Troubleshooting Router Startup Problems

Table 2-2 : Specific Cards and Products: Failure Symptoms and Associated Problems

Card or Product	Symptom
RP (Cisco 7000 series only)	System is down after running a short time; blower on. System will not power up; blower on. Boot Error or CPU halt LEDs might be on. Configuration cannot be written to memory. System will not boot (any combination of processor LEDs on, other than green LED alone). The SP or SSP card is not recognized. Partial boot only or system will not boot. Random reboot occurs after initial boot. System reboots when configuration memory is written. No response from keyboard or apparent problem with console terminal. Configuration memory is wrong size. Error Indicators---Bad checksum for configuration memory, configuration memory not set up, nonvolatile memory not present.
CSC-ENVM	System is down after running a short time; DC voltages off; blower on. System will not power up; DC voltages off; blower on. Configuration cannot be written to memory; system loses memory over time. CSC-ENVM fails to shut system down even with excessive heat or DC voltage. Error Indicators---Bad checksum for configuration memory, configuration memory not set up, nonvolatile memory not present.
CSC/2, CSC/3, and CSC/4 cards	System will not boot (any combination of processor LEDs on, other than green LED alone). Multibus cards are not recognized. The ciscoBus controller is not recognized (CSC/3 and CSC/4 cards only). Partial boot only. Random reboot occurs after initial boot. System will autoboot but cannot boot manually. System will reboot when configuration memory is written. No response from keyboard. Error Indicators---Parity error, software versus hardware error, local timeout, bus error, wrong interface, emulation line error, software-forced crashes, checksum mismatch error.
SP or SSP (Cisco 7000 series only)	Some or all CxBus cards are not recognized. Enabled LED does not go on (processor card not initialized). Error Indicators---MEMD failure, MEMA failure, arbiter/processor card failure.
CSC-CCTL and CSC-CCTL2	Some or all ciscoBus cards are not recognized. No LEDs are on. All LEDs are on. Some or all Multibus cards are not recognized. Error Indicators---MEMD failure, MEMA failure, ciscoBus daughter controller failure.
FIP (Cisco 7000 series only)	Not recognized by arbiter, SP, or SSP. Fiber Distributed Data Interface (FDDI) ring will not come up. FDDI ring comes up, but ping does not work on the FDDI ring or only works intermittently; only certain packet sizes ping. No keyboard response after FDDI ring comes up; keyboard locks up. Cannot see FDDI upstream/downstream neighbors. LEDs are on in the wrong sequence. FDDI ring comes up in "wrap-mode" only---wrap A or wrap B. No ping through FDDI ring or to address of unit under test (UUT); intermittent ping. FDDI ring will intermittently or constantly transition. Ring status LEDs do not go on. Error Indicators---Unknown data error, card in slot n does not respond.
CSC-FCI, CSC-C2FCI, and CSC-C2FCIT cards	Not recognized by ciscoBus controller. FDDI ring will not come up. FDDI ring comes up, but ping does not work on the FDDI ring or only works intermittently; only certain packet sizes ping. No keyboard response after FDDI ring comes up; keyboard locks up. Cannot see FDDI upstream/downstream neighbors. Error Indicators---Unknown data error, MEMD failure, MEMA failure, ciscoBus daughter controller failure.
FDDI appliques (APP-LMM, APP-LMS, APP-LSM, and APP-LSS)	FDDI ring will not come up. LEDs are on in wrong sequence. FDDI ring comes up in "wrap-mode" only---wrap A or wrap B. No ping through FDDI ring or to address of UUT; intermittent ping. FDDI ring intermittently or constantly transitions. Cannot see FDDI upstream/downstream neighbors. Ring status LEDs are not on.
EIP (Cisco 7000 series only)	Card is not recognized by the arbiter, SP, or SSP. Unable to ping on any or some ports; intermittent ping; only certain packet sizes will ping. All LEDs are on. No LEDs are on. Wrong number of LEDs are on. Error Indicators---Timeout, arbiter, SP, or SSP failure, halted output.
CSC-MEC and CSC-C2MEC cards	Card is not recognized by ciscoBus controller. Unable to ping on any or some ports; intermittent ping; only certain packet sizes will ping. All LEDs are on. No LEDs are on. Wrong number of LEDs are on. Error Indicators---Multibus timeout, ciscoBus daughter controller failure, halted output.
FSIP card (Cisco 7000 series only)	Card is not recognized by arbiter or SP or SSP. No LEDs are on. All LEDs are on. No ping on any or some ports; DTE will ping and DCE will not ping (or vice versa); intermittent ping; only certain packet sizes will ping. Ports will not initialize---some or all. Will not netboot or ping to network; no ping to address of unit under test (UUT). Error Indicators---Local timeout, MEMD failure, MEMA failure, halted output, bus or ALU failure, configuration memory not set up, excessive input serial errors, CxBus timeouts, or SxBus timeouts (SxBus timeouts apply to older Cisco 7000 routers only).
FSIP port adapters	Interface up, but ping does not work, or intermittent ping functionality. DTE will ping but DCE will not ping (or vice versa). System reboots.
CSC-MCI and CSC-SCI	Card is not recognized by the processor card. No LEDs are on. All LEDs are on. No ping on any or some ports; DTE will ping but DCE will not ping (or vice versa); intermittent ping; only certain packet sizes will ping. Ports will not initialize---some or all. Will not netboot or ping to network; no ping to address of UUT. CSC-MCI-3 card cannot see random-access memory (RAM) and NVRAM. Wrong number of LEDs on---too many or too few. Error Indicators---Local timeout, MEMD failure, MEMA failure, halted output, bus/ALU failure, configuration memory not set up, excessive input serial error, or Multibus timeouts.
Arbiter, SP, or SSP (Cisco 7000 series only)	Cannot write configuration memory on RP; no memory access; memory access causes reboot. CxBus cards are not recognized. System will not boot or will reboot. No DC voltages---some or all. Bad power supply (caused by shorted backplane). Blower is not working. Systems consistently crash when attempting to boot.
ciscoBus backplane and Multibus backplane	Cannot write configuration to memory; cannot access memory; memory access causes reboot. Multibus or ciscoBus cards are not recognized. System will not boot or will reboot. No DC voltages---some or all. Bad power supply (caused by shorted backplane).
TRIP card (Cisco 7000 series only)	Card is not recognized by the processor. No ping to outside address or address of UUT; intermittent ping. No hardware address recognized. Error Indicators---Halted output, beaconing, local timeout, Open failed: lobe test.
CSC-R, CSC-R16M, CSC-1R, CSC-2R, and CSC-C2CTR cards	Card is not recognized by the processor. No ping to outside address or address of UUT; intermittent ping. No hardware address recognized. Error Indicators---Halted output, beaconing, local timeout, Open failed: lobe test, Multibus timeout.
CSC-M, CSC-MT, CSC-MC, and CSCMC+ cards	NVRAM not recognized by MCI-3, CSC-1R, or CSC-2R card (CSC-MC and CSC-MC+ cards only). Configuration cannot be written to memory. Memory lost over time. Configuration and/or Multibus memory wrong size (CSC-MT card only). Error Indicators---Bad checksum for configuration memory, configuration memory not set up, nonvolatile memory not present.
Serial appliques	Interface up but ping does not work, or intermittent ping functionality. DTE will ping, DCE will not ping (or vice versa). System reboots (with dual-mode V.35, suggests bad ground contact). 5V or 12V power supply LEDs indicate no power detected.
Cisco 4000	System will not boot. Fuse blows. Constant or partial reboot.
IGS, Cisco 2000, Cisco 2500, and Cisco 3000	System will not boot. Fuse blows (except Cisco 2500 series). Fan does not run. Constant or partial reboot.
500-CS	System will not boot. Fuse blows. Fan does not run. LEDs fail to go on.

Troubleshooting Media Problems

Table 2-3 through Table 2-6 summarize general problem-solving guidelines for common media (Ethernet, Token Ring, serial lines, and FDDI).

Table 2-3 : Media Problems: Ethernet

Media Problem

Suggested Actions

Excessive errors or noise on Ethernet

Step 1 Use the show interfaces ethernet EXEC command to determine the status of the interface. Step 2 Use a time domain reflectometer (TDR) to find any unterminated Ethernet cables. Step 3 Check host cables to determine whether any are incorrectly terminated, overly long, or damaged. Step 4 Look for a jabbering transceiver attached to a host (might require host-by-host inspection). Step 5 Look for badly spaced taps causing reflections.

Table 2-4 : Media Problems: Token Ring

Media Problem

Suggested Actions

Nonfunctional Token Ring

Step 1 Use the show interfaces token command to determine the status of the interface. Step 2 If the status line indicates that the interface and line protocol are not up, check the cable from router to Multistation Access Unit (MAU). Make sure that the cable is good; replace if necessary. If you are performing a new installation, make sure that the MAU has been properly initialized. Consult the manufacturer's documentation for information on initializing your MAU. Step 3 If the show interfaces token output indicates that the interface and line protocol are up, use the ping command between routers to test connectivity. Step 4 If the remote router does not respond, check the ring specification on all nodes attached to the Token Ring backbone. Ring speed for all must be the same. The options are 4 Mbps (default) and 16 Mbps. Use the write terminal privileged EXEC command to determine which speed is active. Step 5 If necessary, modify ring speed specifications for clients, servers, and routers. Step 6 Use the ring-speed interface configuration command to modify the ring speed configuration for Token Ring cards that support software speed configuration; change jumpers as needed for modular router platforms that do not support software speed configuration. For more information about ring speed specifications, refer to the hardware installation and maintenance manual for your system.

Table 2-5 : Media Problems: Serial Lines

Media Problem

Suggested Actions

Nonfunctional serial line

Step 1 Use the show interfaces serial EXEC command to determine the status of the interface. Step 2 If the show interfaces serial command indicates that the interface and line protocol are up, use the ping EXEC command between routers to test connectivity. Isolate possible circuit problems by looping the local DTE back to the RTS interface pin. Step 3 If routers do not respond to the ping test, follow the troubleshooting techniques as discussed in the "Troubleshooting Serial Line Problems" chapter. Step 4 If clock and data signals are out of phase, invert the clock signal.

Table 2-6 : Media Problems: FDDI

Media Problem	Suggested Actions
Nonfunctional FDDI ring	Step 1 Use the show interfaces fddi EXEC command to determine status of interface. Step 2 If the show interfaces fddi command indicates that the interface and line protocol are up, use the ping command between routers to test connectivity. Step 3 If the interface and line protocol are up, make sure the media access control (MAC) addresses of upstream and downstream neighbors are as expected. If all zeros appear in either of the address fields for these neighbors, a physical connection problem is likely. Step 4 In this case (or if status line does not indicate that the interface and line protocol are up), check patch-panel connections or use an optical TDR or light meter to check connectivity between neighbors. Ensure that signal strength is within specification.
Upstream neighbor has failed and bypass switch is installed. Bypass switches can cause signal degradation because they do not repeat signals like a normal transceiver.	Step 1 Check upstream neighbor to determine if it is operational. Step 2 If the node is down, and a bypass switch is in place, resolve any problems found in upstream neighbor.

Troubleshooting Router Booting Problems

Routers allow for system initialization (booting) using several methods. Systems can be booted in any of four ways:

• From a file over the network
  

• From Flash memory
  

• From ROM
  

• From a PCMCIA Flash memory card
  

The material that follows addresses problems that might arise during the booting process.

Booting Troubleshooting Information

If you are unable to resolve your booting problem, collect the following information for the technical support representative:

• ROM images (using the show version EXEC command)
  

• Programmable ROM labels
  

• NVRAM configurations for client and adjacent routers (via the write terminal privileged EXEC command)
  

• Debugging output from the adjacent router using the following privileged EXEC commands:
  

  • debug ip packet
    

  • debug arp
    

  • debug ip udp
    

  • debug tftp
    

For more information about these debug commands, refer to the Debug Command Reference publication.

Notes on Netbooting

Routers support netbooting via both the Trivial File Transfer Protocol (TFTP) and the DEC Maintenance Operation Protocol (MOP) across all supported media types such as Ethernet, FDDI, serial lines, Token Ring, and High-Speed Serial Interface (HSSI). During netbooting sessions, routers behave like hosts: they route via proxy Address Resolution Protocol (ARP), Serial Line Address Resolution Protocol (SLARP) information, Internet Control Message Protocol (ICMP) redirects, or a default gateway. When netbooting, routers ignore dynamic routing information, static IP routes, and bridging information. As a result, intermediate routers are responsible for handling ARP and User Datagram Protocol (UDP) requests correctly. For serial and HSSI media, ARP is not used.

If you need to netboot from a server, you should first ping the server from the ROM software. If you are unable to ping the server, first look for a solution in Table 2-7. If none of the problems described in Table 2-7 explains the ping failure, there is probably a problem with the server configuration or hardware. Contact your router or TFTP server technical support representative for assistance.

Using a Fault-Tolerant Boot Strategy

Network failures can make netbooting impossible. After Flash memory is installed and configured, configure the router to boot in the following order to reduce the effects of a server or network failure:

1. Boot an image from Flash memory
   

2. Boot an image from a system filename (netboot)
   

3. Boot from a ROM image
   

Example

The order of the commands needed to implement this strategy is illustrated in the following sample output:

klamath# configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
klamath(config)# boot system flash gsxx
klamath(config)# boot system gsxx 131.108.1.101
klamath(config)# boot system rom
klamath(config)# ^Z
klamath#
%SYS-5-CONFIG_I: Configured from console by console
klamath# write memory
[ok]
klamath#

Using this strategy, a router has three sources from which to boot: Flash memory, netboot, or ROM. Providing alternative sources can help to mitigate any potential failure of the TFTP server or the network.

Booting gs3-bfx from 131.108.1.123: !.!!!!!!!!!!!!!!!!!!!!!!

Booting gs3-bfx from 131.108.1.123: !O.O!!!!!!!!!!!!!!!!!!!!!!

Booting gs3-bfx..........[failed]

Booting gs3-bfx from 131.108.1.123: !O.O!.O..O!!!OOO.O!!.O.O.....

Booting gs3-bfx!OOOO..........[timed out]

Booting gs3-bfx!.O.O.O.O..........[timed out]

Booting gs3-bfx!!!!!!!!!!OOOOOOOOOO..........[timed out]

Booting gs3-bfx!..........[failed]

router(boot)# configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
ip default-gateway 130.108.1.7
^Z
%SYS-5-CONFIG_I: Configured from console by console
router(boot)# copy tftp flash
IP address or name of remote host [255.255.255.255]? server1
Name of tftp filename to copy into flash []? IJ09140Z
copy IJ09140Z from 131.131.101.101 into flash memory? [confirm] <Return>
xxxxxxxx bytes available for writing without erasure.
erase flash before writing? [confirm] <Return>
Clearing and initializing flash memory (please wait)####...
Loading from 131.131.101.101: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!... [OK - 324572/524212 bytes]
Verifying checksum...
VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV...
Flash verification successful. Length = 1204637, checksum = 0x95D9

Booting gs3-bfx..........[timed out]