Interactive
Note:
This document does not cover any software
-
related failures
except for those that are often
Page
1
of 12Cisco
- Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
Hardware Troubleshooting for Cisco
TAC Notice:
Changing on TAC Web
What's
1700 Series Routers
: This document offers customized analysis of your
Cisco device.
Contents
Introduction
Prerequisites
Requirements
Components Used
Hardware-Software Compatibility and Memory Requirements
Error Messages
Conventions
Modules and Cards
DSP Optimization on the Cisco 1751 and Cisco 1760 Routers
WIC-1ENET Module
Identifying the Issue
Troubleshooting ISDN Interfaces
Troubleshooting Router Hangs
Troubleshooting Bus Error Crashes
Router Does Not Boot
Router Crashes
Watchdog Timeouts
Router Is Dropping Packets
Memory Leaks
Summary
Related Information
Help us help you.
#
!"
Excellent
#
!"
Good
#
!"
Average
#
!"
Fair
#
!"
Poor
$%
#
!"
Yes
#
!"
No
#
!"
Just browsing
--
,
.0/
)
(256 character limit)
& '
(*)+
'21
354768
%
Introduction
Valuable time and resources are often wasted replacing hardware that actually functions properly. This
document helps troubleshoot potential hardware issues with Cisco 1700 Series Routers, and can help
you identify which component may be causing a hardware failure, depending on the type of error that
the router is experiencing.
mistaken as hardware issues.
router, refer to the
Memory Requirements
section of
How to Choose
a Cisco IOS Software Release
. Page
2
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
Prerequisites
Requirements
Readers of this document should be knowledgeable of the following:
1700 Series Hardware Installation Guide
Troubleshooting Router Crashes
1700 Series Router Technical Support Page
Components Used
The information in this document is based on the hardware versions below.
Cisco 1700 Series Routers
The information presented in this document was created from devices in a specific lab environment. All
of the devices used in this document started with a cleared (default) configuration. If you are working in
a live network, ensure that you understand the potential impact of any command before using it.
Hardware-Software Compatibility and Memory Requirements
Whenever you install a new card, module, or Cisco IOS® software image, it is important to verify that
the router has enough memory, and that the hardware and software are compatible with the features you
wish to use.
Perform the following recommended steps to check for hardware-software compatibility and memory
requirements:
registered customers only
1. Use the Software Advisor (
device.
Tips:
The Software Support for Hardware (
whether the modules and cards installed on the router are supported by the desired Cisco
IOS software version.
The Software Support for Features (
registered customers only
Cisco IOS software image needed by choosing the types of features you wish to implement.
registered customers only
2. Use the
IOS Upgrade Planner (
(RAM and Flash) required by the Cisco IOS software, and/or download the Cisco IOS software
image. To determine the amount of memory (RAM and Flash) installed on your
) tool to choose software for your network
registered customers only
) section helps you verify
) section helps you determine the
) to check the minimum amount of memory
running on your router,
MALLOCFAIL: Memory allocation of [dec] bytes failed from [hex],
supported:
Page
3
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
Tips:
If you want to keep the same features as the version that is currently
but don't know which feature set you are using, enter the
router and paste it into the Output Interpreter (
registered customers only
show version
command on your
) tool to find out. It is
important to check for feature support, especially if you plan to use recent software features.
If you need to upgrade the Cisco IOS software image to a new version or feature set, refer
to How to Choose a Cisco IOS Software Release for more information.
3. If you determine that a Cisco IOS software upgrade is required, follow the Software Installation
and Upgrade Procedure for the Cisco 1700 Series Router.
If your 1700 does not have a connection to the network or does not have a valid Cisco IOS
Tip:
software image, you may need to use the console port of the router to perform an
Xmodem
Console Download Procedure Using ROMmon. This procedure does not require the use of a
Trivial File Transfer Protocol (TFTP) server.
4. If the router does not boot after a Cisco IOS software upgrade, refer to the Router Does Not Boot
section of this document for more information.
Error Messages
The Error Message Decoder (
) tool allows you to check the meaning of an error
registered customers only
message. Error messages appear on the console of Cisco products, usually in the following form:
%XXX-n-YYYY : [text]
Here is an example error message:
Router# %SYS-2pool [chars], alignment [DEC]
Some error messages are informational only, while others indicate hardware or software failures and
require action. The Error Message Decoder (
registered customers only
) tool provides an explanation of the
message, a recommended action (if needed), and if available, a link to a document that provides
extensive troubleshooting information about that error message.
Conventions
For more information on document conventions, refer to the
Cisco Technical Tips Conventions.
Modules and Cards
The Cisco 1700 Series Routers include two WAN interface card slots. In addition, the Cisco 1750 and
Cisco 1751 routers include one Voice Interface Card (VIC) slot that can be used only for a voice
interface card. The Cisco 1760 router includes two VIC-only slots. However, not all Cisco WAN
Interface Cards (WICs), Voice Interface Cards (VICs), and Voice WAN Interface Cards (VWIC)s are
supported by the 1700 Series Router. The documents below can help you verify that the module/card is
Digital Signal Processor (DSP) resources among
Optimization on
How to Configure Cisco Devices for
hardware issues. It is
cycle
The complete bootup sequence if the router experiences boot errors.
Page
4
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
For WIC compatibility information, refer to the WIC Hardware Compatibility Matrix.
For VIC/VWIC compatibility information, refer to the Voice Hardware Compatibility Matrix.
DSP Optimization on the Cisco 1751 and Cisco 1760 Routers
To find software configuration information for changing the codec complexity of the analog
Voice Interface Cards (VICs) and optimizing the
the analog and BRI VICs on the Cisco 1751 and Cisco 1760 routers, refer to DSP
the Cisco 1751 and Cisco 1760 Routers.
The DSP Calculator tool (
registered customers only
) can help you determine the DSP requirements
for the Cisco 1751 and the Cisco 1760 routers, based on the interface modules selected and their
codec configurations. This tool supports Cisco IOS Software Releases 12.2(8) YN and later.
WIC-1ENET Module
The Cisco WIC-1ENET is a single-port Ethernet interface card supporting 10BASE-T Ethernet. The
card provides the Cisco 1700 Series Router with additional Ethernet interfaces.
The Cisco WIC-1ENET card is only supported on the Cisco 1700 series routers and requires Cisco IOS
Software Release 12.2(2)XJ or later if the card is installed in any WIC slot other than slot 0, or if more
than one card is installed. The card can be used with Cisco IOS Software Release 12.2(4)T, but the card
must be installed in slot 0, and only one Cisco WIC-1ENET card can be installed in the router.
For information on compatibility and hardware/software requirements for the Cisco WIC-1ENET card,
refer to
Cisco WIC-1ENET Requirements.
Identifying the Issue
This section explains what to do to determine the cause of the potential hardware issue(s).
In order to determine the cause, the first step is to capture as much information about the problem as
possible. The following information is essential for determining the cause of the problem:
Console logs - For more information, refer to Applying Correct Terminal Emulator Settings for
Console Connections.
Syslog information - If the router is set up to send logs to a syslog server, you may be able to
obtain information on what happened. For details, refer to the
Syslog section of Resource Manager Essentials and Syslog Analysis: How-To.
show technical-support
compilation of many different commands including
show stacks
. TAC engineers usually ask for this information to troubleshoot
important to collect the
command output - The
show technical-support
show technical-support
show version, show running-config
information before doing a reload or power-
as these actions can cause all information about the problem to be lost.
command is a
, and
If you have the output of a
Page
5
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
support
fixes. To use the
), you can use the Output Interpreter (
Output Interpreter (
command from your Cisco device (including
show
registered customers only
registered customers only
) , you must be logged in and have JavaScript
) to display potential issues and
enabled.
Troubleshooting ISDN Interfaces
Here is a list of references to use for troubleshooting ISDN interfaces:
Troubleshooting ISDN Layer 1
Troubleshooting ISDN Layer 2
Troubleshooting ISDN Layer 3
Troubleshooting Router Hangs
show technical-
Refer to
Troubleshooting Router Hangs for this issue.
Troubleshooting Bus Error Crashes
The system encounters a bus error when the processor tries to access a memory location that either does
not exist (a software error) or does not respond properly (a hardware problem). A bus error can be
identified by looking at the output of the
show version
cycled or manually reloaded).
Here are two examples of bus error crashes:
Router uptime is 2 days, 21 hours, 30 minutes
System restarted by bus error at PC 0x30EE546, address 0xBB4C4
System image file is "flash:igs-j-l.111-24.bin", booted via flash
.........
At the console prompt, the following error message might also be seen during a bus error:
*** System received a Bus Error exception ***
signal= 0xa, code= 0x8, context= 0x608c3a50
PC = 0x60368518, Cause = 0x20, Status Reg = 0x34008002
For more information regarding this issue, refer to Troubleshooting Bus Error Crashes.
command provided by the router (if not power-
Router Does Not Boot
Capturing information from the console of the router is essential for troubleshooting a router that does
not boot. The console output should be logged in a file for later analysis or for the Cisco Technical
Assistance Center (TAC) if a TAC case is opened.
The following table lists symptoms and recommended actions to take if you are encountering boot
problems:
Symptom
Verify that the power supply is plugged in firmly. If
resolve the issue, replace the power
replace the router.
Follow these steps to connect the PC or terminal to
Verify that your PC or terminal emulation
communicate with
the switch using hardware flow control.
Configure the baud rate and character format
match these console
port default characteristics: 9600 baud, 8 data
Using the supplied rollover cable, insert the
9 female
attach an appropriate
Insert the other end of the supplied rollover
Run the terminal emulation program if you are
If that doesn't help, verify that the equipment used
the console is operating properly.
known good
router to check your console equipment. If the
successfully tested, but the problem
seated. First, try to
insert) the Dynamic RAM
(DRAM) single inline memory module (SIMM)
memory. If the router still does not boot, replace the
You can find information on the location of the
1700 Series Routers
Set the configuration register to 0x2102 and reload
Page
6
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
Recommended Action
No LEDs are on after powering on the router
LEDs are on after powering on the router, but there is
nothing on the console
that does not
supply. If the problem persists,
the router:
1.
software is configured to
2.
of the PC or terminal to
bits, 1 stop bit, No parity.
3.
RJ-45 connector into the console port.
4. Attach the supplied RJ-45-to-DBDTE adapter to a PC, or
adapter to the terminal.
Router displays the System Bootstrap version and hangs
at that point, or falls into a booting loop, repeating the
same information over and over:
System Bootstrap, Version 11.1(7)AX (7)AX],
EARLY DEPLOYMENT RELEASE SOFTWARE (fc2)
Copyright (c) 1994-1996 by cisco Systems, Inc.
5.
cable in the attached adapter.
6.
using a PC or a terminal.
for connecting to
You can do this by connecting to a
equipment is
remains, replace the router.
The memory may not be wellreseat (remove and then re-
router.
DRAM SIMMs in the Cisco
Documentation.
the router:
rommon 1 > confreg 0x2102
Router boots in ROMmon; no error messages on the
If the router remains in ROMmon, follow the
Recovery for the
Set the configuration register to 0x2102 and reload
is no need to save the running
register 0x2102
System configuration has been modified.
command is the only
IOS software configuration command that
RAM (NVRAM). It
changes the configuration register immediately, but
The Flash card is removed or badly inserted. Verify
Flash card by running the
Insert or reseat the Flash. If the problem persists,
card. If that doesn't resolve the
The Flash is empty or the filesystem is corrupted.
on the Flash, and while copying,
Flash (if one
valid
This message only occurs when loading images that
1700 routers support images
Page
7
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
console
rommon 2 > reset
procedure described in ROMmon
Cisco 1700 Series Router.
the router. There
configuration before reloading:
Router boots in boot mode; no error messages on the
console
Router boots in boot mode, with the following messages
on the console:
getdevnum warning: device "PCMCIA slot 1" has
size of zero
getdevnum warning: device "PCMCIA slot 1" has
size of zero
pen: read error...requested 0x4 bytes, got 0x0
trouble reading device magic number
boot: cannot open "flash:"
boot: cannot determine first file name on
device "flash:"
router(boot)#
Enter configuration commands,
one per line. End with CNTL/Z.
router(boot)(config)#
router(boot)(config)#
router(boot)#
Save? [yes/no]: no
Proceed with reload? [confirm]
Note:
The
configure terminal
configend
reload
config-register
Cisco
doesn't get saved in nonvolatile
this only takes effect during the next boot.
the presence of the
command:
flash
router(boot)#show flash
%Device in REMOVED state
replace the Flash
problem, replace the router.
Router boots in boot mode, with the following messages
on the console:
Copy a valid image
you will be prompted to erase the old
device does not contain a valid magic
number
boot: cannot open "flash:"
boot: cannot determine first file name on
device "flash:"
exists). Then reload the router.
Refer to Software Installation and Upgrade
Procedures for instructions on how to copy a
image onto the Flash.
run from RAM. Since
detected an unrecoverable
from this process.
that run in RAM and images that run directly
Flash (for example, relocatable images), there are
Load a relocatable image; load an image from
For more information on image types, refer to
For information on memory requirements, refer to
for your version of Cisco IOS
the DRAM
and
The Cisco IOS software image is not appropriate for
it is corrupted. Erase the Flash
software
For more information on loading new software on the
Installation and
Page
8
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
two solutions for the problem:
Upgrade the DRAM
Router boots in boot mode, with the following message
on the console:
SYSTEM INIT: INSUFFICIENT
MEMORY TO BOOT THE IMAGE!
Router boots in boot mode, with the following messages
on the console:
loadprog: error - Invalid image for platform
Flash
Naming Conventions.
the Release Notes
software.
Refer to the Cisco 1700 Series Router
Documentation for where to locate
SIMMs for upgrading the 1720, 1750, 1751,
1760.
the 1700 router, or
and download a new valid Cisco IOS
image.
1700 router, refer to Software
Upgrade Procedure.
Router Crashes
When we refer to a "system crash", we mean a situation where the system has
error and has restarted itself. A crash can be caused by software problems, hardware problems, or both.
This section deals with hardware-caused crashes and crashes that are software-related, but may be
mistaken for hardware problems.
Important:
If the router is reloaded after the crash (for example, through a power-cycle or the
command), important information about the crash will be lost, so try to collect
and
show log
Refer to
output, as well as the crashinfo file (if possible) before reloading the router!
Troubleshooting Router Crashes for more information regarding this issue.
reload
show technical-support
Watchdog Timeouts
Cisco processors have timers that guard against certain types of system hangs. The CPU periodically
resets a watchdog timer. The watchdog timer basically controls the time of each process. If the timer is
not reset, a trap occurs. If a process is longer than it should be, the watchdog timer is used to escape
There are two main types of watchdog timeouts. The first type is usually caused by a software problem
this usually indicates a hardware
remote interface (port)
Page
9
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
and is reported in the router console logs as:
%SYS-2-WATCHDOG: Process aborted on watchdog timeout
The second type of watchdog timeout is usually due to a hardware problem and is reported in one or
both of these two ways:
The
show version
System returned to ROM by watchdog timer expired
The console logs show:
*** Watch Dog Timeout ***
command output shows:
Refer to Troubleshooting Watchdog Timeouts to further troubleshoot this issue.
Router Is Dropping Packets
Packet losses caused by hardware problems are fairly easy to identify. The following section uses the
output of the
show interfaces
command to identify packet losses.
Cyclic Redundancy Check (CRC) and Frame Errors
If CRC errors or frame errors are constantly increasing on the interface,
problem.
router#show interface ethernet 0
Ethernet0/0 is up, line protocol is up
...
121 input errors, 102 CRC, 19 frame, 0 overrun, 0 ignored
An exception to this is when CRC and frame errors are found on channelized interfaces; they can
indicate clocking problems as well. The fault that is causing the errors can be anywhere between two
connected interfaces: on cables, intermediate devices, or on interfaces themselves. Troubleshooting
techniques differ slightly for different interface types.
Ethernet Interfaces
For Ethernet interfaces, troubleshooting differs between a shared environment (devices connected
through a hub or with a coaxial cable) and a switched environment (devices connected to a switch).
In a switched environment, there are only three components that could cause the error:
cable
local interface (port)
Consequently, the troubleshooting steps are simple. For example, if a router
is connected to a switch, the
interfaces. In the latter case, an increment
Page
10
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
troubleshooting steps would be:
1. Replace the cable.
2. If this does not solve the problem, try another port on the switch.
3. If the problem persists, replace the router.
In a shared environment, the source of the problem is a lot harder to find. Every piece of hardware that
makes up the shared segment can be the cause. All components (cables, connectors, and so on) have to
be tested one by one.
Ignored Packets
router#
Ethernet0/0 is up, line protocol is up
...
21 input errors, 0 CRC, 0 frame, 0 overrun,
show interfaces ethernet 0
21 ignored
Packets are ignored if there are no free buffers to accept the new packet. This can happen if the router is
overloaded with traffic, but can also happen if the interface is faulty. If "ignores" are present on all
interfaces, then the router is probably overloaded with traffic, or doesn't have sufficient free buffers in
the pool that match the maximum transmission unit (MTU) on
of the ignored counter is followed by an increment of the no buffer counter:
router#
...
1567 packets input, 0 bytes,
22 input errors, 0 CRC, 0 frame, 0 overrun, 22 ignored, 0 abort
show interfaces serial 0
22 no buffer
You may also see an increase in the buffer failures counter in the pool that matches the MTU size:
router#
...
Big buffers, 1524 bytes (total 50, permanent 50):
50 in free list (5 min, 150 max allowed)
3066 hits, 189 misses, 0 trims, 24 created
show buffers
12 failures
(0 no memory)
The number of preconfigured permanent, free, and maximum allowed buffers may not be completely
compatible for every environment. You can read more about this and how to avoid it in Buffer Tuning.
If "ignores" are only increasing on one interface and are not followed by an increment of the no buffer
counter, and the interface is not heavily loaded, then this interface could be faulty. In that case, capture
the output of the
be viewed in the output of the
router#
...
reliability 255/255, txload 100/255, rxload 122/255
show technical-support
show interfaces
show interfaces serial 0
command and contact the TAC. The load on the interface can
command:
Input and Output Queue Drops
The hardware component that failed and the serial number for the chassis
Page
11
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
Input queue drops are never caused by hardware problems. Output queue drops may be caused by a
hardware problem only if the output queue is constantly full and no packets are being sent out of the
interface. You can read more about these kinds of drops in
Troubleshooting Input Queue Drops and
Output Queue Drops.
If the issue you are having is not listed on this page, please read through the documents in the Related
Information section. If you still need assistance, please contact the Cisco Technical Assistance Center.
Memory Leaks
Memory allocation failure means either that the router has used all available memory (temporarily or
permanently) or that the memory has fragmented into such small pieces that the router cannot find a
usable available block. This can happen with the processor memory (used by the Cisco Internet
Operating System [IOS]) or with the packet memory (used by incoming and outgoing packets).
Symptoms of memory allocation failure include, but are not limited to:
The console or log message: "%SYS-2-MALLOCFAIL: Memory allocation of 1028 bytes failed
from 0x6015EC84, Pool Processor, alignment 0"
Refused Telnet sessions
The
show processor memory
command is displayed no matter what command you type on a
console
There is no output from some
You see "Low on memory" messages
You see the console message "Unable to create EXEC - no memory or too many processes"
The router hangs; there is no console response
commands
show
For more information regarding this issue, refer to Troubleshooting Memory Problems.
Summary
If you have identified a component that needs to be replaced, contact your Cisco partner or reseller to
request a replacement for the hardware component that is causing the issue. If you have a support
registered customers only
contract directly with Cisco, use the TAC Case Open Tool (
and request a hardware replacement. Make sure you attach the following information:
) to open a TAC case
Console captures showing the error messages
Console captures showing the troubleshooting steps taken and the boot sequence during each step
Page
12
of 12Cisco
-
Hardware Troubleshooting for Cisco 1700 Series Routers
01/10/2004
http://kbase:8000/paws/servlet/ViewFile/17959/hwts_1700_17959.xml?convertPaths=1
Troubleshooting logs
Output from the
show technical-support
Related Information
command
Cisco 1700 Series Routers Documentation
ROMmon Recovery for the Cisco 1700 Series Router
Image Naming Conventions
Software Installation and Upgrade Procedures
Capacity and Performance Management: Best Practices
How to Choose a Cisco IOS Software Release –
Buffer Tuning
Troubleshooting Input Queue Drops and Output Queue Drops
Cisco IOS Software Skills
Rebooting Documentation
Loading and Maintaining System Images
Cisco 1700 Series Product Support Page
Technical Support - Cisco Systems
Memory Requirements
All contents are Copyright © 1992-2004 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Loading...