Cisco Systems UBR10012 User Manual

Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
October 2004
Corporate Headquarters
Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000
Fax: 408 526-4100
Text Part Number: OL-1237-01
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESSED OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The following inform ation is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.
The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy. If it is not installed in accordance with Cisco’s installation instructions, it may cause interference with radio and television reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation. However, there is no guarantee that interference will not occur in a particular installation.
Modifying the equipment without Cisco’s written authorization may result in the equipment no longer complying with FCC requirements for Class A or Class B digital devices. In that event, your right to use the equipment may be limited by FCC regulations, and you may be required to correct any interference to radio or television communications at your own expense.
You can determine whether your equipment is causing interference by turning it off. If the interference stops, it was probably caused by the Cisco equipment or one of its peripheral devices. If the equipment causes interference to radio or television reception, try to correct the interference by using one or more of the following measures:
• Turn the television or radio antenna until the interference stops.
• Move the equipment to one side or the other of the television or radio.
• Move the equipment farther away from the television or radio.
• Plug the equipment into an outlet that is on a different circuit from the television or radio. (That is, make certain the equipment and the television or radio are on circuits controlled by different circuit breakers or fuses.)
Modifications to this product not authorized by Cisco Systems, Inc. could void the FCC approval and negate your authority to operate the product.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
CCVP, the Cisco Logo, and the Cisco Square Bridge logo are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems, Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys, MeetingPlace, MGX, Networking Academy, Network Registrar, Pac ke t, PIX, ProConnect, RateMUX, ScriptShare, SlideCast, SMARTnet, StackWise, The Fastest Way to Increase Your Internet Quotient, and TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0704R)
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide OL-1237-01
Copyright © 2001-2004, Cisco Systems, Inc. All rights reserved.
Press,
Preface vii
Purpose vii
Audience vii
Document Organization viii
Related Documentation viii
Obtaining Documentation ix
Cisco.com ix Ordering Documentation ix
Documentation Feedback ix
Obtaining Technical Assistance x
Cisco TAC Website x Opening a TAC Case x TAC Case Priority Definitions xi

CONTENTS

CHAPTER
CHAPTER
Obtaining Additional Publications and Information xi
1 Basic Troubleshooting Tasks and Startup Issues 1-1
Basic Troubleshooting Checklist 1-1
Confirming the Hardware Installation 1-2
Displaying the Cisco IOS Software Version 1-3
Displaying System Environment Information 1-4
Hardware Troubleshooting Flowchart 1-4
Cisco uBR10012 System Startup Sequence 1-5
2 PEM Faults and Fan Assembly Failures 2-1
AC PEM Faults 2-1
DC PEM Faults 2-3
2400W AC-Input Power Shelf 2-5
Other Electrical Problems 2-6
Fan Assembly Module Faults 2-7
CHAPTER
OL-1237-01
3 Troubleshooting PRE-1 Modules 3-1
Information Required for Troubleshooting PRE-1 Modules 3-1
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
iii
Contents
PRE Module Not Supported 3-2
PRE-1 Module Status Screen 3-2
Booting Up with Redundant PRE-1 Modules 3-3
PRE-1 Module Faults 3-4
Ethernet Connection Problems 3-6
Console Port Serial Connection Problems 3-7
Troubleshooting Common System Problems 3-8
Troubleshooting System Crashes 3-8 High CPU Utilization Problems 3-9
ARP Traffic 3-9 CPUHOG Errors 3-11 Debug and System Messages 3-11 Exec and Virtual Exec Processes 3-11 Interrupts are Consuming a Large Amount of Resources 3-12 Invalid Scheduler Allocate Configuration 3-12 IP Input Processing 3-12 One or More Processes is Consuming an Excessive Amount of Resources 3-12 Problems with Access Lists 3-13
SNMP Traffic 3-13 Bus Errors 3-13 Memory Problems 3-15
Alignment Errors 3-15
Low Memory Errors 3-16
Memory Parity Errors 3-16
Particle Pool Fallbacks 3-17
Spurious Interrupts 3-18
Spurious Memory Accesses 3-19
CHAPTER
iv
4 Troubleshooting Line Cards 4-1
General Information for Troubleshooting Line Card Crashes 4-2
Cache Parity Errors 4-4 Bus Errors 4-5 Software-Forced Crashes 4-6
Troubleshooting the Timing, Communication, and Control Plus Card 4-8
Troubleshooting the OC-12 Packet-Over-SONET Line Card 4-12
Troubleshooting the OC-12 Dynamic Packet Transport Spatial Reuse Protocol WAN Card 4-14
Troubleshooting the Cisco uBR10012 OC-48 DPT/POS Line Card 4-16
Troubleshooting the Gigabit Ethernet Line Card 4-18
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Contents
CHAPTER
APPENDIX
APPENDIX
5 Replacing or Recovering Passwords 7-1
Password Recovery Procedure Overview 7-1
Password Recovery Procedure 7-1
A Unsupported Commands A-1
Unsupported Frame Relay Commands A-1
HCCP Commands A-2
MLPPP Commands A-2
Unsupported MPLS VPN Commands A-3
Unsupported PPP Commands A-3
Spectrum Management Commands A-3
Unsupported Telco-Return Commands A-3
B Recommended Tools and Test Equipment B-1
Testing with Digital Multimeters and Cable Testers B-1
Testing with TDRs and OTDRs B-2
Testing with TDRs B-2 Testing with OTDRs B-2
I
NDEX
Testing with Breakout Boxes, Fox Boxes, and BERTs/BLERTs B-3
Testing with Network Monitors B-3
Testing with Network Analyzers B-4
OL-1237-01
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
v
Contents
vi
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01

Preface

This guide documents processes and procedures for user level hardware troubleshooting on the Cisco uBR10012 universal broadband router. For complete configuration instructions, please refer to the Cisco uBR10012 Universal Broadband Router Software Configuration Guide and the documents listed in the “Related Documentation” section on page viii.
Purpose, page vii
Audience, page vii
Document Organization, page viii
Related Documentation, page viii
Obtaining Documentation, page ix

Purpose

Audience

Documentation Feedback, page ix
Obtaining Technical Assistance, page x
Obtaining Additional Publications and Information, page xi
The Cisco uBR10012 router provides data and Voice over IP (VoIP) services to cable modems (CMs) and customer premises equipment (CPE) devices over a cable TV (CATV) network, supplying high-speed Internet and voice connectivity over the coaxial cable that provides TV and other signals. Many of the Cisco uBR10012 modules are available in redundant configurations, so that the failure of one module does not affect systems operations. This guide provides troubleshooting steps for a failed component that you can take before system failure occurs and before intervention from higher level support agencies becomes necessary.
To benefit from this guide, you must be experienced using Cisco IOS and have some responsibility for installing, configuring, or operating the Cisco uBR10012 router. Knowledge of basic cable data network operations and of the Data-Over-Cable Service Interface Specifications (DOCSIS), which define the transmission of data and other services over a coaxial cable TV network.
OL-1237-01
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
vii
Preface

Document Organization

Document Organization
The sections of this guide are as follows:
Chapter Description
Chapter 1, “Basic Troubleshooting Tasks and Startup Issues”
Chapter 2, “PEM Faults and Fan Assembly Failures”
Chapter 3, “Troubleshooting PRE-1 Modules” How to troubleshoot Performance Routing Engine (PRE-1) modules. It
Chapter 4, “Troubleshooting Line Cards” Troubleshooting faults for all following Cisco uBR10012 line cards.
Chapter 5, “Replacing or Recovering Passwords”
Appendix A, “Unsupported Commands” A list of the commands that are not supported in Cisco IOS Release
Appendix B, “Recommended Tools and Test Equipment”
Basic procedures that users should perform before undertaking a detailed troubleshooting analysis of the Cisco uBR10012 router or logging a case with the Cisco Technical Assistance Center (TAC).
Methods for troubleshooting faults involving the Cisco uBR10012 Power Entry Modules (PEMs) and blower modules.
provides information on troubleshooting PRE-1 fault states, the management Ethernet port, and the serial port.
How to recover a lost enable or console login password, and how to replace a lost enable secret password on the Cisco uBR10012 router.
12.2(15)BC1 for the Cisco uBR10012 router.
A list of basic tools and test equipment necessary to perform maintenance and troubleshooting tasks on the Cisco uBR10012 router.

Related Documentation

When troubleshooting the Cisco uBR10012 router, you should use the Cisco uBR10012 Universal Broadband Router Troubleshooting Guide with the following documents:
Cisco uBR10012 Universal Broadband Router Release Notes—Provides the most up-to-date
information about software version requirements for using the router. It also provides information about bugs and workarounds. See the following URL:
http://www.cisco.com/univercd/cc/td/doc/product/cable/ubr10k/ub10krns/index.htm
Cisco uBR10012 Universal Broadband Router Software Configuration Guide—Contains detailed
information on the configuration and administration of the Cisco uBR10012 router. See the following URL:
http://www.cisco.com/univercd/cc/td/doc/product/cable/ubr10k/ubr10012/scg/index.htm
Cisco uBR10012 Universal Broadband Router Hardware Installation Guide—Contains information
about the hardware of the Cisco uBR10012 router, how to install the router, connect its cables, and start the system up for the first time. See the following URL:
http://www.cisco.com/univercd/cc/td/doc/product/cable/ubr10k/ubr10012/hig/index.htm
viii
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Preface
For more information about the IOS software that runs on the Cisco uBR10012 router, see the Cisco IOS command reference books and configuration guides:
Cisco Broadband Cable Command Reference Guide—Describes the cable specific commands used
on the Cisco uBR10012 router. See the following URL:
http://www.cisco.com/univercd/cc/td/doc/product/cable/bbccmref/i ndex.htm
Cisco IOS Release 12.2 Configuration Guides and Command References—Describes the commands
and configuration used in Cisco IOS Release 12.2. See the following URL:
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/ 122cgcr/index.htm

Obtaining Documentation

Cisco documention and additional literature are available on Cisco.com. Cisco also provides several ways to obtain technical assistance and other technical resources. These sections explain how to obtain technical information from Cisco Systems.
Obtaining Documentation

Cisco.com

You can access the most current Cisco documentation on the World Wide Web at this URL:
http://www.cisco.com/univercd/home/home.htm
You can access the Cisco website at this URL:
http://www.cisco.com
International Cisco websites can be accessed from this URL:
http://www.cisco.com/public/countries_languages.shtml

Ordering Documentation

You can find instructions for ordering documentation at this URL:
http://www.cisco.com/univercd/cc/td/doc/es_inpck/pdi.htm
You can order Cisco documentation in these ways:
Registered Cisco.com users (Cisco direct customers) can order Cisco product documentation from
the Ordering tool:
http://www.cisco.com/en/US/partner/ordering/index.shtml
Nonregistered Cisco.com users can order documentation through a local account representative by
calling Cisco Systems Corporate Headquarters (California, USA) at 408 526-7208 or, elsewhere in North America, by calling 800 553-NETS (6387).

Documentation Feedback

You can submit e-mail comments about technical documentation to bug-doc@cisco.com.
OL-1237-01
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
ix

Obtaining Technical Assistance

You can submit comments by using the response card (if present) behind the front cover of your document or by writing to the following address:
Cisco Systems Attn: Customer Document Ordering 170 West Tasman Drive San Jose, CA 95134-9883
We appreciate your comments.
Obtaining Technical Assistance
For all customers, partners, resellers, and distributors who hold valid Cisco service contracts, the Cisco Technical Assistance Center (TAC) provides 24-hour-a-day, award-winning technical support services, online and over the phone. Cisco.com features the Cisco TAC website as an online starting point for technical assistance. If you do not hold a valid Cisco service contract, please contact your reseller.

Cisco TAC Website

Preface
The Cisco TAC website provides online documents and tools for troubleshooting and resolving technical issues with Cisco products and technologies. The Cisco TAC website is available 24 hours a day, 365 days a year. The Cisco TAC website is located at this URL:
http://www.cisco.com/tac
Accessing all the tools on the Cisco TAC website requires a Cisco.com user ID and password. If you have a valid service contract but do not have a login ID or password, register at this URL:
http://tools.cisco.com/RPF/register/register.do

Opening a TAC Case

Using the online TAC Case Open Tool is the fastest way to open P3 and P4 cases. (P3 and P4 cases are those in which your network is minimally impaired or for which you require product information.) After you describe your situation, the TAC Case Open Tool automatically recommends resources for an immediate solution. If your issue is not resolved using the recommended resources, your case will be assigned to a Cisco TAC engineer. The online TAC Case Open Tool is located at this URL:
http://www.cisco.com/tac/caseopen
For P1 or P2 cases (P1 and P2 cases are those in which your production network is down or severely degraded) or if you do not have Internet access, contact Cisco TAC by telephone. Cisco TAC engineers are assigned immediately to P1 and P2 cases to help keep your business operations running smoothly.
To open a case by telephone, use one of the following numbers:
Asia-Pacific: +61 2 8446 7411 (Australia: 1 800 805 227) EMEA: +32 2 704 55 55 USA: 1 800 553-2447
For a complete listing of Cisco TAC contacts, go to this URL:
http://www.cisco.com/warp/public/687/Directory/DirTAC.shtml
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
x
OL-1237-01
Preface

Obtaining Additional Publications and Information

TAC Case Priority Definitions

To ensure that all cases are reported in a standard format, Cisco has established case priority definitions.
Priority 1 (P1)—Your network is “down” or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation.
Priority 2 (P2)—Operation of an existing network is severely degraded, or significant aspects of your business operation are negatively affected by inadequate performance of Cisco products. You and Cisco will commit full-time resources during normal business hours to resolve the situation.
Priority 3 (P3)—Operational performance of your network is impaired, but most business operations remain functional. You and Cisco will commit resources during normal business hours to restore service to satisfactory levels.
Priority 4 (P4)—You require information or assistance with Cisco product capabilities, installation, or configuration. There is little or no effect on your business operations.
Obtaining Additional Publications and Information
Information about Cisco products, technologies, and network solutions is available from various online and printed sources.
Cisco Marketplace provides a variety of Cisco books, reference guides, and logo merchandise. Go
to this URL to visit the company store:
http://www.cisco.com/go/marketplace/
The Cisco Product Catalog describes the networking products offered by Cisco Systems, as well as
ordering and customer support services. Access the Cisco Product Catalog at this URL:
http://cisco.com/univercd/cc/td/doc/pcat/
Cisco Press publishes a wide range of general networking, training and certification titles. Both new
and experienced users will benefit from these publications. For current Cisco Press titles and other information, go to Cisco Press online at this URL:
http://www.ciscopress.com
Packet magazine is the Cisco quarterly publication that provides the latest networking trends,
technology breakthroughs, and Cisco products and solutions to help industry professionals get the most from their networking investment. Included are networking deployment and troubleshooting tips, configuration examples, customer case studies, tutorials and training, certification information, and links to numerous in-depth online resources. You can access Packet magazine at this URL:
http://www.cisco.com/packet
iQ Magazine is the Cisco bimonthly publication that delivers the latest information about Internet
business strategies for executives. You can access iQ Magazine at this URL:
http://www.cisco.com/go/iqmagazine
OL-1237-01
Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering
professionals involved in designing, developing, and operating public and private internets and intranets. You can access the Internet Protocol Journal at this URL:
http://www.cisco.com/ipj
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
xi
Obtaining Additional Publications and Information
Training—Cisco offers world-class networking training. Current offerings in network training are
listed at this URL:
http://www.cisco.com/en/US/learning/index.html
Preface
xii
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
CHAPTER
1

Basic Troubleshooting Tasks and Startup Issues

This section describes the basic procedures that users should perform before undertaking a detailed troubleshooting analysis of the Cisco uBR10012 router or logging a case with the Cisco Technical Assistance Center (TAC).
These basic troubleshooting checks are organized as follows:
Basic Troubleshooting Checklist, page 1-1
Confirming the Hardware Installation, page 1-2
Displaying the Cisco IOS Software Version, page 1-3
Displaying System Environment Information, page 1-4
Hardware Troubleshooting Flowchart, page 1-4
Cisco uBR10012 System Startup Sequence, page 1-5

Basic Troubleshooting Checklist

If you encounter a problem after you install the Cisco uBR10012 router, go through the following troubleshooting checklist to check for the most common error conditions before you contact the Cisco Technical Assistance Center (TAC) or before you perform a detailed troubleshooting analysis:
1. Is the power on?
2. Is each Power Entry Module (PEM) securely inserted into the router? Is each PEM connected to a
power source that is supplying voltage in the proper AC or DC range? Are all power leads and cables firmly connected at both ends?
3. Is the fan assembly module installed in the chassis and operating? Can you hear the fans operating,
and when you put your hand in front of the fan blowers, can you feel the air flow? Are all empty slots covered with blank front panels, to ensure the correct air flow through the chassis for cooling?
4. Is each PRE-1 module firmly seated and securely inserted in the chassis?
5. Is at least one Timing, Communication and Control Plus (TCC+) card installed in the router?
6. Are the other line cards firmly seated and securely screwed to the chassis?
7. Are all data cables firmly connected at both ends?
8. Are the ports properly configured? Refer to the Cisco uBR10012 Universal Broadband Router
Software Configuration Guide for configuration examples.
After going through this checklist, go through the remaining sections in this chapter to verify the installation and to perform basic troubleshooting.
OL-1237-01
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
1-1

Confirming the Hardware Installation

Confirming the Hardware Installation
Start troubleshooting the installation by issuing the show hardware command. The show hardware command displays all hardware components that are recognized by the system. These components can include the following:
Performance Routing Engine (PRE-1) modules (minimum of one, maximum of two)
FastEthernet Interface (onboard the active PRE-1 module)
Cable Interface line cards (minimum of one, maximum of eight):
Cisco uBR10-MC5X20S-D
Cisco uBR-LCP2-MC16C
Cisco uBR-LCP2-MC16E
Cisco uBR-LCP2-MC16S
Cisco uBR-LCP2-MC28C
WAN interface uplink line cards (minimum of one, maximum of four):
Cisco uBR10-1GE Gigabit Ethernet (GigE)
Cisco uBR10-1OC12/P-SMI Packet Over SONET (POS)
Chapter 1 Basic Troubleshooting Tasks and Startup Issues
Cisco uBR10-SRP-OC12SML Dynamic Packet Transport (DPT) Spatial Reuse Protocol (SRP)
Cisco uBR10-OC-48 DPT/POS
Timing, Communication and Control Plus (TCC+) card (minimum of one, maximum of two)
If an installed item does not appear in the command output, make sure the item is properly installed. For example, make sure the line cards are fully inserted into the slot and the captive screws are tightened. If the problem persists, consult the Cisco uBR10012 release notes to confirm that this is not an existing problem. Finally, you should consider replacing the component.
The following example shows typical output from the show hardware command:
UBR10K-ROUTER1#show hardware
Cisco Internetwork Operating System Software IOS (tm) 10000 Software (UBR10K-P6-M), Released Version 12.2(8)BC2 Copyright (c) 1986-2002 by cisco Systems, Inc. Compiled Mon 12-Aug-02 17:53 slacmar Image text-base: 0x60008940, data-base: 0x61730000
ROM: System Bootstrap, Version 12.0(9r)SL2, RELEASE SOFTWARE (fc1) BOOTLDR: 10000 Software (C10K-EBOOT-M), Version 12.0(17)ST, RELEASE SOFTWARE)
UBR10K-ROUTER1 uptime is 3 weeks, 21 hours, 43 minutes System returned to ROM by power-on System restarted at 13:00:51 PDT Mon Dec 13 2003 System image file is “disk0:/ubr10k-k9p6-mz”
cisco uBR10000 (PRE1-RP) processor with 425983K/98304K bytes of memory. Processor board ID DEFGHIJKLMN R7000 CPU at 262Mhz, Implementation 39, Rev 2.1, 256KB L2, 2048KB L3 Cache Backplane version 1.0, 8 slot
1-2
Last reset from power-on PXF processor tmc0 is running. PXF processor tmc1 is running. 2 TCCplus card(s) 1 FastEthernet/IEEE 802.3 interface(s)
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Chapter 1 Basic Troubleshooting Tasks and Startup Issues
2 Gigabit Ethernet/IEEE 802.3 interface(s) 4 Cable Modem network interface(s) 509K bytes of non-volatile configuration memory.
125440K bytes of ATA PCMCIA card at slot 1 (Sector size 512 bytes). 32768K bytes of Flash internal SIMM (Sector size 256KB). Configuration register is 0x2102
UBR10K-ROUTER1#
Displaying the Cisco IOS Software Version
Use the show version command to confirm that the router is running the proper version of Cisco IOS software and has a sufficient amount of system memory. The command also reports the system uptime and the method by which the system was powered up.
In the following sample of output from the show version command, some of the information that may be useful for troubleshooting appears in bold type:
UBR10K-ROUTER1# show version
Displaying the Cisco IOS Software Version
Cisco Internetwork Operating System Software
IOS (tm) 10000 Software (UBR10K-P6-M), Released Version 12.2(8)BC2
Copyright (c) 1986-2002 by cisco Systems, Inc. Compiled Thu 19-Apr-01 13:47 by skabar Image text-base: 0x60008960, data-base: 0x612B0000
ROM: System Bootstrap, Version 12.0(9r)SL1, RELEASE SOFTWARE (fc1) BOOTFLASH: 10000 Software (C10K-EBOOT-M), Released Version 12.2(1)
UBR10K-ROUTER1 uptime is 3 weeks, 21 hours, 43 minutes System returned to ROM by power-on
System restarted at 13:00:51 PDT Mon Dec 13 2003
cisco uBR10000 (PRE-1-RP) processor with 393215K/131072K bytes of memory. Processor board ID DEFGHIJKLMN R7000 CPU at 262Mhz, Implementation 39, Rev 2.1, 256KB L2, 2048KB L3 Cache Backplane version 1.0, 8 slot
Last reset from power-on
PXF processor tmc0 is running. PXF processor tmc1 is running. 2 TCCplus card(s) 1 FastEthernet/IEEE 802.3 interface(s) 2 Gigabit Ethernet/IEEE 802.3 interface(s) 4 Cable Modem network interface(s) 509K bytes of non-volatile configuration memory.
125440K bytes of ATA PCMCIA card at slot 1 (Sector size 512 bytes). 32768K bytes of Flash internal SIMM (Sector size 256KB). Configuration register is 0x2102
OL-1237-01
UBR10K-ROUTER1#
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
1-3
Chapter 1 Basic Troubleshooting Tasks and Startup Issues

Displaying System Environment Information

Displaying System Environment Information
Use the show environment command to display the basic system environment status, to verify the following:
Make sure the system operating temperature is equal to or less than 41° F at the inlet and 104° F
degrees at the core (5° C and 40° C).
That the fan assembly module is installed in the chassis and operating properly.
Report the operational status of the PEMs and blower
If the operating temperature is not between 41° F and 104° F, refer to the “Fan Assembly Module Faults”
section on page 2-7.
The following example is sample output from the show environment command for a system with two DC PEMs installed:
UBR10K-ROUTER1# show environment
Temperature normal:chassis inlet measured at 29C/84F Temperature normal:chassis core measured at 39C/98F Fan: OK Power Entry Module 0 type DC status: OK Power Entry Module 0 Power: 555w Power Entry Module 0 Voltage: 62v Power Entry Module 1 type DC status: OK Power Entry Module 1 Power: 558w Power Entry Module 1 Voltage: 62v
UBR10K-ROUTER1#

Hardware Troubleshooting Flowchart

Use Figure 1-1 to determine which component of your Cisco uBR10012 router is malfunctioning.
Figure 1-1 describes a series of hardware dependent startup events that must take place for a
Cisco uBR10012 router to allow the passage of IP traffic. At each main point of the flowchart, there are pointers to the chapters in this guide that describe how to troubleshoot individual pieces of hardware.
Note This flowchart does not address software configuration problems.
1-4
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Chapter 1 Basic Troubleshooting Tasks and Startup Issues
Figure 1-1 Hardware Troubleshooting Flowchart
Turn
on
PEM

Cisco uBR10012 System Startup Sequence

*Miswire
LED
off
?
Yes
PEM
Power OK
LED on
?
Yes
PRE
Status
LEDs OK
?
Yes
ICC+
Status
LEDs OK
?
Yes
Line card
LEDs on
?
Yes
System
startup
successful
No
No
No
No
No
Correct
input
wiring
Troubleshoot
PEM
Reseat
PRE
and restart
Reseat
TCC+
Reseat
line card
and restart
PEM
Power OK
LED on
?
Yes
PRE
status LEDs
OK?
Yes
ICC+
status LEDs
OK?
Yes
Line
card LEDs
OK
?
Yes
See section
No
No
No
See appropriate
No
"PEM Faults"
See section
"PRE Module
Faults"
See section
"Troubleshooting
TCC+"
line card fault
section
Obtain
technical
assistance
103381
Cisco uBR10012 System Startup Sequence
Table 1-1 describes the visible sequence of events that occur during a typical Cisco uBR10012 power up.
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
1-5
Chapter 1 Basic Troubleshooting Tasks and Startup Issues
Cisco uBR10012 System Startup Sequence
Table 1-1 Cisco\ uBR10000 Series System Startup Sequence
Startup Event Event Description
PEM is powered off The Fault LED on each PEM is lit yellow to indicate that power is being supplied to the PEM
but that the router is not turned on.
Power on the Cisco uBR10012 router
Cisco IOS software loads
1. The Power LED on each PEM is lit green.
2. The yellow Critical, Major, and Minor alarm and Fail LEDs illuminate for about 2 seconds.
3. The alphanumeric display on the active PRE-1 module counts up through a range of
numbers from 1111 to 9999 (1111, 2222, and so on).
4. The alpha numeric display counts up through a sequence of letters from AAA to CCC
(AAA, BBB, and CCC).
5. The message ROM DONE appears on the alphanumeric display.
Note If the system is not configured to auto boot, it stops at the ROM DONE message. The
console displays a
6. The Power LED on each TCC+ card turns green. The Status LED on each TCC+ lights
rommon> prompt.
yellow. After a few seconds, the Status LED on the primary TCC+ card lights green, and the Status LED on the backup TCC+ card begins blinking green.
1. If the system is set to boot from the slot0: file system, the green slot LED lights.
2. The message BOOT IMGE appears on the alphanumeric display on the active PRE-1
module.
3. The console displays a series of pound signs (#) as the IOS software image is decompressed.
4. The following messages appear on the alphanumeric display on the active PRE-1 module.
IOS STRT
IOS EXC
IOS FPGA
IOS FPOK
IOS FILE
IOS STBY
IOS DRVR
IOS LIB
IOS MGMT
IOS CONF
5. The console displays the bootup screen, followed by the prompt:
Press RETURN to get started!
6. The message IOS RUN appears in the alphanumeric display on the active PRE-1 module. In
a redundant configuration, the message IOS STBY appears on the alphanumeric display of the standby PRE-1 module.
If the boot process fails, no console access is available. If you cannot boot the Cisco uBR10012 router, call Cisco TAC.
1-6
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01

PEM Faults and Fan Assembly Failures

The following sections provide methods for troubleshooting faults involving the Cisco uBR10012 DC Power Entry Modules (PEMs), the optional 2400W AC-input power shelf, and fan assembly module. This chapter contains the following major sections:
AC PEM Faults, page 2-1
DC PEM Faults, page 2-3
2400W AC-Input Power Shelf, page 2-5
Other Electrical Problems, page 2-6
Fan Assembly Module Faults, page 2-7

AC PEM Faults

CHAPTER
2
On the Cisco uBR10012 router, two AC PEMs are installed in a redundant configuration, which allows one AC PEM to fail without affecting system operations. A single PEM can power the router for sufficient time to request and install a new PEM to replace the one that failed.
Tip To quickly check the functional status of your PEMs, use the show environment command.
AC PEM faults can occur for the following reasons:
PEM failure
Invalid AC-input power being supplied by the power source
Backplane interface failures or damage
Figure 2-1 illustrates the AC PEM and its indicators. Table 2-1 describes the indicators.
OL-1237-01
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
2-1
AC PEM Faults
Figure 2-1 AC PEM Front Panel
POWER
FAULT
AC power cord clips
AC power plug
Chapter 2 PEM Faults and Fan Assembly Failures
AC power switch
62520
Table 2-1 AC PEM LEDs
LED Color Description
Power Green The PEM is on, is receiving power from the AC power source, and is providing
power to the Cisco uBR10012 chassis (normal operations).
Fault Yellow Indicates that AC-input power is being received by the PEM, but that the PEM is
not supplying power to the chassis, typically because the PEM’s power switch is turned to the standby position.
If the Fault LED is lit when the power switch is in the ON position, the PEM is not operating correctly.
Table 2-2 lists the AC PEM fault symptoms and corrective actions.
Table 2-2 AC PEM Fault Symptoms and Corrective Action
Fault Symptom Corrective Action
Green LED on PEM fails to light
1. Make sure the power switch on the PEM is turned to the ON position.
2. Make sure the PEM is properly seated and that its captive screws have been
tightened.
2-2
3. Make sure that the AC-input power cord is securely plugged into the power
plug on the front panel of the PEM. Secure the cord in the clips to ensure the plug is not accidentally pulled out.
4. Check the external power source and verify that the AC-input power cord is
correctly connected to the power outlet.
5. Move the PEM to the other PEM slot. If the PEM still fails, replace it.
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Chapter 2 PEM Faults and Fan Assembly Failures
Table 2-2 AC PEM Fault Symptoms and Corrective Action (continued)

DC PEM Faults

PEM experiences problems in one slot but operates normally in a different slot
Fault LED is lit yellow
1. Ensure that the input power to both slots is correct.
2. Verify that no connections have been made to the DC-power connectors
underneath each PEM.
3. If the problem persists, contact Cisco TAC.
1. Verify that no connections have been made to the DC-power connectors
underneath each PEM.
2. Verify that the PEM is fully inserted into the power bay and that its captive
screws have been tightened.
3. Check to see if the power switch is set to the standby position. If so, set the
switch to the ON position.
4. If the problem persists, flip the power switch on the PEM to the standby
position, wait several seconds, and then back to the ON position.
5. Replace PEM with a known good replacement.
6. Contact Cisco TAC.
Tip Securely tighten the captive screws on your PEMs to prevent heightened levels of electromagnetic
interference.
DC PEM Faults
On the Cisco uBR10012 router, two DC PEMs are in a redundant configuration, which allows one DC PEM to fail without affecting system operations. A single PEM can usually power the router for sufficient time to request and install a new PEM to replace the one that failed.
Tip To quickly check the functional status of your PEMs, use the show environment command.
DC PEM faults can occur for the following reasons:
PEM failure
Reversed power cables
Backplane interface failures or damage
Two models of the DC PEM exist.
Figure 2-2 shows the front panel of the original DC PEM (UBR10-PWR-DC) that was initially
produced for the Cisco uBR10012 router.
Figure 2-3 shows the front panel of the DC PEM that is currently being produced for the
Cisco uBR10012 router. The new model of the DC PEM (UBR10-PWR-DC-M) is identical in form and function to the first version, except that it includes a connector on the front panel for connecting to the alarm status connectors on the optional 2400-watt AC-input power shelf.
Table 2-3 describes the indicators on the front panel of both models of DC PEM.
OL-1237-01
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
2-3
DC PEM Faults
Chapter 2 PEM Faults and Fan Assembly Failures
Figure 2-2 DC PEM Front Panel (original model, UBR10-PWR-DC)
POWER MISWIRE FAU LT
56480
Figure 2-3 DC PEM Front Panel (current model with alarm connector, UBR10-PWR-DC-M)
ER
POW
IRE
ISW
M FAULT
2-4
62518
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Chapter 2 PEM Faults and Fan Assembly Failures
Table 2-3 DC PEM LEDs
LED Description
Power (green) PEM is powered on and is operational.
Fault (yellow) PEM is not operating correctly or the circuit breaker is in the OFF position.
Miswire (yellow) Input DC power cables are wired incorrectly and should be reversed.
Table 2-4 lists the DC PEM fault symptoms and corrective actions.
Table 2-4 DC PEM Fault Symptoms and Corrective Action
Fault Symptom Corrective Action
Green LED on PEM fails to light

2400W AC-Input Power Shelf

1. Make sure the circuit breaker on the PEM is turned on.
2. Make sure the PEM is properly seated and screwed in place.
3. Make sure power leads are properly connected to power connectors on the
backplane. If connections are loose or their polarity is reversed, the chassis does not receive power.
4. Check the external power source.
5. Move the PEM to the other PEM slot. If the PEM still fails, replace it.
PEM experiences problems in one slot but operates
1. Ensure that the input power to both slots is correct.
2. If the problem persists, contact Cisco TAC.
normally in a different slot
Fault LED is lit yellow
Miswire LED is lit yellow
1. Check to see if the circuit breaker (on/off switch) has tripped. If it has,
return the switch to the ON position.
2. Replace PEM with a known good replacement.
3. Contact Cisco TAC.
If the MISWIRE LED is on, the power cables are reversed. Power off the PEM and the external power source and reconnect the wires correctly. See the Cisco uBR10012 Universal Broadband Router Hardware Installation Guide.
Tip Securely tighten the captive screws on your PEMs to prevent heightened levels of electromagnetic
interference.
2400W AC-Input Power Shelf
OL-1237-01
The 2400W AC-input power shelf converts AC-output power from an external AC power source into DC power that is suitable for powering the Cisco uBR10012 router. The power shelf supplies –54 VDC output power to the two DC PEMs in the Cisco uBR10012 chassis.
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
2-5

Other Electrical Problems

DC OK
AC OK
FAULT
Chapter 2 PEM Faults and Fan Assembly Failures
The power shelf includes three 1200-watt (W) AC-input power modules that plug into a common power backplane in the 2400W AC-input power shelf. Two 1200W AC-input power modules are capable of powering a fully configured Cisco uBR10012 router. The third power module provides full redundancy.
During normal operation, the three AC-input power modules provide automatic load-sharing with each power module supporting 33 percent of the power load. When you remove one of the AC-input power modules, the remaining power modules immediately ramp up to full power and maintain uninterrupted system power for a limited time. This allows you to replace the affected module without impacting system operations.
Faults on the 2400W AC-input power shelf can occur for the following reasons:
The AC-input power to one or more power modules has failed.
The AC power plug to one or more power modules has been removed or unplugged.
One or more power modules has failed and must be replaced.
Figure 2-4 illustrates the AC PEM and its indicators. Table 2-5 describes the indicators.
Figure 2-4 AC-Input Power Shelf Front Panel
D
C O
K
A
C
O
K
F A
U L
T
D
C
O
K
A
C
O
K
F A
U
L T
Table 2-5 AC-Input Power Shelf Module LEDs
LED Color Description
AC OK Green The AC-input power to the power module is present and is within the proper
range.
DC OK Green The power module is producing DC output power in the proper range.
FAULT Red This particular power module has failed and must be replaced. The 2400W
AC-input power shelf can continue operating with only two out of the three power modules installed, but the failed module should still be replaced as soon as possible.
Other Electrical Problems
D
C
O
K
A
C O
K
F A
U
L T
36137
2-6
If the electrical problem cannot be traced to a PEM, check the unit for:
Improper power cable connections to the Cisco uBR10012 router
Improper installation of other field-replaceable units (FRUs)
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Chapter 2 PEM Faults and Fan Assembly Failures
Check the site for:
Improperly grounded equipment, particularly equipment racks and power grounds
Fluctuating voltage, which can result from excessive power drains caused by other equipment (such
as air conditioning units)
Cable corrosion or defective power panels, circuit breakers or fuses, or cable connections
Undersized power cables or excessive power cable lengths
Excessive power demand on backup power systems or batteries when alternate power sources
are used

Fan Assembly Module Faults

The fan assembly module is critical to the operation of the Cisco uBR10012 router because it allows the router to maintain proper operating temperatures. Severe overheating can result in system failure, so a fan assembly module must always be present in the chassis while the router is operating.
Figure 2-5 shows the fan assembly module front panel and its LED indicators.
Fan Assembly Module Faults
Figure 2-5 Fan Assembly Module
FANS OK
SINGLE FAN FAILURE
MULTIPLE FAN FAILURE
Fan assembly
CISCO 10000
CISCO 10000
A UX
AU
X
AC
TIVIT
E
Y
AC
THE
TIVITY
RN
LINK
E
ETH
T
ERN
LIN
ET
K
56479
The Cisco uBR10012 fan assembly module contains four fans in a redundant configuration. One fan can fail without affecting system operations. If more than one fan fails, however, the fan assembly module must be replaced immediately to avoid overheating the system.
The fan assembly module draws air in from the bottom front of the Cisco uBR10012 router, through the air filter at the bottom of the front bezel. The air is drawn up through the line cards, and then exits through the vents at the top rear of the router.
OL-1237-01
Figure 2-6 shows the air circulation pattern of the Cisco uBR10012 router when two DC PEMs are
installed. The air flow when two AC PEMs are installed is similar. The front bezel is not shown for clarity.
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
2-7
Fan Assembly Module Faults
Figure 2-6 Fan Assembly Air Circulation Pattern
Chapter 2 PEM Faults and Fan Assembly Failures
C
IS
C
O
1
0
0
0
0
C O
N
S O
L E
A U
X
A C
T
I V
IT
E
Y
T
H
E R
SLOT 0
SLOT 1
L
N
I
N
E
T
K
SLOT 0
SLOT 1
A
L
A
R
M
S
A
L
A
A
C O
C
R I T
IC
A
M
L A J
O R
M
PERFORMANCE ROUTING ENGINE
I N O
R
S
T A T
U
S
F
A I L
P
O
W
O
IT
R N
L E
Y
E T
PERFORMANCE ROUTING ENGINE
ER
M
ISW
IRE
FA
ULT
PO
W
ER
M
ISW
IR
E
FAU
LT
C
IS
C
O
1
0
0
0
0
C
O
N S
A
U X
A
C T
I V
E
T H
E
L
I N
K
R
M
S
A C
O
C
R IT
I C
A
M
L A J
O
R
M
IN
O R
S
T A
T U
S
F
A I
L
56430
The LEDs on the front panel indicate the current status of the fans. Tab le 2-6 lists the fan assembly module fault indications and recommended actions.
2-8
Cisco uBR10012 Universal Broadband Router Troubleshooting Guide
OL-1237-01
Loading...
+ 60 hidden pages