Cisco CRS-1 User Manual

Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide

Cisco IOS XR Software Release 3.4
Americas Headquarters
Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000
800 553-NETS (6387)
Text Part Number: OL-12571-01
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
CCVP, the Cisco Logo, and the Cisco Square Bridge logo are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems, Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing, FormShare, GigaDrive, GigaStack, HomeLink, Internet Quotient, IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys, MeetingPlace, MGX, Networking Academy, Network Registrar, Pack e t, PIX, ProConnect, RateMUX, ScriptShare, SlideCast, SMARTnet, StackWise, The Fastest Way to Increase Your Internet Quotient, and TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0612R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
© 2007 Cisco Systems, Inc. All rights reserved.
Preface v
Changes to This Document v
Obtaining Documentation v
Cisco.com v Product Documentation DVD vi Ordering Documentation vi
Documentation Feedback vi
Cisco Product Security Overview vi
Reporting Security Problems in Cisco Products vii
Product Alerts and Field Notices vii
Obtaining Technical Assistance viii
Cisco Technical Support & Documentation Website viii Submitting a Service Request ix Definitions of Service Request Severity ix
Obtaining Additional Publications and Information ix
1-1

CONTENTS

Upgrading to a Multishelf System 1-1
Contents 1-1
Prerequisites for Upgrading to a Multishelf System 1-1
Restrictions for Upgrading to a Multishelf System 1-2
Information About Upgrading to a Multishelf System 1-3
How to Upgrade to a Multishelf System 1-3
Upgrading the Fabric Cards and Adding an FCC 1-4 Adding an LCC to a Multishelf System 1-9
Configuration Examples for Upgrading to a Multishelf System 1-11
Displaying Chassis Serial Numbers: Example 1-11 Adding a Fabric Card Chassis: Example 1-12 Adding an LCC to a Multishelf System: Example 1-15
Where to Go Next 1-17
Additional References 1-17
Related Documents 1-17 Technical Assistance 1-18
2-1
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
iii
Contents
Converting to an Integrated Switch System 2-1
Contents 2-1
Prerequisites for the Integrated Switch System 2-2
Cisco CRS-1 Multishelf Conversion to an Integrated Switch System Introduction 2-2
Cisco CRS-1 Multishelf Integrated Switch Solution 2-3
Benefits of the Cisco CRS-1 Multishelf Integrated Switch Solution 2-3
Naming Conventions 2-3
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and Two-FCC Multishelf Systems
2-4
Validating the 22-port SCGE Cards (Single-FCC Multishelf and Two-FCC Multishelf Systems) 2-4 Connecting the 22-port SCGE to an Active Cat6K (Single-FCC Multishelf and Two-FCC Multishelf
Systems)
2-5
Transferring the Backup Cisco Catalyst 65xx Switch Gigabit Ethernet Connections to a Standby 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems)
2-6
Performing a Failover to a Standby 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems)
2-7
Connecting the 22-port SCGEs into a Full Mesh (Single-FCC Multishelf and Two-FCC Multishelf Systems)
2-8
Transferring an Active Cat6K Gigabit Ethernet Connections to the Active 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems)
2-9
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
2-10
Validating the 22-port SCGE Cards (Four-FCC Multishelf System) 2-10 Connecting the 22-port SCGE to an Active Cat6K (Four-FCC Multishelf System) 2-10 Transferring the Backup Cisco Catalyst 65xx Switch Gigabit Ethernet Connections to a Standby
22-port SCGE (Four-FCC Multishelf System)
2-13
Performing a Failover to a Standby 22-port SCGE (Four-FCC Multishelf System) 2-14 Connecting the 22-port SCGEs into a Full Mesh Configuration (Four-FCC Multishelf System) 2-15 Transferring an Active Cat6K Gigabit Ethernet Connections to the Active 22-port SCGE (Four-FCC
Multishelf System)
2-16
Index
iv
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01

Preface

This guide describes how to upgrade a single-shelf Cisco CRS-1 to a multishelf system. The upgrade process is similar to the process for a new multishelf system installation. This guide documents only the differences between the upgrade process and the new installation process.
The preface contains the following sections:
Changes to This Document, page v
Obtaining Documentation, page v
Documentation Feedback, page vi
Cisco Product Security Overview, page vi
Product Alerts and Field Notices, page vii
Obtaining Technical Assistance, page viii
Obtaining Additional Publications and Information, page ix

Changes to This Document

Table 1 lists the technical changes made to this document since it was first printed.
Table 1 Changes to This Document
Revision Date Change Summary
OL-12571-01 February 2007 Initial release of the document.

Obtaining Documentation

Cisco documentation and additional literature are available on Cisco.com. This section explains the product documentation resources that Cisco offers.
Cisco.com
You can access the most current Cisco documentation at this URL:
http://www.cisco.com/techsupport
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
v

Documentation Feedback

You can access the Cisco website at this URL:
http://www.cisco.com
You can access international Cisco websites at this URL:
http://www.cisco.com/public/countries_languages.shtml
Product Documentation DVD
The Product Documentation DVD is a library of technical product documentation on a portable medium. The DVD enables you to access installation, configuration, and command guides for Cisco hardware and software products. With the DVD, you have access to the HTML documentation and some of the PDF files found on the Cisco website at this URL:
http://www.cisco.com/univercd/home/home.htm
The Product Documentation DVD is created and released regularly. DVDs are available singly or by subscription. Registered Cisco.com users can order a Product Documentation DVD (product number DOC-DOCDVD= or DOC-DOCDVD=SUB) from Cisco Marketplace at the Product Documentation Store at this URL:
Preface
http://www.cisco.com/go/marketplace/docstore
Ordering Documentation
You must be a registered Cisco.com user to access Cisco Marketplace. Registered users may order Cisco documentation at the Product Documentation Store at this URL:
http://www.cisco.com/go/marketplace/docstore
If you do not have a user ID or password, you can register at this URL:
http://tools.cisco.com/RPF/register/register.do
Documentation Feedback
You can provide feedback about Cisco technical documentation on the Cisco Technical Support & Documentation site area by entering your comments in the feedback form available in every online document.
Cisco Product Security Overview
Cisco provides a free online Security Vulnerability Policy portal at this URL:
vi
http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html
From this site, you will find information about how to do the following:
Report security vulnerabilities in Cisco products
Obtain assistance with security incidents that involve Cisco products
Register to receive security information from Cisco
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Preface
A current list of security advisories, security notices, and security responses for Cisco products is available at this URL:
http://www.cisco.com/go/psirt
To see security advisories, security notices, and security responses as they are updated in real time, you can subscribe to the Product Security Incident Response Team Really Simple Syndication (PSIRT RSS) feed. Information about how to subscribe to the PSIRT RSS feed is found at this URL:
http://www.cisco.com/en/US/products/products_psirt_rss_feed.html
Reporting Security Problems in Cisco Products
Cisco is committed to delivering secure products. We test our products internally before we release them, and we strive to correct all vulnerabilities quickly. If you think that you have identified a vulnerability in a Cisco product, contact PSIRT:
For emergencies only— security-alert@cisco.com
An emergency is either a condition in which a system is under active attack or a condition for which a severe and urgent security vulnerability should be reported. All other conditions are considered nonemergencies.

Product Alerts and Field Notices

For nonemergencies —psirt@cisco.com
In an emergency, you can also reach PSIRT by telephone:
1 877 228-7302
1 408 525-6532
Tip We encourage you to use Pretty Good Privacy (PGP) or a compatible product (for example, GnuPG) to
encrypt any sensitive information that you send to Cisco. PSIRT can work with information that has been encrypted with PGP versions 2.x through 9.x.
Never use a revoked encryption key or an expired encryption key. The correct public key to use in your correspondence with PSIRT is the one linked in the Contact Summary section of the Security Vulnerability Policy page at this URL:
http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html
The link on this page has the current PGP key ID in use.
If you do not have or use PGP, contact PSIRT to find other means of encrypting the data before sending any sensitive material.
Product Alerts and Field Notices
Modifications to or updates about Cisco products are announced in Cisco Product Alerts and Cisco Field Notices. You can receive Cisco Product Alerts and Cisco Field Notices by using the Product Alert Tool on Cisco.com. This tool enables you to create a profile and choose those products for which you want to receive information.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
vii

Obtaining Technical Assistance

To access the Product Alert Tool, you must be a registered Cisco.com user. (To register as a Cisco.com user, go to this URL: http://tools.cisco.com/RPF/register/register.do) Registered users can access the tool at this URL: http://tools.cisco.com/Support/PAT/do/ViewMyProfiles.do?local=en
Obtaining Technical Assistance
Cisco Technical Support provides 24-hour-a-day award-winning technical assistance. The Cisco Technical Support & Documentation website on Cisco.com features extensive online support resources. In addition, if you have a valid Cisco service contract, Cisco Technical Assistance Center (TAC) engineers provide telephone support. If you do not have a valid Cisco service contract, contact your reseller.
Cisco Technical Support & Documentation Website
The Cisco Technical Support & Documentation website provides online documents and tools for troubleshooting and resolving technical issues with Cisco products and technologies. The website is available 24 hours a day at this URL:
Preface
http://www.cisco.com/techsupport
Access to all tools on the Cisco Technical Support & Documentation website requires a Cisco.com user ID and password. If you have a valid service contract but do not have a user ID or password, you can register at this URL:
http://tools.cisco.com/RPF/register/register.do
Note Use the Cisco Product Identification Tool to locate your product serial number before submitting a
request for service online or by phone. You can access this tool from the Cisco Technical Support & Documentation website by clicking the Tools & Resources link, clicking the All Tools (A-Z) tab, and then choosing Cisco Product Identification Tool from the alphabetical list. This tool offers three search options: by product ID or model name; by tree view; or, for certain products, by copying and pasting show command output. Search results show an illustration of your product with the serial number label location highlighted. Locate the serial number label on your product and record the information before placing a service call.
Tip Displaying and Searching on Cisco.com
If you suspect that the browser is not refreshing a web page, force the browser to update the web page by holding down the Ctrl key while pressing F5.
To find technical information, narrow your search to look in technical documentation, not the entire Cisco.com website. On the Cisco.com home page, click the Advanced Search link under the Search box and then click the Technical Support & Documentation radio button.
viii
To provide feedback about the Cisco.com website or a particular technical document, click Contacts & Feedback at the top of any Cisco.com web page.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Preface
Submitting a Service Request
Using the online TAC Service Request Tool is the fastest way to open S3 and S4 service requests. (S3 and S4 service requests are those in which your network is minimally impaired or for which you require product information.) After you describe your situation, the TAC Service Request Tool provides recommended solutions. If your issue is not resolved using the recommended resources, your service request is assigned to a Cisco engineer. The TAC Service Request Tool is located at this URL:
http://www.cisco.com/techsupport/servicerequest
For S1 or S2 service requests, or if you do not have Internet access, contact the Cisco TAC by telephone. (S1 or S2 service requests are those in which your production network is down or severely degraded.) Cisco engineers are assigned immediately to S1 and S2 service requests to help keep your business operations running smoothly.
To open a service request by telephone, use one of the following numbers:
Asia-Pacific: +61 2 8446 7411 Australia: 1 800 805 227 EMEA: +32 2 704 55 55 USA: 1 800 553 2447

Obtaining Additional Publications and Information

For a complete list of Cisco TAC contacts, go to this URL:
http://www.cisco.com/techsupport/contacts
Definitions of Service Request Severity
To ensure that all service requests are reported in a standard format, Cisco has established severity definitions.
Severity 1 (S1)—An existing network is “down” or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation.
Severity 2 (S2)—Operation of an existing network is severely degraded, or significant aspects of your business operations are negatively affected by inadequate performance of Cisco products. You and Cisco will commit full-time resources during normal business hours to resolve the situation.
Severity 3 (S3)—Operational performance of the network is impaired while most business operations remain functional. You and Cisco will commit resources during normal business hours to restore service to satisfactory levels.
Severity 4 (S4)—You require information or assistance with Cisco product capabilities, installation, or configuration. There is little or no effect on your business operations.
Obtaining Additional Publications and Information
OL-12571-01
Information about Cisco products, technologies, and network solutions is available from various online and printed sources.
The Cisco Online Subscription Center is the website where you can sign up for a variety of
Cisco e-mail newsletters and other communications. Create a profile and then select the subscriptions that you would like to receive. To visit the Cisco Online Subscription Center, go to this URL:
http://www.cisco.com/offer/subscribe
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
ix
Obtaining Additional Publications and Information
The Cisco Product Quick Reference Guide is a handy, compact reference tool that includes brief
product overviews, key features, sample part numbers, and abbreviated technical specifications for many Cisco products that are sold through channel partners. It is updated twice a year and includes the latest Cisco channel product offerings. To order and find out more about the Cisco Product Quick Reference Guide, go to this URL:
http://www.cisco.com/go/guide
Cisco Marketplace provides a variety of Cisco books, reference guides, documentation, and logo
merchandise. Visit Cisco Marketplace, the company store, at this URL:
http://www.cisco.com/go/marketplace/
Cisco Press publishes a wide range of general networking, training, and certification titles. Both new
and experienced users will benefit from these publications. For current Cisco Press titles and other information, go to Cisco Press at this URL:
http://www.ciscopress.com
Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering
professionals involved in designing, developing, and operating public and private internets and intranets. You can access the Internet Protocol Journal at this URL:
http://www.cisco.com/ipj
Preface
Networking products offered by Cisco Systems, as well as customer support services, can be
obtained at this URL:
http://www.cisco.com/en/US/products/index.html
Networking Professionals Connection is an interactive website where networking professionals
share questions, suggestions, and information about networking products and technologies with Cisco experts and other networking professionals. Join a discussion at this URL:
http://www.cisco.com/discuss/networking
“What’s New in Cisco Documentation” is an online publication that provides information about the
latest documentation releases for Cisco products. Updated monthly, this online publication is organized by product category to direct you quickly to the documentation for your products. You can view the latest release of “What’s New in Cisco Documentation” at this URL:
http://www.cisco.com/univercd/cc/td/doc/abtunicd/136957.htm
World-class networking training is available from Cisco. You can view current offerings at
this URL:
http://www.cisco.com/en/US/learning/index.html
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
x
OL-12571-01
CHA PTER
1

Upgrading to a Multishelf System

This chapter describes how to upgrade a single-chassis Cisco CRS-1 Carrier Routing System router to a Cisco CRS-1 Carrier Routing System Multishelf System.
Note For an introduction to the multishelf system, see Cisco CRS-1 Carrier Routing System Multishelf System
Description. For information on planning a multishelf system installation, see Cisco CRS-1 Carrier Routing System Multishelf System Site Planning Guide.
Feature History for the Cisco CRS-1 Multishelf System
Release Modification
Release 3.3.0 The Cisco CRS-1 Multishelf System was introduced.
Release 3.4.0 No modification.

Contents

This chapter contains the following sections:
Prerequisites for Upgrading to a Multishelf System, page 1-1
Restrictions for Upgrading to a Multishelf System, page 1-2
Information About Upgrading to a Multishelf System, page 1-3
How to Upgrade to a Multishelf System, page 1-3
Configuration Examples for Upgrading to a Multishelf System, page 1-11
Where to Go Next, page 1-17
Additional References, page 1-17

Prerequisites for Upgrading to a Multishelf System

An upgrade kit is required to use an existing Cisco CRS-1 16-Slot Line Card Chassis as a component in a multishelf system. The upgrade kit is product ID CRS-16-MC-UPG. The upgrade kit includes fabric cards (S13 cards should replace the S123 cards) and cabling.
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-1

Restrictions for Upgrading to a Multishelf System

Prepare the single chassis system as follows:
Obtain the chassis serial number, which is required for configuration. The serial number is on a chassis label and can be displayed using the show diag chassis, as described in Cisco IOS XR
Getting Started Guide.
Upgrade the ROM Monitor software to Version 1.42 or later, as described in Cisco IOS XR ROM Monitor Guide.
For more information, see the documents in the “Related Cisco CRS-1 Multishelf Hardware
Documentation” section on page 1-17.
Prepare each fabric card chassis (FCC) as follows:
Install the FCCs and distribute the eight fabric card chassis-switch fabric cards (FCC-SFCs, product ID CRS-FCC-SFC) among the FCCs.
Obtain the chassis serial number for each FCC, which is required for configuration. The serial number is on a chassis label and can be displayed using the show diag chassis, as described in
Cisco IOS XR Getting Started Guide.
Ensure that the power to the FCCs is off.
Connect all fabric cables to the FCCs, as described in Cisco CRS-1 Carrier Routing System Multishelf System Interconnection and Cabling Guide.
Chapter 1 Upgrading to a Multishelf System
For more information, see the documents in the “Related Cisco CRS-1 Multishelf Hardware
Documentation” section on page 1-17.
Note You cannot connect the fabric cables to the pre-existing single-chassis rack until you replace the FC/S
cards in the line card chassis (LCC). To avoid service interruption, replace these modules and connect the fabric cables only when instructed to do so in the procedure that appears in the “How to Upgrade to
a Multishelf System” section on page 1-3.
Prepare the additional LCC as follows:
Install the LCC and all cards.
Obtain the chassis serial number, which is required for configuration.
Ensure that the power to the LCC is off.
For more information, see the documents in the “Related Cisco CRS-1 Multishelf Hardware
Documentation” section on page 1-17.
Prepare the control network (Catalyst 6509 switches) as follows:
Install the Catalyst 6509 switches.
Cable the Catalyst 6509 switches to the FCC and both LCCs, as described in Cisco CRS-1 Carrier Routing System Multishelf System Interconnection and Cabling Guide.
Configure the Catalyst 6509 switches, as described in Cisco IOS XR Getting Started Guide.
For more information, see the documents in the “Related Documentation for the Catalyst 6509
Switch” section on page 1-18.
Restrictions for Upgrading to a Multishelf System
Cisco IOS XR Software Release supports the multishelf feature only on the 16-slot LCCs. This release does not support multishelf operation using 8-slot LCCs.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-2
OL-12571-01
Chapter 1 Upgrading to a Multishelf System

Information About Upgrading to a Multishelf System

Information About Upgrading to a Multishelf System
The difference between upgrading a single-chassis system to a multishelf system and installing a multishelf system is the fabric upgrade. A single-chassis system uses fabric cards designed for single-chassis systems (FC/S cards), and an LCC in a multishelf system uses fabric cards designed for a multishelf system (FC/M cards). On a single chassis system, each fabric card represents one fabric plane. To upgrade a single-chassis system to a multishelf system without interrupting service, you must do the following:
1. Use CLI commands to prepare each FC/S fabric card for replacement with an FC/M card.
2. Shut down the plane on each FC/S card before it is replaced.
3. Replace the FC/S card with an FC/M card.
4. Bring up the FC/M card.
5. Repeat Step 2 through Step 4 until all planes (0 through 7) are upgraded.
When you plan to perform a single-shelf to multishelf system upgrade, consider the following:
You need these components:
Fabric card chassis and new line card chassis.
Cisco Catalyst 6509 Switch, which provides the control Ethernet network for the multishelf system control traffic. We highly recommend two switches for redundancy.
S13 cards for the original line card chassis (to replace the original S123 cards in the chassis).
Optical array cables (48 cables for each multishelf system and 24 cables for each line card chassis).
Additional MSCs and PLIMs for the new line card chassis.
To avoid traffic loss, you must upgrade the switch fabric one plane at a time. To do that, you must
replace each FC/S card with a new FC/M card and restore service to that fabric plane before upgrading the next fabric plane.
You might want to perform the upgrade during a network maintenance window or when system
traffic is light.
Note Be sure to review the upgrade procedures before deciding when to schedule the upgrade.
That way, you can familiarize yourself with the upgrade procedure and determine if there are other issues to consider before performing the upgrade.
Are there cabling issues to consider when you add the new chassis to the existing chassis?
For example, will you have to install optical chassis interconnect cables between the chassis before you bolt the chassis to the floor? If so, be sure to protect the cables while moving the chassis.

How to Upgrade to a Multishelf System

To upgrade a single-chassis system to a multishelf system, you must complete the following tasks:
Upgrading the Fabric Cards and Adding an FCC, page 1-4
Adding an LCC to a Multishelf System, page 1-9
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
1-3
How to Upgrade to a Multishelf System
Upgrading the Fabric Cards and Adding an FCC
This section describes how to upgrade the fabric cards in a single-chassis system and establish communications between the designated shelf controller (DSC) LCC and one or more FCCs.
Prerequisites
Software Requirements
Cisco IOS XR Software Release 3.4.0 or later
ROMMON 1.42 or later
Caution The ROM Monitor software must be upgraded to version 1.42 or higher on all RPs before a Cisco CRS-1
system is upgraded to Cisco IOS XR Software Release 3.4.0 or higher release. If the router is brought up with an incompatible version of the ROM Monitor software, then the standby RP may fail to boot. For instructions to overcome a boot block in the standby RP in a single chassis system, see Cisco IOS XR ROM Monitor Guide. If a boot block occurs in a multishelf system, contact your Cisco Systems support representative for assistance. See Obtaining Technical Assistance, page viii.
Chapter 1 Upgrading to a Multishelf System
Restrictions
SUMMARY STEPS
In addition, Cisco CRS-1 Multishelf systems should be upgraded to ROMMON release 1.42 before being upgraded to IOS XR Release 3.4.0 to ensure RPs are assigned the correct rack numbers during system boot.
For more information, see Cisco IOS XR ROM Monitor Guide.
Hardware Requirements
The system hardware should be prepared as described in the “Prerequisites for Upgrading to a
Multishelf System” section on page 1-1.
The control network must be set up as described in the “Prerequisites for Upgrading to a Multishelf
System” section on page 1-1.
The power should be off for the FCCs and any additional LCCs.
None.
1. admin
2. configure
3. dsc serial serialNumber rack 0
1-4
4. dsc serial serialNumber rack Frack
5. controllers fabric plane planeNumber
oim count 1 oim instance 0 location Frack/slot/FM
6. commit
7. end
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System
8. Apply power to the FCC.
9. show platform Frack/**/*
10. configure
11. do show controllers fabric plane all
12. controllers fabric plane planeNumber shutdown
13. commit
14. end
15. In Rack 0, remove the FC/S card for the plane that was shut down in Step 12.
16. In Rack 0, insert the FC/M card for the plane that was shut down in Step 12.
17. show platform 0/smslotNumber/sp
18. In Rack 0, attach the fabric cable connectors to the plane that was shut down in Step 12.
19. In the appropriate FCC, check the LEDs on the appropriate OIM-LED panel for the cables connected
to the plane that is being upgraded.
20. configure
21. do show controllers fabric plane planeNumber detail
How to Upgrade to a Multishelf System
DETAILED STEPS
Command or Action Purpose
Step 1
admin
Example:
RP/0/RP1/CPU0:router# admin
Step 2
configure
Example:
RP/0/RP1/CPU0:router(admin)#configure
Step 3
dsc serial
Example:
RP/0/RP1/CPU0:router(admin-config)# dsc serial TBA08260159 rack 0
22. no controllers fabric plane planeNumber shutdown
23. commit
24. end
25. Repeat Step 9 through Step 24 for each fabric plane.
26. show controllers fabric plane all
Places the router in administration EXEC mode.
All commands listed in this procedure should be
entered on the pre-existing single-chassis system.
Places the router in administration configuration mode.
serialNumber
rack 0
Configures the pre-existing single-chassis system as the DSC rack in the new multishelf system.
Replace serialNumber with the serial number of the
pre-existing single-chassis system.
If you are configuring the system from a remote
location, you can use a command to display the serial number. For more information, see Cisco IOS XR
Getting Started Guide.
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-5
How to Upgrade to a Multishelf System
Command or Action Purpose
Step 4
dsc serial
serialNumber
rack F
rack
Chapter 1 Upgrading to a Multishelf System
Configures the FCC identified by the serial number as an FCC rack within the multishelf system.
Step 5
Example:
RP/0/RP1/CPU0:router(admin-config)# dsc serial TBC0820052000000 rack F0
controllers fabric plane oim count 1 oim instance 0 location F
planeNumber
rack/slot
/FM
Example:
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 0
RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim
instance 0 location F0/SM9/FM
Replace serialNumber with the serial number of the
FCC.
Replace rack with the FCC number as it appears in the
fabric cabling plan, which is described in Cisco CRS-1 Carrier Routing System Multishelf System Interconnection and Cabling Guide.
If you are configuring the system from a remote
location, you can use a command to display the serial number. For more information, see Cisco IOS XR
Getting Started Guide.
Configures a plane to operate in an FCC slot.
Enter this command sequence for each of the eight
fabric planes.
Replace planeNumber with the number of the plane (0
to 7) you want to configure.
Replace rack with the FCC rack number assigned to the
FCC that hosts the plane.
Replace slot with the FCC slot number that supports the
fabric plane you are configuring. Valid slot numbers are SM0 to SM23.
The plane numbers and slot numbers are determined by
the hardware installation and cabling. The software configuration must match the hardware configuration. For more information, see Cisco CRS-1 Carrier
Routing System Multishelf System Interconnection and Cabling Guide.
Step 6
Step 7
Step 8
1-6
Enter this command once for each of the eight planes.
Note These configuration commands are ignored when
the FC/S cards are installed.
commit
Commits the target configuration to the router running configuration.
Example:
RP/0/RP1/CPU0:router(admin-config)# commit
end
Changes the mode from administration configuration mode to administration EXEC mode.
Example:
RP/0/RP1/CPU0:router(admin-config)# end
Apply power to the FCCs. Starts the FCCs.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System
Command or Action Purpose
Step 9
show platform F
rack
/**/*
Example:
RP/0/RP1/CPU0:router(admin)# show platform F0/**/*
Step 10
configure
Example:
RP/0/RP1/CPU0:router(admin)#configure
Step 11
do show controllers fabric plane all
How to Upgrade to a Multishelf System
Displays the status of all FCC modules in the specified rack.
Replace rack with the rack number of the FCC to
examine.
Repeat this command for all FCCs.
The state for all modules should be IOS-XR RUN.
It can take a few minutes for all FCC modules to start.
Note The FCC module status appears only when the show
platform command is executed in administration EXEC mode.
Places the router in administration configuration mode.
Displays the administrative and operational status of all eight fabric planes.
Step 12
Step 13
Step 14
Step 15
Step 16
Example:
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric plane all
controllers fabric plane
planeNumber
shutdown
Example:
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 0 shutdown
commit
Example:
RP/0/RP1/CPU0:router(admin-config)# commit
end
Example:
RP/0/RP1/CPU0:router(admin-config)# end
In Rack 0, remove the FC/S card for the plane that was shut down in Step Step 12.
In Rack 0, insert the FC/M card for the plane that was shut down in Step Step 12.
The do command prefix allows the EXEC mode show
command to execute in administration configuration mode.
Caution To prevent service interruption, do not continue
until the administrative and operational status for all eight planes is UP.
Modifies the target configuration to shut down the specified plane number.
Replace the planeNumber parameter with the number
of the plane you want to shut down.
Commits the target configuration to the router running configuration.
This step shuts down the plane identified in the
previous step.
Changes the mode from administration configuration mode to administration EXEC mode.
Creates room for the FC/M card that is required for multishelf operation.
Provides the hardware required for communication with the FCC.
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-7
How to Upgrade to a Multishelf System
Command or Action Purpose
Step 17
show platform 0/sm
slotNumber
/sp
Chapter 1 Upgrading to a Multishelf System
Displays the status of the Rack 0 fabric slot specified by slotNumber.
Step 18
Step 19
Step 20
Step 21
Step 22
Example:
RP/0/RP1/CPU0:router(admin)# show platform 0/sm0/sp
In Rack 0, attach the fabric cable connectors to the plane that was shut down in Step Step 12.
In the appropriate FCC, check the LEDs on the appropriate OIM-LED panel for the cables connected to the plane that is being upgraded.
configure
Example:
RP/0/RP1/CPU0:router(admin)#configure
do show controllers fabric plane detail
planeNumber
Example:
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric plane 0 detail
no controllers fabric plane shutdown
planeNumber
Note The fabric card status appears only when the show
platform command is executed in administration
EXEC mode.
Completes the connection between a plane in the LCC and the same plane in the FCC.
Green LEDs indicate that the cables are connected correctly.
If the LEDs display a color other than green, see Cisco
IOS XR Getting Started Guide for information on
interpreting the LED display.
Places the router in administration configuration mode.
Displays the status of the plane specified by planeNumber.
Verify that a capital “P” appears in the Down Flags
column.
Wait for the plane to come up before you continue.
Modifies the target configuration to bring up the specified fabric plane.
Step 23
Step 24
Step 25
Step 26
Example:
RP/0/RP1/CPU0:router(admin-config)# no controllers fabric plane 0 shutdown
commit
Example:
RP/0/RP1/CPU0:router(admin-config)# commit
end
Example:
RP/0/RP1/CPU0:router(admin-config)# end
Repeat Step 9 through Step 24 for each fabric plane.
show controllers fabric plane all
Example:
RP/0/RP1/CPU0:router(admin)# show controllers fabric plane all
Commits the target configuration to the router running configuration.
This step brings up the previously shutdown plane,
which is now configured to use the FCC-SFC.
Changes the mode from administration configuration mode to administration EXEC mode.
Displays the administrative and operational status of all eight fabric planes.
Verify that all fabric planes are operational.
1-8
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System
What to Do Next
When the Rack 0 upgrade is completed and Rack 0 is communicating with FCCs, the next step is to add a second LCC to the multishelf system.
Adding an LCC to a Multishelf System
This section describes how to add an LCC to a multishelf system.
Prerequisites
Software Requirements
Cisco IOS XR Software Release 3.4.0 or later
ROMMON 1.42 or later
Hardware Requirements
The LCC to be added must be prepared as described in the “Prerequisites for Upgrading to a
Multishelf System” section on page 1-1.
How to Upgrade to a Multishelf System
Restrictions
SUMMARY STEPS
The control network must be operational and connected to all chassis.
The power should be off for the LCC to be added.
None.
1. admin
2. configure
3. dsc serial serialNumber rack 1
4. controllers fabric rack 1 install-mode
5. commit
6. Apply power to the new LCC (Rack 1).
7. Connect all fabric cables that connect the fabric planes in the new LCC to the FCCs.
8. In the FCCs, check the LEDs for the cables that connect to the new LCC (Rack 1).
9. do show controllers fabric rack-status all detail
10. do show controllers fabric fabric-backpressure summary
11. no controllers fabric rack 1 install-mode
OL-12571-01
12. commit
13. do show controllers rack-status all detail
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-9
How to Upgrade to a Multishelf System
DETAILED STEPS
Command or Action Purpose
Step 1
admin
Example:
RP/0/RP1/CPU0:router# admin
Step 2
configure
Example:
RP/0/RP1/CPU0:router(admin)#configure
Step 3
dsc serial
serialNumber
rack 1
Chapter 1 Upgrading to a Multishelf System
Places the router in administration EXEC mode.
All commands listed in this procedure should be
entered on the pre-existing single-chassis system.
Places the router in administration configuration mode.
Configures the additional LCC as Rack 1 in the multishelf system.
Step 4
Step 5
Step 6
Step 7
Step 8
Step 9
Example:
RP/0/RP1/CPU0:router(admin-config)# dsc serial TBA08440024 rack 1
Replace the serialNumber parameter with the serial
number of the additional LCC.
If you are configuring the system from a remote
location, you can use a command to display the serial number. For more information, see Cisco IOS XR
Getting Started Guide.
controllers fabric rack 1 install-mode
Modifies the target configuration to change the Rack 1 configuration to installation mode.
Example:
RP/0/RP1/CPU0:router(admin-config)# controllers fabric rack 1 install-mode
commit
Commits the target configuration to the router running configuration.
Example:
RP/0/RP1/CPU0:router(admin-config)# commit
Apply power to the new LCC (Rack 1). Starts up the second LCC (Rack 1).
Connect all fabric cables that connect the fabric planes
Interconnects the fabric cards in the LCC and FCC.
in the new LCC to the FCCs.
In the FCCs, check the LEDs for the cables that connect to the new LCC (Rack 1).
Green LEDs indicate that the cables are connected correctly.
If the LEDs display a color other than green, see Cisco
IOS XR Getting Started Guide for information on
interpreting the LED display.
do show controllers fabric rack-status all detail
Example:
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric rack-status all detail
Displays the status of all racks and additional information for racks in installation mode.
Wait for the status in the Rack in Install and Rack out
of Install columns to change to UP for all planes.
1-10
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System
Command or Action Purpose
Step 10
Step 11
Step 12
Step 13
do show controllers fabric fabric-backpressure summary
Example:
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric fabric-backpressure summary
no controllers fabric rack 1 install-mode
Example:
RP/0/RP1/CPU0:router(admin-config)# no controllers fabric rack 1 install-mode
commit
Example:
RP/0/RP1/CPU0:router(admin-config)# commit
do show controllers rack-status all detail

Configuration Examples for Upgrading to a Multishelf System

Displays the backpressure status for all racks.
The status for the row labeled “Rack 1: All Groups
Received? :” should be “Yes.”
Modifies the target configuration to change the Rack 1 configuration to normal mode.
Commits the target configuration to the router running configuration.
Displays the status of all racks in the system.
In a properly operating system, the rack status for all
Example:
RP/0/RP1/CPU0:router(admin-config)# do show controllers rack-status all detail
racks should be Normal, and the server status should be Present.
What to Do Next
When all chassis in the multishelf system are operational and communicating with each other, it is time to continue system configuration, as described in the documents in the “Related Documents” section on
page 1-17.
Troubleshooting Tips
For troubleshooting information, see the documents described in the “Related Documents” section on
page 1-17.
Configuration Examples for Upgrading to a Multishelf System
This section provides examples for the following procedures:
Displaying Chassis Serial Numbers: Example, page 1-11
Adding a Fabric Card Chassis: Example, page 1-12
Adding an LCC to a Multishelf System: Example, page 1-15
Displaying Chassis Serial Numbers: Example
The following example shows how to display the chassis serial numbers in a Cisco CRS-1 Multishelf system.
RP/0/RP0/CPU0:router(admin)# show diag chassis
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
1-11
Configuration Examples for Upgrading to a Multishelf System
RACK 0 : MAIN: board type 0001e0 800-24872-01 rev 20 dev N/A S/N TBA08260159 PCA: 73-7640-05 rev 20 PID: CRS-16-LCC VID: V01 CLEI: IPM6700DRA ECI: 445022 RACK NUM: 0
RACK 1 :
MAIN: board type 0001e0 800-24872-01 rev 20
dev N/A S/N TBA08260159 PCA: 73-7640-05 rev 20 PID: CRS-16-LCC VID: V01 CLEI: IPM6700DRA ECI: 445022 RACK NUM: 0
Chapter 1 Upgrading to a Multishelf System
RACK 240 : MAIN: board type 0001e0 800-24872-01 rev 20 dev N/A S/N TBA08260159 PCA: 73-7640-05 rev 20 PID: CRS-16-LCC VID: V01 CLEI: IPM6700DRA ECI: 445022 RACK NUM: 0
Adding a Fabric Card Chassis: Example
The following example shows how to add a single FCC to an existing single-chassis system.
RP/0/RP1/CPU0:router# admin
RP/0/RP1/CPU0:router(admin)# configure
RP/0/RP1/CPU0:router(admin-config)# dsc serial TBC0820052000001 rack 0
RP/0/RP1/CPU0:router(admin-config)# dsc serial TBC0820052000000 rack F0
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 0 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM9/FM
1-12
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 1 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM6/FM
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 2 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM3/FM
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 3 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM0/FM
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 4 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM12/FM
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 5 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM15/FM
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 6 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM18/FM
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 7 RP/0/RP1/CPU0:router(admin-config)# oim count 1 RP/0/RP1/CPU0:router(admin-config)# oim instance 0 location F0/SM21/FM
RP/0/RP1/CPU0:router(admin-config)# commit
RP/0/RP1/CPU0:router(admin-config)# end
Configuration Examples for Upgrading to a Multishelf System
Power is applied to the FCC at this time.
RP/0/RP1/CPU0:router(admin)# show platform F0/**/*
Node Type PLIM State Config State
----------------------------------------------------------------------------­F0/SM0/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM3/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM6/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM9/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM12/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM15/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM18/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SM21/SP FCC-SFC(SP) FCC-FM-1S IOS XR RUN PWR,NSHUT,MON F0/SC0/CPU0 FCC-SC(Standby) N/A IOS XR RUN PWR,NSHUT,MON F0/SC1/CPU0 FCC-SC(Active) N/A IOS XR RUN PWR,NSHUT,MON F0/AM0/SP ALARM(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/AM1/SP ALARM(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/LM0/SP FCC-LED(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/LM1/SP FCC-LED(SP) N/A IOS XR RUN PWR,NSHUT,MON
RP/0/RP1/CPU0:router(admin)# configure
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric plane all
Flags: P - plane admin down, p - plane oper down C - card admin down, c - card oper down L - link port admin down, l - linkport oper down A - asic admin down, a - asic oper down B - bundle port admin Down, b - bundle port oper down I - bundle admin down, i - bundle oper down N - node admin down, n - node down o - other end of link down d - data down f - failed component downstream m - plane multicast down
OL-12571-01
Plane Admin Oper Id State State
--------------------
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-13
Configuration Examples for Upgrading to a Multishelf System
0 UP UP 1 UP UP 2 UP UP 3 UP UP 4 UP UP 5 UP UP 6 UP UP 7 UP UP
RP/0/RP1/CPU0:router(admin-config)# controllers fabric plane 0 shutdown
RP/0/RP1/CPU0:router(admin-config)# commit
RP/0/RP1/CPU0:router(admin-config)# end
RP/0/RP1/CPU0:Mar 4 18:37:55.055 : fsdb_aserver[173]: %FABRIC-FSDB-1-PLANE_UPDOWN : Plane 0 state changed to DOWN: RP/0/RP1/CPU0:Mar 4 18:37:55.088 : config[65733]: %MGBL-LIBTARCFG-6-ADMIN_COMMIT : Administration configuration committed by user 'user_a'.
The FC/S card is replaced with an FC/M card at this point.
RP/0/RP1/CPU0:Mar 4 18:38:32.680 : oir_daemon[245]: %PLATFORM-OIRD-5-OIROUT : OIR: Node 0/SM0/SP removed RP/0/RP1/CPU0:Mar 4 18:38:54.328 : oir_daemon[245]: %PLATFORM-OIRD-5-OIRIN : OIR: Node 0/SM0/SP inserted SP/0/SM0/SP:Mar 4 18:40:52.575 : alphadisplay[100]: %PLATFORM-ALPHA_DISPLAY-6-CHANGE : Alpha display on node 0/SM0/SP changed to IOS-XR in state default SP/0/SM0/SP:Mar 4 18:41:34.027 : sfe_drvr[108]: %FABRIC-FABRIC_DRVR-6-ASIC_INITIALIZED : Fabric ASICs initialized
Chapter 1 Upgrading to a Multishelf System
RP/0/RP1/CPU0:router(admin)# show platform 0/sm9/sp
0/SM9/SP FC/M(SP) N/A IOS-XR RUN PWR,NSHUT,MON
The fabric cable is attached to the FC/M card at this point.
RP/0/RP1/CPU0:router(admin)# configure
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric plane 0 detail ... Plane Admin Oper Down Total Down Id State State Flags Bundles Bundles
-----------------------------------------------------­ 0 DOWN DOWN pPm 9 6
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric plane 0 detail ... Plane Admin Oper Down Total Down Id State State Flags Bundles Bundles
-----------------------------------------------------­ 0 DOWN DOWN P 9 6
RP/0/RP1/CPU0:iox(admin-config)# no controllers fabric plane 0 shutdown
RP/0/RP1/CPU0:iox(admin-config)# commit
RP/0/RP1/CPU0:Mar 4 18:47:42.930 : fsdb_aserver[173]: %FABRIC-FSDB-1-PLANE_UPDOWN : Plane 0 state changed to UP: RP/0/RP1/CPU0:Mar 4 18:47:42.954 : config[65733]: %MGBL-LIBTARCFG-6-ADMIN_COMMIT : Administration configuration committed by user 'user_a'.
1-14
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System
RP/0/RP1/CPU0:router(admin-config)# end
RP/0/RP1/CPU0:router(admin)# show controllers fabric plane all
Flags: P - plane admin down, p - plane oper down C - card admin down, c - card oper down L - link port admin down, l - linkport oper down A - asic admin down, a - asic oper down B - bundle port admin Down, b - bundle port oper down I - bundle admin down, i - bundle oper down N - node admin down, n - node down o - other end of link down d - data down f - failed component downstream m - plane multicast down
Plane Admin Oper Id State State
-------------------­ 0 UP UP 1 UP UP 2 UP UP 3 UP UP 4 UP UP 5 UP UP 6 UP UP 7 UP UP
Configuration Examples for Upgrading to a Multishelf System
Adding an LCC to a Multishelf System: Example
The following example shows how to add an LCC to a multishelf system.
RP/0/RP1/CPU0:router# admin
RP/0/RP1/CPU0:router(admin)# configure
RP/0/RP1/CPU0:router(admin-config)# dsc serial TBC0820031000000 rack 1
RP/0/RP1/CPU0:router(admin-config)# control fabric rack 1 install-mode
RP/0/RP1/CPU0:router(admin-config)# commit
Apply power, attach cables, and check cable LEDs.
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric rack-status all detail
Rack Rack Server Num Status Status
---- ------ ------
0 NORMAL PRESENT 1 INSTALL PRESENT Oper State
---------------------­ Plane Admin Rack in Rack out Num State Install of Install
----- ----- ------- ---------­ 0 UP UP UP 1 UP UP DOWN 2 UP UP DOWN 3 UP UP DOWN 4 UP UP DOWN
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-15
Configuration Examples for Upgrading to a Multishelf System
5 UP UP DOWN 6 UP UP DOWN 7 UP UP DOWN
F0 NORMAL PRESENT
RP/0/RP1/CPU0:router(admin-config)# do show controllers fabric rack-status all detail
Rack Rack Server Num Status Status
---- ------ ------
0 NORMAL PRESENT 1 INSTALL PRESENT Oper State
---------------------­ Plane Admin Rack in Rack out Num State Install of Install
----- ----- ------- ---------­ 0 UP UP UP 1 UP UP UP 2 UP UP UP 3 UP UP UP 4 UP UP UP 5 UP UP UP 6 UP UP UP 7 UP UP UP
Chapter 1 Upgrading to a Multishelf System
F0 NORMAL PRESENT
RP/0/RP1/CPU0:router(admin-config)# show controllers fabric fabric-backpressure summary
Expected BP Fabric Groups in the System: 0 1 2 3
Rack 0: All Groups Received? : Yes
------
Rack 1: All Groups Received? : Yes
RP/0/RP1/CPU0:router(admin-config)# no controllers fabric rack 1 install-mode
RP/0/RP1/CPU0:router(admin-config)# commit
RP/0/RP1/CPU0:router(admin-config)# do show controllers rack-status all detail
Rack Rack Server Num Status Status
---- ------ ------
0 NORMAL PRESENT 1 NORMAL PRESENT F0 NORMAL PRESENT
1-16
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 1 Upgrading to a Multishelf System

Where to Go Next

When all fabric planes on both LCCs are configured and connected to the FCCs, see Cisco IOS XR Getting Started Guide for information on general system setup and operation.

Additional References

The following sections provide references related to upgrading a single-chassis system to a multishelf system.
Related Documents
Related Cisco CRS-1 Multishelf Hardware Documentation, page 1-17
Related Documentation for the Catalyst 6509 Switch, page 1-18
Related Cisco IOS XR Software Documentation, page 1-18
Where to Go Next
Related Cisco CRS-1 Multishelf Hardware Documentation
For additional documentation related to hardware installation and site planning, see the following Cisco Systems documents:
Related Topic Document Title
Multishelf system description and installation planning Cisco CRS-1 Carrier Routing System Multishelf System Description
Cisco CRS-1 Carrier Routing System Multishelf System Planning Guide
FCC installation Cisco CRS-1 Carrier Routing System Fabric Card Chassis Site
Planning Guide
Installing the Cisco CRS-1 Carrier Routing System Fabric Card Chassis
LCC installation Cisco CRS-1 Carrier Routing System 16-Slot Line Card Chassis
System Description
Installing the Cisco CRS-1 Carrier Routing System 16-Slot Line Card Chassis
Cabling between all system components Cisco CRS-1 Carrier Routing System Multishelf System
Interconnection and Cabling Guide.
Color codes for FCC OIM-LED panel LEDs Cisco CRS-1 Carrier Routing System Multishelf System Description
Troubleshooting Cisco CRS-1 Carrier Routing System Multishelf System
Troubleshooting Guide
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-17
Chapter 1 Upgrading to a Multishelf System
Additional References
Related Documentation for the Catalyst 6509 Switch
For additional documentation related to the installation and configuration of the Catalyst 6509 switch, see the following Cisco Systems documents:
Related Topic Document Title
Catalyst 6509 installation Cisco Catalyst 6500 Series documentation
Catalyst 6509 configuration and general system configuration after the fabric installation and configuration is complete
Cisco IOS XR Getting Started Guide
Related Cisco IOS XR Software Documentation
The Cisco IOS XR software documentation is published at the following URL:
http://www.cisco.com/univercd/cc/td/doc/product/ioxsoft/index.htm
Technical Assistance
Description Link
The Cisco Technical Support website contains thousands of pages of searchable technical content, including links to products, technologies, solutions, technical tips, and tools. Registered Cisco.com users can log in from this page to access even more content.
http://www.cisco.com/techsupport
1-18
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
CHA PTER
2

Converting to an Integrated Switch System

This chapter describes the following concepts:
Convert to a Cisco CRS-1 Multishelf System that uses an external Cisco Catalyst 6509 Switches, to
a system that uses integrated switches that are located on the 22-port shelf controller Gigabit Ethernet (22-port SCGE) card.
Remove the Cisco Catalyst switches and replace them with 22-port SCGE cards.
Remove and replace techniques are used to complete the process.
Feature History for the Cisco CRS-1 Multishelf System
Release Modification
Release 3.4.1 The 22-port shelf controller Gigabit Ethernet (22-port SCGE) card is
introduced.

Contents

This chapter contains the following sections:
Prerequisites for the Integrated Switch System, page 2-2
Cisco CRS-1 Multishelf Conversion to an Integrated Switch System Introduction, page 2-2
Cisco CRS-1 Multishelf Integrated Switch Solution, page 2-3
Benefits of the Cisco CRS-1 Multishelf Integrated Switch Solution, page 2-3
Naming Conventions, page 2-3
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for
Single-FCC Multishelf and Two-FCC Multishelf Systems, page 2-4
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a
Four-FCC Multishelf System, page 2-10
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-1
Chapter 2 Converting to an Integrated Switch System

Prerequisites for the Integrated Switch System

Prerequisites for the Integrated Switch System
Hardware Requirements
Ensure that all fabric card chassis (FCCs) have two power shelves and that all three power entry
modules (PEMs) in each power shelf are running correctly. During the conversion, when one 22-port SCGE is removed, the chassis must have both power shelves populated to keep all the fans in the chassis engaged.
Two 22-port SCGE per fabric chassis.
Four 22-port SCGE for a Two-FCC Multishelf System.
Eight 22-port SCGE for a Four-FCC Multishelf System.
Software Requirements
Cisco IOS XR Software Release 3.4.1 to support 22-port SCGE.
ROMMON version 1.43 or higher
Caution Before performing the procedures that are described in this chapter, it is important to wait until the
previous step is completed before attempting the next step. Failure to do so can result in a failure of the migration procedure. If a previous step fails to complete successfully, contact Cisco Technical Support before proceeding. We do not have a rollover procedure defined.
Cisco CRS-1 Multishelf Conversion to an Integrated Switch System Introduction
The first implementation of the Cisco CRS-1 Multishelf System, the interrack control network was established by connecting the GE ports on each LCCs route processor (RP) to two external Cisco Catalyst 65xx switches, which, in turn, were connected to the FCCs’ shelf controller (SC) cards. Two Cisco Catalyst 65xx switches were used to achieve redundancy, and Rapid Spanning Tree Protocol (RSTP) was used to ensure a loop-free topology.
In the first release of the Cisco CRS-1 Multishelf System (running Cisco IOS XR Software Release 3.2.50), two Cisco Catalyst 65xx switches are interconnected to the multishelf’s fabric card chassis (FCC0, FCC1, FCC2, and FCC3) and two line card chassis (LCC0 and LCC1) to form the interchassis control network.
The 22-port SCGE card replaces the current 2-port SCGE card in the multishelf system (MSS) FCC. The 22-port SCGE card contains built-in bundled Gigabit Ethernet (GE) switches. The 22-port SCGE card is designed to integrate the functionality of the two Cisco Catalyst 65xx switches. With the new 22-port SCGE card, all Layer 2 system switching is now bundled and integrated into the Cisco CRS-1 router. On the 22-port SCGE card, there are 22 ports available on the front panel. Each port on the 22-port SCGE card has two LEDs, which indicate port activity and link state.
When the 22-port SCGE cards are installed, the control network topology ceases to be a simple hub-and-spoke set of connections.
2-2
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
Benefits of the Cisco CRS-1 Multishelf Integrated Switch Solution
Cisco CRS-1 Multishelf Integrated Switch Solution
When the Cisco Catalyst 65xx switches are phased out of the multishelf network, the new 22-port SCGE cards, which are installed in the Cisco CRS-1 Multishelf FCCs, are connected directly to the Cisco CRS-1 Multishelf LCC RPs. The Cisco CRS-1 RP is a system controller that performs route processing and distributes forwarding tables to the line cards. Although each routing system contains two RPs, only one RP is active at a time. The other RP operates in standby mode if the active RP fails.
To perform the conversion, you need to know how to view and use the STP on the Cisco Catalyst 65xx switch through the console port.
Note The Cisco Catalyst 65xx switches are typically configured so that one is the root of the spanning tree,
and the other one is the backup root. All the ports in the root switch are in the forwarding state.
Before the conversion to the 22-port SCGE, if you have two ports from the RP connected to a Cisco Catalyst 65xx switch, you should see one RP port in forwarding state, and the other RP port in blocked state.
RSTP is used to break the interchassis loops in the Layer 2 control network.
Note Loops can happen only when a port is transitioning from a down state to an up state.
Benefits of the Cisco CRS-1 Multishelf Integrated Switch Solution
The following benefits are described for migrating from a Cisco Catalyst 65xx switch-based control network for the multishelf system to a bundled 22-port SCGE integrated switch solution:
The Cisco Catalyst switches are configured with Cisco IOS software through a separate
command-line interface (CLI). But the integrated 22-port SCGE switches are configured from the Cisco CRS-1 router console by using the Cisco IOS XR software, which is the same operating system used by the Cisco CRS-1 router. The control network is easier to connect and manage.
There is no longer any need for two dedicated Catalyst switches per multichassis system.

Naming Conventions

The following naming conventions are used in this document:
The fabric chassis upper cage shelf controller card and its slot are both referred to as SC0.
The lower cage shelf controller card and slot are referred to as the SC1.
There are two SC cards in every FCC for the purpose of redundancy.
OL-12571-01
The active Cat6K remains active in the Cisco CRS-1 Multishelf control network.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-3
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and Two-FCC Multishelf Systems
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and Two-FCC Multishelf Systems
This section contains the following procedures:
Validating the 22-port SCGE Cards (Single-FCC Multishelf and Two-FCC Multishelf Systems),
page 2-4
Connecting the 22-port SCGE to an Active Cat6K (Single-FCC Multishelf and Two-FCC Multishelf
Systems), page 2-5
Transferring the Backup Cisco Catalyst 65xx Switch Gigabit Ethernet Connections to a Standby
22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems), page 2-6
Performing a Failover to a Standby 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf
Systems), page 2-7
Connecting the 22-port SCGEs into a Full Mesh (Single-FCC Multishelf and Two-FCC Multishelf
Systems), page 2-8
Transferring an Active Cat6K Gigabit Ethernet Connections to the Active 22-port SCGE
(Single-FCC Multishelf and Two-FCC Multishelf Systems), page 2-9
Validating the 22-port SCGE Cards (Single-FCC Multishelf and Two-FCC Multishelf Systems)
To validate the 22-port SCGE cards, perform the following steps:
Step 1 Insert the 22-port SCGE card into the standby slot (SC-GE-2) of FCC. Remove the 2-port card that is
already installed.
Step 2 Use the show platform command in EXEC mode or administration EXEC mode to verify that the
22-port SCGE card is in the IOS XR RUN state, as shown in the following example:
RP/0/RP0/CPU0:router(admin)# show platform
Node Type PLIM State Config State
----------------------------------------------------------------------------­0/3/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/3/CPU0 MSC Jacket Card IOS XR RUN PWR,NSHUT,MON 0/3/2 MSC(SPA) 8X1GE OK PWR,NSHUT,MON 0/3/4 MSC(SPA) 8X1GE OK PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/M(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/SM0/SP FCC-SFC(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/SC0/CPU0 FCC-SC(Active) N/A IOS XR RUN PWR,NSHUT,MON
F0/SC1/CPU0 FCC-SC(Standby) N/A IOS XR RUN PWR,NSHUT,MON
Step 3 Use the show redundancy command in EXEC mode to verify that the standby node is ready.
RP/0/RP0/CPU0:router# show redundancy
2-4
Redundancy information for node 0/RP0/CPU0: ========================================== Node 0/RP0/CPU0 is in ACTIVE role
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and
Partner node (0/RP1/CPU0) is in STANDBY role Standby node in 0/RP1/CPU0 is ready
Reload and boot info
---------------------­RP reloaded Fri Feb 2 16:09:51 2007: 3 days, 1 hour, 46 minutes ago Active node booted Fri Feb 2 16:09:51 2007: 3 days, 1 hour, 46 minutes ago Standby node boot Fri Feb 2 16:13:40 2007: 3 days, 1 hour, 43 minutes ago Standby node last went not ready Fri Feb 2 16:30:39 2007: 3 days, 1 hour, 26 minutes ago Standby node last went ready Fri Feb 2 16:30:44 2007: 3 days, 1 hour, 26 minutes ago There have been 0 switch-overs since reload
Redundancy information for node F0/SC0/CPU0: ========================================== Node F0/SC0/CPU0 is in ACTIVE role
Partner node (F0/SC1/CPU0) is in STANDBY role
Standby node in F0/SC1/CPU0 is ready
Reload and boot info
---------------------­RP reloaded Fri Feb 2 16:13:39 2007: 3 days, 1 hour, 43 minutes ago Active node booted Fri Feb 2 16:13:39 2007: 3 days, 1 hour, 43 minutes ago Standby node boot Fri Feb 2 16:13:39 2007: 3 days, 1 hour, 43 minutes ago Standby node last went not ready Fri Feb 2 16:48:07 2007: 3 days, 1 hour, 8 minutes ago Standby node last went ready Fri Feb 2 16:48:12 2007: 3 days, 1 hour, 8 minutes ago There have been 0 switch-overs since reload
Two-FCC Multishelf Systems
Step 4 Repeat Step 1 to Step 3 for the remaining FCC.
Connecting the 22-port SCGE to an Active Cat6K (Single-FCC Multishelf and Two-FCC Multishelf Systems)
To connect the standby 22-port SCGE to an active Cat6K and connect the 22-port SCGEs to each other in a full mesh configuration, perform the following steps:
Step 1 Connect one port of the standby 22-port SCGE in each fabric chassis to the active Cat6K. Enable STP
and portfast on these ports from the Cisco Catalyst 65xx switch side.
Step 2 Connect the two newly inserted standby 22-port SCGE cards to each other in a Two-FCC Multishelf
System.
Step 3 Use the show controller switch inter-rack stp command in administration EXEC mode, with the
location keyword, to verify that all the new connections and ports are working correctly.
Step 4 Verify that the connected ports are bidirectional and that the port neighbors are correct by using the show
controllers switch inter-rack udld command, with the all and location keywords, in administration
EXEC mode, as shown in the following syntax:
show controllers switch inter-rack udld {all location node-id}
Replace the node-id argument with the newly inserted standby 22-port SCGE card.
OL-12571-01
In addition, verify that the port is connected to an active Cat6K and is in the forwarding state (FWD) by using the show controllers switch inter-rack stp command, with the ports and location keywords, in administration EXEC mode, as shown in the following syntax:
show controller switch inter-rack stp {ports interface number location node-id}
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-5
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and Two-FCC Multishelf Systems
Replace the interface number argument with the port that is connected to the active Cat6k.
Replace the node-id argument with the newly inserted standby 22-port SCGE card.
The following example sample output displays information for STP from the show controllers switch inter-rack stp command:
RP/0/RP0/CPU0:router(admin)# show controllers switch inter-rack stp ports 0 location
GE_0 of MST1 is designated forwarding Edge port: no (default) port guard : none (default) Link type: point-to-point (auto) bpdu filter: disable (default) Boundary : internal bpdu guard : disable (default) Bpdus (MRecords) sent 204605, received 8
Instance Role Sts Cost Prio.Nbr Vlans mapped
-------- ---- --- --------- -------- ------------------------------­ 1 Desg FWD 20000 128. 1 1
For the 22-port SCGE ports which are connected to each other, one is in the forwarding state and the other is in the blocked state.
Transferring the Backup Cisco Catalyst 65xx Switch Gigabit Ethernet Connections to a Standby 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems)
To transfer the backup Cisco Catalyst 65xx switch GE connections to a standby 22-port SCGE card and to isolate the backup Cisco Catalyst 65xx switch from the control network, perform the following steps:
Step 1 Shut down the ports on the backup Cat6K one at a time. Make the interCat6K port the last port to shut
down.
Step 2 Move the disabled GE connections between the active RPs (on both the LCC in case of a 2 FCC MC)
and the backup Cat6K to the standby 22-port SCGE.
Step 3 Use the show controllers switch stp command, with the location keyword, in administration EXEC
mode, to verify that the RP ports (GE1) that are connected to the 22-port SCGE are in the blocked state, and that the RP ports (GE0) that are connected to an active Cat6K are in the forwarding state, as shown in the following syntax:
show controllers switch inter-rack stp location node-id
Replace the node-id argument with the location of the 22-port SCGE card. In addition, you can use the newly inserted standby 22-port SCGE card.
2-6
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and
Two-FCC Multishelf Systems
Performing a Failover to a Standby 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems)
To perform a failover to a standby 22-port SCGE and swap the remaining 2-ports SCGE in the system with the new 22-port SCGEs, perform the following steps:
Step 1 Perform a failover of the current active 2-ports SCGE card to the standby 22-port SCGE card. Use the
redundancy switchover command, with the location keyword, in EXEC mode, as shown in the
following syntax:
redundancy switchover location node-id
Replace the node-id argument with the location of the active 2-ports SCGE card.
Step 2 Verify that the failover is completed and that the 22-port SCGE card is now in active mode by using both
the show platform command and show redundancy command.
The show platform command shows that the 22-port SCGE card is in IOS XR RUN state.
The show redundancy command shows that the 22-port SCGE card is in the active role.
Step 3 After the 22-port SCGE becomes active, swap the 2-ports SCGE card (now in standby mode) with a new
22-port SCGE card.
Note Do not attempt this step before Step 2 has completed.
Step 4 Use the show platform command in EXEC mode or administration EXEC mode to verify that the newly
inserted 22-port SCGE card is in the IOS XR RUN state, as shown in the following example:
RP/0/RP0/CPU0:router(admin)# show platform
Node Type PLIM State Config State
----------------------------------------------------------------------------­0/3/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/3/CPU0 MSC Jacket Card IOS XR RUN PWR,NSHUT,MON 0/3/2 MSC(SPA) 8X1GE OK PWR,NSHUT,MON 0/3/4 MSC(SPA) 8X1GE OK PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/M(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/SM0/SP FCC-SFC(SP) N/A IOS XR RUN PWR,NSHUT,MON F0/SC0/CPU0 FCC-SC(Active) N/A IOS XR RUN PWR,NSHUT,MON
F0/SC1/CPU0 FCC-SC(Standby) N/A IOS XR RUN PWR,NSHUT,MON
Use the show redundancy command in EXEC mode to verify that the newly inserted 22-port SCGE cards are in standby ready, as shown in the following example:
RP/0/RP0/CPU0:router# show redundancy
Redundancy information for node 0/RP0/CPU0: ========================================== Node 0/RP0/CPU0 is in ACTIVE role Partner node (0/RP1/CPU0) is in STANDBY role Standby node in 0/RP1/CPU0 is ready
OL-12571-01
Reload and boot info
---------------------­RP reloaded Fri Feb 2 16:09:51 2007: 3 days, 1 hour, 46 minutes ago Active node booted Fri Feb 2 16:09:51 2007: 3 days, 1 hour, 46 minutes ago Standby node boot Fri Feb 2 16:13:40 2007: 3 days, 1 hour, 43 minutes ago
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-7
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and Two-FCC Multishelf Systems
Standby node last went not ready Fri Feb 2 16:30:39 2007: 3 days, 1 hour, 26 minutes ago Standby node last went ready Fri Feb 2 16:30:44 2007: 3 days, 1 hour, 26 minutes ago There have been 0 switch-overs since reload
Redundancy information for node F0/SC0/CPU0: ========================================== Node F0/SC0/CPU0 is in ACTIVE role
Partner node (F0/SC1/CPU0) is in STANDBY role
Standby node in F0/SC1/CPU0 is ready
Reload and boot info
---------------------­RP reloaded Fri Feb 2 16:13:39 2007: 3 days, 1 hour, 43 minutes ago Active node booted Fri Feb 2 16:13:39 2007: 3 days, 1 hour, 43 minutes ago Standby node boot Fri Feb 2 16:13:39 2007: 3 days, 1 hour, 43 minutes ago Standby node last went not ready Fri Feb 2 16:48:07 2007: 3 days, 1 hour, 8 minutes ago Standby node last went ready Fri Feb 2 16:48:12 2007: 3 days, 1 hour, 8 minutes ago There have been 0 switch-overs since reload
Step 5 Repeat Step 1 to Step 4 for the other fabric chassis on a two-FCC MC.
Connecting the 22-port SCGEs into a Full Mesh (Single-FCC Multishelf and Two-FCC Multishelf Systems)
To connect the 22-port SCGEs in a full mesh configuration, perform the following steps:
Step 1 Connect each of the 22-port SCGE card to every other 22-port SCGE card in the system. For a one-FCC
MC, this means one connection from the active 22-port SCGE card to the standby 22-port SCGE card. For a two-FCC MC, this means each 22-port SCGE card has three connections to the other three 22-port SCGE cards in the system. This constitutes a full mesh connectivity between the 22-port SCGE cards.
Step 2 Verify that all of the connected ports are bidirectional and that all of the neighbors are correct by using
the show controllers switch inter-rack udld command, with the all and location keywords, in administration EXEC mode, as shown in the following syntax:
show controllers switch inter-rack udld {all location node-id}
Replace the node-id argument with the location for all of the 22-port SCGE card in the system.
Step 3 Verify that all of the connected ports to 22-port SCGE cards are displayed as either in forwarding (FWD)
state or blocked (BLK) state for all of the 22-port SCGE cards in the system by using the show controller switch inter-rack stp command, with the location keyword, in administration EXEC mode.
The location is used for all of the 22-port SCGE cards in the system.
The following example shows location f0/sc0/cpu0:
RP/0/RP0/CPU0:router(admin)# show controllers switch inter-rack stp location f0/sc0/cpu0
2-8
##### MST 0 vlans mapped: 2-4094 Bridge address 5246.48f0.20ff priority 32768 (32768 sysid 0) Root this switch for the CIST Operational hello time 1, forward delay 6, max age 8, txholdcount 6 Configured hello time 1, forward delay 6, max age 8, max hops 4
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- ---------------------------
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for Single-FCC Multishelf and
##### MST 1 vlans mapped: 1 Bridge address 5246.48f0.20ff priority 32769 (32768 sysid 1) Root this switch for MST1
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------­ GE_13 Desg FWD 20000 128. 14 P2p GE_14 Desg FWD 20000 128. 15 P2p GE_15 Desg FWD 20000 128. 16 P2p GE_17 Desg FWD 20000 128. 18 P2p GE_22 Desg FWD 20000 128. 23 P2p
Two-FCC Multishelf Systems
Transferring an Active Cat6K Gigabit Ethernet Connections to the Active 22-port SCGE (Single-FCC Multishelf and Two-FCC Multishelf Systems)
To transfer the active Cat6K GE connections to the active 22-port SCGE and isolate the active Cat6K from the control network, perform the following steps:
Step 1 Disable the active Cat6K links that are connected to the active RP on the Designated Shelf Controller
(DSC).
Step 2 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify on
the active RPs that the GE1 port, which is connected to the 22-port SCGE, goes into the forwarding (FWD) state.
The location is the location of the active RP.
Step 3 Shut down the ports on the active Cat6K one at a time, with the exception of the four links between an
active Cat6K and the 22-port SCGE. Now, the active Cat6K remains connected to the fabric chassis with four GE links.
Step 4 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify on
all RPs that the GE1 port, which is connected to the 22-port SCGE, goes into the forwarding (FWD) state.
Step 5 Move the disabled GE link connecting the RPs and the active Cat6K link to the 22-port SCGE card.
Step 6 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify on
all RPs that the GE0 port, which is connected to the 22-port SCGE, goes into the blocked (BLK) state.
Step 7 Raise the priority of active Cat6K to 40960, which causes the root to change to 22-port SCGE on rack
FCC0.
Step 8 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify that
the active SC on FCC0 is the new root switch.
Step 9 Shut down the port that is connected to the active Cat6K on each of the 22-port SCGEs.
OL-12571-01
Step 10 Verify that there is no impact to the active topology and the control Ethernet traffic.
The migration is complete after you validate the new connections. You can remove the two Cisco Catalyst 65xx switches.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-9
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
This section contains the following procedures:
Validating the 22-port SCGE Cards (Four-FCC Multishelf System), page 2-10
Connecting the 22-port SCGE to an Active Cat6K (Four-FCC Multishelf System), page 2-10
Transferring the Backup Cisco Catalyst 65xx Switch Gigabit Ethernet Connections to a Standby
22-port SCGE (Four-FCC Multishelf System), page 2-13
Performing a Failover to a Standby 22-port SCGE (Four-FCC Multishelf System), page 2-14
Connecting the 22-port SCGEs into a Full Mesh Configuration (Four-FCC Multishelf System), page
2-15
Transferring an Active Cat6K Gigabit Ethernet Connections to the Active 22-port SCGE (Four-FCC
Multishelf System), page 2-16
Validating the 22-port SCGE Cards (Four-FCC Multishelf System)
To validate the 22-port SCGE cards, perform the following steps:
Step 1 Insert the 22-port SCGE card into the standby slot (SC-GE-2) of FCC. Remove the 2-port card that is
already installed.
Step 2 Use the show platform command in EXEC mode or administration EXEC mode to verify that the
22-port SCGE card is in the IOS XR RUN state.
Step 3 Use the show redundancy command in EXEC mode to verify that the standby node is ready.
Step 4 Repeat Step 1 to Step 3 for all eight 22-port SCGE cards. Verify the two per fabric chassis by using the
standby slot and leaving the second 22-port SCGE in the standby slot to continue with the upgrade.
Connecting the 22-port SCGE to an Active Cat6K (Four-FCC Multishelf System)
To connect the standby 22-port SCGE to an active Cat6K and connect the 22-port SCGEs to each other in a full mesh configuration, perform the following steps:
Step 1 Connect one port of the standby 22-port SCGE in each fabric chassis to the active Cat6K. Enable STP
and portfast on these ports from the Cisco Catalyst 65xx switch side. If there are not enough ports on the active Cat6K, use the ports that were previously connected to the standby SCs in the four fabric chassis.
Step 2 Connect the ports to each other in a full mesh configuration for the 22-port SCGE.
2-10
Step 3 Use the show controller switch inter-rack stp command in administration EXEC mode, with the
location keyword, to verify that all the new connections and ports are working correctly.
Step 4 Use the show controllers switch inter-rack udld command in administration EXEC mode, with the all
and location keywords, to verify that all the ports for the 22-port SCGE are connected to the active Cat6K in the forwarding state.
The node-id argument is the location of the 22-port SCGE cards.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
The following example displays sample output from the show controllers switch inter-rack udld command:
RP/0/RP0/CPU0:router(admin)# show controllers switch inter-rack udld all location f0/sc0/CPU0
Interface Gig port# 0
--­Port enable administrative configuration setting: Enabled Port enable operational state: Enabled Current bidirectional state: Bidirectional Current operational state: Advertisement - Single neighbor detected Message interval: 7 Time out interval: 5
Entry 1
--­ Expiration time: 16 Device ID: 1 Current neighbor state: Bidirectional Device name: 0_RP0_CPU0_Switch Port ID: GE_Port_0 Neighbor echo 1 device: nodeF0_SC0_CPU0 Neighbor echo 1 port: Gig port# 0
Message interval: 7 Time out interval: 5 CDP Device name: BCM_SWITCH
Interface Gig port# 1
--­Port enable administrative configuration setting: Enabled Port enable operational state: Enabled Current bidirectional state: Bidirectional Current operational state: Advertisement - Single neighbor detected Message interval: 7 Time out interval: 5
Entry 1
--­ Expiration time: 16 Device ID: 1 Current neighbor state: Bidirectional Device name: 0_RP1_CPU0_Switch Port ID: GE_Port_0 Neighbor echo 1 device: nodeF0_SC0_CPU0 Neighbor echo 1 port: Gig port# 1
Message interval: 7 Time out interval: 5 CDP Device name: BCM_SWITCH
Interface Gig port# 2
--­Port enable administrative configuration setting: Enabled Port enable operational state: Enabled Current bidirectional state: Unknown Current operational state: Advertisement Message interval: 7 Time out interval: 5 No neighbor cache information stored
OL-12571-01
Interface Gig port# 10
---
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-11
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
Port enable administrative configuration setting: Enabled Port enable operational state: Enabled Current bidirectional state: Bidirectional Current operational state: Advertisement - Single neighbor detected Message interval: 7 Time out interval: 5
Entry 1
--­ Expiration time: 15 Device ID: 1 Current neighbor state: Bidirectional Device name: nodeF0_SC1_CPU0 Port ID: Gig port# 10 Neighbor echo 1 device: nodeF0_SC0_CPU0 Neighbor echo 1 port: Gig port# 10
Message interval: 7 Time out interval: 5 CDP Device name: BCM_SWITCH
Interface Gig port# 12
--­Port enable administrative configuration setting: Enabled Port enable operational state: Enabled Current bidirectional state: Unknown Current operational state: Advertisement Message interval: 7 Time out interval: 5 No neighbor cache information stored
Interface Gig port# 22
--­Port enable administrative configuration setting: Enabled Port enable operational state: Enabled Current bidirectional state: Bidirectional Current operational state: Advertisement - Single neighbor detected Message interval: 7 Time out interval: 5
Entry 1
--­ Expiration time: 19 Device ID: 1 Current neighbor state: Bidirectional Device name: F0_SC0_CPU0_Switch Port ID: GE_Port_0 Neighbor echo 1 device: nodeF0_SC0_CPU0 Neighbor echo 1 port: Gig port# 22
Message interval: 7 Time out interval: 5 CDP Device name: BCM_SWITCH
In addition, verify that the port is connected to an active Cat6K and is in the forwarding (FWD) state by using the following syntax for the show controllers switch inter-rack stp command in administration EXEC mode:
show controllers switch inter-rack stp {ports interface number location node-id}
Replace the interface number argument with the port connected to the active Cat6K.
2-12
Replace the node-id argument with the newly inserted standby 22-port SCGE card.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
The following example shows how to display information for the Spanning Tree Protocol (STP) from the show controllers switch inter-rack stp command:
RP/0/RP0/CPU0:router(admin)# show controllers switch inter-rack stp ports 0 location
GE_0 of MST1 is designated forwarding Edge port: no (default) port guard : none (default) Link type: point-to-point (auto) bpdu filter: disable (default) Boundary : internal bpdu guard : disable (default) Bpdus (MRecords) sent 204605, received 8
Instance Role Sts Cost Prio.Nbr Vlans mapped
-------- ---- --- --------- -------- ------------------------------­ 1 Desg FWD 20000 128. 1 1
For the 22-port SCGE ports which are connected to each other, one is in the forwarding state and the other is in the blocked state.
Step 5 Use the show controller inter-rack stp command in administration EXEC mode, with the location
keyword, to verify the connection between the 22-port SCGE card. The active Cat6K goes into forwarding state.
The location is the location of the 22-port SCGE card.
The following example shows location f0/sc0/cpu0:
RP/0/RP0/CPU0:router(admin)# show controllers switch inter-rack stp location f0/sc0/cpu0
##### MST 0 vlans mapped: 2-4094 Bridge address 5246.48f0.20ff priority 32768 (32768 sysid 0) Root this switch for the CIST Operational hello time 1, forward delay 6, max age 8, txholdcount 6 Configured hello time 1, forward delay 6, max age 8, max hops 4
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- ---------------------------
##### MST 1 vlans mapped: 1 Bridge address 5246.48f0.20ff priority 32769 (32768 sysid 1) Root this switch for MST1
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------­ GE_13 Desg FWD 20000 128. 14 P2p GE_14 Desg FWD 20000 128. 15 P2p GE_15 Desg FWD 20000 128. 16 P2p GE_17 Desg FWD 20000 128. 18 P2p GE_22 Desg FWD 20000 128. 23 P2p
Transferring the Backup Cisco Catalyst 65xx Switch Gigabit Ethernet Connections to a Standby 22-port SCGE (Four-FCC Multishelf System)
OL-12571-01
To transfer the backup Cisco Catalyst 65xx switch GE connections to a standby 22-port SCGE and to isolate the backup Cisco Catalyst 65xx switch from the control network, perform the following steps:
Step 1 Shut down the ports on the backup Cat6K one at a time. Make the interCat6K port the last port to shut
down.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-13
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
Step 2 Move the disabled GE links between RPs and the backup Cat6K to the newly inserted 22-port SCGE
cards. The following locations are listed:
0/rp0-GE1 to F1/SC1-GE0
0/rp1-GE1 to F3/SC1-GE0
1/rp0-GE1 to F0/SC1-GE0
1/rp1-GE1 to F2/SC1-GE0
Step 3 Use the show controllers switch inter-rack udld command in administration EXEC mode, with the all
and location keywords, to verify that all of the new connections are bidirectional and have the correct neighbors.
The location is the location of the 22-port SCGE card.
Step 4 Use the show controllers switch stp location command in administration EXEC mode to verify that RP
ports (GE1) that are connected to the 22-port SCGE are in the blocked state, and that the RP ports (GE0) that are connected to the active Cat6K are in the forwarding state.
Step 5 Verify that there is no impact to the active topology and the control Ethernet traffic because the active
Cat6K remains as the root.
Performing a Failover to a Standby 22-port SCGE (Four-FCC Multishelf System)
To perform a failover to a standby 22-port SCGE and swap the remaining SCGE with the new 22-port SCGEs, perform the following steps:
Step 1 Perform a failover to an FCC0, to the new 22-port SCGE. Use the following syntax for the redundancy
switchover command, with the location keyword, in EXEC mode:
redundancy switchover location node-id
Replace the node-id argument with the location of the active 2-ports SCGE card.
Step 2 Verify that the failover is completed by using the show platform command and show redundancy
command.
The show platform command shows that the 22-port SCGE card is in IOS XR RUN state.
The show redundancy command shows that the 22-port SCGE card is in the active state.
Step 3 After the 22-port SCGE becomes active, swap the 2-ports SCGE (active SC) with the new 22-port SCGE
card.
Note Do not attempt this step before Step 2 has completed.
Step 4 Use the show platform command in EXEC mode or administration EXEC mode to verify that the newly
22-port SCGE is in the IOS XR RUN state.
2-14
Step 5 Use the show redundancy command in EXEC mode to verify that the newly inserted 22-port SCGE
cards are in standby ready.
Step 6 Repeat Step 1 to Step 4 for FCC1, FCC2, and FCC3.
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
Connecting the 22-port SCGEs into a Full Mesh Configuration (Four-FCC Multishelf System)
To connect the 22-port SCGEs in a full mesh configuration, perform the following steps:
Step 1 Connect all of the 22-port SCGE cards to every other 22-port SCGE cards in the system in a mesh
connection topology. Every 22-port SCGE card has seven connections to other 22-port SCGE cards.
Step 2 Verify that all new connections are bidirectional and connected to the right neighbors by using the show
controllers switch inter-rack udld command, with the all and location keywords, in administration
EXEC mode, as shown in the following syntax:
show controllers switch inter-rack udld {all location node-id}
Replace the node-id argument with the location of all of the 22-port SCGE cards.
Step 3 Verify that all of the connected ports to the 22-port SCGE cards are displayed as either in forwarding
(FWD) state or blocked (BLK) state for all of the 22-port SCGE cards in the system by using the show controller switch inter-rack stp command, with the location keyword, in administration EXEC mode.
The location is used for all of the 22-port SCGE cards in the system.
The following example shows location f0/sc0/cpu0:
RP/0/RP0/CPU0:router(admin)# show controllers switch inter-rack stp location f0/sc0/cpu0
##### MST 0 vlans mapped: 2-4094 Bridge address 5246.48f0.20ff priority 32768 (32768 sysid 0) Root this switch for the CIST Operational hello time 1, forward delay 6, max age 8, txholdcount 6 Configured hello time 1, forward delay 6, max age 8, max hops 4
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- ---------------------------
##### MST 1 vlans mapped: 1 Bridge address 5246.48f0.20ff priority 32769 (32768 sysid 1) Root this switch for MST1
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------­ GE_13 Desg FWD 20000 128. 14 P2p GE_14 Desg FWD 20000 128. 15 P2p GE_15 Desg FWD 20000 128. 16 P2p GE_17 Desg FWD 20000 128. 18 P2p GE_22 Desg FWD 20000 128. 23 P2p
OL-12571-01
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
2-15
Chapter 2 Converting to an Integrated Switch System
How to Convert from the Cisco Catalyst 65xx Switch to the Integrated Switch System for a Four-FCC Multishelf System
Transferring an Active Cat6K Gigabit Ethernet Connections to the Active 22-port SCGE (Four-FCC Multishelf System)
To transfer an active Cat6K GE connections to the active 22-port SCGE and isolate the active Cat6K from the control network, perform the following steps:
Step 1 Disable the active Cat6K port that is connected to the active RP on the DSC.
Step 2 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify on
the active RPs that the GE1 port, which is connected to the 22-port SCGE, goes into the forwarding (FWD) state. The location is the location of the active RP.
Step 3 Shut down the ports on the active Cat6K one at a time, with the exception of the four links between the
active Cat6K and the 22-port SCGE. Now, the active Cat6K remains connected to the fabric chassis with four GE links.
Step 4 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify on
all RPs that the GE1 port, which is connected to the 22-port SCGE, goes into the forwarding state.
Step 5 Move the disabled GE links from the RP to the active Cat6K to the 22-port SCGE. The following
locations are listed:
0/rp0-GE0 to F0/SC0-GE0
0/rp1-GE0 to F2/SC0-GE0
1/rp0-GE0 to F1/SC0-GE0
1/rp1-GE0 to F3/SC0-GE0
Step 6 Verify that all of the connections are bidirectional and have the correct neighbors by using the show
controller switch inter-rack udld command, with the all and location keywords, in administration EXEC mode, as shown in the following syntax.
show controllers switch inter-rack udld {all location node-id}
Replace the node-id argument with the location of all of the 22-port SCGE card.
Step 7 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify on
all RPs that the GE0 port, which is connected to the 22-port SCGE, goes into the blocked (BLK) state.
Step 8 Raise the priority of the active Cat6K to 40960, which causes the root to change to the 22-port SCGE on
rack FCC0.
Step 9 Use the show controllers switch inter-rack stp command in administration EXEC mode to verify that
the active SC on FCC0 is the new root switch.
Step 10 Shut down the port that is connected to active Cat6K to each of the 22-port SCGEs.
The migration is complete after you validate the new connections. You can remove the two Cisco Catalyst 65xx switches.
2-16
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
OL-12571-01

INDEX

A
admin configure command 1-5, 1-10
C
cabling documentation 1-17
Catalyst 6509 documentation 1-18
chassis serial number, location 1-2
commit command 1-6, 1-10
considerations
switch fabric upgrade
controllers fabric plane command 1-7, 1-8
controllers fabric rack command 1-10, 1-11
CRS-FCC-SFC cards 1-2
1-3
D
documentation
Catalyst 6509 Switch
Cisco CRS-1 Multishelf hardware 1-17
dsc serial command 1-5, 1-10
1-18
fabric card, single-chassis system
See FC/S cards
fabric card chassis
See FCC
FC/M cards
FC/S cards 1-3
FCC
installation documents
LEDs 1-8, 1-10, 1-17
serial number configuration 1-6
serial number location 1-2
FCC-SFC cards 1-2
1-3
1-17
L
LCC
installation documents
rack 0 serial number configuration 1-5
rack 1 serial number configuration 1-10
serial number location 1-2
line card chassis
See LCC
1-17
E
end command 1-6
F
fabric card, FCC
See FCC-SFC cards
fabric card, multishelf system
See FC/M cards
OL-12571-01
M
multishelf system
documentation
upgrading
configuration examples
information 1-3
prerequisites 1-1
procedure 1-3
restrictions 1-2
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
1-17
1-11
IN-1
Index
P
planning documentation 1-17
R
ROMMON, version required for LCC 1-9
S
serial number
displaying
FCC configuration 1-6
LCC configuration 1-10
location 1-2
rack 0 configuration 1-5
show controllers fabric fabric-backpressure summary command
show controllers fabric plane all command 1-7, 1-8
show controllers fabric plane command 1-8
show controllers fabric rack-status all detail command
show controllers rack-status all detail command 1-11
show platform command 1-7
switch fabric plane
upgrade considerations
1-11
1-11
1-10
1-3
T
technical assistance 1-18
troubleshooting documentation 1-17
Cisco CRS-1 Carrier Routing System Multishelf System Upgrade and Conversion Guide
IN-2
OL-12571-01
Loading...