Alcatel-Lucent 6648, 8800, 6624, 6800 User Manual

Part No. 031496-00, Rev. C September 2005

OmniSwitch 6624/6648/

6800/7700/7800/8800

Troubleshooting Guide

www.alcatel.com

OmniSwitch Troubleshooting Guide September 2005 i

This troubleshooting guide documents OmniSwitch 6624/6648/770/7800/8800

hardware, including chassis and associated components, and Release 5.1 software.

The specifications described in this guide are subject to change without notice.

Alcatel and Alcatel OmniVista

and the Alcatel logo are registered trademarks of Alcatel. Xylan®, OmniSwitch®, OmniStack®,

are registered trademarks of Alcatel Internetworking, Inc.

OmniAccess™, Omni Switch/Router™, PolicyView™, RouterView™, SwitchManager™, VoiceView™, WebView™, X-Cell™, X-Vision™, and the Xylan logo are trademarks of Alcatel Internetworking, Inc.

This OmniSwitch product contains components which may be covered by one or more of the following U.S. Patents:

• U.S. Patent No. 6,339,830

• U.S. Patent No. 6,070,243

• U.S. Patent No. 6,061,368

• U.S. Patent No. 5,394,402

• U.S. Patent No. 6,047,024

• U.S. Patent No. 6,314,106

• U.S. Patent No. 6,542,507

26801 West Agoura Road

Calabasas, CA 91301

(818) 880-3500 FAX (818) 880-3505

info@ind.alcatel.com

US Customer Support—(800) 995-2696

International Customer Support—(818) 878-4507

Internet—http://eservice.ind.alcatel.com

ii OmniSwitch Troubleshooting Guide September 2005

About This Guide .........................................................................................................xv

Supported Platforms ......................................................................................................... xv

Who Should Read this Manual? ...................................................................................... xvi

When Should I Read this Manual? .................................................................................. xvi

What is in this Manual? ................................................................................................... xvi

What is Not in this Manual? ...........................................................................................xvii

How is the Information Organized? ...............................................................................xvii

Related Documentation ..................................................................................................xvii

Before Calling Alcatel’s Technical Assistance Center .................................................... xx

Chapter 1 Troubleshooting the Switch System ......................................................................1-1

In This Chapter ................................................................................................................1-1

Introduction .....................................................................................................................1-2

Troubleshooting System for OS-6624/6648 and OS-7/8XXX .......................................1-3

Advanced Troubleshooting .............................................................................................1-9

Dshell Troubleshooting .................................................................................................1-11

Troubleshooting NIs on OmniSwitch 7700/7800/8800 .........................................1-21

OmniSwitch 6624/6648 Dshell Troubleshooting ...................................................1-23

Accessing Dshell on Idle Switches ..................................................................1-25

Using AlcatelDebug.cfg ................................................................................................1-26

Troubleshooting IPC on OS-6/7/8XXX Series of Switches .........................................1-27

Debugging IPC .......................................................................................................1-27

OmniSwitch 6624/6648 Example ..........................................................................1-34

Port Numbering Conversion Overview .........................................................................1-36

ifindex to gport .......................................................................................................1-36

gport to ifindex .......................................................................................................1-36

Converting from lport .............................................................................................1-36

OmniSwitch 7700/7800/8800 (Falcon/Eagle) Example ..................................1-36

OmniSwitch 6624/6648 (Hawk) Example ......................................................1-37

Chapter 2 Troubleshooting Switched Ethernet Connectivity ..............................................2-1

In This Chapter ................................................................................................................2-1

Overview of Troubleshooting Approach ........................................................................2-2

Verify Physical Layer Connectivity ................................................................................2-3

Verify Current Running Configuration ...........................................................................2-5

OmniSwitch Troubleshooting Guide September 2005 iii

Verify Source Learning ...................................................................................................2-6

Verify Switch Health .......................................................................................................2-7

Verify ARP ......................................................................................................................2-7

Using the Log File ...........................................................................................................2-8

Checking the 7700/7800 Nantucket Fabric ..............................................................2-8

Checking the 7700/7800 Nantucket Fabric for Interrupts, Data Counts and

Error Counts ............................................................................................................2-9

Checking the Traffic Queue on the NI .....................................................................2-9

Check for Catalina (MAC) or Port Lockup ............................................................2-10

Chapter 3 Troubleshooting Source Learning .........................................................................3-1

In This Chapter ................................................................................................................3-1

Introduction .....................................................................................................................3-2

Troubleshooting a Source Learning Problem .................................................................3-3

Advanced Troubleshooting .............................................................................................3-5

Dshell Troubleshooting ...................................................................................................3-7

OS-6600 .................................................................................................................3-10

Chapter 4 Troubleshooting Spanning Tree ............................................................................4-1

In This Chapter ................................................................................................................4-1

Introduction .....................................................................................................................4-1

Troubleshooting Spanning Tree ......................................................................................4-2

Dshell ..............................................................................................................................4-5

Generic Troubleshooting in Dshell ...............................................................................4-10

Event Trace (stpni_traceprint) ................................................................................4-10

PORTATCH ....................................................................................................4-11

PORTDELE .....................................................................................................4-11

ADDVLAN .....................................................................................................4-11

MODVLADM .................................................................................................4-12

MODVLSTP ....................................................................................................4-12

ADDQTAG .....................................................................................................4-12

DELQTAG ......................................................................................................4-12

MDEFVLAN ...................................................................................................4-13

PORTAGGR ....................................................................................................4-13

PORTDISG ......................................................................................................4-13

AGGR_UP .......................................................................................................4-13

AGGRDOWN .................................................................................................4-13

PORTJOIN ......................................................................................................4-14

PORTLEAV ....................................................................................................4-14

BRGPARAM ...................................................................................................4-14

PTSTPMOD ....................................................................................................4-15

PORTMOD ......................................................................................................4-15

PORTVLBK ....................................................................................................4-15

PVLANBLK ....................................................................................................4-15

GMBPDU ........................................................................................................4-16

iv OmniSwitch Troubleshooting Guide September 2005

GMIGBPDU ....................................................................................................4-16

GM2FIXED .....................................................................................................4-17

VMADDVPA ..................................................................................................4-17

VMDELVPA ...................................................................................................4-17

VMDEFVPA ...................................................................................................4-17

TOPOCHGT ....................................................................................................4-18

LINK_UP ........................................................................................................4-18

LINKDOWN ...................................................................................................4-18

NI_UP ..............................................................................................................4-18

NI_DOWN ......................................................................................................4-18

Physical and Logical Port Dumps ..........................................................................4-19

Logical Ports (stpni_debugLport) ....................................................................4-19

Physical Port (stpni_debugPport) ....................................................................4-20

Physical and Logical Port Trace Display (stpni_debugport) ...........................4-22

Socket Handler Traces ...........................................................................................4-22

stpNISock_globals ...........................................................................................4-22

stpNISock_warningprint ..................................................................................4-23

stpNISock_traceprint .......................................................................................4-23

Inter-NI Trace (stpNISock_intraceprint) .........................................................4-24

Time-out Trace (stpNISock_totraceprint) .......................................................4-24

Board Up (stpNISock_boardupprint) ..............................................................4-24

stpNISock_printon ...........................................................................................4-24

StpNISock_printoff .........................................................................................4-24

CMM Spanning Tree Traces ..................................................................................4-25

Trace Menu ......................................................................................................4-25

stpCMM_traceprint .........................................................................................4-25

Writing a PR for Spanning Tree ....................................................................................4-26

Exception in Spanning Tree (NI and CMM case) ..................................................4-26

Port Does Not Forward ..........................................................................................4-26

Spanning Tree Unchanged When Port State Has Changed ....................................4-27

Other Cases ............................................................................................................4-27

Chapter 5 Troubleshooting BOOTP/DHCP/UDP Relay ........................................................5-1

In This Chapter ................................................................................................................5-1

Starting the Troubleshooting Procedure ..........................................................................5-1

Use a Network Diagram ...........................................................................................5-2

Use the OSI Model to Guide Your Troubleshooting ...............................................5-2

UDP Relay Configuration Problems ........................................................................5-2

Incorrect Server IP Address ...............................................................................5-2

Forward Delay Timer ........................................................................................5-3

Displaying DHCP Statistics ..............................................................................5-3

UDP Relay and Group Mobility ...............................................................................5-4

Advanced Troubleshooting for UDP Relay ....................................................................5-5

Dshell ..............................................................................................................................5-6

Chapter 6 Troubleshooting DNS ................................................................................................6-1

In This Chapter ................................................................................................................6-1

Introduction .....................................................................................................................6-1

OmniSwitch Troubleshooting Guide September 2005 v

Troubleshooting a DNS Failure ......................................................................................6-2

Starting the Troubleshooting Procedure ...................................................................6-2

Layer 7 DNS or Name Resolution Issue ..................................................................6-2

DNS Configuration Considerations ................................................................................6-3

Chapter 7 Troubleshooting Link Aggregation .......................................................................7-1

In This Chapter ................................................................................................................7-1

Link Aggregation Limits and Guidelines ........................................................................7-2

OmniSwitch 6624/6648 Restrictions .......................................................................7-2

Troubleshooting a Link Aggregation Failure ..................................................................7-3

Verify the Configuration ..........................................................................................7-3

Source Learning .......................................................................................................7-5

Link Aggregation Affecting Other Traffic ...............................................................7-5

Problems Creating a Group ......................................................................................7-5

Problems Deleting a Group ......................................................................................7-5

LACP 802.3AD ........................................................................................................7-6

Advanced Link Aggregation Troubleshooting ................................................................7-7

6800 Link Aggregation Debug Functions .....................................................................7-10

la_ni_agg_prt ..........................................................................................................7-10

la_ni_port_prt .........................................................................................................7-10

la_ni_port_up_prt ...................................................................................................7-11

la_ni_port_stats_prt ................................................................................................7-11

la_ni_info ...............................................................................................................7-11

lagg_ni_Sock_help .................................................................................................7-11

la_ni_trace_freeze ..................................................................................................7-12

la_ni_trace_unfreeze ..............................................................................................7-12

la_ni_kite_help .......................................................................................................7-12

Chapter 8 Troubleshooting 802.1Q ..........................................................................................8-1

In This Chapter ................................................................................................................8-1

Troubleshooting 802.1Q .................................................................................................8-2

Default VLAN Traffic ..............................................................................................8-3

Tagged Packet on an Untagged Port ........................................................................8-3

802.1Q with VLAN ID of 0 ..............................................................................8-4

802.1Q and 64 Byte Packets ..............................................................................8-4

Advanced Troubleshooting .............................................................................................8-5

Dshell Commands ...........................................................................................................8-7

Chapter 9 Troubleshooting Group Mobility ............................................................................9-1

In This Chapter ................................................................................................................9-1

Troubleshooting a VLAN Mobility Failure ....................................................................9-2

Binding Rules ...........................................................................................................9-3

Port Rules .................................................................................................................9-3

Precedence ................................................................................................................9-4

Advanced Troubleshooting .............................................................................................9-5

vi OmniSwitch Troubleshooting Guide September 2005

Dshell ..............................................................................................................................9-6

NI Debug Dshell .......................................................................................................9-6

6800 Group Mobility Troubleshooting ...........................................................................9-7

show vlan rules .........................................................................................................9-7

gmHelp .....................................................................................................................9-7

gmcKiteDebug .........................................................................................................9-8

gmcShowPorts ..........................................................................................................9-8

gmcShowRules .........................................................................................................9-8

gmnKiteDebug .........................................................................................................9-9

gmnKiteShowRules ..................................................................................................9-9

gmnMacVlanShowBuffer ........................................................................................9-9

Chapter 10 Troubleshooting QoS ...............................................................................................10-1

In This Chapter ..............................................................................................................10-1

QoS Behavior ................................................................................................................10-2

Default ....................................................................................................................10-2

QoS Queues and Ports ............................................................................................10-2

Troubleshooting QoS ....................................................................................................10-3

Information Gathering on Symptoms and Recent Changes ...................................10-3

Starting the Troubleshooting Procedure .................................................................10-3

QoS Activation .......................................................................................................10-3

QoS Apply ..............................................................................................................10-4

Invalid Policies .......................................................................................................10-4

Rules Order ............................................................................................................10-4

Viewing QoS Settings ............................................................................................10-5

Viewing QoS Policy Rules .....................................................................................10-5

Validation ...............................................................................................................10-6

Example 1 ........................................................................................................10-6

Example 2 ........................................................................................................10-6

Example 3 ........................................................................................................10-7

Correction ...............................................................................................................10-8

Reflexive Rules ......................................................................................................10-8

QoS Log .................................................................................................................10-9

QoS Statistics .......................................................................................................10-11

Debug QoS ...........................................................................................................10-11

Debug QoS Internal ..............................................................................................10-12

OmniSwitch 6624/6648 Dshell Troubleshooting .................................................10-13

qosIxHelp ......................................................................................................10-13

qosDBState ....................................................................................................10-13

QoS Dump .....................................................................................................10-13

Example QoS Rules ....................................................................................................10-15

Chapter 11 Troubleshooting ARP ...............................................................................................11-1

In This Chapter ..............................................................................................................11-1

ARP Protocol Failure ....................................................................................................11-2

Common Error Conditions ............................................................................................11-5

Advanced ARP Troubleshooting ..................................................................................11-6

OmniSwitch Troubleshooting Guide September 2005 vii

Dshell Troubleshooting .................................................................................................11-8

Viewing the ARP Table on OmniSwitch 6624/6648 Switches ............................11-10

Chapter 12 Troubleshooting IP Routing ...................................................................................12-1

In This Chapter ..............................................................................................................12-2

Introduction ...................................................................................................................12-3

IP Routing Protocol Failure ..........................................................................................12-3

Troubleshooting via the CLI .........................................................................................12-3

Troubleshooting with Debug CLI ...............................................................................12-11

RIP Troubleshooting ...................................................................................................12-13

OSPF Troubleshooting ................................................................................................12-19

BGP Troubleshooting ..................................................................................................12-27

Dshell Troubleshooting Advanced IP Routing ...........................................................12-29

ipdbg=x .................................................................................................................12-29

ifShow ..................................................................................................................12-29

iprmShowRoutes ..................................................................................................12-30

iprmCountRoutes .................................................................................................12-30

ipni_ifShow ..........................................................................................................12-30

Iprm_routeShow ...................................................................................................12-31

Ipni_routeCount ...................................................................................................12-31

ospfDbgDumpEnv ................................................................................................12-31

Chapter 13 Troubleshooting Virtual Router Redundancy Protocol (VRRP) ....................13-1

In This Chapter ..............................................................................................................13-1

Overview .......................................................................................................................13-2

Protocol Information .....................................................................................................13-3

IP Field Descriptions ..............................................................................................13-3

VRRP Field Descriptions .......................................................................................13-3

VRRP States ...........................................................................................................13-3

OmniSwitch 7700/7800/8800 Implementation .............................................................13-4

VRRP Security .......................................................................................................13-4

OmniSwitch VRRP Limitations .............................................................................13-4

CMM Failover ...............................................................................................................13-5

OmniSwitch VRRP Troubleshooting ............................................................................13-9

ARP Table ...................................................................................................................13-10

Dshell Troubleshooting ...............................................................................................13-11

Chapter 14 Troubleshooting IP Multicast Switching (IPMS) ...............................................14-1

In This Chapter ..............................................................................................................14-1

Troubleshooting a Device that Cannot Join an IP Multicast Stream ............................14-2

Troubleshooting a Device that Drops Out of an IP Multicast Stream ..........................14-3

Troubleshooting IPMS in Debug CLI ...........................................................................14-7

viii OmniSwitch Troubleshooting Guide September 2005

Dshell Troubleshooting .................................................................................................14-9

Chapter 15 Troubleshooting DVMRP ........................................................................................15-1

In This Chapter ..............................................................................................................15-1

Introduction ...................................................................................................................15-2

DVMRP Troubleshooting .............................................................................................15-2

DVMRP Global and Interface Commands .............................................................15-2

DVMRP Debug Commands ...................................................................................15-4

Chapter 16 Troubleshooting PIM-SM ........................................................................................16-1

In This Chapter ..............................................................................................................16-1

Introduction ...................................................................................................................16-2

Definition of Terms .......................................................................................................16-2

Protocol Overview ........................................................................................................16-3

DR Election ............................................................................................................16-3

Simplified Hello Message Format ...................................................................16-3

Debugging Hello Messages .............................................................................16-4

Related CLI Command ....................................................................................16-5

BSR Election .................................................................................................................16-6

Simplified Packet Format .......................................................................................16-7

Debugging BSR/Bootstrap .....................................................................................16-7

Election of a New BSR ....................................................................................16-8

Related CLI Command ....................................................................................16-9

C-RP Advertisements ..................................................................................................16-10

Simplified RP-Advertisement Packet Format ......................................................16-10

Debugging C-RP-Adv ..........................................................................................16-11

Related CLI Command ..................................................................................16-12

RP-SET .......................................................................................................................16-13

Simplified Bootstrap RP-SET Packet Taken on a 192.168.12/24 Network .........16-14

Debugging RP-SET ..............................................................................................16-16

On Non BSR You Should See .......................................................................16-16

Related CLI Command ..................................................................................16-17

Join/Prune ....................................................................................................................16-18

Simplified Join Packet ..........................................................................................16-18

Simplified PRUNE Packet ...................................................................................16-20

Debugging JOIN/PRUNE Event ..........................................................................16-20

Register .......................................................................................................................16-21

Simplified REGISTER Packet Format .................................................................16-22

Shared Tree .................................................................................................................16-23

Related CLI Command ..................................................................................16-24

Source-Based Tree ......................................................................................................16-25

Related CLI Command ..................................................................................16-26

Troubleshooting Examples: Limitations .....................................................................16-27

Incorrect BSR ID ..................................................................................................16-27

OmniSwitch Troubleshooting Guide September 2005 ix

Multicast Group Status is Shown as Disabled .....................................................16-27

PIM-SM Limitations ............................................................................................16-28

Upstream Neighbor/Next Hop Debug Commands ...............................................16-28

Chapter 17 Troubleshooting Server Load Balancing ...........................................................17-1

In This Chapter ..............................................................................................................17-1

Introduction ...................................................................................................................17-2

Server Load Balance Failure .........................................................................................17-2

What is an SLB Failure? ........................................................................................17-2

Description of a Complete Failure of Service ........................................................17-2

Description of a Partial Failure of Service .............................................................17-2

Troubleshooting Commands .........................................................................................17-3

Troubleshooting a Complete Failure .............................................................................17-4

Troubleshooting a Partial Failure ..................................................................................17-5

The Troubleshooting Procedure ....................................................................................17-5

Chapter 18 Troubleshooting Authenticated VLANs ..............................................................18-1

In This Chapter ..............................................................................................................18-1

Introduction ...................................................................................................................18-1

Troubleshooting AVLAN .............................................................................................18-2

DHCP Request Failure ...........................................................................................18-2

Authentication Failure ............................................................................................18-3

Problem Communicating Using Multiple Protocols Simultaneously ....................18-4

Useful Notes on Client Issues ................................................................................18-5

Troubleshooting Using Debug Systrace ........................................................................18-5

Telnet Authentication and De-authentication ........................................................18-5

Get the IP Address from Default VLAN .........................................................18-5

Initiate the Telnet Authentication ....................................................................18-6

Release/Renew IP ............................................................................................18-7

De-Authenticating ...........................................................................................18-7

Release/Renew to Go Back to Default VLAN ................................................18-7

HTTP/S Authentication ..........................................................................................18-8

Start of Authentication using https://x.x.x.253 ................................................18-8

De-Authenticate using https://x.x.x.253 ..........................................................18-9

AVClient ..............................................................................................................18-10

AVClient Authentication Start ......................................................................18-10

AVClient logout: ...........................................................................................18-11

Dshell Troubleshooting ...............................................................................................18-12

Authentication Dispatcher (AD) Debugging Help ...............................................18-12

The Authenticated VLAN adDebugShowContext Function ................................18-13

x OmniSwitch Troubleshooting Guide September 2005

Chapter 19 Troubleshooting 802.1X .........................................................................................19-1

In This Chapter ..............................................................................................................19-1

Troubleshooting with the CLI .......................................................................................19-2

Troubleshooting Using Debug CLI ...............................................................................19-4

Dshell Troubleshooting .................................................................................................19-7

Appendix A OS6600/OS7700/OS8800 Architecture Overview ............................................A-1

In This Chapter ...............................................................................................................A-1

The MAC ASIC .............................................................................................................A-2

Catalina ....................................................................................................................A-2

Firenze .....................................................................................................................A-4

The Coronado ASIC ................................................................................................A-5

Functional Description ............................................................................................ A-6

Coronado: The “Brain” of the System ..............................................................A-7

Coronado Specifications ...................................................................................A-7

Software Module Interaction ............................................................................A-8

Queue Driver Interaction ................................................................................................A-8

Ethernet Driver ........................................................................................................A-8

Queue Dispatcher ....................................................................................................A-8

NI Supervision .........................................................................................................A-9

Source Learning ......................................................................................................A-9

L3 Manager/IPMS ................................................................................................... A-9

QoS Manager ........................................................................................................... A-9

Destination MAC Learning .............................................................................. A-9

L3 Pseudo CAM Learning ................................................................................A-9

QoS Policy Change ........................................................................................... A-9

QoS Policy Deleted ........................................................................................A-10

L2 destination MAC Aged/Deleted ................................................................A-10

L3 PseudoCAM Entry Aged/Deleted ............................................................. A-10

Request to Free Queues Sent to QoS Manager ..............................................A-10

Link Goes Up/Down ....................................................................................... A-10

Link Aggregation .........................................................................................................A-11

Coronado Tables .......................................................................................................... A-11

Layer 2 Tables .......................................................................................................A-11

Layer 3 Tables .......................................................................................................A-11

Source Learning ...........................................................................................................A-12

Hardware Routing Engine (HRE) ................................................................................A-13

QoS/Policy Manager ....................................................................................................A-15

Coronado Egress Logic ................................................................................................ A-15

The Fabric Architecture ...............................................................................................A-16

Nantucket ASIC ...........................................................................................................A-17

Additional Nantucket Specifications .....................................................................A-17

Functional Description: ..................................................................................A-18

Data Flow .......................................................................................................A-18

OmniSwitch Troubleshooting Guide September 2005 xi

Calendar Manager Module ....................................................................................A-19

Data Port Output Module ......................................................................................A-19

Nantucket Redundancy .........................................................................................A-19

Roma ............................................................................................................................A-22

Functional Description .......................................................................................... A-23

Initialization ....................................................................................................A-24

NI Slot Insertion ............................................................................................. A-25

Setup Calendars and Flow Control for New NI .............................................A-25

NI Slot Extraction ...........................................................................................A-25

CMM Takeover and Hot Swap ............................................................................. A-25

Framing Error ........................................................................................................A-26

Chassis Management Module (CMM) .........................................................................A-26

OS7000 CMM .......................................................................................................A-27

OS8800 CMM .......................................................................................................A-27

Functional Description of CMM ...........................................................................A-28

CMM Software Startup Process ..................................................................... A-28

AOS ................................................................................................................A-29

MiniBoot ........................................................................................................A-30

AOS Start ........................................................................................................A-30

Chassis Manager Component of System Services ................................................A-30

CMM Reload of NI Module ..................................................................................A-30

Overall System Architecture .................................................................................A-32

Packet Walk .................................................................................................................A-34

Packet Walk Principles ..........................................................................................A-34

Data Flow Overview .............................................................................................A-34

Specific Packet Flows ..................................................................................................A-35

Unknown L2 Source, Known L2 Destination .......................................................A-35

The Catalina ASIC .........................................................................................A-35

The Coronado ASIC .......................................................................................A-35

The Nantucket ASIC ......................................................................................A-35

The Coronado ASIC .......................................................................................A-35

The Catalina ASIC .........................................................................................A-35

Unknown Destination ................................................................................................... A-36

Known L2 Source, Unknown L2 Destination .......................................................A-36

The Catalina ASIC .........................................................................................A-36

The Coronado ASIC .......................................................................................A-36

The Nantucket ASIC ......................................................................................A-36

The Coronado ASIC .......................................................................................A-37

The Catalina ASIC .........................................................................................A-37

Traffic is Being Passed; the Switch is Attempting to Put a Correct L2 DA

Entry on the NI .....................................................................................................A-37

The Coronado ASIC .......................................................................................A-37

Unknown L3 DA ...................................................................................................A-38

The Coronado ASIC .......................................................................................A-38

Hardware Buses on OmniSwitch 7700/7800/8800 Switches .......................................A-41

Xybus ....................................................................................................................A-41

Fbus .......................................................................................................................A-41

Bbus .......................................................................................................................A-41

xii OmniSwitch Troubleshooting Guide September 2005

Bus Mapping on OmniSwitch 7700/7800/8800 Switches ...........................................A-42

Xybus Mapping .....................................................................................................A-42

Fbus Mapping ........................................................................................................A-42

Falcon (OmniSwitch 7700/7800) Fbus Mapping ...........................................A-42

Eagle (OmniSwitch 8800) Fbus Mapping ......................................................A-42

OS6624/6648 Architecture ...........................................................................................A-43

Hardware Architectural Overview ........................................................................A-44

Layer 2 Forwarding ...............................................................................................A-46

Address Resolution Protocol .......................................................................... A-46

Address Learning ............................................................................................A-47

Location of Address Tables ............................................................................A-47

Address Look-up Methodology ...................................................................... A-48

L2 Data Structures ..........................................................................................A-48

3-Protocol Entry .............................................................................................A-49

Layer 3 Forwarding ...............................................................................................A-50

VLANs ..................................................................................................................A-51

Port Based VLANs ......................................................................................... A-51

Protocol Based VLANs ..................................................................................A-51

Address Based VLANs ...................................................................................A-51

Tag Net ID Entry ............................................................................................A-52

Priority ............................................................................................................A-52

802.1p Priority ................................................................................................A-52

Rules-Based Priority .......................................................................................A-53

QOS Flow .......................................................................................................A-53

Bandwidth Management and QoS ..................................................................A-53

CMM Functionality for OS6600 ..................................................................................A-54

OS6600 IPC Communication .......................................................................................A-58

OS6600 BOOT Sequence ............................................................................................A-59

Appendix B Debug Commands ..................................................................................................... B-1

Appendix C Technical Support Commands ............................................................................... C-1

Appendix D Modifying Files with VI Editor ................................................................................D-1

In This Chapter ...............................................................................................................D-1

Useful VI Commands .....................................................................................................D-2

Sample VI Session ......................................................................................................... D-3

Index ...................................................................................................................... Index-1

OmniSwitch Troubleshooting Guide September 2005 xiii

xiv OmniSwitch Troubleshooting Guide September 2005

About This Guide

This OmniSwitch Troubleshooting Guide describes how to use Command Line Interface (CLI) and Dshell commands available on the OmniSwitch 6600 Family, OmniSwitch 6800 Series, OmniSwitch 7700/7800, and the OmniSwitch 8800 to troubleshoot switch and network problems.

Supported Platforms

This information in this guide applies to the following products:

• OmniSwitch 6624 (OmniSwitch 6600-24)

• OmniSwitch 6648 (OmniSwitch 6600-48)

• OmniSwitch 6600-P24

• OmniSwitch 6600-U24

• OmniSwitch 6602-24

• OmniSwitch 6602-48

• OmniSwitch 6800

• OmniSwitch 7700

• OmniSwitch 7800

• OmniSwitch 8800

Note. All references to OmniSwitch 6624 and 6648 switches also apply to the OmniSwitch 6600-P24, OmniSwitch 6600-U24, OmniSwitch 6602-24, and OmniSwitch 6602-48 unless specified otherwise.

Unsupported Platforms

The information in this guide does not apply to the following products:

• OmniSwitch (original version with no numeric model name)

• Omni Switch/Router

• OmniStack

• OmniAccess

OmniSwitch Troubleshooting Guide September 2005 page -xv

Note. Troubleshooting documentation for legacy products (e.g., Omni Switch/Router) can be downloaded at http://support.ind.alcatel.com/releasefiles/indexpage.cfm.

Who Should Read this Manual?

The principal audience for this user guide is Service and Support personnel who need to troubleshoot switch problems in a live network. In addition, network administrators and IT support personnel who need to configure and maintain switches and routers can use this guide to troubleshoot a problem upon advice from Alcatel Service and Support personnel..

However, this guide is not intended for novice or first-time users of Alcatel OmniSwitches. Misuse or failure to follow procedures in this guide correctly can cause lengthy network down time and/or permanent damage to hardware. Caution must be followed on distribution of this document.

When Should I Read this Manual?

Always read the appropriate section or sections of this guide before you log into a switch to troubleshoot problems. Once you are familiar with the commands and procedures in the appropriate sections you can use this document as reference material when you troubleshoot a problem.

What is in this Manual?

The principal sections (i.e., the chapters numbered numerically) use CLI and Dshell commands to analyze and troubleshoot switch problems. Each section documents a specific switch feature (e.g., hardware, server load balancing, routing).

Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.

Appendix A provides an architecture overview for the OmniSwitch 6600 Family, OmniSwitch 7700/7800, and the OmniSwitch 8800.

Appendices B and C provide the following for debug and technical support CLI commands:

• Command description.

• Syntax.

• Description of keywords and variables included in the syntax.

• Default values.

• Usage guidelines, which include tips on when and how to use the command.

• Examples of command lines using the command.

page -xvi OmniSwitch Troubleshooting Guide September 2005

• Related commands.

• Release history, which indicates the release when the command was introduced.

Appendix D provides a list of useful VI editor commands and a sample VI session that modifies the boot.params file.

What is Not in this Manual?

This guide is intended for troubleshooting switches in live networks. It does not provide step-by-step instructions on how to set up particular features on the switch or a comprehensive reference to all CLI commands available in the OmniSwitch. For detailed syntax on non debug CLI commands and comprehensive information on how to configure particular software features in the switch, consult the user guides, which are listed in “Related Documentation” on page xvii.

How is the Information Organized?

Each chapter in this guide includes troubleshooting guidelines related to a single software feature, such as server load balancing or link aggregation.

Before Calling Alcatel’s Technical Assistance Center

Before calling Alcatel’s Technical Assistance Center (TAC), make sure that you have read through the appropriate section (or sections) and have completed the actions suggested for your system’s problem.

Additionally, do the following and document the results so that the Alcatel TAC can better assist you:

• Have a network diagram ready. Make sure that relevant information is listed, such as all IP addresses

and their associated network masks.

• Have any information that you gathered while troubleshooting the issue to this point available to

provide to the TAC engineer.

• If the problem appears to be with only a few-fewer than four-switches, capture the output from the

show tech-support CLI command on these switches. (See Appendix C, “Technical Support

Commands,” for more information on show tech-support CLI commands.)

When calling Alcatel TAC in order to troubleshoot or report a problem following information can be helpful to get a quick resolution:

• All the dump files that were created, if any

• Output of switch log or the switch log files swlog1.log and swlog2.log

• Configuration file boot.cfg

• A capture of the show microcode command

• A capture of the show module long command

• A capture of the show tech-support command from CLI.

• If a CMM fail over to the redundant CMM happened because of this failure then include this informa-

tion from both of the CMMs.

Dial-in or Telnet access can also helpful for effective problem resolution.

page -xx OmniSwitch Troubleshooting Guide September 2005

1 Troubleshooting the Switch

System

In order to troubleshoot the system, a basic understanding of the operation of Chassis Management Modules (CMMs) and their interaction with Network Interface (NI) modules is required. Some concepts are covered in this chapter:

• Understanding of the “Diagnosing Switch Problems” chapter in the appropriate OmniSwitch Switch

Management Guide.

• Understanding of the “Using Switch Logging” from the appropriate OmniSwitch Network Configura-

tion Guide is highly recommended.

In This Chapter

“Introduction” on page 1-2

“Troubleshooting System for OS-6624/6648 and OS-7/8XXX” on page 1-3

“Advanced Troubleshooting” on page 1-9

“Dshell Troubleshooting” on page 1-11

“Using AlcatelDebug.cfg” on page 1-26

“Troubleshooting IPC on OS-6/7/8XXX Series of Switches” on page 1-27

“Port Numbering Conversion Overview” on page 1-36

OmniSwitch Troubleshooting Guide September 2005 page 1-1

Introduction Troubleshooting the Switch System

Introduction

The CMM is the Management Module of the switch. All of the critical operations of the switch including the monitoring is the responsibility of the CMM. CMM not only provides monitoring but also synchronizes all of the NI for different operations. The operation of the CMM is the same in OS-6/7/8XXX switches. The only difference is that OS-6/7XXX has the switching fabric inherent to the module whereas OS-8800 has fabric at the back of the chassis.

NI has a build in CPU. Each NI has its own CPU, which acts independently of the CMM. The CPU of the NI has to interact with the CPU on the CMM for certain operations. If this operation becomes out of sync then it can create problems specific to that NI.

In order to troubleshoot the system, an understanding of the CMM and NI operation is essential.

page 1-2 OmniSwitch Troubleshooting Guide September 2005

Troubleshooting the Switch System Troubleshooting System for OS-6624/6648 and OS-7/8XXX

Troubleshooting System for OS-6624/6648 and OS-7/8XXX

If the switch is having any problems the first place to look for is the CMM. All the task are supervised by CMM. Any in coherency between CMM and the NI can cause problems to appear.

1 The first step for troubleshooting problems with the switch is to look at the overall general health of the

switch.

OmniSwitch 7700/7800/8800

Verify that all of the modules in the chassis are up and operational, using the command:

-> show module status

Operational Firmware Slot Status Admin-Status Rev MAC

------+-------------+------------+---------+----------------CMM-A UP POWER ON 36 00:d0:95:6b:09:40

NI-1 UP POWER ON 5 00:d0:95:6b:22:5c

NI-3 UP POWER ON 5 00:d0:95:6b:23:2e

NI-5 UP POWER ON 5 00:d0:95:6b:3a:20

OmniSwitch 6624/6648

If the switch is having any problems the first place to look for is the CMM. All the task are supervised by CMM. Any in coherency between CMM and the NI can cause problems to appear. For OS-6600 with 8 units stacked together, the CMM will be:

• Primary

• Secondary

• Idle

The switch with the lowest ID will become the primary CMM.

The first step for troubleshooting problems with the switch is to look at the overall general health of the switch.

Verify that all of the modules in the chassis are up and operational, using the command:

-> show module status Operational Firmware

Slot Status Admin-Status Rev MAC

------+-------------+------------+---------+-----------------

CMM-1 UP POWER ON N/A 00:d0:95:84:4b:d2

CMM-2 SECONDARY POWER ON N/A 00:d0:95:84:4b:d2

NI-1 UP POWER ON N/A 00:d0:95:84:4b:d4

OmniSwitch Troubleshooting Guide September 2005 page 1-3

Troubleshooting System for OS-6624/6648 and OS-7/8XXX Troubleshooting the Switch System

NI-2 UP POWER ON N/A 00:d0:95:84:3d:26

NI-3 UP POWER ON N/A 00:d0:95:86:50:f4

NI-4 UP POWER ON N/A 00:d0:95:84:49:be

NI-5 UP POWER ON N/A 00:d0:95:84:39:be

NI-6 UP POWER ON N/A 00:d0:95:84:4a:90

NI-7 UP POWER ON N/A 00:d0:95:84:39:f4

NI-8 UP POWER ON N/A 00:d0:95:84:3c:44

OmniSwitch 6600 with 8 stackable switches show up. Notice that the switch with ID 1 is the primary CMM and the switch with ID of 2 is the secondary. All the switch also show up as NI because each switch has a CPU and is also a NI.

To verify the stacking topology, use the following command:

-> show stack topology

Link A Link A Link A Link B Link B Link B

NI Role State RemoteNI RemoteLink State RemoteNI RemoteLink

---+-----------+---------+---------+-----------+---------+---------+----------

1 PRIMARY ACTIVE 8 51 ACTIVE 2 52 2 SECONDARY ACTIVE 3 27 ACTIVE 1 52 3 IDLE ACTIVE 2 51 ACTIVE 4 52 4 IDLE ACTIVE 5 51 ACTIVE 3 28 5 IDLE ACTIVE 4 51 ACTIVE 6 52 6 IDLE ACTIVE 7 51 ACTIVE 5 52 7 IDLE ACTIVE 6 51 ACTIVE 8 52 8 IDLE ACTIVE 1 51 ACTIVE 7 2

The above command shows the stacking topology. Switch 1 is the primary connected to Switch 8 on port 51 and Switch 2 on port 52. The state of CPUs for all the switches in the stack is shown by the output of this command.

2 Verify the power supply (or supplies).

OmniSwitch 7700/7800/8800

Omni Switch 7/8XXX has build-in mechanism to power off the modules if the power supply is not enough. Switching off a power supply in a chassis which does not have redundant power supply will result in power off of the modules. Make sure that there is no power involvement.

Check the power supply status, using the command:

-> show power supply 1

Module in slot PS-1

Model Name: OSR-PS-06, Description: OSR-PS-06, Part Number: 901750-10,

page 1-4 OmniSwitch Troubleshooting Guide September 2005

Troubleshooting the Switch System Troubleshooting System for OS-6624/6648 and OS-7/8XXX

Hardware Revision: , Serial Number: B42N101P2, Manufacture Date: OCT 18 2001, Firmware Version: , Admin Status: POWER ON, Operational Status: UP

Make sure that all the known good power supplies are operational.

OmniSwitch 6624/6648

Check the power supply status, using the command:

-> show power supply

Power Supplies in chassis 1 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Power Supplies in chassis 2 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Power Supplies in chassis 3 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Power Supplies in chassis 4 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Power Supplies in chassis 5 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Power Supplies in chassis 6 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Power Supplies in chassis 7 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

OmniSwitch Troubleshooting Guide September 2005 page 1-5

Troubleshooting System for OS-6624/6648 and OS-7/8XXX Troubleshooting the Switch System

Power Supplies in chassis 8 PS Operational Status

-----+-------------------

PS-1 UP PS-2 NOT PRESENT

Make sure that all the known good power supplies are operational.

3 Verify the CPU utilization.

OmniSwitch 6624/6648 and 7700/7800/8800

The CPU utilization of CMM can be viewed by using the command:

-> show health

* - current value exceeds threshold

Device Min 1 Hr 1 Hr Resources Limit Curr Avg Avg Max

----------------+-------+-----+------+-----+-----+-------

Receive 80 00 00 00 0 Transmit/Receive 80 00 00 00 00 Memory 80 43 43 43 43 Cpu 80 02 06 05 07 Temperature Cmm 50 38 37 37 37 Temperature Cmm Cpu 50 32 32 31 32

The above command shows the receive, transmit/receive, memory, CPU, temperature CMM and temperature CMM CPU statistics for current, 1 minimum average, 1 hour average and 1 hour maximum. All the values should be within the threshold. Anything above the threshold depicts that some abnormal behavior. Normally 1 hour average maximum might be high if the switch was booted up in the last hour but it should be fairly steady during normal operation.

If none of the above are above the threshold then the next step is to try to isolate the problem to a particular NI. Due to the distributed architecture every NI has it own CPU to perform some operations locally. It is possible that a particular NI might be at high CPU utilization at a time when other NI as well as the CPU are within the thresholds.

If none of the above are above the threshold then the next step is to try to isolate the problem to a particular NI (or a switch within an OmniSwitch 6624/6648 stack) with the show health slot_number CLI command:

-> show health 5

* - current value exceeds threshold

Slot 05 1 Min 1 Hr 1 Hr Resources Limit Curr Avg Avg Max

----------------+-------+-----+------+-----+-----+-------

Receive 80 01 01 01 01 Transmit/Receive 80 01 01 01 01 Memory 80 39 39 39 39 Cpu 80 21 22 21 24

The principle for the health of an NI is the same as for CMM.

page 1-6 OmniSwitch Troubleshooting Guide September 2005

Troubleshooting the Switch System Troubleshooting System for OS-6624/6648 and OS-7/8XXX

The average on one minute is calculated from the average of 12 samples. Each sample is an average of the CPU utilization during 5 seconds. Those values are stored in a table. The current minute (1 Min Avg or “min”) displays the average of the last 12 samples.

Every 60 seconds the average of the 12 samples is recorded into the average value for this minute. Those values are stored in a form of 60 samples which represent one hour.

Most probably one of the above would help to localize the problem to a particular NI or to CMM. For more details see, Section “Monitoring Switch Health” in the chapter titled “Diagnosing Switch Problems” in the appropriate OmniSwitch Network Configuration Guide.

4 Check the switch log.

OmniSwitch 6624/6648 and 7700/7800/8800

Now, one of the most important things to check is the switch log. Switch log would contain the error messages depending on the settings of the log levels and applications configured to generate error messages. Default settings of the log switch log can be view using the command:

-> show swlog

Switch Logging is:

- INITIALIZED

- RUNNING

Log Device(s)

----------------

flash console

Only Applications not at the level ‘info’ (6) are shown Application ID Level

----------------------------

CHASSIS (64) debug3 (9)

By default, log devices are set to be flash and console. This can be changed and specific log servers can be used to log the messages, please refer to the Switch Management Guide for further details. The application trace level is set for ‘info’. Any error messages or informational messages would be logged in the switch log.

Switch log should be viewed to see if any errors messages were generated by the switch. The command to use is:

-> show log swlog

Displaying file contents for 'swlog2.log' FILEID: fileName[swlog2.log], endPtr[32]

configSize[64000], currentSize[64000], mode[2] Displaying file contents for 'swlog1.log' FILEID: fileName[swlog1.log], endPtr[395]

configSize[64000], currentSize[64000], mode[1]

Time Stamp Application Level Log Message

------------------------+--------------+-------+--------------------------------

MON AUG 21 23:09:57 2023 HSM-CHASSIS info == HSM == GBIC extraction detect ed on NI slot 1, GBIC port 2 MON AUG 21 23:28:33 2023 HSM-CHASSIS info == HSM == GBIC Insertion detecte d on NI slot 1, GBIC port 1 MON AUG 21 23:28:33 2023 HSM-CHASSIS info == HSM == GBIC Insertion detecte

OmniSwitch Troubleshooting Guide September 2005 page 1-7

Troubleshooting System for OS-6624/6648 and OS-7/8XXX Troubleshooting the Switch System

d on NI slot 1, GBIC port 2 MON AUG 21 23:28:39 2023 HSM-CHASSIS info == HSM == GBIC extraction detect ed on NI slot 5, GBIC port 2 MON AUG 21 23:30:39 2023 HSM-CHASSIS info == HSM == GBIC Insertion detecte d on NI slot 5, GBIC port 2 MON AUG 21 23:30:41 2023 HSM-CHASSIS info == HSM == GBIC extraction detect ed on NI slot 1, GBIC port 1 MON AUG 21 23:30:45 2023 HSM-CHASSIS info == HSM == GBIC extraction detect ed on NI slot 1, GBIC port 2 TUE AUG 22 00:05:45 2023 CSM-CHASSIS info == CSM == !!!ACTIVATING!!! TUE AUG 22 00:05:45 2023 CSM-CHASSIS info == CSM == !!! REBOOT !!! TUE AUG 22 00:05:53 2023 SYSTEM alarm System rebooted via ssReboot(), restart type=0x2 (COLD)

The log message are kept in two log files: swlog1.log and swlog2.log in flash. In the above example, log messages show that some GBICs were extracted and inserted at a particular time. In addition, the switch was rebooted. This information helps to relate the time of the problem together with the events happening at the switch. In addition, it also provides an idea about if the source of the problem was external or internal to the switch.

If the log messages do not show enough information then they can be changed for specific applications to a higher log level or for all the applications running in the switch. For setting up different log levels in switch log, please refer to the “Using Switch Logging” chapter in the appropriate OmniSwitch Network

Configuration Guide.

If the switch is running in redundant configuration make sure that the two CMMs are completely synchronized. This can be done using the command:

-> show running-directory Running CMM : PRIMARY, Running configuration : WORKING, Certify/Restore Status : CERTIFIED, Synchronization Status : SYNCHRONIZED

If the two CMMs are not synchronized and the problem leads to the failure of Primary CMM then it will result in re-initialization of all of the modules. If the two CMMs are properly synchronized and primary CMM failed, the take over mechanism will be transparent to the end user. So, for complete redundancy keep the two CMMs synchronized.

Look for any post-mortem dump files that may be created due to the problem with the switch. Post Mortem Dump files have an extension of *.dmp and are created in /flash directory of the CMM (be sure to check the secondary CMM, if running in redundant mode). System dump files are normally named as “cs_system.dmp”, Memory related dump files are normally created as “MemMon000.dmp” and NI related dump files are named as “SloXSliYver1.dmp”, where X is the slot number and Y is the slice number.

The creation of a dump file indicates a problem with the switch. System related dump files can be viewed through CLI but other dump files cannot. For system related dump files use the command:

-> show log pmd cs_system.pmd

Capture the output of this command. In addition, if there are any dump files created in the switch, they should be downloaded through FTP to forward them to technical support. Technical Support can have them analyzed to find the source of the problem.

page 1-8 OmniSwitch Troubleshooting Guide September 2005

Troubleshooting the Switch System Advanced Troubleshooting

Advanced Troubleshooting

One level of switch logging is stored in the two log files located in the /flash directory. There is another low level debug that can be enabled and used for diagnosing the problems. This debug is known as “systrace”, meaning system trace. The information in this trace is stored in NVRAM on the CMM, so it is valid until powered off. Soft reboot of the switch will retain the trace information but powering off the switch will result in loosing all of the information. This is less CMM intensive so can be used to collect all the background information about the different tasks running in the switch.

The command to look at the default settings for systrace is

-> debug systrace show sysTrace is:

- INITIALIZED

- RUNNING

- configured to TRACE CALLERS

- configured to NOT WATCH on stdout

All Applications have their trace level set to level ’info’ (6)

Systrace is set for the level of “info” for all the applications. Any application with trace level other than 6 is displayed in the above command output. Notice that it is initialized by default and is running in the background. By default it is configured not to display messages on the console. The purpose of systrace is to track all the system processes called and the caller.

Application log levels can be changed and specific applications can also be set for the logging purposes. The commands are similar to switch log.

-> debug systrace appid ?

WEB VRRP VLAN TRAP TELNET SYSTEM STP SSL SSH

SNMP SMNI SLB SESSION RSVP RMON QOS QDRIVER QDISPATCHER PSM PRB-CHASSIS PORT-MGR POLICY PMM NOSNMP NI-SUPERVISION NI-INTERFACE NAN-DRIVER MODULE MIPGW LINKAGG LDAP IPX IPMS IPC-MON IPC-LINK IPC-DIAG IP-HELPER IP INTERFACE HSM-CHASSIS HEALTH GMAP GM FTP EPILOGUE EIPC DRC DISTRIB DIAG CVM-CHASSIS CSM-CHASSIS CONFIG CMS-CHASSIS CMM-INTERFACE CLI CHASSIS CCM-CHASSIS BRIDGE AMAP ALL AAA 802.1Q <num>

(System Service & File Mgmt Command Set)

The applications and the log levels are the same as switch log applications. Please refer to the “Section Switch Logging Commands Overview” section in the “Using Switch Logging” chapter in the appropriate OmniSwitch Network Configuration Guide.

Systrace can be enabled using the command:

-> debug systrace enable

To look at the systrace log file use the following command:

swnygb02 > debug systrace show log TStamp(us) AppId Level Task Comment

----------+------+------+----------+--------------------------------------3349119104 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3349118980 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> Event = CS_CSM_HELLO

OmniSwitch Troubleshooting Guide September 2005 page 1-9

Advanced Troubleshooting Troubleshooting the Switch System

_SM_IPCUP_TIMEOUT 3349118948 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> CS_TIMEOUT 3345200526 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3342928783 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3342928661 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> Event = CS_CSM_HELLO _SM_IPCUP_TIMEOUT 3342928628 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> CS_TIMEOUT 3336738410 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3336738287 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> Event = CS_CSM_HELLO _SM_IPCUP_TIMEOUT 3336738256 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> CS_TIMEOUT 3334849145 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3330548020 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3330547902 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> Event = CS_CSM_HELLO _SM_IPCUP_TIMEOUT 3330547869 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> CS_TIMEOUT 3324495309 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3324357940 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3324357816 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> Event = CS_CSM_HELLO _SM_IPCUP_TIMEOUT 3324357782 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> CS_TIMEOUT 3318167293 CSM-CH info tCsCSMtask ***HELLO FSM TRACE*** 3318167171 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> Event = CS_CSM_HELLO _SM_IPCUP_TIMEOUT 3318167139 CSM-CH info tCsCSMtask csCsmHelloReceptio - -> CS_TIMEOUT

This information is useful to analyze the different processes taking place in the switch.

Other useful command to use in case of problem is:

-> show tech-support

This command captures all of the information from the chassis, including the hardware information, configuration, software release active and some other statistics about the number of buffers being used at the time of the use of command. The output of the command is saved in /flash as “tech_support.log”. Other variation of this command is:

-> show tech-support layer2

This command collects Layer 2 data only.

-> show tech-support layer3

This command collects Layer 3 data only.

page 1-10 OmniSwitch Troubleshooting Guide September 2005

+ 584 hidden pages

Alcatel-Lucent 6648, 8800, 6624, 6800 User Manual

Contents

About This Guide

Supported Platforms

Who Should Read this Manual?

When Should I Read this Manual?

What is in this Manual?

What is Not in this Manual?

How is the Information Organized?

Related Documentation

Before Calling Alcatel’s Technical Assistance Center

In This Chapter

Introduction

Troubleshooting System for OS-6624/6648 and OS-7/8XXX

Advanced Troubleshooting