Cisco NCS 2000 series Troubleshooting Manual

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

First Published: 2019-03-08

Americas Headquarters

Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]

800 553-NETS (6387)

Fax: [phone]

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15
of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment
generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications.
Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.

The following information is for FCC compliance of Class B devices: This equipment has been tested and found to comply with the limits for a Class B digital device, pursuant to part 15 of
the FCC rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiate radio
frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference
will not occur in a particular installation. If the equipment causes interference to radio or television reception, which can be determined by turning the equipment off and on, users are
encouraged to try to correct the interference by using one or more of the following measures:

• Reorient or relocate the receiving antenna.

• Increase the separation between the equipment and receiver.

• Connect the equipment into an outlet on a circuit different from that to which the receiver is connected.

• Consult the dealer or an experienced radio/TV technician for help.

Modifications to this product not authorized by Cisco could void the FCC approval and negate your authority to operate the product.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of
the UNIX operating system. All rights reserved. Copyright©1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED "AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown forillustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative contentis unintentional
and coincidental.

All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.

Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. Toview a list of Cisco trademarks, go to this URL: www.cisco.com

go trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any

other company. (1721R)

©

2019 Cisco Systems, Inc. All rights reserved.

CONTENTS

Introduction ?

PREFACE

CHAPTER 1

Preface xxxvii

Document Objectives xxxvii

Audience xxxvii

Document Organization xxxviii

Document Conventions xxxviii

Related Documentation xliv

Obtaining Optical Networking Information xlv

Where to Find Safety and Warning Information xlv

Communications, Services, and Additional Information xlv

General Troubleshooting 1

Loopback Description 1

Facility Loopbacks 2

General Behavior 2

Card Behavior 3

Terminal Loopbacks 5

General Behavior 5

Card Behavior 6

Troubleshooting MXP, TXP, XP, or ADM-10G Circuit Paths With Loopbacks 7

Perform a Facility Loopback on a Source-Node MXP or TXP Port 8

Create the Facility Loopback on the Source-Node MXP, TXP, XP or ADM-10G Port 9

Test and Clear the MXP, TXP, XP or ADM-10G Facility Loopback Circuit 10

Test the MXP, TXP, XP or ADM-10G Card 11

Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port 12

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

iii

Contents

Create the Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port 13

Test and Clear the MXP, TXP, XP, or ADM-10G Port Terminal Loopback Circuit 14

Test the MXP, TXP, XP, or ADM-10G Card 14

Create a Facility Loopback on an Intermediate-Node MXP or TXP Port 15

Create a Facility Loopback on an Intermediate-Node MXP or TXP Port 16

Test and Clear the MXP or TXP Port Facility Loopback Circuit 17

Test the MXP or TXP Card 17

Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports 18

Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports 19

Test and Clear the MXP or TXP Terminal Loopback Circuit 20

Test the MXP or TXP Card 20

Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port 21

Create the Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port 22

Test and Clear the MXP, TXP, XP, or ADM-10G Facility Loopback Circuit 23

Test the MXP, TXP, XP, or ADM-10G Card 23

Perform a Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port 24

Create the Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port 25

Test and Clear the MXP, TXP, XP, or ADM-10G Terminal Loopback Circuit 26

Test the MXP, TXP, XP, or ADM-10G Card 26

Troubleshooting DWDM Circuit Paths With ITU-T G.709 Monitoring 27

ITU-T G.709 Monitoring in Optical Transport Networks 27

Optical Channel Layer 28

Optical Multiplex Section Layer 28

Optical Transmission Section Layer 28

Performance Monitoring Counters and Threshold Crossing Alerts 29

Set Node Default BBE or SES Card Thresholds 29

Provision Individual Card BBE or SES Thresholds in CTC 30

Provision Card PM Thresholds Using TL1 30

Provision Optical TCA Thresholds 31

Forward Error Correction 32

Provision Card FEC Thresholds 32

Sample Trouble Resolutions 32

Using CTC Diagnostics 33

Card LED Lamp Tests 34

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

iv

Verify Card LED Operation 34

Retrieve Tech Support Logs Button 34

Off-Load the Diagnostics File 35

Data Communications Network Tool 37

Onboard Failure Logging 37

Run Time Log for IO Cards 38

Snapshot Log for IO Cards 39

Snapshot Logging in CTC 39

Restoring the Database and Default Settings 40

Restore the Node Database 40

PC Connectivity Troubleshooting 40

Unable to Verify the IP Configuration of Your PC 42

Verify the IP Configuration of Your PC 42

Contents

Browser Login Does Not Launch Java 43

Reconfigure the PC Operating System Java Plug-in Control Panel 43

Reconfigure the Browser 44

Unable to Verify the NIC Connection on Your PC 45

Verify PC Connection to the NCS (ping) 45

Ping the NCS 45

The IP Address of the Node is Unknown 46

Retrieve Unknown Node IP Address 46

CTC Operation Troubleshooting 47

CTC Colors Do Not Appear Correctly on a UNIX Workstation 47

Limit Netscape Colors 47

Unable to Launch CTC Help After Removing Netscape 48

Reset Internet Explorer as the Default Browser for CTC 48

Unable to Change Node View to Network View 48

Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Windows

49

Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Solaris

50

Browser Stalls When Downloading CTC JAR Files From Control Card 50

Disable the VirusScan Download Scan 50

CTC Does Not Launch 51

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

v

Contents

Redirect the Netscape Cache to a Valid Directory 51

Slow CTC Operation or Login Problems 52

Delete the CTC Cache File Automatically 52

Delete the CTC Cache File Manually 53

Node Icon is Gray on CTC Network View 53

Java Runtime Environment Incompatible 54

Launch CTC to Correct the Core Version Build 54

Different CTC Releases Do Not Recognize Each Other 55

Launch CTC to Correct the Core Version Build 55

Username or Password Do Not Match 55

Verify Correct Username and Password 56

DCC Connection Lost 56

Path in Use Error When Creating a Circuit 56

Calculate and Design IP Subnets 57

Timing 57

NCS Switches Timing Reference 57

Holdover Synchronization Alarm 57

Free-Running Synchronization Mode 58

Daisy-Chained BITS Not Functioning 58

Blinking STAT LED after Installing a Card 58

Fiber and Cabling 59

Bit Errors Appear for a Traffic Card 59

Faulty Fiber-Optic Connections 59

Crimp Replacement LAN Cables 60

Replace Faulty SFP, SFP+, or XFP Connectors 61

Remove SFP or XFP Connectors 62

Install an SFP, SFP+, or XFP Connector 62

Power Supply Problems 63

Isolate the Cause of Power Supply Problems 64

Power Up Problems for Node and Cards 65

Network Level (Internode) Problems 65

System Restart after a Fiber Cut 65

Scenario 1: Span Loss Change > 5 dBm and OSC Power Value on the Receiver less than –42

dBm 66

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

vi

Scenario 2: Span Loss Change > 5 dBm and OSC Power Value on the Receiver > –42 dBm 68

Scenario 3: 3 dBm less than Span Loss Change less than 5 dBm 72

Scenario 4: Span Loss Change less than 3 dB 75

OCHNC Circuits Creation Failure 76

Prerequisites for Successful OCHNC Circuit Creation 77

Conditions for OCHNC Circuit Creation Failure 78

Scenarios for OCHNC Circuit Creation Failure 78

Node Level (Intranode) Problems 80

VOA Startup Phases 80

VOA Failure Scenarios 81

Scenario A: Optical Power Level of the Incoming Signal Lower Than Minimum Allowed by MSTP

Supported Optical Interfaces 82

Scenario B: Optical Power Level of the Incoming Signal Lower Than Expected 86

Corrective Actions for Scenario B (Optical Power Level of Incoming Signal Lower than Expected)

87

Contents

CHAPTER 2

Scenario C: Optical Drop Power Level Lower Than Expected 92

Corrective Action for Scenario C (Optical Power Level of Incoming Signal Lower than Expected)

93

Counter-Propagating Light Affecting Operation of 32DMX-C and 32DMX-L Cards 97

Corrective Action for Software Releases Lower than 9.0 97

Alarm Troubleshooting 99

Alarm Indexes 109

Logical Objects 109

Alarm Logical Objects 110

Trouble Characterizations 111

Safety Summary 113

Trouble-Clearing Procedures 114

ACT-SOFT-VERIF-FAIL 115

Clear the ACT-SOFT-VERIF-FAIL Alarm 115

AIS 115

Clear the AIS Condition 116

AIS-L 116

Clear the AIS-L Condition 116

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

vii

Contents

AIS-P 117

Clear the AIS-P Condition 117

ALS 117

ALS-DISABLED 117

Clear the ALS-DISABLED Condition 118

AMPLI-INIT 118

Clear the AMPLI-INIT Condition 118

APC-CORR-SKIPPED 119

APC-DISABLED 119

Clear the APC-DISABLED Alarm 120

APC-END 120

APC-OUT-OF-RANGE 120

Clear the APC-OUT-OF-RANGE Alarm 121

APC-WRONG-GAIN 121

Clear the APC-WRONG-GAIN Alarm 121

APSB 122

Clear the APSB Alarm 122

APSCM 123

Clear the APSCM Alarm 123

APSIMP 124

Clear the APSIMP Alarm 124

APSMM 125

Clear the APSMM Alarm 125

APS-NO-RESPONSE 126

Clear the APS-NO-RESPONSE Alarm 126

APS-PROV-MISM 126

Clear the APS-PROV-MISM Alarm 126

AS-CMD 126

Clear the AS-CMD Condition 127

AS-MT 128

viii

Clear the AS-MT Condition 129

AU-AIS 129

Clear the AU-AIS Condition 129

AU-LOP 130

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Clear the AU-LOP Alarm 130

AUTH-EC 131

Clear the AUTH-EC Alarm 131

AUTO-SENSE 131

AUTO-SENSE-DSBLD 132

Clear the AUTO-SENSE-DSBLD Alarm 132

AUTORESET 132

Clear the AUTORESET Alarm 133

AUTOSW-AIS 133

Clear the AUTOSW-AIS Condition 134

AUTOSW-AIS-SNCP 134

Clear the AUTOSW-AIS-UPSR Condition 134

AUTOSW-LOP (STSMON) 135

Contents

Clear the AUTOSW-LOP (STSMON) Condition 135

AUTOSW-LOP-SNCP 135

Clear the AUTOSW-LOP-SNCP Alarm 136

AUTOSW-PDI 136

Clear the AUTOSW-PDI Condition 136

AUTOSW-PDI-SNCP 137

Clear the AUTOSW-PDI-SNCP Condition 137

AUTOSW-SDBER 137

Clear the AUTOSW-SDBER Condition 138

AUTOSW-SDBER-SNCP 138

Clear the AUTOSW-SDBER-SNCP Condition 138

AUTOSW-SFBER 139

Clear the AUTOSW-SFBER Condition 139

AUTOSW-SFBER-SNCP 139

Clear the AUTOSW-SFBER-SNCP Condition 140

AUTOSW-UNEQ (STSMON) 140

Clear the AUTOSW-UNEQ (STSMON) Condition 140

AUTOSW-UNEQ-SNCP (VCMON-HP) 141

Clear the AUTOSW-UNEQ-SNCP (VCMON-HP) Condition 141

AWG-DEG 142

Clear the AWG-DEG Alarm 142

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

ix

Contents

AWG-FAIL 142

Clear the AWG-FAIL Alarm 142

AWG-OVERTEMP 143

Clear the AWG-OVERTEMP Alarm 143

AWG-WARM-UP 143

BAD-DB-DETECTED 144

Clear the BAD-DB-DETECTED Alarm 144

BAT-FAIL 144

Clear the BAT-FAIL Alarm 145

BP-LPBKFACILITY 145

Clear the BP-LPBKFACILTIY Alarm 145

BP-LPBKTERMINAL 146

Clear the BP-LPBKTERMINAL Alarm 146

CARLOSS (EQPT) 147

Clear the CARLOSS (EQPT) Alarm 147

CARLOSS (FC) 149

Clear the CARLOSS (FC) Alarm 149

CARLOSS (GE) 149

Clear the CARLOSS (GE) Alarm 149

CARLOSS (ISC) 150

Clear the CARLOSS (ISC) Alarm 151

CARLOSS (TRUNK) 151

Clear the CARLOSS (TRUNK) Alarm 151

CASETEMP-DEG 152

Clear the CASETEMP-DEG Alarm 153

CD 153

Clear the CD Alarm 154

CFM-CONFIG-ERROR 154

Clear the CFM-CONFIG-ERROR Condition 154

CFM-LOOP 155

Clear the CFM-LOOP Condition 155

CFM-MEP-DOWN 156

Clear the CFM-MEP-DOWN Condition 156

CFM-XCON-SERVICE 156

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

x

Clear the CFM-XCON-SERVICE Condition 156

CHANLOSS 157

Clear the CHANLOSS Condition 158

CHAN-PWR-THRESHOLD-CHECK 158

Clear the CHAN-PWR-THRESHOLD-CHECK Alarm 158

CLDRESTART 159

Clear the CLDRESTART Condition 159

COMP-CARD-MISSING 159

Clear the COMP-Card-Missing Alarm 160

COMM-FAIL 160

Clear the COMM-FAIL Alarm 161

COOL-MISM 161

Clear the COOL-MISM Alarm 161

Contents

CP-UNVER-CLEARED Alarm 162

CTNEQPT-MISMATCH 162

Clear the CTNEQPT-MISMATCH Condition 163

DATA-CRC 163

Clear the DATA-CRC Alarm 163

DBOSYNC 164

Clear the DBOSYNC Alarm 164

DCU-LOSS-FAIL 164

Clear the DCU-LOSS-FAIL Condition 165

DISCONNECTED 165

Clear the DISCONNECTED Alarm 165

DSP-COMM-FAIL 166

DSP-FAIL 166

Clear the DSP-FAIL Alarm 166

DUP-IPADDR 167

Clear the DUP-IPADDR Alarm 167

DUP-NC 168

Clear the DUP-NC Alarm 168

DUP-NODENAME 168

Clear the DUP-NODENAME Alarm 169

DUP-SHELF-ID 169

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xi

Contents

Clear the DUP-SHELF-ID Alarm 169

EPROM-SUDI-SN-MISMATCH 170

Clear the EPROM-SUDI-SN-MISMATCH Alarm 170

EFM-PEER-MISSING 170

Clear the EFM-PEER-MISSING Condition 170

EFM-RFI-CE 171

Clear the EFM-RFI-CE Alarm 171

EFM-RFI-DG 171

Clear the EFM-RFI-DG Alarm 172

EFM-RFI-LF 172

Clear the EFM-RFI-LF Alarm 172

EFM-RLBK 172

Clear the EFM-RLBK Condition 173

EHIBATVG 173

Clear the EHIBATVG Alarm 173

ELWBATVG 174

Clear the ELWBATVG Alarm 174

ENCAP-MISMATCH-P 174

Clear the ENCAP-MISMATCH-P Alarm 175

EMBEDDED-AMPLIFIER-SATURATED 176

Clear the EMBEDDED-AMPLIFIER-SATURATED Alarm 176

EOC-E 176

Clear the EOC-E Alarm 177

EOC-L 179

Clear the EOC-L Alarm 180

EQPT 180

Clear the EQPT Alarm 181

EQPT-DEGRADE 181

Clear the EQPT-DEGRADE Condition 182

EQPT-DIAG 182

xii

Clear the EQPT-DIAG Alarm 182

EQPT-FAIL 183

Clear the EQPT-FAIL Alarm 183

EQPT-FPGA-IMAGE-AVAILABLE 184

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Clear the EQPT-FPGA-IMAGE-AVAILABLE Condition 184

EQPT-MISS 184

Clear the EQPT-MISS Alarm 184

ERFI-P-SRVR 185

Clear the ERFI-P-SRVR Condition 185

ESMC-FAIL 185

Clear the ESMC-FAIL Alarm 186

ETH-LINKLOSS 186

Clear the ETH-LINKLOSS Alarm 186

EVAL-LIC 187

Clear the EVAL-LIC Alarm 187

EXC-BP 187

Clear the EXC-BP Condition 187

Contents

EXCCOL 188

Clear the EXCCOL Alarm 188

EXT 188

Clear the EXT Alarm 189

FAILTOSW (2R, EQPT, ESCON, FC, GE, ISC, OCN/STMN, TRUNK, OTS) 189

Clear the FAILTOSW (2R, EQPT, ESCON, FC, GE, ISC, OCN/STMN, TRUNK, OTS) Condition

190

FAILTOSW (TRUNK) 190

Clear the FAILTOSW (TRUNK) Condition 190

FAILTOSW-HO 191

Clear the FAILTOSW-HO Condition 191

FAILTOSW-PATH 192

Clear the FAILTOSW-PATH Condition in a Path Protection Configuration 192

FAN 193

Clear the FAN Alarm 193

FAPS 194

Clear the FAPS Alarm 194

FAPS-CONFIG-MISMATCH 194

Clear the FAPS-CONFIG-MISMATCH Condition 194

FC-NO-CREDITS 195

Clear the FC-NO-CREDITS Alarm 195

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xiii

Contents

FDI 196

Clear the FDI Condition 196

FE-FRCDWKSWBK-SPAN 197

Clear the FE-FRCDWKSWBK-SPAN Condition 197

FE-FRCDWKSWPR-SPAN 198

Clear the FE-FRCDWKSWPR-SPAN Condition 198

FE-MANWKSWBK-SPAN 198

Clear the FE-MANWKSWBK-SPAN Condition 199

FE-MANWKSWPR-SPAN 199

Clear the FE-MANWKSWPR-SPAN Condition 199

FEC-MISM 200

Clear the FEC-MISM Alarm 200

FEED-MISMATCH 201

FEPRLF 201

Clear the FEPRLF Alarm on an BLSR 202

FIBERTEMP-DEG 202

Clear the FIBERTEMP-DEG Alarm 202

FIPS-TEST-FAILED 203

Clearing the FIPS-TEST-FAILED Alarm 203

FORCED-REQ 203

Clear the FORCED-REQ Condition 204

FORCED-REQ-SPAN (2R, ESCON, FC, GE, ISC, OCN/STMN, OTS) 204

FORCED-REQ-SPAN (TRUNK) 204

FP-LINK-LOSS 205

Clear the FP-LINK-LOSS Condition 205

FPGA-UPGRADE-FAILED 205

Clear the FPGA-UPGRADE-FAILED Alarm 205

FRCDSWTOINT 206

FRCDSWTOPRI 206

FRCDSWTOSEC 206

xiv

FRCDSWTOTHIRD 206

FRNGSYNC 207

Clear the FRNGSYNC Condition 207

FSTSYNC 208

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

FTA-MISMATCH 208

Clear the FTA-MISMATCH Condition 208

GAIN-HDEG 208

Clear the GAIN-HDEG Alarm 209

GAIN-HFAIL 210

Clear the GAIN-HFAIL Alarm 210

GAIN-LDEG 211

Clear the GAIN-LDEG Alarm 211

GAIN-LFAIL 211

Clear the GAIN-LFAIL Alarm 212

GAIN-NEAR-LIMIT 212

Clear the GAIN-NEAR-LIMIT Alarm 212

GCC-EOC 213

Contents

Clear the GCC-EOC Alarm 213

GE-OOSYNC (FC, GE, ISC) 213

Clear the GE-OOSYNC (FC, GE, ISC) Alarm 213

GE-OOSYNC (TRUNK) 214

Clear the GE-OOSYNC (TRUNK) Alarm 214

GFP-CSF-SIGLOSS 215

Clear the GFP-CSF-SIGLOSS Alarm 215

GFP-CSF-SYNCLOSS 215

Clear the GFP-CSF-SYNCLOSS Alarm 216

GFP-LFD 216

Clear the GFP-LFD Alarm 216

GFP-UP-MISMATCH 217

Clear the GFP-UP-MISMATCH Alarm 217

HELLO 217

Clear the HELLO Alarm 218

HIBATVG 218

Clear the HIBATVG Alarm 218

HI-BER 219

Clear the HI-BER Alarm 219

HI-CCVOLT 220

Clear the HI-CCVOLT Condition 220

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xv

Contents

HI-LASERBIAS 220

Clear the HI-LASERBIAS Alarm 221

HI-LASERTEMP 221

Clear the HI-LASERTEMP Alarm 221

HI-RXPOWER 222

Clear the HI-RXPOWER Alarm 222

HITEMP 223

Clear the HITEMP Alarm 224

HI-RXTEMP 225

Clear the HI-RXTEMP Alarm 225

HI-TXPOWER 225

Clear the HI-TXPOWER Alarm 225

HLDOVRSYNC 226

Clear the HLDOVRSYNC Condition 226

HP-DEG 227

Clear the HP-DEG Condition 228

HP-ENCAP-MISMATCH 228

Clear the HP-ENCAP-MISMATCH Alarm 229

HP-EXC 230

Clear the HP-EXC Condition 230

HP-PLM 230

HP-RFI 230

Clear the HP-RFI Condition 231

HP-TIM 231

Clear the HP-TIM Alarm 231

HP-UNEQ 232

Clear the HP-UNEQ Alarm 232

I-HITEMP 234

Clear the I-HITEMP Alarm 234

ILK-FAIL 234

xvi

Clear the ILK-FAIL Alarm 235

IMPROPRMVL 235

Clear the IMPROPRMVL Alarm 236

INHSWPR 237

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Clear the INHSWPR Condition 237

INHSWWKG 238

Clear the INHSWWKG Condition 238

INCOMPATIBLE-SEND-PDIP 238

Clear the INCOMPATIBLE-SEND-PDIP Alarm 239

INCOMPATIBLE-SW 239

Clear the INCOMPATIBLE-SW Alarm 239

INTRUSION-PSWD 239

Clear the INTRUSION-PSWD Condition 240

INVALID-SYSDB 240

Clear the INVALID-SYSDB Alarm 240

INVALID-MUXCONF 241

Clear the INVALID-MUXCONF Alarm 241

Contents

INVMACADR 241

Clear the INVMACADR Alarm 242

IMPROPRMVL-FS 242

IPC-LASER-FAIL 242

IPC-LOOPBACK-MISS 243

Clear the IPC-LOOPBACK-MISS Alarm 243

IPC-VERIFICATION-DEGRADE 243

IPC-VERIFICATION-FAIL 244

ISIS-ADJ-FAIL 244

Clear the ISIS-ADJ-FAIL Alarm 244

IPC-VERIFICATION-RUNNING 246

Clear the IPC-VERIFICATION-RUNNING Alarm 246

KEY-EX-FAIL 246

Clearing the KEY-EX-FAIL Alarm 247

KEY-WRITE-FAIL 247

Clearing the KEY-WRITE-FAIL Alarm 248

LASER-APR 248

LASER-OFF-WVL-DRIFT 248

Clear the LASER-OFF-WVL-DRIFT Condition 249

LASERBIAS-DEG 249

Clear the LASERBIAS-DEG Alarm 249

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xvii

Contents

LASERBIAS-FAIL 250

Clear the LASERBIAS-FAIL Alarm 250

LASEREOL 250

Clear the LASEREOL Alarm 251

LASERTEMP-DEG 251

Clear the LASERTEMP-DEG Alarm 251

LICENSE-EXPIRED 252

Clear the LICENSE-EXPIRED Alarm 252

LIC-EXPIRING-SHORTLY 252

Clear the LIC-EXPIRING-SHORTLY Alarm 253

LIC-EXPIRING-SOON 253

Clear the LIC-EXPIRING-SOON Alarm 253

LIC-MISSING 254

Clear the LIC-MISSING Alarm 254

LMP-FAIL 254

Clear the LMP-FAIL Alarm 255

LMP-SD 256

Clear the LMP-SD Condition 256

LMP-SF 257

Clear the LMP-SF Condition 257

LMP-UNALLOC 259

LOCAL-CERT-CHAIN-VERIFICATION-FAILED 259

Clear the LOCAL-CERT-CHAIN-VERIFICATION-FAILED Alarm 259

LOCAL-CERT-ISSUED-FOR-FUTURE-DATE 260

Clear the LOCAL-CERT-ISSUED-FOR-FUTURE-DATE Alarm 260

LOCAL-CERT-EXPIRING-WITHIN-30-DAYS 260

Clear the LOCAL-CERT-EXPIRING-WITHIN-30-DAYS Alarm 260

LOCAL-SUDI-CERT-VERIFICATION-FAILED 260

Clear the LOCAL-SUDI-CERT-VERIFICATION-FAILED Alarm 261

LOCAL-CERT-EXPIRED 261

xviii

Clear the LOCAL-CERT-EXPIRED Alarm 261

LOCAL-FAULT 261

Clear the LOCAL-FAULT Alarm 262

LOCKOUT-REQ 262

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Clear the LOCKOUT-REQ Condition 262

LOCKOUT-REQ (2R, EQPT, ESCON, FC, GE, ISC) 263

Clear the LOCKOUT-REQ (2R, EQPT, ESCON, FC, GE, ISC) Condition 263

LOCKOUT-REQ (TRUNK) 263

Clear the LOCKOUT-REQ (TRUNK) Condition 263

LOF (BITS) 264

Clear the LOF (BITS) Alarm 264

LOF (TRUNK) 265

Clear the LOF (TRUNK) Alarm 265

LOGBUFR90 266

LOGBUFROVFL 267

Clear the LOGBUFROVFL Alarm 267

LO-LASERBIAS 267

Contents

Clear the LO-LASERBIAS Alarm 267

LO-LASERTEMP 268

Clear the LO-LASERTEMP Alarm 268

LOM 268

Clear the LOM Alarm 269

LOP-P 270

Clear the LOP-P Alarm 270

LO-RXPOWER 271

Clear the LO-RXPOWER Alarm 271

LOS (2R) 272

Clear the LOS (2R) Alarm 272

LOS (BITS) 273

Clear the LOS (BITS) Alarm 274

LOS (ESCON) 274

Clear the LOS (ESCON) Alarm 274

LOS (ISC) 276

Clear the LOS (ISC) Alarm 276

LOS (OTS) 277

Clear the LOS (OTS) Alarm 277

LOS (TRUNK) 278

Clear the LOS (TRUNK) Alarm 279

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xix

Contents

LOS-O 280

Clear the LOS-O Alarm 280

LOS-P (AOTS, OMS, OTS) 282

Clear the LOS-P (AOTS, OMS, OTS) Alarm 282

LOS-P (OCH) 284

Clear the LOS-P (OCH) Alarm 284

LOS-P (TRUNK) 287

Clear the LOS-P (TRUNK) Alarm 288

LOS-RAMAN (OTS) 289

Clear the LOS-RAMAN Condition 290

LO-TXPOWER 290

Clear the LO-TXPOWER Alarm 291

LPBKCRS 291

Clear the LPBKCRS Condition 292

LPBKFACILITY (ESCON) 292

Clear the LPBKFACILITY (ESCON) Condition 292

LPBKFACILITY (FC) 293

Clear the LPBKFACILITY (FC) Condition 293

LPBKFACILITY (GE) 293

Clear the LPBKFACILITY (GE) Condition 294

LPBKFACILITY (ISC) 294

Clear the LPBKFACILITY (ISC) Condition 294

LPBKFACILITY (TRUNK) 295

Clear the LPBKFACILITY (TRUNK) Condition 295

LPBKTERMINAL (ESCON) 295

Clear the LPBKTERMINAL (ESCON) Condition 296

LPBKTERMINAL (FC) 296

Clear the LPBKTERMINAL (FC) Condition 296

LPBKTERMINAL (GE) 297

Clear the LPBKTERMINAL (GE) Condition 297

LPBKTERMINAL (ISC) 297

Clear the LPBKTERMINAL (ISC) Condition 297

LPBKTERMINAL (TRUNK) 298

Clear the LPBKTERMINAL (TRUNK) Condition 298

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xx

LSC-NOT-PRESENT-MIC-IN-USE 298

Clear the LSC-NOT-PRESENT-MIC-IN-USE Alarm 299

LWBATVG 299

Clear the LWBATVG Alarm 299

MAN-LASER-RESTART 299

Clear the MAN-LASER-RESTART Condition 300

MAN-REQ 300

Clear the MAN-REQ Condition 300

MANRESET 300

MANSWTOINT 301

MANSWTOPRI 301

MANSWTOSEC 301

MANSWTOTHIRD 301

Contents

MANUAL-REQ-SPAN (2R, ESCON, FC, GE, ISC, OCN/STMN, OTS) 302

MANUAL-REQ-SPAN (TRUNK) 302

MEA (AIP) 302

Clear the MEA (AIP) Alarm 302

MEA (PPM) 303

Clear the MEA (PPM) Alarm 303

MEA (SHELF) 304

Clear the MEA (SHELF) Condition 304

MEM-GONE 305

MEM-LOW 305

MFGMEM 305

Clear the MFGMEM Alarm 306

MS-AIS 306

Clear the MS-AIS Condition 306

MS-DEG 307

Clear the MS-DEG Condition 307

MS-EOC 307

Clear the MS-EOC Alarm 307

MS-EXC 308

Clear the MS-EXC Condition 308

MS-RFI 308

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxi

Contents

Clear the MS-RFI Condition 309

MT-OCHNC 309

Clear the MT-OCHNC Condition 309

NO-SHARED-CIPHERS Alarm 309

Clear the NO-SHARED-CIPHERS Alarm 310

NON-CISCO-PPM 310

Clear the NON-CISCO-PPM Condition 310

NODE-FACTORY-MODE 310

Clear the NODE-FACTORY-MODE Alarm 311

NOT-AUTHENTICATED 311

OCHNC-BDI 311

Clear the OCHNC-BDI Alarm 311

OCHNC-INC 312

Clear the OCHNC-INC Alarm 313

OCHNC-SIP 313

Clear the OCHNC-SIP Alarm 313

OCHTERM-INC 314

Clear the OCHTERM-INC Condition 314

ODUK-1-AIS-PM 314

Clear the ODUK-1-AIS-PM Condition 315

ODUK-2-AIS-PM 315

Clear the ODUK-2-AIS-PM Condition 315

ODUK-3-AIS-PM 315

Clear the ODUK-3-AIS-PM Condition 316

ODUK-4-AIS-PM 316

Clear the ODUK-4-AIS-PM Condition 316

ODUK-AIS-PM 317

Clear the ODUK-AIS-PM Condition 317

ODUK-BDI-PM 317

Clear the ODUK-BDI-PM Condition 318

xxii

ODUK-LCK-PM 318

Clear the ODUK-LCK-PM Condition 318

ODUK-OCI-PM 319

Clear the ODUK-OCI-PM Condition 319

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

ODUK-SD-PM 319

Clear the ODUK-SD-PM Condition 319

ODUK-SF-PM 320

Clear the ODUK-SF-PM Condition 320

ODUK-TIM-PM 320

Clear the ODUK-TIM-PM Condition 321

OPEN-SLOT 321

Clear the OPEN-SLOT Alarm 321

OPTNTWMIS 322

Clear the OPTNTWMIS Alarm 322

OPWR-HDEG 322

Clear the OPWR-HDEG Alarm 323

OPWR-HFAIL 325

Contents

Clear the OPWR-HFAIL Alarm 326

OPWR-LDEG 326

Clear the OPWR-LDEG Alarm 326

OPWR-LFAIL 327

Clear the OPWR-LFAIL Alarm 327

OSRION 327

Clear the OSRION Condition 327

OTDR-ABSOLUTE-A-EXCEEDED-RX 328

Clear the OTDR-ABSOLUTE-A-EXCEEDED-RX Alarm 328

OTDR-ABSOLUTE-A-EXCEEDED-TX 328

Clear the OTDR-ABSOLUTE-A-EXCEEDED-TX Alarm 329

OTDR-ABSOLUTE-R-EXCEEDED-RX 329

Clear the OTDR-ABSOLUTE-R-EXCEEDED-RX Alarm 329

OTDR-ABSOLUTE-R-EXCEEDED-TX 329

Clear the OTDR-ABSOLUTE-R-EXCEEDED-TX Alarm 330

OTDR-BASELINE-A-EXCEEDED-RX 330

Clear the OTDR-BASELINE-A-EXCEEDED-RX Alarm 330

OTDR-BASELINE-A-EXCEEDED-TX 330

Clear the OTDR-BASELINE-A-EXCEEDED-TX Alarm 331

OTDR-BASELINE-R-EXCEEDED-RX 331

Clear the OTDR-BASELINE-R-EXCEEDED-RX Alarm 331

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxiii

Contents

OTDR-BASELINE-R-EXCEEDED-TX 331

Clear the OTDR-BASELINE-R-EXCEEDED-TX Alarm 332

OTDR-FAST-FAR-END-IN-PROGRESS 332

Clear the OTDR-FAST-FAR-END-IN-PROGRESS Alarm 332

OTDR-FAST-SCAN-IN-PROGRESS-RX 332

Clear the OTDR-FAST-SCAN-IN-PROGRESS-RX Alarm 332

OTDR-FAST-SCAN-IN-PROGRESS-TX 333

Clear the OTDR-FAST-SCAN-IN-PROGRESS-TX Alarm 333

OTDR-FIBER-END-NOT-DETECTED-RX 333

Clear the OTDR-FIBER-END-NOT-DETECTED-RX Alarm 333

OTDR-FIBER-END-NOT-DETECTED-TX 334

Clear the OTDR-FIBER-END-NOT-DETECTED-TX Alarm 334

OTDR-HYBRID-FAR-END-IN-PROGRESS 334

Clear the OTDR-HYBRID-FAR-END-IN-PROGRESS Alarm 334

OTDR-HYBRID-SCAN-IN-PROGRESS-RX 335

Clear the OTDR-HYBRID-SCAN-IN-PROGRESS-RX Alarm 335

OTDR-HYBRID-SCAN-IN-PROGRESS-TX 335

Clear the OTDR-HYBRID-SCAN-IN-PROGRESS-TX Alarm 335

OTDR-ORL-THRESHOLD-EXCEEDED-RX 335

Clear the OTDR-ORL-THRESHOLD-EXCEEDED-RX Alarm 336

OTDR-ORL-THRESHOLD-EXCEEDED-TX 336

Clear the OTDR-ORL-THRESHOLD-EXCEEDED-TX Alarm 336

OTDR-ORL-TRAINING-FAILED-RX 336

Clear the OTDR-ORL-TRAINING-FAILED-RX Alarm 337

OTDR-ORL-TRAINING-FAILED-TX 337

Clear the OTDR-ORL-TRAINING-FAILED-TX Alarm 337

OTDR-ORL-TRAINING-IN-PROGRESS-RX 337

Clear the OTDR-ORL-TRAINING-IN-PROGRESS-RX Alarm 337

OTDR-ORL-TRAINING-IN-PROGRESS-TX 338

Clear the OTDR-ORL-TRAINING-IN-PROGRESS-TX Alarm 338

xxiv

OTDR-OTDR-TRAINING-FAILED-RX 338

Clear the OTDR-OTDR-TRAINING-FAILED-RX Alarm 338

OTDR-OTDR-TRAINING-FAILED-TX 339

Clear the OTDR-OTDR-TRAINING-FAILED-TX Alarm 339

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

OTDR-SCAN-FAILED 339

Clear the OTDR-SCAN-FAILED Alarm 339

OTDR-SCAN-IN-PROGRESS 339

OTDR-SCAN-NOT-COMPLETED 340

Clear the OTDR-SCAN-NOT-COMPLETED Alarm 340

OTUK-AIS 340

Clear the OTUK-AIS Condition 341

OTUK-BDI 341

Clear the OTUK-BDI Condition 341

OTUK-IAE 342

Clear the OTUK-IAE Alarm 343

OTUK-LOF 343

Clear the OTUK-LOF Alarm 344

Contents

OTUK-SD 344

Clear the OTUK-SD Condition 345

OTUK-SF 346

Clear the OTUK-SF Condition 346

OTUK-TIM 346

Clear the OTUK-TIM Condition 347

OUT-OF-BUNDLE 347

Clear the OUT-OF-BUNDLE Condition 347

OUT-OF-SYNC 348

Clear the OUT-OF-SYNC Condition 348

OVER-TEMP-UNIT-PROT 349

Clearing the OVER-TEMP-UNIT-PROT Alarm 349

PARAM-MISM 350

PATCH-ACTIVATION-FAILED 350

PATCH-DOWNLOAD-FAILED 351

PAYLOAD-UNKNOWN 351

Clear the PAYLOAD-UNKNOWN Alarm 351

PDI-P 352

Clear the PDI-P Condition 353

PEER-CERT-VERIFICATION-FAILED 354

Clear the PEER-CERT-VERIFICATION-FAILED Alarm 354

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxv

Contents

PEER-CSF 354

Clear the PEER-CSF Alarm 354

PEER-NORESPONSE 355

Clear the PEER-NORESPONSE Alarm 355

PMD-DEG 355

Clear the PMD-DEG Alarm 356

PMI 356

Clear the PMI Condition 356

PORT-COMM-FAIL 357

Clear the PORT-COMM-FAIL Alarm 357

PORT-FAIL 358

Clear the PORT-FAIL Alarm 358

PPR-BDI 359

Clear the PPR-BDI Condition 359

PPR-FDI 360

Clear the PPR-FDI Condition 360

PPR-MAINT 360

PPR-TRIG-EXCD 360

Clear the PPR-TRIG-EXCD Condition 361

PRBS-ENABLED 361

Clear the PRBS-ENABLED Alarm 362

PROT-SOFT-VERIF-FAIL 362

Clear the PROT-SOFT-VERIF-FAIL Alarm 362

PROTNA 363

Clear the PROTNA Alarm 363

PROV-MISMATCH 363

Clear the PROV-MISMATCH Alarm 364

PTIM 366

Clear the PTIM Alarm 366

PWR-CON-LMT 367

xxvi

Clear the PWR-CON-LMT Alarm 367

PWR-FAIL-A 367

Clear the PWR-FAIL-A Alarm 368

PWR-FAIL-B 369

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Clear the PWR-FAIL-B Alarm 369

PWR-FAIL-RET-A 369

Clear the PWR-FAIL-RET-A Alarm 369

PWR-FAIL-RET-B 370

Clear the PWR-FAIL-RET-A Alarm 370

PWR-PROT-ON 370

Clear the PWR-PROT-ON Alarm 371

RAMAN-CALIBRATION-FAILED 371

Clear the RAMAN-CALIBRATION-FAILED Alarm 371

RAMAN-CALIBRATION-PENDING 372

RAMAN-CALIBRATION-RUNNING 372

RAMAN-G-NOT-REACHED 372

Clear the RAMAN-G-NOT-REACHED Alarm 373

Contents

REMOTE-FAULT 373

Clear the REMOTE-FAULT Alarm 373

REP-LINK-FLAPPING 374

Clear the REP-LINK-FLAPPING 374

REP-NEIHB-ADJ-FAIL 374

Clear the REP-NEIHB-ADJ-FAIL Alarm 375

REP-SEGMENT-FAULT 375

Clear the REP-SEGMENT-FAULT Condition 375

REROUTE-IN-PROG 376

Clear the REROUTE-IN-PROG Alarm 376

REVERT-IN-PROG 376

Clear the REVERT-IN-PROG Alarm 376

RFI 376

Clear the RFI Condition 377

RFI-L 377

Clear the RFI-L Condition 377

RFI-P 377

Clear the RFI-P Condition 378

RLS 378

Clear the RLS Condition 379

ROUTE-OVERFLOW 380

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxvii

Contents

Clear the ROUTE-OVERFLOW Condition 380

RS-EOC 380

Clear the RS-EOC Alarm 381

RS-TIM 383

Clear the RS-TIM Alarm 383

SBYTCC-NEINTCLK 384

Clear the SBYTCC-NEINTCLK Alarm 384

SD (TRUNK) 385

Clear the SD (TRUNK) Condition 385

SD-L 386

Clear the SD-L Condition 386

SD-L (TRUNK) 387

Clear the SD-L (TRUNK) Condition 387

SD-P 387

Clear the SD-P Condition 388

SDBER-EXCEED-HO 388

Clear the SDBER-EXCEED-HO Condition 389

SEQ-MISMATCH-COUNT 389

Clearing the SEQ-MISMATCH-COUNT Alarm 389

SF (TRUNK) 390

Clear the SF (TRUNK) Condition 390

SF-L 390

Clear the SF-L Condition 391

SF-L (TRUNK) 391

Clear the SF-L (TRUNK) Condition 391

SF-P 392

Clear the SF-P Condition 392

SFTWDOWN 392

SFTWDOWN-FAIL 393

Clear the SFTWDOWN-FAIL Alarm 393

xxviii

SHELF-COMM-FAIL 394

Clear the SHELF-COMM-FAIL Alarm 394

SH-IL-VAR-DEG-HIGH 394

Clear the SH-IL-VAR-DEG-HIGH Alarm 395

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

SH-IL-VAR-DEG-LOW 395

Clear the SH-IL-VAR-DEG-LOW Alarm 395

SHUTTER-OPEN 395

Clear the SHUTTER-OPEN Condition 396

SIGLOSS 396

Clear the SIGLOSS Alarm 396

SNTP-HOST 397

Clear the SNTP-HOST Alarm 397

SOFT-VERIF-FAIL 398

Clear the SOFT-VERIF-FAIL Alarm 398

SPANLEN-OUT-OF-RANGE 398

Clear the SPANLEN-OUT-OF-RANGE Alarm 399

SPAN-NOT-MEASURED 399

Contents

SQUELCHED 400

Clear the SQUELCHED Condition 401

SSM-DUS 402

SSM-FAIL 402

Clear the SSM-FAIL Alarm 402

SSM-LNC 403

SSM-OFF 403

Clear the SSM-OFF Condition 403

SSM-PRC 404

SSM-PRS 404

SSM-RES 404

SSM-SMC 404

SSM-ST2 405

SSM-ST3 405

SSM-ST3E 405

SSM-ST4 406

SSM-STU 406

Clear the SSM-STU Condition 406

SSM-TNC 407

SW-MISMATCH 407

Clear the SW-MISMATCH Condition 407

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxix

Contents

SWTOPRI 407

SWTOSEC 408

Clear the SWTOSEC Condition 408

SWTOTHIRD 408

Clear the SWTOTHIRD Condition 408

SYNC-FREQ 409

Clear the SYNC-FREQ Condition 409

SYNCLOSS 410

Clear the SYNCLOSS Alarm 410

SYNCPRI 410

Clear the SYNCPRI Alarm 411

SYNCSEC 411

Clear the SYNCSEC Alarm 412

SYNCTHIRD 412

Clear the SYNCTHIRD Alarm 412

SYSBOOT 413

TEMP-LIC 413

Clear the TEMP-LIC Alarm 414

TEMP-MISM 414

Clear the TEMP-MISM Condition 414

TIM 415

Clear the TIM Alarm 415

TIM-MON 416

Clear the TIM-MON Alarm 416

TIM-P 416

Clear the TIM-P Alarm 417

TIM-S 417

Clear the TIM-S Alarm 418

TRAF-AFFECT-RESET-REQUIRED 418

Clear the TRAF-AFFECT-RESET-REQUIRED Alarm 418

xxx

TRAIL-SIGNAL-FAIL 419

Clear the TRAIL-SIGNAL-FAIL Condition 419

TRUNK-ODU-AIS 419

Clear the TRUNK-ODU-AIS Condition 419

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

TRAIL-SIGNAL-FAIL 420

Clear the TRAIL-SIGNAL-FAIL Condition 420

TRUNK-OPU-CSF 420

Clear the TRUNK-OPU-CSF Condition 420

TRUNK-PAYLOAD-MISM 421

Clear the TRUNK-PAYLOAD-MISM Alarm 421

TX-OFF-NON-CISCO-PPM 422

Clear the TX-OFF-NON-CISCO-PPM Condition 422

UNC-WORD 422

Clear the UNC-WORD Condition 422

UNEQ-P 423

Clear the UNEQ-P Alarm 424

UNIT-HIGH-TEMP 425

Contents

Clearing the UNIT-HIGH-TEMP Alarm 426

UNQUAL-PPM 427

Clear the UNQUAL-PPM Condition 427

UNREACHABLE-TARGET-POWER 427

USBSYNC 428

Clear the USB-SYNC Alarm 428

USB-MOUNT-FAIL Alarm 428

Clearing the USB-MOUNT-FAIL Alarm 428

USB PORTS DOWN 429

Clear the USB PORTS DOWN Alarm 429

USB-WRITE-FAIL 429

Clear the USB-WRITE-FAIL Alarm 429

UT-COMM-FAIL 430

Clear the UT-COMM-FAIL Alarm 430

UT-FAIL 430

Clear the UT-FAIL Alarm 430

VOA-DISABLED 431

Clear the VOA-DISABLED Condition 431

VOA-HDEG 431

Clear the VOA-HDEG Alarm 432

VOA-HFAIL 432

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxxi

Contents

Clear the VOA-HFAIL Alarm 432

VOA-LMDEG 432

Clear the VOA-LDEG Alarm 433

VOA-LFAIL 433

Clear the VOA-LFAIL Alarm 433

VOLT-MISM 434

Clear the VOLT-MISM Condition 434

WAITING-TO-START 434

WAN-SYNCLOSS 434

Clear the WAN-SYNCLOSS Condition 435

WKSWPR (2R, EQPT, ESCON, FC, GE, ISC, OTS) 435

WKSWPR (TRUNK) 435

WRK-PATH-RECOVERY-CHECK 436

Clear the WRK-PATH-RECOVERY-CHECK Alarm 436

Wait to Restore Condition 436

WTR (TRUNK) 437

WVL-DRIFT-CHAN-OFF 437

Clear the WVL-DRIFT-CHAN-OFF Condition 437

WVL-MISMATCH 438

Clear the WVL-MISMATCH alarm 438

WVL-UNLOCKED Alarm 439

DWDM Card LED Activity 439

DWDM Card LED Activity After Insertion 439

DWDM Card LED Activity During Reset 440

Traffic Card LED Activity 440

Typical Traffic Card LED Activity After Insertion 440

Typical Traffic Card LED Activity During Reset 440

Typical Card LED State After Successful Reset 441

Frequently Used Alarm Troubleshooting Procedures 441

Node and Ring Identification, Change, Visibility, and Termination 441

xxxii

Identify a BLSR Ring Name or Node ID Number 441

Change a BLSR Ring Name 442

Change a BLSR Node ID Number 442

Verify Node Visibility for Other Nodes 443

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Protection Switching, Lock Initiation, and Clearing 443

Initiate a 1+1 Protection Port Force Switch Command 443

Initiate a 1+1 Manual Switch Command 444

Initiate a 1:1 Card Switch Command 445

Clear a 1+1 Force or Manual Switch Command 445

Initiate a Lock-On Command 446

Initiate a Card or Port Lockout Command 447

Clear a Lock-On or Lockout Command 447

Initiate a Lockout on a BLSR Protect Span 448

Clear a BLSR External Switching Command 448

CTC Card Resetting and Switching 449

Reset a Card in CTC 449

Reset an Active Control Card and Activate the Standby Card 450

Contents

Physical Card Reseating, Resetting, and Replacement 451

Remove and Reinsert (Reseat) the Standby Control Card 451

Remove and Reinsert (Reseat) Any Card 452

Physically Replace a Card 453

Generic Signal and Circuit Procedures 454

Verify the Signal BER Threshold Level 454

Delete a Circuit 454

Verify or Create Node Section DCC Terminations 455

Clear an MXP, TXP, GE-XP, 10GE-XP, and ADM-10G Card Loopback Circuit 455

Verify or Create Node RS-DCC Terminations 456

Clear an STM-N Card XC Loopback Circuit 457

Air Filter and Fan Procedures 457

Inspect, Clean, and Replace the Reusable Air Filter 457

Remove and Reinsert a Fan-Tray Assembly 460

Replace the Fan-Tray Assembly 460

Interface Procedures 461

Replace the Alarm Interface Panel 461

CHAPTER 3

Transient Conditions 465

Transients Indexed By Alphabetical Entry 465

Trouble Notifications 468

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxxiii

Contents

Condition Characteristics 468

Condition States 468

Transient Conditions 469

ADMIN-DISABLE 469

ADMIN-DISABLE-CLR 469

ADMIN-LOCKOUT 469

ADMIN-LOCKOUT-CLR 469

ADMIN-LOGOUT 469

ADMIN-SUSPEND 469

ADMIN-SUSPEND-CLR 470

AUD-ARCHIVE-FAIL 470

AUTOWDMANS 470

BLSR-RESYNC 470

DBBACKUP-FAIL 470

DBRESTORE-FAIL 470

EXERCISING-RING 471

EXERCISING-SPAN 471

FIREWALL-DIS 471

FIRMWARE-DOWNLOAD 471

FIRMWARE-UPG 471

FIRMWARE-UPG-COMPLETE 471

FIRMWARE-UPG-FAIL 471

FRCDWKSWBK-NO-TRFSW 472

FRCDWKSWPR-NO-TRFSW 472

INC-BOOTCODE 472

INTRUSION 472

INTRUSION-PSWD 472

IOSCFG-COPY-FAIL 472

LOGIN-FAIL-LOCKOUT 473

LOGIN-FAIL-ONALRDY 473

xxxiv

LOGIN-FAILURE-PSWD 473

LOGIN-FAILURE-USERID 473

LOGOUT-IDLE-USER 473

MASTERKEY-SUCCESS 473

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

MANWKSWBK-NO-TRFSW 474

MANWKSWPR-NO-TRFSW 474

MCAST-MAC-ALIASING 474

MSSP-RESYNC 474

PM-TCA 474

PS 474

REP-PRI-EDGE-ELECTED 474

REP-SEC-EDGE-ELECTED 474

REP-STCN-GENERATED 475

REP-VLB-ACTIVATED 475

REP-VLB-TRIG-DELAY 475

RESTORE-IN-PROG 475

RMON-ALARM 475

Contents

CHAPTER 4

RMON-RESET 475

SESSION-TIME-LIMIT 475

SFTWDOWN-FAIL 475

SPAN-NOT-MEASURED 476

SWFTDOWNFAIL 476

USER-LOCKOUT 476

USER-LOGIN 476

USER-LOGOUT 476

WKSWBK 476

WKSWPR 476

WRMRESTART 477

WTR-SPAN 477

Error Messages 479

Error Messages Reference 479

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxxv

Contents

xxxvi

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Preface

Note

The terms "Unidirectional Path Switched Ring" and "UPSR" may appear in Cisco literature. These terms do
not refer to using Cisco ONS 15xxx products in a unidirectional path switched ring configuration. Rather,
these terms, as well as "Path Protected Mesh Network" and "PPMN," refer generally to Cisco's path protection
feature, which may be used in any topological network configuration. Cisco does not recommend using its
path protection feature in any particular topological network configuration.

This section explains the objectives, intended audience, and organization of this publication and describes the
conventions that convey instructions and other information.

This section provides the following information:

• Document Objectives, on page xxxvii

• Audience, on page xxxvii

• Document Organization, on page xxxviii

• Document Conventions, on page xxxviii

• Related Documentation, on page xliv

• Obtaining Optical Networking Information, on page xlv

• Communications, Services, and Additional Information, on page xlv

Document Objectives

This guide gives troubleshooting information and troubleshooting-related parameters for the Cisco NCS 2002
and Cisco NCS 2006 platforms, specifically the dense wavelength division multiplexing (DWDM) application
that can operate on either platform. Use this guide in conjunction with the appropriate publications listed in
the Related Documentation, on page xliv section.

Audience

To use this publication, you should be familiar with Cisco or equivalent optical transmission hardware and
cabling, telecommunications hardware and cabling, electronic circuitry and wiring practices, and preferably
have experience as a telecommunications technician.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxxvii

Document Organization

Document Organization

This document is organized into the following chapters:

Preface

DescriptionChapter

General Troubleshooting, on page
1

Alarm Troubleshooting, on page 99

Transient Conditions, on page 465

Document Conventions

This document uses the following conventions:

^ or Ctrl

Provides procedures for troubleshooting the most common problems
encountered when operating on Cisco NCS 2000 Seriesplatforms.

Gives a description, severity, and troubleshooting procedure for each
commonly encountered Cisco DWDM alarm and condition.

Gives a description, entity, Simple Network Management Protocol
(SNMP) number, and trap for each commonly encountered transient
condition.

Lists the error messages.Error Messages, on page 479

DescriptionConvention

Both the ^ symbol and Ctrl represent the Control (Ctrl) key on a keyboard.
For example, the key combination ^D or Ctrl-D means that you hold
down the Control key while you press the D key. (Keys are indicated in
capital letters but are not case sensitive.)

Commands and keywords and user-entered text appear in bold font.bold font

xxxviii

Italic font

Courier font

...

|

[x | y]

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Document titles, new or emphasized terms, and arguments for which you
supply values are in italic font.

Terminal sessions and information the system displays appear in courier
font.

Bold Courier font indicates text that the user must enter.Bold Courier font

Elements in square brackets are optional.[x]

An ellipsis (three consecutive nonbolded periods without spaces) after a
syntax element indicates that the element can be repeated.

A vertical line, called a pipe, indicates a choice within a set of keywords
or arguments.

Optional alternative keywords are grouped in brackets and separated by
vertical bars.

Preface

Preface

DescriptionConvention

{x | y}

Required alternative keywords are grouped in braces and separated by
vertical bars.

[x {y | z}]

Nested set of square brackets or braces indicate optional or required
choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.

string

A nonquoted set of characters. Do not use quotation marks around the
string or the string will include the quotation marks.

Nonprinting characters such as passwords are in angle brackets.< >

Default responses to system prompts are in square brackets.[ ]

!, #

An exclamation point (!) or a pound sign (#) at the beginning of a line of
code indicates a comment line.

Reader Alert Conventions

This document uses the following conventions for reader alerts:

Note

Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual.

Tip

Caution

Timesaver

Warning

Means the following information will help you solve a problem.

Means reader be careful. In this situation, you might do something that could result in equipment damage or
loss of data.

Means the described action saves time. You can save time by performing the action described in the paragraph.

Means reader be warned. In this situation, you might perform an action that could result in bodily injury.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xxxix

Preface

Preface

IMPORTANT SAFETY INSTRUCTIONS

This warning symbol means danger. You are in a situation that could
cause bodily injury. Before you work on any equipment, be aware of the
hazards involved with electrical circuitry and be familiar with standard
practices for preventing accidents. Use the statement number provided
at the end of each warning to locate its translation in the translated safety
warnings that accompanied this device. Statement 1071

SAVE THESE INSTRUCTIONS

Waarschuwing

Varoitus

Attention

BELANGRIJKE VEILIGHEIDSINSTRUCTIES

Dit waarschuwingssymbool betekent gevaar. U verkeert in een situatie
die lichamelijk letsel kan veroorzaken. Voordat u aan enige apparatuur
gaat werken, dient u zich bewust te zijn van de bij elektrische schakelingen
betrokken risico's en dient u op de hoogte te zijn van de standaard
praktijken om ongelukken te voorkomen. Gebruik het nummer van de
verklaring onderaan de waarschuwing als u een vertaling van de
waarschuwing die bij het apparaat wordt geleverd, wilt raadplegen.

BEWAAR DEZE INSTRUCTIES

TÄRKEITÄ TURVALLISUUSOHJEITA

Tämä varoitusmerkkimerkitsee vaaraa. Tilanne voi aiheuttaa ruumiillisia
vammoja. Ennen kuin käsittelet laitteistoa, huomioi sähköpiirien
käsittelemiseen liittyvät riskit ja tutustu onnettomuuksien yleisiin
ehkäisytapoihin. Turvallisuusvaroitusten käännökset löytyvät laitteen
mukana toimitettujen käännettyjen turvallisuusvaroitusten joukosta
varoitusten lopussa näkyvien lausuntonumeroiden avulla.

SÄILYTÄ NÄMÄ OHJEET

IMPORTANTES INFORMATIONS DE SÉCURITÉ

Ce symbole d'avertissement indique un danger. Vous vous trouvez dans
une situation pouvant entraîner des blessures ou des dommages corporels.
Avant de travailler sur un équipement, soyez conscient des dangers liés
aux circuits électriques et familiarisez-vous avec les procédures
couramment utilisées pour éviter les accidents. Pour prendreconnaissance
des traductions des avertissements figurant dans les consignes de sécurité
traduites qui accompagnent cet appareil, référez-vous au numéro de
l'instruction situé à la fin de chaque avertissement.

CONSERVEZ CES INFORMATIONS

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xl

Preface

Preface

Warnung

Avvertenza

Advarsel

WICHTIGE SICHERHEITSHINWEISE

Dieses Warnsymbol bedeutet Gefahr. Sie befinden sich in einer Situation,
die zu Verletzungen führen kann. Machen Sie sich vor der Arbeit mit
Geräten mit den Gefahren elektrischer Schaltungen und den üblichen
Verfahren zur Vorbeugung vor Unfällen vertraut. Suchen Sie mit der am
Ende jeder Warnung angegebenen Anweisungsnummer nach der
jeweiligen Übersetzung in den übersetzten Sicherheitshinweisen, die
zusammen mit diesem Gerät ausgeliefert wurden.

BEWAHREN SIE DIESE HINWEISE GUT AUF.

IMPORTANTI ISTRUZIONI SULLA SICUREZZA

Questo simbolo di avvertenza indica un pericolo. La situazione potrebbe
causare infortuni alle persone. Prima di intervenire su qualsiasi
apparecchiatura, occorre essere al corrente dei pericoli relativi ai circuiti
elettrici e conoscere le procedurestandard per la prevenzione di incidenti.
Utilizzare il numero di istruzione presente alla fine di ciascuna avvertenza
per individuare le traduzioni delle avvertenze riportate in questo
documento.

CONSERVARE QUESTE ISTRUZIONI

VIKTIGE SIKKERHETSINSTRUKSJONER

Dette advarselssymbolet betyr fare. Du er i en situasjon som kan føre til
skade på person. Før du begynner å arbeide med noe av utstyret, må du
være oppmerksom på farene forbundet med elektriske kretser, og kjenne
til standardprosedyrer for å forhindre ulykker. Bruk nummeret i slutten
av hver advarsel for å finne oversettelsen i de oversatte
sikkerhetsadvarslene som fulgte med denne enheten.

Aviso

¡Advertencia!

TA VARE PÅ DISSE INSTRUKSJONENE

INSTRUÇÕES IMPORTANTES DE SEGURANÇA

Este símbolo de aviso significa perigo. Você está em uma situação que
poderá ser causadora de lesões corporais. Antes de iniciar a utilização de
qualquer equipamento, tenha conhecimento dos perigos envolvidos no
manuseio de circuitos elétricos e familiarize-se com as práticas habituais
de prevenção de acidentes. Utilize o número da instrução fornecido ao
final de cada aviso para localizar sua tradução nos avisos de segurança
traduzidos que acompanham este dispositivo.

GUARDE ESTAS INSTRUÇÕES

INSTRUCCIONES IMPORTANTES DE SEGURIDAD

Este símbolo de aviso indica peligro. Existe riesgo para su integridad
física. Antes de manipular cualquier equipo, considere los riesgos de la
corriente eléctrica y familiarícese con los procedimientos estándar de
prevención de accidentes. Al final de cada advertencia encontrará el
número que le ayudará a encontrar el texto traducido en el apartado de
traducciones que acompaña a este dispositivo.

GUARDE ESTAS INSTRUCCIONES

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xli

Preface

Preface

Varning!

VIKTIGA SÄKERHETSANVISNINGAR

Denna varningssignal signalerar fara. Du befinner dig i en situation som
kan leda till personskada. Innan du utför arbete på någon utrustning
måste du vara medveten om farorna med elkretsar och känna till vanliga
förfaranden för att förebygga olyckor. Använd det nummer som finns i
slutet av varje varning för att hitta dess översättning i de översatta
säkerhetsvarningar som medföljer denna anordning.

SPARA DESSA ANVISNINGAR

xlii

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

Preface

Preface

Aviso

Advarsel

INSTRUÇÕES IMPORTANTES DE SEGURANÇA

Este símbolo de aviso significa perigo. Você se encontra em uma situação
em que há risco de lesões corporais. Antes de trabalhar com qualquer
equipamento, esteja ciente dos riscos que envolvem os circuitos elétricos
e familiarize-se com as práticas padrão de prevenção de acidentes. Use o
número da declaração fornecido ao final de cada aviso para localizar sua
tradução nos avisos de segurança traduzidos que acompanham o
dispositivo.

GUARDE ESTAS INSTRUÇÕES

VIGTIGE SIKKERHEDSANVISNINGER

Dette advarselssymbol betyder fare. Du befinder dig i en situation med
risiko for legemesbeskadigelse. Før du begynder arbejde på udstyr, skal
du være opmærksom på de involverede risici, der er ved elektriske
kredsløb, og du skal sætte dig ind i standardprocedurer til undgåelse af
ulykker. Brug erklæringsnummeret efter hver advarsel for at finde
oversættelsen i de oversatte advarsler, der fulgte med denne enhed.

GEM DISSE ANVISNINGER

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xliii

Related Documentation

Preface

Related Documentation

Use this guide in conjunction with the following referenced Release 10.x publications:

• Release Notes for Cisco NCS 2000 Series, Release 10.x

• Cisco NCS 2000 Series Line Card Configuration Guide

• Cisco NCS 2000 Series Network Configuration Guide

• Cisco NCS 2000 Series Licensing Configuration Guide

• Cisco NCS 2000 Series Hardware Installation Guide

• Cisco NCS 2000 Series TL1 Command Guide

• Cisco Transport Planner DWDM Operations Guide

• Regulatory Compliance and Safety Information for Cisco NCS Platforms

• Electrostatic Discharge and Grounding Guide for Cisco NCS Platforms

• Installing the GBIC, SFP, SFP+, XFP, CXP, and CFP Optical Modules in Cisco NCS Platforms

• Installing the Cisco NCS 2000 Series Passive Optical Modules

For an update on End-of-Life and End-of-Sale notices, refer to

http://www.cisco.com/en/US/products/hw/optical/ps2006/prod_eol_notices_list.html

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xliv

Preface

Obtaining Optical Networking Information

Obtaining Optical Networking Information

This section contains information that is specific to optical networking products. For information that pertains
to all of Cisco, refer to the Obtaining Documentation and Submitting a Service Request section.

Where to Find Safety and Warning Information

For safety and warning information, refer to the Regulatory Compliance and Safety Information for Cisco
ONS Products document that accompanied the product. This publication describes the international agency
compliance and safety information for the Cisco ONS Products. It also includes translations of the safety
warnings that appear in the ONS system documentation.

Communications, Services, and Additional Information

• To receive timely, relevant information from Cisco, sign up at Cisco Profile Manager.

• To get the business impact you’re looking for with the technologies that matter, visit Cisco Services.

• To submit a service request, visit Cisco Support.

• To discover and browse secure, validated enterprise-class apps, products, solutions and services, visit

Cisco Marketplace.

• To obtain general networking, training, and certification titles, visit Cisco Press.

• To find warranty information for a specific product or product family, access Cisco Warranty Finder.

Cisco Bug Search Tool

Cisco Bug Search Tool (BST) is a web-based tool that acts as a gateway to the Cisco bug tracking system

that maintains a comprehensive list of defects and vulnerabilities in Cisco products and software. BST provides
you with detailed defect information about your products and software.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

xlv

Communications, Services, and Additional Information

Preface

xlvi

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

CHAPTER 1

General Troubleshooting

This chapter provides procedures for troubleshooting the most common problems encountered when operating
a DWDM shelf in ANSI or ETSI platforms. To troubleshoot specific alarms, see Alarm Troubleshooting, on

page 99 If you cannot find what you are looking for, contact Cisco Technical Support (1 800 553-2447).

Alarms can occur even in those cards that are not explicitly mentioned in the Alarm sections. When an alarm
is raised, refer to its clearing procedure.

This chapter includes the following sections on network problems:

• Loopback Description, on page 1

• Troubleshooting MXP, TXP, XP, or ADM-10G Circuit Paths With Loopbacks, on page 7

• Troubleshooting DWDM Circuit Paths With ITU-T G.709 Monitoring, on page 27

• Using CTC Diagnostics, on page 33

• Onboard Failure Logging, on page 37

• Restoring the Database and Default Settings, on page 40

• PC Connectivity Troubleshooting, on page 40

• CTC Operation Troubleshooting, on page 47

• Timing, on page 57

• Fiber and Cabling, on page 59

• Power Supply Problems, on page 63

• Power Up Problems for Node and Cards, on page 65

• Network Level (Internode) Problems, on page 65

• Node Level (Intranode) Problems, on page 80

Loopback Description

To create a loopback on an ANSI or SONET port, the port must be in the Out-of-Service and Management,
Maintenance (OOS-MA,MT) service state. After you create the loopback, the service state becomes
Out-of-Service and Management, Loopback and Maintenance (OOS-MA,LPBK & MT).

To create a loopback on an a port, the port must be in the Locked, maintenance administrative state and the
Locked-Enabled, loopback & maintenance administrative state.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

1

Facility Loopbacks

General Troubleshooting

Caution

Facility or terminal loopbacks can be service-affecting. To protect traffic, apply a lockout or Force switch to
the target loopback port. Basic directions for these procedures exist in Alarm Troubleshooting, on page 99
chapter.

Note

In CTC, a facility loopback is sometimes called facility (line) loopback, and a terminal loopback is sometimes
called a terminal (inward) loopback. This is done to indicate the terminating direction of the signal: a facility
loopback is sent outward toward the span, whereas a terminal loopback is redirected inward toward its
originating port.

Facility Loopbacks

The following sections give general information about facility loopback operations and specific information
about card loopback activity.

General Behavior

A facility loopback tests the line interface unit (LIU) of a card, the electrical interface assembly (EIA), and
related cabling. After applying a facility loopback on a port, use a test set to run traffic over the loopback. A
successful facility loopback isolates the LIU, the EIA, or the cabling plant as the potential cause of a network
problem.

To test a card LIU, connect an optical test set to a trunk or client port and perform a facility loopback.
Alternately, use a loopback or hairpin circuit on a card that is farther along the circuit path. For example,

Figure 1: Facility Loopback Path on a Near-End Transponder Card, on page 3 shows a facility loopback at

a trunk port and at a client port on a TXP card.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

2

General Troubleshooting

Card Behavior

Figure 1: Facility Loopback Path on a Near-End Transponder Card

Caution

Caution

Card Behavior

Note

Before performing a facility loopback on a TXP card, be sure that the card contains at least two data
communications channel (DCC) paths to the node where the card is installed. A second DCC provides a
nonlooped path to log into the node after the loopback is applied, enabling you to remove the facility loopback.
Ensuring a second DCC is not necessary if you are directly connected to the node containing the loopback
card.

Ensure that the facility being loopbacked is not being used by the node for line timing. If it is, a timing loop
will be created.

Port loopbacks either terminate or bridge the loopback signal. All MXP and TXP facility loopbacks are
terminated as shown in the following table.

When a port terminates a facility loopback signal, the signal only loops back to the originating port and is not
transmitted downstream. When a port bridges a loopback signal, the signal loops back to the originating port
and is also transmitted downstream.

In the following table, no alarm indication signal (AIS) is injected if the signal is bridged. If the signal is
terminated, an applicable AIS is injected downstream.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

3

Card Behavior

General Troubleshooting

Table 1: DWDM Card Facility Loopback Behavior

Facility Loopback SignalCard/Port

BridgedTXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L client ports

TerminatedTXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L trunk ports

TerminatedTXP_MR_2.5G/TXPP_MR_2.5G client ports

TerminatedTXP_MR_2.5G/TXPP_MR_2.5G trunk ports

BridgedMXP_2.5G_10E_C/MXP_2.5G_10E_L client ports

TerminatedMXP_2.5G_10E_C/MXP_2.5G_10E_L trunk ports

TerminatedMXP_MR_10DME client ports

TerminatedMXP_MR_10DME trunk ports

BridgedMXP_MR_2.5G/MXPP_MR_2.5G client ports

TerminatedMXP_MR_2.5G/MXPP_MR_2.5G trunk ports

TerminatedGE_XP/10GE_XP client ports

TerminatedGE_XP/10GE_XP trunk ports

BridgedADM-10G client ports

TerminatedADM-10G trunk ports

Bridged40G-MXP-C/40E-MXP-C/40ME-MXP-C client ports

Bridged40G-MXP-C/40E-MXP-C/40ME-MXP-C trunk ports

Bridged40E-TXP-C/40ME-TXP-C client ports

Bridged40E-TXP-C/40ME-TXP-C trunk ports

TerminatedAR_XP/AR_MXP client ports

TerminatedAR_XP/AR_MXP trunk ports

The loopback itself is listed in the Conditions window. For example, the window would list the
LPBKFACILITY condition for a tested port. (The Alarms window would show the AS-MT condition which
means that alarms are suppressed on the facility during loopback unless the default is set to alarm for loopback
while in AS-MT.)

With a client-side SONET or ANSI facility loopback, the client port service state is OOS-MA,LPBK & MT.
However, any remaining client and trunk ports can be in any other service state. For SONET or ANSI cards
in a trunk-side facility loopback, the trunk port service state is OOS-MA,LPBK & MT and the remaining
client and trunk ports can be in any other service state.

With a client-side SDH or ESTI facility loopback, the client port is in the Locked-enabled,maintenance &
loopback service state. However, the remaining client and trunk ports can be in any other service state. For
MXP and TXP cards in a SDH or ETSI trunk-side facility loopback, the trunk port is in the

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

4

General Troubleshooting

Locked-enabled,maintenance & loopback service state and the remaining client and trunk ports can be in any
other service state.

When you apply a facility loopback on the GE_XP, 10GE_XP, GE_XPE, and 10GE_XPE cards, the ifInDiscard
counters increment continuously.

Terminal Loopbacks

The following sections give general information about terminal loopback operations and specific information
about card loopback activity.

General Behavior

A terminal loopback tests a circuit path as it passes through a TXP, MXP, or ADM-10G card and loops back.
For example, as shown in Figure 2: Terminal Loopback on a TXP Card, on page 5, there are two types of
terminal loopbacks shown for a TXP card.

The first is a terminal loopback at the client port. In this situation, the test set traffic comes in through the
TXP trunk port, travels through the card, and turns around because of the terminal loopback in effect on the
card just before it reaches the LIU of the client port. The signal is then sent back through the card to the trunk
port and back to the test set.

Terminal Loopbacks

The second is a terminal loopback at the trunk port. In this situation, the test set traffic comes in through the
TXP client port, travels through the card, and turns around because of the terminal loopback in effect on the
card just before it reaches the LIU of the trunk port. The signal is then sent back through the card to the client
port and back to the test set.

This test verifies that the terminal circuit paths are valid, but does not test the LIU on the TXP card.

Figure 2: Terminal Loopback on a TXP Card

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

5

Card Behavior

Card Behavior

Note

General Troubleshooting

The SDH terminal port loopbacks can either terminate or bridge the signal. TXP terminal loopbacks are
terminated as shown in the following table. During terminal loopbacks, if a port terminates a terminal loopback
signal, the signal only loops back to the originating port and is not transmitted downstream. If the port bridges
a loopback signal, the signal loops back to the originating port and is also transmitted downstream. Client
card terminal loopback bridging and terminating behaviors are listed in the following table.

AIS signal is not injected if the signal is bridged. If the signal is terminated, an applicable AIS is injected
downstream.

Table 2: DWDM Card Terminal Loopback Behavior

Terminal Loopback SignalCard/Port

BridgedTXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L client ports

BridgedTXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L trunk ports

BridgedTXP_MR_2.5G/TXPP_MR_2.5G client ports

BridgedTXP_MR_2.5G/TXPP_MR_2.5G trunk ports

BridgedMXP_2.5G_10E_C/MXP_2.5G_10E_L client ports

BridgedMXP_2.5G_10E_C/MXP_2.5G_10E_L trunk ports

BridgedMXP_MR_10DME client ports

BridgedMXP_MR_10DME trunk ports

BridgedMXP_MR_2.5G/MXPP_MR_2.5G client ports

BridgedMXP_MR_2.5G/MXPP_MR_2.5G trunk ports

BridgedGE_XP/10GE_XP client ports

BridgedGE_XP/10GE_XP trunk ports

BridgedADM-10G client ports

BridgedADM-10G trunk ports

Bridged40G-MXP-C/40E-MXP-C/40ME-MXP-C client ports

Bridged40G-MXP-C/40E-MXP-C/40ME-MXP-C trunk ports

Bridged40E-TXP-C/40ME-TXP-C client ports

Bridged40E-TXP-C/40ME-TXP-C trunk ports

BridgedAR_XP/AR_MXP client ports

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

6

General Troubleshooting

Troubleshooting MXP, TXP, XP, or ADM-10G Circuit Paths With Loopbacks

Terminal Loopback SignalCard/Port

BridgedAR_XP/AR_MXP trunk ports

Important notes about loopback on MXP and TXP trunk and client ports:

• For SONET or ANSI TXP and TXPP cards with a client-side terminal loopback, the client port is in the
OOS-MA,LPBK & MT service state and trunk port must be in IS-NR service state.

• For SONET or ANSI MXP and MXPP cards with a client-side terminal loopback, the client port is in
the OOS-MA,LPBK & MT service state and the remaining client and trunk ports can be in any service
state.

• For ADM-10G cards with Client Terminal Loopback on a SONET Client port, AIS-P is sent forward on
client for the circuits on that port.

• For ADM-10G cards with a Terminal Loopback on a GE Client port, the client port is squelched.

• In SONET or ANSI MXP or TXP trunk-side terminal loopbacks, the trunk port is in the OOS-MA,LPBK
& MT service state and the client ports must be in IS-NR service state for complete loopback functionality.
A terminal loopback affects all client ports because it is performed on the aggregate signal.

• For ADM-10G cards with a Facility Loopback on the Trunk port, AIS-P is sent forward on all the SONET
client ports.

• For ADM-10G cards with a Facility Loopback on the Trunk port, all the GE client ports is squelched

• For ADM-10G Terminal Loopback on the Trunk port, the signal is anyway sent downstream (drop and
continue).

• For SDH or ETSI TXP and TXPP client-side facility loopbacks, the client port is in the
Locked-enabled,maintenance & loopback service state and the trunk port must be in Unlocked-enabled
service state.

• For SDH or ETSI MXP and MXPP cards with a client-side terminal loopback, the client port is in the
Locked-enabled,maintenance & loopback service state and remaining client and trunk ports can be in
any service state.

• In SDH and ETSI MXP or TXP trunk-side terminal loopbacks, the trunk port is in the
Locked-enabled,maintenance & loopback service state and the client ports must be in Unlocked-enabled
service state for complete loopback functionality. A facility loopback affects all client ports because it
is performed on the aggregate signal.

The loopback itself is listed in the Conditions window. For example, the window would list the
LPBKTERMINAL condition or LPBKFACILITY condition for a tested port. (The Alarms window would
show the AS-MT condition, which indicates that all alarms are suppressed on the port during loopback testing
unless the default is set to alarm for loopback while in AS-MT.)

Troubleshooting MXP, TXP, XP, or ADM-10G Circuit Paths With
Loopbacks

Facility loopbacks and terminal loopbacks are often used together to test the circuit path through the network
or to logically isolate a fault. Performing a loopback test at each point along the circuit path systematically

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

7

Perform a Facility Loopback on a Source-Node MXP or TXP Port

isolates possible points of failure. MXP, TXP, XP, or ADM-10G card loopback tests differ from other testing
in that loopback testing does not require circuit creation. MXP, TXP, and XP client ports are statically mapped
to the trunk ports so no signal needs to traverse the cross-connect card (in a circuit) to test the loopback.

You can use these procedures on transponder cards (TXP, TXPP, ADM-10G), muxponder, or xponder cards
(MXP, MXPP, XP, ADM-10G) cards. The example in this section tests an MXP or TXP circuit on a three-node
bidirectional line switched ring (BLSR) or multiplex section-shared protection ring (MS-SPRing). Using a
series of facility loopbacks and terminal loopbacks, the example scenario traces the circuit path, tests the
possible failure points, and eliminates them. The logical progression contains six network test procedures:

Note

MXP, TXP, XP, or ADM-10G card client ports do not appear when you click the Maintenance > Loopback
tab unless they have been provisioned. Do this in the card view by clicking the Provisioning > Pluggable
Port Modules tab.

Note

The test sequence for your circuits will differ according to the type of circuit and network topology.

General Troubleshooting

1. A facility loopback on the source-node MXP, TXP, XP, or ADM-10G port

2. A terminal loopback on the source-node MXP, TXP, XP, or ADM-10G port

3. A facility loopback on the intermediate-node MXP, TXP, XP, or ADM-10G port

4. A terminal loopback on the intermediate-node MXP, TXP, XP, or ADM-10G port

5. A facility loopback on the destination-node MXP, TXP, XP, or ADM-10G port

6. A terminal loopback on the destination-node MXP, TXP, XP, or ADM-10G port

Note

Facility and terminal loopback tests require on-site personnel.

Perform a Facility Loopback on a Source-Node MXP or TXP Port

This facility loopback test is performed on the node source port in the network circuit. In the testing situation
used in this example, the source muxponder, transponder, xponder, or ADM-10G port under test is located
in the source node. Facility loopback can be performed at the trunk port or at a client port. Completing a
successful facility loopback on this port isolates the source MXP, TXP, XP, or ADM-10G port as a possible
failure point. Figure 3: Facility Loopback on a Circuit Source MXP or TXP Port, on page 9 shows the facility
loopback examples on source ONS node TXP ports (client and trunk).

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

8

General Troubleshooting

Create the Facility Loopback on the Source-Node MXP, TXP, XP or ADM-10G Port

Figure 3: Facility Loopback on a Circuit Source MXP or TXP Port

Caution

Performing a loopback on an in-service circuit is service-affecting.

Note

Facility loopbacks require on-site personnel.

Complete the Create the Facility Loopback on the Source-Node MXP, TXP, XP or ADM-10G Port, on page

9.

Create the Facility Loopback on the Source-Node MXP, TXP, XP or ADM-10G Port

SUMMARY STEPS

1. Connect an optical test set to the port you are testing.

2. Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

3. In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card to display the

card view.

4. Click the Maintenance > Loopback tabs.

5. Choose OOS,MT (or locked,maintenance) from the Admin State column for the port being tested. If

this is a multiport card, select the appropriate row for the desired port.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

9

Test and Clear the MXP, TXP, XP or ADM-10G Facility Loopback Circuit

6. Choose Facility (Line) from the Loopback Type column for the port being tested. If this is a multiport

card, select the appropriate row for the desired port.

7. Click Apply.

8. Click Yes in the confirmation dialog box.

9. Complete the Test and Clear the MXP, TXP, XP or ADM-10G Facility Loopback Circuit, on page 10.

DETAILED STEPS

Step 1 Connect an optical test set to the port you are testing.

General Troubleshooting

Note

For specific procedures to connect, set up, and use the test set equipment, consult the manufacturer.

Use appropriate cabling to attach the transmit (Tx) and receive (Rx) terminals of the optical test set to the port you are
testing. The Tx and Rx terminals connect to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

Step 3 In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card to display the card view.
Step 4 Click the Maintenance > Loopback tabs.
Step 5 Choose OOS,MT (or locked,maintenance) from the Admin State column for the port being tested. If this is a multiport

card, select the appropriate row for the desired port.

Step 6 Choose Facility (Line) from the Loopback Type column for the port being tested. If this is a multiport card, select the

appropriate row for the desired port.

Step 7 Click Apply.
Step 8 Click Yes in the confirmation dialog box.

Note

It is normal for the LPBKFACILITY (ESCON) , on page 292, LPBKFACILITY (FC) , on page 293,

LPBKFACILITY (GE) , on page 293, LPBKFACILITY (ISC) , on page 294, or the LPBKFACILITY (TRUNK)
, on page 295 to appear during loopback setup. The condition clears when you remove the loopback.

Step 9 Complete the Test and Clear the MXP, TXP, XP or ADM-10G Facility Loopback Circuit, on page 10.

Test and Clear the MXP, TXP, XP or ADM-10G Facility Loopback Circuit

SUMMARY STEPS

1. If the test set is not already sending traffic, send test traffic on the loopback circuit.

2. Examine the traffic received by the test set. Look for errors or any other signal information that the test

set is capable of indicating.

3. If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the facility

loopback:

4. If the test set indicates errors, complete the Test the MXP, TXP, XP or ADM-10G Card, on page 11.

DETAILED STEPS

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

10

General Troubleshooting

Test the MXP, TXP, XP or ADM-10G Card

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of

indicating.

Step 3 If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the facility loopback:

a) Click the Maintenance > Loopback tabs.
b) Choose None from the Loopback Type column for the port being tested.
c) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

d) Click Apply.
e) Click Yes in the confirmation dialog box.
f) Complete the Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port, on page 12.

Step 4 If the test set indicates errors, complete the Test the MXP, TXP, XP or ADM-10G Card, on page 11.

Test the MXP, TXP, XP or ADM-10G Card

SUMMARY STEPS

1. Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a

known-good one.

2. Resend test traffic on the loopback circuit with a known-good card installed.

3. If the test set indicates no errors, the problem was probably the defective card. Return the defective card

to Cisco through the Return Materials Authorization (RMA) process. Contact Cisco Technical Support 1
800 553 2447.

4. Clear the facility loopback:

5. Complete the Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port, on

page 12.

DETAILED STEPS

Step 1 Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a known-good one.

Warning

High-performance devices on this card can get hot during operation. To remove the card, hold it by the
faceplate and bottom edge. Allow the card to cool before touching any other part of it or before placing
it in an antistatic bag. Statement 201

Caution

Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform
an external switch if a switch has not already occurred. Refer to the procedures in the Protection Switching,

Lock Initiation, and Clearing, on page 443. For more information, refer to the Maintain the Node chapter in the

Cisco NCS 2002 and NCS 2006 Network Operations Guide.

Step 2 Resend test traffic on the loopback circuit with a known-good card installed.

Step 3 If the test set indicates no errors, the problem was probably the defective card. Return the defective card to Cisco through

the Return Materials Authorization (RMA) process. Contact Cisco Technical Support 1 800 553 2447.

Step 4 Clear the facility loopback:

a) Click the Maintenance > Loopback tabs.
b) Choose None from the Loopback Type column for the port being tested.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

11

General Troubleshooting

Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port

c) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

d) Click Apply.
e) Click Yes in the confirmation dialog box.

Step 5 Complete the Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port, on page 12.

Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G
Port

The terminal loopback test is performed on the node source MXP, TXP, XP, or ADM-10G port. For the circuit
in this example, it is the source TXP trunk port or a client port in the source node. Completing a successful
terminal loopback to a node source port verifies that the circuit is through the source port. Figure 4: Terminal

Loopback on a Source-Node MXP or TXP Port, on page 12 shows an example of a terminal loopback on a

source TXP port and a client TXP port.

Figure 4: Terminal Loopback on a Source-Node MXP or TXP Port

12

Caution

Performing a loopback on an in-service circuit is service-affecting.

Note

Terminal loopbacks require on-site personnel.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

General Troubleshooting

Create the Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port

Complete the Create the Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port, on page

13.

Create the Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port

SUMMARY STEPS

1. Connect an optical test set to the port you are testing:

2. Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

3. In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card that requires the

loopback.

4. Click the Maintenance > Loopback tabs.

5. Select OOS,MT(or locked,maintenance) from the Admin State column. If this is a multiport card, select

the row appropriate for the desired port.

6. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row

appropriate for the desired port.

7. Click Apply.

8. Click Yes in the confirmation dialog box.

9. Complete the Test and Clear the MXP, TXP, XP, or ADM-10G Port Terminal Loopback Circuit, on page

14.

DETAILED STEPS

Step 1 Connect an optical test set to the port you are testing:

Note

For specific procedures to connect, set up, and use the test set equipment, consult the manufacturer.

a) If you just completed the Perform a Facility Loopback on a Source-Node MXP or TXP Port, on page 8, leave the

optical test set hooked up to the MXP, TXP, XP, or ADM-10G port in the source node.

b) If you are starting the current procedure without the optical test set hooked up to the source port, use appropriate

cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect
to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

Step 3 In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card that requires the loopback.
Step 4 Click the Maintenance > Loopback tabs.
Step 5 Select OOS,MT (or locked,maintenance) from the Admin State column. If this is a multiport card, select the row

appropriate for the desired port.

Step 6 Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the

desired port.

Step 7 Click Apply.
Step 8 Click Yes in the confirmation dialog box.

Step 9 Complete the Test and Clear the MXP, TXP, XP, or ADM-10G Port Terminal Loopback Circuit, on page 14.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

13

General Troubleshooting

Test and Clear the MXP, TXP, XP, or ADM-10G Port Terminal Loopback Circuit

Test and Clear the MXP, TXP, XP, or ADM-10G Port Terminal Loopback Circuit

SUMMARY STEPS

1. If the test set is not already sending traffic, send test traffic on the loopback circuit.

2. Examine the test traffic being received by the test set. Look for errors or any other signal information that

the test set is capable of indicating.

3. If the test set indicates no errors, no further testing is necessary on the loopback circuit. Clear the terminal

loopback state on the port:

4. If the test set indicates errors, complete the Test the MXP, TXP, XP, or ADM-10G Card, on page 14.

DETAILED STEPS

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is

capable of indicating.

Step 3 If the test set indicates no errors, no further testing is necessary on the loopback circuit. Clear the terminal loopback state

on the port:

a) Double-click the card in the source node with the terminal loopback.
b) Click the Maintenance > Loopback tabs.
c) Select None from the Loopback Type column for the port being tested.
d) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

e) Click Apply.
f) Click Yes in the confirmation dialog box.
g) Complete the Create a Facility Loopback on an Intermediate-Node MXP or TXP Port, on page 15.

Step 4 If the test set indicates errors, complete the Test the MXP, TXP, XP, or ADM-10G Card, on page 14.

Test the MXP, TXP, XP, or ADM-10G Card

SUMMARY STEPS

1. Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a

known-good one.

2. Resend test traffic on the loopback circuit with a known-good card.

3. If the test set indicates no errors, the problem was probably the defective card. Return the defective card

to Cisco through the RMA process. Contact Cisco Technical Support 1 800 553 2447.

4. Clear the terminal loopback on the port before testing the next segment of the network circuit path:

5. Complete the Create a Facility Loopback on an Intermediate-Node MXP or TXP Port, on page 15.

DETAILED STEPS

Step 1 Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a known-good one.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

14

General Troubleshooting

Create a Facility Loopback on an Intermediate-Node MXP or TXP Port

Warning

High-performance devices on this card can get hot during operation. To remove the card, hold it by the
faceplate and bottom edge. Allow the card to cool before touching any other part of it or before placing
it in an antistatic bag. Statement 201

Caution

Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform
an external switch if a switch has not already occurred. Refer to the procedures in the Protection Switching,

Lock Initiation, and Clearing, on page 443.

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates no errors, the problem was probably the defective card. Return the defective card to Cisco through

the RMA process. Contact Cisco Technical Support 1 800 553 2447.

Step 4 Clear the terminal loopback on the port before testing the next segment of the network circuit path:

a) Double-click the card in the source node with the terminal loopback.
b) Click the Maintenance > Loopback tabs.
c) Select None from the Loopback Type column for the port being tested.
d) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

e) Click Apply.
f) Click Yes in the confirmation dialog box.

Step 5 Complete the Create a Facility Loopback on an Intermediate-Node MXP or TXP Port, on page 15.

Create a Facility Loopback on an Intermediate-Node MXP or TXP Port

Performing the facility loopback test on an intermediate port isolates whether this node is causing circuit
failure. In the situation shown in Figure 5: Facility Loopback on an Intermediate-Node MXP or TXP Port,

on page 15, the test is being performed on an intermediate MXP or TXP port.

Figure 5: Facility Loopback on an Intermediate-Node MXP or TXP Port

Caution

Performing a loopback on an in-service circuit is service-affecting.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

15

Create a Facility Loopback on an Intermediate-Node MXP or TXP Port

Note

Facility loopbacks require on-site personnel.

Complete the Create a Facility Loopback on an Intermediate-Node MXP or TXP Port, on page 16.

Create a Facility Loopback on an Intermediate-Node MXP or TXP Port

SUMMARY STEPS

1. Connect an optical test set to the port you are testing:

2. Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

3. In node view (single-shelf mode) or shelf view (multishelf mode), double-click the intermediate-node

card that requires the loopback.

4. Click the Maintenance > Loopback tabs.

5. Select OOS,MT(or locked,maintenance) from the Admin State column. If this is a multiport card, select

the row appropriate for the desired port.

6. Select Facility(Line) from the Loopback Type column. If this is a multiport card, select the row appropriate

for the desired port.

7. Click Apply.

8. Click Yes in the confirmation dialog box.

9. Complete the Test and Clear the MXP or TXP Port Facility Loopback Circuit, on page 17.

General Troubleshooting

DETAILED STEPS

Step 1 Connect an optical test set to the port you are testing:

Note

For specific procedures to connect, set up, and use the test set equipment, consult the manufacturer.

a) If you just completed the Perform a Terminal Loopback on a Source-Node MXP, TXP, XP, or ADM-10G Port, on

page 12, leave the optical test set hooked up to the source-node port.

b) If you are starting the current procedure without the optical test set hooked up to the source port port, use appropriate

cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect
to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

Step 3 In node view (single-shelf mode) or shelf view (multishelf mode), double-click the intermediate-node card that requires

the loopback.

Step 4 Click the Maintenance > Loopback tabs.
Step 5 Select OOS,MT (or locked,maintenance) from the Admin State column. If this is a multiport card, select the row

appropriate for the desired port.

Step 6 Select Facility (Line) from the Loopback Type column. If this is a multiport card, select the row appropriate for the

desired port.

Step 7 Click Apply.
Step 8 Click Yes in the confirmation dialog box.

Step 9 Complete the Test and Clear the MXP or TXP Port Facility Loopback Circuit, on page 17.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

16

General Troubleshooting

Test and Clear the MXP or TXP Port Facility Loopback Circuit

Test and Clear the MXP or TXP Port Facility Loopback Circuit

SUMMARY STEPS

1. If the test set is not already sending traffic, send test traffic on the loopback circuit.

2. Examine the traffic received by the test set. Look for errors or any other signal information that the test

set is capable of indicating.

3. If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the facility

loopback from the port:

4. If the test set indicates errors, complete the Test the MXP or TXP Card, on page 17.

DETAILED STEPS

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of

indicating.

Step 3 If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the facility loopback

from the port:
a) Click the Maintenance > Loopback tabs.

b) Choose None from the Loopback Type column for the port being tested.
c) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

d) Click Apply.
e) Click Yes in the confirmation dialog box.
f) Complete the Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports, on page 18.

Step 4 If the test set indicates errors, complete the Test the MXP or TXP Card, on page 17.

Test the MXP or TXP Card

SUMMARY STEPS

1. Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a

known-good one.

2. Resend test traffic on the loopback circuit with a known-good card installed.

3. If the test set indicates no errors, the problem was probably the defective card. Return the defective card

to Cisco through the RMA process. Contact Cisco Technical Support 1 800 553 2447.

4. Clear the facility loopback from the port:

5. Complete the Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports, on page 18.

DETAILED STEPS

Step 1 Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a known-good one.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

17

Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports

General Troubleshooting

Warning

High-performance devices on this card can get hot during operation. To remove the card, hold it by the
faceplate and bottom edge. Allow the card to cool before touching any other part of it or before placing
it in an antistatic bag. Statement 201

Caution

Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform
an external switch if a switch has not already occurred. Refer to the procedures in the Protection Switching,

Lock Initiation, and Clearing, on page 443.

Step 2 Resend test traffic on the loopback circuit with a known-good card installed.

Step 3 If the test set indicates no errors, the problem was probably the defective card. Return the defective card to Cisco through

the RMA process. Contact Cisco Technical Support 1 800 553 2447.

Step 4 Clear the facility loopback from the port:

a) Click the Maintenance > Loopback tabs.
b) Choose None from the Loopback Type column for the port being tested.
c) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

d) Click Apply.
e) Click Yes in the confirmation dialog box.

Step 5 Complete the Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports, on page 18.

Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports

In the next troubleshooting test, you perform a terminal loopback on the intermediate-node port to isolate
whether the intermediate client or trunk port is causing circuit trouble. In the example situation in Figure 6:

Terminal Loopback on an Intermediate-Node MXP or TXP Port, on page 18, the terminal loopback is

performed on an intermediate MXP or TXP port in the circuit. If you successfully complete a terminal loopback
on the node, this node is excluded from possible sources of circuit trouble.

Figure 6: Terminal Loopback on an Intermediate-Node MXP or TXP Port

Caution

Performing a loopback on an in-service circuit is service-affecting.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

18

General Troubleshooting

Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports

Note

Terminal loopbacks require on-site personnel.

Complete the Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports, on page 19.

Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports

SUMMARY STEPS

1. Connect an optical test set to the port you are testing:

2. Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

3. Create the terminal loopback on the destination port being tested:

4. Complete the Test and Clear the MXP or TXP Terminal Loopback Circuit, on page 20.

DETAILED STEPS

Step 1 Connect an optical test set to the port you are testing:

Note

For specific procedures to connect, set up, and use the test set equipment, consult the manufacturer.

a) If you just completed the Create a Facility Loopback on an Intermediate-Node MXP or TXP Port, on page 15, leave

the optical test set hooked up to the source-node port.

b) If you are starting the current procedure without the optical test set hooked up to the source port, use appropriate

cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect
to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

Step 3 Create the terminal loopback on the destination port being tested:

a) Go to node view (single-shelf mode) or shelf view (multishelf mode) of the intermediate node:

• Choose View > Go To Other Node from the menu bar.

• Choose the node (or shelf) from the drop-down list in the Select Node dialog box and click OK.

b) In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card that requires the loopback.
c) Click the Maintenance > Loopback tabs.
d) Select OOS,MT (or locked,maintenance) from the Admin State column. If this is a multiport card, select the row

appropriate for the desired port.

e) Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate

for the desired port.

f) Click Apply.
g) Click Yes in the confirmation dialog box.

Step 4 Complete the Test and Clear the MXP or TXP Terminal Loopback Circuit, on page 20.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

19

General Troubleshooting

Test and Clear the MXP or TXP Terminal Loopback Circuit

Test and Clear the MXP or TXP Terminal Loopback Circuit

SUMMARY STEPS

1. If the test set is not already sending traffic, send test traffic on the loopback circuit.

2. Examine the test traffic being received by the test set. Look for errors or any other signal information that

the test set is capable of indicating.

3. If the test set indicates no errors, no further testing is necessary on the loopback circuit. Clear the terminal

loopback from the port:

4. If the test set indicates errors, complete the Test the MXP or TXP Card, on page 20.

DETAILED STEPS

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is

capable of indicating.

Step 3 If the test set indicates no errors, no further testing is necessary on the loopback circuit. Clear the terminal loopback from

the port:

a) Double-click the intermediate-node card with the terminal loopback to display the card view.
b) Click the Maintenance > Loopback tabs.
c) Select None from the Loopback Type column for the port being tested.
d) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

e) Click Apply.
f) Click Yes in the confirmation dialog box.
g) Complete the Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port, on page 21.

Step 4 If the test set indicates errors, complete the Test the MXP or TXP Card, on page 20.

Test the MXP or TXP Card

SUMMARY STEPS

1. Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a

known-good one.

2. Resend test traffic on the loopback circuit with a known-good card.

3. If the test set indicates no errors, the problem was probably the defective card. Return the defective card

to Cisco through the RMA process. Contact Cisco Technical Support 1 800 553 2447.

4. Clear the terminal loopback on the port:

5. Complete the Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port,

on page 21.

DETAILED STEPS

Step 1 Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a known-good one.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

20

General Troubleshooting

Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port

Warning

Caution

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates no errors, the problem was probably the defective card. Return the defective card to Cisco through

the RMA process. Contact Cisco Technical Support 1 800 553 2447.

Step 4 Clear the terminal loopback on the port:

a) Double-click the source-node card with the terminal loopback.
b) Click the Maintenance > Loopback tabs.
c) Select None from the Loopback Type column for the port being tested.
d) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

e) Click Apply.
f) Click Yes in the confirmation dialog box.

Step 5 Complete the Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port, on page 21.

High-performance devices on this card can get hot during operation. To remove the card, hold it by the
faceplate and bottom edge. Allow the card to cool before touching any other part of it or before placing
it in an antistatic bag. Statement 201

Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform
an external switch if a switch has not already occurred. Refer to the procedures in the Protection Switching,

Lock Initiation, and Clearing, on page 443.

or automatically in service from the Admin State column for the port being tested.

Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G
Port

You perform a facility loopback test at the destination port to determine whether this local port is the source
of circuit trouble. The example in Figure 7: Facility Loopback on a Destination-Node MXP or TXP Port, on

page 21 shows a facility loopback being performed on a TXP client or trunk port at a destination node.

Figure 7: Facility Loopback on a Destination-Node MXP or TXP Port

Caution

Performing a loopback on an in-service circuit is service-affecting.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

21

General Troubleshooting

Create the Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port

Note

Facility loopbacks require on-site personnel.

Complete the Create the Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port, on

page 22.

Create the Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port

SUMMARY STEPS

1. Connect an optical test set to the port you are testing:

2. Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

3. Create the facility loopback on the destination port being tested:

4. Complete the Test and Clear the MXP, TXP, XP, or ADM-10G Facility Loopback Circuit, on page 23.

DETAILED STEPS

Step 1 Connect an optical test set to the port you are testing:

Note

For specific procedures to connect, set up, and use the test set equipment, consult the manufacturer.

a) If you just completed the Create a Terminal Loopback on Intermediate-Node MXP or TXP Ports, on page 18, leave

the optical test set hooked up to the source-node port.

b) If you are starting the current procedure without the optical test set hooked up to the source port, use appropriate

cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect
to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

Step 3 Create the facility loopback on the destination port being tested:

a) Go to the node view (single-shelf mode) or shelf view (multishelf mode) of the destination node:

• Choose View > Go To Other Node from the menu bar.

• Choose the node (or shelf) from the drop-down list in the Select Node dialog box and click OK.

b) In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card that requires the loopback.
c) Click the Maintenance > Loopback tabs.
d) Select OOS,MT (or locked,maintenance) from the Admin State column. If this is a multiport card, select the row

appropriate for the desired port.

e) Select Facility (Line) from the Loopback Type column. If this is a multiport card, select the row appropriate for the

desired port.

f) Click Apply.
g) Click Yes in the confirmation dialog box.

Step 4 Complete the Test and Clear the MXP, TXP, XP, or ADM-10G Facility Loopback Circuit, on page 23.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

22

General Troubleshooting

Test and Clear the MXP, TXP, XP, or ADM-10G Facility Loopback Circuit

Test and Clear the MXP, TXP, XP, or ADM-10G Facility Loopback Circuit

SUMMARY STEPS

1. If the test set is not already sending traffic, send test traffic on the loopback circuit.

2. Examine the traffic received by the test set. Look for errors or any other signal information that the test

set is capable of indicating.

3. If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the facility

loopback from the port:

4. If the test set indicates errors, complete the Test the MXP, TXP, XP, or ADM-10G Card, on page 23.

DETAILED STEPS

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of

indicating.

Step 3 If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the facility loopback

from the port:
a) Click the Maintenance > Loopback tabs.

b) Choose None from the Loopback Type column for the port being tested.
c) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

d) Click Apply.
e) Click Yes in the confirmation dialog box.
f) Complete the Perform a Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port, on page 24.

Step 4 If the test set indicates errors, complete the Test the MXP, TXP, XP, or ADM-10G Card, on page 23.

Test the MXP, TXP, XP, or ADM-10G Card

SUMMARY STEPS

1. Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a

known-good one.

2. Resend test traffic on the loopback circuit with a known-good card installed.

3. If the test set indicates no errors, the problem was probably the defective card. Return the defective card

to Cisco through the RMA process. Contact Cisco Technical Support 1 800 553 2447.

4. Clear the facility loopback on the port:

5. Complete the Perform a Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port,

on page 24.

DETAILED STEPS

Step 1 Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a known-good one.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

23

Perform a Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port

General Troubleshooting

Warning

Caution

Step 2 Resend test traffic on the loopback circuit with a known-good card installed.

Step 3 If the test set indicates no errors, the problem was probably the defective card. Return the defective card to Cisco through

the RMA process. Contact Cisco Technical Support 1 800 553 2447.

Step 4 Clear the facility loopback on the port:

a) Click the Maintenance > Loopback tabs.
b) Choose None from the Loopback Type column for the port being tested.
c) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

d) Click Apply.
e) Click Yes in the confirmation dialog box.

Step 5 Complete the Perform a Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port, on page 24.

High-performance devices on this card can get hot during operation. To remove the card, hold it by the
faceplate and bottom edge. Allow the card to cool before touching any other part of it or before placing
it in an antistatic bag. Statement 201

Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform
an external switch if a switch has not already occurred. Refer to the procedures in the Protection Switching,

Lock Initiation, and Clearing, on page 443.

or automatically in service from the Admin State column for the port being tested.

Perform a Terminal Loopback on a Destination-Node MXP, TXP, XP, or
ADM-10G Port

The terminal loopback at the destination-node port is the final local hardware error elimination in the circuit
troubleshooting process. If this test is completed successfully, you have verified that the circuit is good up to
the destination port. The example in Figure 8: Terminal Loopback on a Destination-Node MXP or TXP Port,

on page 24 shows a terminal loopback on an destination node TXP port.

Figure 8: Terminal Loopback on a Destination-Node MXP or TXP Port

24

Caution

Performing a loopback on an in-service circuit is service-affecting.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

General Troubleshooting

Create the Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port

Note

Terminal loopbacks require on-site personnel.

Complete the Create the Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port, on

page 25.

Create the Terminal Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port

SUMMARY STEPS

1. Connect an optical test set to the port you are testing:

2. Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

3. Create the terminal loopback on the destination port being tested:

4. Complete the Test and Clear the MXP, TXP, XP, or ADM-10G Terminal Loopback Circuit, on page 26.

DETAILED STEPS

Step 1 Connect an optical test set to the port you are testing:

Note

For specific procedures to connect, set up, and use the test set equipment, consult the manufacturer.

a) If you just completed the Perform a Facility Loopback on a Destination-Node MXP, TXP, XP, or ADM-10G Port,

on page 21, leave the optical test set hooked up to the source port.

b) If you are starting the current procedure without the optical test set hooked up to the source port, use appropriate

cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect
to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

Note

It is normal for the LPBKFACILITY (ESCON) , on page 292, LPBKFACILITY (FC) , on page 293,

LPBKFACILITY (GE) , on page 293, LPBKFACILITY (ISC) , on page 294, or the LPBKFACILITY (TRUNK)
, on page 295 to appear during loopback setup. The condition clears when you remove the loopback.

Step 3 Create the terminal loopback on the destination port being tested:

a) Go to the node view (single-shelf mode) or shelf view (multishelf mode) of the destination node:

• Choose View > Go To Other Node from the menu bar.

• Choose the node (or shelf) from the drop-down list in the Select Node dialog box and click OK.

b) In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card that requires the loopback.
c) Click the Maintenance > Loopback tabs.
d) Select OOS,MT (or locked,maintenance) from the Admin State column. If this is a multiport card, select the row

appropriate for the desired port.

e) Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for

the desired port.

f) Click Apply.
g) Click Yes in the confirmation dialog box.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

25

General Troubleshooting

Test and Clear the MXP, TXP, XP, or ADM-10G Terminal Loopback Circuit

Step 4 Complete the Test and Clear the MXP, TXP, XP, or ADM-10G Terminal Loopback Circuit, on page 26.

Test and Clear the MXP, TXP, XP, or ADM-10G Terminal Loopback Circuit

SUMMARY STEPS

1. If the test set is not already sending traffic, send test traffic on the loopback circuit.

2. Examine the test traffic being received by the test set. Look for errors or any other signal information that

the test set is capable of indicating.

3. If the test set indicates no errors, no further testing is necessary on the loopback circuit. Clear the terminal

loopback from the port:

4. If the test set indicates errors, the problem might be a faulty card.

5. Complete the Test the MXP, TXP, XP, or ADM-10G Card, on page 26.

DETAILED STEPS

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is

capable of indicating.

Step 3 If the test set indicates no errors, no further testing is necessary on the loopback circuit. Clear the terminal loopback from

the port:

a) Double-click the intermediate-node card with the terminal loopback.
b) Click the Maintenance > Loopback tabs.
c) Select None from the Loopback Type column for the port being tested.
d) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

or automatically in service from the Admin State column for the port being tested.

e) Click Apply.
f) Click Yes in the confirmation dialog box.

Step 4 If the test set indicates errors, the problem might be a faulty card.

Step 5 Complete the Test the MXP, TXP, XP, or ADM-10G Card, on page 26.

Test the MXP, TXP, XP, or ADM-10G Card

SUMMARY STEPS

1. Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a

known-good one.

2. Resend test traffic on the loopback circuit with a known-good card.

3. If the test set indicates no errors the problem was probably the defective card. Return the defective card

to Cisco through the RMA process. Contact Cisco Technical Support 1 800 553 2447.

4. Clear the terminal loopback on the port:

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

26

General Troubleshooting

Troubleshooting DWDM Circuit Paths With ITU-T G.709 Monitoring

DETAILED STEPS

Step 1 Complete the Physically Replace a Card, on page 453 for the suspected bad card and replace it with a known-good one.

Warning

Caution

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates no errors the problem was probably the defective card. Return the defective card to Cisco through

the RMA process. Contact Cisco Technical Support 1 800 553 2447.

Step 4 Clear the terminal loopback on the port:

a) Double-click the source-node card with the terminal loopback.
b) Click the Maintenance > Loopback tabs.
c) Select None from the Loopback Type column for the port being tested.
d) Choose the appropriate state to place the port in service, out of service and disabled, out of service for maintenance,

e) Click Apply.
f) Click Yes in the confirmation dialog box.

High-performance devices on this card can get hot during operation. To remove the card, hold it by the
faceplate and bottom edge. Allow the card to cool before touching any other part of it or before placing
it in an antistatic bag. Statement 201

Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform
an external switch if a switch has not already occurred. Refer to the procedures in the Protection Switching,

Lock Initiation, and Clearing, on page 443.

or automatically in service from the Admin State column for the port being tested.

The entire circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live
traffic.

Troubleshooting DWDM Circuit Paths With ITU-T G.709
Monitoring

This section provides an overview of the optical transport network (OTN) specified in ITU-T G.709,Network
Node Interface for the Optical Transport Network , and provides troubleshooting procedures for DWDM

circuit paths in the ITU-T G.709 OTN using PM and TCAs.

ITU-T G.709 Monitoring in Optical Transport Networks

ITU-T Recommendation G.709 is part of a suite of recommendations covering the full functionality of an
OTN. ITU-T G.709 enables single-wavelength SONET transparent optical wavelength-based networks. ITU-T
G.709 adds the Operation, Administration, Maintenance, and Provisioning (OAM&P) functionality of
SONET/SDH to DWDM optical networks. It adds extra overhead to existing SONET, Ethernet, or asynchronous
transfer mode (ATM) bit streams for performance management and improvement.

Like traditional SONET networks, ITU-T G.709 optical networks have a layered design (Figure 9: Optical

Transport Network Layers, on page 28). This structure enables localized monitoring that helps you isolate

and troubleshoot network problems.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

27

Optical Channel Layer

Figure 9: Optical Transport Network Layers

Optical Channel Layer

The optical channel (OCH) layer is the outermost part of the OTN and spans from client to client. The optical
channel is built as follows:

General Troubleshooting

A client signal such as SONET, Gigabit Ethernet, IP, ATM, Fibre Channel, or enterprise system connection
(ESCON) is mapped to a client payload area and combined with an overhead to create the optical channel
payload unit (OPUk).

A second overhead is added to the OPUk unit to create the optical channel data unit (ODUk).

A third overhead including forward error correction (FEC) is added to the ODUk to create the optical channel
transport unit (OTUk).

A fourth overhead is added to the OTUk to create the entire OCH layer.

Optical Multiplex Section Layer

The optical multiplex section (OMS) of the OTN allows carriers to identify errors occurring within DWDM
network sections. The OMS layer consists of a payload and an overhead (OMS-OH). It supports the ability
to monitor multiplexed sections of the network, for example, the span between an optical multiplexer such as
the 32MUX-O card and an optical demultiplexer such as the 32DMX-O card.

Optical Transmission Section Layer

The optical transmission section (OTS) layer supports monitoring partial spans of a network multiplexed
sections. This layer consists of a payload and an overhead (OTS-OH). It is a transmission span between two
elements in an optical network, such as between:

• A multiplexer such as the 32MUX-O card and an amplifier such as the OPT-PRE card

• An amplifier and another amplifier, such as the OPT-BST card and the OPT-PRE card

• An amplifier such as the OPT-BST card and a demultiplexer such as the 32DMX card

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

28

General Troubleshooting

Performance Monitoring Counters and Threshold Crossing Alerts

Performance Monitoring Counters and Threshold Crossing Alerts

PM counters and TCAs can be used for identifying trouble and troubleshooting problems in ITU-T G.709
optical transport networks. ITU-T Recommendation M.2401 recommends that the following PM parameters
be monitored at the ODUk layer:

• SES (severely errored seconds)A one-second period that contains greater than or equal to 30 percent
errored blocks or at least one defect. SES is a subset of the errored second (ES) parameter, which is a
one-second period with one or more errored blocks or at least one defect.

• BBE (background block error counter)An errored block not occurring as part of an SES. BBE is a subset
of the errored block (EB) parameter, which is a block in which one or more bits are in error.

Different PM count parameters are associated with different read points in a network. Figure 10: Performance

Monitoring Points on ONS DWDM, on page 29 illustrates the PM read points that are useful in identifying

DWDM circuit points of failure. The Monitor Performance document lists all PM parameters and provides
block diagrams of signal entry points, exit points, and interconnections between the individual circuit cards.
Consult these specifications to determine which PM parameters are associated with the system points you
want to monitor or provision with CTC or TL1. The monitoring points might vary according to your
configuration.

Note

When LOS, LOS-P, or LOF alarms occur on TXP and MXP trunks, G709/SONET/SDH TCAs are suppressed.
For details, see the .

Figure 10: Performance Monitoring Points on ONS DWDM

TCAs are used to monitor performance through the management interface by indicating whether preset
thresholds have been crossed, or whether a transmission (such as a laser transmission) is degraded. TCAs are
not associated with severity levels. They are usually associated with rate, counter, and percentage parameters
that are available at transponder monitoring points. The Monitor Performance document contains more
information about these alerts.

Select and complete the following procedures according to your network parameters.

Set Node Default BBE or SES Card Thresholds

Complete the following procedure to provision default node ODUk BBE and SES PM thresholds for TXP
cards.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

29

General Troubleshooting

Provision Individual Card BBE or SES Thresholds in CTC

SUMMARY STEPS

1. In node view (single-shelf mode) or multishelf view (multishelf mode), click the Provisioning > Defaults

tabs.

2. In the Defaults Selector field, click the card you wish to provision, then click Opticalthresholds > Trunk
> Warning > 15min in the drop-down list.

DETAILED STEPS

Step 1 In node view (single-shelf mode) or multishelf view (multishelf mode), click the Provisioning > Defaults tabs.
Step 2 In the Defaults Selector field, click the card you wish to provision, then click Opticalthresholds > Trunk > Warning

> 15min in the drop-down list.

Provision Individual Card BBE or SES Thresholds in CTC

Complete the following procedure to provision BBE or SES PM thresholds in CTC for an individual TXP
card.

SUMMARY STEPS

1. In node view (single-shelf mode) or shelf view (multishelf mode), double-click the applicable card (TXP,
MXP, or XP.)

2. Click the Provisioning > OTN > G.709 Thresholds tabs.

3. In the Directions area, click the Near End radio button.

4. In the Intervals area, click the 15 Min radio button.

5. In the Types area, click the PM (ODUk) radio button.

6. In the SES and BBE fields, enter threshold numbers, for example 500 and 10000.

DETAILED STEPS

Step 1 In node view (single-shelf mode) or shelf view (multishelf mode), double-click the applicable card (TXP, MXP, or XP.)
Step 2 Click the Provisioning > OTN > G.709 Thresholds tabs.
Step 3 In the Directions area, click the Near End radio button.
Step 4 In the Intervals area, click the 15 Min radio button.
Step 5 In the Types area, click the PM (ODUk) radio button.

Step 6 In the SES and BBE fields, enter threshold numbers, for example 500 and 10000.

Provision Card PM Thresholds Using TL1

SUMMARY STEPS

30

Complete the following procedure if you wish to provision PM thresholds in TL1 rather than in CTC.

1. Open a TL1 command line (click Tools > Open TL1 Connection).

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

General Troubleshooting

2. In the TL1 command line, enter a command using the following syntax:

DETAILED STEPS

Step 1 Open a TL1 command line (click Tools > Open TL1 Connection).

Step 2 In the TL1 command line, enter a command using the following syntax:

SET-TH-OCH:[<TID>]:<AID>:<CTAG>::<MONTYPE>,<THLEV>,[<LOCN>],,[<TMPER>];

where:

• Access Identifier (AID) identifies the NE to which the command pertains. All the OCH, STS, VT1, facility, and
DS1 AIDs are supported.

• The parameter MONTYPE is the monitored type.

• The parameter THLEV is optional and indicates a threshold count value (the number of errors that must be exceeded
before the threshold is crossed).

• The parameter LOCN specifies the location associated with the particular command.

Provision Optical TCA Thresholds

• The parameter TMPER is optional and is an accumulation time period for performance counters, with possible values
of 1-DAY, 1-HR, 1-MIN, 15-MIN, and RAW-DATA.

Note

For a more information about this command and a list of TL1 commands, refer to the Cisco ONS SONET
TL1 Command Guide and Cisco ONS SDH TL1 Command Guide .

Provision Optical TCA Thresholds

Complete the following procedure to provision TCA thresholds in CTC.

SUMMARY STEPS

1. In card view, click the Provisioning > Optics Thresholds tabs.

2. In the Types area, click TCA.

3. In the Intervals area, click 15 Min.

4. In the Laser Bias High (%) field, enter the threshold value, for example, 81.0 percent.

DETAILED STEPS

Step 1 In card view, click the Provisioning > Optics Thresholds tabs.
Step 2 In the Types area, click TCA.
Step 3 In the Intervals area, click 15 Min.

Step 4 In the Laser Bias High (%) field, enter the threshold value, for example, 81.0 percent.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

31

Forward Error Correction

Forward Error Correction

In DWDM spans, FEC reduces the quantities of retiming, reshaping, and regeneration (3R) needed to maintain
signal quality. The following two PM parameters are associated with FEC:

• BIT-EC: Bit errors corrected (BIT-EC ) indicates the number of bit errors corrected in the DWDM trunk
line during the PM time interval.

• UNC-WORDSThe number of uncorrectable words detected in the DWDM trunk line during the PM
time interval.

Complete the following procedure to provision BIT-EC and UNC-WORDS PM parameters for FEC.

Provision Card FEC Thresholds

SUMMARY STEPS

1. In node view (single-shelf mode) or shelf view (multishelf mode), double-click a transponder, muxponder,

or xponder card to open the card view.

2. Click the Provisioning > OTN > FEC Thresholds tabs.

3. In the Bit Errors Corrected field, enter a threshold number, for example 225837.

4. In the Uncorrectable Words field, enter a threshold number, for example, 2.

5. In the Intervals area, click 15 Min.

General Troubleshooting

DETAILED STEPS

Step 1 In node view (single-shelf mode) or shelf view (multishelf mode), double-click a transponder, muxponder, or xponder

card to open the card view.

Step 2 Click the Provisioning > OTN > FEC Thresholds tabs.

Step 3 In the Bit Errors Corrected field, enter a threshold number, for example 225837.

Step 4 In the Uncorrectable Words field, enter a threshold number, for example, 2.
Step 5 In the Intervals area, click 15 Min.

Sample Trouble Resolutions

The following sample trouble resolutions use PM and TCAs to isolate degrade points.

Problem There is a BBE TCA on a single transponder pair.

Possible Cause The transponder input power is out of range.

Solution Check the input power on the transponder. It should be within the specified/supported range.

Possible Cause There are dirty trunk connectors on the transponder.

Solution Check the connector on the trunk port.

Possible Cause There is a degraded trunk patchcord between the transponder and the DWDM port.

Solution Check the patchcord on the transponder DWDM port.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

32

General Troubleshooting

Using CTC Diagnostics

Possible Cause There are dirty client connectors on the ADxC-xx.x card transmit port or the demultiplexer

(DMX) has crossed the near-end TCA.

Solution Check the connector on the OCH port of the ADxC-xx.x card.

Possible Cause There are dirty client connectors on the ADxC-xx.x card receive port or the multiplexer

(MUX) has crossed the far-end TCA point.

Solution If an optical channel bypass exists along the line, check the connectors.

Problem There is a BBE TCA on all transponders connected to an ADxB-xx.x card.

Possible Cause The transponder input power is out of range.

Solution Check the input power on the transponder. It should be within the specified/supported range.

Possible Cause There is a dirty connector on the 4MD-xx.x card port.

Solution Check the connector on the drop port of the 4MD-xx.x card.

Possible Cause There is a dirty connector on the ADxB-xx.x card drop port, and it has crossed the near-end

TCA point.

Solution Check the connector on the drop port of the ADxB-xx.x card.

Possible Cause There is a dirty connector on the ADxB-xx.x card add port and it has crossed the far-end

TCA.

Solution Check the patchcord on the 4MD-xx.x or AD1B-xx.x card.

Possible Cause There is a degraded patchcord between the ADxB-xx.x and 4MD-xx.x cards.

Solution If an optical band bypass exists along the line, check the band connectors.

Problem There is a BBE TCA on all transponders that the OCH passes through a single OTS section.

Possible Cause This is not a transponder or channel-related issue.

Solution The problem is in the intercabinet signal path preceding the transponder.

Problem You have a laser bias current (LBC) TCA on a single transponder.

Possible Cause The laser of the transponder is degrading.

Solution The problem is within the laser circuitry. Check the OPT-PRE, OPT-BST, OPT-AMP-C, and

OPT-AMP-17C optical amplifier cards.

Using CTC Diagnostics

In Software Release 9.1, CTC provides diagnostics for the following functions:

• Verifying proper card application-specific integrated circuit (ASIC) functionality

• Verifying standby card operation

• Verifying proper card LED operation

• Diagnostic circuit creation

• Customer problem notifications detected by alarms

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

33

Card LED Lamp Tests

• Provision of a downloadable, machine-readable diagnostic information file to be used by Cisco Technical
Support

Some of these functions, such as ASIC verification and standby card operation, are invisibly monitored in
background functions. Change or problem notifications are provided in the Alarms and Conditions windows.
Other diagnostic functionsverifying card LED function, creating bidirectional diagnostic circuits, and also
downloading diagnostic files for technical supportare available to the user in the node view (single-shelf mode)
or shelf view (multishelf mode) Maintenance > Diagnostic tab. The user-operated diagnostic features are
described in the following paragraphs.

Card LED Lamp Tests

A card LED lamp test determines whether card-level indication LEDs are operational. This diagnostic test is
run as part of the initial turn-up, during maintenance routines, or any time you question whether an LED is
in working order. Maintenance or higher-level users can complete the following tasks to verify LED operation.

Verify Card LED Operation

General Troubleshooting

SUMMARY STEPS

1. In node view (single-shelf mode) or shelf view (multishelf mode), click the Maintenance > Diagnostic

tabs.

2. Click Lamp Test.

3. Watch to make sure all the port LEDs illuminate simultaneously for several seconds, with the following

durations:

4. Click OK in the Lamp Test Run dialog box.

DETAILED STEPS

Step 1 In node view (single-shelf mode) or shelf view (multishelf mode), click the Maintenance > Diagnostic tabs.
Step 2 Click Lamp Test.

Step 3 Watch to make sure all the port LEDs illuminate simultaneously for several seconds, with the following durations:

• For tri-color LEDs: three 5-second cycles

• For dual-color LEDs: one 5-second cycle and one 10-second cycle

Step 4 Click OK in the Lamp Test Run dialog box.

Retrieve Tech Support Logs Button

When you click the Retrieve Tech Support Logs button in the Diagnostics tab of the Maintenance window,
CTC retrieves system data that a Retrieve or higher level user can off-load to a local directory and send to
Technical Support for troubleshooting purposes. The diagnostics file is in machine language and is not
human-readable, but can be used by Cisco Technical Support for problem analysis. Complete the following
procedure to off-load the diagnostics file.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

34

General Troubleshooting

Note

In addition to the machine-readable diagnostics file, the system stores an audit trail of all system events such
as user log-ins, remote log-ins, configuration, and changes. This audit trail is considered a record-keeping
feature rather than a troubleshooting feature.

Off-Load the Diagnostics File

Note

The diagnostics operation is performed at a shelf level. Only single-node-related diagnostic information can
be downloaded at a time.

The diagnostic files retrieved by CTC depends on the user privilege levels. Table 3: Diagnostic Files Retrieved

Based on User Privilege, on page 35 lists the user privilege levels and the diagnostic retrieval operations they

can perform.

Table 3: Diagnostic Files Retrieved Based on User Privilege

Off-Load the Diagnostics File

Retrieve

Maintenance

Provisioning

Superuser

Diagnostic File Retrieval OperationUser Privilege Level

• Export the unfiltered alarm table contents

• Export the unfiltered conditions table contents

• Export the unfiltered history table contents

• Export the inventory table contents

• CTC Dump Diagnostics log

• All Retrieve level access operations

• Save the node database

• All Maintenance level access operations

• Retrieve and save the node-level diagnostics report. (If secure mode is not set
on the node.)

• Export the audit table contents. (If the
NODE.security.grantPermission.RetrieveAuditLog NE Default is set to
Provisioning.)

• All Provisioning level access operations

• Retrieve and save the node-level diagnostics report

• Export the audit table contents

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

35

General Troubleshooting

Off-Load the Diagnostics File

SUMMARY STEPS

1. In the node view, click the Maintenance > Diagnostic tabs.

2. Click Retrieve Tech Support Logs in the Controller area.

3. In the Select a Filename for the Tech Support Logs Zip Archive dialog box, add the diagnostics file name

in the format TechSupportLogs_<node_name>.zip by default. Substitute the last 20 alphanumeric
characters of the node name for <node_name>. Navigate to the directory (local or network) where you
want to save the file.

4. Click Save.

5. Click OK.

DETAILED STEPS

Step 1 In the node view, click the Maintenance > Diagnostic tabs.
Step 2 Click Retrieve Tech Support Logs in the Controller area.

Step 3 In the Select a Filename for the Tech Support Logs Zip Archive dialog box, add the diagnostics file name in the format

TechSupportLogs_<node_name>.zip by default. Substitute the last 20 alphanumeric characters of the node name for
<node_name>. Navigate to the directory (local or network) where you want to save the file.

A message appears asking you if you want to overwrite any existing disgnostics file in the selected directory.

Step 4 Click Save.

CTC performs the diagnostic tasks and writes the diagnostic files in a folder named TechSupportLogs_<node_name>
under the location selected in Step Step 3, on page 36. After all the diagnostic files are written to the
TechSupportLogs_<node_name> folder, CTC archives the retrieved diagnostic files as
TechSupportLogs_<node_name>.zip. CTC deletes the TechSupportLogs_<node_name> folder after the archiving
process is successfully completed. CTC retains this folder if the archiving process fails. The retrieved diagnostic files
can be accessed in the TechSupportLogs_<node_name> folder.

A progress bar indicates the percentage of the file that is being saved. The Save Tech Support Logs Completed dialog
box appears when the file is saved. CTC logs any error during the retrieval and archiving of diagnostics file to the CTC
Alerts Log.

Table 4: List of Diagnostic Files, on page 36 lists the diagnostic files retrieved by CTC.

Table 4: List of Diagnostic Files

Diagnostic File ContentDiagnostic File

Alarm Table exportAlarmTableLog.html

Alarm Table exportHistoryTableLog.html

Conditions Table exportConditionsTableLog.html

Inventory Table exportInventoryTableLog.html

Audit Table exportAuditTableLog.html

Audit Table exportCTCDumpDiagLog.txt

NodeDiagnostics.gzNodeDiagnostics.bin

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

36

General Troubleshooting

Data Communications Network Tool

Diagnostic File ContentDiagnostic File

OBFLDiagnostics.binOBFLDiagnostics.bin

Database backupNodeDatabaseBackup.bin

TechSupportLogs_<node_name>.zip

Step 5 Click OK.

Data Communications Network Tool

CTC contains a data communications network (DCN) tool that assists with network troubleshooting for Open
Shortest Path First (OSPF) networks. It executes an internal dump command to retrieve information about all
nodes accessible from the entry point.

The dump, which provides the same information as a dump executed by special networking commands, is
available at the network view in the Maintenance > Diagnostic tab. You can select the access point node in
the Select Node drop-down list. To create the dump, click Retrieve. (To clear the dump, click Clear.)

The contents of the dump file can be saved or printed and furnished to Cisco Technical Support for use in
OSPF network support.

Onboard Failure Logging

Onboard Failure Logging (OBFL) records events that occur during the card operation. In the event of card
failure, the stored log can assist in determining root cause of failure. The OBFL data is stored in two different
formats:

Zip archive of all the diagnostics file

• Run time log for IO cards

• Snapshot log for IO cards

The OBFL feature is supported on the following cards:

• OPT-BST

• OPT-PRE

• 40-SMR1-C

• 40-SMR2-C

• 40G-MXP-C

• 80-WXC-C

Note

To determine if OBFL is supported on the OPT-BST and OPT-PRE cards running in your system, contact
the Cisco Technical Assistance Center (TAC).

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

37

Run Time Log for IO Cards

Note

The stored logs can be retrieved only by the Cisco support team to diagnose the root cause of the card failure.

Run Time Log for IO Cards

Run time log traces events and critical information such as alarms raised and cleared, power variations and
so on, during the working of the card. The stored logs help identify the cause of failure.

For legacy cards (OPT-BST and OPT-PRE), the run time logs are automatically stored in RAM and are deleted
when the card is hard reset. To store the logs in the permanent memory, the user should take the snapshot of
logs as explained in the Snapshot Logging in CTC, on page 39 section. For new cards (40-SMR1-C and
40-SMR2-C), the run time logs are automatically written to the flash memory and are not deleted even after
reset or hard reboot of the card.

The following table lists a few run time logs captured for a specific event:

Table 5: Run Time Logging—Events and Logs

General Troubleshooting

When the change in Rx and Tx optical power in the
active stage is greater than the threshold value, the
unit stores the input and output power every second.
The difference between the two adjacent input power
readings or two adjacent output power readings is
greater than 1 db, and this event occurs more than 10
times in 30 seconds

Target power not reached (0.5 dB or more difference
from set point)

Fiber Temperature Alarm

Laser Temperature Alarm

LogEvent

• Input power of all the active stages (1 for the
BST, 2 for the PRE)

• Output power of all the active stages (1 for the
BST, 2 for the PRE)

• Module status

• Laser pump current—set point and value

• Laser pump power—set point and value

• DCU loss

• VOA loss

• Optical power values

• Temperature of the case

• Temperature of the laser

• Temperature of the case

• Temperature of the fiber

Case Temperature Alarm

• Temperature of the case

• Temperature of the fiber

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

38

General Troubleshooting

Snapshot Log for IO Cards

LogEvent

Communication error with TCC

Snapshot Log for IO Cards

Snapshot log captures the board's information at any given time. In CTC, the user has an option to take a
snapshot of the current status of the card. When the snapshot is taken, a log file will be created that contains
the information from the card. In addition to the information stored in the run time logs, the snapshot log
contains details like card parameters, alarm history, and so on. For legacy and new cards, the snapshot logs
are written to the flash memory. When EQPT-FAIL alarm is detected on the card, a snapshot of the log will
be automatically taken by the card. In the event of card failure due to other reasons, the users must take the
snapshot of logs before swapping the card. Refer to the Snapshot Logging in CTC, on page 39 section.

Snapshot Logging in CTC

The users can take the snapshot of logs in the event of card failure, before replacing the card. This section
explains the steps to take snapshot of logs in CTC:

SUMMARY STEPS

1. Login to CTC.

2. In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card to open it in the

card view.

3. Click the Maintenance > OBFL tabs.

4. Click Start Onboard Failure logging. The OBFL Info dialog box is displayed.

5. Click Yes to continue. The Onboard failure logging feature is launched.

6. Click OK. The snapshot log will be written to the flash memory.

• FPGA dump

• E2PROM dump

DETAILED STEPS

Step 1 Login to CTC.

Step 2 In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card to open it in the card view.
Step 3 Click the Maintenance > OBFL tabs.
Step 4 Click Start Onboard Failure logging. The OBFL Info dialog box is displayed.
Step 5 Click Yes to continue. The Onboard failure logging feature is launched.
Step 6 Click OK. The snapshot log will be written to the flash memory.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

39

Restoring the Database and Default Settings

Restoring the Database and Default Settings

This section contains troubleshooting for node operation errors that require restoration of software data or the
default node setup.

Restore the Node Database

Problem One or more nodes do not function properly or have incorrect data.

Possible Cause Incorrect or corrupted node database.

Solution Complete the procedures in the Maintain the Node chapter of the configuration guide.

PC Connectivity Troubleshooting

This section contains information about system minimum requirements, supported platforms, browsers, and
Java Runtime Environments (JREs) for Software 9.6.x 9.8, and troubleshooting procedures for PC and network
connectivity to the chassis. Table 6: Computer Requirements for CTC lists the requirements for PCs and
UNIX workstations. In addition to the JRE, the Java plug-in is also included on the software CD.

General Troubleshooting

Table 6: Computer Requirements for CTC

Processor (PC only)

Hard drive

Pentium Dual-Core processor or
equivalent

2 GB RAM or moreRAM

20 GB hard drive with 250 MB of
free space required

NotesRequirementsArea

A faster CPU is recommended if
your workstation runs multiple
applications or if CTC manages a
network with a large number of
nodes and circuits.

A minimum of 2 GB is
recommended if your workstation
runs multiple applications or if
CTC manages a network with a
large number of nodes and circuits.

CTC application files are
downloaded from the
TNC/TSC/TNCS/TNCS-O to your
computer. These files occupy
around 100MB (250MB to be safer)
or more space depending on the
number of versions in the network.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

40

General Troubleshooting

PC Connectivity Troubleshooting

NotesRequirementsArea

Operating System

Java Runtime Environment

• PC: Windows 2000, Windows
XP, Windows Vista, Windows
XP, Windows 7, Windows
Server 2003 and 2008.

• Workstation: Solaris versions
9 or 10 on an UltraSPARC-III
or faster processor, with a
minimum of 1 GB RAM and
a minimum of 250 MB of
available hard drive space.

• Apple Mac OS X. CTC needs
to be installed using the
CacheInstaller available on
CCO or the ONS CD.

• JRE 1.6

• JRE 1.7 (R9.4 and later
releases)

Use the latest patch/Service Pack
released by the OS vendor. Check
with the vendor for the latest
patch/Service Pack.

JRE 1.6 is installed by the CTC
Installation Wizard included on the
software CD. JRE 1.6 provides
enhancements to CTC performance,
especially for large networks with
numerous circuits.

We recommend that you use JRE

1.6 for networks with Software
R9.2 nodes. If CTC must be
launched directly from nodes
running software R7.0 or R7.2, We
recommend JRE 1.4.2 or JRE 5.0.
If CTC must be launched directly
from nodes running software R5.0
or R6.0, we recommend JRE 1.4.2.
If CTC must be launched directly
from nodes running software earlier
than R5.0, we recommend JRE

1.3.1_02.

Web browser

• PC: Internet Explorer 8.x, 9.x
(R9.6 and later releases), 10
(R9.4.0.3, R9.6.0.3 and later
releases) 11 (R9.8 and later
releases)

For the PC, use JRE 1.6 or JRE 1.7
with any supported web browser.

The supported browser can be
downloaded from the Web.

• UNIX Workstation: Mozilla

1.7

• MacOS-X PC: Safari

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

41

Unable to Verify the IP Configuration of Your PC

General Troubleshooting

NotesRequirementsArea

Cable

straight-through cable with
RJ-45 connectors on each end
to connect the computer to the
chassis directly or through a
LAN.

• User-supplied cross-over
CAT-5 cable to the DCN port
on the patch panel or to the
Catalyst 2950 (multishelf
mode)

Unable to Verify the IP Configuration of Your PC

Problem When connecting your PC to the system, you are unable to successfully ping the IP address of your

PC to verify the IP configuration.

Possible Cause The IP address was entered incorrectly.

Solution Verify that the IP address used to ping the PC matches the IP address displayed when in the Windows

IP Configuration information retrieved from the system. See the Verify the IP Configuration of Your PC, on

page 42.

Possible Cause The IP configuration of your PC is not properly set.

—• User-supplied CAT-5

Solution Verify the IP configuration of your PC. Complete the Verify the IP Configuration of Your PC, on

page 42. If this procedure is unsuccessful, contact your network administrator for instructions to correct the

IP configuration of your PC.

Verify the IP Configuration of Your PC

SUMMARY STEPS

1. Open a DOS command window by selecting Start > Run from the Start menu.

2. In the Open field, type command and then click OK. The DOS command window appears.

3. At the prompt in the DOS window, type ipconfig and press the Enter key.

4. At the prompt in the DOS window, type ping followed by the IP address shown in the Windows IP

configuration information previously displayed.

5. Press the Enter key to execute the command.

DETAILED STEPS

Step 1 Open a DOS command window by selecting Start > Run from the Start menu.
Step 2 In the Open field, type command and then click OK. The DOS command window appears.
Step 3 At the prompt in the DOS window, type ipconfig and press the Enter key.

The Windows IP configuration information appears, including the IP address, the subnet mask, and the default gateway.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

42

General Troubleshooting

Browser Login Does Not Launch Java

Note

The winipcfg command only returns the information above if you are on a network.

Step 4 At the prompt in the DOS window, type ping followed by the IP address shown in the Windows IP configuration

information previously displayed.

Step 5 Press the Enter key to execute the command.

If the DOS window returns multiple (usually four) replies, the IP configuration is working properly.

If you do not receive a reply, your IP configuration might not be properly set. Contact your network administrator for
instructions to correct the IP configuration of your PC.

Browser Login Does Not Launch Java

Problem The message Loading Java Applet does not appear and the JRE does not launch during the initial

login.

Possible Cause The PC operating system and browser are not properly configured.

Solution Reconfigure the PC operating system Java Plug-in Control Panel and the browser settings. Complete

the Reconfigure the PC Operating System Java Plug-in Control Panel, on page 43 and the Reconfigure the

Browser, on page 44.

Reconfigure the PC Operating System Java Plug-in Control Panel

SUMMARY STEPS

1. From the Windows start menu, click Settings > Control Panel.

2. If Java Plug-in does not appear, the JRE might not be installed on your PC:

3. From the Windows start menu, click Settings > Control Panel.

4. In the Java Plug-in Control Panel window, double-click the Java Plug-in 1.6 icon.

5. Click the Advanced tab on the Java Plug-in Control Panel.

6. Navigate to C:\ProgramFiles\JavaSoft\JRE\1.6.

7. Select JRE 1.6.

8. Click Apply.

9. Close the Java Plug-in Control Panel window.

DETAILED STEPS

Step 1 From the Windows start menu, click Settings > Control Panel.
Step 2 If Java Plug-in does not appear, the JRE might not be installed on your PC:

a) Run the software CD.
b) Open the CD-drive:\Windows\JRE folder.
c) Double-click the j2re-1_6-win icon to run the JRE installation wizard.
d) Follow the JRE installation wizard steps.

Step 3 From the Windows start menu, click Settings > Control Panel.
Step 4 In the Java Plug-in Control Panel window, double-click the Java Plug-in 1.6 icon.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

43

Reconfigure the Browser

Step 5 Click the Advanced tab on the Java Plug-in Control Panel.
Step 6 Navigate to C:\ProgramFiles\JavaSoft\JRE\1.6.
Step 7 Select JRE 1.6.
Step 8 Click Apply.

Step 9 Close the Java Plug-in Control Panel window.

Reconfigure the Browser

SUMMARY STEPS

1. From the Start Menu, launch your browser application.

2. If you are using Netscape Navigator:

3. If you are using Internet Explorer:

4. Temporarily disable any virus-scanning software on the computer. See the Browser Stalls When

Downloading CTC JAR Files From Control Card, on page 50.

5. Verify that the computer does not have two network interface cards (NICs) installed. If the computer does

have two NICs, remove one.

6. Restart the browser and log onto the system.

General Troubleshooting

DETAILED STEPS

Step 1 From the Start Menu, launch your browser application.

Step 2 If you are using Netscape Navigator:

a) From the Netscape Navigator menu bar, click the Edit > Preferences menus.
b) In the Preferences window, click the Advanced > Proxies categories.
c) In the Proxies window, click the Direct connection to the Internet check box and click OK.
d) From the Netscape Navigator menu bar, click the Edit > Preferences menus.
e) In the Preferences window, click the Advanced > Cache categories.
f) Confirm that the Disk Cache Folder field shows one of the following paths:

• For Windows 98/ME: C:\ProgramFiles\Netscape\Communicator\cache

• For Windows NT/2000/XP: C:\ProgramFiles\Netscape\username\Communicator\cache

g) If the Disk Cache Folder field is not correct, click Choose Folder.
h) Navigate to the file listed in Step 2.f, on page 44, and click OK.
i) Click OK in the Preferences window and exit the browser.

Step 3 If you are using Internet Explorer:

a) From the Internet Explorer menu bar, click the Tools > Internet Options menus.
b) In the Internet Options window, click the Advanced tab.
c) In the Settings menu, scroll down to Java (Sun) and click the Use Java 2 v1.4.2 for applet (requires restart) check

box.

d) Click OK in the Internet Options window and exit the browser.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

44

General Troubleshooting

Unable to Verify the NIC Connection on Your PC

Step 4 Temporarily disable any virus-scanning software on the computer. See the Browser Stalls When Downloading CTC JAR

Files From Control Card, on page 50.

Step 5 Verify that the computer does not have two network interface cards (NICs) installed. If the computer does have two NICs,

remove one.

Step 6 Restart the browser and log onto the system.

Unable to Verify the NIC Connection on Your PC

Problem When connecting your PC to the system, you are unable to verify that the NIC connection is working

properly because the link LED is not illuminated or flashing.

Possible Cause The CAT-5 cable is not plugged in properly.

Solution Confirm that both ends of the cable are properly inserted. If the cable is not fully inserted due to a

broken locking clip, the cable should be replaced.

Possible Cause The CAT-5 cable is damaged.

Solution Ensure that the cable is in good condition. If in doubt, use a known-good cable. Often, cabling is

damaged due to pulling or bending.

Possible Cause Incorrect type of CAT-5 cable is being used.

Solution If connecting directly to your laptop, a PC, or a router, use a straight-through CAT-5 cable. When

connecting the chassis to a hub or a LAN switch, use a crossover CAT-5 cable. For details on the types of
CAT-5 cables, see the Crimp Replacement LAN Cables, on page 60.

Possible Cause The NIC is improperly inserted or installed.

Solution If you are using a Personal Computer Memory Card International Association (PCMCIA)-based NIC,

remove and reinsert the NIC to make sure the NIC is fully inserted. (If the NIC is built into the laptop or PC,
verify that the NIC is not faulty.)

Possible Cause The NIC is faulty.

Solution Confirm that the NIC is working properly. If you have no issues connecting to the network (or any

other node), then the NIC should be working correctly. If you have difficulty connecting a to the network (or
any other node), then the NIC might be faulty and needs to be replaced.

Verify PC Connection to the NCS (ping)

Problem The TCP/IP connection was established and then lost.

Possible Cause A lost connection between the PC and the system.

Solution Use a standard ping command to verify the TCP/IP connection between the PC and the card. A ping

command should work if the PC connects directly to the control card or uses a LAN to access the control
card. Complete the Ping the NCS, on page 45.

Ping the NCS

SUMMARY STEPS

1. Display the command prompt:

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

45

The IP Address of the Node is Unknown

2. For both the Sun and Microsoft operating systems, at the prompt enter: ping IP-address

3. If the workstation has connectivity to the NCS, the ping is successful and displays a reply from the IP

address. If the workstation does not have connectivity, a Request timed out message appears.

4. If the ping is successful, it demonstrates that an active TCP/IP connection exists. Restart CTC.

5. If the ping is not successful, and the workstation connects to the NCS through a LAN, check that the

workstation IP address is on the same subnet as the ONS node.

6. If the ping is not successful and the workstation connects directly to the NCS, check that the link light on

the workstation NIC is illuminated.

DETAILED STEPS

Step 1 Display the command prompt:

a) If you are using a Microsoft Windows operating system, from the Start Menu choose Run, enter command in the

Open field of the Run dialog box, and click OK.

b) If you are using a Sun Solaris operating system, from the Common Desktop Environment (CDE) click the Personal

Application tab and click Terminal.

General Troubleshooting

Step 2 For both the Sun and Microsoft operating systems, at the prompt enter: ping IP-address

For example:

ping 198.168.10.10

Step 3 If the workstation has connectivity to the NCS, the ping is successful and displays a reply from the IP address. If the

workstation does not have connectivity, a Request timed out message appears.

Step 4 If the ping is successful, it demonstrates that an active TCP/IP connection exists. Restart CTC.

Step 5 If the ping is not successful, and the workstation connects to the NCS through a LAN, check that the workstation IP

address is on the same subnet as the ONS node.

Step 6 If the ping is not successful and the workstation connects directly to the NCS, check that the link light on the workstation

NIC is illuminated.

The IP Address of the Node is Unknown

Problem The IP address of the node is unknown and you are unable to login.

Possible Cause The node is not set to the default IP address.

Solution Leave one control card in the shelf. Connect a PC directly to the remaining control card and perform

a hardware reset of the card. The control card transmits the IP address after the reset to enable you to capture
the IP address for login. Complete the Retrieve Unknown Node IP Address, on page 46.

Retrieve Unknown Node IP Address

SUMMARY STEPS

1. Connect your PC directly to the active control card Ethernet port on the faceplate.

2. Start the Sniffer application on your PC.

3. Perform a hardware reset by pulling and reseating the active control card.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

46

General Troubleshooting

CTC Operation Troubleshooting

4. After the control card completes resetting, it broadcasts its IP address. The Sniffer software on your PC

will capture the IP address being broadcast.

DETAILED STEPS

Step 1 Connect your PC directly to the active control card Ethernet port on the faceplate.

Step 2 Start the Sniffer application on your PC.

Step 3 Perform a hardware reset by pulling and reseating the active control card.

Step 4 After the control card completes resetting, it broadcasts its IP address. The Sniffer software on your PC will capture the

IP address being broadcast.

CTC Operation Troubleshooting

This section contains troubleshooting procedures for CTC login or operation problems.

CTC Colors Do Not Appear Correctly on a UNIX Workstation

Problem When running CTC on a UNIX workstation, the colors do not appear correctly. For example, both

major and minor alarms appear in the same color.

Possible Cause When running in 256-color mode on a UNIX workstation, color-intensive applications

such as Netscape might use all of the colors.

Solution CTC requires a full 24-color palette to run properly. When logging into CTC on a UNIX workstation,

run as many colors as your adapter will support. In addition, you can use the -install or the -ncols 32 command
line options to limit the number of colors that Netscape uses. Complete the Limit Netscape Colors, on page

47. If the problem persists after limiting Netscape colors, exit any other color-intensive applications in use.

Limit Netscape Colors

SUMMARY STEPS

1. Close the current session of Netscape.

2. Launch Netscape from the command line by entering one of the following commands:

DETAILED STEPS

Step 1 Close the current session of Netscape.

Step 2 Launch Netscape from the command line by entering one of the following commands:

• netscape -install (installs Netscape colors for Netscape use)

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

47

Unable to Launch CTC Help After Removing Netscape

• netscape -ncols 32 (limits Netscape to 32 colors so that if the requested color is not available, Netscape chooses the

closest color option)

Unable to Launch CTC Help After Removing Netscape

Problem After removing Netscape and running CTC using Internet Explorer, you are unable to launch CTC

Help and receive an MSIE is not the default browser error message.

Possible Cause Loss of association between browser and Help files.

Solution When the CTC software and Netscape are installed, the Help files are associated with Netscape by

default. When you remove Netscape, the Help files are not automatically associated with Internet Explorer
as the default browser. Reset Internet Explorer as the default browser so that CTC associates the Help files
to the correct browser. Complete the Reset Internet Explorer as the Default Browser for CTCInternet Explorer

resetting as default browserresetting Internet Explorer as the default browser, on page 48 to associate the

CTC Help files to the correct browser.

General Troubleshooting

Reset Internet Explorer as the Default Browser for CTC

SUMMARY STEPS

1. Open the Internet Explorer browser.

2. From the menu bar, click Tools > Internet Options. The Internet Options window appears.

3. In the Internet Options window, click the Programs tab.

4. Click the Internet Explorer should check to see whether it is the default browser check box.

5. Click OK.

6. Exit all open and running CTC and Internet Explorer applications.

7. Launch Internet Explorer and open a new CTC session. You should now be able to access the CTC Help.

DETAILED STEPS

Step 1 Open the Internet Explorer browser.
Step 2 From the menu bar, click Tools > Internet Options. The Internet Options window appears.
Step 3 In the Internet Options window, click the Programs tab.
Step 4 Click the Internet Explorer should check to see whether it is the default browser check box.
Step 5 Click OK.

Step 6 Exit all open and running CTC and Internet Explorer applications.

Step 7 Launch Internet Explorer and open a new CTC session. You should now be able to access the CTC Help.

Unable to Change Node View to Network View

Problem When activating a large, multinode BLSR from Software R3.2 to Software R3.3, some of the nodes

appear grayed out. Logging into the new CTC, the user is unable to change node view (single-shelf mode) or

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

48

General Troubleshooting

Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Windows

shelf view (multishelf mode) to network view on any nodes, from any workstation. This is accompanied by
an Exception occurred during event dispatching: java.lang.OutOfMemoryError in the java window.

Possible Cause The large, multinode BLSR requires more memory for the graphical user interface (GUI)

environment variables.

Solution Set the system or user CTC_HEAP environment variable to increase the memory limits. Complete

the Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Windows, on page

49 or the Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Solaris, on
page 50 to enable the CTC_HEAP variable change.

Note

This problem typically affects large networks where additional memory is required to manage large numbers
of nodes and circuits.

Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Windows

Note

Before proceeding with the following steps, ensure that your system has a minimum of 4 GB of RAM. If your
system does not have a minimum of 4 GB of RAM, contact the Cisco Technical Assistance Center (TAC).

SUMMARY STEPS

1. Close all open CTC sessions and browser windows.

2. From the Windows Start menu, choose Control Panel > System.

3. In the System Properties window, click the Advanced tab.

4. Click the Environment Variables button to open the Environment Variables window.

5. Click the New button under the System variables field.

6. Type CTC_HEAP in the Variable Name field.

7. Type 896 in the Variable Value field, and then click the OK button to create the variable.

8. Again, click the New button under the System variables field.

9. Type CTC_MAX_PERM_SIZE_HEAP in the Variable Name field.

10. Type 256 in the Variable Value field, and then click the OK button to create the variable.

11. Click the OK button in the Environment Variables window to accept the changes.

12. Click the OK button in the System Properties window to accept the changes.

DETAILED STEPS

Step 1 Close all open CTC sessions and browser windows.
Step 2 From the Windows Start menu, choose Control Panel > System.
Step 3 In the System Properties window, click the Advanced tab.
Step 4 Click the Environment Variables button to open the Environment Variables window.
Step 5 Click the New button under the System variables field.

Step 6 Type CTC_HEAP in the Variable Name field.

Step 7 Type 896 in the Variable Value field, and then click the OK button to create the variable.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

49

General Troubleshooting

Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Solaris

Step 8 Again, click the New button under the System variables field.

Step 9 Type CTC_MAX_PERM_SIZE_HEAP in the Variable Name field.
Step 10 Type 256 in the Variable Value field, and then click the OK button to create the variable.
Step 11 Click the OK button in the Environment Variables window to accept the changes.
Step 12 Click the OK button in the System Properties window to accept the changes.

Set the CTC_HEAP and CTC_MAX_PERM_SIZE_HEAP Environment Variables for Solaris

SUMMARY STEPS

1. From the user shell window, kill any CTC sessions and browser applications.

2. In the user shell window, set the environment variables to increase the heap size.

DETAILED STEPS

Step 1 From the user shell window, kill any CTC sessions and browser applications.

Step 2 In the user shell window, set the environment variables to increase the heap size.

Example:

The following example shows how to set the environment variables in the C shell:

% setenv CTC_HEAP 896
% setenv CTC_MAX_PERM_SIZE_HEAP 256

Browser Stalls When Downloading CTC JAR Files From Control Card

Problem The browser stalls or hangs when downloading a CTC Java archive (JAR) file from the control card.

Possible Cause McAfee VirusScan software might be interfering with the operation. The problem occurs

when the VirusScan Download Scan is enabled on McAfee VirusScan 4.5 or later.

Solution Disable the VirusScan Download Scan feature. Complete the Disable the VirusScan Download Scan,

on page 50.

Disable the VirusScan Download Scan

SUMMARY STEPS

1. From the Windows Start menu, choose Programs > Network Associates > VirusScan Console.

2. Double-click the VShield icon listed in the VirusScan Console dialog box.

3. Click Configure on the lower part of the Task Properties window.

4. Click the Download Scan icon on the left of the System Scan Properties dialog box.

5. Uncheck the Enable Internet download scanning check box.

6. Click Yes when the warning message appears.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

50

General Troubleshooting

7. Click OK in the System Scan Properties dialog box.

8. Click OK in the Task Properties window.

9. Close the McAfee VirusScan window.

DETAILED STEPS

Step 1 From the Windows Start menu, choose Programs > Network Associates > VirusScan Console.
Step 2 Double-click the VShield icon listed in the VirusScan Console dialog box.
Step 3 Click Configure on the lower part of the Task Properties window.
Step 4 Click the Download Scan icon on the left of the System Scan Properties dialog box.
Step 5 Uncheck the Enable Internet download scanning check box.
Step 6 Click Yes when the warning message appears.
Step 7 Click OK in the System Scan Properties dialog box.
Step 8 Click OK in the Task Properties window.

Step 9 Close the McAfee VirusScan window.

CTC Does Not Launch

CTC Does Not Launch

Problem CTC does not launch; usually an error message appears before the login window appears.

Possible Cause The Netscape browser cache might point to an invalid directory.

Solution Redirect the Netscape cache to a valid directory. Complete the Redirect the Netscape Cache to a Valid

Directory, on page 51.

Redirect the Netscape Cache to a Valid Directory

SUMMARY STEPS

1. Launch Netscape.

2. Open the Edit menu.

3. Choose Preferences.

4. In the Category column on the left side, expand the Advanced category and choose the Cache tab.

5. Change your disk cache folder to point to the cache file location.

DETAILED STEPS

Step 1 Launch Netscape.
Step 2 Open the Edit menu.
Step 3 Choose Preferences.
Step 4 In the Category column on the left side, expand the Advanced category and choose the Cache tab.

Step 5 Change your disk cache folder to point to the cache file location.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

51

Slow CTC Operation or Login Problems

The cache file location is usually C:\ProgramFiles\Netscape\Users\yourname\cache. The yourname segment of the file
location is often the same as the user name.

Slow CTC Operation or Login Problems

Problem You experience slow CTC operation or have problems logging into CTC.

Problem Table 7: Slow CTC Operation or Login Problems, on page 52 describes the potential cause of the

symptom and the solution.

Table 7: Slow CTC Operation or Login Problems

SolutionPossible Problem

General Troubleshooting

The CTC cache file might
be corrupted or might
need to be replaced.

Search for and delete cache files. This operation forces the NCS to download a
new set of Java archive (JAR) files to your computer hard drive. Complete the

Delete the CTC Cache File Automatically, on page 52 or the Delete the CTC
Cache File Manually, on page 53.

Insufficient heap memory
allocation.

Increase the heap size if you are using CTC to manage more than 50 nodes
concurrently. See the Set the CTC_HEAPand CTC_MAX_PERM_SIZE_HEAP

Environment Variables for Windows, on page 49 or the Set the CTC_HEAP and
CTC_MAX_PERM_SIZE_HEAP Environment Variables for Solaris, on page

50.

Note

Delete the CTC Cache File Automatically

Before you begin

Caution

All running sessions of CTC must be halted before deleting the CTC cache. Deleting the CTC cache might
cause any CTC running on this system to behave in an unexpected manner.

To avoid network performance issues, Cisco recommends managing
a maximum of 50 nodes concurrently with CTC. To manage more
than 50 nodes, Cisco recommends using Cisco Transport Manager
(CTM). Cisco does not recommend running multiple CTC sessions
when managing two or more large networks.

SUMMARY STEPS

52

1. Enter an NCS IP address into the browser URL field. The initial browser window shows a Delete CTC
Cache button.

2. Close all open CTC sessions and browser windows. The PC operating system does not allow you to delete
files that are in use.

3. Click Delete CTC Cache in the initial browser window to clear the CTC cache.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

General Troubleshooting

Delete the CTC Cache File Manually

DETAILED STEPS

Step 1 Enter an NCS IP address into the browser URL field. The initial browser window shows a Delete CTC Cache button.

Step 2 Close all open CTC sessions and browser windows. The PC operating system does not allow you to delete files that are

in use.

Step 3 Click Delete CTC Cache in the initial browser window to clear the CTC cache.

Delete the CTC Cache File Manually

Before you begin

Caution

All running sessions of CTC must be halted before deleting the CTC cache. Deleting the CTC cache might
cause any CTC running on this system to behave in an unexpected manner.

SUMMARY STEPS

1. To delete the JAR files manually, from the Windows Start menu choose Search > For Files or Folders.

2. In the Search Results dialog box, enter ctc*.jar or cms*.jar in the Search for Files or Folders Named

field and click Search Now.

3. Click the Modified column in the Search Results dialog box to find the JAR files that match the date

when you downloaded the files from the control card.

4. Highlight the files and press the keyboard Delete key.

5. Click Yes in the Confirm dialog box.

DETAILED STEPS

Step 1 To delete the JAR files manually, from the Windows Start menu choose Search > For Files or Folders.
Step 2 In the Search Results dialog box, enter ctc*.jar or cms*.jar in the Search for Files or Folders Named field and click

Search Now.

Step 3 Click the Modified column in the Search Results dialog box to find the JAR files that match the date when you downloaded

the files from the control card.

Step 4 Highlight the files and press the keyboard Delete key.
Step 5 Click Yes in the Confirm dialog box.

Node Icon is Gray on CTC Network View

Problem The CTC network view shows one or more node icons as gray in color and without a node name.

Possible Cause Different CTC releases do not recognize each other.

Solution Correct the core version build as described in the Different CTC Releases Do Not Recognize Each

Other, on page 55.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

53

Java Runtime Environment Incompatible

Possible Cause Username and password do not match.

Solution Correct the username and password as described in the Username or Password Do Not Match, on

page 55.

Possible Cause A lost DCC connection.

Solution Usually accompanied by an embedded operations channel (EOC) alarm. Clear the EOC alarm and

verify the DCC connection as described in the EOC.

Java Runtime Environment Incompatible

Problem The CTC application does not run properly.

Possible Cause The compatible Java JRE is not installed.

Solution The JRE contains the Java virtual machine, runtime class libraries, and Java application launcher that

are necessary to run programs written in the Java programming language. The NCS CTC is a Java application.
A Java application, unlike an applet, cannot rely completely on a web browser for installation and runtime
services. When you run an application written in the Java programming language, you need the correct JRE
installed. The correct JRE for each CTC software release is included on the Cisco NCS software CD. Complete
the Launch CTC to Correct the Core Version Build, on page 55. If you are running multiple CTC software
releases on a network, the JRE installed on the computer must be compatible with the different software
releases. The following table shows JRE compatibility with NCS software releases.

General Troubleshooting

Table 8: JRE Compatibility for NCS

Note

Software Release 4.0 notifies you if an earlier JRE version is running on your PC or UNIX workstation.

Launch CTC to Correct the Core Version Build

SUMMARY STEPS

1. Exit the current CTC session and completely close the browser.

2. Start the browser.

3. Enter the NCS IP address of the node that reported the alarm. This can be the original IP address you

logged in with or an IP address other than the original.

4. Log into CTC. The browser downloads the JAR file from CTC.

DETAILED STEPS

JRE 1.7JRE 1.6JRE 5.0JRE 1.4JRE 1.3JRE 1.2.2Software Release

YesYesNoNoNoNoNCS R10.0

Step 1 Exit the current CTC session and completely close the browser.

Step 2 Start the browser.

Cisco NCS 2000 series Troubleshooting Guide, Release 11.0

54