Lucent Technologies 9077 04S, 9076, RS/6000 User Manual

Download

SP Switch Router Adapter Guide

1.4 Update 2

Part Number: 7820-2039-001

For software version 1.4.20 and later

October, 1999

This material is protected by the copyright laws of the United States and other countries. It may not be reproduced, distributed, or altered in any fashion by an y entity (either internal or external to Lucent Technologies), except in accordance with applicable agreements, contracts, or licensing, without the express written consent of Lucent Technologies.

For permission to reproduce or distribute, please contact: Alison Gowan, 1-612-996-6891

Notice

Every effort was made to ensure that the information in this document was complete and accurate at the time of printing. However, information is subject to change.

Trademarks

GRF is a trademark of Lucent Technologies. Other trademarks and trade names mentioned in this publication belong to their respective owners.

Limited Warranty

Lucent T echnologies pro vides a limited w arranty to this product. See Appendix B, "Limited Warranty," in the GRF 400/1600 Getting Started manual for more information.

Ordering Information

To order copies of this document, contact your Lucent Technologies representative or reseller.

Support Telephone Numbers

For a menu of support and other services, call (800) 272-3634. Or call (510) 769-6001 for an operator.

Lucent Technologies

Contents

About this Guide............................................................................. xiii

About 1.4 Update 2................................................................................................................ xiii

How to use this Guide............................................................................................................ xiii

Manual sets ............................................................................................................................ xiv

SP Switch Router manuals.............................................................................................. xiv

IBM SP system manuals................................................................................................. xiv

Documentation conventions..................................................................................................... xv

IP routing publications........................................................................................................... xvi

Chapter 1 Introduction to the SP Switch Router Adapter card.................... 1-1

What is the RS/6000 SP Switch Router ? ............................................................................. 1-2

SP Switch Router systems for IBM sites ............................................................................. 1-3

Cables included in your system ...................................................................................... 1-3

SP Switch cable ..................................................................................................... 1-3

Ethernet cable ......................................................................................................... 1-4

SP ground strap........................................................................................................ 1-4

PCMCIA 520MB disk .................................................................................................... 1-4

Redundant AC power supplies ....................................................................................... 1-4

Redundant supply safety ......................................................................................... 1-4

Upgrading system memory.................................................................................................... 1-5

Overview of the SP Switch Router Adapter card................................................................... 1-6

Face plate diagram ........................................................................................................ 1-6

Inserting a media card into the SP Switch Router ................................................................. 1-7

ESD requirements........................................................................................................... 1-8

Card insertion procedure................................................................................................. 1-8

SP Switch Router Adapter card LEDs .................................................................................. 1-9

LED activity during boot................................................................................................ 1-9

LED activity during normal operations ........................................................................ 1-11

SP Switch Router Adapter card specifications..................................................................... 1-13

Assigning filters .................................................................................................................. 1-14

SNMP on the SP Switch Router Adapter card..................................................................... 1-15

SP Switch Router Adapter dependent node MIB support ............................................ 1-15

SP Switch Router Adapter media card states (SNMP) ................................................ 1-16

SNMP configuration overview..................................................................................... 1-17

SNMP activity during media card start up .................................................................. 1-18

SP Switch Router Adapter Guide - 1.4 Update 2 iii

Contents

Chapter 2 Conﬁguring the SP Switch Router Adapter.................................. 2-1

Introduction to installation and configuration ...................................................................... 2-2

Location of relevant information.................................................................................... 2-3

Pre-installation assumptions ......................................................................................... 2-3

Order of information....................................................................................................... 2-4

Installing an SP Switch Router Adapter card......................................................................... 2-5

Installation overview....................................................................................................... 2-5

Installing the PCMCIA spinning disk.................................................................................... 2-6

Managing PCMCIA slots................................................................................................ 2-6

Panic dumps sent to external flash device .................................................................... 2-7

Installation steps.............................................................................................................. 2-7

Attaching SP Switch Router cables ................................................................................... 2-10

Ethernet cable................................................................................................................ 2-10

SP switch cable............................................................................................................. 2-10

Procedure for attaching cables to card and SP Switch ................................................. 2-11

Configuration required on the SP system ............................................................................ 2-12

Determining the switch connection for a dependent node............................................ 2-12

Procedure............................................................................................................... 2-13

Sources of configuration information .......................................................................... 2-14

Multiple frames for multiple system connections ........................................................ 2-15

Step-by-step media card configuration ................................................................................ 2-16

Configuration files and their uses ................................................................................ 2-16

Overview of steps 1–5 .................................................................................................. 2-16

Step 1. Check SNMP in the SP Switch Router system....................................................... 2-18

Put SNMP changes into effect...................................................................................... 2-20

Step 2. Assign IP addresses ................................................................................................ 2-21

Method 1: Recommended, use SP SNMP Manager ................................................... 2-21

Method 2: Optional, edit /etc/grifconfig.conf.............................................................. 2-21

Interface name ...................................................................................................... 2-22

Internet address...................................................................................................... 2-22

Netmask ................................................................................................................ 2-22

Broadcast / destination address ............................................................................. 2-22

Argument field ..................................................................................................... 2-23

Putting grifconfig.conf additions into effect ............................................................... 2-23

Step 3. Change profile settings .......................................................................................... 2-24

Specify card-level parameters – Card profile ............................................................. 2-24

Specify ICMP throttling ........................................................................................ 2-24

Specify different executables ................................................................................ 2-25

Specify different dump settings ........................................................................... 2-25

Change executables for all dev1 cards - Load profile ................................................. 2-27

Change dump defaults for all dev1 cards - Dump profile ............................................ 2-28

Dump vectors (read-only)..................................................................................... 2-29

Step 4. Run dev1config to create grdev1.conf.................................................................... 2-31

Method 1: Recommended, use SP SNMP Manager.................................................... 2-31

Method 2: Optional, edit /etc/grdev1.conf................................................................... 2-31

How to run the command.............................................................................................. 2-31

Contents of /etc/grdev1.conf......................................................................................... 2-32

iv SP Switch Router Adapter Guide - 1.4 Update 2

Contents

Parameter definitions.................................................................................................... 2-33

Step 5. Reset card to install files......................................................................................... 2-35

Saving configuration files............................................................................................. 2-35

Verify SP Switch Router Adapter card from router ........................................................... 2-36

Verify media card operation using ping ...................................................................... 2-36

Verify switch node connectivity using ping ................................................................. 2-37

Check media card status using grcard .......................................................................... 2-37

Media card states ................................................................................................... 2-37

Reset media card using grreset ................................................................................... 2-38

Bringing the SP Switch Router Adapter card on-line with the SP ...................................... 2-40

Checking connectivity to the SP system....................................................................... 2-40

Procedure ...................................................................................................................... 2-41

Chapter 3 Monitoring and Management Tools............................................... 3-1

SP Switch Router command overview................................................................................... 3-2

csconfig .......................................................................................................................... 3-2

flashcmd.......................................................................................................................... 3-2

getver............................................................................................................................... 3-3

grarp .............................................................................................................................. 3-3

grcard ............................................................................................................................. 3-3

grfins............................................................................................................................... 3-3

grms ............................................................................................................................... 3-4

grreset.............................................................................................................................. 3-4

grrmb ............................................................................................................................. 3-4

grroute............................................................................................................................. 3-4

grrt................................................................................................................................... 3-4

grsite................................................................................................................................ 3-4

grsnapshot....................................................................................................................... 3-4

grstat ............................................................................................................................... 3-5

grwrite ........................................................................................................................... 3-5

mountf............................................................................................................................. 3-5

setver............................................................................................................................... 3-5

umountf........................................................................................................................... 3-5

vpurge ............................................................................................................................ 3-5

SP Switch Router UNIX tools ............................................................................................... 3-6

ping ............................................................................................................................... 3-6

route ................................................................................................................................ 3-6

tcpdump .......................................................................................................................... 3-7

traceroute ........................................................................................................................ 3-7

Using the netstat command ................................................................................................... 3-8

netstat -rn ...................................................................................................................... 3-8

netstat -rs ....................................................................................................................... 3-9

netstat -in ........................................................................................................................ 3-9

netstat -an ..................................................................................................................... 3-10

netstat -s........................................................................................................................ 3-11

Obtaining layer 2 and 3 statistics - grstat............................................................................. 3-12

Options.......................................................................................................................... 3-12

Layer 3 statistics ........................................................................................................... 3-12

List of IP stats........................................................................................................ 3-12

SP Switch Router Adapter Guide - 1.4 Update 2 v

Contents

Layer 2 statistics ........................................................................................................... 3-14

List of layer 2 stats................................................................................................. 3-14

SP Switch Router Adapter card maint commands............................................................... 3-15

Preparing to use maint commands................................................................................ 3-15

Sample maint commands.............................................................................................. 3-16

Find hardware and software version numbers - maint 2 ....................................... 3-16

Find transmit (tx) binary version - maint 102........................................................ 3-16

Display configuration and status - maint 3............................................................ 3-16

maint 4 - display media statistics .......................................................................... 3-17

maint 5 - display switch statistics.......................................................................... 3-17

maint 6 - display combus statistics........................................................................ 3-18

Filtering commands - maint 50-58, 150-58 .......................................................... 3-18

List the filters per media card - maint 50 .............................................................. 3-19

List where filters are assigned - maint 54 ............................................................. 3-19

Configure UDP packet discards - maint 89 7........................................................ 3-19

Display ARP table - maint 189 1 .......................................................................... 3-20

Flush the ARP cache - maint 189 10 .................................................................... 3-20

Display switch route table - maint 189 2 .............................................................. 3-21

“Switch route not found”....................................................................................... 3-21

Checking for hardware problems - grdiag .......................................................................... 3-22

What is tested ............................................................................................................... 3-22

Where to find the user guide......................................................................................... 3-22

Stopping or halting grdiag ............................................................................................ 3-22

When a media card does not boot after grdiag ............................................................ 3-23

“Switch receive error” can indicate hardware problem ............................................... 3-23

SP Switch Router dumps ..................................................................................................... 3-24

System dumps............................................................................................................... 3-24

Media card dumps ........................................................................................................ 3-24

Use grdinfo to collect logs .......................................................................................... 3-24

Data collection utility - grdinfo............................................................................................ 3-25

Options ........................................................................................................................ 3-25

SP Switch Router example............................................................................................ 3-26

SP Switch Router logs ........................................................................................................ 3-30

Accessing a log file....................................................................................................... 3-30

Sample gr.console log................................................................................................... 3-31

Sample gr.boot log........................................................................................................ 3-32

Sample messages log .................................................................................................... 3-33

Burning in media card flash memory................................................................................... 3-34

Appendix A Part Numbers .................................................................................. A-1

Parts list – model 04S............................................................................................................ A-1

Parts list – model 16S............................................................................................................ A-3

Publication numbers – IBM manuals.................................................................................... A-4

Appendix B Log Messages................................................................................. B-1

Alphabetical list of messages................................................................................................. B-1

Message descriptions ............................................................................................................. B-3

vi SP Switch Router Adapter Guide - 1.4 Update 2

Contents

Appendix C Network Conﬁguration Examples ................................................. C-1

Example 1: Single SP Switch Router Adapter card, single SP partition....................... C-2

Configuration requirements..................................................................................... C-2

Example 2: Multiple cards, single partition................................................................... C-3

Configuration requirements..................................................................................... C-3

Configuration tasks.................................................................................................. C-4

Incoming traffic (going to SP processor nodes)..................................................... C-4

Outgoing traffic (coming from SP processor nodes).............................................. C-4

Recovery procedure if an SP Switch Router Adapter card fails..................................... C-5

Example 3: Multiple cards, multiple SP partitions:........................................................ C-6

Configuration tasks.................................................................................................. C-6

Appendix D Upgrading Router Software........................................................... D-1

The SP Switch Router as an IBM product..................................................................... D-1

Obtaining new machine code ........................................................................................ D-1

Support for code installation.......................................................................................... D-1

IBM License Agreement for Machine Code.................................................................. D-2

Index.......................................................................................... Index-1

SP Switch Router Adapter Guide - 1.4 Update 2 vii

Contents

viii SP Switch Router Adapter Guide - 1.4 Update 2

Figures

Figure 1-1 Connections between the SP Switch Router and an SP system .............. 1-2

Figure 1-2 Expandable area of system memory............................................................. 1-5

Figure 1-3 Media card components................................................................................ 1-7

Figure 1-4 LEDs on the SP Switch Router Adapter card............................................... 1-9

Figure 2-1 Components connecting an SP Switch Router to an SP Switch and control

workstation.................................................................................................... 2-2

Figure 2-2 SP system administrative Ethernet connections ......................................... 2-10

Figure 2-3 SP Switch Router Adapter cable 50-pin connector end.............................. 2-10

Figure 2-4 How frames enable connections to multiple SP Switches.......................... 2-15

Figure 2-5 Components in the SP Switch Router Adapter card’s interface name ....... 2-22

Figure 3-1 SP Switch Router control board memory components................................. 3-2

Figure 3-2 Sample entries in the gr.console log........................................................... 3-31

Figure 3-3 Sample entries in the gr.boot log................................................................ 3-32

Figure 3-4 Sample entries in the messages log ............................................................ 3-33

Figure 3-5 Sample entries in the gr.boot log................................................................ 3-34

Figure C-1 Example 1 – one card, one SP partition network configuration................... C-2

Figure C-2 Example 2 – multiple card, single SP partition configuration...................... C-3

Figure C-3 Recovery from a card failure in a dually-connected configuration.............. C-5

Figure C-4 Example 3 – multiple card, multiple SP partition configuration.................. C-6

SP Switch Router Adapter Guide - 1.4 Update 2 ix

Figures

x SP Switch Router Adapter Guide - 1.4 Update 2

Tables

Table 1-1 SP Switch Router Adapter card LED activity during boot and reset............ 1-9

Table 1-2 SP Switch Router Adapter media card LEDs............................................. 1-11

Table 1-3 SP Switch Router Adapter media card specifications................................. 1-13

Table A-1 IBM and Lucent part numbers for model 04S............................................ A-1

Table A-2 IBM and Lucent part numbers for model 16S............................................. A-3

Table A-3 Publication numbers for related IBM manuals............................................ A-4

SP Switch Router Adapter Guide - 1.4 Update 2 xi

Tables

xii SP Switch Router Adapter Guide - 1.4 Update 2

About this Guide

Lucent’s GRF switched IP routers can be used to pro vide high-speed data communication links between IBM RS/6000 Scalable POWERparallel Systems (SP) and external networks/hosts. When packaged with an IBM SP system, the GRF router is referred to as an RS/6000 SP Switch Router, or SP Switch Router. The SP Switch Router Adapter card is the GRF media card that speciﬁcally supports SP system data transfers. To connect to an SP system, the SP Switch Router Adapter card is cabled directly to an SP Switch port.

The SP Switch Router Adapter Guide describes the media card itself and explains how to install, verify , and conﬁgure the card. The Guide provides the same type of information for this media card as is provided for other GRF media cards. Information speciﬁc to installing and conﬁguring a GRF router is found in the manuals listed below in the “Manual sets” section.

The RS/6000 SP Switch Router is based on the GRF 400 and GRF1600 routers manufactured by Lucent Technologies. For that reason, this manual contains references to the

GRF 400/1600 Getting Started , GRF Reference Guide , and GRF Conﬁguration and Management manuals.

About 1.4 Update 2

The GRF 1.4 Update 2 manual set includes new features added since software release 1.4.12. GateD information is provided in a separate document, the GRF GateD Manual.

How to use this Guide

The Guide contains the following chapters and an index:

• Chapter 1, “Introduction to the SP Switch Router Adapter Card,” describes the SP Switch

Router Adapter media card, its cables, LEDs, and SNMP implementation.

• Chapter 2, “Conﬁguring the SP Switch Router Adapter,” explains how to conﬁgure the SP

Switch Router Adapter media card and how to attach it to the SP Switch.

• Chapter 3, “Monitoring and Management Tools,” contains information about commands,

logs, and dumps useful for maintaining the SP Switch Router and adapter card.

• Appendix A, “Part Numbers,” contains a table of corresponding Lucent and IBM part

numbers for SP Switch Router components.

• Appendix B, “Log Messages, ” contains e xplanations of log messages generated by the SP

Switch Router Adapter media card.

• Appendix C, “Network Conﬁguration Examples,” contains examples and descriptions of

three basic SP Switch network conﬁgurations and their requirements.

• Appendix D, “Upgrading SP Switch Router Software,” describes how to install new

releases of the router’s operating software.

SP Switch Router Adapter Guide - 1.4 Update 2 October 25, 1999 xiii

About this Guide

Manual sets

This section provides a list of relevant GRF manuals. A second list includes IBM system manuals that contain information speciﬁc to the SP supercomputer.

SP Switch Router manuals

The SP Switch Router Adapter media card is described only in the SP Switch Router Adapter

Guide . The SP Switch Router and other media cards are described in the GRF manual set.

The GRF 1.4 Update 2 documentation set consists of the following manuals:

• GRF 400/1600 Getting Started - 1.4 Update 2

• GRF Conﬁguration and Management - 1.4 Update 2

• GRF Reference Guide - 1.4 Update 2

• GRF GateD Manual - 1.4 Update 2

• SP Switch Router Adapter Guide - 1.4 Update 2

Router manuals and software upgrade release notes for the 9077 are available in PDF format from this Web site:

http://www.rs6000.ibm.com/resource/aix_resource/sp_books/sra /index.html

To check which software release your SP Switch Router is running, use the getver command:

super> getver Current Revision: 1.4.20.ibm Version: default super>

IBM SP system manuals

IBM information speciﬁc to the SP Switch Router Adapter card appears in:

• IBM RS/6000 Scalable POWERparallel Systems:

Planning, Volume 1, Hardware and Physical Environment, GA22-7280

• IBM RS/6000 Scalable POWERparallel Systems:

Planning, Volume 2, Control Workstation and Software Environment, GA22-7281

• IBM Parallel System Support Programs for AIX:

Installation and Migration Guide, GA22-7347

• IBM Parallel System Support Programs for AIX:

Administration Guide, SA22-7348

• IBM Parallel System Support Programs for AIX:

Diagnosis Guide, GA22-7350

• IBM Parallel System Support Programs for AIX:

Command and Technical Reference, Volume 1 , SA22-7351

• IBM Parallel System Support Programs for AIX:

Command and Technical Reference, Volume 2, SA22-7351

• IBM Parallel System Support Programs for AIX:

Messages Reference, GA22-7352

xiv October 25, 1999

SP Switch Router Adapter Guide - 1.4 Update 2

You can download PDF versions of these manuals from the RS6000 SP Product Documentation Library at this web site:

http://www.rs6000.ibm.com/resource/aix_resource/sp_books/ http://www.rs6000.ibm.com/resource/aix_resource/sp_books/pssp/inde x.html

Documentation conventions

This manual uses the following standard documentation conventions:

Convention Meaning

Monospace text Represents text that appears on your computer’s screen, or that could

appear on your computer’s screen.

Boldface text

Italics

[ ] Square brackets indicate an optional argument you might add to a

| Separates command choices that are mutually exclusive. Key1-Key2 Represents a combination keystroke. To enter a combination

Press Enter Means press the Enter, or Return, key or its equivalent on your

Note:

Represents characters that you enter exactly as shown (unless the characters are also in names used in text appear in boldface.

In command usage, italic represent variable information. Do not enter the words themselves in the command. Enter the information they represent. In ordinary text, italics are used for titles of publications, for some terms that would otherwise be in quotation marks, and to show emphasis.

command. To include such an argument, type only the information inside the brackets. Do not type the brackets unless they appear in bold type.

keystroke, press the ﬁrst key and hold it down while you press one or more other keys. Release all the keys at the same time. (For example, Ctrl-H means hold down the Control key and press the H key.)

computer. Introduces important additional information.

italics

About this Guide

Documentation conventions

—see Italics , below). Command

Caution:

Warning:

SP Switch Router Adapter Guide - 1.4 Update 2 October 25, 1999 xv

W arns that a failure to follo w the recommended procedure could result in loss of data or damage to equipment.

Warns that a failure to take appropriate safety precautions could result in physical injury.

About this Guide

IP routing publications

Here are some related publications that you may ﬁnd useful:

• Internetworking with TCP/IP, Volume 1 and 2, by Douglas E. Comer, and David L.

Stevens. Prentice-Hall,

• TCP/IP Illustrated, Volumes 1 and 2 , by W. Richard Stevens. Addison-Wesley, 1994.

• Interconnections , Radia Perlman. Addison-Wesley, 1992.

Recommended for information about routers and bridging.

• Routing in the Internet, by Christian Huitema. Prentice Hall PTR, 1995.

Recommended for information about IP, OSPF, CIDR, IP multicast, and mobile IP.

• TCP/IP Network Administration , by Craig Hunt. O’Reilly & Associates, Inc. 1994.

Recommended for network management information.

• Essential System Administration, Æleen Frisch. O’Reilly & Associates, Inc. 1991.

Recommended for network management information.

xvi October 25, 1999

SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

The RS/6000 SP Switch Router is based on the GRF 400 (4-card) and GRF 1600 (16-card) routers manufactured by Lucent Technologies. For that reason, this manual contains references to the GRF 400/1600 Getting Started , GRF Reference Guide , and

GRF Conﬁguration and Management manuals. The SP model of the router is referred to as the

SP Switch Router. SP Switch Routers can be used to provide high-speed data communication links between IBM

RS/6000 Scalable POWERparallel Systems (SP) and external networks/hosts. The SP Switch Router Adapter card is the router media card that speciﬁcally supports SP system data transfers. To connect to an SP system, the SP Switch Router Adapter card is cabled directly to an SP Switch port.

Material in Chapter 1 provides a basic description of the SP Switch Router Adapter card. Please refer to the GRF 400/1600 Getting Started manual for SP Switch Router system installation procedures.

Chapter 1 covers these topics:

What is the RS/6000 SP Switch Router ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2

SP Switch Router systems for IBM sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3

Upgrading system memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5

Overview of the SP Switch Router Adapter card. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6

Inserting a media card into the SP Switch Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7

SP Switch Router Adapter card LEDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9

SP Switch Router Adapter card speciﬁcations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-13

Assigning ﬁlters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14

SNMP on the SP Switch Router Adapter card. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-15

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-1

Introduction to the SP Switch Router Adapter card

What is the RS/6000 SP Switch Router ?

The RS/6000 SP Switch Router is a high-performance switched IP router designed for high-volume, large-scale public and private backbone applications.

It has these main features:

• Performs Layer-3 switching across 4-16 adapter slots, depending upon router model

• Supports large suite of dynamic routing protocols

• Accommodates multiple types of media, including HSSI, 10/100Base-T (fast Ethernet),

ATM OC-3c, ATM OC-12c, SONET OC-3c, HIPPI, and FDDI

• Provides basic ﬁltering, OSPF multicast, SNMP v1

• Manages 150K-entry route table, batch updating with 20 routes per second As shown in Figure 1-1, the SP Switch Router attaches to the SP Switch through the SP Switch

Router Adapter media card:

SP control workstation

Administrative network =

Ethernet hub or bridge

SP Switch Router

Switch

• • •

SP Switch Router Adapter media card

control board

Processor

node

• • •

Processor

node

SP Switch

to/from other networks and hosts

Figure 1-1. Connections between the SP Switch Router and an SP system

Primary node for SP Switch

Conﬁgured with an IBM SP system, the SP Switch Router provides multiple media LAN and WAN connectivity for the SP. The SP Switch Router Adapter card connects directly to the SP Switch. Other components communicate across the administrative Ethernet network. (While using a hub or a bridge to interconnect the administrative Ethernet segments is common, other network components can be used to provide connectivity between the segments.)

1-2 October 22, 1999

SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SP Switch Router systems for IBM sites

By using the appropriate SP Switch Router Adapter card, the SP system can connect to FDDI, fast Ethernet, HSSI, ATM OC-3c, ATM OC-12c, SONET OC-3c, and HIPPI networks and hosts, depending upon which other media cards are installed in the SP Switch Router chassis.

The SP Switch Router supports these connection options:

– a single SP Switch Router Adapter card can be installed in an SP Switch Router – multiple SP Switch Router Adapter cards can be installed in an SP Switch Router – more than one card can connect to the same SP system – a single SP Switch Router can connect to one or multiple SP systems

Each option requires unique IP addressing and network conﬁguration. Appendix C contains examples of speciﬁc SP system–SP Switch Router networks. See the IBM documentation related to planning for the SP Switch Router for more information.

SP Switch Router systems for IBM sites

The SP Switch Router Adapter media card communicates directly with the SP. In the SP system conﬁguration, the SP Switch Router Adapter media card is treated as a dependent node and is assigned a node number. There is only one node number address space in the SP system, and traditional SP nodes and dependent nodes are both assigned node numbers from that address space.

You may be given references to models 9076 and 9077:

– Model 9076 is the IBM SP system. – Model 9077 04S is the 4-card SP Switch Router. – Model 9077 16S is the 16-card SP Switch Router.

This manual uses SP system and SP Switch Router, respectively, as system names.

Cables included in your system

SP Switch cable

The SP Switch Router Adapter media card connects to an SP Switch via an SP Switch cable. Make sure the shipping box contains one ten- or twenty-meter cable for each SP Switch Router Adapter card you receive. If there is no cable, contact your IBM representative and order the special cable required by the SP Switch Router Adapter media card from IBM:

• SP Switch cable, 10m (IBM P/N 46H9699)

• SP Switch cable, 20m (IBM P/N 46H9701) Although it has 50-pin connector ends, the speciﬁed cable has custom signal wiring so that

other 50-pin cables cannot be substituted.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-3

Introduction to the SP Switch Router Adapter card

SP Switch Router systems for IBM sites

Ethernet cable

An Ethernet 10/100Base-T cable is required for connecting the SP Switch Router to the SP control workstation. It is the customer's responsibility to provide the appropriate cable to make this connection as well as any Ethernet hubs or bridges that may be required to connect to the SP LAN.

SP ground strap

A ground strap is included with the SP Switch Router and must be connected between the designated site on the SP Switch Router chassis and an SP frame. Refer to “ESD requirements” on page 1-8 for a diagram showing the location of router ground strap connections.

PCMCIA 520MB disk

The SP Switch Router shipping box also contains a PCMCIA 520MB spinning disk device. The PCMCIA disk installs in the SP Switch Router’s control board, in the PCMCIA “A” slot. Once installed, the SP Switch Router can be conﬁgured to log and dump locally to the 520MB external storage device.

By default, logging is turned off when the SP Switch Router boots and comes up. After the system comes up, one of the ﬁrst tasks is to conﬁgure the PCMCIA disk. The conﬁguration procedure formats and mounts the external PCMCIA device, and places the required logging pointers. The installation procedure is described in Chapter 2.

Redundant AC power supplies

The SP Switch Router you receive is installed with redundant AC power supplies. You must plug the power supply cords directly into an AC wall or rack receptacle.

Note: The SP Switch Router has no power on/off switch. When you plug the power supply cord into a live outlet, the SP Switch Router powers on and,

since the software is already loaded, immediately begins to boot.

Redundant supply safety

Please note the following when powering off (unplugging) the SP Switch Router unit:

Caution: This unit has two power supply cords. For total isolation from electrical shock and energy hazard, disconnect both supply cords. Care must be taken to correctly connect each power supply to separate AC power sources and (optional) UPS devices.

Vorsicht: Dieses Gerät hat zwei Netzanschlusskabel. Um das Gerät vollstandig v on Netz zu trennen ziehen Sie beide Kabel ab, sonst können Sie einen elektrischen Schlag erhalten. Achten Sie darauf, daß jedes Stromkabel mit einer separaten Wechselstromquelle und einem separaten USV-Gerät verbunden wird.

Attention: Cet appareil a deux cordons d’alimentation électrique. Pour une isolation complète de tout choc électrique et de danger énergétique, débrancher les deux cordons d’alimentation.

1-4 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Upgrading system memory

Figure 1-2 shows the area of system memory (control board RAM) that can be expanded to meet site requirements. Memory upgrades are made in 128MB increments up to 512MB.

Introduction to the SP Switch Router Adapter card

Upgrading system memory

expandable to -->

- system software

- config files

- GateD binary

- log files

- route tables

- ATMP tunnels

- kernel runs

- GateD runs

256MB

RAM

212MB

= expandable area of RAM

--> 384MB RAM

340MB

--> 512MB RAM

468MB

Memory size and

organization

128MB RAM

32MB

(fixed size)

84MB

8-12MB

(fixed size)

Figure 1-2. Expandable area of system memory

This chart provides general guidelines for memory required in different routing environments. Although the ﬁgures assume BGP peers with 50K route entries, additional memory may be required for higher average numbers of routes per BGP peer.

If the SP Switch Router is to support dynamic routing or ATMP home agents and mobile nodes, upgrade to at least 256MB. In environments where large numbers of routes are advertised, upgrade to 512MB.

Customer proﬁle

Amount of control board memory needed

Space for dynamic routing, ATMP tables

Route entries on media card

Route entries in dynamic routing database

Typical numbe r of peer sessions

Static routing: (in high-performance environment)

Small POP 256MB 212MB 150K Typical

Medium POP / ISP backbone

Large POP / Exchange point / Route reﬂection server

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-5

128MB 84MB 150K Typical

number: 35,800

number: 199,000

384MB 340MB 150K Typical

number: 362,000

512MB 468MB 150K Typical

number: 521,000

Introduction to the SP Switch Router Adapter card

Overview of the SP Switch Router Adapter card

The SP Switch Router Adapter media card is cabled to a connector jack on an SP Switch. This media card transfers data to/from the SP Switch at 100 MB/s in each direction.

Like other SP Switch Router media interfaces, the SP Switch Router Adapter media card:

– is intelligent, and performs IP-level routing and route look-ups – provides complete speed-decoupling between the connecting media and the

gigabit/second router switch core

– fully buffers data for input and output queuing; each card has 16MB of high-speed

receive buffer memory and 16MB of high-speed transmit buffers

Face plate diagram

The SP Switch Router Adapter card provides one full-duplex interface.

This illustration shows the faceplate, the interface connector, and the card LEDs.

The actual height of the SP Switch Router Adapter card is 10 inches.

The SP Switch Router Adapter cable 50-pin connector end is also shown.

PWR ON 3V

RX HB RX ST0 RX ST1 RX ERR

MD RCV SW XMIT

1-6 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

TX HB TX ST0 TX ST1 TX ERR

MD XMIT SW RCV

Introduction to the SP Switch Router Adapter card

Inserting a media card into the SP Switch Router

Note:

To operate properly, the 16-card SP Switch Router requires that at least two media cards be installed. The 4-card router requires one card be installed. In both models, a face plate cover must be installed in any unused chassis slot to maintain router cooling ﬂows.

Media cards are actually two logic boards joined to make a single component. As shown in Figure 1-3, the smaller board on the right is the serial interface, also called the serial daughter card. The larger one on the left is the media board and has the network ports. Together they comprise an SP Switch Router media card.

Cards install vertically for 16-card SP Switch Router:

Port

Cards install horizontally level for 4-card SP Switch Router:

Media board

Serial/rev number area

Top Bottom

Serial

daughter

card

Figure 1-3. Media card components

The two logic boards are joined by a pair of 100-pin connectors and reinforcing plates. Even so, this joint retains some ﬂex and must be carefully supported, especially when inserting the media card into the chassis.

Warning: The backplane of the SP Switch Router contains hazardous energy levels. When replacing a media card, remove only one card at a time. Removing more than one card will expose the operator to this energy hazard.

Warnung: An den Rückwandplatinen der SP Switch Router liegen gefährliche Hochspannungen ab. Zum Auswechseln der Medienkarte jeweils nur eine Karte entfernen. Bei zwei gleichzeitig entfernten Karten ist der Bediener gefährlichen Spannungen ausgesetzt.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-7

Introduction to the SP Switch Router Adapter card

Inserting a media card into the SP Switch Router

ESD requirements

Caution: Media cards are hot swappable and can be installed when the SP Switch Router is

running. However, media cards are highly susceptible to damage from electrostatic discharge. You must wear a grounded, conductive wrist strap any time you handle a media card. Make sure the metallic elements in the band directly touch your exposed skin.

SP Switch Router - 16 card

Card insertion procedure

1 When you are properly grounded, remove the media card from its anti-static container. 2 Hold the media card with the network ports facing you.

4-card router

Turn the card horizontal, the top of the media card should be on the left, the bottom of the card should be on the right. As you start, make sure you visually identify the left and right guide pair for this particular slot.

Keeping the media card horizontally level, insert the card fully into the slot, you will feel the card joining with the 100-pin connector on the backplane.

SP Switch Router - 4 card

Wrist strap grounding sites

16-card router

Hold the card vertically. As you start, make sure you visually identify the top and bottom guide pair for this particular slot. Have one hand under the card, lightly supporting its weight. Rest just the edge of the bottom corner of the card in the bottom guide. Then, bring the top edge of the card into the top guide. This will help you keep the card level as you slide it in.

Keeping the media card vertically upright, insert the card fully into the slot. You will feel the card joining with the 100-pin connector on the backplane.

3 When fully inserted, the card’s face plate should be ﬂush against the chassis back panel.

Note: Do not force the card into the slot. Doing so can damage the card or slot connector. 4 Tighten the screws at each end of the face plate.

1-8 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card LEDs

The “RX” and “TX” LEDs are under software control and indicate port states on receive and transmit sides. The “MD” and “SW” LEDs are hardware-controlled and reﬂect data activity on the SP Switch Router switch core or interface side of receive and transmit ports.

“Top” end of card “Bottom” end of card

RX HB

RX ST0

RX ST1

PWR ON

Figure 1-4. LEDs on the SP Switch Router Adapter card

LED activity during boot

RX ERR

SW XMIT

MD RCV

During boot and resets, the four software-controlled LEDs indicate different media card activities by ﬂashing in speciﬁc patterns. Refer to Table 1-1 for a description of each pattern.

Table 1-1. SP Switch Router Adapter card LED activity during boot and reset

TX HB

TX ST0

TX ST1

TX ERR

SW RCV

MD XMIT

LED Description

• ON At reset, all LEDs are lit for 1/2 second

• RX HB (green)

• RX ST0 (green)

• RX ST1 (amber)

• RX ERR (amber)

• ON as part of on-board diagnostics.

• ON Also tests that LEDs are working.

• ON

• OFF ERROR - During a boot or reset, this pattern indicates

• OFF a checksum error is detected in

• OFF ﬂash memory.

• ON

• ON ERROR - During a boot or reset, this pattern indicates

• OFF that the SRAM fails the memory test.

• ON

• OFF

• ON -> OFF - During loading, HB and RX ST1 ﬂash

• OFF as each section of the code loads.

• OFF

• ON -> OFF

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-9

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card LEDs

Table 1-1. SP Switch Router Adapter card LED activity during boot and reset (continued)

LED Description

• ON At reset, all LEDs are lit for 1/2 second

• TX HB (green)

• TX ST0 (green)

• TX ST1 (amber)

• TX ERR (amber)

• ON as part of on-board diagnostics.

• ON Also tests that LEDs are working.

• ON

• OFF ERROR - During a boot or reset, this pattern indicates

• OFF a checksum error is detected in

• OFF ﬂash memory.

• ON

• ON ERROR - During a boot or reset, this pattern indicates

• OFF that the SRAM fails the memory test.

• ON

• OFF

• ON -> OFF - During loading, HB and TX ST1 ﬂash

• OFF as each section of the code loads.

• OFF

• ON -> OFF

1-10 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

LED activity during normal operations

Refer to Table 1-2 for a description of SP Switch Router Adapter card LED activity during normal run time operations.

Table 1-2. SP Switch Router Adapter media card LEDs

LED Description

PWR ON This green LED is on when 5 volts are present. Both power LEDs,

5V and 3V, can be on simultaneously. 3V This green LED is on when 3 volts are present. RX HB (green) During normal run-time operations, this green LED blinks a

“heartbeat” pattern for the receive side CPU. The beat is a long off,

short on-off-on pattern .

In the pattern, the LED goes off for 1/2 second, comes on for 1/4

second, goes off for 1/4, comes on for 1/4, and then begins the

pattern again by going off for 1/2 second.

SP Switch Router Adapter card LEDs

• ON STATE_0 - These three LEDs are on during

• RX ST0 (green)

• RX ST1 (amber)

• RX ERR (amber)

These three LEDs light in different combinations to indicate ﬁve operating states for the receive port.

MD RCV (amber) This amber LED lights when data comes into the receive media port

SW XMIT (amber) This amber LED lights when the receive media port sends data to the

• ON hardware initialization.

• ON

• OFF STATE_1 - Bottom two amber LEDs go on during

• ON software initialization, show receive port

• ON is waiting for conﬁguration parameters.

• ON STATE_2 - Middle amber LED goes off when

• OFF conﬁguration parameters are in place and the

• ON receive port is ready to be connected.

• OFF STATE_3 - Bottom amber LED goes on to show that

• OFF the receive port is connected and the card is

• ON ready to be on line

• OFF STATE_4 - These three LEDs are off to show receive

• OFF port is online and running/routing.

• OFF

from an external source.

SP Switch Router switch core (via the serial daughter card). TX HB (green) This green LED blinks a “heartbeat” pattern for the transmit side

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-11

CPU, the beat is a long off, short on-off-on pattern during normal run

time operations.

In the pattern, the LED goes off for 1/2 second, comes on for 1/4

second, goes off for 1/4, comes on for 1/4, and then begins the

pattern again by going off for 1/2 second.

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card LEDs

Table 1-2. SP Switch Router Adapter media card LEDs (continued)

LED Description

• ON STATE_0 - These three LEDs are on during

• TX ST0 (green)

• TX ST1 (amber)

• TX ERR (amber)

These three LEDs light in different combinations to indicate ﬁve operating states for the transmit port.

• ON hardware initialization.

• ON

• OFF STATE_1 - Bottom two LEDs (amber) go on during

• ON software initialization, show transmit port

• ON is waiting for conﬁguration parameters

• ON STATE_2 - Middle LED goes off when

• OFF conﬁguration parameters are in place and the

• ON transmit port is ready to be connected.

• OFF STATE_3 - Bottom LED (amber) goes on to show that

• OFF the transmit port is connected and the card is

• ON ready to be on line.

• OFF STATE_4 - These three LEDs are off to show transmit

• OFF port is online and running/routing.

• OFF

MD XMIT (amber) This amber LED comes on as data leaves the transmit media side

going to an external destination. SW RCV (amber) This amber LED lights when data from the SP Switch Router switch

core (via the serial daughter card) goes to the transmit media side.

Note: The MD RCV, SW XMIT, MD XMIT, and SW RCV LEDs increase in brightness with increasing data trafﬁc. When there is little data trafﬁc, it may be difﬁcult to see that the LED is blinking.

1-12 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card specifications

SP Switch Router Adapter card speciﬁcations

Refer to Table 1-3 for SP Switch Router Adapter media card characteristics:

Table 1-3. SP Switch Router Adapter media card speciﬁcations

Element Value

Attachment density One full-duplex interface Media transfer rate 100 megabytes per second Processors 40 MHz SPARC, one transmit, one receive Data buffers 16MB input, 16MB output Route table support 150K entries Max transmission unit The default MTU is 65520 bytes Card connector 2-row, 50-pin panel-mount receptacle Cable connector 2-row, 50-pin shielded tab connector Cables Twisted-pair copper, 10- or 20-meter length, available from IBM Power consumption Approximately 50 watts per media card

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-13

Introduction to the SP Switch Router Adapter card

Assigning filters

Assigning ﬁlters

The SP Switch Router Adapter card supports IP packet ﬁltering. You can apply ﬁlters to the receive and/or transmit path of a logical interface as described in the “IP Packet Filtering” chapter of the GRF Conﬁguration and Management manual.

The ﬁlter conﬁguration ﬁle is /etc/filterd.conf. The maint 50 – 58 commands report statistics and information for ﬁlters assigned to the receive side of the card. The maint 150 – 158 commands report on transmit side ﬁlters.

The “IP Packet Filtering” chapter describes the entries in /etc/filterd.conf and tells you how to design several types of ﬁlters. The binding statement in /etc/filterd.conf is where you assign a ﬁlter you have created to a particular logical interface on a speciﬁc media card. This statement has two variables that are media card speciﬁc,

In a binding statement, media is the type of media card and vlif is the logical interface number to which the ﬁlter is assigned. For the SP Switch Router Adapter card, media is always dev1 and the vlif is always 0 since the card has a single interface.

Here is a binding statement for an SP Switch Router Adapter card in slot 5, gt050 (the card is connected to node 8 on an SP switch):

media and vlif.

tcpdump

media dev1 5 { # bind no_host_22_22 { vlif 0; # this is the switch node 8 interface direction out; # outbound trafﬁc to node 8 action filter; } }

the ﬁlter named “no_host_22” blocks all packets from remote host 192.168.22.22

Here are the supported media names:

atm (OC-3c) dev1 ether fddi hssi hippi sonet

(OC-3c)

Please refer to the “IP Packet Filtering” chapter of the GRF Conﬁguration and Management manual for conﬁguration information and examples.

Filtering supports the standard UNIX tcpdump utility that enables you to examine the data crossing an SP Switch Router Adapter interface. A tcpdump “listen” command for interface

gt030 is:

# tcpdump -i gt030

1-14 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SNMP on the SP Switch Router Adapter card

This section describes the SNMP implementation on the SP Switch Router Adapter card as a way of providing information for staff supporting the card from the SP control workstation. Chapter 2 describes the actual conﬁguration procedure performed on the SP Switch Router.

Although the mib2d daemon within the SP Switch Router supports several MIBs, the SET command is supported only for the SP Switch Router Adapter card MIB, ibmSPDepNode.

For SP Switch Router Adapter cards speciﬁcally, mib2d creates a table of

ibmSPDepNodeEntry MIB objects, one entry for each possible media card slot.

SNMP write access mode is supported for SP Switch Router Adapter conﬁguration parameters and for an object representing the Administrative state of the adapter.

When a SET command is received for an object deﬁned in the ibmSPDepNode MIB, the

/etc/grdev1.conf

Switch Router Adapter card’s run-time software reports board status to mib2d.

For more information about the use of SNMP to conﬁgure the SP Switch Router Adapter card, please refer to the “Managing Extension Nodes” chapter in the PSSP Administration Guide.

SP Switch Router Adapter dependent node MIB support

conﬁguration ﬁle is also updated with the newly-set value. The SP

SP Switch Router Adapter MIB support complies with the dependent node MIB deﬁnition and

provides these objects:

– ibmSPDepNode – ibmSPDepNodeTable – ibmSPDepNodeEntry – ibmSPDepNodeName – ibmSPDepNodeNumber – ibmSPDepSwToken – ibmSPDepSwARP – ibmSPDepSwNodeNumber – ibmSPDepIPaddr – ibmSPDepNetMask – ibmSPDepIPMaxLinkPkt – ibmSPDepIPHostOffset – ibmSPDepConfigState – ibmSPDepSysName – ibmSPDepNodeState – ibmSPDepSwChipLink – ibmSPDepNodeDelay – ibmSPDepAdminStatus

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-15

Introduction to the SP Switch Router Adapter card

SNMP on the SP Switch Router Adapter card

The object ibmSPDepNodeName serves as an index for the ibmSPDepNodeTable and is initialized to a constant text string equivalent to the corresponding chassis slot number: 00–15.

The slot numbers support SP Switch Router chassis with 4 or 16 card slots. The objects ibmSPDepConfigState, ibmSPDepNodeState, and ibmSPDepNodeName are

read-only. At start up, ibmSPDepConfigState is set to 1 (not-conﬁgured), and is changed as the SP Switch Router Adapter media card state changes.

SP Switch Router Adapter media card states (SNMP)

These states are available as possible instance values for the SNMP ibmSPDepConfigState object, but are not directly viewed by the user:

1 - notConfigured (card is initialized) 4 - diagnosticFailed (card’s own on-board diagnostics fail) 5 - microcodeLoadFailed (card is waiting for valid, not null, conﬁguration parameters) 6 - fullyConfigured (card is initialized, conﬁgured, ready to be brought on line

with the SP)

1-16 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

SNMP conﬁguration overview

After the SP Switch Router is powered on and booted, the network administrator logs on to conﬁgure the router as a system. The following is a description of the steps that can be taken during router conﬁguration. The actual procedure is described in Chapter 2.

The conﬁguration for the snmpd daemon must be updated to identify the SP SNMP Manager(s) that will conﬁgure and maintain status of the SP Switch Router Adapter card. The community name to be used for communications with the SP SNMP Manager must be the same community name that is speciﬁed in the Extension Node conﬁguration data on the SP control workstation where the SP SNMP Manager resides.

If no community name is speciﬁed on the SP control workstation, specify spenmgmt as the community name for the SP SNMP Manager. The community name must allow read, write, and trap capabilities. All community names used to communicate with managers other than the SP SNMP Manager(s) should be restricted to read-only and trap capabilities. Traps should be sent to the same UDP port on which the SP SNMP Manager is listening. This port will most often be port number 162.

Using a UNIX editor, the administrator edits the conﬁguration ﬁles required for each type of media card. Conﬁguration parameters for the SP Switch Router Adapter card could also be entered in the SNMP. Save this ﬁle using the grwrite command.

/etc/grdev1.conf ﬁle at this time if the card is not going to be conﬁgured via

Introduction to the SP Switch Router Adapter card

SNMP on the SP Switch Router Adapter card

The administrator now resets the SP Switch Router system to actually install the conﬁguration parameters. During a reset, system daemons restart and reread their ﬁles. The media cards also boot, loading their software, conﬁguration information, and the current route table.

Each time the SP Switch Router software boots, mib2d starts up. Unless the administrator has already entered SP Switch Router Adapter conﬁguration values in /etc/grdev1.conf, ﬁle parameters for SP Switch Router Adapter cards will all contain null values.

mib2d generates and sends a coldStart/warmStart trap message to all SNMP Managers conﬁgured. It creates and initializes its MIB object instances. To support SET commands for any instances of the SP Switch Router Adapter conﬁguration objects, mib2d creates an

ibmSPDepNodeTable . The table contains an ibmSPDepNodeEntry MIB entry (there are 16

of these) for each available media card slot in various SP Switch Router models.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-17

Introduction to the SP Switch Router Adapter card

SNMP on the SP Switch Router Adapter card

SNMP activity during media card start up

After the SP Switch Router software boots, the media cards boot and load their boot diagnostics. The SP Switch Router Adapter media card runs its diagnostics as a check for hardware defects. If no failure is detected, the card’s run-time software is loaded. The screen displays diagnostic and boot reports from all the media cards, interleaved as received. If a hardware problem is found, the diagnostic forwards a ConfigState trap request to mib2d. In turn, mib2d sends a switchConfigState trap message (with ConfigState=diagnosticFailed) to the SP SNMP Manager. The SP Switch Router Adapter card continues to execute its self-test software until the card is powered off (removed from the chassis).

Note: The execution of diagnostics at boot time is an option. The default is for diagnostics to run each time the SP Switch Router Adapter card boots.

After an SP Switch Router Adapter card successfully loads its run-time software, the card requests its conﬁguration parameters. After the run-time software veriﬁes the parameters are valid (not null), the card is ready to begin normal operation.

If the conﬁguration parameters contain null values, the card informs mib2d that the conﬁguration parameters have not been sent via a trap request message. mib2d sends the SP SNMP Manager a switchConfigState trap message (ConfigState=microcodeLoadFailed) and also a switchInfoNeeded trap message.

The card remains in this state (5, microcodeLoadFailed) until it receives v alid conﬁguration parameters, or until the card is reset. The card periodically requests conﬁguration parameters and sends trap request messages to mib2d.

If the SP SNMP Manager is conﬁgured and operational, it responds to the mib2d trap messages by sending SET commands to put the parameters in place. If the SP SNMP Manager does not respond, there may be a conﬁguration error that can be detected using the procedures for diagnosing dependent node conﬁguration problems in the PSSP Diagnosis Guide.

As a last resort, the network administrator can perform the following operations to install the updated parameters on the SP Switch Router Adapter card:

– run the dev1conﬁg command – edit /etc/grdev1.conf as required – use grreset slot to reset the SP Switch Router Adapter media card

The SP Switch Router Adapter card can receive valid parameters either way. It does not begin normal operation until it is brought on line with the SP system as the IP router interface.

Refer to the section in Chapter 2 on “Bringing the SP Switch Router Adapter card on-line with the SP” for a continuation of the start up scenario discussed here.

1-18 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Conﬁguring the SP Switch Router Adapter

This chapter describes conﬁguration and installation tasks required to connect an SP Switch Router Adapter media card to an IBM SP System.

The RS/6000 SP Switch Router is based on the GRF 400 and GRF1600 routers manufactured by Lucent Technologies. For that reason, this manual contains references to the

GRF 400/1600 Getting Started, GRF Reference Guide, and GRF Conﬁguration and Management manuals. The SP model of a GRF router is referred to as the SP Switch Router .

For more information about conﬁguration as related to the SP, see the PSSP Administration Guide and the PSSP Command and Technical Reference. For additional information on

troubleshooting your conﬁguration, see the PSSP Diagnosis Guide.

Chapter 2 covers these topics:

Introduction to installation and conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2

Installing an SP Switch Router Adapter card. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5

Installing the PCMCIA spinning disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6

Attaching SP Switch Router cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10

Conﬁguration required on the SP system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12

Step-by-step media card conﬁguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-16

Step 1. Check SNMP in the SP Switch Router system . . . . . . . . . . . . . . . . . . . . . . . . 2-18

Step 2. Assign IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-21

Step 3. Change proﬁle settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24

Step 4. Run dev1conﬁg to create grdev1.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-31

Step 5. Reset card to install ﬁles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35

Verify SP Switch Router Adapter card from router . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-36

Bringing the SP Switch Router Adapter card on-line with the SP. . . . . . . . . . . . . . . . 2-40

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-1

Configuring the SP Switch Router Adapter

Introduction to installation and configuration

Introduction to installation and conﬁguration

The SP Switch Router functions as an IP router to provide high-speed data communication links between SP processor nodes and external networks/hosts. The SP Switch Router Adapter media card connects to the SP Switch board in an SP system as shown in Figure 2-1.

SP control workstation

Administrative network =

Ethernet hub or bridge

SP Switch Router

Switch

• • •

SP Switch Router Adapter media card

control board

Processor

node

• • •

Processor

node

Primary node for SP Switch

SP Switch

to/from other networks and hosts

Figure 2-1. Components connecting an SP Switch Router to an SP Switch and control

workstation

The SP Switch Router Adapter card also transmits data to/from other types of media cards across the SP Switch Router’s internal switch core. These media include HIPPI, HSSI, FDDI, ATM OC-3c, ATM OC-12c, SONET OC-3c, and 100Base-T (Fast Ethernet).

The SP system manages the SP Switch Router Adapter card as a dependent node, under the control of the SP SNMP Manager running on the SP control workstation and the primary node of the SP Switch.

Once powered on and started up, the SP Switch Router can be conﬁgured and managed remotely, via a site’s administrative network, including T elnet from the SP control w orkstation. Information about procedures performed from the SP control workstation are found in the “Managing Extension Nodes” chapter in the PSSP Administration Guide.

2-2 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Location of relevant information

The intent of this chapter is to either provide or refer you to the necessary information to enable you to attach an SP Switch Router to an IBM SP system, including:

• Information to physically connect the two independent systems across cables is complete in this chapter.

• Information to start up, conﬁgure, and begin operations on the SP Switch Router is contained in the GRF 400/1600 Getting Started manual.

• Information to conﬁgure the SP Switch Router Adapter card as required for SP Switch Router functionality is complete in this chapter.

• Information to conﬁgure the SP Switch Router Adapter card as required for SP system functionality is only partially described in this chapter. Detailed information is contained in the “Managing Extension Nodes” chapter in the PSSP Administration Guide.

Pre-installation assumptions

The presentation of information in this chapter assumes the following:

• The SP Switch Router is powered on and has a VT-100 terminal or administrative Ethernet network connected to its control board.

• The SP Switch Router’s basic system parameters, primarily IP address and host name, have been conﬁgured during the ﬁrst time power-on conﬁguration script.

You use the terminal or network to log in to the SP Switch Router system and enter those basic conﬁguration parameters. Procedures for starting and setting up the SP Switch Router are found in the GRF 400/1600 Getting Started manual.

• Remote T elnet access is enabled. This requires editing the /etc/ttys ﬁle and creating an entry for each remote session needed. Refer to Chapter 2 in the GRF Conﬁguration and Management manual.

Use the following command when telneting to the SP Switch Router from an X terminal:

xterm -sb -e tn <

This command ensures the screen output of the SP Switch Router is displayed correctly.

• The SP Switch Router’s operating parameters, such as SNMP and IP addressing, have been conﬁgured and the system rebooted so it is up and operating. Procedures to conﬁgure operating parameters are found in the GRF Conﬁguration and Management manual, Chapter 2.

Descriptions of conﬁguration and system management commands you may need are found in the GRF Reference Guide.

• You are ready to conﬁgure media cards. Procedures to conﬁgure media cards other than the SP Switch Router Adapter card are found in the GRF Conﬁguration and Management manual

• The IBM SP system is up and operating.

• The SP system administrator has given you one of these pieces of information:

hostname_of_SP_Switch_Router

Configuring the SP Switch Router Adapter

Introduction to installation and configuration

– the node number assigned to each SP Switch Router Adapter card to be attached to an

SP Switch port

or – the port location on each SP Switch reserved for speciﬁc SP Switch Router Adapter

cards

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-3

Configuring the SP Switch Router Adapter

Introduction to installation and configuration

Order of information

Here is the order in which installation information is presented:

– an installation overview of tasks involving the SP Switch Router, the SP Switch

Router Adapter card, and the SP system

– the conﬁguration procedure for the PCMCIA 520MB disk, which also initiates

system logging

– a description of which cables to attach between the SP Switch Router and the SP

control workstation, and between the SP Switch Router Adapter card and the SP Switch

– methods to determine node number and SP Switch port for an SP Switch Router

Adapter card – a step-by-step conﬁguration of an SP Switch Router Adapter card – a list of ways to verify that the SP Switch Router Adapter card is correctly installed in

the SP Switch Router – a description of what needs to occur to bring the card on-line with the SP system

2-4 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Installing an SP Switch Router Adapter card

This section contains the procedure for physical installation and minimal conﬁguration of the SP Switch Router Adapter card for use as an SP dependent node. This includes cabling the card to the SP control workstation and the appropriate SP switch port.

Note: There must be an Ethernet twisted-pair connection between the SP Switch Router control board and the SP control workstation. This is normally done through an Ethernet hub .

Installation overview

IBM support personnel who install the SP Switch Router (9077) will perform the physical installation and minimal conﬁguration described below with help from the customer's system administrator. The system administrator must provide basic conﬁguration information to complete the steps in this procedure.

1 Locate all the components of the SP Switch Router ship group. 2 Perform the complete physical installation of the SP Switch Router unit as described in the

“Power On and Initial Conﬁguration” chapter of the GRF 400/1600 Getting Started manual.

Make sure that when the “First-time power on conﬁguration script” runs at system boot, the required conﬁguration information is provided by or entered by the customer. This information includes the SP Switch Router unit IP address and host name.

3 Perform the procedure to conﬁgure the PCMCIA 520 MB disk. The procedure is included

in this chapter.

4 Route the Ethernet twisted-pair cable between the SP Switch Router unit and the Ethernet

hub, then connect the cable to the SP Switch Router control board and to the Ethernet hub.

5 Verify that the SP control workstation has a connection to this same Ethernet hub.

If the SP control workstation Ethernet adapter is conﬁgured by the system administrator, then a ping test from the SP control workstation to the conﬁgured SP Switch Router Ethernet address can be done to test Ethernet connectivity.

Physical installation and minimal conﬁguration will be complete at this point.

Review the “Attaching SP Switch Router cables” section in this chapter before connecting the SP Switch Router Adapter card cables to the SP switch ports speciﬁed for this conﬁguration.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-5

Configuring the SP Switch Router Adapter

Installing the PCMCIA spinning disk

Your system is shipped with a PCMCIA disk device that is required to collect the system log ﬁles. This disk can hold up to 520MB of data.

You can install the disk any time after the SP Switch Router is powered on and is running. Logging is not enabled until you install the disk and complete this conﬁguration procedure. Logged messages can be helpful while you are conﬁguring media cards.

The conﬁguration is done only once to set up local logs and dumps, and is not affected by software updates or system reboots.

Note that the disk is used only for storage. You cannot boot the router from an external device. Three logs provide speciﬁc information useful for monitoring and debugging SP Switch

Router operations. If you are working with Customer Support, these are the three logs they will need to see:

– /var/log/gr.console

– /var/log/messages – /var/log/gr.boot

The /var/log directory contains other log ﬁles that collect low-level information useful primarily to system developers.

The procedure formats and initializes an external device (/dev/wd2a), temporarily mounts it on /mnt, creates subdirectories and symbolic links, and creates a permanent site ﬁle for storing the symbolic links.

Note that the iﬂash command can be used with a -f option that forces any data on the target device to be overwritten. When you use iﬂash without -f, you are informed if there is a ﬁle system already on the device and reminded that you must use the -f option to overwrite it. Because of its “force” capability, use the iﬂash -f command with caution.

The /var/portcards directory only contains media card dump ﬁles. These include the dumps from media card panics and dumps created when automatic dumping is selected via the grreset -D command (media card dumps when it comes back up). The /var/crash directory contains dumps from BSD kernel crashes.

Managing PCMCIA slots

Two commands enable remote management of PCMCIA slots. The csconﬁg slot_number command returns status while csconﬁg slot_number up and csconﬁg slot_number down mark the speciﬁed PCMCIA slot up or down, respectively.

2-6 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Panic dumps sent to external ﬂash device

The mountf and grdump commands enable the grdump program to work with an external ﬂash device to capture media card dumps.

When a media card panics and there is a formatted external ﬂash device plugged into a PCMCIA slot, a copy of the dump is automatically saved to the external ﬂash in a directory called /var/portcards.

Installation steps

1 Insert the PCMCIA disk into slot A on the SP Switch Router control board (the thickness

of the 520MB device requires it be installed in slot A).

2 Log in as root to the SP Switch Router, start the UNIX shell, and execute these

commands from the shell:

prompt> sh # # cd / # iflash -A # grsnapshot --dup PA # mountf -A -w -m /mnt # mkdir /mnt/crash # mkdir /mnt/portcards # cd /var # mv crash crash.orig # mv portcards portcards.orig # ln -s /var/log/portcards /var/portcards # ln -s /var/log/crash /var/crash # grsite --perm portcards crash # cd /var/log # pax -rw -pe -v . /mnt # umountf -A

Installing the PCMCIA spinning disk

Note:

The grsnapshot --dup command places a copy of the internal ﬂash boot and conﬁguration ﬁles on the spinning disk. Customer Support may use these ﬁles to rebuild internal ﬂash if a problem occurs.

3 Edit the ﬁle /etc/fstab and add this line as shown in the excerpt below:

/dev/wd2a /var/log ufs rw 0 2 #PCMCIA slot A, use wd3a for B

# Filesystem mount table information. See the fstab(5) man page # and the /etc/fstab.sample file for more information and examples. # # Each line is of the form: # device mount_point type flags dump fsck_pass # # Note that multiple flags (when used) are specified as a # comma separated list without spaces. #

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-7

Configuring the SP Switch Router Adapter

Installing the PCMCIA spinning disk

# Blank lines and lines beginning with `#' are comments. # /dev/rd0a / ufs rw 0 0 /dev/wd2a /var/log ufs rw 0 2 #PCMCIA slot A, use wd3a for B

4 Edit the ﬁle /etc/syslog.conf to specify the location where the logs will be kept.

Uncomment the local log conﬁguration lines in the “Log messages to Disk” section by removing #disk# from each line and specify /var/log as the directory for each log:

These are the ﬁrst four lines in the section:

#disk#*.err;*.notice;kern.debug;lpr,auth.info;mail.crit /var/log/messages #disk#

#disk# #disk#

cron.info /var/log/cron local0.info /var/log/gritd.packets local1.info /var/log/gr.console

The ﬁle entries should now look like the following:

*.err;*.notice;kern.debug;lpr,auth.info;mail.crit /var/log/messages cron.info /var/log/cron local0.info /var/log/gritd.packets local1.info /var/log/gr.console local2.* /var/log/gr.boot local3.* /var/log/grinchd.log local4.* /var/log/gr.conferrs local5.* /var/log/mib2d.log

If you had previously conﬁgured your SP Switch Router to log messages to a directory other than /var/log, you changed settings in /etc/grclean.conf and

/etc/grclean.logs.conf ﬁles.

Go back into those ﬁles now and change the log directory.

5 To install your changes, restart syslog:

Determine the PID (process ID) for the syslog daemon and then use the kill -HUP PID command to restart it:

# ps -ax | grep syslogd

# kill -HUP

PID

6 Modify /etc/grclean.conf and /etc/grclean.logs.conf to reﬂect the new log

directory. The /etc/grclean.conf ﬁle speciﬁes which log and dump ﬁles the grclean program

compresses, archives, and deletes. The /etc/grclean.conf ﬁle entries should look like the following:

################################################################# # port card dump files. #################################################################

2-8 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Installing the PCMCIA spinning disk

hold=4 size=1 remove=y local=y logfile=/var/portcards/grdump.* ################################################################# # cleanup our own log file, if necessary. ################################################################# DEFAULTS hold=2 local=y size=10000 logfile=/var/log/grclean.log

The /etc/grclean.logs.conf ﬁle is used to set size limits on log ﬁles. Here are some sample entries:

***************************************************************** * Log files that used to be archived by the */etc/{daily|weekly|monthly} scripts. ***************************************************************** size=150000 logfile=/var/log/gr.console size=11000 logfile=/var/log/gr.boot

7 Save all changes and reboot:

# grwrite -v # reboot -i

8 Verify that the PCMCIA interface and device are up:

# csconfig -a Slot 0: flags=0x3<UP,RUNNING> Attached device: wdc2 Manufacturer Name: "Kingston Technology" Product Name: "DataPak 520" Function ID: 4 (PC card ATA) Assigned IRQ: 11 Assigned I/O port1: 0x3d0-0x3df Slot 1: flags=0x5<UP,EMPTY>

9 To run a quick test, execute the grconslog command. If the command runs, these steps

have been performed correctly:

# grconslog

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-9

Configuring the SP Switch Router Adapter

Attaching SP Switch Router cables

Three types of cables must be attached:

– the administrative Ethernet LAN cable – the SP Switch Router Adapter card–SP Switch cable(s) – the ground strap to the SP frame

Ethernet cable

Route the Ethernet twisted-pair cable between the SP Switch Router unit and the Ethernet hub, then connect the cable to the SP Switch Router control board and to the Ethernet hub. (While using a hub or a bridge to interconnect the administrative Ethernet segments is common, other network components can be used to provide connectivity between the segments.)

SP control workstation

SP Switch Router

Figure 2-2. SP system administrative Ethernet connections

SP switch cable

The SP Switch Router Adapter card provides one full-duplex attachment and requires a speciﬁc cable with 50-pin connector ends obtainable from IBM. The cable has a unique signal wiring map, and is not replaceable by a 50-pin HSSI cable, for example.

Figure 2-3 illustrates the cables 50-pin connector end.

Figure 2-3. SP Switch Router Adapter cable 50-pin connector end

Control board

Hub

Administrative Ethernet network

SP Switch Router Adapter card cables are available in 10- and 20-meter lengths (32 or 65 feet).

Excess cable lengths should be bound in a ﬁgure-eight pattern. Do not wind excess cable into circular coils.

Do not damage the connector ends

Each connector end has 50 fragile pins. Pins can become bent while making the connection to the media card if alignment is wrong. If an SP Switch Router Adapter card link does not work after cabling, check both ends of the cable for bent pins.

2-10 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Attaching SP Switch Router cables

Keep the plastic cap on

Cables ship with the connector pins protected by a plastic cap. Keep this cap on while you lay out the cabling. Remove the cap only when you are ready to plug in the connector.

Procedure for attaching cables to card and SP Switch

This procedure connects the SP Switch Router Adapter card(s) to the SP Switch. Before the SP Switch Router unit can begin full operation, all other router media cards must be conﬁgured with appropriate customer conﬁguration information.

Be careful with the cable ends. Because they use high-density pins, cable ends are susceptible to physical damage if not handled correctly.

Warning: A connector can build up enough charge to disable the media card. One way to discharge a cable is to run your ﬁnger gently over the tips of the pins, touching pins and the connector shell at the same time.

Warnung: Es kann sich genug Ladung in einem Stecker ansammeln, daß die Datenträgerkarte deaktiviert wird. So wird das Kabel entladen: Fahren Sie mit dem Finger vorsichtig über die Spitzen der Stifte; berühren Sie dabei gleichzeitig die Stifte und das Steckergehäuse.

Make sure you have identiﬁed and labeled the SP Switch cable as to which media card and SP Switch port it will be connected to.

1 If there are any terminators on the media card or the switch assembly where you need to

attach the switch cable, remove them now.

2 Using appropriate frame entry and exit holes for cable management, route the SP Switch

cable between the SP Switch Router unit and the SP Switch.

3 Connect the SP Switch cable to both the media card and the correct SP Switch port:

Connection to media card

The EMI shielding ﬁtted inside the connector end can make insertion difﬁcult. Insert the connector end very nearly perpendicular to the card. Pins can be damaged when the connector is inserted at too much of an angle. Seat the connector ﬁrmly so the spring clips are engaged.

Connection to SP Switch port

The cable ends should click onto the connectors. Determining the correct switch port is described in the section, “Determining the switch

connection for a dependent node.”

4 Make sure there is positive retention at both ends of the cable by pulling lightly on the

cable.

At this point, the SP Switch Router Adapter card conﬁguration information must be entered on the SP control workstation to enable the PSSP code and SP Switch to recognize the adapter. These tasks are discussed in the “Conﬁguration required on the SP system” section.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-11

Configuring the SP Switch Router Adapter

Configuration required on the SP system

Conﬁguration required on the SP system

This section describes the SP Switch Router-related conﬁguration information that should be deﬁned by the SP administrator and then entered from the SP control workstation before conﬁguring the SP Switch Router Adapter card.

The SP Switch Router-related conﬁguration information includes:

– SP Switch Router Ethernet IP address – SP Switch Router Ethernet host name

(this host name is the SP Switch Router’s administrative Ethernet host name) – unique node number(s) for SP Switch Router Adapter card(s)

The SP Switch Router and Adapter card conﬁguration information enables the PSSP code and the SP Switch to recognize and communicate with this card.

Determining the switch connection for a dependent node

The SP Switch Router Adapter connection replaces an SP node connection to the SP Switch. Each SP Switch Router Adapter media card is referred to as a dependent node, and is assigned a node number that corresponds to its speciﬁc connection on the SP Switch.

The node number is determined by the SP system administrator based on an understanding of how node numbers are assigned in the SP system and the rules for choosing a valid, unused SP Switch port. The rules are described in the conﬁguration planning section of the PSSP Planning, Volume 2, Control Workstation and Software Environment manual.

If proper planning has been done to assign the node number, the system administrator will know which SP frame, switch board, and node slot corresponds to a dependent node. Given this information, you can determine which jack on the switch board should be used by consulting the “Switch Cable Charts” for the SP Switch in the PSSP Maintenance Information Volume 1, Installation and Customer Engineer Operations manual.

You should not attempt to connect an SP Switch Router Adapter to the SP Switch until proper planning has been done to assign the node number.

Once the node number is assigned, the SP system administrator can deﬁne the corresponding dependent node using SMIT as described in the “Managing Extension Nodes” section of the PSSP for AIX: Administration Guide.

After deﬁning new dependent nodes on the SP, the administrator should use the Eannotator command to annotate the SP switch topology ﬁle. With the ﬁle annotated, even if the administrator is not sure of the frame, switch board, or node slot for the dependent node, you can determine the corresponding switch connection with the following procedure.

2-12 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Procedure

Configuring the SP Switch Router Adapter

Configuration required on the SP system

1 From the SP control workstation, determine the switch_node_number by entering:

SDRGetObjects DependentNode node_number==n

where n is the node_number of the dependent node. This command produces output that looks like:

node_number switch_node_number ...

47 31 ...

2 From the SP control workstation, determine the host name of the switch primary node by

entering:

splstdata -s

This command produces output that lists information about nodes on the SP switch. It includes the following data and identiﬁes the primary node host name:

switch_port topology primary arp switch_node number filename name enabled nos._used

---------------------------------------------------------------- 1 top.7 ournode1 no yes

In this case, the primary node host name is "ournode1".

3 Log into the primary node by entering:

tn node_hostname

where node_hostname is the host name of the primary node.

4 From the primary node, enter:

pg /var/adm/SPlogs/css/out.top

5 Within the /out.top ﬁle, look for the lines containing "tb3". tb3 is immediately

followed by the value for the switch_node_number.

For switch_node_number 31, the following line identiﬁes the SP switch jack E02-S17-BH-J15 (Frame 2, switch, bulkhead, jack J15):

s 27 3 tb3 31 0 E02-S17-BH-J15 to E03-N47 # Dependent Node

If you need help interpreting this identifying string, see the PSSP Maintenance Information, Volume 2, Maintenance Analysis Procedures and Parts Catalog for an

explanation of the naming standard for RS/6000 SP components.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-13

Configuring the SP Switch Router Adapter

Configuration required on the SP system

Sources of conﬁguration information

Here are several ways to determine the node number and other conﬁguration data for a particular card.

• Check the SP Switch out.top ﬁle: – The value of the switch_node_number follows the "tb3" ﬁeld in ﬁle output. In the

example below, the switch_node_number is 31:

s 27 3 tb3 31 0 E02-S17-BH-J15 to E03-N47 # Dependent Node

– The value of the node_number can be found by entering:

SDRGetObjects DependentNode switch_node_number==n

where n is the switch_node_number.

This produces output that looks like:

node_number switch_node_number ... 47 31 ...

• If the node is already deﬁned to the SP system, but you do not know other conﬁguration information, this command returns conﬁguration information about the dependent nodes associated with SP Switch Router Adapters in your SP system. From the SP control workstation, enter:

SDRGetObjects DependentNode

• The system administrator may have labeled the switch cable connecting to the media card with the node number.

• If there is no terminal directly-attached to the SP Switch Router, check the router host name from the SP control workstation. From the control workstation, enter:

SDRGetObjects DependentNode node_number reliable_hostname

This command returns host names and their corresponding node numbers for the attached SP Switch Routers.

Several dependent nodes will be associated with the same reliable host name for SP Switch Routers that have more than one SP Switch Router Adapter card.

2-14 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Configuration required on the SP system

Multiple frames for multiple system connections

SP Switch Router Adapter cards in an SP Switch Router can connect to dif ferent switch boards in the same SP system. A conﬁguration problem could arise in which the SP Switch Router Adapter cards would be assigned the same node number if each card plugged into the same port position on each switch board.

The construct of a frame removes the conﬁguration problem. The example belo w demonstrates the organization of three SP frames, 1, 2, and 3 with switch boards in each.

SP system A

SP Switch A1

J31

Frame 1

gt000 node 9

J23 J15

gt010 node 16

J31

Frame 2 Frame 3

SP Switch A2

gt020 node 25

J23 J15

gt030 node 41

SP Switch A3

J31

J23 J15

SP Switch Router

Figure 2-4. How frames enable connections to multiple SP Switches

Figure 2-4 shows how the frame numbering differentiates each SP Switch Router connection:

– the SP Switch Router card connected to port J31 of SP Switch A1 is node number 9 – the SP Switch Router card connected to port J31 of SP Switch A2 is node number 25 – the SP Switch Router card connected to port J31 of SP Switch A3 is node number 41

The SP Switch Router card connected to port J15 of SP Switch A1 is node number 16.

You can now conﬁgure the SP Switch Router Adapter card(s).

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-15

Configuring the SP Switch Router Adapter

Step-by-step media card configuration

Step-by-step media card conﬁguration

This section provides a conﬁguration overview and the steps required to conﬁgure an SP Switch Router Adapter media card.

Conﬁguration ﬁles and their uses

These are the /etc conﬁguration ﬁles discussed in this chapter:

grifconfig.conf - identiﬁes each logical interface on a media card snmpd.conf - enables SNMP capabilities grdev1.conf - conﬁgures SP Switch Router Adapter cards

Refer to the GRF Reference Guide for templates of all conﬁguration ﬁles.

Overview of steps 1–5

Steps 1 through 5 are described following this list.

1 Check SNMP in the SP Switch Router system.

Edit the SNMP conﬁguration ﬁle and start SNMP daemon on the SP Switch Router.

2 Assign IP addresses

Assign an IP address and other parameters to each SP Switch Router Adapter interface.

There are two ways to conﬁgure these parameters: a. We recommend using the procedures documented in the “Managing Extension Nodes”

chapter of the PSSP Administration Guide. or: b. As an alternative, you can log in to the SP Switch Router and use a UNIX editor to enter

the address and other parameters in the /etc/grifconfig.conf ﬁle.

3 Change proﬁle settings (optional)

Change default boot diagnostics and dump settings in proﬁles. To change the defaults for one card, change settings in the appropriate Card proﬁle. T o change the def aults for all installed SP Switch Router Adapter cards, change settings in

the Dump and Load proﬁles.

The default settings are: – enable/disable diagnostic boot sequence: default, off – when to dump: default, at card reset or hang – additional dumps to save: default, two

Default settings save the ﬁrst dump of the day, the current dump, and the next two recent dumps.

2-16 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step-by-step media card configuration

4 Run dev1conﬁg to create /etc/grdev1.conf.

The SP Switch Router requires a speciﬁc conﬁguration ﬁle, /etc/grdev1.conf, for the SP Switch Router Adapter card to operate. The de v1conﬁg command creates this skeleton ﬁle using conﬁguration information passed to the router in either of two ways:

a. We recommend using the procedures documented in the “Managing Extension Nodes” chapter of the PSSP Administration Guide. to install the parameters. (This is the same conﬁguration information as described in the recommended method of step 2.) These parameters will only be available to dev1conﬁg after the SP Switch Router Adapter card is activated on the SP switch.

or: b. As an alternative, you can log on to the SP Switch Router and use a UNIX editor to

enter the parameters in the

/etc/grdev1.conf ﬁle.

5 Reset the SP Switch Router Adapter card to install conﬁguration ﬁles.

Details about each step are provided in the next several sections.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-17

Configuring the SP Switch Router Adapter

Step 1. Check SNMP in the SP Switch Router system

Check the /etc/snmpd.conf ﬁle to see if a management station and community are deﬁned, and if traps are enabled. Network monitoring devices (management stations) can request or access the SP Switch Router’s SNMP information.

Follow the procedure described in Chapter 2 of the GRF Conﬁguration and Management manual, “Conﬁgure SNMP.” Note that the ALLOW and COMMUNITY public statements are in the default /etc/snmpd.conf ﬁle, and must not be removed.

Here is an /etc/snmpd.conf ﬁle appropriate for the SP Switch Router connected to an SP system. This conﬁguration assumes two SP control workstations manage the SP Switch Router Adapter cards. Refer to the “SNMP conﬁguration overview” section in Chapter 1 of this manual for background about the values to specify on the MANAGER and COMMUNITY statements that correspond to the SP SNMP Managers.

In the example, the IP addresses of the two SP control workstations are 129.40.61.12 and

129.40.61.5. You will see them deﬁned under “MANAGER”. The example assumes that both

SP SNMP Managers listen for traps on UDP port 162 and that no other SNMP management programs that need UDP port 162 are installed on the SP control workstations.

# Default Agent Configuration File # # This file allows MANAGERS to be specified. This is used to # specify which managers will be receiving which traps. # # Also, COMMUNITYs can be specified. This allows that agent to # be configured such that it will only accept requests from # certain managers and with certain community strings. # # GRAMMAR: # # INITIAL <name> <String> # # TRANSPORT <name> # [SNMP | SMUX] # [OVER [UNIX | UDP | TCP] [SOCKET | TLI]] # [AT <addr>] # # MANAGER <addr> [ON TRANSPORT <name>] # [SEND [ALL | NO | traplist] TRAPS # [TO PORT <#> ] # [WITH COMMUNITY <name>]] # # COMMUNITY <name> # ALLOW op [,op]* [OPERATIONS] # [AS <name>] # [USE encrypt ENCRYPTION] # [MEMBERS <addr> [,<addr>] ] # # ALLOW <subagentId> [ON <hostSpec>] # WITH <passwordSpec> [<entitySpec>] [<timeout>] # # DENY <subagentId> ON <hostSpec> WITH <passwordSpec> #

2-18 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step 1. Check SNMP in the SP Switch Router system

# ENTITY <EntityName> DESCRIPTION <String> # # LOCAL CONTEXT <contextName> [USES] VIEW <viewName> # REFERS TO ENTITY <entityName> AS <oid> # # PROXY CONTEXT <oid> [USES] SOURCE PARTY <oid> # DESTINATION PARTY <oid> # [AND] CONTEXT <oid> # # VIEW <viewName> [[INCLUDE | EXCLUDE] SUBTREE <oid> [MASK <bitmask>]]+ # # ALLOW op [,op]* [OPERATIONS] <sugar> SOURCE PARTY <partyName> # DESTINATION PARTY <partyName> # [AND] CONTEXT <contextName> # # <partyDefinition> ::= [LOCAL] PARTY <name> ON TRANSPORT <transport> # AT <addr> <AuthPriv> # AS <oid> # # <transport> ::= [ snmpUDPDomain | snmpCLNSDomain | snmpCONSDomain # | snmpDDPDomain | snmpIPXDomain | rfc1157Domain # <transport> ::= # <AuthPriv> ::= <noAuth> <noPriv> | # <md5Auth> <noPriv> | # <md5Auth> <desPriv> # # <noAuth> ::= <sugar> NO AUTHENTICATION # # <sugar> ::= [AND] [[WITH | USING]] # # <noPriv> ::= <sugar> NO ENCRYPTION # # <md5Auth> ::= <sugar> MD5 AUTHENTICATION <key> # # <key> ::= <sugar> <string> AS KEY # # <desPriv> ::= <sugar> DES ENCRYPTION <key> # # <subagentId> ::= SUBAGENT <oid> | # SMUX SUBAGENT <oid> | # UNSPECIFIED SUBAGENTS # # <hostSpec> ::= HOST <hostid> | # UNSPECIFIED HOST[S] # # <passwordSpec> ::= PASSWORD <string> # UNSPECIFIED PASSWORDS # # entitySpec ::= AS ENTITY <entityName> # # <timeout> ::= USING <specificTimeout> TIMEOUT # <specificTimeout> ::= <number> SECOND[S] | # NO # # addr ::= <ip-kind> | <rfc1449addr> | <full-ip>

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-19

Configuring the SP Switch Router Adapter

Step 1. Check SNMP in the SP Switch Router system

# ip-kind ::= <hostid> | # <hostid> <portid> | # <portid> # # hostid ::= <hostname> | <ip> # where: hostname is defined in /etc/host # portid ::= [PORT | : ] <#> # # full-ip ::= <ip>:<#> # ip ::= <#>.<#>.<#>.<#> # traplist ::= trap [, trap]* # trap ::= <trap_name> # # op ::= ALL | GET | SET | TRAP # encrypt ::= NO | <name> # # rfc1449addr ::= tcp_ip_addr | osi_addr # tcp_ip_addr ::= <ip>/<#> # osi_addr ::= <nsap>/<tsel> # nsap ::= hexes # tsel ::= hexes # hexes = hexbyte[: hexbyte]* # # ALLOW SUBAGENT 1.3.6.1.4.1.1080.1.1.1 WITH OTHER PASSWORD USE 15 SECOND TIMEOUT COMMUNITY public ALLOW GET, TRAP OPERATIONS USE NO ENCRYPTION MANAGER 129.40.61.12 SEND ALL TRAPS TO PORT 162 WITH COMMUNITY spenmgmt MANAGER 129.40.61.5 SEND ALL TRAPS TO PORT 162 WITH COMMUNITY spenmgmt COMMUNITY spenmgmt ALLOW ALL OPERATIONS USE NO ENCRYPTION #

Put SNMP changes into effect

To have changes to /etc/snmpd.conf take effect, kill snmpd. It is automatically restarted. On the SP Switch Router, log in as super user, ﬁnd the snmpd PID (process ID), and then kill

the SNMP daemon, as follows:

% ps -ax | grep snmpd 26053 p2 S+ 0:00.05 grep snmpd

127 co- S 1:59.55 snmpd /etc/snmpd.conf /var/run/snmpd.OCT % kill 127

2-20 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step 2. Assign IP addresses

You need to assign an IP address and related parameters to the SP Switch Router Adapter interface. You assign a primary IP address and, if needed, an alias address to support multiple subnets on the same physical interface. Netmasks for each address can differ in length. Refer to Appendix C, “Network Conﬁguration Examples,” for ways to use IP addressing and subnet masks.

There are two methods to assign IP addresses, one recommended, one optional.

Method 1: Recommended, use SP SNMP Manager

This is the method recommended for conﬁguring the SP Switch Router Adapter card. From a system point of view, it is appropriate to treat the SP Switch Router Adapter card as an

extension node in the SP system. All conﬁguration parameters should be entered using the SMIT panels. Remember that if you enter conﬁguration information into SP Switch Router conﬁguration ﬁles, you will also need to access the SMIT panels and re-enter information those panels require.

Refer speciﬁcally to the “Managing Extension Nodes” chapter in the PSSP Administration Guide for information about setting up SNMP to monitor the SP Switch Router system and conﬁgure the SP Switch Router Adapter media card.

Method 2: Optional, edit /etc/grifconﬁg.conf

Edit the /etc/grifconfig.conf ﬁle to assign an IP address to each logical SP Switch Router interface. You also can provide other information about the logical IP netw ork to which that interface is physically attached.

Each logical interface is identiﬁed in /etc/grifconfig.conf as to its:

– interface name (an SP Switch Router convention, deﬁned below) – Internet address – netmask – broadcast/destination address – argument ﬁeld (can specify a default MTU value here)

The format for an entry (all lower case) in the /etc/grifconfig.conf ﬁle is:

# /etc/grifconfig.conf #name address netmask broad-dest arguments gt020 192.168.15 255.255.255.0 - mtu 65520 gt020 192.168.16 255.255.255.192 - mtu 65520

Remember that if you enter conﬁguration information into SP Switch Router conﬁguration ﬁles, you will also need to access the SMIT panels and re-enter information those panels require.

# primary # alias

If you have not done so, use the super> prompt to establish the UNIX shell you will use to conﬁgure the SP Switch Router Adapter card.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-21

Configuring the SP Switch Router Adapter

Step 2. Assign IP addresses

Type sh at the super> prompt for the UNIX shell prompt to appear:

super> sh #

Interface name

The SP Switch Router interface name has ﬁve components that describe an individual interface in terms of its physical slot location in the chassis, and its speciﬁc and virtual locations on a media card.

In a 4-card SP Switch Router, model 04S, the SP Switch Router Adapter card interface names look like these: gt000, gt010, gt020, gt030 (only the slot # changes, 0 – 4). Figure 2-5 deﬁnes the components in the SP Switch Router Adapter card interface name:

Figure 2-5. Components in the SP Switch Router Adapter card’s interface name

Internet address

The Internet address is the 32-bit IP address for the speciﬁed logical interface. Enter this address in standard dotted-decimal (octet) notation: xx.xx.xx.xx. This value must match any entered into /etc/grdev1.conf.

1st:

always “g” for GRF media type, t (SP Adapter), f (FDDI), h (HIPPI), e (100Base-T), etc

2nd:

chassis number, always “0” (zero)

3rd:

slot number in hex

4th:

logical interface number in hex, always “0” (zero)

5th

g t 0 y 0

Netmask

Netmask is the 32-bit address for the logical IP network on the physical network to which the speciﬁc SP Switch Router or media card physical interface is attached. The netmask is entered in standard dotted-decimal (octet) notation. If no destination/broadcast address is supplied, a netmask is required. This value must match any entered into /etc/grdev1.conf.

Broadcast / destination address

The broadcast or destination address is the 32-bit address for this network. Enter the broadcast or destination address in standard dotted-decimal (octet) notation. When a broadcast IP address is assigned to a logical interface, the netmask value is ignored. A dash (-) can be entered in the netmask column, or it can be left blank.

The connection of the SP Switch Router Adapter card to the SP system is point-to-multipoint.

SP Switch boardSP Adapter card

point multipoint

2-22 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Argument ﬁeld

This ﬁeld is optional for SP Switch Router Adapter cards. The arguments ﬁeld usually speciﬁes an MTU value different from the coded default value of 65520.

Default MTU values

The default MTUs for SP Switch Router media are:

SP Switch Router Adapter: 65520 bytes HIPPI: 65280 bytes FDDI: 4352 bytes ATM OC-3c: 9180 bytes ATM OC-12c: 9180 bytes 10/100Base-T: 1500 bytes

The default MTUs for these framing protocols are:

Frame Relay: 4352 bytes HDLC: 4352 bytes Point-to-Point Protocol: 1496 bytes

Configuring the SP Switch Router Adapter

Step 2. Assign IP addresses

MTU discovery facility

Maximum Transmission Unit (MTU) sizes are generally selected at the host end of the route. This is accomplished by turning on the host’s MTU disco very facility and allowing the host to send packets. The MTU discovery facility operates by default on the SP Switch Router.

In effect, the discovery facility tells the router not to fragment, but to advise the host when the packet size is larger than the giv en path can handle. This allows the host to discover the largest packet which the most restrictive of the media components within the same path can handle. Once “discovered”, the host then sends only packets in sizes matching the reported maximum, and packets are not fragmented.

Putting grifconﬁg.conf additions into effect

Additions made to /etc/grifconfig.conf after ﬁrst-time installation will take effect only after the ﬁle is reloaded and the media card(s) reset.

Use the grreset command to reset each conﬁgured SP Switch Router Adapter card by specifying the slot number where each card is installed.

Enter:

# grreset <

Refer to the GRF Reference Guide for more information about using grreset.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-23

card_slot_number

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

Step 3. Change proﬁle settings

Specify card-level parameters – Card proﬁle

Set speciﬁc dev1 card conﬁguration parameters at the Card proﬁle. The ﬁelds to set are:

– OPTIONAL: specify ICMP throttling settings – OPTIONAL: change run-time binaries – OPTIONAL: change dump variables

This is the card proﬁle for the SP Switch Router Adapter card residing in slot 8. Media card type, dev1, is automatically read into the read-only media-type ﬁeld. Other values shown are defaults. At the top level, you see conﬁg and ICMP throttling ﬁelds:

super> read card 8 CARD/8 read super> list card 8 card-num* = 8 media-type = dev1 debug-level = 0 hssi-frame-protocol = Frame-Relay sonet-frame-protocol = PPP ether-verbose = 0 ports = <{ 0{off on 10 3} {single off} {"" "" 1 sonet internal-osc+ load = { 0 < > 1 0 0 } dump = { 0 < > off off} config = { 0 1 1 4 0 0 } icmp-throttling = { 10 10 2147483647 10 10 10 }

Specify ICMP throttling

You can specify ICMP throttling changes for this card in these settings. Refer to Chapter 1 in the GRF Conﬁguration and Management manual for an explanation of each ﬁeld, or do a set

<field-name>?

super> list ic echo-reply = 10 unreachable = 10 redirect = 2147483647 TTL-timeout = 10 param-problem = 10 time-stamp-reply = 10

Change default echo reply and TTL settings with this series of commands:

super> set echo-reply = 4 super> set TTL-timeout = 12 super> write CARD/8 written

You do not have to do a write until you have ﬁnished all changes in the Card proﬁle. Howe v er, you get a warning message if you try to exit a proﬁle without saving your changes.

for a brief description. Default values are shown:

2-24 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Specify different executables

A media card’s Card proﬁle can be used to specify a custom binary setting that overrides the standard media binary conﬁgured in the system Load proﬁle.

Card-speciﬁc executables can be set at the Card proﬁle in the load / hw-table ﬁeld. The

hw-table ﬁeld is empty until you specify the path name of a new run-time binary. This

speciﬁed run-time binary will execute in this SP Switch Router Adapter card only.

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

super> list load config = 0 hw-table = < > boot-seq-index = 1 boot-seq-state = 0 boot-seq-diagcode = 0

If you want to try a test binary, specify the new path in the hw-table ﬁeld:

super> set hw-table = /usr/libexec/portcard/test_exec_for_dev1 super> write CARD/8 written

Specify different dump settings

A media card’s Card proﬁle can be used to specify custom dump settings that override the system options conﬁgured in the system Dump proﬁle.

Card-speciﬁc dump ﬁle names can be set at the Card proﬁle in the dump / hw-table ﬁeld. The

hw-table ﬁeld is empty until you specify a new path name where you want dumps for this

card to be kept.

super> read card 8 card/8 read super> list dump config = 0 hw-table = < > config-spontaneous = off dump-on-boot = off

In a Card proﬁle’s config ﬁeld you can specify when dumps will be taken for a particular card. You set the ﬁeld equal to a value that represents a certain ev ent or sev eral such ev ents. Y ou can specify the value in either hex or decimal. However, after you save (write) your speciﬁed

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-25

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

value and you later read the ﬁeld, the setting is always displayed in decimal. Here are the hex values for dump events:

0x0001 - dump always (override other bits) 0x0002 - dump just the next time the card reboots 0x0004 - dump on card panic 0x0008 - dump whenever card is reset 0x0010 - dump whenever card is hung 0x0020 - dump on power up

To specify dump on panic and dump when card hangs, you OR together 0x0004 (dump on card panic) and 0x0010 (dump whenever card is hung). The result in hex is 0x0014, or 20 in decimal. You could specify either of these settings to get the same result:

super> set config = 0x14

or:

super> set config = 20 super> write DUMP/ written

After you save (write) your speciﬁed value and you later read the ﬁeld, you will see the panic and hang setting displayed as config = 20.

To specify dump during panic, reset, and power up, you OR together 0004, 0008, and 0020. The result in hex is 0x2c, the decimal equivalent is 44. Both settings produce the same result:

super> set config = 0x2c

or:

super> set config = 44 super> write DUMP/ written

After you save (write) your speciﬁed value and you later read the ﬁeld, you will see the panic, reset, and power up setting displayed as 44.

Note: Dump settings for all installed media cards are speciﬁed in the system Dump proﬁle. There are two ﬁelds, keep-count and config. These ﬁelds are overridden by settings in a particular Card proﬁle.

The keep-count ﬁeld enables you to specify numbers of dumps to be saved for all installed cards. The default setting is keep-count = 0, which means two more dumps are saved in addition to the current dump and the ﬁrst dump of the day. A total of four dumps will be available with this default setting. At minimum, the ﬁrst dump of the day and the current dump are always saved.

The config ﬁeld in the system Dump is the same as the config ﬁeld in a Card proﬁle. The default for this setting is 20, expressed in decimal, and means that dumps are taken when cards panic and hang.

Refer to the GRF Conﬁguration and Management manual for a description of other system Dump proﬁle ﬁelds.

2-26 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

Change executables for all dev1 cards - Load proﬁle

Global values for executable binaries are set at the Load proﬁle in the hw-table ﬁeld. These only change when you want to execute new run-time code in all SP Switch Router Adapter cards.

Here is the path, default settings are shown:

super> read load LOAD read

super> list hippi = {"" N/A on 0 1 <{1 /usr/libexec/portcards/xlxload.run N/A}+ rmb = { /usr/libexec/portcards/rm.run N/A off 0 1 < > } hssi = {/usr/libexec/portcards/hssi_rx.run /usr/libexec/portcards+ dev1 = {/usr/libexec/portcards/dev1_rx.run /usr/libexec/portcards+ atm-oc3-v2 = {/usr/libexec/portcards/atmq_rx.run /usr/libexec/por+ fddi-v2 = {/usr/libexec/portcards/fddiq-0.run /usr/libexec/portca+ atm-oc12-v1 = { /usr/libexec/portcards/atm-12.run N/A off 0 1 < >+ ethernet-v1 = {/usr/libexec/portcards/ether_rx.run /usr/libexec/p+ sonet-v1 = {/usr/libexec/portcards/sonet_rx.run /usr/libexec/port+ atm-oc12-v2 = {/usr/libexec/portcards/atm-12v2.run N/A off 0 1 < +

Look at the SP Switch Router Adapter card settings:

super> list dev1 type = dev1 rx-config = 0 rx-path = /usr/libexec/portcards/dev1_rx.run tx-config = 0 tx-path = /usr/libexec/portcards/dev1_tx.run enable-boot-seq = off mode = 0 iterations = 1 boot-seq-table = < >

To execute different run-time code on the receive side of the SP Switch Router Adapter card, replace /usr/libexec/portcards/dev1_rx.run with the path to the new code:

super> set rx-path = /usr/libexec/portcards/newdev1_rx.run super> write LOAD written

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-27

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

Change dump defaults for all dev1 cards - Dump proﬁle

Global values for dump settings are at the Dump proﬁle. These settings are usually changed only for debug purposes. Default settings are shown in this example.

The keep-count ﬁeld speciﬁes how many dumps are compressed and stored at one time for each media card. The ﬁle system can store the default setting of 2 which actually stores four dumps per day (the current dump, the ﬁrst dump of the day, and two additional dumps). Use caution if you change the recommended default.

Here is the path, default settings are shown:

super> read dump DUMP read

super> list hw-table = <{hippi 20 /var/portcards/grdump 0} {rmb 20 /var/portc+ dump-vector-table = <{3 rmb "RMB default dump vectors" < {1 SRAM 2+ config-spontaneous = off keep-count = 2

The hw-table ﬁeld has settings to specify when dumps are taken and where dumps are stored. Here is the path to examine the global SP Switch Router Adapter settings:

super> list hw-table hippi = { hippi 20 /var/portcards/grdump 0 } rmb = { rmb 20 /var/portcards/grdump 3 } hssi = { hssi 20 /var/portcards/grdump 7 } dev1 = { dev1 20 /var/portcards/grdump 9 } atm-oc3-v2 = { atm-oc3-v2 20 /var/portcards/grdump 5 } fddi-v2 = { fddi-v2 20 /var/portcards/grdump 6 } atm-oc12-v1 = { atm-oc12-v1 20 /var/portcards/grdump 10 } ethernet-v1 = { ethernet-v1 20 /var/portcards/grdump 8 } sonet-v1 = { sonet-v1 20 /var/portcards/grdump 11 } atm-oc12-v2 = { atm-oc12-v2 20 /var/portcards/grdump 14 }

super> list dev1 media = dev1 config = 20 path = /var/portcards/grdump

In the config ﬁeld you can specify when dumps will be taken by using a value that represents a certain event or several such events. You can specify the value in either hex or decimal. However, after you save (write) your speciﬁed value and you later read the ﬁeld, the setting is always displayed in decimal. Here are the hex values for dump events:

2-28 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

super> set config = 0x14

or:

super> set config = 20 super> write DUMP/ written

After you save (write) your speciﬁed value and you later read the ﬁeld, you will see the panic and hang setting displayed as config = 20.

To specify dump during panic, reset, and power up, you OR together 0004, 0008, and 0020. The result in hex is 0x2c, the decimal equivalent is 44. Both settings produce the same result:

super> set config = 0x2c

or:

super> set config = 44 super> write DUMP/ written

After you save (write) your speciﬁed value and you later read the ﬁeld, you will see the panic, reset, and power up setting displayed as 44.

Dump vectors (read-only)

The segment-table ﬁelds in the dump-vector-table describe the areas in core memory that will be dumped for all SP Switch Router Adapter cards. These are read-only settings, you cannot change them.

Here is the path, cd .. go back up to the main level if necessary:

super> cd .. super> list dump-vector-table 3 = {3 rmb "RMB default dump vectors" < {1 SRAM 262144 524288} > + 5 = {5 atm-oc3-v2 "ATM/Q default dump vectors" <{1 "atm inst memo+ 6 = {6 fddi-v2 "FDDI/Q default dump vectors" < {1 "fddi/Q CPU0 co+ 7 = {7 hssi "HSSI default dump vectors" < {1 "hssi rx SRAM memory+ 8 = {8 ethernet-v1 "ETHERNET default dump vectors" <{1 "Ethernet + 9 = {9 dev1 "DEV1 default dump vectors" <{1 "dev1 rx SRAM memory"+ 10 = {10 atm-oc12-v1 "ATM OC-12 default dump vectors" <{1 "ATM-12+ 11 = {11 sonet-v1 "SONET default dump vectors" <{1 "SONET rx SRAM+ 14 = {14 atm-oc12-v2 "ATM OC-12-V2 default dump vectors" <{1 "ATM+

This sequence shows the segments in the SP Switch Router Adapter card that are dumped:

super> list 9 index = 9 hw-type = dev1 description = "DEV1 default dump vectors" segment-table = <{1 "dev1 rx SRAM memory" 2097152 2097152} {2"dev1+

super> list seg 1 = { 1 "dev1 rx SRAM memory" 2097152 2097152 } 2 = { 2 "dev1 shared SRAM memory" 131072 32768 }

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-29

Configuring the SP Switch Router Adapter

Step 3. Change profile settings

super> list 1 index = 1 description = "dev1 rx SRAM memory" start = 2097152 length = 2097152

super> cd .. index = 9 hw-type = dev1 description = "DEV1 default dump vectors" segment-table = <{1 "dev1 rx SRAM memory" 2097152 2097152}{2 "dev1+

super> list seg 2 index = 2 description = "dev1 shared SRAM memory" start = 131072 length = 32768

2-30 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step 4. Run dev1config to create grdev1.conf

Step 4. Run dev1conﬁg to create grdev1.conf

The dev1conﬁg command creates conﬁguration ﬁles that are necessary for the set up of your SP Switch Router Adapter card, including /etc/grdev1.conf ﬁle.

This command creates the /etc/grdev1.conf ﬁle that contains the null values for the SP Switch Router Adapter MIB. Your SP Switch Router Adapter card conﬁguration will fail if you do not run this command.

Method 1: Recommended, use SP SNMP Manager

Caution: You cannot verify that the parameters have been set in /etc/grdev1.conf until

after the SP Switch Router Adapter card is activated on the SP switch.

There are two ways to get conﬁguration parameters into the /etc/grdev1.conf ﬁle.

This is the method recommended for conﬁguring the SP Switch Router Adapter card. From a system point of view, it is appropriate to treat the SP Switch Router Adapter card as an extension node in the SP system. All conﬁguration parameters should be entered using the SMIT panels.

This is the same conﬁguration information as described earlier in Method 1 of Step 2. These parameters will only be available in the /etc/grdev1.conf ﬁle after the SP Switch Router Adapter card is activated on the SP switch.

Method 2: Optional, edit /etc/grdev1.conf

On the SP Switch Router, edit the /etc/grdev1.conf ﬁle to assign parameters to each logical SP Switch Router interface. These parameters are described on the next page.

Remember that if you enter conﬁguration information into SP Switch Router conﬁguration ﬁles, you also need to access the SMIT panels and re-enter information those panels require.

How to run the command

You must run the dev1conﬁg command while you are logged on to the SP Switch Router. Log on and start the UNIX shell using the sh command:

super> sh #

Enter the dev1conﬁg command:

# dev1config

Then, after the SP Switch Router Adapter card is activated on the SP switch, use the vi editor command to open the /etc/grdev1.conf ﬁle and verify that it does contain the conﬁguration data.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-31

Configuring the SP Switch Router Adapter

Step 4. Run dev1config to create grdev1.conf

Contents of /etc/grdev1.conf

Here is an excerpt from the /etc/grdev1.conf conﬁguration ﬁle. It shows only the slot 0 card interface entry.

– SNMP expects 16 interface entries, so do not remove any even if they will be unused. – Leave the “x” in place in front of the IP address and Net Mask parameters.

# more /etc/grdev1.conf ##################################################################### # DEV1 Configuration ##################################################################### # There are several variables that an SP Adapter card needs at # start up. These are handled by a set of GRINCHES whose descriptors # are indexed by card number and interface number as follows: # # 2.21.{CARD+1}.1.{INTERFACE+1} # # This template specifies the start-up values for all potential cards # in a 16-card GRF router. Initially these are default values # indicating that the card needs to be configured. # # The descriptors are grouped by card and interface so that a # particular interface can be easily configured. ##################################################################### # CARD 0 Interface 0

-> 2.21.1.1.1.1 "00" # Extension Node Identifier

2.21.1.1.1.2 -1 # Node Name

2.21.1.1.1.3 "00:00:00:01:00:00:00:02:00:03" # Switch Token

2.21.1.1.1.4 1 # Switch ARP

2.21.1.1.1.5 0 # Switch Node Number

2.21.1.1.1.6 x0.0.0.0 # IP Address

2.21.1.1.1.7 x0.0.0.0 # Net Mask

-> 2.21.1.1.1.8 1024 # Max Link Pckt Len.(bytes)

2.21.1.1.1.9 0 # IP Host Offset

-> 2.21.1.1.1.10 1 # Configuration State

2.21.1.1.1.11 "sp_00" # System Name

-> 2.21.1.1.1.12 2 # Node State

2.21.1.1.1.13 0 # Switch Chip Link

2.21.1.1.1.14 31 # Node Delay (cycles)

2.21.1.1.1.15 1 # Admin Status #

The arrows (->) indicate read-only values that will not be changed.

2-32 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Parameter deﬁnitions

This section describes the parameters in the /etc/grdev1.conf ﬁle that are read from SNMP variables:

Extension Node Identiﬁer ibmSPDepNodeName (read-only) is a 2-digit value that corresponds to the SP Switch Router slot number for the resident SP Switch Router Adapter card. Valid values are between 00 and 15.

Node Name ibmSPDepNodeNumber is an integer value that is the node number assigned to this card based on its physical connection to an SP Switch port.

Switch T ok en ibmSPDepSwToken is an octet string size 10 value that internally identiﬁes the SP Switch Router Adapter card’s connection to the SP Switch network.

When you edit the grdev1.conf ﬁle, you should consult the SP system administrator to determine what the Switch Token value should be since the makeup of this ﬁeld could change in future releases. For the initial release of SP Switch Router support, the ﬁeld includes the switch board number (octets 1–4), switch chip number (octets 5–8), and chip port number (octets 9 and 10) to which the adapter is connected. The SP system administrator can determine these values from the SP System Data Repository (SDR) object for the dependent node.

Configuring the SP Switch Router Adapter

Step 4. Run dev1config to create grdev1.conf

Switch ARP ibmSPDepSwARP is an integer, either 1 or 2. A value of 2, enabled, indicates that ARP is to be used to resolve network protocol addresses to switch node numbers for the switch network.

A value of 1, disabled, indicates that ARP is not to be used to resolve network protocol addresses to switch node numbers for the switch network. Instead, the mapping of IP addresses to switch node numbers is determined via the IP Host Offset.

Switch Node Number ibmSPDepSwNodeNumber is an integer that indicates the physical address or switch node number for the SP Switch Router Adapter card on the SP switch network (also called the Node Device ID).

IP Address ibmSPDepIPaddr is the IP address of the card expressed in standard IP notation, x00.00.00.00, and requires the x preﬁx to be attached.

Net Mask ibmSPDepNetMask is the subnet netmask for the SP Switch Router Adapter card’s IP address expressed in standard IP notation, x255.255.255.0, and requires the x preﬁx to be attached. All network bits must be set to 1, all host bits set to 0.

Max Link Pckt Length ibmSPDepIPMaxLinkPkt (read-only) is the maximum number of bytes carried in the data portion of the IP datagram transmitted on the SP Switch network, the required setting is 1024.

IP Host Offset ibmSPDepIPHostOffset Speciﬁed when ARP is disabled, is an integer whose value is the difference between switch node numbers and the host portion of the corresponding IP addresses. The value to subtract from the host portion of an IP address to calculate the corresponding switch node number.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-33

Configuring the SP Switch Router Adapter

Step 4. Run dev1config to create grdev1.conf

Conﬁguration State ibmSPDepConfigState (read-only) is an integer that indicates the ﬁnal conﬁguration state of the SP Switch Router Adapter card.

1 = not conﬁgured 2 = ﬁrmware load failed 3 = driver load failed 4 = diagnostic failed 5 = microcode load failed 6 = fully conﬁgured

System Name ibmSPDepSysName is a quoted string which is the host name of the control workstation for the system partition to which the SP Switch Router Adapter card is connected. The format could be anything and is determined by the value speciﬁed in the SP system conﬁguration.

Node State

ibmSPDepNodeState (read-only)

is an integer, either 1 or 2, that indicates whether the SP Switch Router Adapter card is responding. A v alue of 2, do wn, indicates the card is not responding. A value of 1, up, indicates the card is up and can be unfenced if it had previously been fenced from the SP Switch network.

Switch Chip Link ibmSPDepSwChipLink is an integer with a value that corresponds to the send port number for packets from the adjacent switch chip to the SP Switch Router Adapter card. The card uses this value in the wake-up packet it sends to the adjacent switch chip. The SP system administrator can determine this value from the SDR object for the dependent node.

Node Delay ibmSPDepNodeDelay is an integer that expresses an estimated amount of clock cycles that can elapse before the switch chip loads a synchronization register in the SP Switch Router Adapter card. The card uses this value in the wake-up packet it sends to the adjacent switch chip. The SP system administrator can determine this value from the SDR object for the switch partition that contains the dependent node.

Admin Status ibmSPDepAdminStatus is an integer, either 1, 2, or 3, that indicates whether the SP Switch Router Adapter card is set to communicate with the Switch Manager protocol. The SP SNMP Manager sets this value to trigger resetting or reconﬁguration of the SP Switch Router Adapter card.

1 = the card is ready 2 = the card is not ready and should be reset 3 = the card needs to be reconﬁgured and made ready

2-34 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Step 5. Reset card to install files

Step 5. Reset card to install ﬁles

T o install the SP Switch Router Adapter conﬁguration ﬁles, ﬁrst sa ve the ﬁles and then reset the SP Switch Router Adapter card.

Save the ﬁles after you complete the system parameters and again after you conﬁgure the media cards and any network services such as ﬁltering or dynamic routing.

# grwrite -v # grreset <

Saving conﬁguration ﬁles

Use the grwrite command to save the /etc conﬁguration directory from RAM to the internal ﬂash device. This preserves the conﬁguration ﬁles over a reboot.

# grwrite -v

card_slot_number

With the /etc directory saved, use the grsnapshot command to save an alternate version of the operating software to the internal ﬂash that will be based upon the conﬁguration currently running on the internal ﬂash device:

# grsnapshot -sP=

In the command, -sP speciﬁes the source as primary internal ﬂash (the SP Switch Router has only a primary internal ﬂash), and -dp speciﬁes the destination device as the primary internal ﬂash. A speciﬁc conﬁguration is deﬁned as a release and version. For example, if the current operating release is 1.4.20, then the source might be 1.4.20.ibm,default and you may name the saved conﬁguration as 1.4.20.ibm,backup.

For more information about these commands, see the GRF Reference Guide.

revision,version

-dP=

revision,version

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-35

Configuring the SP Switch Router Adapter

Verify SP Switch Router Adapter card from router

This section describes tools available from the SP Switch Router system software to check out newly-installed media cards. These tools are to be used on the SP Switch Router.

• The ping command tests whether an SP Switch Router Adapter media card can process and return a message.

• The grcard command tells you the operating state of an installed SP Switch Router Adapter card.

• The grreset command allows you to reset all or an individual card.

Note: Output from logs and other system reporting functions refer to the SP Switch Router Adapter card as DEV1 or dev1.

Verify media card operation using ping

Check SP Switch Router Adapter media card viability using the ping command. This UNIX command is modiﬁed to support SP Switch Router board components. This use of ping only tests internal communication between the SP Switch Router control board and the speciﬁed media card. It does not test message routing between media cards or communication between media cards and external devices.

Note: The ping command does not disturb normal SP Switch Router operations.

The ping -P grid <slot number> command sends a message to a speciﬁed SP Switch Router Adapter card asking the card to respond back with another message.

1 Log in as root to the SP Switch Router.

2 Enter a ping -P grid command to send a ping from the control board. Specify the

appropriate media card by its chassis slot number.

For example, to act on the SP Switch Router Adapter media card in slot 3, enter:

# ping -P grid 3

This is what you see when the media card responds:

68 bytes from 0:0x3:0: time=0.293 ms 68 bytes from 0:0x3:0: time=0.251 ms 68 bytes from 0:0x3:0: time=0.288 ms

•

Do a Control-C to stop the ping and view ping statistics:

-- 2 GRID ECHO Statistics -2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.969/1.060/1.172 ms

To act on the IP switch control board, enter:

# ping -P grid 66

2-36 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Verify SP Switch Router Adapter card from router

Verify switch node connectivity using ping

You can execute a ping command that will verify the connection between the switch node and the SP Switch Router Adapter card’s inbound and outbound data paths.

Note: You must make sure that the SP Switch Router Adapter card has been unfenced before executing the ping.

1 Log in as root to the SP Switch Router. 2 Enter a ping ip_address command where ip_address is the IP address of the target SP

Switch Router Adapter card.

# ping xxx.yyy.zzz.nnn

Refer to the GRF Reference Guide for a description of the ping command.

Check media card status using grcard

The grcard command returns information about the status of all installed media cards. Output from logs and other system reporting functions refer to the SP Switch Router Adapter card as

DEV1_v1 or dev1.

Enter:

# grcard -v

Here is a sample of the slot, media, and state information returned from the grcard command:

# grcard -v

Slot 0 HSSI_V1 running 1 ATM_OC3_V1 running 2 DEV1_V1 running 3 FDDI_V2 running

The SP Switch Router Adapter card resides in slot 2 and its state is reported as running. Refer to the command descriptions in the GRF Reference Guide for a description of grcard.

Media card states

The following SP Switch Router media card states are reported:

POWER-UP - initial state of a card at system power on BOOT-REQUESTED - card has requested its run-time code DUMPING - card is being dumped LOADING - card is receiving run-time code CONFIGURING - card has requested its conﬁguration tables RUNNING - card is conﬁgured and operating NOT-RESPONDING - card does not respond to requests from the management software PANIC - card has encountered a system fault HELD_RESET - card is being held in reset state STATE_UNKNOWN - state cannot be determined

HWtype State

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-37

Configuring the SP Switch Router Adapter

Verify SP Switch Router Adapter card from router

The POWER-UP state is the normal condition as power is being applied to the media card.

The BOOT-REQUESTED state is one step in the progress of a card during initial power-up, while a card is coming up, or while a card is being reset due to user or software direction.

DUMPING is the resulting state when a card is directed to dump at reset, at a panic, or as user-speciﬁed in the Dump or appropriate Card proﬁle.

LOADING is a state during which the card receives its run-time image.

CONFIGURING is a waiting state for the media card after it issues a request for conﬁguration parameters. The media card stays in the CONFIGURING state after being loaded until it has all necessary conﬁguration information, is initialized, and the kernel recognizes the card is up.

RUNNING is the normal operating state, the card is able to receive and send packets.

NOT-RESPONDING is a state in which the card does not answer requests from the operating software. The card could be hung. If it decides a card is hung, the kernel begins an automatic reset.

A system fault can cause a card to go into the PANIC state. After a card panics, it needs operator or kernel intervention to start up. The kernel either reboots the card, or dumps and then reboots, depending upon how variables are user-speciﬁed in the Dump or appropriate Card proﬁle.

An operator can use the grreset -h command to put a card into the HELD_RESET state to keep it up but not transferring packets.

When the kernel cannot determine what a media card is doing, it places the card into STATE_UNKNOWN, and either reboots the card, or dumps and then reboots the card, depending upon how variables are user-speciﬁed in the Dump or appropriate Card proﬁle.

Reset media card using grreset

Use the grreset command to reset a media card from the UNIX prompt. Note: The grreset command can be used on a media card without disturbing normal SP

Switch Router system operations.

1 Log in as root on the SP Switch Router.

2 Enter the grreset command, specifying the appropriate media card by its slot number.

To reset all the media cards, enter:

# grreset all

To reset the media cards in slot 0, enter:

# grreset 0

To reset the card in slot 4 and dump its memory, enter:

# grreset -D 4

2-38 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Configuring the SP Switch Router Adapter

Verify SP Switch Router Adapter card from router

To reset the card in slot 4 and return debug information, enter:

# grreset -d 4

The hold reset option (-h) has numerous uses, here are three.

– To isolate a possible problem, set all cards to hold reset and bring them on-line one at

a time.

– To determine whether a card is affecting other media cards, put that one in hold reset

to verify what is happening.

– If certain cards cannot restart while under heavy load from the router switch, you can

put all cards in hold reset and then bring up those cards having trouble ﬁrst while the switch load is light.

These commands hold either all or one media card in reset:

# grreset all -h # grreset

slot

-h

Refer to the command section in the GRF Reference Guide for a description of grreset.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-39

Configuring the SP Switch Router Adapter

Bringing the SP Switch Router Adapter card on-line with the SP

After the SP Switch Router Adapter media card completes initialization, its state machine enters the Configured state (6). The media card sends an up-trap request to mib2d. mib2d sends the SP Switch manager a pair of switchNodeUp and switchConfigState (ConfigState=FullyConfigured) trap messages.

The SP system administrator now decides which action is required to bring the IP router interface on-line.

If the SP Switch Router Adapter was previously fenced from the switch network with the

-autojoin option, the SP SNMP Manager will automatically unfence the adapter. Otherwise,

the SP system administrator must perform one of the following actions to bring the SP Switch Router Adapter card on-line. Possible actions include:

- a switch initialization

- an unfencing sequence

- other switch management sequence

The appropriate action depends on what state the SP system is in with respect to the dependent node. For example, if no Estart command has been issued to re-initialize the SP switch since the dependent node (the SP Switch Router Adapter) was installed, then an Estart command is needed. If the dependent node was fenced from the SP switch without the -autojoin option, then an Eunfence command is needed.

Many different states are possible. Consult the PSSP Installation and Migration Guide and the PSSP Administration Guide for descriptions of the administrative actions needed to bring extension nodes on-line (dependent nodes are speciﬁc types of extension nodes). See the PSSP Diagnosis Guide” for information on diagnosing extension node conﬁguration problems.

The SP Switch Router Adapter media card remains in fullyConfigured state until it is actually brought on line via a switch initialization or unfencing sequence. Should the

switchNodeUp trap message not reach the SP SNMP Manager, use the grcard command to

check the card’s readiness and state. The grcard command returns “running” if the card is ready to be brought on-line.

Checking connectivity to the SP system

The procedure in this section is useful when a problem is suspected with the SP Switch Router Adapter media card, its connection to the SP Switch, or its connection to the SP Switch Router hardware. This section is intended for hardware service personnel, although parts may be applicable to customer problem determination.

Before beginning this procedure, it may be helpful to verify the conﬁguration of the media adapter. Use the vi editor to determine whether proper or NULL parameters are set in the

/etc/grdev1.conf ﬁle. You can also use the maint 3 command to view the parameters for a

single card. If you are unable to ﬁnd a conﬁguration problem or are unable to correct the conﬁguration due

to potential hardware problems, the procedure should be used to check the connection to the SP switch. Each SP Switch Router Adapter media card is considered a dependent node for the

2-40 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Procedure

Configuring the SP Switch Router Adapter

Bringing the SP Switch Router Adapter card on-line with the SP

SP System. Each dependent node has a node_number and other conﬁguration and status information which is unique to that dependent node.

1 Check the SP Switch cable for obvious problems such as a loose or disconnected

connector. If any problems are found, correct the problem.

2 Check the 10Base-T twisted-pair connection between the SP Switch Router control board

and the SP control workstation. This connection is normally routed through an Ethernet hub.

3 If there is no terminal directly-attached to the SP Switch Router, check the SP Switch

Router host name from the SP control workstation. From the control workstation, enter:

SDRGetObjects DependentNode node_number reliable_hostname

This will return the node numbers and the corresponding host names for the SP Switch Router systems.

4 Test Ethernet connectivity by performing a ping test from the SP control workstation to

the SP Switch Router administrative Ethernet address.

5 Check the status of the SP Switch Router Adapter LEDs.

Use the tables in the “SP Switch Router Adapter LEDs” section in Chapter 1 to determine the state of the card.

Generally, “RX ST0/ST1/ERR” and “TX ST0/ST1/ERR” will indicate a problem. The problem might be due to connection, conﬁguration, hardware, or software.

To further test the SP Switch Router Adapter card hardware, you can reset or reseat the card, and then use the tables under “LED activity during boot” (Chapter 1) to interpret the results.

Note: Before you reset an SP Switch Router Adapter card or re-seat a card in the router chassis, be sure that the switch node has been fenced. Use an Efence command to take the dependent node off the switch node.

6 From the SP control workstation, use an Eunfence and/or Estart command to bring the

dependent node back into the conﬁguration. From the control workstation, check switch_responds for a good value. If switch_responds returns "1" or shows green in Perspectives, then the dependent

node is active again.

7 You may need to log in to the SP Switch Router to perform additional analysis before

determining whether any hardware needs replacement.

8 If problems remain, you will have to contact the next le vel of Customer Support for further

direction. They may log into the SP Switch Router to perform additional analysis. If you were directed here by the RS/6000 SP Maintenance Information Manual Dependent Node MAP, return to that procedure.

For more information about conﬁguration as related to the SP, see the PSSP Administration Guide and the PSSP Command and Technical Reference. For additional information on troubleshooting your conﬁguration, see the PSSP Diagnosis Guide.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 2-41

Configuring the SP Switch Router Adapter

Bringing the SP Switch Router Adapter card on-line with the SP

2-42 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Monitoring and Management Tools

This chapter describes tools used to monitor day-to-day operations of the SP Switch Router Adapter card or to indicate the causes of problems which may develop.

These tools operate from the SP Switch Router, and apply to the SP Switch Router and the SP Switch Router Adapter card. The “Management Commands and Tools” and “Management T asks” chapters in the GRF Conﬁgur ation and Mana gement manual pro vide more information related to monitoring and managing the router.

Other tools that apply to the SP Switch Router Adapter card, notably SNMP, operate from the SP control workstation. Information about procedures performed from the SP control workstation are found in the “Managing Extension Nodes” chapter in the PSSP Administration Guide.

Chapter 3 contains these topics:

SP Switch Router command overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2

SP Switch Router UNIX tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6

Using the netstat command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8

Obtaining layer 2 and 3 statistics - grstat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12

SP Switch Router Adapter card maint commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15

Checking for hardware problems - grdiag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22

SP Switch Router dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24

Data collection utility - grdinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25

SP Switch Router logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-30

Burning in media card ﬂash memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-34

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-1

Monitoring and Management Tools

SP Switch Router command overview

This section provides a brief overview of frequently-used management commands. These are administrative and conﬁguration commands, most are preﬁxed with gr and most operate on the router’s internal ﬂash.

These commands manage memory and support multiple conﬁguration versions. These include: ﬂashcmd, getver, grﬁns, grsite, grsnapshot, grwrite, mountf, setver , umountf, and vpurge.

Refer to the GRF Reference Guide for command syntax and examples. Man pages are a vailable for most of these commands.

Many of the commands read/write the internal ﬂash device. Those commands will mount the ﬂash (mountf -w), perform their function, and then unmount the ﬂash. Mounting takes several seconds. If you are doing several commands in a row, mount the ﬂash yourself to avoid the repeated mount/unmount delay. The commands do not mount ﬂash if it is already mounted and do not unmount it if they did not mount it.

This diagram of the control board memory structure provides a reference point as you review the memory commands.

System RAM

Permanently reserved for file system

128MB

---> 256 MB ---> 384 MB ---> 512 MB 64

Internal flash

device

85MB

csconﬁg

ﬂashcmd

Slot B

modem

Control board

CPU

Administrative

LAN (de0)

PCMCIA

Slot A

Disk

device options

(g0001)

Figure 3-1. SP Switch Router control board memory components

csconﬁg sets a PCMCIA slot interface on (up) or off (down), and reports general interf ace and device status. This command is useful for remote management of PCMCIA devices to verify the status of device and slot interface readiness.

This SP Switch Router command mounts the speciﬁed ﬂash device, executes a command (such as write, read, ls, df) on the device, and then unmounts the device. For example, to use the df command to determine device capacity, use ﬂashcmd df.

3-2 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

getver

grarp

Monitoring and Management Tools

SP Switch Router command overview

This command tells you the version of the operating system that is currently running. It can also report which release version will be run the next time the system is booted. In this case, getver is used in conjunction with setver . The setver command speciﬁes which release will be run at the next system boot.

The grarp command displays the ARP table for a given IP address. If you specify grarp with the IP address of the SP Switch Router Adapter card, it returns a physical address that is the switch node number to which this card connects.

# grarp 192.146.162.67 gt030 (0): 192.146.162.67 at 0:0:0:0:0:2

In this example, 192.146.162.67 is the IP address of the SP Switch Router Adapter card in slot 3 (interface name gt030), and 0:0:0:0:0:2 is the switch node number. Other grar p options edit, add, or remove ARP table entries.

grcard

grﬁns

This command displays slot number, media type, and current operating status of the installed media cards.

super> grcard 0 FDDI_V2 running 1 FDDI_V2 running 2 ETHER_V1 running 3 HSSI_V1 running 5 HSSI_V1 running 6 SONET_V1 running 7 ETHER_V1 running 8 FDDI_V2 running 9 ATM_OC3_V2 running 10 HIPPI_V1 running 11 HSSI_V1 running 12 DEV1_V1 running 13 DEV1_V1 running 14 ATM_OC12_V2 running 15 ATM_OC12_V2 running

This command installs a release onto the internal ﬂash device. In the process, it installs all the new ﬁles and converts the system conﬁguration ﬁles as required. Rebooting moves the image to RAM.

As an example, if a release has a new /etc/gratm.conf ﬁle, grﬁns does not write ov er your current version. Instead, it installs a new version of the /etc/gratm.conf.template ﬁle. In this way, you can copy the current conﬁguration information into the new .template ﬁle, make any changes, and then save that ﬁle as the new /etc/gratm.conf.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-3

Monitoring and Management Tools

SP Switch Router command overview

grms

This command enables non-privileged users to reboot, halt, or shutdown the SP Switch Router in an orderly manner that precludes damage to the system.

grreset

This command resets one or more speciﬁed media cards. Options can direct that memory be dumped when the media card comes back up (grreset -D) or that the media card be held in reset (grreset -h).

grrmb

This command enables you to use the maint command set to display media card statistics. maint commands require the GR ##> screen prompt which is invok ed by executing the grrmb command.When grrmb is entered, the screen prompt changes to:

# grrmb GR ##>

grroute

grrt

grsite

where ## is the number of a chassis slot. The default is 66, specifying that the command will act on slot 66, the router control board. The port command changes the prompt slot number. Using grrmb and the maint commands is described in the section “SP Switch Router Adapter card maint commands” on page 3-15.

This command adds the routes speciﬁed in the /etc/grroute.conf conﬁguration ﬁle. This ﬁle maintains the set of static routes to remote nodes. If you are running GateD, do not use grroute, you must use the GateD Static statement to create static routes.

The grrt -p slot -S command displays the route table for an individual media card. Other options delete table entries, display the route to a speciﬁed address, and so on.

This command enables a site to manage and install special ﬁles after the main release is loaded into RAM. A special ﬁle could be a single binary image for a type of media card to be used for debug or testing, for example, an experimental version of the FDDI run-time ﬁle

/var/portcards/fddi.run. grsite has options to add, delete, or list ﬁles in the current

release, the next boot release, or an arbitrary release set.

grsnapshot

This command runs a script that can be speciﬁed to copy conﬁguration ﬁles (or release images) to a target ﬂash device under a ne w or the current version name. For example, grsnapshot can

3-4 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

grstat

grwrite

Monitoring and Management Tools

SP Switch Router command overview

be set up to initialize an external (PCMCIA) ﬂash device, copy the entire contents of the internal ﬂash device to it, and rename the image as a backup.

The grstat command options report layer 3 (IP and ICMP) forwarding statistics for the SP Switch Router Adapter card and all other media card types excpt HIPPI. Error reporting includes the saved source and destination IP addresses of the packet that caused the last error of each type reported.

The grstat l2 command reports many of the Layer 2 (data link layer) statistics currently reported by individual media card maint commands for SP Switch Router Adapter, ATM OC-3c, HSSI, Ethernet, and SONET OC-3c cards. Examples are included in “Obtaining layer 2 and 3 statistics - grstat” on page 3-12.

The grwrite command is crucial on the SP Switch Router because it saves conﬁguration changes made in the /etc directory to a ﬂash device so the changes survive the boot-up process. By default, grwrite saves a copy of those ﬁles with a newer time stamp than the last boot.

mountf

setver

umountf

vpurge

This command mounts an external device so that the device looks like a ﬁle system to the operating system. Mounting an external device enables various processes to be applied to the device. A device is mounted as read only (default) or writable. mountf veriﬁes (fsck) the device before doing the mount. When a media card panics, mountf works with grdump to write a panic dump out to an external storage device in a PCMCIA slot. See also umountf.

This command speciﬁes the software version that will load during the next system reboot. The general form of the command is setver that the speciﬁed appropriate release ﬁles, start-up scripts, and conﬁguration entities are in place. You see a message if these release components are incomplete. See also getver.

This command unmounts a ﬂash device previously mounted by the mountf command. See also mountf.

release_name

can actually be loaded by checking to see that the

release_name

. When setver executes, it veriﬁes

This command removes a speciﬁed release or conﬁguration version from a speciﬁed ﬂash device.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-5

Monitoring and Management Tools

SP Switch Router UNIX tools

Determining that TCP/IP routing is conﬁgured properly between the SP system and the SP Switch Router is another task for a system administrator. This section details some commands that can be useful in performing this task. For more information on managing the SP Switch Router network data, please see the sections of the IBM PSSP Administration Guide that deal with managing extension nodes.

ping

This standard tool generates and receives ICMP/IP echo request and reply messages. It is used to test connectivity to a speciﬁc interface or to a host such as the SP. The version running on the SP Switch Router is modiﬁed for use with media card protocols.

The most common use of ping is to use it from an SP standard node to determine if you have a path to your SP Switch Router Adapter on the SP Switch network. This is necessary if you are to reach or be reached by external networks that the SP Switch Router can access. From an SP standard node, you can ping the IP address for the SP dependent node that corresponds to the SP Switch Router Adapter. This IP address can be obtained by using the host command if you have domain host name resolution conﬁgured in your network. If not, then this address is the address associated with the switch network for that node, and not the address that would be associated with the administrative network for nodes in your SP.

route

The following example shows how you might use host and ping to verify your path to the SP Switch Router Adapter on an SP node named system is sp_sn2.

[sp_n3][/]> host sp_sn2 sp_sn2.ppd.pok.ibm.com is 129.40.85.130 [sp_n3][/]> ping 129.40.85.130 PING 129.40.85.130: (129.40.85.130): 56 data bytes 64 bytes from 129.40.85.130: icmp_seq=0 ttl=255 time=0 ms 64 bytes from 129.40.85.130: icmp_seq=1 ttl=255 time=0 ms 64 bytes from 129.40.85.130: icmp_seq=2 ttl=255 time=0 ms

----129.40.85.130 PING Statistics---3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0/0/0 ms [sp_n3][/]>

In this example, note the difference between sp_nX host names that refer to the Ethernet IP address of a node, and the sp_snX host names that refer to the switch network IP address of a node.

Refer to the GRF Reference Guide for other ping examples.

sp_n3. The SP dependent node in the same SP

Static routing can be conﬁgured by using either the UNIX route command or the SP Switch Router grroute command. Routing is the primary function of a router that allows IP trafﬁc from one network to reach another network. The SP Switch Router and the SP Switch Router Adapter card support both static and dynamic IP routing.

3-6 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Monitoring and Management Tools

SP Switch Router UNIX tools

The UNIX route command can be used to manually add or delete routes. When route is used, no media card or system reset is needed to install the new routes, the new routes are updated in the kernel and downloaded into each media card automatically.

The SP Switch Router grrt command can also be used to examine the routing table on a speciﬁc media card but it is not recommended for routing table conﬁguration because it does not ensure that routing tables are synchronized among the various media cards.

tcpdump

Static routes can also be set by editing the made via this conﬁguration ﬁle do not take effect until the affected media card is reset or the SP Switch Router system is reset.

IP dynamic routing can be conﬁgured by editing the running the gated daemon in the SP Switch Router. GateD implements complex routing protocols. Please refer to the GRF Reference Guide and the GRF Conﬁguration Guide for information about using GateD on the SP Switch Router.

Note: If you plan to run GateD, set up your static routes in /etc/gated.conf by using the Static statement. If you add routes using the route command when GateD is active, those routes are not maintained and are removed by GateD.

/etc/grroute.conf conﬁguration ﬁle. Changes

/etc/gated.conf conﬁguration ﬁle and

grroute.conf ﬁle

Route tables can also be set by editing the /etc/grroute.conf conﬁguration ﬁle. Changes made via this conﬁguration ﬁle do not take effect until the affected media card is reset. Information in

This standard UNIX media examination tool is modiﬁed for use with media card protocols. tcpdump prints out all packet headers or a speciﬁed type of header transmitting on the target network. Note that tcpdump can interfere with network operations and performance.

grroute.conf is eventually turned into route commands.

tcpdump works on the router’s Ethernet LAN interface located on the SP Switch Router

control board (de0), and the router’s communications b us (rmb0). The TCP Dump utility also functions on the SP Switch Router Adapter card.

To collect packet headers from an SP Switch Router Adapter card in slot 6, enter:

# tcpdump -i gt060

traceroute

This standard UNIX command prints the route that packets likely take to a destination network host. traceroute uses the ICMP/IP parameters time-to-live and time-exceeded to trace a route between two IP entities.

You can use the traceroute command to determine if packets from an SP node are actually being routed through the SP Switch Router to get to the target destination address.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-7

Monitoring and Management Tools

Using the netstat command

The UNIX netstat command reports status and information about SP Switch Router media card physical interfaces. netstat is available from the CLI and the UNIX shell.

– netstat -r -s prints routing statistics – netstat -i -n shows all conﬁgured interfaces – netstat -a -n prints a list of all active connections – netstat -g -n prints the multicast route table – netstat -r -n prints the current table of installed routes

In the output from netstat -r -n, the => symbol next to a route means it is a duplicate key, but with a different netmask.

– netstat -rn | wc -l returns the number of entries in the routing table, here is an

example of a 50-entry table:

# netstat -rn | wc -l 50

– netstat -s prints comprehensive statistics for protocols, including: IP, ICMP, TCP , and

UDP, and GRIT, GRIEF, and GRID for SP Switch Router entities.

Refer to the man page for a complete list of netstat options. Examples of netstat usage follow.

netstat -rn

Use this netstat command to determine that the SP Switch Router Adapter card has the correct routing table entries. You must have properly conﬁgured routing table entries so that TCP/IP packets can be routed to and from the SP system to the SP Switch Router Adapter card. Use netstat with the -rn options to view the card’s current routing table entries:

# netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Interface default 129.40.85.126 UGS 3 19356 tn0

127.0.0.1 127.0.0.1 UH 0 19 lo0

129.40.85.64/26 link#1 UC 0 0 tn0

129.40.85.121 0:c0:f2:0:26:8a UHL 4 490 lo0

129.40.85.126 0:0:a2:1:ff:e8 UHL 1 2 tn0

129.40.85.128/26 129.40.85.130 U 0 0 gt020

129.40.158.128/26 129.40.158.139 U 0 88652 gt030

129.40.192.128/26 129.40.192.139 U 0 34 gt060 224/8 link#1 UC 0 0 tn0

In the netstat -rn output, you will see the routing entries for the various SP Switch Router Adapters installed in the SP Switch Router. In the example above there are three. These are identiﬁed by their Interface names having the form gt0y0 where y is the number of the chassis slot in which a speciﬁc SP Switch Router Adapter is installed. F or more information on the logical interface naming convention, refer to Chapter 2.

Each Interface in the netstat -rn output should correspond to at least one route that speciﬁes the reachable network in the Destination column. The TCP/IP address of the SP

3-8 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

dependent node for this SP Switch Router Adapter will be speciﬁed in the Gateway column. This is the address to which packets for the destination network will be sent when they enter the SP Switch Router from external networks or from other SP Switch Router Adapter networks. This is how you display the route paths that have been built for packets destined for or sent from the SP system.

netstat -rs

Using both the -r and -s options, netstat prints routing statistics:

netstat -in

Monitoring and Management Tools

Using the netstat command

% netstat -rs

routing: 0 bad routing redirects 0 dynamically created routes 0 new gateways due to redirects 44 destinations found unreachable 0 uses of a wildcard route

Here is an example of output from netstat -in with interface information. Interface gt020 is the SP Switch Router Adapter card from slot 2:

# netstat -in Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll de0 1500 <link1> 00:c0:80:0b:30:53 492665 0 8099 0 2584 de0 1500 198.174.11 198.174.11.249 492665 0 8099 0 2584 rmb0 596 <link2> 00:00:00:00:00:00 130022 0 129726 0 0 rmb0 596 <GRIT> 0:0x40:0 130022 0 129726 0 0 lo0 1536 <link3> 496 0 496 0 0 lo0 1536 <GRIT> 0:0x48:0 496 0 496 0 0 gl000* 1524 <link4> 0 0 0 0 0 gh010* 65280 <link11> 0 0 0 0 0 gt020 65520 <link5> 0 0 0 0 0 gt020 65520 206.146.162 206.146.162.67 0 0 0 0 0 gf080 4352 <link7> 00:c0:80:00:04:db 0 0 0 0 0 gf080 4352 206.146.162 206.146.162.129 0 0 0 0 0 gf081* 4352 <link8> 00:c0:80:00:04:dc 0 0 0 0 0 gf082* 4352 <link9> 00:c0:80:00:04:dd 0 0 0 0 0 gf083* 4352 <link10> 00:c0:80:00:04:de 0 0 0 0 0 gs0b0* 4352 <link6> 0 0 0 0 0

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-9

Monitoring and Management Tools

Using the netstat command

netstat -an

Here is an excerpt from netstat -an showing active connections:

# netstat -an Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp 0 0 198.174.11.249.23 198.174.11.38.1073 ESTABLISHED tcp 0 0 198.174.11.249.199 198.174.11.249.1026 ESTABLISHED tcp 0 0 198.174.11.249.1026 198.174.11.249.199 ESTABLISHED tcp 0 0 *.199 *.* LISTEN tcp 0 0 198.174.11.249.199 198.174.11.249.1024 ESTABLISHED tcp 0 0 198.174.11.249.1024 198.174.11.249.199 ESTABLISHED tcp 0 0 *.23 *.* LISTEN udp 0 0 *.* *.* udp 0 0 *.161 *.* udp 0 0 198.174.11.249.1056 198.174.11.239.2049 udp 0 0 198.174.11.249.1054 198.174.11.239.2049 udp 0 0 198.174.11.249.1046 198.174.11.239.2049 udp 0 0 198.174.11.249.1044 198.174.11.239.2049 udp 0 0 *.* *.* Active GRIT connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) grit 0 0 *:25 *:* grit 0 0 *:* *:* grit 0 0 *:32 *:* grit 0 0 *:* *:* grit 0 0 *:27 *:* Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr f0d5ff80 dgram 0 0 0 f0793b94 0 f0c4d694 f0cd4500 dgram 0 0 0 f0793b94 0 f0879494 f09eb200 stream 0 0 0 0 0 0 f0c22980 stream 0 0 0 0 0 0 f0dc0f80 stream 0 0 0 0 0 0 f0cd4e00 dgram 0 0 0 f0793b94 0 f0c8a414 f0cd4b00 dgram 0 0 0 f0793b94 0 0 #

3-10 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

netstat -s

Monitoring and Management Tools

Using the netstat command

netstat -s shows the statistics reported for all protocols, this excerpt shows the statistics reported for IP:

# netstat -s ip: 211338 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 29285 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 4885 packets reassembled ok 171636 packets for this host 2948 packets for unknown/unsupported protocol 0 packets forwarded 12295 packets not forwardable 0 redirects sent 8049 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-11

Monitoring and Management Tools

Obtaining layer 2 and 3 statistics - grstat

Options

- grid, displays the combus and other messaging statistics.

- ipstat, lists IP statistics.

- ipdrop, lists IPDROP statistics.

- ip, lists IPSTAT and IPDROP statistics.

- icmpin, lists ICMPIN (input) statistics.

- icmpout, lists ICMPOUT (output) statistics.

- icmperr, lists ICMPERR (error) statistics.

- icmp, lists ICMPERR, ICMPIN, and ICMPOUT statistics.

- l2, displays the Layer 2 media I/O statistics:

grstat l2rx , displays receive side statistics. grstat l2tx, displays transmit side statistics.

- switch, displays the switch statistics.

Layer 3 statistics

The grstat command reports layer 3 (IP and ICMP) forwarding statistics for SP Switch Router Adapter and all other media card types except HIPPI. Error reporting includes the saved source and destination IP addresses of the packet that caused the last error of each type reported.

List of IP stats

This is a list of all IP statistics returned with grstat, the -a option returns all entries even when 0 (they are not displayed without -a):

# grstat -a ip all cards (17 interfaces found) ipstat totals count description 13751 total packets received 1 packets dropped 2665 packets forwarded normally 0 packets redirected out receiving interface 0 packets fragmented 0 fragments created 0 packets forwarded locally to card 0 packets handled by the card 11085 packets forwarded to the RMS 0 packets segmented to the RMS 7593 multicast packets received

3-12 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Monitoring and Management Tools

Obtaining layer 2 and 3 statistics - grstat

0 multicast packets attempted to route 7593 multicast packets sent to RMS 0 multicast packets received on rincorrect interface 0 multicast packets for forwarding 0 multicast packets (copies) transmitted 0 packets ATMP encapsulated 0 packets ATMP decapsulated 0 packets forwarded to resolved bridge destinations 0 packets ATMP encapsulated with pre-fragmentation 0 packets ATMP encapsulated with pre-fragmentation and mtu-limit override 0 packets ATMP encapsulated with pre-fragmentation, clearing DF ipdrop totals count description 1 packet received on down interface 0 received data below IP header minimum 0 not version 4 IP 0 header length below minimum 0 bad header checksum 0 header length longer than packet length 0 received data less than packet length 0 source address from 127.*.*.* 0 source address from 192.0.2.* 0 packet filtered on input from media 0 no route to destination address 0 destination interface is down 0 TTL expired 0 needed to frag packet but DF set 0 multicast routing not yet supported 0 routing to bridge groups not supported 0 route table says to drop dest address 0 unknown special processing code 0 IP option with length <= 0 0 IP option length past header length 0 Record Route offset less than minimum 0 couln't find interface address for dest 0 TimeStamp offset less than minimum 0 bad TimeStamp option flag 0 TimeStamp option overflow beyond maximum 0 Source Route offset less than minimum 0 next Strict Source Route address not ours 0 bad ICMP checksum 0 local interface is down for ICMP ECHO 0 no route back for reply to ICMP ECHO 0 no next route for Strict Source Route 0 no next route for Loose Source Route 0 no interface addr for Source Route 0 can't forward link-layer broadcast 0 can't forward link-layer multicast 0 loose source routing disabled 0 strict source routing disabled 0 no buffer to generate packet 0 Rate overflow on ICMP generation 0 packet filtered into-me 0 ATMP err: can't find Home Network entry 0 ATMP err: bad GRE header

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-13

Monitoring and Management Tools

Obtaining layer 2 and 3 statistics - grstat

0 ATMP err: can't find mobile node entry 0 ATMP err: Invalid IP header 0 multicast replicated, original dropped 0 multicast TTL expired 0 multicast packet on wrong interface 0 can't forward to/through control board

Layer 2 statistics

The grstat command reports many of the Layer 2 (data link layer) statistics currently reported by individual media card maint commands. Layer 2 statistics are reported for SP Switch Router Adapter, ATM OC-3c (ATM/Q), HSSI, Ethernet, and SONET OC-3c media cards.

List of layer 2 stats

The grstat command has options to return layer 2 data per interface or per media card.

# grstat -a l2 gt000 gt000 Layer 2 statistics physical port 0 count description 297233 RX packets 107210916 RX bytes 0 RX errors 0 RX discard 9847 TX packets 708984 TX bytes 0 FIFO full 0 Line down

Please refer to the GRF Reference Guide for a description of grstat options and uses.

3-14 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

Each type of media card has a set of maintenance or maint commands. Some of the commands operate on most types of media cards, others are media-speciﬁc.

A small set of the maint commands provide card-speciﬁc status information. The majority of maint commands are useful only to system developers. You can use the maint commands listed below at any time, they do not interrupt or affect media card operations.

Several maint commands that return useful feedback on card status useful to administrators and support staff are:

– maint 3, Display Conﬁguration and Status – maint 4, Display Media Statistics – maint 5, Display SWITCH Statistics – maint 6, Display Combus Statistics – maint 189 displays the ARP table for the speciﬁed interface’s IP address

The SP Switch Router Adapter card has two processors, one on the receive side, one on the transmit side. One set of maint commands returns developer-level information and information speciﬁc to the receive side. A small set of maint commands returns developer-level information speciﬁc to the transmit side. Commands 81–89 are reserved because certain of their effects upon the card can be destructive.

You must start the grrmb program before you can enter a maint command.

Preparing to use maint commands

First, start the grrmb program to invoke the GR 66> prompt. From the UNIX shell enter:

# grrmb

The new prompt appears:

GR 66>

Use the port command to change the prompt number to the slot occupied by the SP Switch Router Adapter media card. For example, if the card is in slot 7, enter:

GR 66> port 7

This message is returned along with the changed prompt:

Current port card is 07 GR 07>

To exit the grrmb program, enter quit:

GR 07> quit #

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-15

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

Sample maint commands

The next pages show examples of maint commands you may ﬁnd useful.

Find hardware and software version numbers - maint 2

The maint 2 command returns the system software version and the card’s receive side (rx) binary version:

GR 0> maint 2 [RX] [RX] DEV1 Port Card Hardware and Software Revisions: [RX] =============================================== [RX] [RX] HW: [RX] Power-On Self-Test (POST) result code: 0x0. [RX] DEV1 Media Board HW Rev: 0x6, with 4M Sram. [RX] DEV1 Xilinx Version: 0x0. [RX] SDC Board HW Rev: 0x9 (SDC2). [RX] SDC2 Combus Xilinx version: 0x6. [RX] SDC2 Switch Transmit Xilinx version: 0x5. [RX] SDC2 Switch Receive Xilinx version: 0x8. [RX] SW: [RX] DEV1 Code Version: A1_4_20,Compiled Mon Aug 30 15:20:26 CDT [RX] 1999,in directory: /nit/A1_4_20/dev1/rx. [RX] IF Library Version: 1.1.0.0, Compiled on Mon Aug 30 [RX] 15:16:07 CDT 1999.

Find transmit (tx) binary version - maint 102

The maint 102 command returns the card’s transmit side (tx) binary version:

GR 0> maint 102 [TX] [TX] Code Version: A1_4_20 [TX] Compiled in: /nit/A1_4_20/dev1/tx, [TX] on: Mon Aug 30 15:21:35 CDT 1999.

Display conﬁguration and status - maint 3

Use maint 3 to display internal SP Switch Router Adapter card conﬁguration parameters. All of the reported values relate to the SP Switch Router Adapter card, they do not reﬂect the SP Switch.

GR 07> maint 3 [RX] Configuration Parameters:

[RX] Slot Number..........: 7

[RX] Node Number..........: 9

[RX] Node Name............: 07

[RX] SW Token.............: 0001000600

[RX] Arp Enabled..........: 1

[RX] SW Node Number.......: 2

[RX] IP...................: 0xce92a243

[RX] IP Mask..............: 0xffffffc0

[RX] Alias IP.............: 0x0

[RX] Max Link Size........: 1024

3-16 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

[RX] Host Offset..........: 1

[RX] Config State.........: 1

[RX] System Name..........: giga_SP

[RX] Node State...........: 2

[RX] Switch Chip Link.....: 0

[RX] Transmit Delay.......: 31

maint 4 - display media statistics

Use maint 4 to return a set of data transfer statistics for the input and output ports (ignore the report for port 1):

GR 07> maint 4 [RX] Media Statistics [RX] input: [RX] Port Bytes Packets Errors Discards [RX]

-----------------------------------------------------------------[RX] 0 00000000055233360 00000000000000843 0000000002 0000000002 [RX] 1 00000000000000000 00000000000000000 0000000000 0000000000 [RX] [RX] output: [RX] Port Bytes Packets Discards [RX] ------------------------------------------------------[RX] 0 00000000000339524640 00000000000000005182 0000000000 [RX] 1 00000000000000000000 00000000000000000000 0000000000 [RX] [RX] Port 0: [RX] Odd Length TX Packets: 0 [RX] TX Dropped Fifo Full: 0 [RX] TX Dropped Line Down: 0 [RX] Port 1: [RX] Odd Length TX Packets: 0 [RX] TX Dropped Fifo Full: 0 [RX] TX Dropped Line Down: 0

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

maint 5 - display switch statistics

This command provides you with information about data transfers between the SP Switch Router’s internal switch and the SP Switch Router Adapter card. Enter: maint 5

GR 07> maint 5 [RX] Switch Statistics [RX] input: [RX] Bytes Packets Errors [RX] --------------------------------------------------------[RX] 000000000000000000 000000000000000000 000000000 [RX] [RX] output: [RX] Bytes Packets Errors [RX] --------------------------------------------------------[RX] 000000000000000000 000000000000000000 000000000 [RX] [RX] Switch Transmit Data Errors: 0 [RX] Switch Transmit Fifo Parity Errors: 0 [RX] Switch Transmit Internal Parity Errors: 0

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-17

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

[RX] Switch Transmit Connection Rejects: 0 [RX] Switch Receive Encoding Errors: 0 [RX] Switch Receive Running Disparity Errors: 0 [RX] Switch Receive Receiver Errors: 0 [RX] Switch Receive Running Checksum Errors: 0 [RX] dont-free packets: 0 [RX] fifo-full packets: 0

maint 6 - display combus statistics

To look at information about the exchanges between the system software and the SP Switch Router Adapter card, enter maint 6 for the receive side and maint 106 for the transmit side.

GR 07> maint 6 [RX] Combus Status: [RX] Last interrupt status: 0x50502055 [RX] Combus Statistics: [RX] Message ready interrupts: 16153 [RX] Truncated input messages: 0 [RX] Grit messages for TX-CPU: 155 [RX] Ip messages Rcvd (non-bypass): 0 [RX] Raw messages: 0 [RX] ISO messages: 0 [RX] Grid messages: 15998 [RX] Grid echo requests: 15865 [RX] Port available messages: 0 [RX] Segmented Packets: 0 [RX] Segments Sent: 0 [RX] Combus Errors: [RX] Bus in timeouts: 0 Bus out timeouts: 0 [RX] Out of buffer cond.: 0 Bad packet type: 0 [RX] Dropped IP packets: 0 Bad packet dest: 0 [RX] Receive Msg Errors: 0 Receive Format Errors: 0 [RX] Receive Past End: 0 Received Long Message: 0

Filtering commands - maint 50-58, 150-58

Odd-numbered commands 51–55 return information based on ﬁlter ID. Even-numbered commands 50–54 return information based on interface number. These are parameters added to some ﬁlter commands:

– detail_level is an optional parameter that speciﬁes how much information is

returned, useful levels are 0 and 1.

– IF is an optional parameter that speciﬁes the interface number. – ID is an optional parameter that speciﬁes the ﬁlter ID randomly assigned by ﬁlterd.

[RX] 50: Filtering filter list: [detail_level [ID]] [RX] 51: Filtering filter list: [detail_level [IF]] [RX] 52: Filtering action list: [detail_level [ID]] [RX] 53: Filtering action list: [detail_level [IF]] [RX] 54: Filtering binding list: [detail_level [ID]] [RX] 55: Filtering binding list: [detail_level [IF]] [RX] 56: Display filtering statistics: [IF#] [RX] 57: Reset filtering statistics: [IF#] [RX] 58: Show filter protocol statistics

3-18 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

[RX] note, IF/ID may be '-1' to indicate all of the given [RX] item while detail level is 0|1|2.

Please refer to the “IP Packet Filtering” chapter in the GRF Conﬁguration and Management manual for a full discussion of the ﬁltering command set. A few examples are shown here.

List the ﬁlters per media card - maint 50

The ﬁlters are listed per interface:

GR 0> maint 50 gt000 [RX] [RX] filterID type status access [RX] 00000022 ctable (loaded) 0002

You need the grfutil command to decipher the ﬁlterID value and get the ﬁlter name:

# grfutil -f 22 filterID temp BPF SPARC SSPARC C30 CTABLE fname 00000022 No Yes No No No Yes mail_server_allow

You see a message if no ﬁlters are found:

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

GR 0> maint 50 [RX] [RX] No filters found.

List where ﬁlters are assigned - maint 54

This command shows which logical interface numbers (vlif) have ﬁlter “ID 22” assigned:

GR 00> maint 54 22 [RX] vlif BindID state location filterID action_cnt [RX] 0000 00000040 FastOp IPin 22 1 [RX] 0000 00000041 FastOp IPout 22 1 [RX] 0001 00000042 FastOp IPin 22 1 [RX] 0001 00000043 FastOp IPout 22 1

Conﬁgure UDP packet discards - maint 89 7

The maint 89 7 command provides a way to drop certain packets such as HATS trafﬁc that are known to be unnecessary. Minimizing unnecessary trafﬁc to the SP Switch Router management system frees the router’s communication bus for other internal messaging.

A site uses the command to specify that all UDP packets bound for a speciﬁed port are discarded. After the speciﬁed UDP port number is entered via maint 89 7 port_number, all UDP messages with source and destination ports set to port_number are discarded.

This example discards HATS packets and shows how to specify the target UDP port of 10000 on the SP Switch Router Adapter card in slot 13. The “port set” message conﬁrms the setting:

# grrmb GR 1> port 13 Current port is 13. GR 13> GR 13> maint 89 7 10000

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-19

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

[RX] [RX] SP HATS UDP port set to: 10000

Use maint 89 8 to view the current port number setting and number of messages discarded:

GR 13> maint 89 8 [RX] SP HATS UDP port number: 10000 [RX] SP HATS (RMS) discards: 654

To disable UDP discards, set the port number to an unused number (the range is 0 to 65535):

GR 13> maint 89 7 65533 [RX] SP HATS UDP port set to: 65533

If you enter an out-of-range port number, here is the response:

GR 13> maint 89 7 101010 [RX] Illegal port number (0 << port# << 65535

Display ARP table - maint 189 1

The maint 189 1 command returns the current ARP entries. IP= is the address for the node, SW-node= is a physical address assigned to an SP switch node, and state=3 means the node is active.

GR 0> maint 189 1 [TX] [TX] ARP Entry: IP= 192.168.178.202, SW-node=5, state=3 [TX] ARP Entry: IP= 192.168.178.203, SW-node=6, state=3 [TX] ARP Entry: IP= 192.168.178.204, SW-node=7, state=3 [TX] ARP Entry: IP= 192.168.178.205, SW-node=1, state=3 [TX] ARP Entry: IP= 192.168.178.206, SW-node=2, state=3 [TX] ARP Entry: IP= 192.168.178.207, SW-node=3, state=3 [TX] ARP Entry: IP= 192.168.178.208, SW-node=4, state=3

Note:

The SW-node= is the same as the Node_ID: parameter in maint 189 2 data.

Flush the ARP cache - maint 189 10

This command removes all ARP entries:

GR 0> maint 189 10 [TX] [TX] Flush ARP Entry: IP= 192.168.178.202, SW-node=5, state=3 [TX] Flush ARP Entry: IP= 192.168.178.203, SW-node=6, state=3 [TX] Flush ARP Entry: IP= 192.168.178.204, SW-node=7, state=3 [TX] Flush ARP Entry: IP= 192.168.178.205, SW-node=1, state=3 [TX] Flush ARP Entry: IP= 192.168.178.206, SW-node=2, state=3 [TX] Flush ARP Entry: IP= 192.168.178.207, SW-node=3, state=3 [TX] Flush ARP Entry: IP= 192.168.178.208, SW-node=4, state=3 [TX] Flush ARP cache complete

3-20 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Monitoring and Management Tools

SP Switch Router Adapter card maint commands

Display switch route table - maint 189 2

The SP Switch sends the SP Switch Router Adapter card routes for the active nodes. The SP Switch Router Adapter card receives these routes and appends the hex addresses to packets going to the target node on the SP Switch (identiﬁed by Node_ID.

GR 0> maint 189 2 [TX] Switch Route Table: [TX] Node_ID: 0, Loopback Routes [TX] Route-1: [0x80000000, 0x1], Route-2: [0x80000000, 0x1] [TX] Route-3: [0x80000000, 0x1], Route-4: [0x80000000, 0x1] [TX] Node_ID: 1 [TX] Route-1: [0x8a000000, 0x1], Route-2: [0x8a000000, 0x1] [TX] Route-3: [0x8a000000, 0x1], Route-4: [0x8a000000, 0x1] [TX] Node_ID: 2 [TX] Route-1: [0x46800000, 0x2], Route-2: [0x5e800000, 0x2] [TX] Route-3: [0x6e800000, 0x2], Route-4: [0x76800000, 0x2] [TX] Node_ID: 3 [TX] Route-1: [0x46890000, 0x2], Route-2: [0x5e890000, 0x2] [TX] Route-3: [0x6e890000, 0x2], Route-4: [0x76890000, 0x2] [TX] Node_ID: 4 [TX] Route-1: [0x89000000, 0x1], Route-2: [0x89000000, 0x1] [TX] Route-3: [0x89000000, 0x1], Route-4: [0x89000000, 0x1] [TX] Node_ID: 5 [TX] Route-1: [0x80000000, 0x1], Route-2: [0x80000000, 0x1] [TX] Route-3: [0x80000000, 0x1], Route-4: [0x80000000, 0x1] [TX] Node_ID: 6 [TX] Route-1: [0x468a0000, 0x2], Route-2: [0x5e8a0000, 0x2] [TX] Route-3: [0x6e8a0000, 0x2], Route-4: [0x768a0000, 0x2] [TX] Node_ID: 7 [TX] Route-1: [0x46830000, 0x2], Route-2: [0x5e830000, 0x2] [TX] Route-3: [0x6e830000, 0x2], Route-4: [0x76830000, 0x2]

Note: The Node_ID: is the same as the SW-node= parameter in maint 189 1 and 10 data. If switch route table addresses do not match the SP Switch Router Adapter card entries, the

gr.console log displays a series of “Switch route not found” messages.

“Switch route not found”

The message is sent during ARP processing when a searched-for entry is not found in the switch route table. The table mismatch is due either to a conﬁguration error or to corrupted packets coming from the the SP Switch. Here is an example of the message:

Switch route not found, node=33386

The node= value is the SP Switch node number in decimal. If the node= value is lo w, the cause is probably a conﬁguration error. With ARP mode set to 0, the card’s ARP table does not have a switch route from the switch to match the destination IP address/netmask for an ARP to be created. The message also appears when the SP Switch Router Adapter media card receives an ARP from the switch, but the hardware address in the ARP packet does not match any switch routes in the switch route table. In either case, the message means the switch route table that the SP Switch Router Adapter card receives from the SP Switch is not matching up with addresses that are attempting to ﬂow through the media card.

If the node= value is high, as in the example, it is likely due to corrupted packets coming from the SP Switch.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-21

Monitoring and Management Tools

Checking for hardware problems - grdiag

This section describes the diagnostic capability provided by the grdiag command. Users can run a set of internal BIST-level diagnostics to verify media card (including SP Switch Router Adapter card) hardware. A card that f ails this set of diagnostics must be replaced. HIPPI media cards do not support the grdiag command.

The grdiag script puts the selected media card(s) into diagnostic mode and runs the diagnostics. After the diagnostics complete, grdiag reloads the media card’s software and conﬁguration currently saved in ﬂash memory, then reboots the card. For this reason, it is very important that you save any conﬁguration changes before you run grdiag. Unsa ved media card changes will be lost. These diagnostics affect the operation of only the target card or cards.

You can run diagnostics on all the chassis cards at the same time. The length of time needed for the diagnostic to run depends on the type of media card and how many cards are being tested at one time.

What is tested

grdiag is intended to help users determine whether hardware is causing a problem that is being seen. These diagnostics do not determine which type of hardware failure occurred. The diagnostics report no error information, only pass-fail results.

The diagnostics verify the following media card and slot functions:

– all memory – all media hardware logic (media card) – all serial hardware logic (serial daughter card) – the connection between the slot and the switch – the connection between the slot and the communications bus – the connection between the slot and power delivery

The diagnostics do not exercise the physical interfaces or transceivers. Generally, you can expect to test 95% of the media card.

Where to ﬁnd the user guide

The “Management Command and Tools” chapter in the GRF Conﬁguration and Management manual contains a complete usage guide and examples for using grdiag.

Stopping or halting grdiag

You can use Control-C to stop the diagnostic sequence at any time. After you enter Control-C, grdiag reloads the card’s run-time binary and last-saved conﬁguration, and then reboots the card.

3-22 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Monitoring and Management Tools

Checking for hardware problems - grdiag

When a media card does not boot after grdiag

For grdiag to run, a card must be able to boot. If the grcard display does not include the slot in which the problem card resides, grdiag cannot operate on that card.

For example, grdiag cannot run diagnostics on the card in slot 1 of this router:

# grcard 0 ATM_OC3_V2 running 2 DEV1_V1 running 3 HSSI_V1 running

“Switch receive error” can indicate hardware problem

The Switch Receive Error means a media card has received a packet from the SP Switch Router backplane switch that contains errors. Here are examples of such packets sent by the card in slot 3 and received by the Ethernet card in slot 4:

Jul 20 02:25:04 grf400.site.com gritd: from 0:0x3:0: dst=0:0x40:16, src=0:0x3:0, type=GRID: hwtype=ETHER_V1 cmd=MSGP '[RX] Switch Recieve Error. Status: 0xb04\r\n'

•

• Jul 20 02:26:16 grf400.site.com gritd: from 0:0x3:0: dst=0:0x40:16, src=0:0x3:0, type=GRID: hwtype=ETHER_V1 cmd=MSGP '[RX] Switch Recieve Error. Status: 0xb04\r\n'

An occasional occurrence of this error in the gr.console log is part of normal operations and can safely be ignored.

However, if clusters of many such errors are reported, it is likely that there is a problem with the switch hardware in either the sending media card or the receiving media card. A card may need to be replaced. If you see the message repeated many times, please contact Customer Service.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 3-23

Monitoring and Management Tools

SP Switch Router dumps

The SP Switch Router can be conﬁgured to send dumps to an external PCMCIA 520MB disk rather than to its own system memory.

On the SP Switch Router, output from dumps, logs, and other system reporting functions refer to the SP Switch Router Adapter card as DEV1_v1 or dev1. Dumps are maintained in the directories /var/portcards and /var/crash.

Dumps are compressed to save space, and the compressed ﬁles are appended with .gz. Media card dumps are stored in /var/portcards in a ﬁle named with the convention

grdump.n.x.gz

where n is the card slot number and x is the number of the saved dump, 1, 2, 3....

Dumps provide speciﬁc information useful for monitoring and debugging the SP Switch Router and SP Switch Router Adapter card operations. If you are working with Customer Support, these are dumps they might need to see:

– bsdx.core - a kernel dump resulting from a system reset or panic,

sent to /var/crash (x is number of the dump, 1, 2, 3...)

– grdump.n.x.gz - a dump resulting from a media card reset or panic,

sent to /var/portcards

System dumps

If the SP Switch Router is reset or panics, a dump is saved in /var/crash under the naming convention bsdx.core.

This dump is generally too large to send by e-mail. Customer Support will tell you how to send it to them.

The grsavecore command copies and formats information generated from a kernel panic as the data is written to standard output. The formatted data is written to grsavecore.out in the

/var/crash directory.

Media card dumps

The grdump program saves and manages media card dumps. As described in Chapter 2, you use the Dump or appropriate Card proﬁle to specify the number of dumps to be saved in addition to the ﬁrst dump of the day and the most current dump . The default number is two per day for each media card. Dumps are collected from media cards when they panic or when they are reset by the system administrator using the grreset -D command.

This dump is generally 4–8MB, and may possibly be e-mailed. Customer Support will tell you how to send it to them.

Use grdinfo to collect logs

With a single command, grdinfo collects the ﬁles in local /var/log/* (including compressed ﬁles)and compresses them in a log ﬁle. Refer to the grdinfo section in this chapter for more information.

3-24 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Lucent Technologies 9077 04S, 9076, RS/6000 User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual