Lucent Technologies 9077 16S, 9077 04S, 9076 User Manual

SP Switch Router Adapter Guide

1.4 Update 2

Part Number: 7820-2039-001

For software version 1.4.20 and later

October, 1999

This material is protected by the copyright laws of the United States and other countries. It may not be reproduced, distributed, or altered in any fashion by an y entity (either internal or external to Lucent Technologies), except in accordance with applicable agreements, contracts, or licensing, without the express written consent of Lucent Technologies.

For permission to reproduce or distribute, please contact: Alison Gowan, 1-612-996-6891

Notice

Every effort was made to ensure that the information in this document was complete and accurate at the time of printing. However, information is subject to change.

Trademarks

GRF is a trademark of Lucent Technologies. Other trademarks and trade names mentioned in this publication belong to their respective owners.

Limited Warranty

Lucent T echnologies pro vides a limited w arranty to this product. See Appendix B, "Limited Warranty," in the GRF 400/1600 Getting Started manual for more information.

Ordering Information

To order copies of this document, contact your Lucent Technologies representative or reseller.

Support Telephone Numbers

For a menu of support and other services, call (800) 272-3634. Or call (510) 769-6001 for an operator.

Lucent Technologies

Contents

About this Guide............................................................................. xiii

About 1.4 Update 2................................................................................................................ xiii

How to use this Guide............................................................................................................ xiii

Manual sets ............................................................................................................................ xiv

SP Switch Router manuals.............................................................................................. xiv

IBM SP system manuals................................................................................................. xiv

Documentation conventions..................................................................................................... xv

IP routing publications........................................................................................................... xvi

Chapter 1 Introduction to the SP Switch Router Adapter card.................... 1-1

What is the RS/6000 SP Switch Router ? ............................................................................. 1-2

SP Switch Router systems for IBM sites ............................................................................. 1-3

Cables included in your system ...................................................................................... 1-3

SP Switch cable ..................................................................................................... 1-3

Ethernet cable ......................................................................................................... 1-4

SP ground strap........................................................................................................ 1-4

PCMCIA 520MB disk .................................................................................................... 1-4

Redundant AC power supplies ....................................................................................... 1-4

Redundant supply safety ......................................................................................... 1-4

Upgrading system memory.................................................................................................... 1-5

Overview of the SP Switch Router Adapter card................................................................... 1-6

Face plate diagram ........................................................................................................ 1-6

Inserting a media card into the SP Switch Router ................................................................. 1-7

ESD requirements........................................................................................................... 1-8

Card insertion procedure................................................................................................. 1-8

SP Switch Router Adapter card LEDs .................................................................................. 1-9

LED activity during boot................................................................................................ 1-9

LED activity during normal operations ........................................................................ 1-11

SP Switch Router Adapter card specifications..................................................................... 1-13

Assigning filters .................................................................................................................. 1-14

SNMP on the SP Switch Router Adapter card..................................................................... 1-15

SP Switch Router Adapter dependent node MIB support ............................................ 1-15

SP Switch Router Adapter media card states (SNMP) ................................................ 1-16

SNMP configuration overview..................................................................................... 1-17

SNMP activity during media card start up .................................................................. 1-18

SP Switch Router Adapter Guide - 1.4 Update 2 iii

Contents

Chapter 2 Conﬁguring the SP Switch Router Adapter.................................. 2-1

Introduction to installation and configuration ...................................................................... 2-2

Location of relevant information.................................................................................... 2-3

Pre-installation assumptions ......................................................................................... 2-3

Order of information....................................................................................................... 2-4

Installing an SP Switch Router Adapter card......................................................................... 2-5

Installation overview....................................................................................................... 2-5

Installing the PCMCIA spinning disk.................................................................................... 2-6

Managing PCMCIA slots................................................................................................ 2-6

Panic dumps sent to external flash device .................................................................... 2-7

Installation steps.............................................................................................................. 2-7

Attaching SP Switch Router cables ................................................................................... 2-10

Ethernet cable................................................................................................................ 2-10

SP switch cable............................................................................................................. 2-10

Procedure for attaching cables to card and SP Switch ................................................. 2-11

Configuration required on the SP system ............................................................................ 2-12

Determining the switch connection for a dependent node............................................ 2-12

Procedure............................................................................................................... 2-13

Sources of configuration information .......................................................................... 2-14

Multiple frames for multiple system connections ........................................................ 2-15

Step-by-step media card configuration ................................................................................ 2-16

Configuration files and their uses ................................................................................ 2-16

Overview of steps 1–5 .................................................................................................. 2-16

Step 1. Check SNMP in the SP Switch Router system....................................................... 2-18

Put SNMP changes into effect...................................................................................... 2-20

Step 2. Assign IP addresses ................................................................................................ 2-21

Method 1: Recommended, use SP SNMP Manager ................................................... 2-21

Method 2: Optional, edit /etc/grifconfig.conf.............................................................. 2-21

Interface name ...................................................................................................... 2-22

Internet address...................................................................................................... 2-22

Netmask ................................................................................................................ 2-22

Broadcast / destination address ............................................................................. 2-22

Argument field ..................................................................................................... 2-23

Putting grifconfig.conf additions into effect ............................................................... 2-23

Step 3. Change profile settings .......................................................................................... 2-24

Specify card-level parameters – Card profile ............................................................. 2-24

Specify ICMP throttling ........................................................................................ 2-24

Specify different executables ................................................................................ 2-25

Specify different dump settings ........................................................................... 2-25

Change executables for all dev1 cards - Load profile ................................................. 2-27

Change dump defaults for all dev1 cards - Dump profile ............................................ 2-28

Dump vectors (read-only)..................................................................................... 2-29

Step 4. Run dev1config to create grdev1.conf.................................................................... 2-31

Method 1: Recommended, use SP SNMP Manager.................................................... 2-31

Method 2: Optional, edit /etc/grdev1.conf................................................................... 2-31

How to run the command.............................................................................................. 2-31

Contents of /etc/grdev1.conf......................................................................................... 2-32

iv SP Switch Router Adapter Guide - 1.4 Update 2

Contents

Parameter definitions.................................................................................................... 2-33

Step 5. Reset card to install files......................................................................................... 2-35

Saving configuration files............................................................................................. 2-35

Verify SP Switch Router Adapter card from router ........................................................... 2-36

Verify media card operation using ping ...................................................................... 2-36

Verify switch node connectivity using ping ................................................................. 2-37

Check media card status using grcard .......................................................................... 2-37

Media card states ................................................................................................... 2-37

Reset media card using grreset ................................................................................... 2-38

Bringing the SP Switch Router Adapter card on-line with the SP ...................................... 2-40

Checking connectivity to the SP system....................................................................... 2-40

Procedure ...................................................................................................................... 2-41

Chapter 3 Monitoring and Management Tools............................................... 3-1

SP Switch Router command overview................................................................................... 3-2

csconfig .......................................................................................................................... 3-2

flashcmd.......................................................................................................................... 3-2

getver............................................................................................................................... 3-3

grarp .............................................................................................................................. 3-3

grcard ............................................................................................................................. 3-3

grfins............................................................................................................................... 3-3

grms ............................................................................................................................... 3-4

grreset.............................................................................................................................. 3-4

grrmb ............................................................................................................................. 3-4

grroute............................................................................................................................. 3-4

grrt................................................................................................................................... 3-4

grsite................................................................................................................................ 3-4

grsnapshot....................................................................................................................... 3-4

grstat ............................................................................................................................... 3-5

grwrite ........................................................................................................................... 3-5

mountf............................................................................................................................. 3-5

setver............................................................................................................................... 3-5

umountf........................................................................................................................... 3-5

vpurge ............................................................................................................................ 3-5

SP Switch Router UNIX tools ............................................................................................... 3-6

ping ............................................................................................................................... 3-6

route ................................................................................................................................ 3-6

tcpdump .......................................................................................................................... 3-7

traceroute ........................................................................................................................ 3-7

Using the netstat command ................................................................................................... 3-8

netstat -rn ...................................................................................................................... 3-8

netstat -rs ....................................................................................................................... 3-9

netstat -in ........................................................................................................................ 3-9

netstat -an ..................................................................................................................... 3-10

netstat -s........................................................................................................................ 3-11

Obtaining layer 2 and 3 statistics - grstat............................................................................. 3-12

Options.......................................................................................................................... 3-12

Layer 3 statistics ........................................................................................................... 3-12

List of IP stats........................................................................................................ 3-12

SP Switch Router Adapter Guide - 1.4 Update 2 v

Contents

Layer 2 statistics ........................................................................................................... 3-14

List of layer 2 stats................................................................................................. 3-14

SP Switch Router Adapter card maint commands............................................................... 3-15

Preparing to use maint commands................................................................................ 3-15

Sample maint commands.............................................................................................. 3-16

Find hardware and software version numbers - maint 2 ....................................... 3-16

Find transmit (tx) binary version - maint 102........................................................ 3-16

Display configuration and status - maint 3............................................................ 3-16

maint 4 - display media statistics .......................................................................... 3-17

maint 5 - display switch statistics.......................................................................... 3-17

maint 6 - display combus statistics........................................................................ 3-18

Filtering commands - maint 50-58, 150-58 .......................................................... 3-18

List the filters per media card - maint 50 .............................................................. 3-19

List where filters are assigned - maint 54 ............................................................. 3-19

Configure UDP packet discards - maint 89 7........................................................ 3-19

Display ARP table - maint 189 1 .......................................................................... 3-20

Flush the ARP cache - maint 189 10 .................................................................... 3-20

Display switch route table - maint 189 2 .............................................................. 3-21

“Switch route not found”....................................................................................... 3-21

Checking for hardware problems - grdiag .......................................................................... 3-22

What is tested ............................................................................................................... 3-22

Where to find the user guide......................................................................................... 3-22

Stopping or halting grdiag ............................................................................................ 3-22

When a media card does not boot after grdiag ............................................................ 3-23

“Switch receive error” can indicate hardware problem ............................................... 3-23

SP Switch Router dumps ..................................................................................................... 3-24

System dumps............................................................................................................... 3-24

Media card dumps ........................................................................................................ 3-24

Use grdinfo to collect logs .......................................................................................... 3-24

Data collection utility - grdinfo............................................................................................ 3-25

Options ........................................................................................................................ 3-25

SP Switch Router example............................................................................................ 3-26

SP Switch Router logs ........................................................................................................ 3-30

Accessing a log file....................................................................................................... 3-30

Sample gr.console log................................................................................................... 3-31

Sample gr.boot log........................................................................................................ 3-32

Sample messages log .................................................................................................... 3-33

Burning in media card flash memory................................................................................... 3-34

Appendix A Part Numbers .................................................................................. A-1

Parts list – model 04S............................................................................................................ A-1

Parts list – model 16S............................................................................................................ A-3

Publication numbers – IBM manuals.................................................................................... A-4

Appendix B Log Messages................................................................................. B-1

Alphabetical list of messages................................................................................................. B-1

Message descriptions ............................................................................................................. B-3

vi SP Switch Router Adapter Guide - 1.4 Update 2

Contents

Appendix C Network Conﬁguration Examples ................................................. C-1

Example 1: Single SP Switch Router Adapter card, single SP partition....................... C-2

Configuration requirements..................................................................................... C-2

Example 2: Multiple cards, single partition................................................................... C-3

Configuration requirements..................................................................................... C-3

Configuration tasks.................................................................................................. C-4

Incoming traffic (going to SP processor nodes)..................................................... C-4

Outgoing traffic (coming from SP processor nodes).............................................. C-4

Recovery procedure if an SP Switch Router Adapter card fails..................................... C-5

Example 3: Multiple cards, multiple SP partitions:........................................................ C-6

Configuration tasks.................................................................................................. C-6

Appendix D Upgrading Router Software........................................................... D-1

The SP Switch Router as an IBM product..................................................................... D-1

Obtaining new machine code ........................................................................................ D-1

Support for code installation.......................................................................................... D-1

IBM License Agreement for Machine Code.................................................................. D-2

Index.......................................................................................... Index-1

SP Switch Router Adapter Guide - 1.4 Update 2 vii

Contents

viii SP Switch Router Adapter Guide - 1.4 Update 2

Figures

Figure 1-1 Connections between the SP Switch Router and an SP system .............. 1-2

Figure 1-2 Expandable area of system memory............................................................. 1-5

Figure 1-3 Media card components................................................................................ 1-7

Figure 1-4 LEDs on the SP Switch Router Adapter card............................................... 1-9

Figure 2-1 Components connecting an SP Switch Router to an SP Switch and control

workstation.................................................................................................... 2-2

Figure 2-2 SP system administrative Ethernet connections ......................................... 2-10

Figure 2-3 SP Switch Router Adapter cable 50-pin connector end.............................. 2-10

Figure 2-4 How frames enable connections to multiple SP Switches.......................... 2-15

Figure 2-5 Components in the SP Switch Router Adapter card’s interface name ....... 2-22

Figure 3-1 SP Switch Router control board memory components................................. 3-2

Figure 3-2 Sample entries in the gr.console log........................................................... 3-31

Figure 3-3 Sample entries in the gr.boot log................................................................ 3-32

Figure 3-4 Sample entries in the messages log ............................................................ 3-33

Figure 3-5 Sample entries in the gr.boot log................................................................ 3-34

Figure C-1 Example 1 – one card, one SP partition network configuration................... C-2

Figure C-2 Example 2 – multiple card, single SP partition configuration...................... C-3

Figure C-3 Recovery from a card failure in a dually-connected configuration.............. C-5

Figure C-4 Example 3 – multiple card, multiple SP partition configuration.................. C-6

SP Switch Router Adapter Guide - 1.4 Update 2 ix

Figures

x SP Switch Router Adapter Guide - 1.4 Update 2

Tables

Table 1-1 SP Switch Router Adapter card LED activity during boot and reset............ 1-9

Table 1-2 SP Switch Router Adapter media card LEDs............................................. 1-11

Table 1-3 SP Switch Router Adapter media card specifications................................. 1-13

Table A-1 IBM and Lucent part numbers for model 04S............................................ A-1

Table A-2 IBM and Lucent part numbers for model 16S............................................. A-3

Table A-3 Publication numbers for related IBM manuals............................................ A-4

SP Switch Router Adapter Guide - 1.4 Update 2 xi

Tables

xii SP Switch Router Adapter Guide - 1.4 Update 2

About this Guide

Lucent’s GRF switched IP routers can be used to pro vide high-speed data communication links between IBM RS/6000 Scalable POWERparallel Systems (SP) and external networks/hosts. When packaged with an IBM SP system, the GRF router is referred to as an RS/6000 SP Switch Router, or SP Switch Router. The SP Switch Router Adapter card is the GRF media card that speciﬁcally supports SP system data transfers. To connect to an SP system, the SP Switch Router Adapter card is cabled directly to an SP Switch port.

The SP Switch Router Adapter Guide describes the media card itself and explains how to install, verify , and conﬁgure the card. The Guide provides the same type of information for this media card as is provided for other GRF media cards. Information speciﬁc to installing and conﬁguring a GRF router is found in the manuals listed below in the “Manual sets” section.

The RS/6000 SP Switch Router is based on the GRF 400 and GRF1600 routers manufactured by Lucent Technologies. For that reason, this manual contains references to the

GRF 400/1600 Getting Started , GRF Reference Guide , and GRF Conﬁguration and Management manuals.

About 1.4 Update 2

The GRF 1.4 Update 2 manual set includes new features added since software release 1.4.12. GateD information is provided in a separate document, the GRF GateD Manual.

How to use this Guide

The Guide contains the following chapters and an index:

• Chapter 1, “Introduction to the SP Switch Router Adapter Card,” describes the SP Switch

Router Adapter media card, its cables, LEDs, and SNMP implementation.

• Chapter 2, “Conﬁguring the SP Switch Router Adapter,” explains how to conﬁgure the SP

Switch Router Adapter media card and how to attach it to the SP Switch.

• Chapter 3, “Monitoring and Management Tools,” contains information about commands,

logs, and dumps useful for maintaining the SP Switch Router and adapter card.

• Appendix A, “Part Numbers,” contains a table of corresponding Lucent and IBM part

numbers for SP Switch Router components.

• Appendix B, “Log Messages, ” contains e xplanations of log messages generated by the SP

Switch Router Adapter media card.

• Appendix C, “Network Conﬁguration Examples,” contains examples and descriptions of

three basic SP Switch network conﬁgurations and their requirements.

• Appendix D, “Upgrading SP Switch Router Software,” describes how to install new

releases of the router’s operating software.

SP Switch Router Adapter Guide - 1.4 Update 2 October 25, 1999 xiii

About this Guide

Manual sets

This section provides a list of relevant GRF manuals. A second list includes IBM system manuals that contain information speciﬁc to the SP supercomputer.

SP Switch Router manuals

The SP Switch Router Adapter media card is described only in the SP Switch Router Adapter

Guide . The SP Switch Router and other media cards are described in the GRF manual set.

The GRF 1.4 Update 2 documentation set consists of the following manuals:

• GRF 400/1600 Getting Started - 1.4 Update 2

• GRF Conﬁguration and Management - 1.4 Update 2

• GRF Reference Guide - 1.4 Update 2

• GRF GateD Manual - 1.4 Update 2

• SP Switch Router Adapter Guide - 1.4 Update 2

Router manuals and software upgrade release notes for the 9077 are available in PDF format from this Web site:

http://www.rs6000.ibm.com/resource/aix_resource/sp_books/sra /index.html

To check which software release your SP Switch Router is running, use the getver command:

super> getver Current Revision: 1.4.20.ibm Version: default super>

IBM SP system manuals

IBM information speciﬁc to the SP Switch Router Adapter card appears in:

• IBM RS/6000 Scalable POWERparallel Systems:

Planning, Volume 1, Hardware and Physical Environment, GA22-7280

• IBM RS/6000 Scalable POWERparallel Systems:

Planning, Volume 2, Control Workstation and Software Environment, GA22-7281

• IBM Parallel System Support Programs for AIX:

Installation and Migration Guide, GA22-7347

• IBM Parallel System Support Programs for AIX:

Administration Guide, SA22-7348

• IBM Parallel System Support Programs for AIX:

Diagnosis Guide, GA22-7350

• IBM Parallel System Support Programs for AIX:

Command and Technical Reference, Volume 1 , SA22-7351

• IBM Parallel System Support Programs for AIX:

Command and Technical Reference, Volume 2, SA22-7351

• IBM Parallel System Support Programs for AIX:

Messages Reference, GA22-7352

xiv October 25, 1999

SP Switch Router Adapter Guide - 1.4 Update 2

You can download PDF versions of these manuals from the RS6000 SP Product Documentation Library at this web site:

http://www.rs6000.ibm.com/resource/aix_resource/sp_books/ http://www.rs6000.ibm.com/resource/aix_resource/sp_books/pssp/inde x.html

Documentation conventions

This manual uses the following standard documentation conventions:

Convention Meaning

Monospace text Represents text that appears on your computer’s screen, or that could

appear on your computer’s screen.

Boldface text

Italics

[ ] Square brackets indicate an optional argument you might add to a

| Separates command choices that are mutually exclusive. Key1-Key2 Represents a combination keystroke. To enter a combination

Press Enter Means press the Enter, or Return, key or its equivalent on your

Note:

Represents characters that you enter exactly as shown (unless the characters are also in names used in text appear in boldface.

In command usage, italic represent variable information. Do not enter the words themselves in the command. Enter the information they represent. In ordinary text, italics are used for titles of publications, for some terms that would otherwise be in quotation marks, and to show emphasis.

command. To include such an argument, type only the information inside the brackets. Do not type the brackets unless they appear in bold type.

keystroke, press the ﬁrst key and hold it down while you press one or more other keys. Release all the keys at the same time. (For example, Ctrl-H means hold down the Control key and press the H key.)

computer. Introduces important additional information.

italics

About this Guide

Documentation conventions

—see Italics , below). Command

Caution:

Warning:

SP Switch Router Adapter Guide - 1.4 Update 2 October 25, 1999 xv

W arns that a failure to follo w the recommended procedure could result in loss of data or damage to equipment.

Warns that a failure to take appropriate safety precautions could result in physical injury.

About this Guide

IP routing publications

Here are some related publications that you may ﬁnd useful:

• Internetworking with TCP/IP, Volume 1 and 2, by Douglas E. Comer, and David L.

Stevens. Prentice-Hall,

• TCP/IP Illustrated, Volumes 1 and 2 , by W. Richard Stevens. Addison-Wesley, 1994.

• Interconnections , Radia Perlman. Addison-Wesley, 1992.

Recommended for information about routers and bridging.

• Routing in the Internet, by Christian Huitema. Prentice Hall PTR, 1995.

Recommended for information about IP, OSPF, CIDR, IP multicast, and mobile IP.

• TCP/IP Network Administration , by Craig Hunt. O’Reilly & Associates, Inc. 1994.

Recommended for network management information.

• Essential System Administration, Æleen Frisch. O’Reilly & Associates, Inc. 1991.

Recommended for network management information.

xvi October 25, 1999

SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

The RS/6000 SP Switch Router is based on the GRF 400 (4-card) and GRF 1600 (16-card) routers manufactured by Lucent Technologies. For that reason, this manual contains references to the GRF 400/1600 Getting Started , GRF Reference Guide , and

GRF Conﬁguration and Management manuals. The SP model of the router is referred to as the

SP Switch Router. SP Switch Routers can be used to provide high-speed data communication links between IBM

RS/6000 Scalable POWERparallel Systems (SP) and external networks/hosts. The SP Switch Router Adapter card is the router media card that speciﬁcally supports SP system data transfers. To connect to an SP system, the SP Switch Router Adapter card is cabled directly to an SP Switch port.

Material in Chapter 1 provides a basic description of the SP Switch Router Adapter card. Please refer to the GRF 400/1600 Getting Started manual for SP Switch Router system installation procedures.

Chapter 1 covers these topics:

What is the RS/6000 SP Switch Router ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2

SP Switch Router systems for IBM sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3

Upgrading system memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5

Overview of the SP Switch Router Adapter card. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6

Inserting a media card into the SP Switch Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7

SP Switch Router Adapter card LEDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9

SP Switch Router Adapter card speciﬁcations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-13

Assigning ﬁlters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14

SNMP on the SP Switch Router Adapter card. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-15

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-1

Introduction to the SP Switch Router Adapter card

What is the RS/6000 SP Switch Router ?

The RS/6000 SP Switch Router is a high-performance switched IP router designed for high-volume, large-scale public and private backbone applications.

It has these main features:

• Performs Layer-3 switching across 4-16 adapter slots, depending upon router model

• Supports large suite of dynamic routing protocols

• Accommodates multiple types of media, including HSSI, 10/100Base-T (fast Ethernet),

ATM OC-3c, ATM OC-12c, SONET OC-3c, HIPPI, and FDDI

• Provides basic ﬁltering, OSPF multicast, SNMP v1

• Manages 150K-entry route table, batch updating with 20 routes per second As shown in Figure 1-1, the SP Switch Router attaches to the SP Switch through the SP Switch

Router Adapter media card:

SP control workstation

Administrative network =

Ethernet hub or bridge

SP Switch Router

Switch

• • •

SP Switch Router Adapter media card

control board

Processor

node

• • •

Processor

node

SP Switch

to/from other networks and hosts

Figure 1-1. Connections between the SP Switch Router and an SP system

Primary node for SP Switch

1-2 October 22, 1999

Conﬁgured with an IBM SP system, the SP Switch Router provides multiple media LAN and WAN connectivity for the SP. The SP Switch Router Adapter card connects directly to the SP Switch. Other components communicate across the administrative Ethernet network. (While using a hub or a bridge to interconnect the administrative Ethernet segments is common, other network components can be used to provide connectivity between the segments.)

SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SP Switch Router systems for IBM sites

By using the appropriate SP Switch Router Adapter card, the SP system can connect to FDDI, fast Ethernet, HSSI, ATM OC-3c, ATM OC-12c, SONET OC-3c, and HIPPI networks and hosts, depending upon which other media cards are installed in the SP Switch Router chassis.

The SP Switch Router supports these connection options:

– a single SP Switch Router Adapter card can be installed in an SP Switch Router – multiple SP Switch Router Adapter cards can be installed in an SP Switch Router – more than one card can connect to the same SP system – a single SP Switch Router can connect to one or multiple SP systems

Each option requires unique IP addressing and network conﬁguration. Appendix C contains examples of speciﬁc SP system–SP Switch Router networks. See the IBM documentation related to planning for the SP Switch Router for more information.

SP Switch Router systems for IBM sites

The SP Switch Router Adapter media card communicates directly with the SP. In the SP system conﬁguration, the SP Switch Router Adapter media card is treated as a dependent node and is assigned a node number. There is only one node number address space in the SP system, and traditional SP nodes and dependent nodes are both assigned node numbers from that address space.

You may be given references to models 9076 and 9077:

– Model 9076 is the IBM SP system. – Model 9077 04S is the 4-card SP Switch Router. – Model 9077 16S is the 16-card SP Switch Router.

This manual uses SP system and SP Switch Router, respectively, as system names.

Cables included in your system

SP Switch cable

The SP Switch Router Adapter media card connects to an SP Switch via an SP Switch cable. Make sure the shipping box contains one ten- or twenty-meter cable for each SP Switch Router Adapter card you receive. If there is no cable, contact your IBM representative and order the special cable required by the SP Switch Router Adapter media card from IBM:

• SP Switch cable, 10m (IBM P/N 46H9699)

• SP Switch cable, 20m (IBM P/N 46H9701) Although it has 50-pin connector ends, the speciﬁed cable has custom signal wiring so that

other 50-pin cables cannot be substituted.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-3

Introduction to the SP Switch Router Adapter card

SP Switch Router systems for IBM sites

Ethernet cable

An Ethernet 10/100Base-T cable is required for connecting the SP Switch Router to the SP control workstation. It is the customer's responsibility to provide the appropriate cable to make this connection as well as any Ethernet hubs or bridges that may be required to connect to the SP LAN.

SP ground strap

A ground strap is included with the SP Switch Router and must be connected between the designated site on the SP Switch Router chassis and an SP frame. Refer to “ESD requirements” on page 1-8 for a diagram showing the location of router ground strap connections.

PCMCIA 520MB disk

The SP Switch Router shipping box also contains a PCMCIA 520MB spinning disk device. The PCMCIA disk installs in the SP Switch Router’s control board, in the PCMCIA “A” slot. Once installed, the SP Switch Router can be conﬁgured to log and dump locally to the 520MB external storage device.

By default, logging is turned off when the SP Switch Router boots and comes up. After the system comes up, one of the ﬁrst tasks is to conﬁgure the PCMCIA disk. The conﬁguration procedure formats and mounts the external PCMCIA device, and places the required logging pointers. The installation procedure is described in Chapter 2.

Redundant AC power supplies

The SP Switch Router you receive is installed with redundant AC power supplies. You must plug the power supply cords directly into an AC wall or rack receptacle.

Note: The SP Switch Router has no power on/off switch. When you plug the power supply cord into a live outlet, the SP Switch Router powers on and,

since the software is already loaded, immediately begins to boot.

Redundant supply safety

Please note the following when powering off (unplugging) the SP Switch Router unit:

Caution: This unit has two power supply cords. For total isolation from electrical shock and energy hazard, disconnect both supply cords. Care must be taken to correctly connect each power supply to separate AC power sources and (optional) UPS devices.

Vorsicht: Dieses Gerät hat zwei Netzanschlusskabel. Um das Gerät vollstandig v on Netz zu trennen ziehen Sie beide Kabel ab, sonst können Sie einen elektrischen Schlag erhalten. Achten Sie darauf, daß jedes Stromkabel mit einer separaten Wechselstromquelle und einem separaten USV-Gerät verbunden wird.

Attention: Cet appareil a deux cordons d’alimentation électrique. Pour une isolation complète de tout choc électrique et de danger énergétique, débrancher les deux cordons d’alimentation.

1-4 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Upgrading system memory

Figure 1-2 shows the area of system memory (control board RAM) that can be expanded to meet site requirements. Memory upgrades are made in 128MB increments up to 512MB.

Introduction to the SP Switch Router Adapter card

Upgrading system memory

expandable to -->

- system software

- config files

- GateD binary

- log files

- route tables

- ATMP tunnels

- kernel runs

- GateD runs

256MB

RAM

212MB

= expandable area of RAM

--> 384MB RAM

340MB

--> 512MB RAM

468MB

Memory size and

organization

128MB RAM

32MB

(fixed size)

84MB

8-12MB

(fixed size)

Figure 1-2. Expandable area of system memory

This chart provides general guidelines for memory required in different routing environments. Although the ﬁgures assume BGP peers with 50K route entries, additional memory may be required for higher average numbers of routes per BGP peer.

If the SP Switch Router is to support dynamic routing or ATMP home agents and mobile nodes, upgrade to at least 256MB. In environments where large numbers of routes are advertised, upgrade to 512MB.

Customer proﬁle

Amount of control board memory needed

Space for dynamic routing, ATMP tables

Route entries on media card

Route entries in dynamic routing database

Typical numbe r of peer sessions

Static routing: (in high-performance environment)

Small POP 256MB 212MB 150K Typical

128MB 84MB 150K Typical

number: 35,800

3 number: 199,000

Medium POP / ISP backbone

384MB 340MB 150K Typical

number:

362,000

Large POP / Exchange point / Route reﬂection server

512MB 468MB 150K Typical

number: 521,000

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-5

Introduction to the SP Switch Router Adapter card

Overview of the SP Switch Router Adapter card

The SP Switch Router Adapter media card is cabled to a connector jack on an SP Switch. This media card transfers data to/from the SP Switch at 100 MB/s in each direction.

Like other SP Switch Router media interfaces, the SP Switch Router Adapter media card:

– is intelligent, and performs IP-level routing and route look-ups – provides complete speed-decoupling between the connecting media and the

gigabit/second router switch core

– fully buffers data for input and output queuing; each card has 16MB of high-speed

receive buffer memory and 16MB of high-speed transmit buffers

Face plate diagram

The SP Switch Router Adapter card provides one full-duplex interface.

This illustration shows the faceplate, the interface connector, and the card LEDs.

The actual height of the SP Switch Router Adapter card is 10 inches.

The SP Switch Router Adapter cable 50-pin connector end is also shown.

PWR ON 3V

RX HB RX ST0 RX ST1 RX ERR

MD RCV SW XMIT

TX HB TX ST0 TX ST1 TX ERR

MD XMIT SW RCV

1-6 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

Inserting a media card into the SP Switch Router

Note:

To operate properly, the 16-card SP Switch Router requires that at least two media cards be installed. The 4-card router requires one card be installed. In both models, a face plate cover must be installed in any unused chassis slot to maintain router cooling ﬂows.

Media cards are actually two logic boards joined to make a single component. As shown in Figure 1-3, the smaller board on the right is the serial interface, also called the serial daughter card. The larger one on the left is the media board and has the network ports. Together they comprise an SP Switch Router media card.

Cards install vertically for 16-card SP Switch Router:

Port

Cards install horizontally level for 4-card SP Switch Router:

Media board

Serial/rev number area

Top Bottom

Serial

daughter

card

Figure 1-3. Media card components

The two logic boards are joined by a pair of 100-pin connectors and reinforcing plates. Even so, this joint retains some ﬂex and must be carefully supported, especially when inserting the media card into the chassis.

Warning: The backplane of the SP Switch Router contains hazardous energy levels. When replacing a media card, remove only one card at a time. Removing more than one card will expose the operator to this energy hazard.

Warnung: An den Rückwandplatinen der SP Switch Router liegen gefährliche Hochspannungen ab. Zum Auswechseln der Medienkarte jeweils nur eine Karte entfernen. Bei zwei gleichzeitig entfernten Karten ist der Bediener gefährlichen Spannungen ausgesetzt.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-7

Introduction to the SP Switch Router Adapter card

Inserting a media card into the SP Switch Router

ESD requirements

Caution: Media cards are hot swappable and can be installed when the SP Switch Router is

running. However, media cards are highly susceptible to damage from electrostatic discharge. You must wear a grounded, conductive wrist strap any time you handle a media card. Make sure the metallic elements in the band directly touch your exposed skin.

SP Switch Router - 16 card

Card insertion procedure

1 When you are properly grounded, remove the media card from its anti-static container. 2 Hold the media card with the network ports facing you.

4-card router

Turn the card horizontal, the top of the media card should be on the left, the bottom of the card should be on the right. As you start, make sure you visually identify the left and right guide pair for this particular slot.

Keeping the media card horizontally level, insert the card fully into the slot, you will feel the card joining with the 100-pin connector on the backplane.

SP Switch Router - 4 card

Wrist strap grounding sites

16-card router

Hold the card vertically. As you start, make sure you visually identify the top and bottom guide pair for this particular slot. Have one hand under the card, lightly supporting its weight. Rest just the edge of the bottom corner of the card in the bottom guide. Then, bring the top edge of the card into the top guide. This will help you keep the card level as you slide it in.

Keeping the media card vertically upright, insert the card fully into the slot. You will feel the card joining with the 100-pin connector on the backplane.

3 When fully inserted, the card’s face plate should be ﬂush against the chassis back panel.

Note: Do not force the card into the slot. Doing so can damage the card or slot connector. 4 Tighten the screws at each end of the face plate.

1-8 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card LEDs

The “RX” and “TX” LEDs are under software control and indicate port states on receive and transmit sides. The “MD” and “SW” LEDs are hardware-controlled and reﬂect data activity on the SP Switch Router switch core or interface side of receive and transmit ports.

“Top” end of card “Bottom” end of card

RX HB

RX ST0

RX ST1

PWR ON

Figure 1-4. LEDs on the SP Switch Router Adapter card

LED activity during boot

RX ERR

SW XMIT

MD RCV

During boot and resets, the four software-controlled LEDs indicate different media card activities by ﬂashing in speciﬁc patterns. Refer to Table 1-1 for a description of each pattern.

Table 1-1. SP Switch Router Adapter card LED activity during boot and reset

TX HB

TX ST0

TX ST1

TX ERR

SW RCV

MD XMIT

LED Description

• ON At reset, all LEDs are lit for 1/2 second

• RX HB (green)

• RX ST0 (green)

• RX ST1 (amber)

• RX ERR (amber)

• ON as part of on-board diagnostics.

• ON Also tests that LEDs are working.

• ON

• OFF ERROR - During a boot or reset, this pattern indicates

• OFF a checksum error is detected in

• OFF ﬂash memory.

• ON

• ON ERROR - During a boot or reset, this pattern indicates

• OFF that the SRAM fails the memory test.

• ON

• OFF

• ON -> OFF - During loading, HB and RX ST1 ﬂash

• OFF as each section of the code loads.

• OFF

• ON -> OFF

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-9

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card LEDs

Table 1-1. SP Switch Router Adapter card LED activity during boot and reset (continued)

LED Description

• ON At reset, all LEDs are lit for 1/2 second

• TX HB (green)

• TX ST0 (green)

• TX ST1 (amber)

• TX ERR (amber)

• ON as part of on-board diagnostics.

• ON Also tests that LEDs are working.

• ON

• OFF ERROR - During a boot or reset, this pattern indicates

• OFF a checksum error is detected in

• OFF ﬂash memory.

• ON

• ON ERROR - During a boot or reset, this pattern indicates

• OFF that the SRAM fails the memory test.

• ON

• OFF

• ON -> OFF - During loading, HB and TX ST1 ﬂash

• OFF as each section of the code loads.

• OFF

• ON -> OFF

1-10 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

LED activity during normal operations

Refer to Table 1-2 for a description of SP Switch Router Adapter card LED activity during normal run time operations.

Table 1-2. SP Switch Router Adapter media card LEDs

LED Description

PWR ON This green LED is on when 5 volts are present. Both power LEDs,

5V and 3V, can be on simultaneously. 3V This green LED is on when 3 volts are present. RX HB (green) During normal run-time operations, this green LED blinks a

“heartbeat” pattern for the receive side CPU. The beat is a long off,

short on-off-on pattern .

In the pattern, the LED goes off for 1/2 second, comes on for 1/4

second, goes off for 1/4, comes on for 1/4, and then begins the

pattern again by going off for 1/2 second.

SP Switch Router Adapter card LEDs

• ON STATE_0 - These three LEDs are on during

• RX ST0 (green)

• RX ST1 (amber)

• RX ERR (amber)

These three LEDs light in different combinations to indicate ﬁve operating states for the receive port.

MD RCV (amber) This amber LED lights when data comes into the receive media port

SW XMIT (amber) This amber LED lights when the receive media port sends data to the

• ON hardware initialization.

• ON

• OFF STATE_1 - Bottom two amber LEDs go on during

• ON software initialization, show receive port

• ON is waiting for conﬁguration parameters.

• ON STATE_2 - Middle amber LED goes off when

• OFF conﬁguration parameters are in place and the

• ON receive port is ready to be connected.

• OFF STATE_3 - Bottom amber LED goes on to show that

• OFF the receive port is connected and the card is

• ON ready to be on line

• OFF STATE_4 - These three LEDs are off to show receive

• OFF port is online and running/routing.

• OFF

from an external source.

SP Switch Router switch core (via the serial daughter card). TX HB (green) This green LED blinks a “heartbeat” pattern for the transmit side

CPU, the beat is a long off, short on-off-on pattern during normal run

time operations.

In the pattern, the LED goes off for 1/2 second, comes on for 1/4

second, goes off for 1/4, comes on for 1/4, and then begins the

pattern again by going off for 1/2 second.

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-11

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card LEDs

Table 1-2. SP Switch Router Adapter media card LEDs (continued)

LED Description

• ON STATE_0 - These three LEDs are on during

• TX ST0 (green)

• TX ST1 (amber)

• TX ERR (amber)

These three LEDs light in different combinations to indicate ﬁve operating states for the transmit port.

• ON hardware initialization.

• ON

• OFF STATE_1 - Bottom two LEDs (amber) go on during

• ON software initialization, show transmit port

• ON is waiting for conﬁguration parameters

• ON STATE_2 - Middle LED goes off when

• OFF conﬁguration parameters are in place and the

• ON transmit port is ready to be connected.

• OFF STATE_3 - Bottom LED (amber) goes on to show that

• OFF the transmit port is connected and the card is

• ON ready to be on line.

• OFF STATE_4 - These three LEDs are off to show transmit

• OFF port is online and running/routing.

• OFF

MD XMIT (amber) This amber LED comes on as data leaves the transmit media side

going to an external destination. SW RCV (amber) This amber LED lights when data from the SP Switch Router switch

core (via the serial daughter card) goes to the transmit media side.

Note: The MD RCV, SW XMIT, MD XMIT, and SW RCV LEDs increase in brightness with increasing data trafﬁc. When there is little data trafﬁc, it may be difﬁcult to see that the LED is blinking.

1-12 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

Introduction to the SP Switch Router Adapter card

SP Switch Router Adapter card specifications

SP Switch Router Adapter card speciﬁcations

Refer to Table 1-3 for SP Switch Router Adapter media card characteristics:

Table 1-3. SP Switch Router Adapter media card speciﬁcations

Element Value

Attachment density One full-duplex interface Media transfer rate 100 megabytes per second Processors 40 MHz SPARC, one transmit, one receive Data buffers 16MB input, 16MB output Route table support 150K entries Max transmission unit The default MTU is 65520 bytes Card connector 2-row, 50-pin panel-mount receptacle Cable connector 2-row, 50-pin shielded tab connector Cables Twisted-pair copper, 10- or 20-meter length, available from IBM Power consumption Approximately 50 watts per media card

SP Switch Router Adapter Guide - 1.4 Update 2 October 22, 1999 1-13

Introduction to the SP Switch Router Adapter card

Assigning filters

Assigning ﬁlters

The SP Switch Router Adapter card supports IP packet ﬁltering. You can apply ﬁlters to the receive and/or transmit path of a logical interface as described in the “IP Packet Filtering” chapter of the GRF Conﬁguration and Management manual.

The ﬁlter conﬁguration ﬁle is /etc/filterd.conf. The maint 50 – 58 commands report statistics and information for ﬁlters assigned to the receive side of the card. The maint 150 – 158 commands report on transmit side ﬁlters.

The “IP Packet Filtering” chapter describes the entries in /etc/filterd.conf and tells you how to design several types of ﬁlters. The binding statement in /etc/filterd.conf is where you assign a ﬁlter you have created to a particular logical interface on a speciﬁc media card. This statement has two variables that are media card speciﬁc,

In a binding statement, media is the type of media card and vlif is the logical interface number to which the ﬁlter is assigned. For the SP Switch Router Adapter card, media is always dev1 and the vlif is always 0 since the card has a single interface.

Here is a binding statement for an SP Switch Router Adapter card in slot 5, gt050 (the card is connected to node 8 on an SP switch):

media and vlif.

tcpdump

media dev1 5 { # bind no_host_22_22 { vlif 0; # this is the switch node 8 interface direction out; # outbound trafﬁc to node 8 action filter; } }

the ﬁlter named “no_host_22” blocks all packets from remote host 192.168.22.22

Here are the supported media names:

atm (OC-3c) dev1 ether fddi hssi hippi sonet

(OC-3c)

Please refer to the “IP Packet Filtering” chapter of the GRF Conﬁguration and Management manual for conﬁguration information and examples.

Filtering supports the standard UNIX tcpdump utility that enables you to examine the data crossing an SP Switch Router Adapter interface. A tcpdump “listen” command for interface

gt030 is:

# tcpdump -i gt030

1-14 October 22, 1999 SP Switch Router Adapter Guide - 1.4 Update 2

+ 120 hidden pages