Sun Microsystems CP3240 User Manual

Download

Sun Netra™CP3240 Switch

Installation Guide

Sun Microsystems, Inc. www.sun.com

Part No. 820-3251-13 September 2009, Revision A

Submit comments about this document at: http://www.sun.com/hwdocs/feedback

Sun Microsystems,Inc. hasintellectual property rightsrelating totechnology embodiedin theproduct that is described in this document. In particular,and withoutlimitation, theseintellectual property rights may include one or more ofthe U.S.patents listedat http://www.sun.com/patents and one or more additionalpatents orpending patentapplications inthe U.S.and inother countries.

This distributionmay includematerials developedby thirdparties.

Parts ofthe productmay bederived from BerkeleyBSD systems,licensed fromthe Universityof California.UNIX isa registered trademarkin the U.S.and inother countries,exclusively licensedthrough X/OpenCompany, Ltd.

Sun, SunMicrosystems, the Sun logo, Solaris,Netra andthe Netra logo are trademarksor registered trademarksof SunMicrosystems, Inc., orits subsidiaries, inthe U.S.and othercountries.

Products covered byand informationcontained inthis servicemanual arecontrolled by U.S. Export Control laws and may be subject to the export orimport lawsin othercountries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct orindirect, are strictly prohibited. Exportor reexport to countries subject to U.S. embargo orto entitiesidentiﬁed onU.S. exportexclusion lists, including,but notlimited to,the deniedpersons andspecially designatednationals listsis strictlyprohibited.

Use ofany spareor replacement CPUsis limitedto repairor one-for-one replacementof CPUsin products exportedin compliancewith U.S. export laws.Use ofCPUs asproduct upgradesunless authorizedby theU.S. Governmentis strictlyprohibited.

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Sun Microsystems,Inc. détientles droits depropriété intellectuelsrelatifs à la technologie incorporée dans le produit qui est décrit dans ce document. Enparticulier,et cesans limitation,ces droits de propriété intellectuellepeuvent inclure un ou plus des brevetsaméricains listésà l’adresse http://www.sun.com/patents et un ou les brevets supplémentaires oules applicationsde brevet enattente auxEtats -Unis etdans les autres pays.

Cette distributionpeut comprendre descomposants développéspar destierces parties.

Des partiesde ceproduit pourront êtredérivées dessystèmes BerkeleyBSD licenciéspar l’Universitéde Californie.UNIX estune marque déposée auxEtats-Unis etdans d’autrespays etlicenciée exclusivementpar X/OpenCompany, Ltd.

Sun, SunMicrosystems, lelogo Sun,Solaris, Netraet lelogo Netrasont desmarques de fabrique ou des marques déposées de Sun Microsystems, Inc.,ou sesﬁliales, auxEtats-Unis etdans d’autres pays.

Ce produitest soumisà lalégislation américaineen matière decontrôle desexportations etpeut êtresoumis àla règlementationen vigueur dans d’autrespays dansle domainedes exportationset importations.Les utilisations, ouutilisateurs ﬁnaux,pour desarmes nucléaires, des missiles, desarmes biologiqueset chimiquesou dunucléaire maritime,directement ou indirectement, sont strictement interdites. Les exportations oureexportations versles payssous embargo américain,ou versdes entitésﬁgurant surles listesd’exclusion d’exportation américaines, ycompris, maisde manièrenon exhaustive,la listede personnesqui fontobjet d’unordre de ne pas participer, d’une façon directe ou indirecte,aux exportationsdes produits oudes servicesqui sontrégis parla législationaméricaine enmatière decontrôle desexportations et la listede ressortissantsspéciﬁquement désignés,sont rigoureusement interdites.L’utilisation de pièces détachées ou d’unités centrales de remplacement estlimitée auxréparations ouà l’échangestandard d’unités centrales pour les produits exportés, conformément à la législation américaine enmatière d’exportation.Sauf autorisationpar les autorités des Etats-Unis, l’utilisation d’unités centrales pour procéder à des mises à jourde produitsest rigoureusement interdite.

LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENTEXCLUES, DANSLA MESUREAUTORISEE PARLA LOIAPPLICABLE, YCOMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFACON.

Please

Recycle

Please

Recycle

Preface xix

1. Getting Started 1–1

1.1 System Requirements 1–2

1.1.1 Connectivity 1–2

1.1.1.1 Hub Connectivity 1–2

1.1.2 Electrical and Environmental 1–2

1.2 Unpacking 1–4

1.3 Handling Switches 1–4

1.4 Connectors 1–5

1.5 Jumper Options 1–5

1.6 Switch and RTM Faceplates 1–6

1.6.1 Base 10/100/1000 Uplink Ports (RJ-45) 1–7

1.6.2 10/100 Management Port (RJ-45) 1–7

1.6.3 Serial Management Port (RJ-45) 1–7

1.7 RTM Airflows 1–7

1.7.1 RTM Airflow Requirements 1–7

1.7.2 RTM Pressure Drop Versus Airflows (Impedence Curves) 1–8

1.8 Removing and Installing Switches 1–10

1.8.1 Removing a Switch Set 1–10

1.8.1.1 Removing a Switch From the Front of the Server 1–11

1.8.1.2 Removing a Rear Transition Module for a Switch 1–13

1.8.2 Installing a Switch Set 1–14

1.8.2.1 Installing the Rear Transition Module for a Switch 1–14

1.8.2.2 Installing a Switch 1–14

1.9 Switch LEDs 1–15

1.9.1 ATCA Board Status LEDs 1–15

1.9.2 Hot-Swap LED 1–16

2. Overview 2–1

2.1 Features 2–2

2.1.1 General 2–2

2.1.2 Base Interface 2–3

2.1.3 Fabric Gigabit Interface 2–4

2.1.4 AMC Sites 2–5

2.2 Switch Components 2–6

2.2.1 Broadcom StrataXGS 3 BCM56503 Ethernet Switch 2–6

2.2.2 Broadcom StrataXGS 3 BCM56800 Ethernet Switch 2–6

2.2.3 Broadcom BCM5464R and BCM5461S 10/100/1000Base-T Ethernet PHY 2–6

2.2.4 Freescale PowerQUICC II MPC8247 Communications Processor 2–7

2.2.5 Pigeon Point BMR-H8S-AMCc AdvancedTCA IPMI Subsystem 2–7

2.3 Protocols, RFCs, and MIBs Support 2–8

2.3.1 FASTPATH Switching 2–8

2.3.1.1 Additional Layer 2 Functionality 2–9

2.3.1.2 System Facilities 2–9

2.3.1.3 Switching MIBs 2–10

2.3.1.4 Routing MIBs 2–10

vi Sun Netra CP3240 Switch Installation Guide • September 2009

2.3.2 FASTPATH Routing 2–11

2.3.3 FASTPATH Quality of Service 2–12

2.3.3.1 DiffServ 2–12

2.3.3.2 Access Control Lists (ACLs) 2–12

2.3.3.3 Class of Service (CoS) 2–13

2.3.3.4 Quality of Service MIBs 2–13

2.3.4 FASTPATH Multicast 2–13

2.3.4.1 Multicast MIBs 2–14

2.3.5 FASTPATH IPv6 Routing 2–14

2.3.5.1 IPv6 Routing MIBs 2–15

2.3.6 FASTPATH Management 2–15

2.3.6.1 SSL 3.0 and TLS 1.0 2–16

2.3.6.2 SSH 1.5 and 2.0 2–16

2.3.6.3 Additional Management Features 2–16

2.4 Functional Diagrams and Port Maps 2–17

2.4.1 Switch 2–17

2.4.2 Rear Transition Modules (RTM)s 2–18

2.4.2.1 Copper RTM 2–19

2.4.2.2 Fiber Optic RTM 2–21

2.4.3 Base Fabric Switch Subsystem 2–23

2.4.4 Expansion Fabric Switch Subsystem 2–24

2.4.5 AdvancedMC Sites 2–25

2.4.5.1 AMC Port Maps 2–26

2.4.5.2 AMC Module Support by Site 2–30

2.4.5.3 AMC Port Restrictions for RTMs 2–30

3. Configuring Jumper Settings 3–1

3.1 Jumper Settings 3–2

3.1.1 P4 Cross-Connect Control 3–2

Contents vii

3.1.2 P6(1-2) Fabric Zero Reset Configuration Word 3–3

3.1.3 P6(3-4) Base Zero Reset Configuration Word 3–3

3.1.4 P8(1-2) Base Write Protect 3–3

3.1.5 P8(3-4) Fabric Write Protect 3–4

3.1.6 P9 IPMC Firmware Program 3–4

3.1.7 P10(1-2) IPMC Reset 3–4

3.1.8 P10(3-4) IPMC FWE 3–5

3.1.9 P11(1-2) Forced-Board Enable 3–5

3.1.10 P11(3-4) IPMC Board Reset 3–5

3.1.11 P12(1-2) and P12 (3-4) EMI Ground to Logic Ground 3–6

3.1.12 P13 Serial Direction 3–6

3.2 Jumper Locations 3–7

4. Configuring Switch Software 4–1

4.1 IPMI Firmware Sensors 4–2

4.1.1 State Sensors 4–2

4.1.2 Threshold Sensors 4–4

4.2 uBoot 4–5

4.2.1 uBoot Console 4–5

4.2.2 E-Keying Control in uBoot 4–6

4.2.3 Serial-Baud Rate Control in uBoot 4–7

4.3 Linux 4–7

4.3.1 e-Keying 4–7

4.3.2 e-Keying Bypass 4–8

4.3.3 ATCA LEDs 4–8

4.4 Serial Select 4–8

4.5 Serial Location 4–9

4.5.1 Changing Serial Location from UBoot 4–9

4.5.2 Changing Serial Location from FASTPATH 4–9

viii Sun Netra CP3240 Switch Installation Guide • September 2009

4.6 Boot Sequence 4–10

4.6.1 Boot Utility Menu 4–11

4.6.1.1 Load Code Update Package using TFTP/FTP 4–12

4.6.1.2 Erase Current Configuration 4–12

4.6.1.3 Erase Permanent Storage 4–12

4.6.1.4 Select Boot Method 4–12

4.6.1.5 Start Diagnostic Application 4–13

4.7 Primary and Backup Flash 4–13

4.8 Dual Firmware Images 4–13

4.8.1 Booting the Non-Active Image 4–13

4.8.2 Updating the Non-Active Image 4–14

4.8.3 Fabric 1G/10G Auto-negotiation 4–14

4.9 Network Boot 4–14

4.10 FASTPATH 4–18

4.10.1 Management Options 4–18

4.10.1.1 CLI 4–18

4.10.2 Basic CLI Commands 4–19

4.10.3 Logins and Prompts 4–20

4.10.3.1 CLI Defaults 4–20

4.10.3.2 Web Interface Defaults 4–20

4.10.3.3 SNMP Defaults 4–21

4.10.4 Secure Remote Access 4–21

4.10.4.1 SSH and SSL/TLS Keys 4–21

4.10.4.2 Enabling SSH and SSL 4–21

4.10.5 Default Settings 4–22

4.10.6 Port Ordering 4–22

4.11 Firmware Updates 4–24

4.11.1 Firmware List 4–24

Contents ix

4.11.2 Firmware Upgrades 4–24

4.12 Fiber Optic RTM Configuration 4–25

4.12.1 Module Support 4–25

4.12.2 Backplane Ports Versus RTM Optical Ports 4–25

4.12.3 10G Configuration for RTM Optical Port 4–25

4.12.4 1G Configuration for RTM Optical Port 4–26

A. Environment Specifications A–1

A.1 Electrical and Environmental A–2

A.1.1 Absolute Maximum Ratings A–2

A.1.2 Normal Operating Ranges A–2

A.2 Reliability A–3

A.3 Mechanical A–3

A.3.1 Board Dimensions and Weight A–3

B. Connectors B–1

B.1 Connector Assignments B–2

B.2 Connector Locations (Topside) B–3

B.3 ATCA Zone 2 P20 ZD Connector (J5) B–5

B.4 ATCA Zone 2 P21 ZD Connector (J4) B–6

B.5 ATCA Zone 2 P22 ZD Connector (J3) B–6

B.6 ATCA Zone 2 P23 ZD Connector (J2) B–7

B.7 ATCA Zone 2 P24 ZD Connector (J1) B–8

B.8 Serial RJ-45 Connector (J13 Top) B–8

B.9 Serial Cable B–9

B.10 10/100/1000Base-T RJ-45 Connector (J13 3rd, J13 Bottom) B–10

B.11 10/100Base-TX RJ-45 Connector (J13 2nd) B–10

B.12 ATCA Zone 3 RTM Connector Top (J8) B–11

B.13 ATCA Zone 3 RTM Connector (J7-Middle) B–11

x Sun Netra CP3240 Switch Installation Guide • September 2009

B.14 ATCA Zone 3 RTM Connector (J6-Bottom) B–12

C. Datasheet Reference C–1

D. Agency Certifications D–1

D.1 CE Certification D–1

D.2 NEBS/ETSI D–1

D.3 Safety D–2

D.4 Emissions Test Regulations D–3

D.4.1 EN 50081-1 Emissions D–3

D.4.2 EN 55024 Immunity D–3

D.5 Regulatory Information D–4

D.5.1 D.4.1FCC (USA) D–4

D.5.2 Industry Canada (Canada) D–5

Index Index–1

Contents xi

xii Sun Netra CP3240 Switch Installation Guide • September 2009

Figures

FIGURE 1-1 Switch and RTM Faceplates 1–6

FIGURE 1-2 Netra CP3240H-RTM-CU Airflow (Pa) 1–8

FIGURE 1-3 Netra CP3240H-RTM-CU Airflow (InchesH20) 1–8

FIGURE 1-4 Netra CP3240H-RTM-OP Airflow (Pa) 1–9

FIGURE 1-5 Netra CP3240H-RTM-OP Airflow (InchesH20) 1–9

FIGURE 1-6 Front Cable Management Bracket in Lower Position 1–11

FIGURE 1-7 Injector/Ejector on the Switch (Open Position) 1–12

FIGURE 2-1 Switch Functional Block Diagram 2–17

FIGURE 2-2 Copper RTM Functional Block Diagram 2–19

FIGURE 2-3 Fiber Optic RTM Functional Block Diagram 2–21

FIGURE 2-4 Base Fabric Switch Subsystem 2–23

FIGURE 2-5 Expansion Fabric Switch Subsystem 2–24

FIGURE 2-6 AMC Port Map Diagram 2–26

FIGURE 3-1 Switch Jumper Locations 3–7

FIGURE A-1 PCB Dimensions A–4

FIGURE B-1 Connector Locations (Topside) B–3

FIGURE B-2 ATCA Zone 1 Connector (J9) B–5

xiii

xiv Sun Netra CP3240 Switch Installation Guide • September 2009

Tables

TABLE 1-1 Power Requirements 1–2

TABLE 1-2 Airflow Requirements 1–7

TABLE 1-3 ATCA Board Status LEDs 1–15

TABLE 1-4 Hot-Swap LED States 1–16

TABLE 2-1 Switch External Ports 2–18

TABLE 2-2 Zone 3 RTM Ports 2–18

TABLE 2-3 Copper RTM External Ports 2–20

TABLE 2-4 Copper RTM Port Restrictions 2–20

TABLE 2-5 Fiber Optic External RTM Ports 2–22

TABLE 2-6 Fiber Optic RTM Port Restrictions 2–22

TABLE 2-7 AMC Ethernet Port Availability 2–27

TABLE 2-8 AMC Site 1: AMC.0 Mid-size, AMC.2 Type E1, AMC.2 Type 5 (Optional), LED Module

Support 2–27

TABLE 2-9 AMC Site #2 AMC.0 Mid-Size, AMC.2 Type E1, AMC.2 Type 5, Double module support, LED

Module Support 2–28

TABLE 2-10 AMC Site 3: AMC.0 Mid-size, AMC.2 Type E1, Master Clock Generator Support, LED

Support 2–29

TABLE 2-11 AMC Modules by Site 2–30

TABLE 2-12 AMC Port Restrictions for Copper RTMs 2–30

TABLE 2-13 AMC Port Restrictions for Fiber Optic RTMs 2–30

TABLE 3-1 Switch Configuration Jumper Settings 3–2

TABLE 3-2 P4 Cross-Connect Jumper Settings 3–2

TABLE 3-3 P6 (1-2) Fabric Zero Reset Jumper Settings 3–3

TABLE 3-4 P6 (3-4) Base Zero Reset Jumper Settings 3–3

TABLE 3-5 P8 (1-2) Base Write-Protect Jumper Settings 3–3

TABLE 3-6 P8 (3-4) Fabric Write Protect Jumper Settings 3–4

TABLE 3-7 P9 IPMC Firmware Program Jumper Settings 3–4

TABLE 3-8 P10 (1-2) IPMC Reset Jumper Settings 3–4

TABLE 3-9 P10 (3-4) IPMC FWE Reserved Jumper Settings 3–5

TABLE 3-10 P11 (1-2) Forced-Board Enable Jumper Settings 3–5

TABLE 3-11 P11 (3-4) IPMC Board Reset Jumper Settings 3–5

TABLE 3-12 P12 (1-2) and P12 (3-4) Grounding Jumper Settings 3–6

TABLE 3-13 P13 Serial Direction Jumper Settings 3–6

TABLE 4-1 IPMI State Sensors 4–2

TABLE 4-2 0 4–2

TABLE 4-3 IPMI Sensor Logic 4–3

TABLE 4-4 IPMI Thresold Sensors 4–4

TABLE 4-5 uBoot Console Commands 4–6

TABLE 4-6 Basic CLI Commands 4–19

TABLE 4-7 Port Order List 4–22

TABLE A-1 Absolute Maximum Electrical and Temperature Ratings A–2

TABLE A-2 Normal Operating Electrical and Temperature Ratings A–2

TABLE A-3 Board Dimensions and Weight A–3

TABLE B-1 Connector Assignments B–2

TABLE B-2 ATCA Zone 1 Connector (J9) Pins B–4

TABLE B-3 ATCA Zone 2 P20 ZD Connector (J5) B–5

TABLE B-4 ATCA Zone 2 P21 ZD Connector (J4) B–6

TABLE B-5 ATCA Zone 2 P22 ZD Connector (J3) B–6

TABLE B-6 ATCA Zone 2 P23 ZD Connector (J2) B–7

TABLE B-7 ATCA Zone 2 P24 ZD Connector (J1) B–8

TABLE B-8 Serial RJ-45 Connector (J13-Top) B–8

TABLE B-9 Minimum Serial Cable Pinouts B–9

xvi Sun Netra CP3240 Switch Installation Guide • September 2009

TABLE B-10 10/100/1000Base-T RJ-45 Connector (J13-3rd and Bottom) B–10

TABLE B-11 10/100Base-TX RJ-45 Connector (J13-2nd) B–10

TABLE B-12 ATCA Zone 3 RTM Connector (J8-Top)) B–11

TABLE B-13 ATCA Zone 3 RTM Connector (J7-Middle) B–11

TABLE B-14 ATCA Zone 3 RTM Connector (J6-Bottom) B–12

Tables xvii

xviii Sun Netra CP3240 Switch Installation Guide • September 2009

Preface

The Sun Netra CP3240 Switch Installation Guide describes the installation and configuration of the Sun Netra CP3240 switch. This guide also includes information about software, environment specifications, connectors, and certifications.

Before You Read This Document

Obtain and read the following documents:

■ Sun Netra CP3x40 Switch Safety and Compliance Manual (820-3505)

■ Sun Netra CP3240 Switch Product Notes (820-3260)

xix

How This Document Is Organized

Chapter 1 provides unpacking instructions and initial setup information for the

switch. It provides information and procedures needed to install and make the Sun Netra CP3240 switch operational.

Chapter 2 introduces the key features of the switch. This chapter includes a product

definition, a list of product features, and a functional block diagram with a brief description of each block. This chapter can be used to compare the features of the switch against the needs of a specific application.

Chapter 3 describes the jumper settings on the switch. This chapter details factory

default settings and provides information about tailoring the board to the needs of specific applications.

Chapter 4 describes the software packages running on the switch. This section serves

as a primer for using the software on the switch.

Appendix A contains the electrical, environmental, and mechanical specifications for

the switch.

Appendix B This appendix provides a connector location illustration and connector

pin out tables. A detailed description and pin out for each connector is given.

Appendix C provides links to websites with information about many of the devices

and technologies used in the switch.

Appendix D presents UL, CE, and FCC agency approval and certification

information for the switch.

xx Sun Netra CP3240 Switch Installation Guide • September 2009

Typographic Conventions

Typeface Meaning Examples

AaBbCc123 The names of commands, files,

and directories; on-screen computer output

AaBbCc123 What you type, when contrasted

with on-screen computer output

AaBbCc123 Book titles, new words or terms,

words to be emphasized. Replace command-line variables with real names or values.

Note – Characters display differently depending on browser settings. If characters

do not display correctly, change the character encoding in your browser to Unicode UTF-8.

Edit your.login file.

Use ls -a to list all files.

% You have mail.

Password:

Read Chapter 6 in the User ’s Guide.

These are called class options.

Yo u must be superuser to do this.

To delete a file, type rm filename.

Preface xxi

Getting Started

This chapter provides information and procedures needed to install and make the Sun Netra CP3240 switch operational. This chapter should be read before using the board.

In addition to this chapter, refer to the following safety documentation:

■ Netra CP3X20 Switch Safety and Compliance Manual (820-3505)

■ Important Safety Information for Sun Hardware Systems (816-7190)

Caution – When the system is plugged in, energy hazards are present on the

midplane. Do not reach into the enclosure.

Caution – Static electricity can damage electronic components. Wear a wrist strap

grounded through one of the system’s ESD ground jacks when removing and replacing hot-swappable components.

This chapter contains the following topics:

■ Section 1.1, “System Requirements” on page 1-2

■ Section 1.2, “Unpacking” on page 1-4

■ Section 1.3, “Handling Switches” on page 1-4

■ Section 1.4, “Connectors” on page 1-5

■ Section 1.5, “Jumper Options” on page 1-5

■ Section 1.6, “Switch and RTM Faceplates” on page 1-6

■ Section 1.7, “RTM Airflows” on page 1-7

■ Section 1.8, “Removing and Installing Switches” on page 1-10

■ Section 1.9, “Switch LEDs” on page 1-15

1-1

1.1 System Requirements

The following sections briefly describe the minimum system requirements and the configurable features. Links are provided to other chapters and appendices containing more detailed information.

1.1.1 Connectivity

The switch can work in any AdvancedTCA shelf. It is mainly designed to be used as hub blade, however, it can also operate a node or a full-mesh blade.

1.1.1.1 Hub Connectivity

In AdvancedTCA. the hub board defines the system, so it is important that it be put in the correct slot. In all shelves there are two slots specifically designated for hub boards, logical slots 1 and 2. If the slots are not labeled, these slots are easily identifiable because they have a larger number of Zone 2 connectors than the other slots (the only slots with all five Zone 2 connectors).

Base is always routed in a dual star. This means every node slot has a Base channel routed to each of the hub slots. Independent of how the Fabric is used, a hub board is always needed for Base. If full mesh node boards are used in a full mesh shelf, a hub board is not needed for Fabric.

1.1.2 Electrical and Environmental

The switch has the following power requirements:

TABLE 1-1 Power Requirements

State Power in Watts (W)

Idle, no AMCs 45W

All ports linked with RTM, no AMCs 72W

All ports linked with RTM, heavy traffic, no AMCs 78W

Designed max power with 15W RTM, and 100W shared for all AMC sites

1-2 Sun Netra CP3240 Switch Installation Guide • September 2009

200W

The 48VDC has a tolerance of 0VDC to 75VDC without damage. The switch will operate when 48VDC is 36VDC to 72VDC, inclusive.

Caution – Any input voltage outside the range of 0VDC to 75VDC can damage the

switch.

The switch requires air flow. A minimum of 100 LFM should be kept on the board at all times. If an RTM is being used, active RTM cooling is not required.

Caution – None of the integrated chips junction temperature should exceed 125˚C.

The switch requires air flow to meet this requirement. Testing should be done in your shelf to find the quantity of air flow needed. 100 LFM is the recommended minimum quantity of air flow.

The switch might contain materials that require regulation upon disposal. Please dispose of this product in accordance with local rules and regulations. For disposal or recycling information, please contact your local authorities or the Electronic Industries Alliance at http://www.eiae.org/.

Chapter 1 Getting Started 1-3

1.2 Unpacking

Check the shipping carton for damage. If the shipping carton or contents are damaged, notify the carrier and Sun. Retain the shipping carton and packing material for inspection by the carrier. Obtain authorization before returning any product to Sun. Refer to the Netra CP3240 Switch Getting Started Guide (820-3254) for return instructions.

Caution – This board must be protected from static discharge and physical shock.

Never remove any of the socketed parts except at a static-free workstation. Use the anti-static bag shipped with the product to handle the board. Wear a wrist strap grounded through one of the system's ESD ground jacks when installing or servicing system components.

1.3 Handling Switches

Caution – The system is sensitive to static electricity. To prevent damage to the

assembly, always connect an antistatic wrist strap between you and the system.

Caution – Do not flex the switches; the surface-mounted components can break if

the switch is bent.

To minimize the amount of switch flexing, observe the following precautions:

■ When removing a switch from an electrostatic discharge bag, keep it vertical until

you place the switch on the electrostatic discharge mat.

■ Do not place a switch on a hard surface. Use a cushioned antistatic mat. The

switch connectors and components have very thin pins that bend easily.

■ Be careful of small parts located on the component side of a switch.

■ Do not use an oscilloscope probe on the components. The soldered pins are easily

damaged or shorted by the probe point.

■ Transport a switch in an antistatic bag.

1-4 Sun Netra CP3240 Switch Installation Guide • September 2009

Caution – The heat sinks on a switch can be damaged by incorrect handling. Do not

touch the heat sinks while installing or removing a switch. Hold a switch only by the edges. If a heat sink is loose or broken, obtain a replacement switch.

Caution – The heat sinks on a switch can be damaged by improper packaging.

When storing or shipping a switch, ensure that the heat sinks have sufficient protection.

1.4 Connectors

The switch includes several connectors to interface to application-specific devices. Refer to the Chapter B for complete connector descriptions and pin outs.

1.5 Jumper Options

The switch provides several jumper configuration options for features. Location figures and descriptions are provided in Chapter 3.

Chapter 1 Getting Started 1-5

1.6 Switch and RTM Faceplates

The following shows the Sun Netra CP3240 switch and the copper RTM (XCP3240H-RTM-CU-Z) and the fiber optic RTM (XCP3240H-RTM-OP-Z).

FIGURE 1-1 Switch and RTM Faceplates

Copper RTM Fiber Optic

Switch Faceplate

RTM

1-6 Sun Netra CP3240 Switch Installation Guide • September 2009

1.6.1 Base 10/100/1000 Uplink Ports (RJ-45)

There are two front panel Base 10/100/1000Base-T Ethernet uplink ports on the faceplate. They are ports number 18 and 19 on the Base network.

1.6.2 10/100 Management Port (RJ-45)

There is a 10/100 management Ethernet port on the faceplate. This port can be used to manage the Base and Fabric. This port and the 10/100 management port on the RTM can be used at the same time.

1.6.3 Serial Management Port (RJ-45)

There is a RS-232 serial management port on the faceplate. This port can be used to manage the Base and Fabric. See Section 4.4, “Serial Select” on page 4-8 for information about how to switch between managing the Base and the Fabric.

1.7 RTM Airflows

1.7.1 RTM Airflow Requirements

The following chart provides the RTM airflow requirements. No alarms are triggered at any of the listed temperatures for the given airflow.

TABLE 1-2 Airflow Requirements

Watts m3/min CFM

Power Level

25 0.07 0.07 0.14 0.14 2.5 2.5 5 5

25˚C 28˚C 40˚C 55˚C 25˚C 28˚C 40˚C 55˚C

Chapter 1 Getting Started 1-7

1.7.2 RTM Pressure Drop Versus Airflows (Impedence Curves)

The following graphs plot the impedance curves for the RTMs, per PICMG 3.0 requirements.

FIGURE 1-2 Netra CP3240H-RTM-CU Airflow (Pa)

FIGURE 1-3 Netra CP3240H-RTM-CU Airflow (InchesH

1-8 Sun Netra CP3240 Switch Installation Guide • September 2009

FIGURE 1-4 Netra CP3240H-RTM-OP Airflow (Pa)

FIGURE 1-5 Netra CP3240H-RTM-OP Airflow (InchesH20)

Chapter 1 Getting Started 1-9

1.8 Removing and Installing Switches

This section describes how to remove and install switches.

If you are hot-swapping the rear transition module for a switch, you must remove the switch from the front of the system before removing the rear transition module from the rear of the system. Do not remove the rear transition module without first removing the accompanying switch from the front.

1.8.1 Removing a Switch Set

Following are the instructions for removing a switch and, if necessary, the accompanying rear transition module.

1-10 Sun Netra CP3240 Switch Installation Guide • September 2009

1.8.1.1 Removing a Switch From the Front of the Server

1. Move the front cable management bracket to the lower position (FIGURE 1-6).

FIGURE 1-6 Front Cable Management Bracket in Lower Position

2. Disengage the injector/ejector mechanisms at the top and bottom of the board to notify software that the board is about to be removed. Wait for the Hot-Swap LED to light.

FIGURE 1-7 shows the proper way to remove or insert a hot-swappable board into a

system.

Chapter 1 Getting Started 1-11

FIGURE 1-7 Injector/Ejector on the Switch (Open Position)

injector/ejector mechanism

3. Disconnect all cables connected to the switch.

4. Loosen the two board retention screws that fasten the board to the enclosure.

5. Open the ejectors fully, rotating the handles outward until the board disengages from the midplane.

6. Slide the board evenly out of the enclosure.

7. Determine if you are going to replace the rear transition module.

■ If you are going to replace the rear transition module, go to “Removing a Rear

Transition Module for a Switch” on page 1-13.

■ If you are not going to replace the rear transition module, you must install a

replacement switch or a filler panel to maintain the enclosures shielding and cooling performance. Refer to Section 1.8.2.2, “Installing a Switch” on page 1-14.

1-12 Sun Netra CP3240 Switch Installation Guide • September 2009

Note – As long as the switch is removed from the slot, and the system is running

with only the single remaining switch, you do not have redundancy for that component. Both switches must be installed and running to have redundancy for that component.

1.8.1.2 Removing a Rear Transition Module for a Switch

1. Remove the switch from the front of the server, if you have not already done so.

Refer to “Removing a Switch From the Front of the Server” on page 1-11 for those instructions. Do not remove the rear transition module until you have first removed the switch.

Note – Opening the handle on the RTM shuts down both the RTM and the switch.

2. Disconnect any cables connected to the rear transition module for the switch.

3. Loosen any retention screws that fasten the module to the enclosure.

4. Open the ejectors fully, rotating the handles outward until the board disengages from the midplane.

5. Slide the module evenly out of the enclosure.

6. Install a replacement rear transition module or a filler panel to maintain the enclosure’s shielding and cooling performance.

Refer to Section 1.8.2.1, “Installing the Rear Transition Module for a Switch” on

page 1-14.

Caution – Failure to cover all rear panel slots with rear transition modules or filler

panels can negatively impact the cooling of the system.

Chapter 1 Getting Started 1-13

1.8.2 Installing a Switch Set

If you have a rear-access server, you must install the rear transition module before installing the front switch.

■ If you want to install the rear transition module, go to Section 1.8.2.1, “Installing

the Rear Transition Module for a Switch” on page 1-14.

■ If you want to install a switch, go to Section 1.8.2.2, “Installing a Switch” on

page 1-14.

1.8.2.1 Installing the Rear Transition Module for a Switch

1. Choose an appropriate slot for the rear transition module.

Rear transition modules must be installed inline behind the accompanying front board. For example, if the accompanying front board is installed in slot 7, its rear transition module must be installed at the back of the system in slot 7.

2. Remove the filler panel, if necessary.

3. Prepare the rear transition module by opening its injector/ejector handles.

4. Carefully align the edges of the board with the guides in the slot.

Look into the enclosure to verify correct alignment of the rails in the guides.

Caution – Do not force the module into the slot. If it does not fit properly, check to

ensure that you have the correct matching RTM for the switch.

5. Keeping the board aligned in the guides, slide the board in until the injector/ejector mechanisms engage the retention bar.

6. Simultaneously push in the board and rotate the injector/ejector handles to their closed positions (rotate inward) to seat the midplane connectors.

7. Tighten the board retention screws to anchor the board in the shelf.

8. Connect the cables to the rear transition module.

1.8.2.2 Installing a Switch

1. Locate the switch slots.

The switches can be inserted only in slot 7 or slot 8 in the Netra CT 900 server.

2. Move the front cable management bracket to the lower position (

3. Remove the filler panel, if necessary.

1-14 Sun Netra CP3240 Switch Installation Guide • September 2009

FIGURE 1-6).

4. Prepare the switch by opening its injector/ejector handles (FIGURE 1-7).

5. Carefully align the edges of the board with the guides in the slot.

Look into the enclosure to verify correct alignment of the rails in the guides.

6. Keeping the board aligned in the guides, slide the board in until the injector/ejector mechanisms engage the retention bar.

7. Simultaneously push in the board and rotate the injector/ejector handles to their closed positions (rotate inward) to seat the midplane connectors.

If system power is on, the Hot-Swap LED should light up. The Hot-Swap LED should blink for several seconds, and then go off. If the Hot-Swap LED does not go off after several seconds, push harder on the injector/ejector handles to verify that they are pushed in all the way.

8. Tighten the board retention screws to anchor the board in the shelf.

9. Connect the cables to the switch.

1.9 Switch LEDs

The following sections give the status information for all of the LEDs on the switch.

■ “ATCA Board Status LEDs” on page 1-15

■ “Hot-Swap LED” on page 1-16

1.9.1 ATCA Board Status LEDs

TABLE 1-3 describes the three LEDs defined by ATCA to monitor board status.

TABLE 1-3 ATCA Board Status LEDs

LED Color Normal Operation Description

OOS Red Off Out of service. This LED lights on a critical switch

error, such that the board should be removed.

OK Green On This LED is lit when the switch is booted and

switching

A Amber Off Minor Error/User Defined. This LED can be defined

by the user via software applicatons.

Chapter 1 Getting Started 1-15

Note that both the OOS and MINOR LEDs are lit when the board is powered but not booted. This includes all Hot-Swap states M1 through M3. Refer to “Hot-Swap LED”

on page 1-16.

1.9.2 Hot-Swap LED

This blue LED communicates the Hot-Swap status of the switch. TABLE 1-4 shows the different states of the Hot-Swap LED.

TABLE 1-4 Hot-Swap LED States

Order Visible State State Description

1 Solid M1 FRU Inactive The Intelligent Platform

2 Blinking (from solid) M2 Activation

3 Off M3-M4 Active The IPMI microcontroller has

4 Blinking (from off) M5-M6

Back to 1

Request

Deactivation Request

Management Interface (IPMI) microcontroller is booted, but the payload is not. The bottom latch is not fully closed.

The IPMI microcontroller has requested permission to boot the payload from the shelf management controller.

received permission to boot the payload, and has done so. This should be the state under normal operation.

The IPMI microcontroller has requested permission to shut down the payload. Opening the bottom latch activates this state.

Note – A board should be hot-swapped only when the LED is solid blue.

1-16 Sun Netra CP3240 Switch Installation Guide • September 2009

CHAPTER

Overview

This chapter introduces the key features of the switch and RTMs. This chapter includes a product definition, a list of product features, and functional block diagrams with brief descriptions. This chapter can be used to compare the features of the switch against the needs of a specific application.

This chapter contains the following topics:

■ Section 2.1, “Features” on page 2-2

■ Section 2.2, “Switch Components” on page 2-6

■ Section 2.3, “Protocols, RFCs, and MIBs Support” on page 2-8

■ Section 2.4, “Functional Diagrams and Port Maps” on page 2-17

2-1

2.1 Features

Part of Sun’s ATCA platform, the Sun Netra CP3240 switch complies with PICMG

3.0 R2.0 ECN002 and PCIMG 3.1 Option 1 and Option 9. The Sun Netra CP3240 switch implements two separate switched networks on a single PCB. By separating the Base and Fabric networks, the Sun Netra CP3240 switch provides a separate control plane and data plane. It provides 1Gigabyte Ethernet switching on the 3.0 Base Fabric, and the 3.1 Expansion Fabric provides 1Gigabyte/10Gigabyte Ethernet switching. Both of these networks are fully managed with the robust FASTPATH management suite. Both networks support Layer 2 switching as well as Layer 3 routing.

The Sun Netra CP3240 switch can host three AdvancedMC mid-size modules. A variety of different AMC types can be used, including AMC.1, AMC.2, and AMC.3. Netra CP3240 switch supports connectivity to three AMCs and to RTMs with multiple 10Gigabyte-Ethernet links.

The XCP3240H-RTM-CU-Z (copper) and XCP3240H-RTM-OP-Z (fiber optic) are Sun’s RTMs that are paired with the Sun Netra CP3240 switch.

The following sections briefly outline the features of the Sun Netra CP3240 switch.

2.1.1 General

■ PICMG 3.0 AdvancedTCA form factor

■ PICMG 3.0 R2.0 ECN002 compliant

■ PICMG 3.0 compliant 10/100/1000Base-T Ethernet Base Fabric hub board

■ PICMG 3.1 Option 1 (1000Base-BX) and Option 9 (10GBase-BX4) compliant

Ethernet Expansion Fabric hub board

■ Two separate subsystems providing two separate networks on a single PCB

■ 14-slot shelf supported

■ Manage both switches with a single serial port and Ethernet management port

■ Low power < 100 watts under heavy load (without AMCs)

■ Operates with or without an RTM

■ Changeable RTM to support different technologies (for example, RJ-45 versus SFP,

XFP versus CX4)

■ AdvancedTCA and AdvancedMC IPMI support with Pigeon Point Systems

BMR-H8S-AMCc Board Manager

■ Dual-image IPMI firmware

2-2 Sun Netra CP3240 Switch Installation Guide • September 2009

2.1.2 Base Interface

■ Broadcom BCM56503-based design

■ Full gigabit non-blocking, wire-speed switching/routing

■ Layer 2 switching

■ Layer 3 routing

■ 27 ports:

■ 14 10/100/1000Base-T Ethernet ports for node slots

■ 1 10/100/1000Base-T Ethernet port for redundant switch

■ 2 10/100/1000Base-T Ethernet ports for front panel access

■ 2 10/100/1000Base-T Ethernet ports for rear panel access (with RTM)

■ 3 1000Base-BX Ethernet ports for AMC sites (AMC.2 Type E1 compliant)

■ 1 10/100/1000Base-T Ethernet port primary shelf manager

■ 1 10/100Base-TX Ethernet port for redundant shelf manager (cross-connect)

■ 1 10GBase-BX4 Ethernet port to the redundant hub board via the Update

Channel

■ 2 10GBase-CX4 Ethernet ports for rear-panel access (with RTM)

■ 400MHz Freescale MPC8247 control CPU

■ 256MB PC100 SDRAM

■ 64MB on-board flash

■ Hardware redundant flash for guaranteed flash recovery

■ 10/100BaseTX out-of-band management Ethernet port

■ Front- and rear-access RJ-45 serial management port

■ Full IPv6 support

■ L2 and L3 Multicast support

Chapter 2 Overview 2-3

2.1.3 Fabric Gigabit Interface

■ Broadcom BCM56800-based design

■ Full gigabit non-blocking, wire-speed switching/routing

■ Layer 2 switching

■ Layer 3 routing

■ 20 total ports:

■ 14 10GBase-BX4 Ethernet ports for node slots

■ 1 10GBase-BX4 Ethernet port for redundant switch

■ 1 10GBase-BX4 for AMC site (AMC.2 Type 5)

■ 4 10GBase-CX4 for rear-panel access (with RTM)

■ Auto-negotiation between 1GbE (1000Base-BX 3.1 Option 1) and 10GbE

(10GbEBase-BX4 3.1 Option 9)

■ 400MHz Freescale MPC8247 control CPU

■ 256MB PC100 SDRAM

■ 64MB on board flash

■ Hardware redundant flash for guaranteed flash recovery

■ 10/100BaseTX out-of-band RJ-45 management Ethernet port

■ Front- and rear-access RJ-45 serial management port

■ Full IPv6 support

■ L2 and L3 multicast support

2-4 Sun Netra CP3240 Switch Installation Guide • September 2009

2.1.4 AMC Sites

■ Site 1 1000Base-BX to 3.0 network (AMC port 0) is ethernet port 22

■ Site 2 1000Base-BX to 3.0 network (AMC port 0) is ethernet port 23

■ Site 2 10GBase-BX to 3.1 network (AMC port 8-11) is ethernet port 16

■ Site 3 1000Base-BX to 3.0 network (AMC port 0) is ethernet port 24

Site 1: AMC.0 Mid-size, AMC.2 Type E1

■ 1000Base-BX to 3.0 network (AMC port 0)

■ AMC ports 4-7 connected to AMC site 2

■ SAS/SATA ports routed between AMCs (AMC port 2-3)

Site 2: AMC.0 Mid-size, AMC.2 Type 5 and E1

■ 10GBase-BX4 to 3.1 network (AMC port 8-11)

■ 1000Base-BX to 3.0 network (AMC port 0)

■ AMC ports 4-7 connected to AMC site 1

■ SAS/SATA ports routed between AMCs (AMC port 2-3)

Site 3: AMC.0 Mid-size, AMC.2 Type E1

■ 1 1000Base-BX to 3.0 network (AMC port 0)

■ SAS/SATA ports routed between AMCs (AMC port 2-3)

■ Update channel support for redundant master clock generators (AMC port 15)

■ PCI-E Fabric clock support

Chapter 2 Overview 2-5

2.2 Switch Components

The following sections list and describe key components of the Sun Netra CP3240 switch.

2.2.1 Broadcom StrataXGS 3 BCM56503 Ethernet Switch

The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56503 for Base Ethernet switching/routing. This chip is an Ethernet switch with 24 1Gigabyte ports and 3 10Gigabyte ports. It provides non-blocking, wire-speed switching and routing on all ports under 100% load. The BCM56503 features many advanced features presented to the user via the FASTPATH software. See the Section 4.10, “FASTPATH”

on page 4-18 for more information.

2.2.2 Broadcom StrataXGS 3 BCM56800 Ethernet Switch

The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56800 for Fabric Ethernet switching/routing. This chip is an Ethernet switch with 20 10Gigabyte ports. It provides non-blocking, wire-speed switching and routing on all ports under 100% load. The BCM56800 features many advanced features presented to the user via the FASTPATH software. See Section 4.10, “FASTPATH” on page 4-18 for more information.

2.2.3 Broadcom BCM5464R and BCM5461S 10/100/1000Base-T Ethernet PHY

Quad and single ports respectively, the Broadcom PHYs provide the physical interfacing for 10/100/1000Base-T. They are low-power devices and provide features such as jumbo frames support, auto-MDIX, and cable testing.

2-6 Sun Netra CP3240 Switch Installation Guide • September 2009

2.2.4 Freescale PowerQUICC II MPC8247 Communications Processor

The Freescale MPC8247 is a microprocessor designed for maximum flexibility. It features a dual core architecture with a PPC G2 LE core and a RISC core controlling the peripherals. Running at 400MHz with only 1 watt of power, the MPC8247 provides high performance with incredibly low power consumption. Paired with the 256MB PC100 SDRAM and 64MB flash, the CPU subsystem of the Sun Netra CP3240 switch is more than enough to handle the switching application and future upgrades.

2.2.5 Pigeon Point BMR-H8S-AMCc AdvancedTCA IPMI Subsystem

The Sun Netra CP3240 switch uses the near-industry standard Pigeon Point Systems BMR-H8S-AMCc for dedicated IPMI management. It controls all interaction between the Sun Netra CP3240 switch and the shelf management controller. It fully supports the PICMG 3.0 R2.0 ECN002 and the AMC.0 R2.0 specifications. The BMC-H8S-AMCc provides all required management for the AMC modules, in addition to the voltage monitoring, temperature monitoring, e-Keying, and other services it provides for the Sun Netra CP3240 switch. Pigeon Point Systems is the leading provider of IPMI firmware for AdvancedTCA, and the firmware has been thoroughly tested to be fully compliant with the specification.

Chapter 2 Overview 2-7

2.3 Protocols, RFCs, and MIBs Support

The Sun Netra CP3240 switch features Level7 FastPath 2340 switching software, version 4.4.4 and newer. This software provides layer 2 switching, quality of service, IPv4 routing, IPv6 routing, and IP multicast.

2.3.1 FASTPATH Switching

■ IEEE 802.1ab—Link Level Discovery

■ IEEE 802.1D—Spanning Tree

■ IEEE 802.1p—Ethernet Priority with User Provisioning and Mapping

■ IEEE 802.1Q—Virtual LANs with Port-based VLANs

■ IEEE 802.1S—Multiple Spanning Tree

■ IEEE 802.1v—Protocol-based VLANs

■ IEEE 802.1W—Rapid Spanning Tree

■ IEEE 802.1X—Port-based Authentication

■ IEEE 802.3—10 Base-T

■ IEEE 802.3ab—1000 Base-T

■ IEEE 802.3ac—VLAN Tagging

■ IEEE 802.3ad—Link Aggregation

■ IEEE 802.3ae—10 Gigabyte

■ IEEE 802.3u—100 Base-T

■ GARP—Generic Attribute Registration Protocol: Clause 12, 802.1D-2004

■ GMRP—Dynamic L2 Multicast Registration: Clause 10, 802.1D-2004

■ GVRP—Dynamic VLAN Registration: Clause 11.2, 802.1Q-2003

■ IEEE 802.3x—Flow Control

■ draft-ietf-magma-snoop-10.txt—Considerations for IGMP and MLD Snooping

Switches

2-8 Sun Netra CP3240 Switch Installation Guide • September 2009

2.3.1.1 Additional Layer 2 Functionality

■ Broadcast/Multicast/Unknown Unicast storm recovery

■ Double VLAN/vMAN tagging

■ DHCP filtering

■ Jumbo Ethernet frames

■ Many-to-one port mirroring (Tx, Rx, both)

■ Static MAC filtering

■ MAC-based VLANs

■ IP subnet-based VLANs

■ Port description

■ Protected ports

■ Network and host DoS attack suppression

2.3.1.2 System Facilities

■ Event and Error Logging Facility

■ Run-time and configuration download capability

■ PING utility

■ XMODEM

■ RFC 768—UDP

■ RFC 783—TFTP

■ RFC 791—IP

■ RFC 792—ICMP

■ RFC 793—TCP

■ RFC 951—BootP

■ RFC 1321—Message Digest Algorithm

■ RFC 1534—Interoperation between BootP and DHCP

■ RFC 2030—Simple Network Time Protocol (SNTP) V4 for IPv4, IPv6, and OSI

■ RFC 2131—DHCP client/server

■ RFC 2132—DHCP options and BootP vendor extensions

■ RFC 2865—RADIUS client

■ RFC 2866—RADIUS accounting

■ RFC 2868—RADIUS attributes for Tunnel protocol support

■ RFC 2869—RADIUS extensions

■ rfc2869bis—RADIUS support for Extensible Authentication Protocol (EAP)

Chapter 2 Overview 2-9

■ RFC 3164—BSD Syslog protocol

■ RFC 3396—Encoding long options in the Dynamic Host Configuration Protocol

(DHCPv4)

■ RFC 3580—802.1X RADIUS usage guidelines

2.3.1.3 Switching MIBs

■ RFC 1213 — MIB-II

■ RFC 1493 — Bridge MIB

■ RFC 1643 — Ethernet-like MIB

■ RFC 2233 — The Interfaces Group MIB using SMI v2

■ RFC 2618 — RADIUS authentication client MIB

■ RFC 2620 — RADIUS accounting MIB

■ RFC 2674 — VLAN MIB

■ RFC 2737—Entity MIB version 2

■ RFC 2819—RMON groups 1, 2, 3 and 9

■ RFC 3291—Textual conventions for Internet Network Addresses

■ RFC 3635—Definitions of managed objects for the Ethernet-like interface types

■ IEEE 802.1X MIB (IEEE802.1-PAE-MIB)

■ IEEE 802.3AD MIB (IEEE802.3-AD-MIB)

■ FASTPATH Enterprise MIBs supporting switching features

2.3.1.4 Routing MIBs

■ RFC 1724—RIP v2 MIB extension

■ RFC 1850—OSPF MIB

■ RFC 2096—IP forwarding table MIB

■ RFC 2787—VRRP MIB

■ RFC 2863—The Interfaces Group MIB

■ FASTPATH Enterprise MIBs supporting routing features

2-10 Sun Netra CP3240 Switch Installation Guide • September 2009

2.3.2 FASTPATH Routing

■ RFC 826—Ethernet ARP

■ RFC 894—Transmission of IP Datagrams over Ethernet networks

■ RFC 896—Congestion control in IP/TCP networks

■ RFC 919—IP Broadcast

■ RFC 922—IP Broadcast in the presence of subnets

■ RFC 950—IP subnetting

■ RFC 1027—Using ARP to implement Transparent Subnet Gateways (proxy ARP)

■ RFC 1058—RIP v1

■ RFC 1256—ICMP router discovery messages

■ RFC 1321—Message Digest Algorithm

■ RFC 1765—OSPF database overflow

■ RFC 1812—Requirements for IP version 4 routers

■ RFC 2082—RIP-2 MD5 authentication

■ RFC 2131—DHCP relay

■ RFC 2328—OSPFv2 (FASTPATH supports Broadcast interfaces.)

■ RFC 2453—RIP v2

■ RFC 3046—DHCP/BootP relay

■ RFC 3101—OSPF “Not So Stubby Area” (NSSA) option

■ RFC 3768—Virtual Router Redundancy Protocol

■ Route redistribution across RIP and OSPF

■ VLAN routing

■ Weighted static routes

Chapter 2 Overview 2-11

2.3.3 FASTPATH Quality of Service

2.3.3.1 DiffServ

■ RFC 2474—Definition of the Differentiated Services field (DS field) in the IPv4

and IPv6 headers

■ RFC 2475—Architecture for differentiated services

■ RFC 2597—Assured Forwarding PHB Group

■ RFC 3246—Expedited forwarding PHB (Per-Hop Behavior)

■ RFC 3260—New terminology and clarifications for DiffServ

Optional policy attributes:

■ Assign matching traffic flow to a specific queue

■ Redirect or mirror (flow-based mirroring) matching traffic flow to a specific port

2.3.3.2 Access Control Lists (ACLs)

Permit/Deny actions for inbound or outbound IP traffic classification based on:

■ Type of Service (TOS) or Differentiated Services (DS) DSCP field

■ Source IP address

■ Destination IP address

■ TCP/UDP source port

■ TCP/UDP destination port

■ IP protocol number

Permit/Deny actions for inbound or outbound Layer 2 traffic classification based on:

■ Source MAC address

■ Destination MAC address

■ Ethertype

■ 802.1p user priority (outer and/or inner VLAN tag)

■ VLAN identifier value or range (outer and/or inner VLAN tag)

Optional rule attributes:

■ Assign matching traffic flow to a specific queue

■ Redirect or mirror (flow-based mirroring) matching traffic flow to a specific port

■ Generate trap log entries (ACL logging) containing rule hit counts

2-12 Sun Netra CP3240 Switch Installation Guide • September 2009

2.3.3.3 Class of Service (CoS)

Direct user configuration of the following:

■ IP DSCP to traffic-class mapping

■ Interface trust mode: 802.1p, IP DSCP, or untrusted

■ Interface traffic-shaping rate

■ Minimum and maximum bandwidth per queue

■ Strict priority versus weighted (WRR/WFQ) scheduling per queue

■ Tail drop versus weighted random early detection (WRED) queue depth

management

2.3.3.4 Quality of Service MIBs

■ RFC 3289—MIB for the Differentiated Services Architecture (read-only)

■ Private MIBs for full configuration of DiffServ, ACL, and CoS functionality

2.3.4 FASTPATH Multicast

■ RFC 1112—Host Extensions for IP multicasting

■ RFC 2236—IGMPv2

■ RFC 2362—PIM-SM

■ RFC 2365—Administratively scoped boundaries

■ RFC 3376—IGMPv3

■ IP multicast traceroute

■ Draft-ietf-pim-v2-dm-03—PIM-DM

■ Draft-ietf-idmr-dvmrp-v3-10—DVMRP

■ Static RP configuration

■ Draft-holbrook-idmr-igmpv3-ssm-08.txt—IGMPv3/MLDv2 for SSM

■ Draft ietf-magma-igmp-proxy-06.txt—IGMP/MLD-based multicast forwarding

(IGMP/MLD Pproxying)

■ Draft ietf-smm-arch-06.txt—Source specific multicast for IP

■ Draft ietf-magma-igmpv3-and-routing-05.txt—IGMPv3 and Multicast Routing

Protocol interaction

Chapter 2 Overview 2-13

2.3.4.1 Multicast MIBs

■ RFC 2932—IPv4 multicast routing MIB

■ RFC 2933—IGMP MIB

■ RFC 2934—PIM MIB for IPv4

■ Draft-ietf-magma-mgmd-mib-03.txt—Multicast Group Membership Discovery

MIB

■ Draft-ietf-idmr-dvmrp-mib-11.txt—DVMRP MIB

■ IANA-RTP-PROTO-MIB

■ FASTPATH Enterprise MIBs supporting multicast features

2.3.5 FASTPATH IPv6 Routing

■ RFC 1981—Path MTU for IPv6

■ RFC 2373—IPv6 addressing

■ RFC 2460—IPv6 protocol specification

■ RFC 2461—Neighbor discovery

■ RFC 2462—Stateless autoconfiguration

■ RFC 2463—ICMPv6

■ RFC 2464—IPv6 over Ethernet

■ RFC 2711—IPv6 router alert

■ RFC 2740—OSPFv3 (FASTPATH supports Broadcast and Point-to-Point

interfaces.)

■ RFC 2893—Transition mechanisms for IPv6 hosts and routers (6over4 configured)

■ RFC 3315—DHCPv6 (stateless + relay)

■ RFC 3484—Default address selection for IPv6

■ RFC 3493—Basic Socket Interface for IPv6

■ RFC 3513—Addressing architecture for IPv6

■ RFC 3542—Advanced Sockets API for IPv6

■ RFC 3587—IPv6 Global Unicast Address format

■ RFC 3736—Stateless DHCPv6

■ Dual IPv4/IPv6 TCP/IP stack

2-14 Sun Netra CP3240 Switch Installation Guide • September 2009

2.3.5.1 IPv6 Routing MIBs

■ RFC 2465—IPv6 MIB

■ RFC 2466—ICMPv6 MIB

■ FASTPATH Enterprise MIBs supporting IPv6 features

2.3.6 FASTPATH Management

■ RFC 854—Telnet

■ RFC 855—Telnet option specifications

■ RFC 1155—SMI v1

■ RFC 1157—SNMP

■ RFC 1212—Concise MIB definitions

■ RFC 1867—HTML/2.0 forms with file upload extensions

■ RFC 1901—Community based SNMP v2

■ RFC 1908—Coexistence between SNMP v1 and SNMP v2

■ RFC 2068—HTTP/1.1 protocol as updated by draft-ietf-http-v11-spec-rev-03

■ RFC 2271—SNMP framework MIB

■ RFC 2295—Transparent content negotiation

■ RFC 2296—Remote variant selection; RSVA/1.0 state management cookies

draft-ietf-http-state-mgmt-05

■ RFC 2570—Introduction to SNMPv3

■ RFC 2576—Coexistence between SNMP v1, v2 and v3

■ RFC 2578—SMI v2

■ RFC 2579—Textual conventions for SMI v2

■ RFC 2580—Conformance statements for SMI v2

■ RFC 3410—Introduction and applicability statements for Internet-Standard

Management Framework

■ RFC 3411—Architecture for describing SNMP management frameworks

■ RFC 3412—Message processing and dispatching for SNMP

■ RFC 3413—SNMP applications

■ RFC 3414—User-based security model for SNMP v3

■ RFC 3415—View-based access control model for SNMP

■ RFC 3416—Version 2 of the protocol operations for SNMP

Chapter 2 Overview 2-15

■ RFC 3417—Transport mappings for SNMP

■ RFC 3418—MIB for SNMP

■ Configurable management VLAN

2.3.6.1 SSL 3.0 and TLS 1.0

■ RFC 2246—TLS protocol, version 1.0

■ RFC 2346—AES Ciphersuites for transport layer security

■ RFC 2818—HTTP over TLS

2.3.6.2 SSH 1.5 and 2.0

■ Draft-ietf-secsh-transport-16—SSH Transport Layer Protocol

■ Draft-ietf-secsh-userauth-17—SSH Authentication Protocol

■ Draft-ietf-secsh-connect-17—SSH Connection Protocol

■ Draft-ietf-secsh-architecture-14—SSH protocol architecture

■ Draft-ietf-secsh-publickeyfile-03—SECSH Public Key File format

■ Draft-ietf-secsh-dh-group-exchange-04—Diffie-Hellman Group Exchange for the

SSH Transport Layer Protocol

■ HTML 4.0 Specification—December, 1997

■ Java and Java Script 1.3

2.3.6.3 Additional Management Features

■ Industry Standard CLI with the following features:

■ Scripting capability

■ Command completion

■ Context sensitive help

■ User password encryption

■ Multi-session Telnet server

■ TACACS+

■ Dual firmware image support

2-16 Sun Netra CP3240 Switch Installation Guide • September 2009

2.4 Functional Diagrams and Port Maps

2.4.1 Switch

FIGURE 2-1 Switch Functional Block Diagram

Chapter 2 Overview 2-17

TABLE 2-1 Switch External Ports

Front Panel Port Type Fiber Optic AMC Module Logical Port Comments

RJ-45 #1 Serial RS-232 RS-232 RJ-45

RJ-45 #2 Management 10/100 Base T RJ-45

RJ-45 #3 Base 1GBE 10/100/1000Base T RJ-45 Base - 18 Standard I/F is copper

RJ-45 #4 Base 1GBE 10/100/1000Base T RJ-45 Base - 19 Standard I/F is copper

2.4.2 Rear Transition Modules (RTM)s

The Sun Netra CP3240 switch supports RTMs through ATCA Zone 3 connectors (see

TABLE 2-2). The Sun Netra CP3240 switch was designed to support multiple RTM

designs.

4 Fabric 10Gigabyte, 2 Base 10Gigabyte, 2 Base 1Gigabyte, AMC I/O, and management ports are run to the RTM. Several voltage rails are supplied, and all of the signals needed to design an ECN002 compliant RTM are present.

TABLE 2-2 Zone 3 RTM Ports

Base Ports 1G Infrastructure 10G Infrastructure

Base Port 20 1G SGMII None

Base Port 21 1G SGMII None

Base Port 26 None 10G XAUI

Base Port 27 None 10G XAUI

Fabirc Port 17 1G SGMII 10G XAUI

Fabirc Port 18 1G SGMII 10G XAUI

Fabirc Port 19 1G SGMII 10G XAUI

Fabirc Port 20 1G SGMII 10G XAUI

The Sun Netra CP3240 switch is paired with either a copper or fiber optic RTM. The following sections describe each RTM.

2-18 Sun Netra CP3240 Switch Installation Guide • September 2009

2.4.2.1 Copper RTM

The XCP3240H-RTM-CU-Z is the Sun Netra CP3240 switch’s copper RTM pairing. This RTM supports all the ports the Sun Netra CP3240 switch provides using 10GBase-CX4 for the 10Gigabyte and 10/100/1000Base-T for the 1Gigabyte ports.

FIGURE 2-2 Copper RTM Functional Block Diagram

Chapter 2 Overview 2-19

TABLE 2-3 Copper RTM External Ports

Port Type RTM Module Logical Port Comments

Base 10GBE XAUI CX4 Base - 27

Base 10GBE XAUI CX4 Base - 26

Base 1GBE 10/100/1000Base T RJ-45 Base - 20 Standard I/F is copper

Base 1GBE 10/100/1000Base T RJ-45 Base - 21 Standard I/F is copper

Management 10/100 Base T RJ-45

Serial RS-232 RS-232 RJ-45

Fabric 10GBE XAUI CX4 Fabric - 17

Fabric 10GBE XAUI CX4 Fabric - 18

Fabric 10GBE XAUI CX4 Fabric - 19

Fabric 10GBE XAUI CX4 Fabric - 20

TABLE 2-4 Copper RTM Port Restrictions

Port 1G Infrastructure 1G Port 10G Infrastructure 10G Port

Base Port 20 SERDES 1000Base-T RJ45 None None

Base Port 21 SERDES 1000Base-T RJ45 None None

Base Port 26 None None XAUI 10GBase-CX4

Base Port 27 None None XAUI 10GBase-CX4

Fabirc Port 17 None None XAUI 10GBase-CX4

Fabirc Port 18 None None XAUI 10GBase-CX4

Fabirc Port 19 None None XAUI 10GBase-CX4

Fabirc Port 20 None None XAUI 10GBase-CX4

2-20 Sun Netra CP3240 Switch Installation Guide • September 2009

2.4.2.2 Fiber Optic RTM

The XCP3240H-RTM-OP-Z is the Sun Netra CP3240 switch’s fiber optic RTM pairing. This RTM supports all the ports the Netra CP3240 switch provides using 10GBase-X for the 10Gigabyte and 1000Base-X for the 1Gigabyte ports.

FIGURE 2-3 Fiber Optic RTM Functional Block Diagram

Chapter 2 Overview 2-21

TABLE 2-5 Fiber Optic External RTM Ports

Port Type Fiber Optic RTM Module Logical Port Comments

Management 10/100 Base T RJ-45 Not a module, 10/100Base T

Base 10GBE 10GBase-X SFP+ Base - 27 Cannot Support 1G Operation

Base 10GBE 10GBase-X SFP+ Base - 26 Cannot Support 1G Operation

Base 1GBE 1000Base-X SFP Base - 20 Can Support 1G SFP Copper

Base 1GBE 1000Base-X SFP Base - 21 Can Support 1G SFP Copper

Serial RS-232 10GBase-X SFP+ Fabric - 17 Can Support 1G SFP Copper

Fabric 10GBE 10GBase-X SFP+ Fabric - 18 Can Support 1G SFP Copper

Fabric 10GBE 10GBase-X SFP+ Fabric - 19 Can Support 1G SFP Copper

Fabric 10GBE 10GBase-X SFP+ Fabric - 20 Can Support 1G SFP Copper

Serial RS-232 Serial RS232 RJ-45 Not a module, RS232 Serial

Note – For enabling 1G operation of Fabric ports 17-20, see Section 4.12, “Fiber

Optic RTM Configuration” on page 4-25.

TABLE 2-6 Fiber Optic RTM Port Restrictions

Port 1G Infrastructure 1G Port 10G Infrastructure 10G Port

Base Port 20 SERDES SFP LX SX None None

Base Port 21 SERDES SFP LX SX None None

Base Port 26 None None XAUI SFP+ LX SX LRM

Base Port 27 None None XAUI SFP+ LX SX LRM

Fabric Port 17 SERDES SFP LX SX XAUI SFP+ LX SX LRM

Fabric Port 18 SERDES SFP LX SX XAUI SFP+ LX SX LRM

Fabric Port 19 SERDES SFP LX SX XAUI SFP+ LX SX LRM

Fabric Port 20 SERDES SFP LX SX XAUI SFP+ LX SX LRM

2-22 Sun Netra CP3240 Switch Installation Guide • September 2009

2.4.3 Base Fabric Switch Subsystem

FIGURE 2-4 Base Fabric Switch Subsystem

PICMG 3.0 AdvancedTCA defines 10/100/1000Base-T Ethernet over the Base Fabric, called “Base” in this guide. The Base is designed to be the control plane for the shelf. Scaling from 10Mbps to 1000Mbps, the Base interface can accommodate a wide variety of node boards. The Base interface on the Sun Netra CP3240 switch is based around three main components: the Broadcom BCM56503, the Broadcom BCM5464x, and the Freescale MPC8247.

Chapter 2 Overview 2-23

2.4.4 Expansion Fabric Switch Subsystem

FIGURE 2-5 Expansion Fabric Switch Subsystem

PICMG 3.0 AdvancedTCA provides an agnostic mesh on the backplane called the Expansion Fabric. This interface is the data plane in the shelf. The Expansion Fabric is called “Fabric” in this guide. This fabric can be several different technologies defined by AdvancedTCA sub-specifications. The Sun Netra CP3240 switch is designed to comply with PICMG 3.1 Ethernet/Fibre Channel for AdvancedTCA Systems, options 1 and option 9. That means the Sun Netra CP3240 switch provides a single 1/10Gigabyte port to each node board.

The Fabric uses 10GBase-BX4 Ethernet to provide connectivity between boards though the backplane. The Fabric can scale down to 1Gigabyte and work with 1000Base-BX boards. The Fabric subsystem is based around two main components: the Broadcom BCM56800 and the Freescale MPC8247.

2-24 Sun Netra CP3240 Switch Installation Guide • September 2009

2.4.5 AdvancedMC Sites

PICMG AdvancedMC AMC.0 defines hot-swappable daughter cards to be used in PICMG architectures. The Sun Netra CP3240 switch supports three AMC sites so that the functionality of the Sun Netra CP3240 switch can be expanded. AMC.2 (Ethernet) AMCs are supported with direct connections to the Base and Fabric subsystems. Other types of AMCs, such as AMC.1 (PCI-Express) or AMC.3 (SAS/SATA), can be used as well. Nearly any type of AMC can be supported, because the Sun Netra CP3240 switch connects the AMCs sites directly together on certain ports.

Chapter 2 Overview 2-25

2.4.5.1 AMC Port Maps

FIGURE 2-6 AMC Port Map Diagram

2-26 Sun Netra CP3240 Switch Installation Guide • September 2009

TABLE 2-7 AMC Ethernet Port Availability

Slot Location 1G Infrastructure 10G Infrastructure

AMC 1 Top 1G Base SGMII None

AMC 2 Middle 1G Base SGMII XAUI from BCM56800 Fabric Port 16

AMC 3 Bottom 1G Base SGMII None

TABLE 2-8 AMC Site 1: AMC.0 Mid-size, AMC.2 Type E1, AMC.2 Type 5 (Optional),

LED Module Support

Port Type Connected

0 GB Base (3.0) switch

1 Any Not connected

2 Any Port 3 of AMC #3, usually used for SAS/SATA

3 Any Port 3 of AMC #2, usually used for SAS/SATA

4 Any Ports 4-7 of AMC #2, usually used for PCIE

8 Not connected

12 Custom LED board support

13 Not connected

14 Not connected

15 Not connected

17 Any RTM for future use

18 Any RTM for future use

19 Any RTM for future use

20 Any RTM for future use

TCLK1 CLK IN FPGA and Zarlink

Chapter 2 Overview 2-27

TABLE 2-8 AMC Site 1: AMC.0 Mid-size, AMC.2 Type E1, AMC.2 Type 5 (Optional),

LED Module Support (Continued)

TCLK2 CLK OUT FPGA and Zarlink

TCLK3 CLK IN FPGA and Zarlink

TCLK4 CLK OUT FPGA and Zarlink

FCLK 100MHz 100MHz

TABLE 2-9 AMC Site #2 AMC.0 Mid-Size, AMC.2 Type E1, AMC.2 Type 5, Double

module support, LED Module Support

Port Type Connected

0 GbE Base (3.0) switch

1 Any Port 1 of AMC #3, usually used for GbE

2 Any Port 2 of AMC #3, usually used for SAS/SATA

3 Any Port 3 of AMC #1, usually used for SAS/SATA

4 Any Ports 4-7 of AMC #1, usually used for PCIE

8 10GbE Fabric (3.1) switch.

12 custom LED board support

13 Not connected

14 Not connected

15 Not connected

17 Any RTM for future use

18 Any RTM for future use

19 Any RTM for future use

20 Any RTM for future use

TCLK1 CLK IN FPGA & Zarlink

TCLK2 CLK OUT FPGA & Zarlink

TCLK3 CLK IN FPGA & Zarlink

TCLK4 CLK OUT FPGA & Zarlink

FCLK PCIE CLK 100MHz

2-28 Sun Netra CP3240 Switch Installation Guide • September 2009

TABLE 2-10 AMC Site 3: AMC.0 Mid-size, AMC.2 Type E1, Master Clock Generator

Support, LED Support

Port Type Connected

0 GB Base (3.0) switch

1 Any Port 1 of AMC #2, usually used for GB

2 Any Port 2 of AMC #2, usually used for SAS/SATA

3 Any Port 2 of AMC #1, usually used for SAS/SATA

4 Not connected

5 Not connected

6 Not connected

7 Not connected

8 Not connected

9 Not connected

10 Not connected

11 Not connected

12 custom LED board support

13 Not connected

14 Not connected

15 Any Update channel, usually used for clock sync

17 Any RTM for future use

18 Any RTM for future use

19 Any RTM for future use

20 Any RTM for future use

TCLK1 CLK IN FPGA and Zarlink

TCLK2 CLK OUT FPGA and Zarlink

TCLK3 CLK IN FPGA and Zarlink

TCLK4 CLK OUT FPGA and Zarlink

FCLK 100MHz 100MHz

Chapter 2 Overview 2-29

2.4.5.2 AMC Module Support by Site

TABLE 2-11 AMC Modules by Site

AMC Module

Site 1 (top most when the board is vertical) Site 2 (middle)

Site 3 (bottom most site when the board is ver tical)

AMC10G-XFP No* Yes No

AMC10G-CX4 No* Yes No

AMCLED001 Yes Yes Yes

PCI-Express AMC Yes Yes No

1G Common Options

Ye s Ye s Ye s

Ethernet AMC

10G Fat Pipe Ethernet AMC No* Yes No

2.4.5.3 AMC Port Restrictions for RTMs

The following tables list AMC port restrictions for RTMs

TABLE 2-12 AMC Port Restrictions for Copper RTMs

1G Infrastructure 1G Base Ports 10G Infrastructure 10G Fabric Ports

BCM5464R Copper PHY Port2210/100/1000 BaseT/RJ-45 XAUI Pass through 10G Fabric XAUI CX4

BCM5464R Copper PHY Port2310/100/1000 BaseT/RJ-45 None None

BCM5464R Copper PHY Port2410/100/1000 BaseT/RJ-45 None None

Front panel Base Port 18 J13 3rd BCM5464R Copper PHY

Front panel Base Port 19 J13 Bottom BCM5464R Copper PHY

TABLE 2-13 AMC Port Restrictions for Fiber Optic RTMs

1G Infrastructure 1G Base Ports 10G Infrastructure 10G Fabric Ports

SERDES SFP - LX SX SFP+ LRM

SERDES SFP - LX SX None

10/100/1000 BaseT/RJ-45

2-30 Sun Netra CP3240 Switch Installation Guide • September 2009

CHAPTER

Configuring Jumper Settings

This chapter describes jumper settings for configuring the switch. Other configuration options are software controlled. Software configuration options are described in Chapter 4.

This chapter contains the following topics:

■ Section 3.1, “Jumper Settings” on page 3-2

■ Section 3.2, “Jumper Locations” on page 3-7

3-1

3.1 Jumper Settings

The following jumper settings are available for configuring the switch. Each of jumper settings is described in the subsections.

TABLE 3-1 Switch Configuration Jumper Settings

Jumper Default Purpose

P4 L2 Cross Connect Control

P6(1-2) OFF Fabric Zero Reset Configuration Word

P6(3-4) OFF Base Zero Reset Configuration Word

P8(1-2) OFF Base Write Protect Disable

P8(3-4) OFF Fabric Write Protect Disable

P9 OFF IPMC Firmware Write

P10(1-2) OFF IPMC Disable

P10(3-4) OFF IPMC FWE

P11(1-2) OFF Forced Board Enable

P11(3-4) OFF IPMC Board Reset Disable

P12 OFF EMI Ground to Logic Ground

P13 OFF Serial Direction

3.1.1 P4 Cross-Connect Control

This jumper is used to control ShMC cross connect. ShMC cross connect is the ability to connect to two ShMCs at 10/100 each rather than use a single 10/100/1000 for a single ShMC. The first ShMC port is number 1 on the Base. The second ShMC port is number 17 on the Base.

TABLE 3-2 P4 Cross-Connect Jumper Settings

P4 Default Function

OFF Software control of cross connect.

1-2 Default Force cross connect enabled, base channel 1 is two 10/100BaseTX

ports.

3-4 No cross connect; base channel 1 is 10/100/1000Base-T.

3-2 Sun Netra CP3240 Switch Installation Guide • September 2009

3.1.2 P6(1-2) Fabric Zero Reset Configuration Word

This jumper is used to configure the Fabric Gigabyte CPU to use the default Reset Configuration Word, or use the one in the flash.

TABLE 3-3 P6 (1-2) Fabric Zero Reset Jumper Settings

P6(1-2) Default Function

ON Use default Reset Configuration Word (all zeros)

OFF Default Use Reset Configuration Word in flash

3.1.3 P6(3-4) Base Zero Reset Configuration Word

This jumper is used to configure the Base CPU to use the default Reset Configuration Word, or use the one in the flash.

TABLE 3-4 P6 (3-4) Base Zero Reset Jumper Settings

P6(3-4) Default Function

ON Use default Reset Configuration Word (all zeros)

OFF Default Use Reset Configuration Word in flash

3.1.4 P8(1-2) Base Write Protect

This jumper configures write protection on the Base reset configuration word.

TABLE 3-5 P8 (1-2) Base Write-Protect Jumper Settings

P8(1-2) Default Function

ON Disable the write protection of the Base reset configuration word

OFF Default Write protect the Base reset configuration word

Chapter 3 Configuring Jumper Settings 3-3

3.1.5 P8(3-4) Fabric Write Protect

This jumper configures write protection on the Fabric reset configuration word.

TABLE 3-6 P8 (3-4) Fabric Write Protect Jumper Settings

P8(3-4) Default Function

ON Disable the write protection of the Fabric reset configuration word

OFF Default Write protect the Fabric reset configuration word

3.1.6 P9 IPMC Firmware Program

These jumpers configure IPMC firmware write actions via the debug serial port.

TABLE 3-7 P9 IPMC Firmware Program Jumper Settings

P4 Default Function

OFF Default Normal operation

1-2 Install to program IPMC firmware via the debug serial port

3-4 Install to program IPMC firmware via the debug serial port

3.1.7 P10(1-2) IPMC Reset

This jumper configures the IPMI subsystem reset function.

TABLE 3-8 P10 (1-2) IPMC Reset Jumper Settings

P10 (1-2) Default Function

ON Disable IPMI subsystem (hold it in reset)

OFF Default Enable IPMI subsystem

3-4 Sun Netra CP3240 Switch Installation Guide • September 2009

3.1.8 P10(3-4) IPMC FWE

These are reserved and should be left configured for the default setting.

TABLE 3-9 P10 (3-4) IPMC FWE Reserved Jumper Settings

P10 (3-4) Default Function

OFF Default Normal operation

3.1.9 P11(1-2) Forced-Board Enable

This jumper controls whether the switch is forced on when it is powered up, or if the IPMI subsystem controls the power up of the switch. Forcing power to the board is not enough for the board to boot. The board could still be held in reset. Configre P11 (3-4) jumper setting as well to bring the board out of reset.

TABLE 3-10 P11 (1-2) Forced-Board Enable Jumper Settings

P11 (1-2) Default Function

ON Force power on. Use this to run without a ShMC.

OFF Default IPMI controls power to the board.

3.1.10 P11(3-4) IPMC Board Reset

This jumper configres the IPMI to send a reset signal that will reset the entire board

TABLE 3-11 P11 (3-4) IPMC Board Reset Jumper Settings

P11 (3-4) Default Function

ON IPMI subsystem cannot reset the switch. Use this to run without a

ShMC.

OFF Default IPMI subsystem can reset the switch and hold it in reset.

Chapter 3 Configuring Jumper Settings 3-5

3.1.11 P12(1-2) and P12 (3-4) EMI Ground to Logic Ground

The switch and the entire AdvancedTCA shelf separate the ground of the chassis itself from digital ground for EMI protection. This jumper configures the two grounds.

TABLE 3-12 P12 (1-2) and P12 (3-4) Grounding Jumper Settings

P12 (1-2) (3-4) Default Function

OFF Default Separate EMI ground and logic ground.

1-2 Connect EMI ground to logic ground.

3-4 Connect EMI ground to logic ground.

3.1.12 P13 Serial Direction

The front panel serial port and the RTM serial port are mutually exclusive; only one can be used at a time. The serial port can be forced to the front or the RTM, or it can be controlled by the software.

TABLE 3-13 P13 Serial Direction Jumper Settings

P13 Default Function

OFF Default Software control of serial direction.

1-2 Front serial port active; RTM serial port disabled.

3-4 Front serial port disabled; RTM serial port active.

3-6 Sun Netra CP3240 Switch Installation Guide • September 2009

3.2 Jumper Locations

FIGURE 3-1 shows the locations of the jumper settings.

FIGURE 3-1 Switch Jumper Locations

Chapter 3 Configuring Jumper Settings 3-7

3-8 Sun Netra CP3240 Switch Installation Guide • September 2009

CHAPTER

Configuring Switch Software

This chapter describes how to configure the switch software.

The switch incorporates four main software components:

■ IPMI firmware, which runs on a separate microcontroller.

■ uBoot loads the boot for the system. It can be compared to a BIOS on a node

board.

■ Operating system (OS), which uses Monta Vista 3.1 Pro, based on the 2.4.20 Linux

kernel.

■ LVL7 Systems’ FASTPATH software, which provides all of the management and

control features of the switch.

The IPMI firmware, uBoot, and OS are covered in this chapter. FASTPATH is described, however, for detailed information and command syntax, refer to the Netra CP3240 Switch Command Reference Manual (820-3253).

This chapter contains the following topics:

■ Section 4.1, “IPMI Firmware Sensors” on page 4-2

■ Section 4.2, “uBoot” on page 4-5

■ Section 4.3, “Linux” on page 4-7

■ Section 4.4, “Serial Select” on page 4-8

■ Section 4.5, “Serial Location” on page 4-9

■ Section 4.6, “Boot Sequence” on page 4-10

■ Section 4.7, “Primary and Backup Flash” on page 4-13

■ Section 4.8, “Dual Firmware Images” on page 4-13

■ Section 4.9, “Network Boot” on page 4-14

■ Section 4.10, “FASTPATH” on page 4-18

■ Section 4.11, “Firmware Updates” on page 4-24

■ Section 4.12, “Fiber Optic RTM Configuration” on page 4-25

4-1

4.1 IPMI Firmware Sensors

This section describes the IPMI firmware sensors.

4.1.1 State Sensors

TABLE 4-1 IPMI State Sensors

Sensor Name Type Description Discrete States Returned

FRU 0

TABLE 4-2

HOT_SWAP

1 RTM hotswap State hotswap discrete

2 AMC 0

hotswap

3 AMC 1

hotswap

4 AMC 2

hotswap

5 IPMB physical State IPMB Link sensor Defined in ATCA spec.

6 BMC Watchdog State Watchdog timer Discrete, Watchdog 2 (per the

11 RTM Presence State Rear Module

12 Base Early* State Base firmware

13 Base Full* State Base firmware

14 Base Good* State Base firmware

15 Fabric Early* State Fabric firmware

16 Fabric Full* State Fabric firmware

17 Fabric Good* State Fabric firmware

State hotswap discrete

state

State hotswap discrete

state

State hotswap discrete

state

State hotswap discrete

state

Present Signal

signal

M state, per ATCA spec

M state, per ATCA and AMC spec

IPMI spec)

State 01h = Device absent, 02h = Device present

Asserted indicates Uboot prompt

Asserted indicates Linux has booted

Asserted indicates FASTPATH has loaded

Asserted indicates Uboot prompt

Asserted indicates Linux has booted

Asserted indicates FASTPATH has loaded

4-2 Sun Netra CP3240 Switch Installation Guide • September 2009

*These sensors represent the software state of the switch.

The following tables lists the logic that drives these sensors.

TABLE 4-3 IPMI Sensor Logic

State Timeout Failover Condition Early Full Good

Initial power-on state 5 sec Uboot corrupted or

erased

uBoot initial state 5 sec Uboot finds no

image

uBoot console n/a Failover timer stops 1 0 0

Linux booted 90 sec Soft-off now occurs 0 1 0

Linux boot menu n/a Failover timer stops

(user interruption)

FASTPATH booted 90 sec Timer stops if this

state is reached in time

Critical code update n/a An update is in

progress; the failover time stops

001

000

110

011

111

Chapter 4 Configuring Switch Software 4-3

4.1.2 Threshold Sensors

TABLE 4-4 IPMI Thresold Sensors

Sensor Name Type Description Units

# Minor Major Critical Minor Major Critical

7 +12.0V Thresh +12V

Volts 11.016 10.21 9.02 13.45 13.72 13.77

Lower Thresholds

Upper Thresholds

main power

8 +3.3V Thresh +3.3V line Volts 3.126 3.02 2.914 3.563 3.654 3.805

9 +2.5V Thresh +2.5V line Volts 2.308 2.202 2.106 2.701 2.808 2.904

10 +1.25V Thresh +1.25V

Volts 1.156 1.009 0.7546 1.5582 1.754 1.9502

line

18 Base

CPU

Thresh Base CPU

Temp

C n/a n/a n/a 60 70 80

Temp

19 RTM

temp

Thresh RTM

Temperatu

C n/a n/a n/a 55 65 75

20 Fabric

CPU

Thresh Fabric

CPU Temp

C n/a n/a n/a 60 70 80

Temp

21 +1.5V Thresh +1.5V line Volts 1.4014 1.303 1.1074 1.6072 1.7052 1.9012

22 +1.8V Thresh +1.8V line Volts 1.656 1.509 1.303 1.9502 2.107 2.303

23 +1.0V Thresh +1.0V line Volts 0.902 0.706 0.51 1.1074 1.303 1.509

24 +1.2V Thresh +1.2V line Volts 1.107 1.009 0.804 1.303 1.4014 1.607

25 Site 1

Pwr Cur

26 Site 1

Pwr

27 Site 1MPThresh AMC Slot

Thresh AMC Slot

current sense

Thresh AMC Slot

+12V

Amps n/a n/a n/a 5.5 6.023 6.517

Volts n/a n/a n/a 13.048 13.44 13.608

Volts n/a n/a n/a 3.456 3.514 3.6 Managem ent Pwr

28 Site 2

Pwr Cur

29 Site 2

Pwr

Thresh AMC Slot

current sense

Thresh AMC Slot

+12V

Amps n/a n/a n/a 5.5 6.023 6.517

Volts n/a n/a n/a 13.048 13.44 13.608

4-4 Sun Netra CP3240 Switch Installation Guide • September 2009

TABLE 4-4 IPMI Thresold Sensors (Continued)

Sensor Name Type Description Units

30 Site 2MPThresh AMC Slot

Managem ent Pwr

31 Site 3

Pwr Cur

32 Site 3

Pwr

33 Site 3MPThresh AMC Slot

Thresh AMC Slot

current sense

Thresh AMC Slot

+12V

Managem ent Pwr

Volts n/a n/a n/a 3.456 3.514 3.6

Amps n/a n/a n/a 5.5 6.023 6.517

Volts n/a n/a n/a 13.048 13.44 13.608

Volts n/a n/a n/a 3.456 3.514 3.6

4.2 uBoot

uBoot is the boot loader. Much like a BIOS, it brings the system to a usable state for the operating system (OS) to boot. It also performs a POST of the CPU subsystem. It can be used as a recovery console if the firmware image becomes corrupt or a firmware update fails. Several important environment variables are stored in uBoot, only some of which should ever be changed, namely noekey and baudrate.

Lower Thresholds

Upper Thresholds

4.2.1 uBoot Console

To get to a uBoot console, you must prevent the switch from booting into the OS. Shown here is an example of the start of a boot sequence.

CPU: 400 MHz DRAM: 256 MB FLASH: 64 MB, Base, Primary Flash

### JFFS2 loading 'image1' to 0x400000 Scanning JFFS2 FS: . done. ### JFFS2 load complete: 10026940 bytes loaded to 0x400000 Booting ... Loading Ramdisk to 0fdf6000, end 0ff8d551 ... OK

Press any key before ### JFFS2 loading 'image1' to 0x400000 appears.

Chapter 4 Configuring Switch Software 4-5

There is only a one-second delay to press a key.

When the uBoot prompt is displayed, you can then enter any of the following commands.

TABLE 4-5 uBoot Console Commands

Command Result

print or printenv

set

or setenv

save or saveenv

Shows the current environment variables.

Followed by an environment variable, this command changes the environment variable.

Writes the variables to flash. You must save the changes if you want them to persist through a reset.

4.2.2 E-Keying Control in uBoot

See the Section 4.3.1, “e-Keying” on page 4-7 for a description of e-Keying, and how it is supported.

To disable e-Keying, use the noekey environment variable.

List the ports to be disabled separated by only commas.

set noekey 1,2,3,4

Or use the word all to disable e-Keying completely.

set noekey all

To re-enable e-Keying clear the variable.

set noekey

After changing the environment variables you must always save if you would like the change to persist though a reset.

4-6 Sun Netra CP3240 Switch Installation Guide • September 2009

4.2.3 Serial-Baud Rate Control in uBoot

You can change the serial baud rate from within FASTPATH, the boot menu, or in uBoot. Changing it in uBoot is the only method that allows it to persist through a reset. Only standard baud rates are accepted.

Note – Change the baud rate on the Fabric first, then on the Base. If the baud rate

on the Fabric and Base do not match, the Fabric will not be accessible. If the Fabric is not accessible, change the baud rate on the Base until the Fabric is accessible.

To change the baud rate in uBoot, enter the command as in the following example.

set baudrate <115200>

After changing the rate, you must save if you would like the change to persist though a reset.

4.3 Linux

The switch uses Linux as an operating system. The Monta Vista 3.1 Pro 2.4.20 kernel is used. There are no settings to change, and the OS is completely transparent to the user.

4.3.1 e-Keying

e-Keying is implemented as a Linux driver. The CPUs for both Base and Fabric have a direct connection to the IPMI controller, which is used to communicate e-Keying messages. The CPU is interrupted when an e-Keying event occurs. The driver handles these interrupts and disables ports based on the information it receives. This disables the port at the PHY level. e-Keying can be ignored by creating a uBoot environment variable as described in the Section 4.2.2, “E-Keying Control in uBoot”

on page 4-6.

Chapter 4 Configuring Switch Software 4-7

4.3.2 e-Keying Bypass

e-Keying can be ignored by creating a uBoot environment variable as described in

Section 4.2.2, “E-Keying Control in uBoot” on page 4-6. When e-Keying is bypassed,

ports are no longer enabled or disabled based on the e-Keying state. The e-Keying information is still shared with the shelf manager as usual, but now ports are enabled regardless if the shelf manager enables them or not.

4.3.3 ATCA LEDs

The ATCA LEDs can be turned on from several sources, one of which is a Linux driver. Both the Base and Fabric drive these LEDs. The red OOS LED is driven until FASTPATH has loaded, at which point the green Healthy LED is driven. The OOS LED signals are “OR” so that if one or more are active, the LED is on. The Healthy LED signals are “AND.” The Base and Fabric must be booted into FASTPATH as well as several hardware conditions must be met for this LED to be on.

4.4 Serial Select

A single-serial port is provided to control both CPUs. The ` key, usually in the upper left key on the keyboard and the same key as ~, can be used to switch between controlling the Base (3.0) and the Fabric (3.1). The ` key works in uBoot, Linux, the BCM debug shell, and FASTPATH.

When you switch between Base and Fabric [Base] or [Fabric] is printed to the console so that you know which one you are currently controlling.

(switch Base) # (and then ENTER is pressed here) (switch Base) #[Fabric] (and then ENTER is pressed here) (switch Fabric) #[ Base ] (switch Base) # User:

4-8 Sun Netra CP3240 Switch Installation Guide • September 2009

4.5 Serial Location

The switch supports running its serial port to both the front panel and the RTM. It does not support both the front panel and RTM at the same time. You must configure the port to go to the location needed. By default, the port is routed to the front panel.

4.5.1 Changing Serial Location from UBoot

In this example, the serial port direction is to the RTM.

CODE EXAMPLE 4-1 Serial Port Change from uBoot

CPU: 400 MHz DRAM: 256 MB FLASH: 64 MB, Base, Primary Flash

Use the [`] key above the TAB key to switch between BASE and FABRIC. [Break UBoot by pressing a key when the switch boots] => set console rtm [set console to front for the front panel] => save Saving Environment to Flash... Un-Protected 1 sectors Erasing Flash... . done Erased 1 sectors Writing to Flash... done Protected 1 sectors => boot

4.5.2 Changing Serial Location from FASTPATH

In this example, the serial port direction is changed to the RTM. Note that the change is instantaneous and lasts though a reboot.

CODE EXAMPLE 4-2 Serial Port Change from FASTPATH

configure lineconfig serial location rtm [set location to front for the front panel]

Chapter 4 Configuring Switch Software 4-9

4.6 Boot Sequence

The following is an example of a boot sequence.

CODE EXAMPLE 4-3 Boot Sequence Example

CPU: 400 MHz DRAM: 256 MB FLASH: 64 MB, Base, Primary Flash

### JFFS2 loading 'image1' to 0x400000 Scanning JFFS2 FS: . done. ### JFFS2 load complete: 10026500 bytes loaded to 0x400000 Booting ... Loading Ramdisk to 0fdf6000, end 0ff8d4c2 ... OK

Mounting /dev/mtdblock2 at /mnt/fastpath...done. Mounting tmpfs at /mnt/application...done.

***************************************************************** *** * Use the [`] key above the TAB to switch between BASE and FABRIC. * ***************************************************************** ***

switch, Base, Primary Flash, image1

Select startup mode. If no selection is made within 5 seconds, the Application will start automatically...

--- Main Menu ---

1 - Start Switch Application 2 - Display Utility Menu Select (1, 2):

Extracting Application from image1.....done

Loading Application...done. SOC unit 0 attached to PCI device BCM56503_B2

FASTPATH starting...started!

(Unit 1)>

User:

4-10 Sun Netra CP3240 Switch Installation Guide • September 2009

The switch takes approximately 50 seconds to boot to a console and be fully functional.

4.6.1 Boot Utility Menu

There is a utility menu that can be used before FASTPATH boots. During boot there is a two-option menu displayed for five seconds. This menu allows access to the utility menu if you press number 2 and Enter.

CODE EXAMPLE 4-4 Boot Utility Menu Access

switch, Base, Primary Flash, image2

--- Utility Menu ---

1 - Start Switch Application 2 - Load Code Update Package using TFTP/FTP 3 - Load Code Update Package using XMODEM/YMODEM/ZMODEM 4 - Copy Primary Flash to Backup Flash 5 - Select Serial Speed 6 - Retrieve Error Log using TFTP 7 - Retrieve Error Log using XMODEM/YMODEM/ZMODEM 8 - Erase Current Configuration 9 - Erase Permanent Storage 10 - Select Boot Method 11 - Activate Backup Image 12 - Start Diagnostic Application 13 - Reboot

Select option (1-13):

Most of the options are self explanatory based on their names. The following sections describe some of the options more fully.

Chapter 4 Configuring Switch Software 4-11

4.6.1.1 Load Code Update Package using TFTP/FTP

FASTPATH can be updated from within FASTPATH itself, but it can also be updated from this menu.

■ The update image must be on a TFTP server or FTP server.

■ The server must be on the same network as the out-of-band management port.

■ You need to have the IP of the server, the desired IP of the board being updated,

the gateway (if needed), and the file name.

Note – DHCP can be used to obtain an IP for the switch during this update. Enter

dhcp as the IP address. This entry begins the update and provides status information as it is updating.

4.6.1.2 Erase Current Configuration

This is the same as clear config from within FASTPATH. This option can be used if the switch is in an unknown state, and restoring the default settings is desired.

4.6.1.3 Erase Permanent Storage

This command completely erases FASTPATH, any log files, and any configurations. It does not erase uBoot or Linux. Use of this command is never recommended. Updates can safely be installed without running this option, and configurations and log files are preserved.

4.6.1.4 Select Boot Method

The switch supports three boot methods:

■ from the local image on the flash

■ from an image over the network

■ from an image over the serial port

The default option is booting from the flash. See Section 4.9, “Network Boot” on

page 4-14 for more information.

4-12 Sun Netra CP3240 Switch Installation Guide • September 2009

4.6.1.5 Start Diagnostic Application

This option boots the Broadcom diag shell, currently SDK version 5.4.1. This environment is provided as is, with no support. Several commands provided in this shell are not supported and will not work. This shell is provided mainly for debug, testing, and diagnostics purposes. This shell has many low-level tests, and low-level register access. It can be used to check the integrity of particular boards. Help is provided in the shell with ?? and commands followed by a single question mark. Some commands of interest are TestList and TestRun.

4.7 Primary and Backup Flash

The switch features hardware flash redundancy. If the switch fails to boot from the primary flash for any reason, the onboard IPMC will reset the board and attempt to boot out of the backup flash. No user interaction is required.

The switch backup flash comes preprogrammed with default settings. You can change what is loaded on the backup flash by using the Copy Primary Flash to Backup Flash command from the utility menu.

4.8 Dual Firmware Images

The switch supports dual firmware images of the Base and Fabric software. While this feature does not provide high availability (the hardware redundant flash provides that), it does provide a way to store two copies of the firmware on the switch. This allows you to switch between firmware, as needed.

4.8.1 Booting the Non-Active Image

Which image is active can be changed several ways.

From the boot menu, the Activate Backup Image option can be used.

From within FASTPATH the boot system command in combination with the update bootcode commands can be used.

Chapter 4 Configuring Switch Software 4-13

4.8.2 Updating the Non-Active Image

All of the update methods supported by the active image (image1 by default) are also supported for the non-active image (image2).

4.8.3 Fabric 1G/10G Auto-negotiation

The Fabric networks support both 1Gigabyte and 10Gigabyte operation. No configuration is required to put the switch into either a 1Gigabyte or 10Gigabyte mode. The switch will dynamically connect at the appropriate speed.

4.9 Network Boot

The switch supports network booting. This feature can be used to test updates to FASTPATH without installing them. Updates containing changes to uBoot, Linux, or the IPMC must be installed to the flash before they can be used.

1. Setup a TFTP server on the update network.

Tftpd is the standard TFTP server for Linux and Solaris.

For Windows, Solarwinds (

http://www.solarwinds.net) offers a free TFTP server.

2. Place the update image on the TFTP server.

It will have a .tgz file extension.

3. Connect to the serial management port of the network to be updated, and connect the MGMT port to the update network.

4. Reboot the switch using the reboot button, hotswap handle, or reload command.

5. As the switch boots, press number 2 to enter the Utility Menu.

6. Press number 8 to change the boot method.

7. Press number 3 to select network as the boot method.

8. Enter the information for your TFTP server.

For host IP, you can enter dhcp if you would like to use DHCP to obtain a valid IP address.

4-14 Sun Netra CP3240 Switch Installation Guide • September 2009

9. Press number 1 to boot the system.

CODE EXAMPLE 4-5 Example Console Output

CPU: 400 MHz DRAM: 256 MB FLASH: 64 MB, Base, Primary Flash

### JFFS2 loading 'image1' to 0x400000 Scanning JFFS2 FS: . done. ### JFFS2 load complete: 10026500 bytes loaded to 0x400000 Booting ... Loading Ramdisk to 0fdf6000, end 0ff8d4c2 ... OK

Mounting /dev/mtdblock2 at /mnt/fastpath...done. Mounting tmpfs at /mnt/application...done.

switch, Base, Primary Flash, image1

Select startup mode. If no selection is made within 5 seconds, the Application will start automatically...

--- Main Menu ---

1 - Start Switch Application 2 - Display Utility Menu Select (1, 2): 2

switch, Base, Primary Flash, image1

--- Utility Menu ---

1 - Start Switch Application 2 - Load Code Update Package using TFTP/FTP 3 - Load Code Update Package using XMODEM/YMODEM/ZMODEM 4 - Copy Primary Flash to Backup Flash 5 - Select Serial Speed

Chapter 4 Configuring Switch Software 4-15

CODE EXAMPLE 4-5 Example Console Output (Continued)

6 - Retrieve Error Log using TFTP 7 - Retrieve Error Log using XMODEM/YMODEM/ZMODEM 8 - Erase Current Configuration 9 - Erase Permanent Storage 10 - Select Boot Method 11 - Activate Backup Image 12 - Start Diagnostic Application 13 - Reboot

Select option (1-13): 10

Current Boot Method: FLASH

1 - Flash Boot 2 - Network Boot 3 - Serial Boot 4 - Exit without change Select option (1-4): 2 Enter Server IP []:10.10.3.199 Enter Host IP (# or dhcp) []:dhcp Enter Transfer Method (tftp or ftp) [ftp]: Enter Filename []:/switch.1.0.1.0.stk Accept changes? Press(Y/N): y

switch, Base, Primary Flash, image1

--- Utility Menu ---

4-16 Sun Netra CP3240 Switch Installation Guide • September 2009

CODE EXAMPLE 4-5 Example Console Output (Continued)

Select option (1-13): 1 Creating tmpfs filesystem on tmpfs for download...done. Bringing up eth0 interface...done. HOST IPADDR=10.10.2.246 Transferring '/switch.1.0.1.0.stk' from '10.10.3.199' via ftp...done. Bringing down eth0 interface...done. Extracting Application...done. Destroying tmpfs filesystem on tmpfs...done. Loading Application...done. SOC unit 0 attached to PCI device BCM56503_B2

FASTPATH starting...started!

(Unit 1)>

User:

10. Use option 8 of the Utility Menu to change the boot method back to flash, when desired.

Chapter 4 Configuring Switch Software 4-17

4.10 FASTPATH

The switch features LVL7 Systems’ FASTPATH 2340 software, version 4.4. FASTPATH is a software package providing the robust management needed to control a modern switch-router. This section serves as a quick primer on using FASTPATH on the switch. FASTPATH is not covered in detail in this guide, but is covered in the Netra CP3240 Switch Command Reference Manual (820-3253).

4.10.1 Management Options

The switch can be controlled by a CLI, a web interface, and through SNMP. All management interfaces, other than serial, can be enabled or disabled, provided over both the out-of-band management port and/or any in-band ports, and be limited to certain in-band ports.

4.10.1.1 CLI

The switch provides an industry standard CLI. The CLI is provided over the serial port, telnet and SSH. This section only describes some basic commands. For detailed command syntax, refer to the Netra CP3240 Switch Command Reference Manual (820-3253).

The CLI is mode based. It works similarly to a console in Linux or Windows. Commands are grouped under modes, and those commands only work when the user is in the current mode. There are very few global commands.

To return to one mode higher than the current mode, use exit.

For example to shutdown port 17, switch to enable mode, configure mode, interface 17 mode, then issue the shutdown command. Use exit to return to configure mode, and exit again to return to enable mode.

Many commands have a no form. The no form is used to disable the command. With the previous example, to re-enable port 17, once you are in the interface 17 mode, you would issue the no shutdown command.

4-18 Sun Netra CP3240 Switch Installation Guide • September 2009

4.10.2 Basic CLI Commands

TABLE 4-6 Basic CLI Commands

Command Function Mode

enable Switch to privilege mode. You must be in enabled

mode for most options.

show port all Show status of the ports. privilege

show interface ethernet 0/x Show detailed statistics on port 0/x. privilege

clear counters Clear all statistics. privilege

clear config Restore the default configuration. privilege

show running-config Show the current configuration of the switch.

This command shows everything that is not set to the default value. The output is a script that can be copied to a file for later use, or for another switch.

copy system:running-config nvram:startup-config

serviceport protocol dhcp Use DHCP on the out-of band port. Only one of

serviceport protocol none Use user assigned IP NOTE: the same command

serviceport ip <ip> <netmask>

network parms <ip> <netmask> <gateway>

show network View the in-band management settings. privilege

show serviceport View the out-of-band management settings. privilege

serial baudrate Change the serial baudrate. privilege

vlan database Switch to vlan database mode. Create and delete

vlan x Create a VLAN with number x. vlan

exit Return to one mode higher.

configure Switch to configure mode. You must be in

Save the current configuration though a reboot. privilege

the serviceport and network can use DHCP. To enable DHCP on one, you must disable it on the other. The same command works for network.

works for network.

Force an IP for serviceport. privilege

Force an IP for network. privilege

VLANs here.

configure mode to change most settings.

default

privilege

Chapter 4 Configuring Switch Software 4-19

TABLE 4-6 Basic CLI Commands (Continued)

interface 0/x Switch to interface mode. You must be in

interface mode to change most port specific settings.

vlan participation include x Add interface to VLAN x. interface

vlan pvid x Change interface’s pvid to VLAN x. interface

configure

4.10.3 Logins and Prompts

FASTPATH supports multiple users with different security levels. By default, there is one admin user with no password. In the CLI, privilege mode is password-protected separately from the default mode, but also has a default of no password.

4.10.3.1 CLI Defaults

A CLI is provided on the serial console, telnet console, and SSH console. The serial

console is always enabled. The telnet console is enabled by default. SSH is disabled by default.

Username:admin

Password: >enable Password:

The prompts always show the current mode. Here are a few examples.

default mode # privilege mode # configure mode (interface 0/2) # interface 0/2 mode

4.10.3.2 Web Interface Defaults

The username and password are the same as the CLI. The web interface is enabled by default. Java is enabled by default. HTTPS is disabled by default.

4-20 Sun Netra CP3240 Switch Installation Guide • September 2009

4.10.3.3 SNMP Defaults

SNMP is enabled by default. The default read-only community string is public. The default read-write community string is private.

4.10.4 Secure Remote Access

4.10.4.1 SSH and SSL/TLS Keys

The switch supports SSH for a secure CLI console as well as SSL/TLS for secure HTTP. By default, SSH and SSL are disabled. The switch cannot generate its own keys. Keys must be generated on an external PC, and uploaded to the switch via TFTP. Once the keys are on the switch, SSH and HTTPS must be enabled to be used.

4.10.4.2 Enabling SSH and SSL

First, the certifications and keys must be uploaded to the switch using a command such as the following:

copy tftp://<ip>/<file> <location>

Upload the following files to the switch:.

File Location

rsa1.key nvram:sshkey-rsa1

rsa2.key nvram:sshkey-rsa2

dsa.key nvram:sshkey-dsa

dh512.pem nvram:sslpem-dhweak

dh1024.pem nvram:sslpem-dhstrong

server.pem nvram:sslpem-server

rootcert.pem nvram:sslpem-root

Enable secure access (SSH/Telnet) and disable non-secure access (SSL/HTTP).

ip ssh configure

Chapter 4 Configuring Switch Software 4-21

lineconfig no transport input telnet

exit

ip http secure-server no ip http server

4.10.5 Default Settings

The switch comes configured with a default configuration. This configuration boots the board to Layer 2 switching. This configuration is very basic and should be updated for your environment. The default settings are as follows:

■ every port is in VLAN 1

■ every port is configured in switching mode

■ management interfaces are enabled

■ spanning-tree is enabled on every port

■ everything else is disabled

The settings of the switch can be checked with the show running-config command. This command shows how the current configuration is different than the default configuration. This command can be very useful because the output is in script format. This output can be backed up or copied to another switch.

4.10.6 Port Ordering

The ports are ordered the same way as the ATCA channels. The ATCA specification allows for backplanes to be routed differently. This means that logical slots and physical slots often do not match.

The switch supports an abstraction layer that allows the port order to be changed. This allows vendors to match logical slots and physical slots independent of routing. Contact Sun if you would like a non-standard port order. Sun’s port ordering is shown in the following table for shelves with middle hub slots, that is, slots 7 and 8.

TABLE 4-7 Port Order List

Shelf Physical Slot Base Port Fabric Port

11312

21110

4-22 Sun Netra CP3240 Switch Installation Guide • September 2009

Sun Microsystems CP3240 User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Contents

Figures

Tables

Preface

Getting Started

1.1 System Requirements

1.1.1 Connectivity

1.1.1.1 Hub Connectivity

1.1.2 Electrical and Environmental

1.2 Unpacking

1.3 Handling Switches

1.4 Connectors

1.5 Jumper Options

1.6 Switch and RTM Faceplates

1.6.1 Base 10/100/1000 Uplink Ports (RJ-45)

1.6.2 10/100 Management Port (RJ-45)

1.6.3 Serial Management Port (RJ-45)

1.7 RTM Airflows

1.7.1 RTM Airflow Requirements

1.7.2 RTM Pressure Drop Versus Airflows (Impedence Curves)

1.8 Removing and Installing Switches

1.8.1 Removing a Switch Set

1.8.1.1 Removing a Switch From the Front of the Server

1.8.1.2 Removing a Rear Transition Module for a Switch

1.8.2 Installing a Switch Set

1.8.2.1 Installing the Rear Transition Module for a Switch

1.8.2.2 Installing a Switch

1.9 Switch LEDs

1.9.1 ATCA Board Status LEDs

1.9.2 Hot-Swap LED

Overview

2.1 Features

2.1.1 General

2.1.2 Base Interface

2.1.3 Fabric Gigabit Interface

2.1.4 AMC Sites

2.2 Switch Components

2.2.1 Broadcom StrataXGS 3 BCM56503 Ethernet Switch

2.2.2 Broadcom StrataXGS 3 BCM56800 Ethernet Switch

2.2.3 Broadcom BCM5464R and BCM5461S 10/100/1000Base-T Ethernet PHY

2.2.4 Freescale PowerQUICC II MPC8247 Communications Processor

2.2.5 Pigeon Point BMR-H8S-AMCc AdvancedTCA IPMI Subsystem

2.3 Protocols, RFCs, and MIBs Support

2.3.1 FASTPATH Switching

2.3.1.1 Additional Layer 2 Functionality

2.3.1.2 System Facilities

2.3.1.3 Switching MIBs

2.3.1.4 Routing MIBs

2.3.2 FASTPATH Routing

2.3.3 FASTPATH Quality of Service

2.3.3.1 DiffServ

2.3.3.2 Access Control Lists (ACLs)

2.3.3.3 Class of Service (CoS)

2.3.3.4 Quality of Service MIBs

2.3.4 FASTPATH Multicast

2.3.4.1 Multicast MIBs

2.3.5 FASTPATH IPv6 Routing

2.3.5.1 IPv6 Routing MIBs

2.3.6 FASTPATH Management

2.3.6.1 SSL 3.0 and TLS 1.0

2.3.6.2 SSH 1.5 and 2.0

2.3.6.3 Additional Management Features

2.4 Functional Diagrams and Port Maps

2.4.1 Switch

2.4.2 Rear Transition Modules (RTM)s

2.4.2.1 Copper RTM

2.4.2.2 Fiber Optic RTM

2.4.3 Base Fabric Switch Subsystem

2.4.4 Expansion Fabric Switch Subsystem

2.4.5 AdvancedMC Sites

2.4.5.1 AMC Port Maps

2.4.5.2 AMC Module Support by Site

2.4.5.3 AMC Port Restrictions for RTMs

Configuring Jumper Settings

3.1 Jumper Settings

3.1.1 P4 Cross-Connect Control