Sun Microsystems,Inc. hasintellectual property rightsrelating totechnology embodiedin theproduct that is described in this document. In
particular,and withoutlimitation, theseintellectual property rights may include one or more ofthe U.S.patents listedat
http://www.sun.com/patents and one or more additionalpatents orpending patentapplications inthe U.S.and inother countries.
This distributionmay includematerials developedby thirdparties.
Parts ofthe productmay bederived from BerkeleyBSD systems,licensed fromthe Universityof California.UNIX isa registered trademarkin
the U.S.and inother countries,exclusively licensedthrough X/OpenCompany, Ltd.
Sun, SunMicrosystems, the Sun logo, Solaris,Netra andthe Netra logo are trademarksor registered trademarksof SunMicrosystems, Inc., orits
subsidiaries, inthe U.S.and othercountries.
Products covered byand informationcontained inthis servicemanual arecontrolled by U.S. Export Control laws and may be subject to the
export orimport lawsin othercountries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether
direct orindirect, are strictly prohibited. Exportor reexport to countries subject to U.S. embargo orto entitiesidentified onU.S. exportexclusion
lists, including,but notlimited to,the deniedpersons andspecially designatednationals listsis strictlyprohibited.
DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
INCLUDING ANY IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,
ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Sun Microsystems,Inc. détientles droits depropriété intellectuelsrelatifs à la technologie incorporée dans le produit qui est décrit dans ce
document. Enparticulier,et cesans limitation,ces droits de propriété intellectuellepeuvent inclure un ou plus des brevetsaméricains listésà
l’adresse http://www.sun.com/patents et un ou les brevets supplémentaires oules applicationsde brevet enattente auxEtats -Unis etdans les
autres pays.
Cette distributionpeut comprendre descomposants développéspar destierces parties.
Sun, SunMicrosystems, lelogo Sun,Solaris, Netraet lelogo Netrasont desmarques de fabrique ou des marques déposées de Sun
Microsystems, Inc.,ou sesfiliales, auxEtats-Unis etdans d’autres pays.
Ce produitest soumisà lalégislation américaineen matière decontrôle desexportations etpeut êtresoumis àla règlementationen vigueur
dans d’autrespays dansle domainedes exportationset importations.Les utilisations, ouutilisateurs finaux,pour desarmes nucléaires, des
missiles, desarmes biologiqueset chimiquesou dunucléaire maritime,directement ou indirectement, sont strictement interdites. Les
exportations oureexportations versles payssous embargo américain,ou versdes entitésfigurant surles listesd’exclusion d’exportation
américaines, ycompris, maisde manièrenon exhaustive,la listede personnesqui fontobjet d’unordre de ne pas participer, d’une façon directe
ou indirecte,aux exportationsdes produits oudes servicesqui sontrégis parla législationaméricaine enmatière decontrôle desexportations et
la listede ressortissantsspécifiquement désignés,sont rigoureusement interdites.L’utilisation de pièces détachées ou d’unités centrales de
remplacement estlimitée auxréparations ouà l’échangestandard d’unités centrales pour les produits exportés, conformément à la législation
américaine enmatière d’exportation.Sauf autorisationpar les autorités des Etats-Unis, l’utilisation d’unités centrales pour procéder à des mises
à jourde produitsest rigoureusement interdite.
LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES
OU TACITES SONT FORMELLEMENTEXCLUES, DANSLA MESUREAUTORISEE PARLA LOIAPPLICABLE, YCOMPRIS NOTAMMENT
TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A
L’ABSENCE DE CONTREFACON.
Please
Recycle
Please
Recycle
Contents
Prefacexix
1.Getting Started1–1
1.1System Requirements1–2
1.1.1Connectivity1–2
1.1.1.1Hub Connectivity1–2
1.1.2Electrical and Environmental1–2
1.2Unpacking1–4
1.3Handling Switches1–4
1.4Connectors1–5
1.5Jumper Options1–5
1.6Switch and RTM Faceplates1–6
1.6.1Base 10/100/1000 Uplink Ports (RJ-45)1–7
1.6.210/100 Management Port (RJ-45)1–7
1.6.3Serial Management Port (RJ-45)1–7
1.7RTM Airflows1–7
1.7.1RTM Airflow Requirements1–7
1.7.2RTM Pressure Drop Versus Airflows (Impedence Curves)1–8
1.8Removing and Installing Switches1–10
1.8.1Removing a Switch Set1–10
v
1.8.1.1Removing a Switch From the Front of the Server1–11
1.8.1.2Removing a Rear Transition Module for a Switch1–13
1.8.2Installing a Switch Set1–14
1.8.2.1Installing the Rear Transition Module for a Switch1–14
TABLE B-12ATCA Zone 3 RTM Connector (J8-Top)) B–11
TABLE B-13ATCA Zone 3 RTM Connector (J7-Middle) B–11
TABLE B-14ATCA Zone 3 RTM Connector (J6-Bottom) B–12
Tablesxvii
xviiiSun Netra CP3240 Switch Installation Guide • September 2009
Preface
The Sun Netra CP3240 Switch Installation Guide describes the installation and
configuration of the Sun Netra CP3240 switch. This guide also includes information
about software, environment specifications, connectors, and certifications.
Before You Read This Document
Obtain and read the following documents:
■ Sun Netra CP3x40 Switch Safety and Compliance Manual (820-3505)
■ Sun Netra CP3240 Switch Product Notes (820-3260)
xix
How This Document Is Organized
Chapter 1 provides unpacking instructions and initial setup information for the
switch. It provides information and procedures needed to install and make the Sun
Netra CP3240 switch operational.
Chapter 2 introduces the key features of the switch. This chapter includes a product
definition, a list of product features, and a functional block diagram with a brief
description of each block. This chapter can be used to compare the features of the
switch against the needs of a specific application.
Chapter 3 describes the jumper settings on the switch. This chapter details factory
default settings and provides information about tailoring the board to the needs of
specific applications.
Chapter 4 describes the software packages running on the switch. This section serves
as a primer for using the software on the switch.
Appendix A contains the electrical, environmental, and mechanical specifications for
the switch.
Appendix B This appendix provides a connector location illustration and connector
pin out tables. A detailed description and pin out for each connector is given.
Appendix C provides links to websites with information about many of the devices
and technologies used in the switch.
Appendix D presents UL, CE, and FCC agency approval and certification
information for the switch.
xxSun Netra CP3240 Switch Installation Guide • September 2009
Typographic Conventions
TypefaceMeaningExamples
AaBbCc123The names of commands, files,
and directories; on-screen
computer output
AaBbCc123What you type, when contrasted
with on-screen computer output
AaBbCc123Book titles, new words or terms,
words to be emphasized.
Replace command-line variables
with real names or values.
Note – Characters display differently depending on browser settings. If characters
do not display correctly, change the character encoding in your browser to Unicode
UTF-8.
Edit your.login file.
Use ls -a to list all files.
% You have mail.
su
%
Password:
Read Chapter 6 in the User ’s Guide.
These are called class options.
Yo u must be superuser to do this.
To delete a file, type rm filename.
Prefacexxi
Related Documentation
The following table lists the documentation for this product. The online
documentation is available at:
Sun Netra CP3240 Advanced
Mezzanine Card Getting Started
Guide
Sun Netra CP3240 Advanced
Mezzanine Card 10G Copper User’s
Guide
Sun Netra CP3240 Advanced
Mezzanine Card 10G Optic User’s
Guide
Sun Netra CP3240 Advanced
Mezzanine Card LED User’s Guide
and Compliance Manual
820-3260-xx PDFOnline
820-3254-xx PrintedShipping Kit
820-3252-xx PDFOnline
820-3253-xx PDFOnline
820-7260-xx PrintedShipping Kit
820-7261-xx PDFOnline
820-7262-xx PDFOnline
820-7263-xx PDFOnline
820-3505-xx PDFOnline
The following table lists the documentation that is related to this product. The online
documentation is available at:
http://docs.sun.com/app/docs/prod/n900.srvr#hic
xxiiSun Netra CP3240 Switch Installation Guide • September 2009
.
ApplicationTitlePart NumberFormatLocation
Latest
information
Pointer DocNetra CT 900 Server Getting Started
OverviewNetra CT 900 Server Overview819-1174-xx PDFOnline
InstallationNetra CT 900 Server Installation
ServiceNetra CT 900 Server Service Manual 819-1176-xx PDFOnline
AdministrationNetra CT 900 Server Administration
ProgrammingNetra CT 900 Software Developer’s
SafetyNetra CT 900 Server Safety and
SetupNetra CT 900 Server Hardware
SafetyImportant Safety Information for
Netra CT 900 Server Product Notes819-1180-xx PDFOnline
819-1173-xx PrintedShipping kit
Guide
819-1175-xx PDFOnline
Guide
819-1177-xx PDFOnline
and Reference Manual
819-1178-xx PDFOnline
Guide
819-1179-xx PDFOnline
Compliance Guide
819-1647-xx PDFOnline
Setup Guide
816-7190-xx PrintedShipping kit
Sun Hardware Systems
Third-Party Web Sites
Sun is not responsible for the availability of third-party web sites mentioned in this
document. Sun does not endorse and is not responsible or liable for any content,
advertising, products, or other materials that are available on or through such sites
or resources. Sun will not be responsible or liable for any actual or alleged damage
or loss caused by or in connection with the use of or reliance on any such content,
goods, or services that are available on or through such sites or resources.
Sun Welcomes Your Comments
Sun is interested in improving its documentation and welcomes your comments and
suggestions. You can submit your comments by going to:
Prefacexxiii
http://www.sun.com/hwdocs/feedback
Please include the title and part number of your document with your feedback:
Sun Netra CP3240 Switch Installation Guide, part number 820-3251-13.
xxivSun Netra CP3240 Switch Installation Guide • September 2009
CHAPTER
1
Getting Started
This chapter provides information and procedures needed to install and make the
Sun Netra CP3240 switch operational. This chapter should be read before using the
board.
In addition to this chapter, refer to the following safety documentation:
■ Netra CP3X20 Switch Safety and Compliance Manual (820-3505)
■ Important Safety Information for Sun Hardware Systems (816-7190)
Caution – When the system is plugged in, energy hazards are present on the
midplane. Do not reach into the enclosure.
Caution – Static electricity can damage electronic components. Wear a wrist strap
grounded through one of the system’s ESD ground jacks when removing and
replacing hot-swappable components.
This chapter contains the following topics:
■ Section 1.1, “System Requirements” on page 1-2
■ Section 1.2, “Unpacking” on page 1-4
■ Section 1.3, “Handling Switches” on page 1-4
■ Section 1.4, “Connectors” on page 1-5
■ Section 1.5, “Jumper Options” on page 1-5
■ Section 1.6, “Switch and RTM Faceplates” on page 1-6
■ Section 1.7, “RTM Airflows” on page 1-7
■ Section 1.8, “Removing and Installing Switches” on page 1-10
■ Section 1.9, “Switch LEDs” on page 1-15
1-1
1.1System Requirements
The following sections briefly describe the minimum system requirements and the
configurable features. Links are provided to other chapters and appendices
containing more detailed information.
1.1.1Connectivity
The switch can work in any AdvancedTCA shelf. It is mainly designed to be used as
hub blade, however, it can also operate a node or a full-mesh blade.
1.1.1.1Hub Connectivity
In AdvancedTCA. the hub board defines the system, so it is important that it be put
in the correct slot. In all shelves there are two slots specifically designated for hub
boards, logical slots 1 and 2. If the slots are not labeled, these slots are easily
identifiable because they have a larger number of Zone 2 connectors than the other
slots (the only slots with all five Zone 2 connectors).
Base is always routed in a dual star. This means every node slot has a Base channel
routed to each of the hub slots. Independent of how the Fabric is used, a hub board
is always needed for Base. If full mesh node boards are used in a full mesh shelf, a
hub board is not needed for Fabric.
1.1.2Electrical and Environmental
The switch has the following power requirements:
TABLE 1-1Power Requirements
StatePower in Watts (W)
Idle, no AMCs45W
All ports linked with RTM, no AMCs72W
All ports linked with RTM, heavy traffic, no AMCs78W
Designed max power with 15W RTM, and 100W
shared for all AMC sites
1-2Sun Netra CP3240 Switch Installation Guide • September 2009
200W
The 48VDC has a tolerance of 0VDC to 75VDC without damage. The switch will
operate when 48VDC is 36VDC to 72VDC, inclusive.
Caution – Any input voltage outside the range of 0VDC to 75VDC can damage the
switch.
The switch requires air flow. A minimum of 100 LFM should be kept on the board at
all times. If an RTM is being used, active RTM cooling is not required.
Caution – None of the integrated chips junction temperature should exceed 125˚C.
The switch requires air flow to meet this requirement. Testing should be done in
your shelf to find the quantity of air flow needed. 100 LFM is the recommended
minimum quantity of air flow.
The switch might contain materials that require regulation upon disposal. Please
dispose of this product in accordance with local rules and regulations. For disposal
or recycling information, please contact your local authorities or the Electronic
Industries Alliance at http://www.eiae.org/.
Chapter 1 Getting Started1-3
1.2Unpacking
Check the shipping carton for damage. If the shipping carton or contents are
damaged, notify the carrier and Sun. Retain the shipping carton and packing
material for inspection by the carrier. Obtain authorization before returning any
product to Sun. Refer to the Netra CP3240 Switch Getting Started Guide (820-3254) for
return instructions.
Caution – This board must be protected from static discharge and physical shock.
Never remove any of the socketed parts except at a static-free workstation. Use the
anti-static bag shipped with the product to handle the board. Wear a wrist strap
grounded through one of the system's ESD ground jacks when installing or servicing
system components.
1.3Handling Switches
Caution – The system is sensitive to static electricity. To prevent damage to the
assembly, always connect an antistatic wrist strap between you and the system.
Caution – Do not flex the switches; the surface-mounted components can break if
the switch is bent.
To minimize the amount of switch flexing, observe the following precautions:
■ When removing a switch from an electrostatic discharge bag, keep it vertical until
you place the switch on the electrostatic discharge mat.
■ Do not place a switch on a hard surface. Use a cushioned antistatic mat. The
switch connectors and components have very thin pins that bend easily.
■ Be careful of small parts located on the component side of a switch.
■ Do not use an oscilloscope probe on the components. The soldered pins are easily
damaged or shorted by the probe point.
■ Transport a switch in an antistatic bag.
1-4Sun Netra CP3240 Switch Installation Guide • September 2009
Caution – The heat sinks on a switch can be damaged by incorrect handling. Do not
touch the heat sinks while installing or removing a switch. Hold a switch only by the
edges. If a heat sink is loose or broken, obtain a replacement switch.
Caution – The heat sinks on a switch can be damaged by improper packaging.
When storing or shipping a switch, ensure that the heat sinks have sufficient
protection.
1.4Connectors
The switch includes several connectors to interface to application-specific devices.
Refer to the Chapter B for complete connector descriptions and pin outs.
1.5Jumper Options
The switch provides several jumper configuration options for features. Location
figures and descriptions are provided in Chapter 3.
Chapter 1 Getting Started1-5
1.6Switch and RTM Faceplates
The following shows the Sun Netra CP3240 switch and the copper RTM
(XCP3240H-RTM-CU-Z) and the fiber optic RTM (XCP3240H-RTM-OP-Z).
FIGURE 1-1 Switch and RTM Faceplates
Copper RTMFiber Optic
Switch
Faceplate
RTM
1-6Sun Netra CP3240 Switch Installation Guide • September 2009
1.6.1Base 10/100/1000 Uplink Ports (RJ-45)
There are two front panel Base 10/100/1000Base-T Ethernet uplink ports on the
faceplate. They are ports number 18 and 19 on the Base network.
1.6.210/100 Management Port (RJ-45)
There is a 10/100 management Ethernet port on the faceplate. This port can be used
to manage the Base and Fabric. This port and the 10/100 management port on the
RTM can be used at the same time.
1.6.3Serial Management Port (RJ-45)
There is a RS-232 serial management port on the faceplate. This port can be used to
manage the Base and Fabric. See Section 4.4, “Serial Select” on page 4-8 for
information about how to switch between managing the Base and the Fabric.
1.7RTM Airflows
1.7.1RTM Airflow Requirements
The following chart provides the RTM airflow requirements. No alarms are triggered
at any of the listed temperatures for the given airflow.
TABLE 1-2Airflow Requirements
Wattsm3/minCFM
Power Level
250.070.070.140.142.52.555
25˚C28˚C40˚C55˚C25˚C28˚C40˚C55˚C
Chapter 1 Getting Started1-7
1.7.2RTM Pressure Drop Versus Airflows (Impedence
Curves)
The following graphs plot the impedance curves for the RTMs, per PICMG 3.0
requirements.
FIGURE 1-2 Netra CP3240H-RTM-CU Airflow (Pa)
FIGURE 1-3 Netra CP3240H-RTM-CU Airflow (InchesH
1-8Sun Netra CP3240 Switch Installation Guide • September 2009
This section describes how to remove and install switches.
If you are hot-swapping the rear transition module for a switch, you must remove
the switch from the front of the system before removing the rear transition module
from the rear of the system. Do not remove the rear transition module without first
removing the accompanying switch from the front.
1.8.1Removing a Switch Set
Following are the instructions for removing a switch and, if necessary, the
accompanying rear transition module.
1-10Sun Netra CP3240 Switch Installation Guide • September 2009
1.8.1.1Removing a Switch From the Front of the Server
1. Move the front cable management bracket to the lower position (FIGURE 1-6).
FIGURE 1-6 Front Cable Management Bracket in Lower Position
2. Disengage the injector/ejector mechanisms at the top and bottom of the board to
notify software that the board is about to be removed. Wait for the Hot-Swap LED
to light.
FIGURE 1-7 shows the proper way to remove or insert a hot-swappable board into a
system.
Chapter 1 Getting Started1-11
FIGURE 1-7 Injector/Ejector on the Switch (Open Position)
injector/ejector mechanism
3. Disconnect all cables connected to the switch.
4. Loosen the two board retention screws that fasten the board to the enclosure.
5. Open the ejectors fully, rotating the handles outward until the board disengages
from the midplane.
6. Slide the board evenly out of the enclosure.
7. Determine if you are going to replace the rear transition module.
■ If you are going to replace the rear transition module, go to “Removing a Rear
Transition Module for a Switch” on page 1-13.
■ If you are not going to replace the rear transition module, you must install a
replacement switch or a filler panel to maintain the enclosures shielding and
cooling performance. Refer to Section 1.8.2.2, “Installing a Switch” on page 1-14.
1-12Sun Netra CP3240 Switch Installation Guide • September 2009
Note – As long as the switch is removed from the slot, and the system is running
with only the single remaining switch, you do not have redundancy for that
component. Both switches must be installed and running to have redundancy for
that component.
1.8.1.2Removing a Rear Transition Module for a Switch
1. Remove the switch from the front of the server, if you have not already done so.
Refer to “Removing a Switch From the Front of the Server” on page 1-11 for those
instructions. Do not remove the rear transition module until you have first removed
the switch.
Note – Opening the handle on the RTM shuts down both the RTM and the switch.
2. Disconnect any cables connected to the rear transition module for the switch.
3. Loosen any retention screws that fasten the module to the enclosure.
4. Open the ejectors fully, rotating the handles outward until the board disengages
from the midplane.
5. Slide the module evenly out of the enclosure.
6. Install a replacement rear transition module or a filler panel to maintain the
enclosure’s shielding and cooling performance.
Refer to Section 1.8.2.1, “Installing the Rear Transition Module for a Switch” on
page 1-14.
Caution – Failure to cover all rear panel slots with rear transition modules or filler
panels can negatively impact the cooling of the system.
Chapter 1 Getting Started1-13
1.8.2Installing a Switch Set
If you have a rear-access server, you must install the rear transition module before
installing the front switch.
■ If you want to install the rear transition module, go to Section 1.8.2.1, “Installing
the Rear Transition Module for a Switch” on page 1-14.
■ If you want to install a switch, go to Section 1.8.2.2, “Installing a Switch” on
page 1-14.
1.8.2.1Installing the Rear Transition Module for a Switch
1. Choose an appropriate slot for the rear transition module.
Rear transition modules must be installed inline behind the accompanying front
board. For example, if the accompanying front board is installed in slot 7, its rear
transition module must be installed at the back of the system in slot 7.
2. Remove the filler panel, if necessary.
3. Prepare the rear transition module by opening its injector/ejector handles.
4. Carefully align the edges of the board with the guides in the slot.
Look into the enclosure to verify correct alignment of the rails in the guides.
Caution – Do not force the module into the slot. If it does not fit properly, check to
ensure that you have the correct matching RTM for the switch.
5. Keeping the board aligned in the guides, slide the board in until the
injector/ejector mechanisms engage the retention bar.
6. Simultaneously push in the board and rotate the injector/ejector handles to their
closed positions (rotate inward) to seat the midplane connectors.
7. Tighten the board retention screws to anchor the board in the shelf.
8. Connect the cables to the rear transition module.
1.8.2.2Installing a Switch
1. Locate the switch slots.
The switches can be inserted only in slot 7 or slot 8 in the Netra CT 900 server.
2. Move the front cable management bracket to the lower position (
3. Remove the filler panel, if necessary.
1-14Sun Netra CP3240 Switch Installation Guide • September 2009
FIGURE 1-6).
4. Prepare the switch by opening its injector/ejector handles (FIGURE 1-7).
5. Carefully align the edges of the board with the guides in the slot.
Look into the enclosure to verify correct alignment of the rails in the guides.
6. Keeping the board aligned in the guides, slide the board in until the
injector/ejector mechanisms engage the retention bar.
7. Simultaneously push in the board and rotate the injector/ejector handles to their
closed positions (rotate inward) to seat the midplane connectors.
If system power is on, the Hot-Swap LED should light up. The Hot-Swap LED
should blink for several seconds, and then go off. If the Hot-Swap LED does not go
off after several seconds, push harder on the injector/ejector handles to verify that
they are pushed in all the way.
8. Tighten the board retention screws to anchor the board in the shelf.
9. Connect the cables to the switch.
1.9Switch LEDs
The following sections give the status information for all of the LEDs on the switch.
■ “ATCA Board Status LEDs” on page 1-15
■ “Hot-Swap LED” on page 1-16
1.9.1ATCA Board Status LEDs
TABLE 1-3 describes the three LEDs defined by ATCA to monitor board status.
TABLE 1-3ATCA Board Status LEDs
LEDColorNormal OperationDescription
OOSRedOffOut of service. This LED lights on a critical switch
error, such that the board should be removed.
OKGreenOnThis LED is lit when the switch is booted and
switching
AAmberOffMinor Error/User Defined. This LED can be defined
by the user via software applicatons.
Chapter 1 Getting Started1-15
Note that both the OOS and MINOR LEDs are lit when the board is powered but not
booted. This includes all Hot-Swap states M1 through M3. Refer to “Hot-Swap LED”
on page 1-16.
1.9.2Hot-Swap LED
This blue LED communicates the Hot-Swap status of the switch. TABLE 1-4 shows the
different states of the Hot-Swap LED.
TABLE 1-4Hot-Swap LED States
OrderVisible StateStateDescription
1SolidM1 FRU InactiveThe Intelligent Platform
2Blinking (from solid)M2 Activation
3OffM3-M4 ActiveThe IPMI microcontroller has
4Blinking (from off)M5-M6
Back to 1
Request
Deactivation
Request
Management Interface (IPMI)
microcontroller is booted, but the
payload is not. The bottom latch is
not fully closed.
The IPMI microcontroller has
requested permission to boot the
payload from the shelf
management controller.
received permission to boot the
payload, and has done so. This
should be the state under normal
operation.
The IPMI microcontroller has
requested permission to shut
down the payload. Opening the
bottom latch activates this state.
Note – A board should be hot-swapped only when the LED is solid blue.
1-16Sun Netra CP3240 Switch Installation Guide • September 2009
CHAPTER
2
Overview
This chapter introduces the key features of the switch and RTMs. This chapter
includes a product definition, a list of product features, and functional block
diagrams with brief descriptions. This chapter can be used to compare the features
of the switch against the needs of a specific application.
This chapter contains the following topics:
■ Section 2.1, “Features” on page 2-2
■ Section 2.2, “Switch Components” on page 2-6
■ Section 2.3, “Protocols, RFCs, and MIBs Support” on page 2-8
■ Section 2.4, “Functional Diagrams and Port Maps” on page 2-17
2-1
2.1Features
Part of Sun’s ATCA platform, the Sun Netra CP3240 switch complies with PICMG
3.0 R2.0 ECN002 and PCIMG 3.1 Option 1 and Option 9. The Sun Netra CP3240
switch implements two separate switched networks on a single PCB. By separating
the Base and Fabric networks, the Sun Netra CP3240 switch provides a separate
control plane and data plane. It provides 1Gigabyte Ethernet switching on the 3.0
Base Fabric, and the 3.1 Expansion Fabric provides 1Gigabyte/10Gigabyte Ethernet
switching. Both of these networks are fully managed with the robust FASTPATH
management suite. Both networks support Layer 2 switching as well as Layer 3
routing.
The Sun Netra CP3240 switch can host three AdvancedMC mid-size modules. A
variety of different AMC types can be used, including AMC.1, AMC.2, and AMC.3.
Netra CP3240 switch supports connectivity to three AMCs and to RTMs with
multiple 10Gigabyte-Ethernet links.
The XCP3240H-RTM-CU-Z (copper) and XCP3240H-RTM-OP-Z (fiber optic) are
Sun’s RTMs that are paired with the Sun Netra CP3240 switch.
The following sections briefly outline the features of the Sun Netra CP3240 switch.
2.1.1General
■ PICMG 3.0 AdvancedTCA form factor
■ PICMG 3.0 R2.0 ECN002 compliant
■ PICMG 3.0 compliant 10/100/1000Base-T Ethernet Base Fabric hub board
The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56503 for Base
Ethernet switching/routing. This chip is an Ethernet switch with 24 1Gigabyte ports
and 3 10Gigabyte ports. It provides non-blocking, wire-speed switching and routing
on all ports under 100% load. The BCM56503 features many advanced features
presented to the user via the FASTPATH software. See the Section 4.10, “FASTPATH”
The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56800 for Fabric
Ethernet switching/routing. This chip is an Ethernet switch with 20 10Gigabyte
ports. It provides non-blocking, wire-speed switching and routing on all ports under
100% load. The BCM56800 features many advanced features presented to the user
via the FASTPATH software. See Section 4.10, “FASTPATH” on page 4-18 for more
information.
2.2.3Broadcom BCM5464R and BCM5461S
10/100/1000Base-T Ethernet PHY
Quad and single ports respectively, the Broadcom PHYs provide the physical
interfacing for 10/100/1000Base-T. They are low-power devices and provide features
such as jumbo frames support, auto-MDIX, and cable testing.
2-6Sun Netra CP3240 Switch Installation Guide • September 2009
2.2.4Freescale PowerQUICC II MPC8247
Communications Processor
The Freescale MPC8247 is a microprocessor designed for maximum flexibility. It
features a dual core architecture with a PPC G2 LE core and a RISC core controlling
the peripherals. Running at 400MHz with only 1 watt of power, the MPC8247
provides high performance with incredibly low power consumption. Paired with the
256MB PC100 SDRAM and 64MB flash, the CPU subsystem of the Sun Netra CP3240
switch is more than enough to handle the switching application and future
upgrades.
2.2.5Pigeon Point BMR-H8S-AMCc AdvancedTCA
IPMI Subsystem
The Sun Netra CP3240 switch uses the near-industry standard Pigeon Point Systems
BMR-H8S-AMCc for dedicated IPMI management. It controls all interaction between
the Sun Netra CP3240 switch and the shelf management controller. It fully supports
the PICMG 3.0 R2.0 ECN002 and the AMC.0 R2.0 specifications. The
BMC-H8S-AMCc provides all required management for the AMC modules, in
addition to the voltage monitoring, temperature monitoring, e-Keying, and other
services it provides for the Sun Netra CP3240 switch. Pigeon Point Systems is the
leading provider of IPMI firmware for AdvancedTCA, and the firmware has been
thoroughly tested to be fully compliant with the specification.
Chapter 2 Overview2-7
2.3Protocols, RFCs, and MIBs Support
The Sun Netra CP3240 switch features Level7 FastPath 2340 switching software,
version 4.4.4 and newer. This software provides layer 2 switching, quality of service,
IPv4 routing, IPv6 routing, and IP multicast.
2.3.1FASTPATH Switching
■ IEEE 802.1ab—Link Level Discovery
■ IEEE 802.1D—Spanning Tree
■ IEEE 802.1p—Ethernet Priority with User Provisioning and Mapping
■ Draft-ietf-secsh-publickeyfile-03—SECSH Public Key File format
■ Draft-ietf-secsh-dh-group-exchange-04—Diffie-Hellman Group Exchange for the
SSH Transport Layer Protocol
■ HTML 4.0 Specification—December, 1997
■ Java and Java Script 1.3
2.3.6.3Additional Management Features
■ Industry Standard CLI with the following features:
■ Scripting capability
■ Command completion
■ Context sensitive help
■ User password encryption
■ Multi-session Telnet server
■ TACACS+
■ Dual firmware image support
2-16Sun Netra CP3240 Switch Installation Guide • September 2009
2.4Functional Diagrams and Port Maps
2.4.1Switch
FIGURE 2-1 Switch Functional Block Diagram
Chapter 2 Overview2-17
TABLE 2-1Switch External Ports
Front PanelPort TypeFiber Optic AMCModuleLogical PortComments
RJ-45 #1Serial RS-232RS-232RJ-45
RJ-45 #2Management10/100 Base TRJ-45
RJ-45 #3Base 1GBE10/100/1000Base T RJ-45Base - 18Standard I/F is copper
RJ-45 #4Base 1GBE10/100/1000Base T RJ-45Base - 19Standard I/F is copper
2.4.2Rear Transition Modules (RTM)s
The Sun Netra CP3240 switch supports RTMs through ATCA Zone 3 connectors (see
TABLE 2-2). The Sun Netra CP3240 switch was designed to support multiple RTM
designs.
4 Fabric 10Gigabyte, 2 Base 10Gigabyte, 2 Base 1Gigabyte, AMC I/O, and
management ports are run to the RTM. Several voltage rails are supplied, and all of
the signals needed to design an ECN002 compliant RTM are present.
TABLE 2-2Zone 3 RTM Ports
Base Ports1G Infrastructure10G Infrastructure
Base Port 201G SGMIINone
Base Port 211G SGMIINone
Base Port 26None10G XAUI
Base Port 27None10G XAUI
Fabirc Port 171G SGMII10G XAUI
Fabirc Port 181G SGMII10G XAUI
Fabirc Port 191G SGMII10G XAUI
Fabirc Port 201G SGMII10G XAUI
The Sun Netra CP3240 switch is paired with either a copper or fiber optic RTM. The
following sections describe each RTM.
2-18Sun Netra CP3240 Switch Installation Guide • September 2009
2.4.2.1Copper RTM
The XCP3240H-RTM-CU-Z is the Sun Netra CP3240 switch’s copper RTM pairing.
This RTM supports all the ports the Sun Netra CP3240 switch provides using
10GBase-CX4 for the 10Gigabyte and 10/100/1000Base-T for the 1Gigabyte ports.
FIGURE 2-2 Copper RTM Functional Block Diagram
Chapter 2 Overview2-19
TABLE 2-3Copper RTM External Ports
Port TypeRTMModuleLogical PortComments
Base 10GBEXAUICX4Base - 27
Base 10GBEXAUICX4Base - 26
Base 1GBE10/100/1000Base TRJ-45Base - 20Standard I/F is copper
Base 1GBE10/100/1000Base TRJ-45Base - 21Standard I/F is copper
Management10/100 Base TRJ-45
Serial RS-232RS-232RJ-45
Fabric 10GBEXAUICX4Fabric - 17
Fabric 10GBEXAUICX4Fabric - 18
Fabric 10GBEXAUICX4Fabric - 19
Fabric 10GBEXAUICX4Fabric - 20
TABLE 2-4Copper RTM Port Restrictions
Port1G Infrastructure1G Port10G Infrastructure10G Port
Base Port 20SERDES1000Base-T RJ45 NoneNone
Base Port 21SERDES1000Base-T RJ45 NoneNone
Base Port 26NoneNoneXAUI10GBase-CX4
Base Port 27NoneNoneXAUI10GBase-CX4
Fabirc Port 17NoneNoneXAUI10GBase-CX4
Fabirc Port 18NoneNoneXAUI10GBase-CX4
Fabirc Port 19NoneNoneXAUI10GBase-CX4
Fabirc Port 19NoneNoneXAUI10GBase-CX4
Fabirc Port 20NoneNoneXAUI10GBase-CX4
2-20Sun Netra CP3240 Switch Installation Guide • September 2009
2.4.2.2Fiber Optic RTM
The XCP3240H-RTM-OP-Z is the Sun Netra CP3240 switch’s fiber optic RTM pairing.
This RTM supports all the ports the Netra CP3240 switch provides using 10GBase-X
for the 10Gigabyte and 1000Base-X for the 1Gigabyte ports.
Port TypeFiber Optic RTMModuleLogical PortComments
Management10/100 Base TRJ-45Not a module, 10/100Base T
Base 10GBE10GBase-XSFP+Base - 27Cannot Support 1G Operation
Base 10GBE10GBase-XSFP+Base - 26Cannot Support 1G Operation
Base 1GBE1000Base-XSFPBase - 20Can Support 1G SFP Copper
Base 1GBE1000Base-XSFPBase - 21Can Support 1G SFP Copper
Serial RS-23210GBase-XSFP+Fabric - 17Can Support 1G SFP Copper
Fabric 10GBE10GBase-XSFP+Fabric - 18Can Support 1G SFP Copper
Fabric 10GBE10GBase-XSFP+Fabric - 19Can Support 1G SFP Copper
Fabric 10GBE10GBase-XSFP+Fabric - 20Can Support 1G SFP Copper
Serial RS-232Serial RS232RJ-45Not a module, RS232 Serial
Note – For enabling 1G operation of Fabric ports 17-20, see Section 4.12, “Fiber
Optic RTM Configuration” on page 4-25.
TABLE 2-6Fiber Optic RTM Port Restrictions
Port1G Infrastructure1G Port10G Infrastructure10G Port
Base Port 20SERDESSFP LX SXNoneNone
Base Port 21SERDESSFP LX SXNoneNone
Base Port 26NoneNoneXAUISFP+ LX SX LRM
Base Port 27NoneNoneXAUISFP+ LX SX LRM
Fabric Port 17SERDESSFP LX SXXAUISFP+ LX SX LRM
Fabric Port 18SERDESSFP LX SXXAUISFP+ LX SX LRM
Fabric Port 19SERDESSFP LX SXXAUISFP+ LX SX LRM
Fabric Port 20SERDESSFP LX SXXAUISFP+ LX SX LRM
2-22Sun Netra CP3240 Switch Installation Guide • September 2009
2.4.3Base Fabric Switch Subsystem
FIGURE 2-4 Base Fabric Switch Subsystem
PICMG 3.0 AdvancedTCA defines 10/100/1000Base-T Ethernet over the Base Fabric,
called “Base” in this guide. The Base is designed to be the control plane for the shelf.
Scaling from 10Mbps to 1000Mbps, the Base interface can accommodate a wide
variety of node boards. The Base interface on the Sun Netra CP3240 switch is based
around three main components: the Broadcom BCM56503, the Broadcom BCM5464x,
and the Freescale MPC8247.
Chapter 2 Overview2-23
2.4.4Expansion Fabric Switch Subsystem
FIGURE 2-5 Expansion Fabric Switch Subsystem
PICMG 3.0 AdvancedTCA provides an agnostic mesh on the backplane called the
Expansion Fabric. This interface is the data plane in the shelf. The Expansion Fabric
is called “Fabric” in this guide. This fabric can be several different technologies
defined by AdvancedTCA sub-specifications. The Sun Netra CP3240 switch is
designed to comply with PICMG 3.1 Ethernet/Fibre Channel for AdvancedTCA Systems,
options 1 and option 9. That means the Sun Netra CP3240 switch provides a single
1/10Gigabyte port to each node board.
The Fabric uses 10GBase-BX4 Ethernet to provide connectivity between boards
though the backplane. The Fabric can scale down to 1Gigabyte and work with
1000Base-BX boards. The Fabric subsystem is based around two main components:
the Broadcom BCM56800 and the Freescale MPC8247.
2-24Sun Netra CP3240 Switch Installation Guide • September 2009
2.4.5AdvancedMC Sites
PICMG AdvancedMC AMC.0 defines hot-swappable daughter cards to be used in
PICMG architectures. The Sun Netra CP3240 switch supports three AMC sites so
that the functionality of the Sun Netra CP3240 switch can be expanded. AMC.2
(Ethernet) AMCs are supported with direct connections to the Base and Fabric
subsystems. Other types of AMCs, such as AMC.1 (PCI-Express) or AMC.3
(SAS/SATA), can be used as well. Nearly any type of AMC can be supported,
because the Sun Netra CP3240 switch connects the AMCs sites directly together on
certain ports.
Chapter 2 Overview2-25
2.4.5.1AMC Port Maps
FIGURE 2-6 AMC Port Map Diagram
2-26Sun Netra CP3240 Switch Installation Guide • September 2009
TABLE 2-7AMC Ethernet Port Availability
SlotLocation1G Infrastructure10G Infrastructure
AMC 1Top1G Base SGMIINone
AMC 2Middle1G Base SGMIIXAUI from BCM56800 Fabric Port 16
AMC 3Bottom1G Base SGMIINone
TABLE 2-8AMC Site 1: AMC.0 Mid-size, AMC.2 Type E1, AMC.2 Type 5 (Optional),
LED Module Support
PortTypeConnected
0GBBase (3.0) switch
1AnyNot connected
2AnyPort 3 of AMC #3, usually used for SAS/SATA
3AnyPort 3 of AMC #2, usually used for SAS/SATA
4AnyPorts 4-7 of AMC #2, usually used for PCIE
5
6
7
8Not connected
9
10
11
12Custom LED board support
13Not connected
14Not connected
15Not connected
17AnyRTM for future use
18AnyRTM for future use
19AnyRTM for future use
20AnyRTM for future use
TCLK1 CLK INFPGA and Zarlink
Chapter 2 Overview2-27
TABLE 2-8AMC Site 1: AMC.0 Mid-size, AMC.2 Type E1, AMC.2 Type 5 (Optional),
LED Module Support (Continued)
TCLK2 CLK OUTFPGA and Zarlink
TCLK3 CLK INFPGA and Zarlink
TCLK4 CLK OUTFPGA and Zarlink
FCLK100MHz100MHz
TABLE 2-9AMC Site #2 AMC.0 Mid-Size, AMC.2 Type E1, AMC.2 Type 5, Double
module support, LED Module Support
PortTypeConnected
0GbEBase (3.0) switch
1AnyPort 1 of AMC #3, usually used for GbE
2AnyPort 2 of AMC #3, usually used for SAS/SATA
3AnyPort 3 of AMC #1, usually used for SAS/SATA
4AnyPorts 4-7 of AMC #1, usually used for PCIE
5
6
7
810GbEFabric (3.1) switch.
9
10
11
12custom LED board support
13Not connected
14Not connected
15Not connected
17AnyRTM for future use
18AnyRTM for future use
19AnyRTM for future use
20AnyRTM for future use
TCLK1CLK INFPGA & Zarlink
TCLK2CLK OUTFPGA & Zarlink
TCLK3CLK INFPGA & Zarlink
TCLK4CLK OUTFPGA & Zarlink
FCLKPCIE CLK100MHz
2-28Sun Netra CP3240 Switch Installation Guide • September 2009
TABLE 2-10 AMC Site 3: AMC.0 Mid-size, AMC.2 Type E1, Master Clock Generator
Support, LED Support
PortTypeConnected
0GBBase (3.0) switch
1AnyPort 1 of AMC #2, usually used for GB
2AnyPort 2 of AMC #2, usually used for SAS/SATA
3AnyPort 2 of AMC #1, usually used for SAS/SATA
4Not connected
5Not connected
6Not connected
7Not connected
8Not connected
9Not connected
10Not connected
11Not connected
12custom LED board support
13Not connected
14Not connected
15AnyUpdate channel, usually used for clock sync
17AnyRTM for future use
18AnyRTM for future use
19AnyRTM for future use
20AnyRTM for future use
TCLK1CLK INFPGA and Zarlink
TCLK2CLK OUTFPGA and Zarlink
TCLK3CLK INFPGA and Zarlink
TCLK4CLK OUTFPGA and Zarlink
FCLK100MHz100MHz
Chapter 2 Overview2-29
2.4.5.2AMC Module Support by Site
TABLE 2-11 AMC Modules by Site
AMC Module
Site 1 (top most
when the board is
vertical)Site 2 (middle)
Site 3 (bottom most
site when the board
is ver tical)
AMC10G-XFPNo*YesNo
AMC10G-CX4No*YesNo
AMCLED001YesYesYes
PCI-Express AMCYesYesNo
1G Common Options
Ye sYe sYe s
Ethernet AMC
10G Fat Pipe Ethernet AMCNo*YesNo
2.4.5.3AMC Port Restrictions for RTMs
The following tables list AMC port restrictions for RTMs
TABLE 2-12 AMC Port Restrictions for Copper RTMs
1G Infrastructure1G Base Ports10G Infrastructure10G Fabric Ports
Front panel Base Port 18J13 3rdBCM5464R Copper PHY
Front panel Base Port 19J13 BottomBCM5464R Copper PHY
TABLE 2-13 AMC Port Restrictions for Fiber Optic RTMs
1G Infrastructure1G Base Ports10G Infrastructure10G Fabric Ports
SERDESSFP - LX SXSFP+ LRM
SERDESSFP - LX SXNone
SERDESSFP - LX SXNone
10/100/1000 BaseT/RJ-45
10/100/1000 BaseT/RJ-45
2-30Sun Netra CP3240 Switch Installation Guide • September 2009
CHAPTER
3
Configuring Jumper Settings
This chapter describes jumper settings for configuring the switch. Other
configuration options are software controlled. Software configuration options are
described in Chapter 4.
This chapter contains the following topics:
■ Section 3.1, “Jumper Settings” on page 3-2
■ Section 3.2, “Jumper Locations” on page 3-7
3-1
3.1Jumper Settings
The following jumper settings are available for configuring the switch. Each of
jumper settings is described in the subsections.
TABLE 3-1Switch Configuration Jumper Settings
JumperDefaultPurpose
P4L2Cross Connect Control
P6(1-2)OFFFabric Zero Reset Configuration Word
P6(3-4)OFFBase Zero Reset Configuration Word
P8(1-2)OFFBase Write Protect Disable
P8(3-4)OFFFabric Write Protect Disable
P9OFFIPMC Firmware Write
P10(1-2)OFFIPMC Disable
P10(3-4)OFFIPMC FWE
P11(1-2)OFFForced Board Enable
P11(3-4)OFFIPMC Board Reset Disable
P12OFFEMI Ground to Logic Ground
P13OFFSerial Direction
3.1.1P4 Cross-Connect Control
This jumper is used to control ShMC cross connect. ShMC cross connect is the
ability to connect to two ShMCs at 10/100 each rather than use a single 10/100/1000
for a single ShMC. The first ShMC port is number 1 on the Base. The second ShMC
port is number 17 on the Base.
TABLE 3-2P4 Cross-Connect Jumper Settings
P4DefaultFunction
OFFSoftware control of cross connect.
1-2DefaultForce cross connect enabled, base channel 1 is two 10/100BaseTX
ports.
3-4No cross connect; base channel 1 is 10/100/1000Base-T.
3-2Sun Netra CP3240 Switch Installation Guide • September 2009
3.1.2P6(1-2) Fabric Zero Reset Configuration Word
This jumper is used to configure the Fabric Gigabyte CPU to use the default Reset
Configuration Word, or use the one in the flash.
TABLE 3-3P6 (1-2) Fabric Zero Reset Jumper Settings
P6(1-2)DefaultFunction
ONUse default Reset Configuration Word (all zeros)
OFFDefaultUse Reset Configuration Word in flash
3.1.3P6(3-4) Base Zero Reset Configuration Word
This jumper is used to configure the Base CPU to use the default Reset
Configuration Word, or use the one in the flash.
TABLE 3-4P6 (3-4) Base Zero Reset Jumper Settings
P6(3-4)DefaultFunction
ONUse default Reset Configuration Word (all zeros)
OFFDefaultUse Reset Configuration Word in flash
3.1.4P8(1-2) Base Write Protect
This jumper configures write protection on the Base reset configuration word.
TABLE 3-5P8 (1-2) Base Write-Protect Jumper Settings
P8(1-2)DefaultFunction
ONDisable the write protection of the Base reset configuration word
OFFDefaultWrite protect the Base reset configuration word
Chapter 3 Configuring Jumper Settings3-3
3.1.5P8(3-4) Fabric Write Protect
This jumper configures write protection on the Fabric reset configuration word.
This jumper controls whether the switch is forced on when it is powered up, or if the
IPMI subsystem controls the power up of the switch. Forcing power to the board is
not enough for the board to boot. The board could still be held in reset. Configre P11
(3-4) jumper setting as well to bring the board out of reset.
ONIPMI subsystem cannot reset the switch. Use this to run without a
ShMC.
OFFDefaultIPMI subsystem can reset the switch and hold it in reset.
Chapter 3 Configuring Jumper Settings3-5
3.1.11P12(1-2) and P12 (3-4) EMI Ground to Logic
Ground
The switch and the entire AdvancedTCA shelf separate the ground of the chassis
itself from digital ground for EMI protection. This jumper configures the two
grounds.
TABLE 3-12 P12 (1-2) and P12 (3-4) Grounding Jumper Settings
P12
(1-2)
(3-4)DefaultFunction
OFFDefaultSeparate EMI ground and logic ground.
1-2Connect EMI ground to logic ground.
3-4Connect EMI ground to logic ground.
3.1.12P13 Serial Direction
The front panel serial port and the RTM serial port are mutually exclusive; only one
can be used at a time. The serial port can be forced to the front or the RTM, or it can
be controlled by the software.
TABLE 3-13 P13 Serial Direction Jumper Settings
P13DefaultFunction
OFFDefaultSoftware control of serial direction.
1-2Front serial port active; RTM serial port disabled.
3-4Front serial port disabled; RTM serial port active.
3-6Sun Netra CP3240 Switch Installation Guide • September 2009
3.2Jumper Locations
FIGURE 3-1 shows the locations of the jumper settings.
FIGURE 3-1 Switch Jumper Locations
Chapter 3 Configuring Jumper Settings3-7
3-8Sun Netra CP3240 Switch Installation Guide • September 2009
CHAPTER
4
Configuring Switch Software
This chapter describes how to configure the switch software.
The switch incorporates four main software components:
■ IPMI firmware, which runs on a separate microcontroller.
■ uBoot loads the boot for the system. It can be compared to a BIOS on a node
board.
■ Operating system (OS), which uses Monta Vista 3.1 Pro, based on the 2.4.20 Linux
kernel.
■ LVL7 Systems’ FASTPATH software, which provides all of the management and
control features of the switch.
The IPMI firmware, uBoot, and OS are covered in this chapter. FASTPATH is
described, however, for detailed information and command syntax, refer to the NetraCP3240 Switch Command Reference Manual (820-3253).
This chapter contains the following topics:
■ Section 4.1, “IPMI Firmware Sensors” on page 4-2
■ Section 4.2, “uBoot” on page 4-5
■ Section 4.3, “Linux” on page 4-7
■ Section 4.4, “Serial Select” on page 4-8
■ Section 4.5, “Serial Location” on page 4-9
■ Section 4.6, “Boot Sequence” on page 4-10
■ Section 4.7, “Primary and Backup Flash” on page 4-13
■ Section 4.8, “Dual Firmware Images” on page 4-13
■ Section 4.9, “Network Boot” on page 4-14
■ Section 4.10, “FASTPATH” on page 4-18
■ Section 4.11, “Firmware Updates” on page 4-24
■ Section 4.12, “Fiber Optic RTM Configuration” on page 4-25
4-1
4.1IPMI Firmware Sensors
This section describes the IPMI firmware sensors.
4.1.1State Sensors
TABLE 4-1IPMI State Sensors
SensorNameTypeDescriptionDiscrete States Returned
FRU 0
TABLE 4-2
0
HOT_SWAP
1RTM hotswapStatehotswap discrete
2AMC 0
hotswap
3AMC 1
hotswap
4AMC 2
hotswap
5IPMB physicalStateIPMB Link sensorDefined in ATCA spec.
6BMC WatchdogStateWatchdog timerDiscrete, Watchdog 2 (per the
11RTM PresenceStateRear Module
12Base Early*StateBase firmware
13Base Full*StateBase firmware
14Base Good*StateBase firmware
15Fabric Early*StateFabric firmware
16Fabric Full*StateFabric firmware
17Fabric Good*StateFabric firmware
Statehotswap discrete
state
state
Statehotswap discrete
state
Statehotswap discrete
state
Statehotswap discrete
state
Present Signal
signal
signal
signal
signal
signal
signal
M state, per ATCA spec
M state, per ATCA spec
M state, per ATCA and AMC
spec
M state, per ATCA and AMC
spec
M state, per ATCA and AMC
spec
IPMI spec)
State 01h = Device absent, 02h
= Device present
Asserted indicates Uboot
prompt
Asserted indicates Linux has
booted
Asserted indicates FASTPATH
has loaded
Asserted indicates Uboot
prompt
Asserted indicates Linux has
booted
Asserted indicates FASTPATH
has loaded
4-2Sun Netra CP3240 Switch Installation Guide • September 2009
*These sensors represent the software state of the switch.
The following tables lists the logic that drives these sensors.
TABLE 4-3IPMI Sensor Logic
StateTimeoutFailover ConditionEarlyFullGood
Initial power-on state5 secUboot corrupted or
erased
uBoot initial state5 secUboot finds no
image
uBoot consolen/aFailover timer stops100
Linux booted90 secSoft-off now occurs010
Linux boot menun/aFailover timer stops
(user interruption)
FASTPATH booted90 secTimer stops if this
state is reached in
time
Critical code updaten/aAn update is in
progress; the failover
time stops
001
000
110
011
111
Chapter 4 Configuring Switch Software4-3
4.1.2Threshold Sensors
TABLE 4-4IPMI Thresold Sensors
SensorNameTypeDescription Units
#MinorMajorCriticalMinorMajorCritical
7+12.0VThresh+12V
Volts11.01610.219.0213.4513.7213.77
Lower
Thresholds
Upper
Thresholds
main
power
8+3.3VThresh+3.3V line Volts3.1263.022.9143.5633.6543.805
9+2.5VThresh+2.5V line Volts2.3082.2022.1062.7012.8082.904
10+1.25VThresh+1.25V
Volts1.1561.0090.75461.55821.7541.9502
line
o
18Base
CPU
ThreshBase CPU
Temp
Cn/an/an/a607080
Temp
o
19RTM
temp
ThreshRTM
Temperatu
Cn/an/an/a556575
re
o
20Fabric
CPU
ThreshFabric
CPU Temp
Cn/an/an/a607080
Temp
21+1.5VThresh+1.5V line Volts1.40141.3031.10741.60721.70521.9012
22+1.8VThresh+1.8V line Volts1.6561.5091.3031.95022.1072.303
23+1.0VThresh+1.0V line Volts0.9020.7060.511.10741.3031.509
24+1.2VThresh+1.2V line Volts1.1071.0090.8041.3031.40141.607
25Site 1
Pwr
Cur
26Site 1
Pwr
27Site 1MPThreshAMC Slot
ThreshAMC Slot
current
sense
ThreshAMC Slot
+12V
Ampsn/an/an/a5.56.0236.517
Voltsn/an/an/a13.04813.4413.608
Voltsn/an/an/a3.4563.5143.6
Managem
ent Pwr
28Site 2
Pwr
Cur
29Site 2
Pwr
ThreshAMC Slot
current
sense
ThreshAMC Slot
+12V
Ampsn/an/an/a5.56.0236.517
Voltsn/an/an/a13.04813.4413.608
4-4Sun Netra CP3240 Switch Installation Guide • September 2009
TABLE 4-4IPMI Thresold Sensors (Continued)
SensorNameTypeDescription Units
30Site 2MPThreshAMC Slot
Managem
ent Pwr
31Site 3
Pwr
Cur
32Site 3
Pwr
33Site 3MPThreshAMC Slot
ThreshAMC Slot
current
sense
ThreshAMC Slot
+12V
Managem
ent Pwr
Voltsn/an/an/a3.4563.5143.6
Ampsn/an/an/a5.56.0236.517
Voltsn/an/an/a13.04813.4413.608
Voltsn/an/an/a3.4563.5143.6
4.2uBoot
uBoot is the boot loader. Much like a BIOS, it brings the system to a usable state for
the operating system (OS) to boot. It also performs a POST of the CPU subsystem. It
can be used as a recovery console if the firmware image becomes corrupt or a
firmware update fails. Several important environment variables are stored in uBoot,
only some of which should ever be changed, namely noekey and baudrate.
Lower
Thresholds
Upper
Thresholds
4.2.1uBoot Console
To get to a uBoot console, you must prevent the switch from booting into the OS.
Shown here is an example of the start of a boot sequence.
### JFFS2 loading 'image1' to 0x400000
Scanning JFFS2 FS: . done.
### JFFS2 load complete: 10026940 bytes loaded to 0x400000
Booting ...
Loading Ramdisk to 0fdf6000, end 0ff8d551 ... OK
Press any key before ### JFFS2 loading 'image1' to 0x400000 appears.
Chapter 4 Configuring Switch Software4-5
There is only a one-second delay to press a key.
When the uBoot prompt is displayed, you can then enter any of the following
commands.
TABLE 4-5uBoot Console Commands
CommandResult
print or
printenv
set
<variable>
or setenv
save or
saveenv
Shows the current environment variables.
Followed by an environment variable, this command changes the
environment variable.
Writes the variables to flash. You must save the changes if you want them
to persist through a reset.
4.2.2E-Keying Control in uBoot
See the Section 4.3.1, “e-Keying” on page 4-7 for a description of e-Keying, and how
it is supported.
To disable e-Keying, use the noekey environment variable.
List the ports to be disabled separated by only commas.
set noekey 1,2,3,4
Or use the word all to disable e-Keying completely.
set noekey all
To re-enable e-Keying clear the variable.
set noekey
After changing the environment variables you must always save if you would like
the change to persist though a reset.
4-6Sun Netra CP3240 Switch Installation Guide • September 2009
4.2.3Serial-Baud Rate Control in uBoot
You can change the serial baud rate from within FASTPATH, the boot menu, or in
uBoot. Changing it in uBoot is the only method that allows it to persist through a
reset. Only standard baud rates are accepted.
Note – Change the baud rate on the Fabric first, then on the Base. If the baud rate
on the Fabric and Base do not match, the Fabric will not be accessible. If the Fabric is
not accessible, change the baud rate on the Base until the Fabric is accessible.
To change the baud rate in uBoot, enter the command as in the following example.
set baudrate <115200>
After changing the rate, you must save if you would like the change to persist
though a reset.
4.3Linux
The switch uses Linux as an operating system. The Monta Vista 3.1 Pro 2.4.20 kernel
is used. There are no settings to change, and the OS is completely transparent to the
user.
4.3.1e-Keying
e-Keying is implemented as a Linux driver. The CPUs for both Base and Fabric have
a direct connection to the IPMI controller, which is used to communicate e-Keying
messages. The CPU is interrupted when an e-Keying event occurs. The driver
handles these interrupts and disables ports based on the information it receives. This
disables the port at the PHY level. e-Keying can be ignored by creating a uBoot
environment variable as described in the Section 4.2.2, “E-Keying Control in uBoot”
on page 4-6.
Chapter 4 Configuring Switch Software4-7
4.3.2e-Keying Bypass
e-Keying can be ignored by creating a uBoot environment variable as described in
Section 4.2.2, “E-Keying Control in uBoot” on page 4-6. When e-Keying is bypassed,
ports are no longer enabled or disabled based on the e-Keying state. The e-Keying
information is still shared with the shelf manager as usual, but now ports are
enabled regardless if the shelf manager enables them or not.
4.3.3ATCA LEDs
The ATCA LEDs can be turned on from several sources, one of which is a Linux
driver. Both the Base and Fabric drive these LEDs. The red OOS LED is driven until
FASTPATH has loaded, at which point the green Healthy LED is driven. The OOS
LED signals are “OR” so that if one or more are active, the LED is on. The Healthy
LED signals are “AND.” The Base and Fabric must be booted into FASTPATH as
well as several hardware conditions must be met for this LED to be on.
4.4Serial Select
A single-serial port is provided to control both CPUs. The ` key, usually in the
upper left key on the keyboard and the same key as ~, can be used to switch
between controlling the Base (3.0) and the Fabric (3.1). The ` key works in uBoot,
Linux, the BCM debug shell, and FASTPATH.
When you switch between Base and Fabric [Base] or [Fabric] is printed to the
console so that you know which one you are currently controlling.
(switch Base) #
(and then ENTER is pressed here)
(switch Base) #[Fabric]
(and then ENTER is pressed here)
(switch Fabric) #[ Base ]
(switch Base) #
User:
4-8Sun Netra CP3240 Switch Installation Guide • September 2009
4.5Serial Location
The switch supports running its serial port to both the front panel and the RTM. It
does not support both the front panel and RTM at the same time. You must
configure the port to go to the location needed. By default, the port is routed to the
front panel.
4.5.1Changing Serial Location from UBoot
In this example, the serial port direction is to the RTM.
Use the [`] key above the TAB key to switch between BASE and FABRIC.
[Break UBoot by pressing a key when the switch boots]
=> set console rtm
[set console to front for the front panel]
=> save
Saving Environment to Flash...
Un-Protected 1 sectors
Erasing Flash...
. done
Erased 1 sectors
Writing to Flash... done
Protected 1 sectors
=> boot
4.5.2Changing Serial Location from FASTPATH
In this example, the serial port direction is changed to the RTM. Note that the
change is instantaneous and lasts though a reboot.
CODE EXAMPLE 4-2Serial Port Change from FASTPATH
configure
lineconfig
serial location rtm
[set location to front for the front panel]
### JFFS2 loading 'image1' to 0x400000
Scanning JFFS2 FS: . done.
### JFFS2 load complete: 10026500 bytes loaded to 0x400000
Booting ...
Loading Ramdisk to 0fdf6000, end 0ff8d4c2 ... OK
Mounting /dev/mtdblock2 at /mnt/fastpath...done.
Mounting tmpfs at /mnt/application...done.
*****************************************************************
***
* Use the [`] key above the TAB to switch between BASE and FABRIC. *
*****************************************************************
***
switch, Base, Primary Flash, image1
Select startup mode. If no selection is made within 5 seconds,
the Application will start automatically...
Loading Application...done.
SOC unit 0 attached to PCI device BCM56503_B2
FASTPATH starting...started!
(Unit 1)>
User:
4-10Sun Netra CP3240 Switch Installation Guide • September 2009
The switch takes approximately 50 seconds to boot to a console and be fully
functional.
4.6.1Boot Utility Menu
There is a utility menu that can be used before FASTPATH boots. During boot there
is a two-option menu displayed for five seconds. This menu allows access to the
utility menu if you press number 2 and Enter.
Most of the options are self explanatory based on their names. The following
sections describe some of the options more fully.
Chapter 4 Configuring Switch Software4-11
4.6.1.1Load Code Update Package using TFTP/FTP
FASTPATH can be updated from within FASTPATH itself, but it can also be updated
from this menu.
■ The update image must be on a TFTP server or FTP server.
■ The server must be on the same network as the out-of-band management port.
■ You need to have the IP of the server, the desired IP of the board being updated,
the gateway (if needed), and the file name.
Note – DHCP can be used to obtain an IP for the switch during this update. Enter
dhcp as the IP address. This entry begins the update and provides status
information as it is updating.
4.6.1.2Erase Current Configuration
This is the same as clear config from within FASTPATH. This option can be used
if the switch is in an unknown state, and restoring the default settings is desired.
4.6.1.3Erase Permanent Storage
This command completely erases FASTPATH, any log files, and any configurations.
It does not erase uBoot or Linux. Use of this command is never recommended.
Updates can safely be installed without running this option, and configurations and
log files are preserved.
4.6.1.4Select Boot Method
The switch supports three boot methods:
■ from the local image on the flash
■ from an image over the network
■ from an image over the serial port
The default option is booting from the flash. See Section 4.9, “Network Boot” on
page 4-14 for more information.
4-12Sun Netra CP3240 Switch Installation Guide • September 2009
4.6.1.5Start Diagnostic Application
This option boots the Broadcom diag shell, currently SDK version 5.4.1. This
environment is provided as is, with no support. Several commands provided in this
shell are not supported and will not work. This shell is provided mainly for debug,
testing, and diagnostics purposes. This shell has many low-level tests, and low-level
register access. It can be used to check the integrity of particular boards. Help is
provided in the shell with ?? and commands followed by a single question mark.
Some commands of interest are TestList and TestRun.
4.7Primary and Backup Flash
The switch features hardware flash redundancy. If the switch fails to boot from the
primary flash for any reason, the onboard IPMC will reset the board and attempt to
boot out of the backup flash. No user interaction is required.
The switch backup flash comes preprogrammed with default settings. You can
change what is loaded on the backup flash by using the Copy Primary Flash toBackup Flash command from the utility menu.
4.8Dual Firmware Images
The switch supports dual firmware images of the Base and Fabric software. While
this feature does not provide high availability (the hardware redundant flash
provides that), it does provide a way to store two copies of the firmware on the
switch. This allows you to switch between firmware, as needed.
4.8.1Booting the Non-Active Image
Which image is active can be changed several ways.
From the boot menu, the Activate Backup Image option can be used.
From within FASTPATH the boot system command in combination with the
update bootcode commands can be used.
Chapter 4 Configuring Switch Software4-13
4.8.2Updating the Non-Active Image
All of the update methods supported by the active image (image1 by default) are
also supported for the non-active image (image2).
4.8.3Fabric 1G/10G Auto-negotiation
The Fabric networks support both 1Gigabyte and 10Gigabyte operation. No
configuration is required to put the switch into either a 1Gigabyte or 10Gigabyte
mode. The switch will dynamically connect at the appropriate speed.
4.9Network Boot
The switch supports network booting. This feature can be used to test updates to
FASTPATH without installing them. Updates containing changes to uBoot, Linux, or
the IPMC must be installed to the flash before they can be used.
1. Setup a TFTP server on the update network.
Tftpd is the standard TFTP server for Linux and Solaris.
For Windows, Solarwinds (
http://www.solarwinds.net) offers a free TFTP server.
2. Place the update image on the TFTP server.
It will have a .tgz file extension.
3. Connect to the serial management port of the network to be updated, and
connect the MGMT port to the update network.
4. Reboot the switch using the reboot button, hotswap handle, or reload
command.
5. As the switch boots, press number 2 to enter the Utility Menu.
6. Press number 8 to change the boot method.
7. Press number 3 to select network as the boot method.
8. Enter the information for your TFTP server.
For host IP, you can enter dhcp if you would like to use DHCP to obtain a valid
IP address.
4-14Sun Netra CP3240 Switch Installation Guide • September 2009
### JFFS2 loading 'image1' to 0x400000
Scanning JFFS2 FS: . done.
### JFFS2 load complete: 10026500 bytes loaded to 0x400000
Booting ...
Loading Ramdisk to 0fdf6000, end 0ff8d4c2 ... OK
Mounting /dev/mtdblock2 at /mnt/fastpath...done.
Mounting tmpfs at /mnt/application...done.
*****************************************************************
***
* Use the [`] key above the TAB to switch between BASE and FABRIC. *
*****************************************************************
***
switch, Base, Primary Flash, image1
Select startup mode. If no selection is made within 5 seconds,
the Application will start automatically...
*****************************************************************
***
* Use the [`] key above the TAB to switch between BASE and FABRIC. *
*****************************************************************
***
switch, Base, Primary Flash, image1
--- Utility Menu ---
1 - Start Switch Application
2 - Load Code Update Package using TFTP/FTP
3 - Load Code Update Package using XMODEM/YMODEM/ZMODEM
4 - Copy Primary Flash to Backup Flash
5 - Select Serial Speed
Chapter 4 Configuring Switch Software4-15
CODE EXAMPLE 4-5Example Console Output (Continued)
1 - Flash Boot
2 - Network Boot
3 - Serial Boot
4 - Exit without change
Select option (1-4): 2
Enter Server IP []:10.10.3.199
Enter Host IP (# or dhcp) []:dhcp
Enter Transfer Method (tftp or ftp) [ftp]:
Enter Filename []:/switch.1.0.1.0.stk
Accept changes? Press(Y/N): y
*****************************************************************
***
* Use the [`] key above the TAB to switch between BASE and FABRIC. *
*****************************************************************
***
4-16Sun Netra CP3240 Switch Installation Guide • September 2009
CODE EXAMPLE 4-5Example Console Output (Continued)
Select option (1-13): 1
Creating tmpfs filesystem on tmpfs for download...done.
Bringing up eth0 interface...done.
HOST IPADDR=10.10.2.246
Transferring '/switch.1.0.1.0.stk' from '10.10.3.199' via
ftp...done.
Bringing down eth0 interface...done.
Extracting Application...done.
Destroying tmpfs filesystem on tmpfs...done.
Loading Application...done.
SOC unit 0 attached to PCI device BCM56503_B2
FASTPATH starting...started!
(Unit 1)>
User:
10. Use option 8 of the Utility Menu to change the boot method back to flash, when
desired.
Chapter 4 Configuring Switch Software4-17
4.10FASTPATH
The switch features LVL7 Systems’ FASTPATH 2340 software, version 4.4.
FASTPATH is a software package providing the robust management needed to
control a modern switch-router. This section serves as a quick primer on using
FASTPATH on the switch. FASTPATH is not covered in detail in this guide, but is
covered in the Netra CP3240 Switch Command Reference Manual (820-3253).
4.10.1Management Options
The switch can be controlled by a CLI, a web interface, and through SNMP. All
management interfaces, other than serial, can be enabled or disabled, provided over
both the out-of-band management port and/or any in-band ports, and be limited to
certain in-band ports.
4.10.1.1CLI
The switch provides an industry standard CLI. The CLI is provided over the serial
port, telnet and SSH. This section only describes some basic commands. For detailed
command syntax, refer to the Netra CP3240 Switch Command Reference Manual
(820-3253).
The CLI is mode based. It works similarly to a console in Linux or Windows.
Commands are grouped under modes, and those commands only work when the
user is in the current mode. There are very few global commands.
To return to one mode higher than the current mode, use exit.
For example to shutdown port 17, switch to enable mode, configure mode, interface
17 mode, then issue the shutdown command. Use exit to return to configure
mode, and exit again to return to enable mode.
Many commands have a no form. The no form is used to disable the command. With
the previous example, to re-enable port 17, once you are in the interface 17 mode,
you would issue the no shutdown command.
4-18Sun Netra CP3240 Switch Installation Guide • September 2009
4.10.2Basic CLI Commands
TABLE 4-6Basic CLI Commands
CommandFunctionMode
enableSwitch to privilege mode. You must be in enabled
mode for most options.
show port allShow status of the ports.privilege
show interface ethernet 0/xShow detailed statistics on port 0/x.privilege
clear countersClear all statistics.privilege
clear configRestore the default configuration.privilege
show running-configShow the current configuration of the switch.
This command shows everything that is not set to
the default value. The output is a script that can
be copied to a file for later use, or for another
switch.
copy system:running-config
nvram:startup-config
serviceport protocol dhcpUse DHCP on the out-of band port. Only one of
serviceport protocol noneUse user assigned IP NOTE: the same command
serviceport ip <ip> <netmask>
<gateway>
network parms <ip> <netmask>
<gateway>
show networkView the in-band management settings.privilege
show serviceportView the out-of-band management settings.privilege
serial baudrateChange the serial baudrate.privilege
vlan databaseSwitch to vlan database mode. Create and delete
vlan xCreate a VLAN with number x.vlan
exitReturn to one mode higher.
configureSwitch to configure mode. You must be in
Save the current configuration though a reboot.privilege
the serviceport and network can use DHCP. To
enable DHCP on one, you must disable it on the
other. The same command works for network.
works for network.
Force an IP for serviceport.privilege
Force an IP for network.privilege
VLANs here.
configure mode to change most settings.
default
privilege
privilege
privilege
privilege
privilege
Chapter 4 Configuring Switch Software4-19
TABLE 4-6Basic CLI Commands (Continued)
interface 0/xSwitch to interface mode. You must be in
interface mode to change most port specific
settings.
vlan participation include xAdd interface to VLAN x.interface
vlan pvid xChange interface’s pvid to VLAN x.interface
configure
4.10.3Logins and Prompts
FASTPATH supports multiple users with different security levels. By default, thereis one admin user with no password. In the CLI, privilege mode is
password-protected separately from the default mode, but also has a default of no
password.
4.10.3.1CLI Defaults
A CLI is provided on the serial console, telnet console, and SSH console. The serial
console is always enabled. The telnet console is enabled by default. SSH is
disabled by default.
Username:admin
Password:
>enable
Password:
#
The prompts always show the current mode. Here are a few examples.
The username and password are the same as the CLI. The web interface is enabled
by default. Java is enabled by default. HTTPS is disabled by default.
4-20Sun Netra CP3240 Switch Installation Guide • September 2009
4.10.3.3SNMP Defaults
SNMP is enabled by default. The default read-only community string is public.
The default read-write community string is private.
4.10.4Secure Remote Access
4.10.4.1SSH and SSL/TLS Keys
The switch supports SSH for a secure CLI console as well as SSL/TLS for secure
HTTP. By default, SSH and SSL are disabled. The switch cannot generate its own
keys. Keys must be generated on an external PC, and uploaded to the switch via
TFTP. Once the keys are on the switch, SSH and HTTPS must be enabled to be used.
4.10.4.2Enabling SSH and SSL
First, the certifications and keys must be uploaded to the switch using a command
such as the following:
copy tftp://<ip>/<file> <location>
Upload the following files to the switch:.
FileLocation
rsa1.keynvram:sshkey-rsa1
rsa2.keynvram:sshkey-rsa2
dsa.keynvram:sshkey-dsa
dh512.pemnvram:sslpem-dhweak
dh1024.pemnvram:sslpem-dhstrong
server.pemnvram:sslpem-server
rootcert.pemnvram:sslpem-root
Enable secure access (SSH/Telnet) and disable non-secure access (SSL/HTTP).
ip ssh
configure
Chapter 4 Configuring Switch Software4-21
lineconfig
no transport input telnet
exit
ip http secure-server
no ip http server
4.10.5Default Settings
The switch comes configured with a default configuration. This configuration boots
the board to Layer 2 switching. This configuration is very basic and should be
updated for your environment. The default settings are as follows:
■ every port is in VLAN 1
■ every port is configured in switching mode
■ management interfaces are enabled
■ spanning-tree is enabled on every port
■ everything else is disabled
The settings of the switch can be checked with the show running-config
command. This command shows how the current configuration is different than the
default configuration. This command can be very useful because the output is in
script format. This output can be backed up or copied to another switch.
4.10.6Port Ordering
The ports are ordered the same way as the ATCA channels. The ATCA specification
allows for backplanes to be routed differently. This means that logical slots and
physical slots often do not match.
The switch supports an abstraction layer that allows the port order to be changed.
This allows vendors to match logical slots and physical slots independent of routing.
Contact Sun if you would like a non-standard port order. Sun’s port ordering is
shown in the following table for shelves with middle hub slots, that is, slots 7 and 8.
TABLE 4-7Port Order List
Shelf Physical SlotBase PortFabric Port
11312
21110
4-22Sun Netra CP3240 Switch Installation Guide • September 2009
Loading...
+ hidden pages
You need points to download manuals.
1 point = 1 manual.
You can buy points or you can get point for every manual you upload.