IBM BNT Installation Manual

BNT Virtual Fabric 10Gb Switch Module for IBM BladeCenter

Installation Guide



BNT Virtual Fabric 10Gb Switch Module for IBM BladeCenter

Installation Guide



Note: Before using this information and the product it supports, read the general information in Appendix B, “Warranty,” on page 45 and Appendix C, “Notices,” on page 65; and read the IBM Safety Information and the IBM Systems Environmental Notices and User Guide on the IBM Documentation CD

Second Edition (November 2010)

US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Safety ...............v

Chapter 1. The BNT Virtual Fabric 10Gb

Switch Module ............1

Specifications ..............3

Safety

Before installing this product, read the Safety Information.

Antes de instalar este produto, leia as Informações de Segurança.

Pred instalací tohoto produktu si prectete prírucku bezpecnostních instrukcí.

Læs sikkerhedsforskrifterne, før du installerer dette produkt.

Lees voordat u dit product installeert eerst de veiligheidsvoorschriften.

Ennen kuin asennat tämän tuotteen, lue turvaohjeet kohdasta Safety Information.

Avant d'installer ce produit, lisez les consignes de sécurité.

Vor der Installation dieses Produkts die Sicherheitshinweise lesen.

Prima di installare questo prodotto, leggere le Informazioni sulla Sicurezza.

Les sikkerhetsinformasjonen (Safety Information) før du installerer dette produktet.

Antes de instalar este produto, leia as Informações sobre Segurança.

Antes de instalar este producto, lea la información de seguridad.

Läs säkerhetsinformationen innan du installerar den här produkten.

Important:

Each caution and danger statement in this document is labeled with a number. This number is used to cross reference an English-language caution or danger statement with translated versions of the caution or danger statement in the Safety Information document.

For example, if a caution statement is labeled “Statement 1,” translations for that caution statement are in the Safety Information document under “Statement 1.”

Be sure to read all caution and danger statements in this document before you perform the procedures. Read any additional safety information that comes with the server or optional device before you install the device.

This device is intended for use with UL Listed IBM BladeCenters.

vi BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Statement 1:

DANGER

Electrical current from power, telephone, and communication cables is hazardous.

To avoid a shock hazard:

v Do not connect or disconnect any cables or perform installation,

maintenance, or reconfiguration of this product during an electrical storm.

v Connect all power cords to a properly wired and grounded electrical outlet.

v Connect to properly wired outlets any equipment that will be attached to

this product.

v When possible, use one hand only to connect or disconnect signal cables.

v Never turn on any equipment when there is evidence of fire, water, or

structural damage.

v Disconnect the attached power cords, telecommunications systems,

networks, and modems before you open the device covers, unless instructed otherwise in the installation and configuration procedures.

v Connect and disconnect cables as described in the following table when

installing, moving, or opening covers on this product or attached devices.

To Connect: To Disconnect:

1. Turn everything OFF.

2. First, attach all cables to devices.

3. Attach signal cables to connectors.

4. Attach power cords to outlet.

5. Turn device ON.

1. Turn everything OFF.

2. First, remove power cords from outlet.

3. Remove signal cables from connectors.

4. Remove all cables from devices.

Safety vii

Statement 3:

CAUTION: When laser products (such as CD-ROMs, DVD drives, fiber optic devices, or transmitters) are installed, note the following:

DANGER

Some laser products contain an embedded Class 3A or Class 3B laser diode. Note the following.

Laser radiation when open. Do not stare into the beam, do not view directly with optical instruments, and avoid direct exposure to the beam.

Class 1 Laser Product Laser Klasse 1 Laser Klass 1 Luokan 1 Laserlaite Appareil A Laser de Classe 1

viii BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Statement 8:

CAUTION: Never remove the cover on a power supply or any part that has the following label attached.

Hazardous voltage, current, and energy levels are present inside any component that has this label attached. There are no serviceable parts inside these components. If you suspect a problem with one of these parts, contact a service technician.

Safety ix

x BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Chapter 1. The BNT Virtual Fabric 10Gb Switch Module

The BNT Virtual Fabric 10Gb Switch Module for IBM BladeCenter is a high-speed Ethernet component that is installed into a BladeCenter®unit that supports high-speed I/O modules.

This Installation Guide contains instructions for and information about: v Setting up and installing or replacing the BNT Virtual Fabric 10Gb Switch

Module for IBM BladeCenter

v Installing and removing optional devices in the switch module v Using the information panel, LEDs, and external ports on the switch module v Cabling the switch module and its optional devices v Configuring the switch module v Updating the switch-module software v Solving problems with the switch module

For installation instructions, see Chapter 2, “Installing and replacing a switch module,” on page 7 and Chapter 3, “Installing and removing a 10 Gb SFP+ module,” on page 15. For additional information about switch modules and other BladeCenter components, see the BladeCenter documentation that comes with these devices.

To support each BNT Virtual Fabric 10Gb Switch Module that you install in the BladeCenter unit, you must also install a compatible high-speed Ethernet expansion card (also known as an Ethernet I/O card) in each blade server that you want to communicate with the switch module. In this environment, the expansion card operates as a host channel adapter (HCA). For additional information, see Chapter 2, “Installing and replacing a switch module,” on page 7 and the installation information for the Ethernet expansion card.

For information about the types of compatible expansion cards for the blade server, contact your IBM marketing representative or authorized reseller. For a list of supported optional devices for the blade server, see http://www.ibm.com/servers/ eserver/serverproven/compat/us/. For details about compatible expansion card installation, configuration, and use, see the documentation that comes with the adapter.

You can obtain up-to-date information about the BNT 10-Port 10Gb Ethernet Switch Module for IBM bladecenter/.

Notes:

1. Throughout this document, the BNT Virtual Fabric 10Gb Switch Module for IBM BladeCenter is referred to as the high-speed switch module, the HSSM, the switch module, or the I/O module.

2. Unless otherwise stated, references to the BladeCenter unit apply to all BladeCenter units that support high-speed I/O modules, such as the BladeCenter H unit.

3. Changes are made periodically to the IBM Web site. Procedures for locating firmware and documentation might vary slightly from what is described in this document.

BladeCenter at http://www.ibm.com/systems/

4. The illustrations in this document might differ slightly from your hardware.

5. The screens that are described or referenced in this document might differ

slightly from the screens that are displayed by your system. Screen content varies according to the type of BladeCenter unit and the firmware versions and options that are installed.

6. Unless otherwise stated, references to the management module apply only to the BladeCenter Advanced Management Module, which is the only type of management module that supports the switch module.

The switch module has the following components: v Fourteen internal 10 Gb ports, one connected to each of the blade servers in the

BladeCenter unit

v Two internal 1 Gb ports to connect to the management module v Ten external 10 Gb user ports for connecting small-form-factor pluggable (SFP+)

modules

v One external 1 Gb Ethernet port v One external RS-232 serial port for management use

You can manage and configure the switch module through multiple interfaces:

v A Telnet connection to the embedded command-line interface (CLI) v A terminal emulation program connection to the serial-port interface v A Web browser-based interface (BBI) connection to the switch module

For more information, see Chapter 6, “Configuring the switch module,” on page

27.

Record information about the switch module in the following table. The product name and serial number are on the identification label on the bottom cover of the switch module. The media access control (MAC) address is on a separate label on the bottom cover of the switch module. For an illustration that shows the locations of these labels, see “Major components of the switch module” on page 5. You will need this information when you register the switch module with IBM. You can register the switch module at http://www.ibm.com/support/mysupport/.

Product name BNT Virtual Fabric 10Gb Switch Module for IBM

BladeCenter

Model number _____________________________________________ Serial number _____________________________________________ Part number _____________________________________________ Media access control (MAC)

address for switch module MAC addresses for other

components

_____________________________________________

2 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Specifications

For detailed information about the switch-module hardware and software features, specifications, and standards, see the switch module Application Guide.

Inventory checklist

Make sure that the shipping carton contains the following items:

v One switch module v The BNT Virtual Fabric 10Gb Switch Module for IBM BladeCenter Installation Guide

(this document)

v One serial console cable v One filler module v Safety flyer v End User License Agreement

If any of these items are missing or damaged, contact your authorized reseller for replacement.

Notices and statements in this document

The caution and danger statements in this document are also in the multilingual Safety Information document, which is on the IBM BladeCenter Documentation CD for the BladeCenter unit. Each statement is numbered for reference to the corresponding statement in your language in the Safety Information document.

The following notices and statements are used in this document:

v Note: These notices provide important tips, guidance, or advice. v Important: These notices provide information or advice that might help you

avoid inconvenient or problem situations.

v Attention: These notices indicate potential damage to programs, devices, or data.

An attention notice is placed just before the instruction or situation in which damage could occur.

v Caution: These statements indicate situations that can be potentially hazardous

to you. A caution statement is placed just before the description of a potentially hazardous procedure step or situation.

v Danger: These statements indicate situations that can be potentially lethal or

extremely hazardous to you. A danger statement is placed just before the description of a potentially lethal or extremely hazardous procedure step or situation.

4 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Major components of the switch module

The following illustration shows the major components of the switch module.

Note: The illustrations in this document might differ slightly from your hardware,

and your switch module might have labels that are not shown in the illustrations in this document.

For more information about the components of the information panel, see Chapter 5, “Information panels, LEDs, and external ports,” on page 23.

Chapter 1. The BNT Virtual Fabric 10Gb Switch Module 5

6 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Chapter 2. Installing and replacing a switch module

This chapter provides instructions for installing a switch module in the BladeCenter unit and for removing a switch module from the BladeCenter unit. See the documentation for your BladeCenter unit for information about I/O-module bay locations and the components that can be installed in them that is specific to your BladeCenter unit type.

The following illustration shows an example of a BladeCenter unit with the I/O-module bays identified. In this example, these bays are in the rear of the BladeCenter chassis. In a different type of BladeCenter unit, the bays might be in a different location.

I/O module bay 7

I/O module bay 8

Power connector 2

I/O module bay 1 I/O module bay 5

Power connector 1

Management module 1

I/O module bay 3

Blower module 1 error LED

Blower module 1

I/O module bay 2 I/O module bay 6

Blower module 2 error LED

Rear system LED panel

Serial connector

I/O module bay 9

I/O module bay 10

Management module bay 2

I/O module bay 4

Blower module 2

An expansion card or host channel adapter (HCA) must be installed in each blade server that you want to communicate with. To enable the switch module to communicate with a blade server, at least one switch module must be installed in the BladeCenter unit. For details about expansion-card installation, configuration, and use, see the documentation that comes with the expansion card.

Installing a second switch module enables a redundant path and a separate connection from the blade server to the external Ethernet network.

The BladeCenter unit supports a maximum of four BNT Virtual Fabric 10Gb Switch Modules. Depending on the type of BladeCenter unit that you are using, the BladeCenter unit supports a maximum of 10 or 14 expansion cards.

Notes:

v The blade servers or BladeCenter units that are described or shown in this

document might be different from your blade server or BladeCenter unit. For additional information, see the documentation that comes with your blade server or BladeCenter unit.

v If you are installing only one switch module, use I/O-module bay 7 or 9. v When the switch module is installed in a BladeCenter unit, the internal ports

operate at 10 Gbps. The external ports can operate at 10 Gbps or 1 Gbps, depending on the SFP module type.

v Configuration requirements for the switch module and the BladeCenter unit

might vary. You can obtain up-to-date information about the switch module and the BladeCenter unit at http://www.ibm.com/systems/bladecenter/.

Installation guidelines

Before you install the switch module in the BladeCenter unit, read the following information:

v Read the safety information that begins on page v, “Handling static-sensitive

devices” on page 9, and the safety statements in the BladeCenter unit documentation. This information will help you work safely.

v Observe good housekeeping in the area where you are working. Place removed

covers and other parts in a safe place.

v Blue on a component indicates touch points, where you can grip the component

to remove it from or install it in the blade server or BladeCenter unit, open or close a latch, and so on.

v Orange on a component or an orange label on or near a component on the

switch module, blade server, or BladeCenter unit indicates that the component can be hot-swapped, which means that if the BladeCenter unit and operating system support hot-swap capability, you can remove or install the component while the BladeCenter unit is running. (Orange can also indicate touch points on hot-swap components.) See the instructions for removing or installing a specific hot-swap component for any additional procedures that you might have to perform before you remove or install the component.

v You do not have to turn off the BladeCenter unit to install or replace any of the

hot-swap modules on the front or rear of the BladeCenter unit.

v When you install a switch module in the BladeCenter unit, you must also install

a compatible I/O expansion card in the blade server to support the switch module.

v When you are finished working on the blade server or BladeCenter unit,

reinstall all safety shields, guards, labels, and ground wires.

v For a list of supported optional devices for the BladeCenter unit and other IBM

products, see http://www.ibm.com/servers/eserver/serverproven/compat/us/.

System reliability guidelines

To help ensure proper cooling, performance, and system reliability, make sure that the following requirements are met:

v Each of the module bays on the rear of the BladeCenter unit contains either a

module or a filler module.

v A removed hot-swap module is replaced with an identical module or filler

module within 1 minute of removal.

v A removed hot-swap blade server is replaced with another blade server or filler

blade within 1 minute of removal.

8 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

v The ventilation areas on the sides of the blade server are not blocked. v You have followed the reliability guidelines in the documentation that comes

with the BladeCenter unit.

Cable requirements for the switch module are described in the IBM Configuration and Options Guide at http://www.ibm.com/servers/eserver/xseries/cog/. See the documentation that comes with the blade server for cable-routing information.

Handling static-sensitive devices

Attention: Static electricity can damage the BladeCenter unit and other electronic devices. To avoid damage, keep static-sensitive devices in their static-protective packages until you are ready to install them.

To reduce the possibility of electrostatic discharge, observe the following precautions:

v Limit your movement. Movement can cause static electricity to build up around

you.

v Handle the device carefully, holding it by its edges or its frame. v Do not touch solder joints, pins, or exposed printed circuitry. v Do not leave the device where others can handle and damage it. v While the device is still in its static-protective package, touch it to an unpainted

metal surface of the BladeCenter unit chassis or an unpainted metal surface on any other grounded rack component in the rack that you are installing the device in for at least 2 seconds. This drains static electricity from the package and from your body.

v Remove the device from its package and install it directly into the BladeCenter

unit without setting down the device. If it is necessary to set down the device, put it back into its static-protective package. Do not place the device on the BladeCenter unit or on a metal surface.

v Take additional care when you handle devices during cold weather. Heating

reduces indoor humidity and increases static electricity.

v Some types of BladeCenter units come with electrostatic discharge (ESD)

connectors. If the BladeCenter unit is equipped with an ESD connector, see the documentation that comes with the BladeCenter unit for using the ESD connector.

Chapter 2. Installing and replacing a switch module 9

Installing a switch module

Note: The following illustration shows how to install a switch module in a Type

8852 BladeCenter unit. The appearance of your BladeCenter unit might be different; see the documentation for your BladeCenter unit for additional information.

To install a switch module, complete the following steps:

1. Read the safety information that begins on page v and “Installation guidelines” on page 8.

2. Select I/O-module bay in which to install the switch module.

Note: For details about I/O-module bay requirements and bay locations, see

the documentation for the BladeCenter unit and blade servers.

3. Remove the filler module from the selected bay. Store the filler module for future use.

4. If you have not already done so, touch the static-protective package that contains the switch module to an unpainted metal surface of the BladeCenter unit or an unpainted metal surface on any other grounded rack-component for at least 2 seconds.

5. If the removed filler module (from step 3) occupied two bays:

v Remove the single-high filler module from its static-protective package. v Install the single-high filler module into the unused bay.

6. Remove the switch module from its static-protective package.

7. Make sure that the release levers on the switch module are in the open

position (perpendicular to the module). For specific instructions for installing a switch module in the BladeCenter

unit, see the documentation that comes with the BladeCenter unit.

8. Slide the switch module into the applicable I/O-module bay until it stops.

9. Push the release levers on the front of the switch module to the closed

position. After you insert and lock the switch module, it is turned on, and a power-on self-test (POST) occurs to verify that the switch module is operating correctly.

10 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Notes:

a. The switch module takes approximately 60 seconds to complete the POST.

When the switch module is turned on, an LED test occurs. All LEDs are lit and remain lit during POST; then, all the LEDs except the OK LED turn off. This indicates normal POST results.

b. To maintain proper airflow, make sure that the ventilation holes on the

front of the switch module are not blocked.

10. Make sure that the LEDs on the switch module indicate that it is operating correctly (see “Information LEDs” on page 24).

11. If you have another switch module to install, repeat step 3 on page 10 through

step 10; otherwise, go to the next step.

12. Install the SFP+ modules in the switch module. For information and instructions, see Chapter 3, “Installing and removing a 10 Gb SFP+ module,” on page 15 and the documentation that comes with the SFP+ module.

13. Attach any cables that are required by the switch module. For additional information about cabling the switch module, see Chapter 4, “Cabling the switch module and the SFP+ module,” on page 19, the documentation that comes with the cables, and the optional network devices to which the cables have been connected. For the locations of the connectors on the BladeCenter unit, see the documentation that comes with the BladeCenter unit. Then, continue with the next step.

14. Make sure that the external ports on the switch module are enabled through one of the management-module interfaces, such as the Web-based interface or the CLI.

Chapter 2. Installing and replacing a switch module 11

Removing or replacing a switch module

Note: The following illustration shows how to remove and replace a switch

module from a Type 8852 BladeCenter unit. The appearance of your BladeCenter unit might be different; see the documentation for your BladeCenter unit for additional information.

To replace a switch module, complete the following steps:

1. Read the safety information that begins on page v, and “Installation guidelines” on page 8.

2. Disconnect any cables from the switch module that you are removing. Removing these cables (especially an Ethernet cable) disrupts the network connection from the external Ethernet port to any connected external Ethernet devices. If you plan to replace the switch module with another switch module, you can use the existing Ethernet cable, provided that it remains securely attached to the Ethernet network. For additional information about cabling the switch module, see Chapter 4, “Cabling the switch module and the SFP+ module,” on page 19, the documentation that comes with the cables, and the optional network devices to which the cables have been connected. For the locations of the connectors on the BladeCenter unit, see the documentation that comes with the BladeCenter unit. Then, continue with step 3.

3. Pull the release latches out from the switch module. The switch module moves out of the bay approximately 0.6 cm (0.25 inch).

4. Slide the switch module out of the bay and set it aside.

5. Place either another switch module or a filler module in the bay.

Important: Complete this step within 1 minute. (For more information, see steps 9 and 10 on page 11.)

6. If you placed a filler module in the bay, continue with Chapter 3, “Installing and removing a 10 Gb SFP+ module,” on page 15.

7. If you placed a switch module in the bay, reconnect the other cables that you disconnected. Attach any additional cables that are required by the switch module. For additional information about cabling the switch module, see Chapter 4, “Cabling the switch module and the SFP+ module,” on page 19, the documentation that comes with the cables, and the optional network devices to which the cables have been connected. For the locations of the connectors on

12 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

the BladeCenter unit, see the documentation that comes with the BladeCenter unit. Then, continue with Chapter 3, “Installing and removing a 10 Gb SFP+ module,” on page 15.

Chapter 2. Installing and replacing a switch module 13

14 BNT Virtual Fabric 10Gb Switch Module: Installation Guide

Chapter 3. Installing and removing a 10 Gb SFP+ module

The switch module supports both the 10 Gb small-form-factor pluggable (SFP+) module and the 1 Gb small-form-factor pluggable (SFP) module. The SFP+ and SFP modules are laser products that convert electrical signals to optical signals.

For additional information about the location of the switch module, the network interface requirements, and expansion options, see the documentation for your BladeCenter unit.

Notes:

1. The illustrations in this document might differ slightly from your hardware.

2. While the information in this section describes the 10 Gb small-form-factor

pluggable (SFP+) module, it also applies to the 1 Gb small-form-factor pluggable (SFP) module.

3. The switch module also supports MSA-compliant copper direct-attach cables (DAC), up to 7 m (23 ft) in length.

Handling an SFP+ module

Before you install an SFP+ module, read the following information: v The module housing of the SFP+ has an integral guide key that is designed to

prevent you from inserting the module incorrectly.

v Use minimal pressure when you insert the module into the port. Forcing the

module into the port can cause damage to the module or the module port.

v You can insert or remove the module while the BladeCenter unit is turned on. v You must first insert the module into the port before you can connect the cables. v You must remove the cable from the SFP+ module before you remove the SFP+

module from the switch module.

+ 61 hidden pages

IBM BNT Installation Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Contents

Safety

Chapter 1. The BNT Virtual Fabric 10Gb Switch Module

Specifications

Related documentation

Inventory checklist

Notices and statements in this document

Major components of the switch module

Chapter 2. Installing and replacing a switch module

Installation guidelines

System reliability guidelines

Handling static-sensitive devices

Installing a switch module

Removing or replacing a switch module

Chapter 3. Installing and removing a 10 Gb SFP+ module

Handling an SFP+ module