Dell M4001F, M4001Q User Manual

Download

Third party information brought

to you courtesy of Dell.

SwitchX 56Gb/s InfiniBand Blade Switch I/O

Module for PowerEdge M-Series User Manual

PN: M4001F, M4001Q

Rev 1.0

www.mellanox.com

Rev 1.0

NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUS TOMER'S MANUF ACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CAN NOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MER CHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEM PLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, S TRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE US E OF THE PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale, CA 94085 U.S.A. www.mellanox.com Tel: (408) 970-3400 Fax: (408) 970-3403

Mellanox, BridgeX, ConnectX, CORE-Direct, InfiniBridge, InfiniHost, InfiniScale, PhyX, SwitchX, Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies, Ltd. FabricIT, MLNX-OS and Unbreakable-Link are trademarks of Mellanox Technologies, Ltd. All othe r trademarks are property of their respective owners.

All other trademarks are property of their respect ive owners.

InfiniBand Blade Switch I/O Module for PowerEdge M-Series

Mellanox Technologies

Mellanox Technologies, Ltd. Beit Mellanox PO Box 586 Yokneam 20692 Israel Tel: +972 (0)4 909 7200 ; +972 (0)74 723 7200 Fax: +972 (0)4 959 3245

Document Number: 3513

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

Contents 3 Revision History 4 About this Manual 5 Chapter 1 Overview 7

1.1 InfiniBand Connectors 8

1.2 Switch Status Lights 9

1.3 I2C Interface RJ-45 Connector 10

Chapter 2 Installation and Basic Operation 11

2.1 Installation Safety Warnin gs 11

2.2 Mechanical Installation 13

2.2.1 Removing a n Old Switch From the Chassis 13

2.2.2 Removing an I/O Module Blank From the Chassis 14

2.2.3 Installing the Mellanox M4001[F, Q] Switch Into the Dell Chassis 14

2.3 Power Connections and Initial Power On 14

2.4 InfiniBand Cable Installation 14

Chapter 3 Cluster Management and Firmware 16

3.1 Network Management and Clustering Software 16

3.2 Updating Firmware 16

3.2.1 Instructions for Reprogramming Over the InfiniBand Network 16

Chapter 4 Troubleshooting 17 Appendix A Specifications 18 Appendix B EMC Certification Statements 20 Appendix C QSFP Interface 23 Appendix D Avertissements de sécurité d’installation 25 Appendix E Installation - Sicherheitshinweise 27

Mellanox Technolog ie s

Rev 1.0

Revision History

Table 1 - Revision History Table

Date Revision Description

November 2011 1.0 Initial release

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

About this Manual

This manual describes the installation and basic use of the SwitchX M4001F 56 Gb/s, and the M4001Q 40 Gb/s InfiniBand blade switch I/O modules for the PowerEdge M-Series chassis.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting up the switch platforms listed above.

The manual assumes familiarity with the Infi niBand® Architecture Specification.

Online Resources

• Mellanox Technologies Web pages: http://www.mellanox.com

• Dell Support Web pages: http://support.dell.com

Conventions

Throughout this manual, the names M4001F and M4001Q and the terms switch, I/O module, and IOM are used to describe the M4001F 56 Gb/s, and the M4001Q 40 Gb/s InfiniBand blade switch I/O modules, unless explicitly indicated otherwise.

The following pictur es a re used throughout this document to indicate inf ormation that is import an t to the user.

InfiniBand architecture specification descriptions

The MFT (Mellanox Firmwa re Tools) package is a set of firmware tools. The manual supplied with this package provides an overview of the firmware its installation and replacem ent. Th e MFT can be down loaded wit h its do cumenta tion at:

http://www.mellanox.com > Downloads > Firmware Tools.

This symbol makes recommendations to the user.

Mellanox Technolog ie s

Rev 1.0

This symbol indicates information that is helpful to the user.

This symbol indicates a situation that can potentially cause damage to hardware or software.

BEWARE! This symbol indicates a situation that can potentially cause personal in jury or damage to hardware or software.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

1Overview

The SwitchX M4001F 56 Gb/s, and the M4001Q 40 Gb/s InfiniBand blade switch I/O module for the PowerEdge M-Series chassis provides a high bandwidth, low latency fabric for Enterprise Data Centers (EDC), High-Perf ormance Computing (HPC), and Embedded environments. Based on the

generation InfiniBand switch device (1st Generation SwitchX). This I/O module (IOM) deliv-

5 ers up to 56Gb/s at FDR speed and 40 Gb/s at QDR speed full bisectional bandwidth per port. When used in conju nct ion with ConnectX®-3 InfiniBand dual por t Me zz ani ne I/O cards, clustered data bases, parallelized applications and transactional services applications these switches will achieve significant performance improvements resulting in reduced completion time and lower cost per operation.

The I/O module comes pre-installed with all necessary firmware, and configured for standard operation within an InfiniBand fabric, and only requires an InfiniBand compliant Subnet Manager running from one of the hosts. All that is required for normal operation is to follow the usual pre cautions for installation and connection from the switch to the HCAs or other switches. Once connected, the Subnet Management software automatically configures the switch and begins utilizing the switch.

It is recommended that Mellanox OpenFabrics software package be installed on all nodes connected to the M4001[F, Q]. The software package provides a subnet manager and network management tools as well as connectivity software for servers and storage, and is available on the Mellanox web site. See Chapter 3 for more information.

Basic installation, hot -swappi ng component s and hardware main tenanc e is cover ed in “Installation and Basic Operation” on page 11.

The M4001F and M4001Q switches have a Hot Swap controller and a PSOC Management IC.

Mellanox Technolog ie s

Figure 1: Switch Front Panel and Locking Mechanism

Locking Me chanism

Switch Status Lights

Port LEDs

OverviewRev 1.0

1.1 InfiniBand Connectors

This switch has 16 QSFP connectors through the front panel. The remaining 16 interfaces are through the Midplane Connector out of the back of the switch.

Each of the InfiniBand ports has two LEDs located between the connectors and the I2C interface. The green LED, when lit, indicates that a valid physical connection to the other system (switch or HCA port) exists. See

Mellanox Technologies

Figure 2,“Physical and Logical Link Indication LEDs” for more details.

Figure 1 shows the front 16 ports.

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

LED Name Connectio n Status

Physical Link - Green Off – No Physical Link

ON – Physical Link

Data Activity - Yellow

Blinking – indicat e s Dat a Transfer Constant on – indicates Link exists with no Data Transfer taking place Off with green LED lit – indicates that the Subnet Manager may not be running

The IO Module is on and ready when both the blue and green LEDs are lit.

This LED can be blue or amber. A fault is indicated when the am ber LED is blinking. See Table 3.

Figure 2: Physical and Logical Link Indication LEDs

1.2 Switch Status Lights

The switch Status lights indicate whether the switch is receiving power from the chassis, and the state of the switch.

Table 3 - IOM states and LED configurations:

Indication Green Status

Figure 3: Indicator LEDs

LED

Switch Statu s

OFF OFF OFF

ON OFF Boot in Progress

Switch not ready

Mellanox Technolog ie s

Indication Green Status

Blinking Blue

OverviewRev 1.0

LED

Switch Status

ON BLINKING BLUE The CMC is identify ing

the newly installed switch

ON ON BLUE Switch is on and operating Normally

ON ON or BLINKING

OFF ON or BLINKING

1.3 I2C Interface RJ-45 Connector

There is an “I2C”interface on the front panel.

Figure 4: I2C RJ-45 Interface

The I2C connection provides access to Flash and EEPROMs. The I2C connector is used to update firmware sho uld in -ba nd firm war e upda te not work. You will need to order an RJ-45 to DB9 I2C adapter to make use of the I2C interface. You can order this adapter from Mellanox Technologies. The Ordering Part Number is MTUSB-1.

AMBER

Fault in System

Self-diagnosed

Fault in System

CMC-detected

All firmware updates should be done in-band using Mellanox Firmware Management Tools. This is for Advanced users and FAEs only.

These switches are Plug and Play and all firmware updates should be done in-band. The I2C connection should only be used if the FW image was corrupted to the point that the regular FW tools cannot successfully reburn the correct image.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

2 Installation and Basic Operation

Installation and initialization of the I/O module are straightforward processes, requiring attention to the normal mechanical , power, and thermal precautions for rack-mounted equipment. The I/O module does not require any programmin g or confi gurati on to oper ate as a bas ic Infi niBand switch and includes all of the necessary functionality to operate with external standard InfiniBand Subnet Management software.

This section describes the installation process and basic operation of the I/O module. Please first read the warnings sub-section carefully before carrying on with installation procedures.

Suitable electrical, mechanical and fire enclosure shall be provided by the end user.

2.1 Installation Safety Warnings

1. I nst all at ion Inst ructions

Read all installation instructions before conn ecting the equipment to the power source.

2. Over-temperature

This equipment should not be operated in an area with an ambient temperature exceeding the maximum recommended: 40°C (104°F). Moreover, to guarantee proper air flow, allow at least 8cm (3 inches) of clearance around the ventilation openings.

3. Stacking the Chassis

The chassis should not be stacked on any other equipment. If the chassis falls, it can cause bodily injury and equipment damage.

4. Dur ing Light ning - Electrical Hazard

During periods of lightning activity, do not work on the equipment or connect or disconnect cables.

Mellanox Technolog ie s

5. Copper InfiniBand Cable Connecting/Disconnecting

Copper InfiniBand cables are heavy and not flexible, as su ch th ey should be carefully attached to or detached from the connectors. Support the cable’s weight using the rack and a cable management system. Refer to the cable manuf acturer fo r special warnings and instructions.

6. Rack Mounting and Servicing

When this product is mounted or serviced in a rack, special precautions must be taken to ensure that the system remai ns s tabl e. In general you should fill the rack with equipment starting from the bottom to th e top.

7. Equipment Installation

This equipment should be installed, replaced, or serviced only by trained and qualified personnel.

8. Equipment Disposal

Installation and Basic OperationRev 1.0

Disposal of this equipment should be in accordance to all national laws and regulations.

9. Local and National Electrical Codes

This equipment should be install e d in compliance with local and national electrical codes.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

Latch Release Buttons

Locking Arms

C2 B2 A2 Channel 2 slots

A1 B1 C1 Channel 1 slots

2.2 Mechanical Installation

These switches are hot pluggable. It is not necessary to power down the Dell Chassis to install a new switch or to replace an old switch with a new one.

Figure 5: Rear View of the Dell PowerEdge M1000e Chassis

M4001[F, Q] blades are not allowed in the Fabric A slots.

The switch can only be placed in channel slots B1, C1, C2 or B2.

2.2.1 Removing an Old Switch From the Chassis

1. Remove any locking cables or screws that secure the old switch into the chassis.

2. Disconnect all of the InfiniBand cables from the front of the switch to be removed.

3. Unlock the switch from the chassis by pushing the red latch release button.

4. Pull the locking arm down to a position perpendicular to the fr ont of the chassis.

5. Pull the switch out of the chassis using the locking arm.

Mellanox Technolog ie s

Installation and Basic OperationRev 1.0

2.2.2 Removing an I/O Module Blank From the Chassis

1. Unlock the I/O Module Blank by pushing the red latch release button.

2. Pull the locking arm down to a position perpendicular to the fr ont of the chassis.

3. Pull the I/O Module Blank out of the chassis using the locking arm.

2.2.3 Installing the Mellanox M4001[F, Q] Switch Into the Dell Chassis

Make sure the rack is stable on a solid floor and that the rack is filled from the bo ttom up. This will keep the center of gravity as low as possible reducing risk of tipping.

Refer to the Dell PowerEdge M1000e Hardware Owners Manual for more information.

1. Follow the instructions in Section 2.2.1 or Section 2.2.2 to rem ove an old switch or an I/O Module Blank.

2. On the new switch, push the red latch release button.

3. Pull the lever forward until the lever is perpendicular to the front panel.

4. Push the switch into the slot until the latching mechanism is against the bar.

5. Push the lever on the latching mechanism up, making sure that the latching mechanism catches the locking bar. The lever should now be parallel to the front panel.

6. Check the indicator lights to make sure the switch has power.

The rack mounting is designed to fit the PowerEdge Chassis. Take precautions to guarantee proper ventilation for air intake at the front of the chassis and exhaust at the rear in order to maintain good airflow at ambient temperature. Cable routing in partic ular should not impede the air exhaust from the chassis.

2.3 Power Connections and Initial Power On

The I/O module will automat ically power up whe n AC power i s appli ed. Ther e is no power swi tch. Immediately upon closing the latching mechanism check to make sure that the green switch LED is lit.

2.4 InfiniBand Cable Installation

All cables can be inserted or removed with the unit powered on. To insert a cable, press the connector into the port receptacle until the connector is firmly seated. The green LED indicator accompanying each port will light when the physical connection is established (that is, when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port). After plugging in a cable, lock the connector using the latching mecha nism particular to the cable vendor. The yellow LED will light if the subnet manager is running (non blinking indicating that no data is being transferred yet). When a logical connection is made the yellow LED will blink signifying data is being transferred.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

To remove, disengage the locks and slowly pull the connector away from the port receptacle. Both LED indicators will tur n off when the cable is unseated.

Care should be taken not to impede the air exhaust flow through the chassis.

Cable lengths should be used whi ch allow f or routin g horizont ally around to the s ide of the chassis before bending upward or dow nward in the rack.

Mellanox Technolog ie s

3 Cluster Management and Firmware

3.1 Network Management and Clustering Software

Download and install, on all nodes, the Mellanox OpenFabric software package for Linux, Windows, or other operating systems from the Mellanox software website:

http://www.mellanox.com >Downloads > InfiniBand SW/Drivers.

This softw are p a cka g e pr ovi d es c onn e ct ivi ty f or s erv er a n d st or a ge sy st em s ut il iz ing H i gh P e r fo rmance Computing (HPC) or enterprise data center (EDC) applications across an InfiniBand fabric. It also provides a Subnet Manager for simple network configuration, network administration, and diagnostic tools for network management.

3.2 Updating Firmware

The switch is delivered with the latest Firmware available at the time of production. New firmware versions will be posted on the Mellanox firmware download page. Firmware can be updated inband using the Mellanox Fi rmware Tools (MFT).

Cluster Management and FirmwareRev 1.0

You will need the Mellanox Firmware Tools package available in MLNX_OFED to update firmware for this switch. It also can be downloaded from:

http://www.mellanox.com > Downloads > Firmware Tools.

The latest firmware can be found at:

http://www.mellanox.com > Support > Dell.

3.2.1 Instructions for Reprogramming Over the InfiniBand Network

To reprogram the switch see http://mellanox.com/content/pages.php?pg=firmware_table_dell

for the latest firmware and the burning instructions.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

4 Troubleshooting

As soon as a switch is inserted in make sure that the power LED comes on.

The power LED for the switch does not come on:

1. Check that the chassis has power.

2. Remove and reinstall the switch.

The status LED for the switch is blinking amber:

1. Remove the switch fr om the chass is and re-ins ert it (verif y that the sw itch is all the way in the chassis and the lever is firmly clos ed).

2. If the amber LED continues to blink, replace the switch.

The link LED for the InfiniBand connector d oes not come on:

1. Check that both ends of the cable are connected.

2. Check that the locks on the ends are secured.

3. Make sure that the latest FW version is installed on both the Mezzanine I/O card and the switch.

4. Make sure that at lea st one blade h as a mat ching Mez zanine I/O ca rd ins tall ed to s upport data flow.

5. If media adapters are used check that the all connections ar e good, tight, and secure.

The activity LED does not come on:

Check that the Subnet Manager has been started.

The power LED for the switch shuts off:

1. Check that the there is adequate ventilatio n.

2. Make sure that there is nothing blocking the front or rear ventilation openings of the chassis.

Mellanox Technolog ie s

Rev 1.0

Appendix A: Specifications

Table 4 - M4001[F, Q] Specification Data

Physical Power and Environmental

H x W x D:

Weight:

Mounting:

SerDes Speeds

Connectors:

12.2 x 1.34 x 10.7 inches

310.7 x 34.1 x 271.8 mm

2.6Kg fully configured Vertically mounted rack

10, 20, 40, 56 Gb/s per port

QSFP 16 external QSFP connectors 16 internal backplane connectors

Typ Power:

FDR

Passive:

Active:

QDR

Passive:

Active:

Max Power:

FDR

Passive:

Active:

QDR

Passive:

Active:

Dissipated Power:

Power through

connector:

Temperature:

Humidity:

80.66W or 275.22B TU s /hr

108.26W or 369.40BTUs/hr

60.42W or 206.16B TU s /hr

88.02W or 300.34B TU s /hr

84.46W or 288.19B TU s /hr

112.06 or 38 2.36BTUs/hr

75.83W or 258.74B TU s /hr

103.43W or 352.92BTUs/hr

2.0 W per port maximum 0°C to 40°C 10% - 90% non-cond ensing

Protocol Support Regulatory Compliance

InfiniBand:

QoS:

Management:

Auto-Negotiation of 10Gb/s, 20Gb/s, or 40 Gb/s, or 56Gb/s

9 InfiniBand Virtual Lanes for all ports 8 Data transport lane s and 1 management lane

Baseboard, Performance, and Device management Agents for full InfiniBand InBand Manageme nt

Safety:

EMC:

ENVIRONMEN-

TAL

US/Canada: cTUVus EU: IEC60950 International: CB TUV Bauart

USA: FCC, Class A Canada: ICES, Class A EU: CE Mark (EN55022 Class A, EN55024, EN61000-3-2, EN61000-3-3) Japan: VCCI, Class A Korea: RRL (MIC), Class A Australia/New Zealand: C-Tick Class A

EU: IEC 60068- 2-64: Random Vibration EU: IEC 60068-2-29: Shocks, Type I / II EU: IEC 60068-2-32: Fall Test

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

Physical Power and Environmental

Scalability and Performance

Switching

Performance:

Simultaneous wire-speed any port to any port

Addressing:

Switchin g C ap a city

48K Unicast Addresses Ma x. per Subnet 16K Multicast Addresses per Subnet

2.56 Tb/s

Mellanox Technolog ie s

Rev 1.0

Appendix B: EMC Certification Statements

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

Appendix B.1: FCC Statements (USA)

Class A Statements:

§ 15.21

Statement Warning! Changes or modifications to this equipment not expressly approved by the party responsible fo r

compliance (Mella nox Tech nologies) could void the user's authority to operate the equipment.

§15.105(a)

Statement

NOTE: This equipment has been tested an d found to co mply with th e limit s for a Cla ss A digi tal devi ce, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruc tion manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense.

B.1.1 EN Statements (Europe)

EN55022 Class A Statement:

Warning! This is a class A product. In a domestic environment this product may cause radio interferenc e, in which case the us e r may b e re quired to take adeq ua t e m ea s ures.

B.1.2 ICES Statements (Canada)

Class A Statement:

“This Class A digital apparatus com p lies with Canadian ICES-003. Cet appareil numérique de la classe A est conforme à la norme NMB-003 du Canada.”

B.1.3 VCCI Statements (Japan)

Class A Statement:

Mellanox Technolog ie s

Rev 1.0

(Translation - "This is a Class A product based on the standard of the Voluntary Control Council for Interference by Information Technology Equipment (VCCI). If this equipment is used in a domestic environment, radio interference may occur, in which case the user may be required to take corrective actions.")

B.1.4 MIC Certification (Korea)

Korea's "Regulation for Certification of Information and Communication Equipment," requires EMC testing and certification for many electronic products. Korean EMC certifications are issued by Radio Research Laboratory (RRL), which is organized under the Ministry of Information and Communications (MIC). EMC testing includes electromagnetic emissions (EMI) and susceptibility (EMS). Certified equipment is labeled with the MIC mark and certification number.

Translation: Class A Device This device is registered for EMC requirements for industrial use. The seller or buyer should

be aware of this. If this type was sold or purchased by mistake, it should be replaced with a residential-use type.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

GND Rx2n

GND

20 21 22 23

24 25 26 27 28 29

30 31

32 33

34 35 36 37

Rx1n

Rx2p

Rx1p

GND

Tx2n

Tx1n

Tx2pTx1p

Tx4n

Tx3n

Tx4p

Tx3p

GND

Rx4n

Rx3n

Rx4p

Rx3p

GND

SDA

SCL

Vcc Rx

ResetL

ModSelLLPMode

Vcc1

VccTx

IntL

ModPrsL

Table 5 - InfiniBand QSFP Connector Pinout

Connector

Pin Number

Connector

Pin Name

Signal Description

1 GND Ground 2 Tx2n Transmitter Inverted Data Input 3 Tx2p Transmitter Non-Inverted Data Input 4 GND Ground 5 Tx4n Transmitter Inverted Data Input 6 Tx4p Transmitter Non-Inverted Data Input 7 GND Ground 8 ModSelL Module Select 9 ResetL Module Reset 10 Vcc Rx +3.3 V Power supply receiver 11 SCL 2-wire serial interface clock 12 SDA 2-wire serial interface data 13 GND Ground 14 Rx3p Receiver Non-Inverted Data Output 15 Rx3n Receiver Inverted Data Output 16 GND Ground 17 Rx1p Receiver Non-Inverted Data Output 18 Rx1n Receiver Inverted Data Output 19 GND Ground 20 GND Ground 21 Rx2n Receiver Inverted Data Output 3 22 Rx2p Receiver Non-Inverted Data Output 3 23 GND Ground 24 Rx4n Receiver Inverted Data Output 3 25 Rx4p Receiver Non-Inverted Data Output 3 26 GND Ground 27 ModPrsL Module Present 28 IntL Interrupt 29 Vcc Tx +3.3 V Power supply transmitter 30 Vcc 1 +3.3 V Power Supply 31 LPMode Low Power Mode 32 GND Ground 33 Tx3p Transmitter Non-Inverted Data Input 34 Tx3n Transmitter Inverted Data Input 35 GND Ground 36 Tx1p Transmitter Non-Inverted Data Input 37 Tx1n Transmitter Inverted Data Input 38 GND Ground

Appendix C: QSFP Interface

Mellanox Technolog ie s

Rev 1.0

18.35

8.50

Top

ResetL

SDA

SCL

ModSelL

RX3p

TX2n

GND

VccRx

TX2p

GND

TX4n

TX4p

GND

RX3n

RX1p

RX1n

RX4p

GND

RX4n

RX2p

RX2n

TX1n

GND

TX1p

GND

TX3n

TX3p

GND

IntL

ModPrsL

VccTx

Vcc1

LPMode

18.35

8.50

Top

ResetL

SDA

SCL

ModSelL

RX3p

TX2n

GND

VccRx

TX2p

GND

TX4n

TX4p

GND

RX3n

RX1p

RX1n

RX4p

GND

RX4n

RX2p

RX2n

TX1n

GND

TX1p

GND

TX3n

TX3p

GND

IntL

ModPrsL

VccTx

Vcc1

LPMode

View into Front of Cage

View into Rear of

Connector

Figure 6: Pinout Looking Into the Rear of the Connector and the Front of the Cage

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

Appendix D: Avertissements de sécurité d’installation

1. Instructions d’installation

Lisez toutes les instructions d’installation avant de brancher le matériel à la source d’alimentation électrique.

2. Température excessive

Ce matériel ne doit pas fonctionner dans une zone avec une température ambiante dépassant le maximum recommandé de 40°C (104°F). En outre, pour garantir un bon écouleme nt de l’ air, laissez au moins 8 cm (3 pouces) d’espa ce libre autour des ouvertures de ventilation.

3. Empilage du châssis

Le châssis ne doit pas être empilé sur un autre matériel. Si le châssis tombe, il peut provoquer des blessures corporelles et des dégradations de biens.

4. Orages – dangers électriques

Pendant un orage, il ne fa ut pas utiliser le matériel et il ne f aut pas brancher ou débrancher les câbles.

5. Branchement/débranchement des câbles InfiniBand en cuivre

Les câbles Infin iBand en c uivre s ont lour ds et ne sont pa s flexible s, il f aut donc faire très attention en les branchant et en les débranchant des connecteurs. Consultez le fabricant des câbles pour connaître les mises en garde et les instructions spéciales.

6. Montage et entretien sur baie

Lorsque ce produit est monté ou entretenu sur baie, il faut prendre des précautions spéciales pour s’assurer que le système reste stable. En général, il faut remplir la baie avec du matériel de bas en haut.

7. Installation du matériel

Ce matériel ne doit être installé, remplacé ou entretenu que par du personnel formé et qualifié.

8. Elimination du matériel

L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur.

Mellanox Technolog ie s

Rev 1.0

9. Codes électriques locaux et nationaux

Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux.

Mellanox Technologies

InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0

Appendix E: Installation - Sicherheitshinweise

1. Installationsanleitungen

Lesen Sie alle Installationsanleitungen, bevor Sie das Gerät an die Stromversorgung anschließen.

2. Übertemperatur

Dieses Gerät sollte nicht in einem Bereich mit einer Umgebungstemperatur über der maximal empfohlenen Temperatur von 40°C (104°F) betrieben werden. Außerdem sollte n mindes tens 8 cm (3 i n.) Fre iraum um d ie Bel üftung söffnungen sein, um einen einwandfreien Luftstrom zu gewährleisten.

3. Stapeln des Chassis

Das Chassis sollte nicht auf andere Gerät e gestapel t werden. Wenn das Chassis herunterfällt, kann es zu Verletzungen und Beschädigungen an Geräten führen.

4. Bei Gewitter - Elektrische Gefahr

Arbeiten Sie während eines Gewitters und Blitzschlag nicht am Gerät, schließen Sie keine Kabel an oder ab.

5. Anschließen/Trennen von InfiniBand-Kupferkabel

InfiniBand-Kupferkabel sind schwer und nicht flexible. Deshalb müssen sie vorsichtig an die Anschlüsse angebracht bzw. davon getrennt werden. Lesen Sie die speziellen Warnungen und Anleitungen des Kabelherstellers.

6. Rack-Montage und Wartung

Wenn dieses Produkt in einem Rack montiert oder gewartet wird, sind besondere Vorsichtsmaßnahmen zu ergreifen, um die Stabilität des Systems zu gewährleisten. Im Allgemeinen sollten Sie das Gestell von unten nach oben mit Geräten füllen.

7. Geräteinstallation

Diese Gerät sollte nur von geschultem und qualifiziertem Personal installiert, ausgetauscht oder gewartet werden.

8. Geräteentsorgung

Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestimmungen erfolgen.

Mellanox Technolog ie s

Rev 1.0

9. Regionale und nationale elektrische Bestimmungen

Dieses Gerät sollte unter Beachtung der regional en und national en elektrisc hen Bestimmungen installiert werden.

Mellanox Technologies

Dell M4001F, M4001Q User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Contents

Revision History

About this Manual

Intended Audience

Related Documentation

Online Resources

Conventions

1Overview

1.1 InfiniBand Connectors

1.2 Switch Status Lights

1.3 I2C Interface RJ-45 Connector

2 Installation and Basic Operation

2.1 Installation Safety Warnings

2.2 Mechanical Installation

2.2.1 Removing an Old Switch From the Chassis

2.2.2 Removing an I/O Module Blank From the Chassis

2.2.3 Installing the Mellanox M4001[F, Q] Switch Into the Dell Chassis

2.3 Power Connections and Initial Power On

2.4 InfiniBand Cable Installation

3 Cluster Management and Firmware

3.1 Network Management and Clustering Software

3.2 Updating Firmware

3.2.1 Instructions for Reprogramming Over the InfiniBand Network

4 Troubleshooting

Appendix A: Specifications

Appendix B: EMC Certification Statements

Appendix C: QSFP Interface

Appendix D: Avertissements de sécurité d’installation

Appendix E: Installation - Sicherheitshinweise