Dell MCQH29-XDR User Manual

Dell and the Dell logo are
trademarks of Dell Inc.

ConnectX®-2 Dual Port I/O Card for Dell C6100

User Manual

MCQH29-XDR

Rev 1.0

www.mellanox.com

NOTE:
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTA TION ARE PRO­VIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF
AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUS
TOMER'S MANUF ACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES
TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CAN
NOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY.
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MER­CHANT ABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT
SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEM­PLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCURE­MENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED
DOCUMENT ATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-

-

Mellanox Technologies
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: [phone]
Fax: [phone]

© Copyright 2010. Mellanox Technologies, Inc. All Rights Reserved.
Mellanox®, ConnectX®, BridgeX®, InfiniBlast®, InfiniBridge®, InfiniHost®, InfiniRISC®, InfiniScale®, and InfiniPCI® are
registered trademarks of Mellanox Technologies, Ltd.
PhyX and Virtual Protocol Interconnect are trademarks of Mellanox Technologies, Ltd.

All other marks and names mentioned herein may be trademarks of their respective companies.

ConnectX®-2 Dual Port VPI I/O Card for Dell C6100

2

Mellanox Technologies

Mellanox Technologies, Ltd.
PO Box 586 Hermon Building
Yokneam 20692
Israel
Tel: [phone]
Fax: [phone]

Document Number: 3198

ConnectX®-2 Dual Port VPI I/O Card for Dell C6100 Rev 1.0

Table of Contents

Chapter 1 Overview 8

1.1 I/O Card 8

1.2 Finding the GUID/MAC and Serial Number on the Adapter Cards 9

Chapter 2 I/O Card Interfaces 10

2.1 I/O Interfaces 10

2.1.1 InfiniBand Interface 10

2.1.2 Ethernet Interface 10

2.2 LED Assignment 11

2.3 PCI Express Interface 11

2.4 Memory 11

2.4.1 System Memory 11

2.4.2 EEPROM 12

2.4.3 Flash 12

2.5 EEPROM VPD 12

Chapter 3 Driver Software and Firmware 14

3.1 Driver Software 14

3.1.1 Linux 14

3.1.2 Windows 14

3.2 Updating Card Firmware 14

3.3 FlexBoot 14

Chapter 4 I/O Card Installation 15

4.1 Hardware and Software Requirements 15

4.2 Installation Kit 15

4.3 Installation Instructions 16

4.4 Safety Warnings 19

Chapter 5 Cables and Modules 21

5.1 Cable Installation 21

5.1.1 Inserting a Cable into the Card 21

5.1.2 Removing a Cable from the Card 22

Appendix A Specifications 23
Appendix B QSFP Interface 27
Appendix C Avertissements de sécurité d’installation (French) 29
Appendix D Installation - Sicherheitshinweise (German) 30

Mellanox Technologies

3

Rev 1.0

List of Figures

Figure 1: Card Product Labels 9
Figure 2: PCI Connection 10
Figure 3: Front Panel 11
Figure 4: Flash Jumper 12
Figure 5: Installation Kit Parts 16
Figure 6: Removing From the Server 17
Figure 7: Top View 17
Figure 8: Install the Plastic Leg onto the Board 18
Figure 9: Installing the Connector Board into the HCA Card 18
Figure 10: Dell Card Installed 18
Figure 11: Front Panel 19
Figure 12: Top View 19
Figure 13: Connector Orientation 22
Figure 14: Mechanical Drawing 23
Figure 15: QSFP Connector Male and Female Views 28

4

Mellanox Technologies

ConnectX®-2 Dual Port VPI I/O Card for Dell C6100 Rev 1.0

List of Tables

Table 1: Revision History Table 6
Table 2: Documents List 7
Table 3: ConnectX-2 I/O Card Details 8
Table 4: VPI Support 10
Table 5: Jumper Configuration 12
Table 6: MCQH29-XDR VPD 12
Table 7: Hardware and Software Requirements 15
Table 8: ConnectX® Dual-Port InfiniBand I/O Card Installation Kit 16
Table 9: ConnectX-2 MCQH29-XDR Specifications 26

Mellanox Technologies

5

Rev 1.0

Revision History

This document was first printed on 5/6/10.

Table 1 - Revision History Table

Date Rev Comments/Changes

April 2010 1.0 Initial release

6

Mellanox Technologies

ConnectX®-2 Dual Port VPI I/O Card for Dell C6100 Rev 1.0

About this Manual

This User Manual describes ConnectX-2 Dual Port VPI I/O cards for the Dell C6100 chassis.

It provides details as to the interfaces of the board, specifications, required software and firmware
for operating the cards, installation instructions, and relevant documentation.

Intended Audience

This manual is intended for the installer and user of the I/O cards.

The manual assumes basic familiarity with the Infiniband

Related Documentation

Table 2 - Documents List

InfiniBand® Architecture Specification Volume 1 Release 1.2 and Volume 2 release

1.2.1– Infiniband Architecture Specifications Descriptions
PCI Express Base 2.0 Specification (1.1 compatible)
PCI Local Bus Specification Rev 2.3

Online Resources

• Mellanox Technologies Web pages: http://www.mellanox.com

• Dell Support Web pages: http://support.dell.com

Document Conventions

®

architecture specifications.

These symbols indicate a situation, status, or condition that may cause harm
to people or damage to the equipment.

When discussing memory sizes, MB and MBytes are used in this document to mean size in mega
bytes. The use of Mb or Mbits (small b) indicates size in mega bits.

Mellanox Technologies

7

1 Overview

This document is a User Manual for the Mellanox ConnectX-2 20 and 40Gb/s InfiniBand /
10GigE Ethernet dual port QSFP I/O cards for the Dell C6100 chassis.

The cards described in this manual have the following main features:

•1μs MPI ping latency

• Selectable 10, 20, or 40Gb/s InfiniBand or 10GigE per port

• CPU offload of transport operations

• End-to-end QoS and congestion control

• Hardware-based I/O virtualization

• PCI Express 2.0 (up to 5GT/s)

• TCP/UDP/IP stateless offload

• Fibre Channel encapsulation (FCoIB or FCoE)

1.1 I/O Card

OverviewRev 1.0

Table 3 - ConnectX-2 I/O Card Details

Ordering Part

Number (OPN)

MCQH29-XDR PCIe Gen2

PCI Express

SERDES

Speed

5.0 GT/s

Transmission

40 Gb/s QDR

IB Data

Rate

InfiniBand

Eth Data

Transmission

Rate

10 Gb/s

RoHS Adapter IC Part Number

R-6 MT25408B0-FCCR-QI

8

Mellanox Technologies

ConnectX®-2 Dual Port VPI I/O Card for Dell C6100 Rev 1.0

IL-0JR3P1-74031-980-0001

REV:A00

Made in IL

P/N: 0JR3P1 REV: X1

Made in IL

GUID: 0024E89097FF24DC

MAC: 0002C9011226 FC

Model No: MCQH29-XDR

1.2 Finding the GUID/MAC and Serial Number on the Adapter Cards

All cards have a label on the printed side of the card that has the card serial number, the card MAC
for Ethernet protocol, and the card GUID for InfiniBand protocol. VPI Cards have both a MAC
and a GUID.

Figure 1: Card Product Labels

Mellanox Technologies

9