Supermicro AOC-MHIBE-M1CG User Manual

AOC-MHIBE-m1CG

User's Guide

Revision 1.0

The information in this User's Guide has been carefully reviewed and is believed to be accurate.
The vendor assumes no responsibility for any inaccuracies that may be contained in this document,
and makes no commitment to update or to keep current the information in this user's guide, or to
notify any person or organization of the updates. Please Note: For the most up-to-date version

of this user's guide, please see our Website at www.supermicro.com.

Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product
described in this user's guide at any time and without notice. This product, including software and
documentation, is the property of Supermicro and/or its licensors, and is supplied only under a
license. Any use or reproduction of this product is not allowed, except as expressly permitted by
the terms of said license.

IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE FOR DIRECT, INDIRECT,
SPECIAL, INCIDENTAL, SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE
USE OR INABILITY TO USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGES. IN PARTICULAR, SUPER MICRO COMPUTER, INC.
SHALL NOT HAVE LIABILITY FOR ANY HARDWARE, SOFTWARE, OR DATA STORED OR USED
WITH THE PRODUCT, INCLUDING THE COSTS OF REPAIRING, REPLACING, INTEGRATING,
INSTALLING OR RECOVERING SUCH HARDWARE, SOFTWARE, OR DATA.

Any disputes arising between the manufacturer and the customer shall be governed by the laws of
Santa Clara County in the State of California, USA. The State of California, County of Santa Clara
shall be the exclusive venue for the resolution of any such disputes. Supermicro's total liability for
all claims will not exceed the price paid for the hardware product.

FCC Statement: This equipment has been tested and found to comply with the limits for a Class
A digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide
reasonable protection against harmful interference when the equipment is operated in a commercial
environment. This equipment generates, uses, and can radiate radio frequency energy and, if not
installed and used in accordance with the manufacturer’s instruction manual, may cause harmful
interference with radio communications. Operation of this equipment in a residential area is likely
to cause harmful interference, in which case you will be required to correct the interference at your
own expense.

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate
warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate
Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”.

WARNING: Handling of lead solder materials used in this
product may expose you to lead, a chemical known to
the State of California to cause birth defects and other
reproductive harm.

User's Guide Revision 1.0

Release Date: Mar 28, 2018

Unless you request and receive written permission from Super Micro Computer, Inc., you may not
copy any part of this document.

Information in this document is subject to change without notice. Other products and companies
referred to herein are trademarks or registered trademarks of their respective companies or mark
holders.

Copyright © 2018 by Super Micro Computer, Inc.
All rights reserved.

Printed in the United States of America

Preface

About this User's Guide

This user's guide is written for system integrators, PC technicians and

knowledgeable PC users. It provides information for the installation and use of the

AOC-MHIBE-m1CG add-on card.

About this Add-on Card

The AOC-MHIBE-m1CG is the most powerful InniBand controller in the market.

In small form factor SIOM, it provides 100Gbps InniBand EDR or 100Gbps Eth-

ernet for the fastest networking bandwidth in the market, and an additional Gigabit

LAN controller for IPMI remote management. Based on Mellanox® ConnectX-4

VPI and Intel® i210, this add-on card provides the highest performing and most

exible interconnect solution for servers used in Enterprise Data Centers and High-

Performance Computing.

An Important Note to the User

All images and layouts shown in this user's guide are based upon the latest PCB

Revision available at the time of publishing. The card you have received may or

may not look exactly the same as the graphics shown in this user's guide.

Preface

Returning Merchandise for Service

A receipt or copy of your invoice marked with the date of purchase is required before

any warranty service will be rendered. You can obtain service by calling your ven-

dor for a Returned Merchandise Authorization (RMA) number. When returning the

add-on-card to the manufacturer, the RMA number should be prominently displayed

on the outside of the shipping carton, and the shipping package is mailed prepaid

or hand-carried. Shipping and handling charges will be applied for all orders that

must be mailed when service is complete. For faster service, You can also request

a RMA authorization online (http://www.supermicro.com).

This warranty only covers normal consumer use and does not cover damages in-

curred in shipping or from failure due to the alternation, misuse, abuse or improper

maintenance of products.

During the warranty period, contact your distributor rst for any product problems.

iii

AOC-MHIBE-m1CG Add-on Card User's Guide

SMC Confidential

Naming Convention for Networking Adapters

Confidential

Supermicro Networking Adapter List

Conventions Used in the User's Guide

Pay special attention to the following symbols for proper system installation and to

prevent damage to the system or injury to yourself:

Warning: Important information given to ensure proper system installation

or to prevent damage to the components or injury to yourself.

Note: Additional information given to differentiate between various models

or provides information for correct system setup.

Naming Convention

Character Representation Options

1st Product Family AOC: Add On Card

2nd Form Factor S: Standard, P: Proprietary, C: MicroLP,

3rd Product Type/Speed G: GbE (1Gb/s), TG: 10GbE (10Gb/s) , 25G: 25GbE (25Gb/s), 40G:

4th Chipset Model (Optional) N: Niantec (82599), P: Powerville (i350), S: Sageville (X550),

5th Chipset Manufacturer i: Intel, m: Mellanox, b: Broadcom

6th Number of Ports 1: 1 port, 2: 2 ports, 4: 4 ports

7th Connector Type (Optional) S: SFP+/SFP28, T: 10GBase-T, Q: QSFP+, C: QSFP28

8th 2

nd

Controller/Connector Type

(Optional)

M: Super IO Module (SIOM), MH: SIOM Hybrid

40GbE (40Gb/s), 50G: 50GbE (50Gb/s), 100G: 100Gb E (100Gb/s),
IBE: EDR IB (100Gb/s), IBF: FDR IB (56Gb/s), IBQ: QDR IB
(40Gb/s), HFI: Host Fabric Interface

F: Fortville (XL710/X710), L: Lewisburg (PCH)

G: 1x GbE RJ45, 2G: GbE 2x RJ45, S: 1x 10G SFP+, T: 10GBase-T,
2T: 2x 10GBase-T

SMC Networking Add-on Cards

Model Type Form Factor Controller Connection

AOC-MGP -i2 GbE SIOM Intel® i350 AM2 2 RJ45 (1Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 3.7

AOC-MGP -i4 GbE SIOM Intel® i350 AM4 4 RJ45 (1Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 4.4

AOC-MT GN-i2S 10GbE SIOM Intel® 82599ES 2 SFP+ (10 Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 7 .2

AOC-MTG-i4S 10GbE SIOM Intel® XL710-BM 1 4 SFP+ (10Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 7

AOC-MTG-i2T 10GbE SIOM Intel® X550-AT2 2 RJ45 (10GBase-T) 3.622" (92mm) x 3.428" (87.08mm) 13

AOC-MTG-i4T 10GbE SIOM 2x Intel® X550-AT2 4 RJ45 (10GBase-T) 3.622" (92mm) x 3.428" (87.08mm) 26

AOC-MHIBF-m1Q2G

AOC-MHIBF-m2Q2G

AOC-MHIBE-m1 CG

AOC-MH25G-b2S2G 25GbE SIOM

AOC-MH25G-m2S2T 25GbE SIOM

AOC-M25G-m4S 25GbE SIOM Me llanox® ConnectX-4 Lx EN 4 SFP28 (25Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 20

AOC-M25G-i2S 25GbE SIOM Intel® XXV71 0 2 SFP28 (25Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 11.8

AOC-MH FI-i1C

FDR IB

GbE

FDR IB

GbE

EDR IB

GbE

Omni-

Path

SIOM

SIOM

SIOM

SIOM

Mellanox ® ConnectX -3 Pro
Intel® i35 0

Mellanox ® ConnectX -3 Pro
Intel® i35 0

Mellanox ® ConnectX -4 VPI
Intel® i21 0

Broadcom® BCM57414
Intel® i35 0

Mellanox ® ConnectX -4 Lx EN
Intel® X5 50-AT2

Intel® OP HFI ASIC
(Wolf River WFR-B)

iv

Dimension

(w/o Bra ckets) (L x H)

1 QSFP (5 6Gb/port)
2 RJ45 (1Gb/port)

2 QSFP (5 6Gb/port)
2 RJ45 (1Gb/port)

1 QSFP28 (100Gb/port)
1 RJ45 (1Gb/port)

2 SFP28 (25Gb/port)
2 RJ45 (1Gb/port)

2 SFP28 (25Gb/port)
2 RJ45 (10GBase-T)

1 QSFP28 (100Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 15

3.622" (92mm) x 3.428" (87.08mm) 9

3.622" (92mm) x 3.428" (87.08mm) 11

3.622" (92mm) x 3.428" (87.08mm) 19

3.622" (92mm) x 3.428" (87.08mm) 9

3.622" (92mm) x 3.428" (87.08mm) 25

Power

(W)

Confidential

Supermicro Networking Adapter List

Confidential

Supermicro Networking Adapter List

Model T ype Form Factor Interface Controller Connection

Dimension

(w/o Brackets) (L x H)

Power

(W)

Preface

Model T ype Form Factor Interface Controller Connection

AOC-SGP-i2 GbE Standard LP PCI-E x4 Intel® i350 AM2 2 RJ45 (1Gb/port) 3.9” (99mm) x 2.73” (69mm) 3.5

AOC-SGP-i4 GbE Standard LP PCI-E x4 Intel® i350 AM4 4 RJ45 (1Gb/port) 3.9” (99mm) x 2.73” (69mm) 5

AOC-STG-i2T 10GbE Standar d LP PCI-E x8 Intel® X540-AT2 2 RJ45 (10GBase-T) 5.9” (150mm) x 2.73” (69mm) 13

AOC-STGS-i1T 10GbE Standard LP PCI-E x4 Intel® X550-AT 1 RJ45 (10GB ase-T) 5.9” (150mm) x 2.73” (69mm) 9

AOC-STGS-i2T 10GbE Standard LP PCI-E x4 Intel® X550-AT2 2 RJ45 (10GBase-T) 5.9” (150mm) x 2.73” (69mm) 11

AOC-STG-i4T 10GbE Standar d LP PCI-E x8 Intel® XL710-BM1 4 RJ45 (10GBase-T) 5.9” (14.99cm) x 2.73”(6.9cm) 15.5

AOC-STGN-i1S 10GbE Standard LP PCI-E x8 Intel® 82598EN 1 SFP+ (10Gb/port) 4.0” (102mm) x 2.73” (69mm) 10

AOC-STGN-i2S 10GbE Standard LP PCI-E x8 Intel® 82599ES 2 S FP+ (10Gb/port) 4.0” (102mm) x 2.73” (69mm) 11.2

AOC-STGF-i2S 10GbE Standard LP PCI-E x8 Intel® X710-BM2 2 SFP+ (10Gb/port) 5.19” (132mm) x 2.73” (69mm) 5.6

AOC-STG-b4S 10GbE Standard LP PCI-E x8 Broadcom® BCM57840S 4 SFP+ (10Gb/port) 5.4” (137mm) x 2.73” (69mm) 14

AOC-STG-i4S 10GbE Standard LP PCI-E x8 Intel® XL710-BM1 4 SFP+ (10Gb/port) 5.9” (150mm) x 2.73” (69mm) 8

AOC-S25G-m2S 25GbE Standard LP PCI-E x8 Mellanox® CX-4 LX 2 SFP28 (25Gb/port) 5.6" (142mm) x 2.713“ (69mm) 8.7

AOC-S40G-i1Q 40GbE Standard LP PCI-E x8 Intel® XL710-BM1 1 QSFP+ (40Gb/port ) 5.9” (150mm) x 2.73” (69mm) 6.5

AOC-S40G-i2Q 40GbE Standard LP PCI-E x8 Intel® XL710-BM2 2 QSFP+ (40Gb/port ) 5.9” (150mm) x 2.73” (69mm) 7

AOC-S100G-m2C 100GbE Standard LP PCI-E x16 Mellanox® CX-4 EN 2 QSFP28 (100Gb/port) 6.6” (168mm) x 2.73” (69mm) 16.3

AOC-PTG-i1S 10GbE Proprietary PCI-E x8 Intel® 82599EN 1 SFP+ (10Gb/port) 10.04” (255mm) x .78” (20mm) 7.5

AOC-UG-i4 GbE UIO FH PCI-E x8 Intel® 82571EB 4 RJ45 (1Gb/port) 6.6” (167mm) x 3.9” (98mm) 10

AOC-CGP-i2 GbE MicroLP PCI-E x4 Intel® i350 AM2 2 RJ45 (1Gb/port) 4.45” (113mm) x 1.54” (39mm) 4

AOC-CG-i2 GbE MicroLP P CI-E x4 Intel® 82580 2 RJ45 (1Gb/port) 4.45” (113mm) x 1.3” (34mm) 4

AOC-CTG-i1S 10GbE MicroLP PCI-E x8 Intel® 82599EN 1 S FP+ (10Gb/port) 4.85" (123mm) x 1.54" (39mm) 10

AOC-CTG-i2S 10GbE MicroLP PCI-E x8 Intel® 82599ES 2 SFP+ (10Gb/port) 4.85" (123mm) x 1.54" (39mm) 11

AOC-CTG-i2T 10GbE MicroLP PCI-E x8 Intel® X540-AT2 2 RJ45 (10GBase-T) 4.8” (123mm) x 2.75” (77mm) 13

AOC-CTGS-i2T 10GbE MicroLP PCI-E x4 Intel® X550-AT2 2 R J45 (10GBase-T) 4.45” (113mm) x 1.54” (39mm) 12

AOC-C25G-m1S 25GbE MicroLP PCI-E x8 Mellanox® CX -4 Lx EN 1 S FP28 (28Gb/port) 4.45” (113mm) x 1.54” (39mm) 8.5

Dimension

(w/o Brackets) (L x H)

Power

(W)

v

AOC-MHIBE-m1CG Add-on Card User's Guide

Contacting Supermicro

Headquarters

Address: Super Micro Computer, Inc.

980 Rock Ave.

San Jose, CA 95131 U.S.A.

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Technical Support)

Website: www.supermicro.com

Europe

Address: Super Micro Computer B.V.

Het Sterrenbeeld 28, 5215 ML

's-Hertogenbosch, The Netherlands

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Technical Support)

[e-mail] (Customer Support)

Asia-Pacic

Address: Super Micro Computer, Inc.

4F, No. 232-1, Liancheng Rd.

Chung-Ho Dist., New Taipei City 235

Taiwan, R.O.C.

Tel: [phone]

Fax: [phone]

Website: www.supermicro.com.tw

Email: [e-mail] (Technical Support)

Tel: [phone] (Technical Support)

vi

Preface

Table of Contents

Preface

Chapter 1 Overview

1-1 Overview ......................................................................................................... 1-1

1-2 Key Features ................................................................................................... 1-1

1-3 Specications .................................................................................................. 1-1

1-4 InniBand EDR QSFP28 Specications ......................................................... 1-3

1-5 GbE RJ45 Specications ................................................................................ 1-5

1-6 Available SKUs ............................................................................................... 1-7

1-7 Similar Products .............................................................................................. 1-7

Chapter 2 Hardware Components

2-1 Add-On Card Image and Layout ..................................................................... 2-1

2-2 Jumpers and Major Components .................................................................... 2-2

2-3 QSFP28 Ethernet Connections ....................................................................... 2-3

2-4 RJ45 LAN Port and LAN LED indicator .......................................................... 2-4

Chapter 3 Installation

3-1 Static-Sensitive Devices .................................................................................. 3-1

3-2 Before Installation ........................................................................................... 3-2

3-3 Installing the Add-on Card .............................................................................. 3-2

3-4 Installing Drivers on Windows (for Intel® i210) .............................................. 3-5

3-5 Installing Drivers on Linux (for Intel® i210) .................................................... 3-5

3-6 Installing Drivers on FreeBSD (for Intel® i210) .............................................. 3-7

3-7 Installing Drivers (for Mellanox® ConnectX®-4 VPI) ...................................... 3-9

vii

AOC-MHIBE-m1CG Add-on Card User's Guide

Notes

Notes

viii

Chapter 1: Overview

Chapter 1

Overview

1-1 Overview

Congratulations on purchasing your add-on card from an acknowledged leader in

the industry. Supermicro products are designed with the utmost attention to detail

to provide you with the highest standards in quality and performance. For product

support and updates, please refer to our website at http://www.supermicro.com/

products/nfo/networking.cfm#adapter.

1-2 Key Features

The key features of this add-on card include the following:

• Super I/O Module (SIOM) form factor

• Mellanox® ConnectX-4 VPI InniBand EDR controller, single QSFP28 Con-

nector

• Up to 100Gbps InniBand or 100Gbps Ethernet

• Virtual Protocol Interconnect (VPI)

• VXLAN and NVGRE

• Intel® i210 GbE controller, single RJ45 Connector

• SR-IOV for virtualization

• NC-SI for remote management

• Asset Management with thermal sensor

• RoHS compliant 6/6

1-3 Specications

General

• Super I/O Module (SIOM) form factor

1-1

AOC-MHIBE-m1CG Add-on Card User's Guide

• Mellanox

®

ConnectX-4 VPI EDR controller

• Single QSFP28 port with speed up to 100Gbps per port

• Intel® i210 GbE controller

• Single RJ45 port with speed up to 1Gbps per port

Cables Support

• InniBand EDR/100GbE:

• QSFP28 100Gbps copper cable

• Fiber optic cable (with required optional QSFP28 optical transceiver)

• GbE: RJ45 Category 5/5e up to 100m

Note: Please check Supermicro website for supported cable/transceiver

part numbers.

Power Consumption

• Maximum 19W

Operating Condition

• Operating temperature: 0°C to 55°C (32°F to 131°F)

• Storage temperature: -40°C to 70°C (-40°F to 158°F)

• Storage humidity: 90% non-condensing relative humidity at 35°C

Physical Dimensions

• Card PCB dimensions: 92mm (3.62in) x 87.1mm (3.43in) (WxD)

Supported Platforms

• Supermicro® motherboards with Super I/O Module slot

1-2

Chapter 1: Overview

• Supermicro® server systems with Super I/O Module slot (see SIOM Compat-

ibility Matrix online at http://www.supermicro.com/support/resources/AOC/

AOC_Compatibility_SIOM.cfm)

1-4 InniBandEDRQSFP28Specications

Mellanox® ConnectX-4 VPI EDR controller with single QSFP28 port and speed

up to 100Gbps.

InniBand

• IBTA Specication 1.3 compliant

• Hardware-based congestion control

• 256 to 4Kbyte MTU, 2GB messages

Enhanced Features

• Hardware-based reliable transport

• Collective operations ofoads

• Mellanox PeerDirectTM RDMA communication acceleration

• Extended reliable connected transport

• Enhanced atomic operations

Ethernet

• IEEE 802.3bj, 802.3bm 100 Gigabit Ethernet

• IEEE 802.3az Energy Efcient Ethernet

• IEEE 802.3ad, 802.1AX Link Aggregation

• IEEE 1588v2

• Jumbo frame support (9.6KB)

1-3

AOC-MHIBE-m1CG Add-on Card User's Guide

Overlay Networks

• Hardware ofoad of encapsulation and decapsulation of NVGRE and VXLAN

overlay networks

Hardware-based I/O Virtualization

• Single Root IOV

• Multi-function per port

• Address translation and protection

• Multiple queues per virtual machine

• Enhanced QoS for vNICs

• VMware NetQueue support

Virtualization

• SR-IOV: Up to 256 virtual functions

• SR-IOV: Up to 16 physical functions per port

CPUOfoads

• RDMA over Converged Ethernet (RoCE)

• TCP/UDP/IP stateless ofoad

• LSO, LRO, checksum ofoad

• Intelligent interrupt coalescence

Remote Boot

• Remote boot over InniBand

• Remote boot over Ethernet

1-4

Chapter 1: Overview

• Remote boot over iSCSI

• PXE and UEFI

Protocol Support

• OpenMPI, IBM PE, OSU MPI (MVAPICH/2), Intel MPI

• Platform MPI, UPC, Mellanox SHMEM

• TCP/UDP, EoIB, IPoIB, SDP, RDS, MPLS, VxLAN, NVGRE, GENEVE

• SRP, iSER, NFS RDMA, SMB Direct

• uDAPL

Operating System Support

• Linux

• Windows

• FreeBSD

• VMware

1-5 GbERJ45Specications

Intel® i210 GbE controller with Single RJ45 port and speed up to 1Gbps.

PerformanceandEfciencyFeatures

• Energy Efcient Ethernet (EEE)

• Jumbo frames support

• Low latency

• TCP/UDP, IPv4 checksum ofoads

1-5

AOC-MHIBE-m1CG Add-on Card User's Guide

• IPv6 support for IP/TCP and IP/UDP receive checksum ofoad

• Low latency interrupts

Manageability Features

• NC-SI

Operating System Support

• Linux

• Windows

• FreeBSD

• VMware

Note: This product is sold only as part of an integrated solution with

Supermicro server systems.

1-6

1-6 Available SKUs

SKUs Part Number Description

AOC-MHIBE-m1CG

AOC-MHIBE-m1CGM

AOC-MHIBE-m1CG 1-port IB EDR Adapter

BKT-0142L Swappable bracket for 2U+ chassis

AOC-MHIBE-m1CGM

BKT-0141L Internal bracket

1-port IB EDR Adapter

1-7 Similar Products

Chapter 1: Overview

Product Part

Number

AOC-MGP-i2 SIOM 1GbE RJ45 2 Intel® i350

AOC-MGP-i4 SIOM 1GbE RJ45 4 Intel® i350

AOC-MTGN-i2S SIOM 10GbE SFP+ 2 Intel® 82599

AOC-MTG-i4S SIOM 10GbE SFP+ 4 Intel® XL710

AOC-MTG-i2T SIOM 10GbE RJ45 2 Intel® X550

AOC-MTG-i4T SIOM 10GbE RJ45 4 Intel® X550

AOC-MH25G-m2S2G SIOM

AOC-MH25G-b2S2G SIOM

AOC-M25G-m4S SIOM 25GbE SFP28 4 Mellanox® CX4 Lx EN

AOC-MHIBF-m2Q2G SIOM InniBand FDR GbE

AOC-MHIBF-m1Q2G SIOM InniBand FDR GbE

AOC-MHFI-i1C SIOM Omni-Path QSFP28 1 Intel® OP HFI ASIC

Form

Factor

Protocols Connector

25GbE

10GbE

25GbE

1GbE

Type

SFP28

RJ45

SFP28

RJ45

QSFP

RJ45

QSFP

RJ45

Total

Ports

2 2Mellanox® CX4 Lx EN

2 2Broadcom® BCM57414

2 2Mellanox® ConnectX-3 Pro

1 2Mellanox® ConnectX-3 Pro

Controller

Intel® X550

Intel® i350

Intel® i350

Intel® i350

1-7

AOC-MHIBE-m1CG Add-on Card User's Guide

Notes

Notes

1-8

Chapter 2: Hardware Components

8

8

Chapter 2

Hardware Components

2-1 Add-On Card Image and Layout

1

5

2

1-2LAN
2-3IB
NCSI

6

4

3

The AOC-MHIBE-m1CG Image

7

5

LAN1

JPL1:1-2 EN

2-3 DIS

2

Intel i210

MH2

7

4

PRESS FIT

J1

LED1

JP1

6

MH1

3

1

J3J4

SIOM1

Mellanox®

ConnectX-4

VPI

AOC-MHIBE-m1CG

DESIGNED IN USA

REV:1.00

The AOC-MHIBE-m1CG Layout

1. Mellanox® ConnectX-4 VPI 5. LAN1: RJ45 Port1

2. Intel® i210 6. Jumper JP1: Fix address for Nuvoton

3. QSFP1: QSFP28 Port1 7. Jumper J1: NC-SI LAN/IB Port select

4. LED1: QSFP28 Port1 LED 8. Jumper JPL1: RJ45 LAN Port Enable/Disable

2-1

AOC-MHIBE-m1CG Add-on Card User's Guide

2-2 Jumpers and Major Components

The table below include jumpers on the AOC-MHIBE-m1CG:

Jumper Description Default Setting

JP1 Fix address for Nuvoton rmware burn-in Not installed

J1 NC-SI LAN/IB Port select 1-2 LAN

JPL1 RJ45 LAN Port Enable/Disable 1-2 Enable

The following major components are installed on the AOC-MHIBE-m1CG:

1. Intel® i210

2. Mellanox® ConnectX-4 VPI

3. Single RJ45 LAN Port

4. Single QSFP28 (Small Form Factor Pluggable) Port

5. QSFP28 Link/Activity LED

MH1

4

SIOM1

PRESS FIT

J3J4

5

LED1

JP1

1-2LAN
2-3IB
NCSI

J1

3

LAN1

JPL1:1-2 EN

2-3 DIS

1

Intel i210

MH2

2

Mellanox®

ConnectX-4

VPI

AOC-MHIBE-m1CG

REV:1.00

DESIGNED IN USA

2-2

Chapter 2: Hardware Components

2-3 QSFP28 Ethernet Connections

QSFP28 (QSFP1) Connector

One small form-factor pluggable (QSFP28) optical transceiver connector (QSFP1)

is located on the add-on card. Connect a QSFP28 cable to this port to provide

Inniband (100Gb/s) and Ethernet (100GbE) communication. See the layout below

for the locations.

QSFP28 (QSFP1) Link/Activity LED Indicators

One Link/Activity LED indicator is located at LED1 on the add-on card. LED1 is

used for the QSFP28 (QSFP1) connector. See the tables below for the LED states.

Port Activity LED

LED Color Port Assigned

Yellow Active

2

1-2LAN
2-3IB

NCSI

PRESS FIT

J1

LED1

JP1

1

MH1

J3J4

SIOM1

Mellanox®
ConnectX-4

VPI

AOC-MHIBE-m1CG

DESIGNED IN USA

Port Link LED

LED Color Denition

Green Link Established

1. QSFP Connector

2. QSFP Link/Activity
LED Indicator

LAN1

JPL1:1-2 EN

2-3 DIS

Intel i210

REV:1.00

MH2

2-3

AOC-MHIBE-m1CG Add-on Card User's Guide

2-4 RJ45 LAN Port and LAN LED indicator

RJ45 LAN Port (LAN1)

There is one RJ45 LAN port (LAN1) on the AOC-MHIBE-m1CG. The LAN port sup-

ports connection speed of 1Gbps. Use a direct-attach RJ45 type LAN cable. See

the layout below for the location.

Note: Please refer to page 1-2 for recommended cables.

RJ45 LAN Port LED Indicators

The LAN port has two LEDs to indicate speed and data activities. Refer to the tables

below for LED colors and denitions.

RJ45 LAN Port Link LED (Left)

LED State

LED Color Denition

Off No Link

Amber 1 Gbps

Green 100 Mbp

PRESS FIT

LED1

JP1

MH1

J3J4

SIOM1

Link LED

1-2LAN
2-3IB

NCSI

J1

Mellanox®

ConnectX-4

VPI

AOC-MHIBE-m1CG

DESIGNED IN USA

Activity LED

RJ45 LAN Port Activity LED (Right)

LED State

LED Color Status Denition

Off Off No Connection

Green Solid Link

Green Flashing Active

1. RJ45 LAN Port

1

LAN1

JPL1:1-2 EN

2-3 DIS

Intel i210

REV:1.00

MH2

2-4

Chapter 3: Installation

Chapter 3

Installation

3-1 Static-Sensitive Devices

Electrostatic Discharge (ESD) can damage electronic com ponents. To avoid dam-

aging your add-on card, it is important to handle it very carefully. The following

measures are generally sufcient to protect your equipment from ESD.

Precautions

• Use a grounded wrist strap designed to prevent static discharge.

• Touch a grounded metal object before removing the add-on card from the

antistatic bag.

• Handle the add-on card by its edges only; do not touch its components.

• Put the add-on card back into the antistatic bags when not in use.

• For grounding purposes, make sure that your system chassis provides excellent

conductivity between the power supply, the case, the mounting fasteners and

the add-on card.

Unpacking

The add-on card is shipped in antistatic packaging to avoid static damage. When

unpacking your component or system, make sure you are static protected.

Note: To avoid damaging your components and to ensure proper instal-

lation, always connect the power cord last, and always unplug it before

adding, removing or changing any hardware components.

3-1

AOC-MHIBE-m1CG Add-on Card User's Guide

3-2 Before Installation

Before you install the add-on card, follow the instructions below.

1. Power down the system.

2. Unplug the power cord.

3. Use industry-standard anti-static equipment such as gloves or a wrist strap

and follow the precautions on page 3-1 to avoid damage caused by ESD.

4. Familiarize yourself with the server, motherboard, and/or chassis documenta-

tion.

5. Conrm that your operating system includes the latest updates and hotxes.

3-3 Installing the Add-on Card

Follow the steps below to install the add-on card into your system.

1. Remove the server cover and, if any, set aside any screws for later use.

2. Remove the add-on card slot cover. If the slot cover has a screw, place it

aside for later use.

3. Position the add-on card in front of the SIOM slot and gently push in both

sides of the card until it slides into the slot.

Insert the SIOM

SIOM card with the
bracket installed

Note: This add-on card does not support hot plug. Please turn off the AC

power and remove the power cord from the wall socket before you install

or remove the add-on card.

card into this slot

3-2

Chapter 3: Installation

1

2

3

4

4. Secure the add-on card to the chassis. If required, use the screw that you

previously removed.

5. Attach any necessary external cables to the add-on card.

6. Replace the system cover.

7. Plug in the power cord and power up the system.

Follow this step to install the add-on card if your system does not support a swap-

pable bracket. Insert the SIOM card in the motherboard and then install the moth-

erboard in the chassis. An internal bracket comes with the SIOM card 1U chassis

SKU. It needs to be installed onto the chassis.

Insert the SIOM card
into the motherboard

Tighten the screw

Install the bracket
onto the chassis

Install the motherboard
in the chassis

Note: It is recommended that the SIOM card installation above be com-

pleted by a system integrator or the manufacturer.

3-3

AOC-MHIBE-m1CG Add-on Card User's Guide

Follow the steps below to install the add-on card into your system that supports a

swappable bracket. The add-on card must be installed in the swappable bracket

before it can be installed in your system

1. Install the add-on card into the swappable bracket.

2. Position the add-on card in front of the SIOM slot and gently push in both

sides of the card until it slides into the slot.

3. Once the card is in the slot, push both knobs in and turn to the right to lock

the card in the system. The left knob has the unlock/lock symbols next to it.

To ensure that the add-on is locked, make sure that the knob position indica-

tor is pointing to the lock symbol.

3-4

Chapter 3: Installation

3-4 Installing Drivers on Windows (for Intel® i210)

Follow the steps below to install the drivers for Windows operation systems. Down-

load the drivers from Intel Support website or from Supermicro CDR-NIC LAN driver

CD or FTP site at ftp://ftp.supermicro.com/Networking_Drivers/.

1. Run CDR-NIC.

2. When the SUPERMICRO window appears, click on the computer icon next to

the product model.

3-5 Installing Drivers on Linux (for Intel® i210)

Follow the steps below to install the drivers for Linux.

Build a Binary RPM Package

1. Run ‘rpmbuild -tb <lename.tar.gz>’

2. Replace <lename.tar.gz> with the specic lename of the driver.

Note: For the build to work properly, the current running kernel MUST

match the version and conguration of the installed kernel sources. If you

have just recompiled the kernel, reboot the system at this time. Follow the

instructions below to build the driver manually.

3-5

AOC-MHIBE-m1CG Add-on Card User's Guide

Follow the instructions below to build the driver manually.

1. Move the base driver tar le to the directory of your choice. For example:

/home/username/igb

or

/usr/local/src/igb

2. Untar/unzip archive, where <x.x.x> is the version number for the driver tar le:

tar xvzf igb-x.x.x.tar.gz

3. Change to the driver src directory, where <x.x.x> is the version number for

the driver tar:

cd igb-x.x.x/src/

4. Compile the driver module:

make install

The binary will be installed as:

/lib/modules/[KERNEL_VERSION]/kernel/drivers/net/ethernet/

intel/ixgbe/ixgbe.ko.xz

The install locations listed above are the default locations. They may not be

correct for certain Linux distributions. For more information, see the ldistrib.txt

le included in the driver tar.

Note: IXGBE_NO_LRO is a compile time ag. The user can enable it at

compile time to remove support for LRO from the driver. The ag is used

by adding CFLAGS_EXTRA=-”DIXGBE_NO_LRO” to the make le when

it’s being compiled.

make CFLAGS_EXTRA=”-DIXGBE_NO_LRO” install

5. Load the module:

For kernel 2.6.x, use the modprobe command:

modprobe ixgbe <parameter>=<value>

3-6

Chapter 3: Installation

For 2.6 kernels, the insmod command can be used if the full path to the driver

module is specied. For example:

insmod /lib/modules/[KERNEL_VERSION]/kernel/drivers/net/

ethernet/intel/ixgbe/ixgbe.ko.xz

For more driver installation information, please refer to Intel Support website.

3-6 Installing Drivers on FreeBSD (for Intel® i210)

Follow the instructions below to install the drivers for FreeBSD kernel 4.8 or later.

In the instructions below, x.x.x is the driver version as indicated in the name of the

drive tar le.

Note: You must have kernel sources installed in order to compile the

driver module.

Move the base driver tar le to the dirctory of your choice. For example, use

/home/username/ixgb or /usr/local/src/ixgb.

6. Untar/unzip the archive:

tar xfz ixgb-x.x.x directory

7. To install man page:

cd ixgb-x.x.x

gzip -c ixgb.4 > /usr/share/man/man4/ixgb.4.gz

8. To load the driver onto a running system, perform the following steps:

cd ixgb-x.x.x

make

or

cd ixgb-x.x.x/src

make load

9. To assign an IP address to the interface, enter the following:

ifcong ixgb<interface_num> <IP_address>

10. Verify that the interface works. Enter the following, where <IP_address> is the

IP address for another machine on the same subnet as the interface that is

being tested:

ping <IP_address>

3-7

AOC-MHIBE-m1CG Add-on Card User's Guide

11. If you want the driver to load automatically when the system is booted:

cd ixgb-x.x.x/src

make load

cp if_ixgb.ko /modules

Edit /boot/loader.conf, and add the following line:

if_ixgb_load="YES"

or

compile the driver into the kernel (see item 8). Edit /etc/rc.conf, and create the

appropriate ifcong_ixgb<interface_num> entry:

ifcong_ixgb<interface_num>="<ifcong_settings>"

Example usage:

ifcong_ixgb0="inet 192.168.10.1 netmask 255.255.255.0"

12. If you want to compile the driver into the kernel, enter:

cd ixgb-x.x.x/src

mkdir /usr/src/sys/dev/ixgb

cp if_ixgb* /usr/src/sys/dev/ixgb

cp ixgb* /usr/src/sys/dev/ixgb

cp Makele.kernel /usr/src/sys/modules/ixgb/Makele

Edit the /usr/src/sys/conf/les.i386 le, and add the following line:

dev/ixgb/ixgb_hw.c optional ixgb

dev/ixgb/ixgb_ee.c optional ixgb

dev/ixgb/if_ixgb.c optional ixgb

Remove the following lines from the /usr/src/sys/conf/les.i386 le, if they exist:

/dev/ixgb/if_ixgb_fx_hw.c optional ixgb

/dev/ixgb/if_ixgb_phy.c optional ixgb

Edit the kernel conguration le (i.e., GENERIC or MYKERNEL) in /usr/src/sys/

i386/conf, and ensure the following line is present:

device ixgb

Compile and install the kernel. Reboot the system for the kernel updates to

take affect.

3-8

Chapter 3: Installation

3-7 Installing Drivers (for Mellanox® ConnectX®-4 VPI)

Use the procedures below to install drivers for Linux.

Linux Drivers

Use the following procedures to install drivers on the Linux operating system.

Installing InniBand Drivers for the Linux Operating System

1. Go to Mellanox Support website to download the driver or from Supermicro

CDR-NIC LAN driver CD or FTP site, go to the following directory:

Mellanox > InniBand > Linux.

2. Choose the InniBand Linux driver package le.

3. Install the driver by entering the following commands:

tar xzvf MLNX_OFED_LINUX-<ver>.tgz

cd MLNX_OFED_LINUX–<ver>

./mlnxofedinstall --without-fw-update

This installs the Linux drivers to your system. For more driver installation infor-

mation, please refer to Mellanox Support website.

Windows Drivers

Use the following procedures to install drivers on the Windows operating system.

Installing InniBand Drivers for the Windows Operating System

1. Go to Mellanox Support website to download the driver or from the

Supermicro CDR-NIC LAN driver CD or FTP site, go to the following direc-

tory: Mellanox > InniBand > Windows.

2. Choose the desired InniBand Windows driver package le.

3. Double-click to run and install the driver package le.

3-9

AOC-MHIBE-m1CG Add-on Card User's Guide

Notes

Notes

3-10

(Disclaimer Continued)

The products sold by Supermicro are not intended for and will not be used in life support systems,
medical equipment, nuclear facilities or systems, aircraft, aircraft devices, aircraft/emergency com­munication devices or other critical systems whose failure to perform be reasonably expected to result

in signicant injury or loss of life or catastrophic property damage. Accordingly, Supermicro disclaims

any and all liability, and should buyer use or sell such products for use in such ultra-hazardous ap­plications, it does so entirely at its own risk. Furthermore, buyer agrees to fully indemnify, defend
and hold Supermicro harmless for and against any and all claims, demands, actions, litigation, and
proceedings of any kind arising out of or related to such ultra-hazardous use or sale.