Supermicro AOC-MHIBF-m2Q2G, AOC-MHIBF-m1Q2G User Manual

AOC-MHIBF-m2Q2G

AOC-MHIBF-m1Q2G

User's Guide

Revision 1.0b

The information in this User's Guide has been carefully reviewed and is believed to be accurate.

!

The vendor assumes no responsibility for any inaccuracies that may be contained in this document,
and makes no commitment to update or to keep current the information in this user's guide, or to
notify any person or organization of the updates. Please Note: For the most up-to-date version

of this user's guide, please see our Website at www.supermicro.com.

Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product
described in this user's guide at any time and without notice. This product, including software and
documentation, is the property of Supermicro and/or its licensors, and is supplied only under a
license. Any use or reproduction of this product is not allowed, except as expressly permitted by
the terms of said license.

IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE FOR DIRECT, INDIRECT,
SPECIAL, INCIDENTAL, SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE
USE OR INABILITY TO USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGES. IN PARTICULAR, SUPER MICRO COMPUTER, INC.
SHALL NOT HAVE LIABILITY FOR ANY HARDWARE, SOFTWARE, OR DATA STORED OR USED
WITH THE PRODUCT, INCLUDING THE COSTS OF REPAIRING, REPLACING, INTEGRATING,
INSTALLING OR RECOVERING SUCH HARDWARE, SOFTWARE, OR DATA.

Any disputes arising between the manufacturer and the customer shall be governed by the laws of
Santa Clara County in the State of California, USA. The State of California, County of Santa Clara
shall be the exclusive venue for the resolution of any such disputes. Supermicro's total liability for
all claims will not exceed the price paid for the hardware product.

FCC Statement: This equipment has been tested and found to comply with the limits for a Class
A digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide
reasonable protection against harmful interference when the equipment is operated in a commercial
environment. This equipment generates, uses, and can radiate radio frequency energy and, if not
installed and used in accordance with the manufacturer’s instruction manual, may cause harmful
interference with radio communications. Operation of this equipment in a residential area is likely
to cause harmful interference, in which case you will be required to correct the interference at your
own expense.

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate
warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate
Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”.

WARNING: This product can expose you to chemica ls including
lead, known to the State of California to cause ca ncer and birth
defects or other reproductive harm. For more information, go
to www.P65Warnings.ca.gov.

The products sold by Supermicro are not intended for and will not be used in life support systems,
medical equipment, nuclear facilities or systems, aircraft, aircraft devices, aircraft/emergency
communication devices or other critical systems whose failure to perform be reasonably expected

to result in signicant injury or loss of life or catastrophic property damage. Accordingly, Supermicro

disclaims any and all liability, and should buyer use or sell such products for use in such ultra­hazardous applications, it does so entirely at its own risk. Furthermore, buyer agrees to fully
indemnify, defend and hold Supermicro harmless for and against any and all claims, demands,
actions, litigation, and proceedings of any kind arising out of or related to such ultra-hazardous
use or sale.

User's Guide Revision 1.0b

Release Date: May 16, 2019

Unless you request and receive written permission from Super Micro Computer, Inc., you may not
copy any part of this document.

Information in this document is subject to change without notice. Other products and companies
referred to herein are trademarks or registered trademarks of their respective companies or mark
holders.

Copyright © 2019 by Super Micro Computer, Inc.

All rights reserved.

Printed in the United States of America

Preface

About this User's Guide

This user's guide is written for system integrators, PC technicians, and

knowledgeable PC users. It provides information for the installation and use of the

AOC-MHIBF-m2Q2G and AOC-MHIBF-m1Q2G add-on cards.

About this Add-on Card

The AOC-MHIBF-m2Q2G and AOC-MHIBF-m1Q2G are the most powerful Inni-

Band controllers in the market. In the small form factor SIOM, available as part of

an integrated solution with Supermicro servers, they provide both InniBand FDR

56Gbps and 40Gbps Ethernet as well as remote server management through a

dedicated Gigabit LAN controller. Based on Mellanox® ConnectX-3 Pro and Intel®

i350 controllers, they provide unparalleled performance and exible interconnect

solutions for Supermicro servers in data centers and enterprise environments.

An Important Note to the User

All images and layouts shown in this user's guide are based upon the latest PCB

revision available at the time of publishing. The card you have received may or may

not look exactly the same as the graphics shown in this user's guide.

Preface

Returning Merchandise for Service

A receipt or copy of your invoice marked with the date of purchase is required before

any warranty service will be rendered. You can obtain service by calling your ven-

dor for a Returned Merchandise Authorization (RMA) number. When returning the

add-on card to the manufacturer, the RMA number should be prominently displayed

on the outside of the shipping carton, and the shipping package is mailed prepaid

or hand-carried. Shipping and handling charges will be applied for all orders that

must be mailed when service is complete. For faster service, You can also request

a RMA authorization online (http://www.supermicro.com).

This warranty only covers normal consumer use and does not cover damages in-

curred in shipping or from failure due to the alternation, misuse, abuse or improper

maintenance of products.

During the warranty period, contact your distributor rst for any product problems.

iii

AOC-MHIBF-m2Q2G/m1Q2G Add-on Card User's Guide

Confidential

Naming Convention for Network Adapters

Confidential

Supermicro Networking Adapter List

Conventions Used in the User's Guide

Pay special attention to the following symbols for proper system installation and to

prevent damage to the system or injury to yourself:

Note: Additional information given to provide information for correct sys-

tem setup.

Naming Convention

Character Representation Options

1st Product Family AOC: Add On Card

2nd Form Factor S: Standard, P: Proprietary, C: MicroLP,

3rd Product Type/Speed G: GbE (1Gb/s), TG: 10GbE (10Gb/s), 25G : 25GbE (25Gb/s), 40G:

4th Chipset Model (Optional) N: Niantec (82599), P: Powerville (i350), S: Sageville (X550)

5th Chipset Manufacturer i: Intel, m: Mellanox, b: Broadcom

6th Number of Ports 1: 1 port, 2: 2 ports, 4: 4 ports

7th Connector Type (Optional) S: SFP+/SFP28, T: 10GBase-T, Q: QSFP+, C: QSFP28

8th 2

nd

Controller/Connector Type

(Optional)

M: Super IO Module (SIOM), MH: SIOM Hybrid

40GbE (40Gb/s), 50G: 50GbE (50Gb/s), 100G: 100GbE (100G b/s),
IBE: EDR IB (100Gb/s), IBF: FDR IB (56Gb/s), IBQ: QDR IB
(40Gb/s), HFI: Host Fabric Interface

G: 1x GbE RJ45, 2G: GbE 2x RJ45, S: 1x 10G SFP+, T: 10GBase-T,
2T: 2x 10GBase-T

SMC Networking Add-on Cards

Model Type Form Factor Controller Connection

AOC-MGP-i2 GbE SIOM Intel® i350 AM2 2 RJ45 (1Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 3.7

AOC-MGP-i4 GbE SIOM Intel® i350 AM4 4 RJ45 (1Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 4.4

AOC-MTGN-i2S 10GbE SIOM Intel® 82599ES 2 SFP+ (10Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 7.2

AOC-MTG-i4S 10GbE SIOM Intel® XL710-BM1 4 SFP+ (10Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 7

AOC-MTG-b2T 10GbE SIOM Broadcom® BCM57416 2 RJ45 (10GBase-T) 3.622" (92mm) x 3.428" (87.08mm) 11

AOC-MTG-i2T 10GbE SIOM Intel® X550-AT2 2 RJ45 (10GB ase-T) 3.622" (92mm) x 3.428" (87.08mm) 13

AOC-MTG-i4T 10GbE SIOM 2x Intel® X550-AT2 4 RJ45 (10GBase-T) 3.622" (92mm) x 3.428" (87.08mm) 26

AOC-MHIBF-m1Q2G

AOC-MHIBF-m2Q2G

AOC-MHIBE-m1CG

AOC-MH25G-b2S2G 25GbE SIOM

AOC-MH25G-m2S2T 25GbE SIOM

AOC-M25G-m4S 25GbE SIOM Mellanox® ConnectX-4 Lx EN 4 SFP28 (25Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 20

AOC-M25G-i2S 25GbE SIOM Intel® XXV710 2 SFP28 (25Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 11.8

AOC-MHFI-i1C

FDR IB

GbE

FDR IB

GbE

EDR IB

GbE

Omni-

Path

SIOM

SIOM

SIOM

SIOM

Mellanox® ConnectX-3 Pro
Intel® i350

Mellanox® ConnectX-3 Pro
Intel® i350

Mellanox® ConnectX-4 VPI

Intel® i210

Broadcom® BCM57414
Intel® i350

Mellanox® ConnectX-4 Lx EN
Intel® X550-AT2

Intel® OP HFI ASIC
(Wolf River WFR-B)

1 QSFP (56Gb/port)
2 RJ45 (1Gb/port)

2 QSFP (56Gb/port)
2 RJ45 (1Gb/port)

1 QSFP28 (100Gb/port)

1 RJ45 (1Gb/port)

2 SFP28 (25Gb/port)
2 RJ45 (1Gb/port)

2 SFP28 (25Gb/port)
2 RJ45 (10GBase-T)

1 QSFP28 (100Gb/port) 3.622" (92mm) x 3.428" (87.08mm) 15

iv

Dimension

(w/o Brackets) (L x H)

3.622" (92mm) x 3.428" (87.08mm) 9

3.622" (92mm) x 3.428" (87.08mm) 11

3.622" (92mm) x 3.428" (87.08mm) 19

3.622" (92mm) x 3.428" (87.08mm) 9

3.622" (92mm) x 3.428" (87.08mm) 25

Power

(W)

Confidential

Supermicro Networking Adapter List

Confidential

Supermicro Networking Adapter List

Model T ype Form Factor Interface Controller Connection

Dimension

(w/o Brackets) (L x H)

Power

(W)

Preface

Model T ype Form Factor Interface Controller Connection

AOC-SGP-i2 GbE Standard LP PCI-E x4 Intel® i350 AM2 2 RJ45 (1Gb/port) 3.9” (99mm) x 2.73” (69mm) 3.5

AOC-SGP-i4 GbE Standard LP PCI-E x4 Intel® i350 AM4 4 RJ45 (1Gb/port) 3.9” (99mm) x 2.73” (69mm) 5

AOC-STG-i2T 10GbE Standard LP PCI-E x8 Intel® X540-AT2 2 RJ45 (10GBase-T) 5.9” (150mm) x 2.73” (69mm) 13

AOC-STGS-i1T 10GbE Standar d LP PCI-E x4 Intel® X550-AT 1 RJ45 (10GBase-T) 5.9” (150mm) x 2.73” (69mm) 9

AOC-STGS-i2T 10GbE Standar d LP PCI-E x4 Intel® X550-AT2 2 RJ45 (10GBase-T) 5.9” (150mm) x 2.73” (69mm) 11

AOC-STG-b2T 10GbE Standard LP PCI-E x8 Broadcom® BCM 57416 2 RJ45 (10GBase-T) 5.6” (142mm) x 2.73”(69mm) 13. 1

AOC-STG-i4T 10GbE Standard LP PCI-E x8 Intel® XL710-BM1 4 RJ45 (10GBase-T) 5.9” (149mm) x 2.73”(69mm) 15.5

AOC-STGN-i1S 10GbE Standard LP PCI-E x8 Intel® 82598EN 1 SFP+ (10Gb/port) 4.0” (102mm) x 2.73” (69mm) 10

AOC-STGN-i2S 10GbE Standard LP PCI-E x8 Intel® 82599ES 2 SFP+ (10Gb/port) 4.0” (102mm) x 2.73” (69mm) 11.2

AOC-STGF-i2S 10GbE Standar d LP PCI-E x8 Intel® X710-BM 2 2 SFP+ (10Gb/port) 5.19” (132mm ) x 2.73” (69mm) 5.6

AOC-STG-b4S 10GbE Standard LP PCI-E x8 Broadcom® BCM57840S 4 SFP+ (10Gb/port) 5.4” (137mm) x 2.73” (69mm) 14

AOC-STG-i4S 10GbE Standard LP PCI-E x8 Intel® XL710-BM1 4 SFP+ (10Gb/port) 5.9” (150mm) x 2.73” (69mm) 8

AOC-S25G-m2S 25GbE Standard LP PCI-E x8 Mellanox® CX-4 LX 2 SFP28 (25Gb/port) 5.6" (142mm) x 2.713“ (69mm) 8.7

AOC-S25G-b2S 25GbE Standar d LP PCI-E x8 Br oadcom® BCM57414 2 SFP28 (25Gb/port) 5.6" (142mm) x 2.713“ (69mm) 5.2

AOC-S25G-i2S 25GbE Standard LP PCI-E x8 Intel® XXV710 2 SFP28 (25Gb/port) 6.1" (155mm) x 2.713“ (69mm) 7.2

AOC-CGP-i2 GbE MicroLP PCI-E x4 Intel® i350 AM2 2 RJ45 (1Gb/port) 4.45” (113mm) x 1.54” (39mm) 4

AOC-S40G-i1Q 40GbE Standard LP PCI-E x8 Intel® XL710-BM1 1 QSFP+ (40Gb/port)

AOC-CTG-i1S 10GbE MicroLP PCI-E x8 Intel® 82599E N 1 SFP+ (10Gb/port) 4.85" (123mm) x 1.54" (39mm) 10

AOC-S40G-i2Q 40GbE Standard LP PCI-E x8 Intel® XL710-BM2 2 QSFP+ (40Gb/port) 5.9” (150mm) x 2.73” (69mm) 7

AOC-CTG-i2S 10GbE MicroLP PCI-E x8 Intel® 82599E S 2 SFP+ (10Gb/port) 4.85" (123mm) x 1.54" (39mm) 11

AOC-CTG-i2T 10GbE M icroLP PCI-E x8 Intel® X540-AT2 2 RJ45 (10GBase-T) 4.8” (123mm) x 2.75” (77mm) 13

AOC-CTGS-i2T 10GbE MicroLP PCI-E x4 Intel® X550-AT2 2 RJ45 (10GBase-T) 4.45” (113mm) x 1.54” (39mm) 12

AOC-C25G-m1S 25GbE M icroLP PCI-E x8 Mellanox® CX-4 Lx EN 1 SFP28 (28Gb/port) 4.45” (113mm) x 1.54” (39mm) 8.5

Dimension

(w/o Brackets) (L x H)

5.9” (150mm) x 2.73” (69mm) 6.5

Power

(W)

v

AOC-MHIBF-m2Q2G/m1Q2G Add-on Card User's Guide

Contacting Supermicro

Headquarters

Address: Super Micro Computer, Inc.

980 Rock Ave.

San Jose, CA 95131 U.S.A.

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Technical Support)

Website: www.supermicro.com

Europe

Address: Super Micro Computer B.V.

Het Sterrenbeeld 28, 5215 ML

's-Hertogenbosch, The Netherlands

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Technical Support)

[e-mail] (Customer Support)

Asia-Pacic

Address: Super Micro Computer, Inc.

4F, No. 232-1, Liancheng Rd.

Chung-Ho Dist., New Taipei City 235

Taiwan, R.O.C.

Tel: [phone]

Fax: [phone]

Website: www.supermicro.com.tw

Email: [e-mail] (Technical Support)

Tel: [phone] (Technical Support)

vi

Table of Contents

Table of Contents

Preface

Chapter 1 Overview

1-1 Overview ......................................................................................................... 1-1

1-2 Key Features ................................................................................................... 1-1

1-3 Specications .................................................................................................. 1-1

1-4 Available SKUs ............................................................................................... 1-7

1-5 Similar Products .............................................................................................. 1-7

1-6 Optional Accessories ....................................................................................... 1-8

Chapter 2 Hardware Components

2-1 Add-On Card Image and Layout ..................................................................... 2-1

2-2 Major Components .......................................................................................... 2-3

2-3 QSFP Ethernet Connections and LED indicators ........................................... 2-4

2-4 RJ45 LAN Ports and LAN LED indicators ...................................................... 2-6

2-5 RJ45 LAN Ports Enable/Disable ..................................................................... 2-7

Chapter 3 Installation

3-1 Static-Sensitive Devices .................................................................................. 3-1

3-2 Before Installation ........................................................................................... 3-2

3-3 Installing the Add-on Card .............................................................................. 3-2

3-4 Installing Drivers on Windows (for Intel® i350) .............................................. 3-5

3-5 Installing Drivers on Linux (for Intel® i350) .................................................... 3-5

3-6 Installing Drivers on FreeBSD (for Intel® i350) .............................................. 3-7

3-7 Installing Drivers (for Mellanox® ConnectX®-3 Pro) .................................... 3-10

3-8 Changing from InniBand to Ethernet mode .................................................3-11

3-9 Using Mellanox Controller for PXE Boot ....................................................... 3-15

vii

Chapter 1: Overview

Chapter 1

Overview

1-1 Overview

Congratulations on purchasing your add-on card from an acknowledged leader in

the industry. Supermicro products are designed with the utmost attention to detail

to provide you with the highest standards in quality and performance. For product

support and updates, please refer to our website at http://www.supermicro.com/

products/nfo/networking.cfm#adapter.

1-2 Key Features

The key features of this add-on card include the following:

• Super I/O Module (SIOM) form factor

• Mellanox® ConnectX-3 Pro InniBand FDR controller

• Dual and single QSFP connector models

• Up to 56Gbps InniBand or 40Gbps Ethernet

• Virtual Protocol Interconnect (VPI)

• VXLAN and NVGRE

• Intel® i350 GbE controller

• Dual RJ45 connectors

• NC-SI for remote management

• Asset Management features with thermal sensor

• RoHS compliant 6/6

1-3 Specications

General

• Super I/O Module (SIOM) form factor

1-1

AOC-MHIBF-m2Q2G/m1Q2G Add-on Card User's Guide

• Mellanox® ConnectX-3 Pro FDR controller

• Intel® i350 GbE controller

• Dual QSFP ports and dual RJ45 ports (AOC-MHIBF-m2Q2G)

• Single QSFP port and dual RJ45 ports (AOC-MHIBF-m1Q2G)

Cables Support

• InniBand FDR: QSFP 56Gbps copper cables

• GbE: RJ45 category 5/5e up to 100m

Power Consumption

• AOC-MHIBF-m2Q2G: Maximum 11W

• AOC-MHIBF-m1Q2G: Maximum 9W

Operating Condition

• Operating temperature: 0°C to 55°C (32°F to 131°F)

• Storage temperature: -40°C to 70°C (-40°F to 158°F)

• Storage humidity: 90% non-condensing relative humidity at 35°C

Physical Dimensions

• Card PCB dimensions: 92mm (3.62in) x 87.1mm (3.43in) (WxD)

Supported Platforms

• Supermicro® motherboards with Super I/O Module (SIOM) slot

• Supermicro® server systems with Super I/O Module slot (see SIOM Compat-

ibility Matrix online at http://www.supermicro.com/support/resources/AOC/

AOC_Compatibility_SIOM.cfm)

Note: This product is sold only as part of an integrated solution with

Supermicro server systems.

1-2

InniBand FDR QSFP Specications

Connectivity

• Interoperable with InniBand or 40GbE switches

• Passive copper cable with ESD protection

InniBand

• IBTA specication 1.2.1 compliant

• Hardware-based congestion control

• 256 to 4K byte MTU, 2GB messages

Enhanced InniBand

• Hardware-based reliable transport

• Collective operations ofoads

Chapter 1: Overview

• Hardware-based reliable multicast

• Extended reliable connected transport

• Enhanced atomic operations

Ethernet

• IEEE 802.3ba 40 Gigabit Ethernet

• IEEE Std 802.3ad Link Aggregation

• IEEE Std 802.3az Energy Efcient Ethernet

• IEEE Std 802.1Q, 802.1P VLAN tags and priority

• IEEE P802.1bb D1.0 priority-based ow control

• IEEE 1588

• Jumbo frame support (9600B)

1-3

AOC-MHIBF-m2Q2G/m1Q2G Add-on Card User's Guide

Hardware-based I/O Virtualization

• Single root IOV

• Address translation and protection

• Dedicated adapter resources

• Multiple queues per virtual machine

• Enhanced QoS for vNICs

• VMware NetQueue support

Overlay Networks

• VXLAN and NVGRE

Additional CPU Ofoads

• RDMA over converged Ethernet (RoCE)

• TCP/UDP/IP stateless ofoad

• Intelligent interrupt coalescence

FlexBoot™ Technoloty

• Remote boot over InniBand

• Remote boot over Ethernet

• Remote boot over iSCSI

Protocol Support

• Open MPI, OSU MVAPICH, Intel® MPI, MS

• MPI, Platform MPI

• TCP/UDP, EoIB, IPoIB, RDS

• SRP, iSER, NFS RDMA

1-4

Chapter 1: Overview

• uDAPL

Operating System Support

• Citrix XenServer 6.1

• RHEL/CentOS 5.X and 6.X, Novell SLES10 SP4; SLES11 SP1, SLES11 SP2,

OEL, Fedora 14, 15, 17, Ubuntu 12.04

• Windows Server 2008/2012/2012 R2

• FreeBSD

• OpenFabrics Enterprise Distribution (OFED)

• OpenFabrics Windows Distribution (WinOF)

• VMware ESXi 4.x and 5.x

GbE RJ45 Specications

Ethernet and Virtualization Features

• PC-SIG SR-IOV support

• Jumbo frames support

Performance and Efciency Features

• Energy Efcient Ethernet (EEE)

• TCP/UDP, IPv4, and IPv6 checksum ofoads to improve CPU usage

• Low latency interrupts

Management Features

• Asset Management support on Supermicro® platforms

• Preboot eXecution Environment (PXE) support

1-5