HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Overview
1
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Hewlett Packard Enterprise supports InfiniBand (IB) products that include Host Channel Adapters (HCA), HPE FlexLOM Adapters,
switches, and cables for
For 100Gbps EDR InfiniBand adapter products, please refer to
adapters.html
catalog/servers/server
For details on the InfiniBand support for
BladeS
Hewlett Packard Enterprise supports
The following InfiniBand
The
function adapter that
InfiniBand ports. This
The
interface. The 544+
of InfiniBand on Port 1 and Ethernet on Port 2
Gen 9
Gen 9
InfiniBand host stack software (driver) is required to run on servers connected to the InfiniBand fabric. For HCAs based on M
technologies,
WinOF driver stack on Microsoft Windows
An InfiniBand fabric is constructed with one or more InfiniBand switches connected via inter
required to manage an InfiniBand fabric. OpenSM is a host
fabric.
Mellanox Unified Fabric Manager
products.
The following InfiniBand
The switch
The following Mellanox software for InfiniBand switches and
HPE ProLiant servers and HPE Apollo Servers. This QuickSpecs covers 56Gbps FDR InfiniBand products.
• HPE IB FDR/EN 10/40Gb 2P 544+FLR-QSFP Adapter (Based on Mellanox ConnectX-3 Pro technology)
• HPE IB FDR/EN 10/40Gb 2P 544+QSFP Adapter (Based on Mellanox ConnectX-3 Pro technology)
547 adapter is based on the Mellanox ConnectX-5 technology. This adapter utilizes the PCI Express 3.0 x8 interface. It is a dual
is capable of supporting two Ethernet ports, a mix of InfiniBand on Port 1 and Ethernet on Port 2 or two
adapter is designed for PCI Express 3.0 x8 FlexibleLOM expansion slots on HPE Gen 9 and Gen 10 servers.
544+ adapters are based on the Mellanox ConnectX-3 Pro technology. The 544+ adapters utilize the PCI Express 3.0 x8
adapters are dual function adapters that also are capable of supporting single or dual Ethernet ports and a mix
. The 544+QSFP adapter is designed for PCI Express 3.0 x8 expansion slots on HPE
and Gen 10 servers. The 544+FLR-QSFP adapters are designed for PCI Express 3.0 x8 FlexibleLOM expansion slots on HPE
and Gen 10 servers.
Hewlett Packard Enterprise supports Mellanox OFED driver stacks on Linux 64-bit operating systems, and Mellanox
Operating System.
-switch links. A subnet manager is
-based subnet manager that runs on a server connected to the InfiniBand
The Mellanox OFED software stack includes OpenSM for Linux. For comprehensive management and monitoring capabilities,
Advanced is recommended for managing the InfiniBand fabric based on Mellanox InfiniBand
FDR switch products based on Mellanox technologies are available from HPE
• Mellanox IB FDR 36-port Managed switch
• Mellanox IB FDR 36-port switch
• Mellanox IB QDR/FDR 648, 324 and 216 port Modular Switches
air flow for cooling is from the power side (front of the rack) to the ports side (rear of the rack).
adapters is available from HPE
• Mellanox Unified Fabric Manager Advanced (UFM)
ellanox
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Overview
2
Mellanox Unified Fabric Manager™ Advanced (UFM™) is a powerful platform for managing scale-out computing environments. UFM
enables data center operators to efficiently provision, monitor and operate the modern data center fabric.
UFM runs on a server and is used to monitor and analyze Mellanox fabrics health and performance. UFM also can be used to
automate provisioning and device management tasks. For example, UFM can communicate with devices, to reset or shut down ports
or devices, perform firmware and software upgrades, etc. UFM's extensive API enables it to easily integrate with existing
management tools for a unifie
systems and to activate user made scripts based on system events.
Hewlett Packard Enterprise supports
What's New
• Support for HPE IB FDR/EN 40/50Gb 2P 547FLR-QSFP Adapter
At A Glance
• Dual function InfiniBand/Ethernet cards based on Mellanox ConnectX-5 technology
FDR IB on port 1 and Ethernet on port 2
d cluster view. UFM also includes the ability to save historical information, to send alerts to external
InfiniBand copper and fiber optic cables with QSFP to QSFP connectors.
• QSFP to QSFP FDR copper cables range from .5M to 3M for HCA to switch, or inter-switch links at FDR speed.
• QSFP to QSFP FDR fiber optic cables range from 3M to 30M for HCA to switch, or inter-switch links at FDR speed.
Dual QSFP ports PCI Express 3.0 card based on ConnectX-5 technology
Support PCI Express 3.0 x8
Single or dual ports FDR InfiniBand
Single or dual ports 10/40/50Gb Ethernet
A mix of InfiniBand on Port 1 and Ethernet on Port 2 (Default)
Use UEFI to change the port personality
NCSI Shared-Network-port (SNP) and Wake-on-Lan are supported on Port 2 only
InfiniBand features: Improved message rate, Tag matching, Adaptive routing, Atomic operations
Additional features: RoCE, DPDK, SRIOV, VXLAN, NVGRE
Support the following Hewlett Packard Enterprise servers: Apollo 2000/XL170r Gen10, Apollo
2000/XL190r Gen10, Apollo 4200 Gen9, Apollo 4500/XL450 Gen10, DL360
Gen10, DL380 Gen10, DL385 Gen10, DL560 Gen10, DL580 Gen10
For firmware release notes, please visit the support website and search by part number and
operating system. Look for the most recent release.
• Dual function InfiniBand/Ethernet cards based on Mellanox ConnectX-3 Pro technology
Dual QSFP ports PCI Express 3.0 card based on ConnectX-3 Pro technology
Support PCI Express 3.0 x8
Single or dual ports FDR InfiniBand
Single or dual ports 10/40Gb Ethernet
FDR IB on port 1 and Ethernet on port 2
Support the following Hewlett Packard Enterprise servers: DL120 Gen9, DL160 Gen9, DL180 Gen9,
2000/XL1x0 Node, Apollo 4200, Apollo 4500/XL450 Gen9, Apollo 6000/XL230b Gen9
For firmware release notes, please visit the support website and search by part number and operating
system. Look for the most recent release.
Dual QSFP ports HPE Flexible LOM Adapter based on ConnectX-3 Pro technology
Support PCI Express 3.0 x8
Single or dual port FDR InfiniBand
Single or dual port 10/40Gb Ethernet
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Overview
3
Support the following Hewlett Packard Enterprise servers: DL180 Gen9, DL360 Gen9, DL380 Gen9,
fan that has air flow from the front to the rear (ports side.
DL560 Gen9, DL580 Gen8, DL580 Gen9, Apollo 2000/XL1x0 Node, Apollo 4200, Apollo 4500/XL450
Gen9, Apollo 6000/XL230a Gen9, Apollo 6000/XL230b Gen9, Apollo 6000/XL250a Gen9
For firmware release notes, please visit the support website and search by part number and operating
• Mellanox Software
o Mellanox Unified Fabric Manager Advanced (UFM)
• InfiniBand switches based on Mellanox SwitchX-2 technology
o Mellanox IB FDR 36-port unmanaged switch (part number: 670767-B21)
system. Look for the most recent release.
Can run on one server, or on two servers for high availability.
NOTE: Supported on 64 bit Linux operating systems only.
Activates user made scripts based on system events
Advanced multicast optimizations through multicast tree management
User management with control and authorization groups
Multicast tree routing optimizations
User authorization management
36 FDR QSFP ports, support FDR InfiniBand copper and fiber optic cables, with the front-to-rear cooling
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Overview
4
Dual power supplies for redundancy.
o 10 Gb Ethernet requires the use of a QSFP to SFP+ Adapter 655874-B21
o Mellanox IB FDR 36-port unmanaged switch with reversed airflow fan unit (part number: 670768-B21)
o Mellanox IB FDR 36-port managed switch (part number: 670769-B21)
o Mellanox IB FDR 36-port managed switch with reversed airflow fan unit (670770-B21)
• InfiniBand cables
o For InfiniBand cables, please see the cables section below. Please also consult product’s firmware release notes for
• Ethernet cables
o The Dual Function cards support most HPE Ethernet cables. Consult product's firmware release notes for list of
36 FDR QSFP ports, support FDR InfiniBand copper and fiber optic cables, with the rear-to-front cooling
fan that has air flow from the rear (ports side) to the front.
Dual power supplies for redundancy.
36 FDR QSFP ports, support FDR InfiniBand copper and fiber optic cables
Integrated management module for Fabric Management
Dual power supplies for redundancy.
36 FDR QSFP ports, support FDR InfiniBand copper and fiber optic cables, with the rear-to-front cooling
fan that has air flow from the rear (ports side) to the front.
Integrated management module for Fabric Management
Dual power supplies for redundancy.
list of support cables.
supported cables.
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Models
5
HPE Dual Function Adapters
InfiniBand/Ethernet based on Mellanox ConnectX-5 technology
HPE BladeSystem c-Class 10GbE SFP+ to SFP+ 1m Direct Attach Copper Cable
487652-B21
HPE BladeSystem c-Class 10GbE SFP+ to SFP+ 3m Direct Attach Copper Cable
487655-B21
HPE BladeSystem c-Class 10GbE SFP+ to SFP+ 5m Direct Attach Copper Cable
537963-B21
HPE BladeSystem c-Class 10Gb SFP+ SR Transceiver
455883-B21
HPE BladeSystem c-Class 10Gb SFP+ LR Transceiver
455886-B21
the support website and search by part number and operating system.
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Performance
7
Bandwidth
All ports on FDR HCA card and switch are capable of supporting 56Gbps signaling rate, with a peak data rate of
52 Gbps in each direction.
All ports on
of 32 Gbps in each direction
QDR HCA card and switch are capable of supporting 40Gbps signaling rate, with a peak data rate
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Operating Systems Support
8
Operating systems
The 547 series device configured as IB only mode, Ethernet only mode, or Port 1 IB and Port 2 Ethernet mode
on the following Windows
Windows
also supported in the following virtualized
support
is supported on MLNX_OFED 4.3-1.0.1.0 on the following Linux operating systems:
• RHEL: 7.5; 7.4; 7.3
• CentOS: 7.5; 7.4; 7.3
• SLES: 12 SP3; 12 SP2; 11 SP4
The 547 series device configured as IB or Ethernet mode is supported on WinOF2 1.9
operating systems (switch based subnet manager or Linux host based subnet manager required)
•Windows Server 2016 (x64 only)
The 544+ series devices configured as IB only mode, Ethernet only mode, or Port 1 IB and Port 2 Ethernet mode
are supported on MLNX_OFED 4.3-1.0.1.0 on the following Linux operating systems:
• RHEL: 7.5; 7.4; 7.3
• CentOS: 7.5; 7.4; 7.3
• SLES: 12 SP3; 12 SP2; 11 SP4
The 544+ series devices configured as Ethernet mode are supported on WinOF on the following
operating systems:
• Windows Server 2016 (x64 only) with WinOF 5.40
• Windows Server 2012 R2 (x64 only) with WinOF 5.35
The 544+ series devices configured as Ethernet only mode are
environments:
• ESXi: 6.7 with inbox driver
• ESXi: 6.5 with VMware 3.16.11.6
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
InfiniBand Fabric Management
9
Auto-negotiation/
InfiniBand ports support auto-negotiation down to lower rate. For example, ports of a FDR switch will negotiate
down to QDR speed if they are connected to a device which can only operate at QDR.
Self
Management
Mellanox OFED includes drivers and utilities to configure and manage the HCA in a Linux environment and
Mellanox WinOF includes drivers and utilities to configure and manage the HCA in the Microsoft Windows
server environment.
InfiniBand fabric requires a subnet manager: see the subnet manager discussion above.
self-discovery
-discovery is supported by subnet manager.
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Service and Support
10
Service and
HPE Technology Services
HPE
infrastructure problems. We help keep your business running, boost availability and avoid downtime.
Protect your business beyond warranty with
When you buy
Pointnext operational services
designed to meet your IT and busine
HPE
standard servers
business needs. If you are running business critical environments,
Care. These services help you deliver high levels of a
management and advanced technical response.
Here is the support service recommendation from the Foundation Care and Proactive Care portfolio. For
customized support service solution,
for your unique support requirements using broader services portfolio of Foundation Care and Proactive Care.
Recommended
Recommended HPE Pointnext operational services for optimal satisfaction with your HPE product
Hewlett Packard Enterprise recommends
prevent problems with proactive advice and assistance, resolve issues and rapidly recover by connecting to The
Hewlett Packard Enterpr
Related Services
HPE Proactive Select Service
Provides a flexible way to purchase
Proactive Select Service Credits when you purchase your hardware and then use the credits over the next 12
months.
Parts and
Hewlett Packard Enterprise will provide HPE-supported replacement parts and materials necessary to maintain
the covered hardware product in operating condition, including parts an
recommended engineering improvements.
Parts and components that have reached their maximum supported lifetime and/or the maximum usage
limitations as set forth in the manufacturer's operating manual, product quick
data sheet will not be provided, repaired, or replaced as part of these services.
The defective media retention service feature option applies only to Disk or eligible SSD/Flash Drives replaced
by Hewlett Packard Enterprise due to malfunction.
Support
Services
Technology Services offers you consultants and support experts to solve your most complex
HPE Pointnext operational services
HPE Options, it's also a good time to think about what level of service you may need. HPE
provide total care and support expertise with committed response choices
ss need.
Foundation Care services deliver scalable support-packages for Hewlett Packard Enterprise industry-
and software. You can choose the type and level of service that is most suitable for your
Hewlett Packard Enterprise offers Proactive
pplication availability through proactive service
Hewlett Packard Enterprise can work with you to tailor a service solution
Proactive Care for your servers, enclosures and components to
ise enhanced call experience and advanced technical expertise.
HPE best-in-class consultancy and technical services. You can buy
NOTE: Please refer to Mellanox product specifications for Mellanox switch models.
HPE IB FDR/EN 40/50G 2P 547FLR QSFP
Adapter:
− With Active Optical Cables: typical
19W, max 21W
− With Passive Copper Cables: typical
14.1W, max 16.1W
• HPE IB FDR/EN 10/40G 2P 544+QSFP
Adapter:
− With Active Optical Cables: typical
13.6W, max 14.2W
− With Passive Copper Cables: typical
8.7W, max 10.6W
• HPE IB FDR/EN 10/40G 2P 544+FLR-QSFP
Adapter:
− With Active Optical Cables: typical
13.6W, max 14.2W
− With Passive Copper Cables: typical
NOTE: Please refer to Mellanox product specifications for Mellanox switch models.
NOTE: Please refer to Mellanox product specification for Mellanox switch models.
friendly Products
and Approach
return, trade-in, and recycling programs in many geographic areas. For trade-in
information, please go to:
http://www.hpe.com/info/recycle
please go to:
Canada: ICES, Class A
EU: EN55022, Class A
EU: EN55024, Class A
EU : EN61000-3-2, Class A
EU : EN61000-3-3, Class A
Japan : VCCI, Class A
Korea: MIC, Class A
Taiwan: BSMI, Class A
or contact your nearest Hewlett
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Technical Specifications
12
Packard Enterprise sales office. Products returned to Hewlett Packard Enterprise will be
equipment.
recycled, recovered or disposed of in a responsible manner.
The EU WEEE directive (2002/95/EC) requires manufacturers to provide treatment
information for each product type for use by treatment facilities. This information
(product disassembly instructions) is posted on the Hewlett Packard Enterprise web
site at:
http://www.hpe.com/info/recycle. These instructions may be used by
recyclers and other WEEE treatment facilities as well as HEWLETT PACKARD
ENTERPRISE OEM customers who integrate and re-sell Hewlett Packard Enterprise
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers
Summary of Changes
13
Date
Version History
Action
Description of Change
04-Jun-2018
Version 29
Changed
HPE IB FDR/EN 40/50Gb 2P 547FLR-QSFP Adapter
Obsolete SKUs were deleted: 764286-B21, 702211-B21, 702212-
and host-based subnet manager from HPE in the Models section.
30-Jan-2009
From Version 2 to 3
Changed
Changed Power Supply amounts in the NOTE for Voltaire IB DDR
section.
15-Sep-2008
From Version 1 to 2
Added
What's New section to the Overview
Cables sections to the Models
Changed
NOTEs section of Voltaire IB DDR Rev B 96-port Switch Chassis,
and options and QLogic BLc 2-Port QDR IB Management Module
Mellanox OFED
Introduction in the Overview
DL160 G6 throughout At A Glance in the Overview section
Mellanox IB QDR 324P Switch Chassis throughout the QuickSpecs
Completely revised the Scalability and Reliability section and Autonegotiation/self-discovery and Management in the InfiniBand fabric
Management section
Updated InfiniBand HCA cards and Power requirement in the
PCI HCA cards and drivers for ProLiant and Integrity severs in the
Models section
Subnet Manager from the InfiniBand Fabric Management section
Standard Support section from HPE Services and Support in the
cards, and drives for ProLiant and Integrity servers and Voltaire IB 2Port QDR 36-port Internally Managed Switch to Voltaire Switches
Rev B96-port Switch Chassis and removed "with 3 yrs software
support" from Voltaire Gridstack per HCA software license and Cisco
IB OFED-based driver per node software license in the Models
Voltaire 22P IB 10GE Line board to the Voltaire switches and hostbased subnet manager from HPE section of the Models
InfiniBand QSFP/CX4 Copper Cables and InfiniBand QSFP Copper
Voltaire IB DDR Rev b 288-port Switch Chassis and Voltaire IB DDR
Rev b 24-port Line Board to the Voltaire switches and host-based
subnet manager from HPE section of the Models
InfiniBand CX4 Copper Cables and InfiniBand CX4 Fiber Optic Cables
section in the Models
Page
QuickSpecs
HPE InfiniBand Options for HPE ProLiant and Apollo Servers