HPE InfiniBand Options for HPE BladeSystems c-Class
Overview
Page 1
HPE InfiniBand Options for HPE BladeSystems c-Class
Hewlett Packard Enterprise supports 56 Gbps Fourteen Data Rate (FDR) and 40Gbps Quad Data Rate (QDR) InfiniBand (IB)
products that include mezzanine Host Channel Adapters (HCA) for server blades, dual mode InfiniBand or Ethernet mezzanine
adaptors, switch blades for c
focuses
on mezzanine adaptors for server blades, and InfiniBand switch blades for c
switches, standup PCI Express HCAs and FlexLOM Adap
servers, as well as InfiniBand cables, please
refer to the
at:
The following InfiniBand products based on Mellanox technology are available for the
The 544+M Mezzanine adapters
Mezzanine adapters utilize the PCI Express 3.0 x8 interface. The 544+M Mezzanine adapters are dual function adapters that are
capable of support
(a mix mode of one InfiniBand port and one Ethernet port is not supported on 544+M Mezzanine adapters). The 544+M Mezzanine
adapter are supported on
The 545M Mezzanine Adapter for
utilizes
and low
with a
NOTE:
InfiniBand host stack software (driver) is required to run on servers connected to the InfiniBand network. For mezzanine HCAs based
on Mellanox technologies,
Mellanox WinOF on Microsoft Windows.
For the latest information on Mellanox OFED software, please refer
to:
The
enclosure. It is based on the Mellanox Switch X technology. The FDR IB switch blade has 16 downlink ports to connect up to 16
server blades in the c7000 enclosure, and 18 QSFP uplink ports for inter
capable of supporting 56Gbps (FDR) bandwidth. A subnet manager has to be provided; see the paragraph on subnet managers for
more details.
The
Platinum enclosure. It is based on the Mellanox Switch X technology. The FDR IB switch blade has 16 downlink ports to connect
to 16 server blades in the c7000 enclosure, and 18 QSFP uplink ports for inter
ports are capable of supporting 56Gbps (FDR) bandwidth. The switch contains an integrated management module that is capable of
managing a cluster of up to 648 nodes.
links. The most commonly deployed fabric topology is a fat tree or its variations. A subnet manager is required to manage and
control an InfiniBand fabric. The subnet manager functionality can be provided by either t
rack-mount InfiniBand switch with an embedded fabric manager (aka internally managed switch) or host-based subnet manager
-Class enclosures, and rack switches and cables for building scale-out solutions. This QuickSpecs
-Class enclosures; for details on the InfiniBand rack
tors for HPE ProLiant and Apollo
HPE InfiniBand for HPE ProLiant and Apollo servers QuickSpecs
HPE BLc FDR IB Switch for HPE BladeSystem c-Class is a double wide switch for the HPE BladeSystem c7000 Platinum
HPE BLc FDR Managed IB Switch for HPE BladeSystem c-Class is a double wide switch for the HPE BladeSystem c7000
HPE BladeSystem c-Class is based on the Mellanox Connect-IB technology. The 545M adapter
-switch links or to connect to external servers. All ports are
-switch links or to connect to external servers. All
An InfiniBand fabric consists of one or more InfiniBand switches connected via inter-switch
he HPE BLc FDR Managed IB Switch or a
up
QuickSpecs
HPE InfiniBand Options for HPE BladeSystems c-
Class
Overview
Page 2
software on a server connected to the fabric. OpenSM is a host-based subnet manager that runs on a server connected to the
InfiniBand fabric. Mellanox OFED software stack includes OpenSM for Linux, and Mellanox WinOF includes OpenSM for Windows. For
comprehensive management and monitoring capabilities, Mellanox Unified Fabric Manager
managing the InfiniBand fabric based on Mellanox/ InfiniBand switch products and Mellanox based mezzanine HCA.
An embedded fabric manager is available on Mellanox internally managed 36
.
Please refer to:
mount
HP
from 1M to 7M for HCA to switch, or inter
for HCA to switch, or inter
switch, or inter
Please refer to:
InfiniBand cables.
What's New
•
− PCI Express 3.0 x8
™ Advanced (UFM) is recommended for
-port FDR switch, and modular FDR and QDR switches
https://www.hpe.com/h20195/v2/GetHTML.aspx?docname=c04154440 for information about InfiniBand rack
switches.
E supports InfiniBand copper and fiber optic cables with QSFP to QSFP connectors. The QSFP to QSFP QDR copper cables range
-switch links at QDR speed. The QSFP to QSFP FDR copper cables range from 0.5M to 3M
-switch links at FDR speed. The QSFP to QSFP FDR optical cables range from 3M to 30M for HCA to
-switch links at FDR speed.
https://www.hpe.com/h20195/v2/GetHTML.aspx?docname=c04154440 for more details on supported
Mellanox Connect-IB technology
Dual port FDR InfiniBand
PCI Express 3.0 x16 interface for increased throughput
Improved message rate
Designed for use with HPE BLc7000 Platinum enclosures
− Support Mellanox OFED Linux driver stack r
For firmware release notes, please visit the support website
system.
− HPE BLc FDR IB Switch for HPE BladeSystem c-Class
Double wide switch blade for the Platinum c7000 enclosure with 16 downlink ports
to connect server blades via the midplane, and 18 QSFP unlink ports for the interswitch links or to connect to external servers. All ports are capable of supporting
56Gbps (FDR) bandwidth. All uplink ports support copper and fiber optic cables. One
version of the switch (648312-B21) is externally managed, i.e. a subnet manager has
to be provided on the fabric (see the subnet manager discussion above). The other
version of the switch (648311-B21)
subnet manager.
then select your operating
Reliability
:
• All ports on FDR HCA cards and switch are capable of supporting 56Gbps signaling rate, with a peak
data rate of 52 Gbps in each direction.
• All ports on QDR HCA card and switch are capable of supporting 40Gbps signaling rate, with a peak
• The 545M Mezzanine Adapter: PCI Express revision 3.0x16 compliant
• IBTA version 1.2 compatible
Page
QuickSpecs
HPE InfiniBand Options for HPE BladeSystems c-
Class
Standard Features
Page 5
Operating
The 544+M series devices configured as IB only mode, or Ethernet only mode are supported on MLNX_OFED
WinOF 5.10 on
0) are
Interconnect Bays
HPE IB switch blades are designed to fit into the double wide switch bays on the c-Class enclosures. Depending
InfiniBand Fabric
Auto-negotiation/self-discovery:
systems support
3.1-1.0.3 on the following Linux operating systems:
• RHEL: 6.5; 6.6; 6.7; 7.0; 7.1
• CentOS: 6.5; 6.6; 6.7; 7.0; 7.1
• SLES: 11 SP1; SP2; SP3; SP4; 12
• OEL: 6.3; 6.4; 6.5; 6.6; 6.7; 7.0; 7.1
• Ubuntu: 12.04; 14.04; 14.10; 15.04
The 544+M series devices configured as IB only mode or Ethernet only mode are supported on
the following Windows operating systems:
• Windows Server 2012 R2 (x64 only)
• Windows Server 2012 (x64 only)
• Windows Server 2008 R2 (x64 only)
The 544+M series devices configured as Ethernet only mode (with VMware 4.15.2.0 or VMware 1.9.10also supported in the following virtualized environments:
• ESXI 6.x
• ESX 5.5
• ESX 5.1
The 545M adapter is supported on MLNX_OFED 3.1-1.0.3 on the following Linux operating systems:
• RHEL: 6.5; 6.6; 6.7; 7.0; 7.1
• CentOS: 6.5; 6.6; 6.7; 7.0; 7.1
• SLES: 11 SP1; SP2; SP3; SP4; 12
• OEL: 6.3; 6.4; 6.5; 6.6; 6.7; 7.0; 7.1
• Ubuntu: 12.04; 14.04; 14.10; 15.04
on c-Class
enclosure
on the mezzanine connectors used for IB HCA mezzanine cards, the IB switch blade has to be inserted into
corresponding interconnect bays.
• The FDR switch may be installed in interconnect bays 3 and 4, 5 and 6, 7 and 8 in the HPE BLc7000
InfiniBand FDR or QDR ports support auto-negotiation down to lower frequencies. For example, external uplink
ports of a FDR switch blade will negotiate down to QDR or DDR speed if they are connected to a device which
can only operate at those speeds.
Self-discovery is supported by subnet manager.
Management Support:
Mellanox OFED includes drivers and utilities to configure and manage the HCA in a Linux environment and
Mellanox WinOF includes drivers and utilities to configure and manage the HCA in the Microsoft Windows
environment.
QuickSpecs
HPE InfiniBand Options for HPE BladeSystems c-
Class
Standard Features
Page 6
InfiniBand fabric has to be set up with a subnet manager: see the subnet manager discussion above.
Warranty
Hewlett Packard Enterprise branded hardware options qualified for BladeSystem c-Class and p-Class servers
service agreements.
are covered by a global limited warranty and supported by HPE Services and a worldwide network of Hewlett
Packard Enterprise Authorized Channel Partners. The Hewlett Packard Enterprise branded hardware option
diagnostic support and repair is available for one year from date of purchase, or the length of the server they are
attached to, whichever is greater. Support for software and initial setup is available for 90 days from date of
purchase. Additional support may be covered under the warranty or available for an additional fee.
Enhancements to warranty services are available through HPE Pointnext operational services or customized
QuickSpecs
HPE InfiniBand Options for HPE BladeSystems c-
Class
Service and Support
7
Service and
HPE Technology Services
HPE
infrastructure problems. We help keep your business running, boost availability and avoid downtime.
Protect your business beyond warranty with
When you buy
Pointnext operational services
designed to meet your IT and busine
HPE
standard servers
business needs. If you are running business critical environments,
Care. These services help you deliver high levels of a
management and advanced technical response.
Here is the support service recommendation from the Foundation Care and Proactive Care portfolio. For
customized support service solution,
for your unique support requirements using broader services portfolio of Foundation Care and Proactive Care.
Recommended
Recommended HPE Pointnext operational services for optimal satisfaction with your HPE product
Hewlett Packard Enterprise recommends
components to prevent problems with proactive advice and assistance, resolve issues and rapidly recover by
connecting to
HPE
Provides for the basic hardware installation of
servers, storage devices and
networking options to ass
manner.
Related Services
HPE Proactive Select Service
Provides a flexible way to purchase
Proactive Select Service Credits when you purchase your hardware and then use the credits over the next 12
months.
Parts and
Hewlett Packard Enterprise will provide HPE-supported replacement parts and materials necessary to maintain
the covered hardware product in operating condition, including parts and materials for available and
recommended engineering improvements.
Parts and components that have reached their maximum supported lifetime and/or the maximum usage
limitations as set
data sheet will not be provided, repaired, or replaced as part of these services.
The defective media retention service feature option applies only to Disk or elig
by Hewlett Packard Enterprise due to malfunction.
Support
Services
Technology Services offers you consultants and support experts to solve your most complex
HPE Pointnext operational services
HPE Options, it's also a good time to think about what level of service you may need. HPE
provide total care and support expertise with committed response choices
ss need.
Foundation Care services deliver scalable support-packages for Hewlett Packard Enterprise industry-
and software. You can choose the type and level of service that is most suitable for your
Hewlett Packard Enterprise offers Proactive
pplication availability through proactive service
Hewlett Packard Enterprise can work with you to tailor a service solution
Proactive Care for your BladeSystem servers, enclosures and
The Hewlett Packard Enterprise enhanced call experience and advanced technical expertise
ProLiant Server Hardware Installation
Hewlett Packard Enterprise branded
ist you in bringing your new hardware into operation in a timely and professional