Cavium QL45211HLCU-BK, QL45212HLCU-CK, QL45212HLCU-SP, QL45212HLCU-BK, QL45411HLCU-BK User Manual
...
User’s Guide
Converged Network Adapters and
Intelligent Ethernet Adapters
FastLinQ 45000 Series
BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Document Revision History
Revision A, April 19, 2016
Revision B, April 26, 2016
Revision C, May 11, 2016
Revision D, August 29, 2016
Revision E, August 30, 2016
Revision F, September 30, 2016
Revision G, March 10, 2017
Revision H, June 26, 2017
Revision J, November 21, 2017
Revision K, February 7, 2018
Revision L, May 22, 2018
Revision M, June 18, 2018
Revision N, August 24, 2018
Revision P, January 17, 2019
Changes Sections Affected
Updated the Cavium logos, preface content, and
copyright information.
Removed references to Windows Nano Server. “System Requirements” on page 7 (Tab le 2 -2 )
Before Ta bl e 3- 4, added “Cavium has certified
qedrntv for ESXi 6.5 and ESXi 6.7, which are bundled along with qedentv as one package in vCG
listings.”
Updated the Linux firmware update procedure. “Upgrading Adapter Firmware on Linux” on
Updated the Windows firmware update procedure. “Upgrading Adapter Firmware on Windows” on
Front cover, Preface, and back page
Chapter 3 Driver Installation
Chapter 4 Firmware Upgrade Utility
Chapter 14 Windows Server 2016
“VMware Drivers and Driver Packages” on
page 24
page 31
page 31
Added L2B Firmware Version as a read-only item
on the Firmware Information page.
For device-level configuration, added the Port
Mode parameter to the bullet list.
“Displaying Firmware Information” on page 38
“Configuring Device-level Parameters” on page 38
ii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
For port-level configuration:
Updated Figure 5-7.
In Step 4, updated the RoCE Priority descrip-
tion.
In Step 10, added iWARP+RoCE as a new
option.
Added a footnote referencing the QConvergeConsole Help system for procedures for changing the
adapter port mode in QConvergeConsole GUI.
Moved iSCSI boot from SAN information from the
iSCSI Configuration chapter to a new chapter.
Moved FCoE boot from SAN information from the
FCoE Configuration chapter to a new chapter.
In the VMware bullet, expanded the information to
explain the limitations of iSCSI boot in the ESXi
OS.
Following Step 1 of the To install SLES 12 SP3
and later procedure, added this note: “Although
they are not necessarily required for iSCSI boot
from SAN for SUSE, Cavium recommends that
you also complete Step 2 through Step 15 of the
Configuring iSCSI Boot from SAN for RHEL 7.5
and Later procedure”
“Configuring Port-level Parameters” on page 41
“Changing the Adapter Port Mode” on page 53
Chapter 6 Boot from SAN Configuration
Chapter 6 Boot from SAN Configuration
“Supported Operating Systems for iSCSI Boot” on
page 60
“Configuring iSCSI Boot from SAN for SLES 12
SP3 and Later” on page 91
In Step 4 of the To install RHEL 7.4 and later pro-
cedure, changed the command to inst.dd
modprobe.blacklist=qed,qede,qedr,qedi
,qedf.
In Table 7-1 for OFED:
Removed support for RHEL 7.4, SLES 12 SP2.
Added support for RHEL 7.6, SLES 12 SP4.
Replaced CentOS 7.4 with CentOS 7.6.
Changed Ubuntu 16.04 LTS support for RoCE
v2 to include “(4.5 + kernel)”.
In Table 7-2, replaced the RDMA Mode property
with the NetworkDirect Technology property.
“Configuring FCoE Boot from SAN for RHEL 7.4
and Later” on page 120
“Supported Operating Systems and OFED” on
page 127
“Configuring RoCE on the Adapter for Windows
Server” on page 135
iii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Updated the following iWARP procedures:
In Step 1 c of the To enable iWARP on the
Windows host and verify RDMA procedure,
changed the parameter to NetworkDirect
Tec h n ology (was RDMA Mode).
In Step 4 b of the To verify iWARP traffic pro-
cedure, changed the command to netstat
-xan (was net -xan).
In the To configure the NFS server procedure:
Added a new Step 1 that describes how to
create an nfs-server directory and grant
permission.
Replaced Step 4 for loading the service.
In the To configure the NFS client procedure:
Added a new Step 1 that describes how to
create an nfs-client directory and grant
permission.
In Step 3, updated the command used for NFS
Version 4.
Following Step 12, added a fourth bullet to the
note: “Switch dependent teaming (IEEE 802.3ad
LACP and Generic/Static Link Aggregation (Trunking) cannot use a switch independent partitioned
virtual adapter. IEEE standards require Switch
Dependent Teaming (IEEE 802.3ad LACP and
Generic/Static Link Aggregation (Trunking)) mode
to work per the entire port instead of just the MAC
address (fraction of a port) granularity.”
“Configuring iWARP on Windows” on page 169
“Configuring NFS-RDMA” on page 175
“Configuring NFS-RDMA” on page 175
“Configuring Microsoft Initiator to Use Cavium’s
iSCSI Offload” on page 197
Added new section addressing an issue with iSCSI
offload on RHEL 7.6.
In the note, added the following: “For NDKPI
Mode-2, you must enable Network Direct Functionality of the physical interface.”
“” on page 208
“Configuring RoCE Interfaces with Hyper-V” on
page 241
iv BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
In the To create a Hyper-V virtual switch with an
RDMA NIC procedure:
Changed the section title to “...RDMA NIC” (was
“...RDMA Virtual NIC”.
Changed procedure title (was To create a
Hyper-V virtual switch with an RDMA virtual
NIC),
Added a new Step 1: “On all physical interfaces,
set the value of the NetworkDirect Functional-
ity parameter to Enabled.”
In the To add host virtual NICs procedure:
In Step 2, corrected the last digit of the com-
mand to “S” (was “5”).
Changed the note’s first bullet, second sen-
tence to: “The same VLAN ID must be assigned
to ports on the switch” (was: “The same VLAN
ID must be assigned to all the interfaces, and
on the switch”).
Added terms and definitions. Glossary
“Creating a Hyper-V Virtual Switch with an RDMA
NIC” on page 242
“Adding a vLAN ID to Host Virtual NIC” on
page 243
v BC0154501-00 P
Table of Contents
Preface
Supported Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
What Is in This Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Related Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx
Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
Technical Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Downloading Updates and Documentation . . . . . . . . . . . . . . . . . . . . . xxiii
Knowledgebase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Laser Safety—FDA Notice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Agency Certification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
EMI and EMC Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
KCC: Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
VCCI: Class A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
Product Safety Compliance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
1 Product Overview
Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Adapter Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
QLogic Control Suite CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
QLogic QConvergeConsole GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
QLogic QConvergeConsole vCenter Plug-in. . . . . . . . . . . . . . . . . . . . 4
QConvergeConsole PowerKit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
FastLinQ ESXCLI VMware Plug-in . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Adapter Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Physical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Standards Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Hardware Installation
System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Safety Precautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Preinstallation Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Installing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
vi BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
3 Driver Installation
Installing Linux Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Installing the Linux Drivers Without RDMA . . . . . . . . . . . . . . . . . . . . . 13
Removing the Linux Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Installing Linux Drivers Using the src RPM Package . . . . . . . . . 15
Installing Linux Drivers Using the kmp/kmod RPM Package . . . 16
Installing Ubuntu Linux Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . 16
Installing Linux Drivers Using the TAR File. . . . . . . . . . . . . . . . . 16
Installing the Linux Drivers with RDMA . . . . . . . . . . . . . . . . . . . . . . . . 17
Linux Driver Optional Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Linux Driver Operation Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Linux Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Importing a Public Key for Secure Boot. . . . . . . . . . . . . . . . . . . . . . . . 19
Installing Windows Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Installing the Windows Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Removing the Windows Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Managing Adapter Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Setting Power Management Options. . . . . . . . . . . . . . . . . . . . . . . . . . 23
Installing VMware Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
VMware Drivers and Driver Packages. . . . . . . . . . . . . . . . . . . . . . . . . 24
Installing VMware Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
VMware NIC Driver Optional Parameters . . . . . . . . . . . . . . . . . . . . . . 26
VMware Driver Parameter Defaults. . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Removing the VMware Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
FCoE Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
iSCSI Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Firmware Upgrade Utility
Image Verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Upgrading Adapter Firmware on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Upgrading Adapter Firmware on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . 31
Converting a 100G Adapter to Four-port 25G Adapter. . . . . . . . . . . . . . . . . 32
5 Adapter Preboot Configuration
Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Displaying Firmware Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Configuring Device-level Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Configuring Port-level Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Configuring FCoE Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
vii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Configuring Partitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Changing the Adapter Port Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Converting a 100G Adapter to Four-port 25G Adapter . . . . . . . . . . . . 53
Converting a 25G Adapter to Single-port 100G Adapter . . . . . . . . . . . 54
Converting a 40G Adapter to a Four-port 10G Adapter. . . . . . . . . . . . 55
Converting a 10G Adapter to a Four-port 40G Adapter. . . . . . . . . . . . 56
6 Boot from SAN Configuration
iSCSI Boot from SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
iSCSI Out-of-Box and Inbox Support. . . . . . . . . . . . . . . . . . . . . . . . . . 59
iSCSI Preboot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Supported Operating Systems for iSCSI Boot . . . . . . . . . . . . . . 60
Configuring Adapter UEFI Boot Mode . . . . . . . . . . . . . . . . . . . . 61
Selecting the iSCSI UEFI Boot Protocol. . . . . . . . . . . . . . . . . . . 64
Configuring the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Configuring iSCSI Boot Parameters . . . . . . . . . . . . . . . . . . . . . . 65
Configuring Adapter UEFI Boot Mode . . . . . . . . . . . . . . . . . . . . 65
Configuring iSCSI Boot Options . . . . . . . . . . . . . . . . . . . . . . . . . 67
Configuring the DHCP Server to Support iSCSI Boot . . . . . . . . 77
Configuring iSCSI Boot from SAN on Windows . . . . . . . . . . . . . . . . . 81
Before You Begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Selecting the Preferred iSCSI Boot Mode . . . . . . . . . . . . . . . . . 82
Configuring iSCSI General Parameters . . . . . . . . . . . . . . . . . . . 82
Configuring the iSCSI Initiator . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Configuring the iSCSI Targets . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Detecting the iSCSI LUN and Injecting the QLogic Drivers . . . . 84
Configuring iSCSI Boot from SAN on Linux . . . . . . . . . . . . . . . . . . . . 86
Configuring iSCSI Boot from SAN for RHEL 7.4 . . . . . . . . . . . . 86
Configuring iSCSI Boot from SAN for RHEL 7.5 and Later . . . . 90
Configuring iSCSI Boot from SAN for SLES 12 SP3
and Later . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Configuring iSCSI Boot from SAN for Other
Linux Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Configuring iSCSI Boot from SAN on VMware . . . . . . . . . . . . . . . . . . 107
FCoE Boot from SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
FCoE Out-of-Box and Inbox Support. . . . . . . . . . . . . . . . . . . . . . . . . . 107
FCoE Preboot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Specifying the BIOS Boot Protocol. . . . . . . . . . . . . . . . . . . . . . . 108
Configuring Adapter UEFI Boot Mode . . . . . . . . . . . . . . . . . . . . 109
viii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring FCoE Boot from SAN on Windows . . . . . . . . . . . . . . . . . 114
Windows Server 2012 R2 and 2016 FCoE Boot Installation . . . 114
Configuring FCoE on Windows . . . . . . . . . . . . . . . . . . . . . . . . . 115
FCoE Crash Dump on Windows. . . . . . . . . . . . . . . . . . . . . . . . . 115
Injecting (Slipstreaming) Adapter Drivers into Windows
Image Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Configuring FCoE Boot from SAN on Linux . . . . . . . . . . . . . . . . . . . . 117
Prerequisites for Linux FCoE Boot from SAN. . . . . . . . . . . . . . . 117
Configuring Linux FCoE Boot from SAN . . . . . . . . . . . . . . . . . . 118
Configuring FCoE Boot from SAN on VMware . . . . . . . . . . . . . . . . . . 123
Injecting (Slipstreaming) ESXi Adapter Drivers into
Image Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Installing the Customized ESXi ISO . . . . . . . . . . . . . . . . . . . . . . 124
7 RoCE Configuration
Supported Operating Systems and OFED . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Planning for RoCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Preparing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Preparing the Ethernet Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Configuring the Cisco Nexus 6000 Ethernet Switch . . . . . . . . . . . . . . 129
Configuring the Dell Z9100 Ethernet Switch . . . . . . . . . . . . . . . . . . . . 131
Configuring the Arista 7060X Ethernet Switch. . . . . . . . . . . . . . . . . . . 133
Configuring RoCE on the Adapter for Windows Server . . . . . . . . . . . . . . . . 135
Viewing RDMA Counters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Configuring RoCE on the Adapter for Linux . . . . . . . . . . . . . . . . . . . . . . . . . 143
RoCE Configuration for RHEL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
RoCE Configuration for SLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
RoCE Configuration for Ubuntu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Verifying the RoCE Configuration on Linux . . . . . . . . . . . . . . . . . . . . . 147
vLAN Interfaces and GID Index Values. . . . . . . . . . . . . . . . . . . . . . . . 150
RoCE v2 Configuration for Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Identifying the RoCE v2 GID Index or Address . . . . . . . . . . . . . 151
Verifying the RoCE v1 or RoCE v2 GID Index and Address
from sys and class Parameters . . . . . . . . . . . . . . . . . . . . . . . . 151
Verifying the RoCE v1 or RoCE v2 Function Through
perftest Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Configuring RoCE on the Adapter for VMware ESX . . . . . . . . . . . . . . . . . . 156
Configuring RDMA Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Configuring MTU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
RoCE Mode and Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Configuring a Paravirtual RDMA Device (PVRDMA). . . . . . . . . . . . . . 159
ix BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring DCQCN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
DCQCN Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
DCQCN Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
DCB-related Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Global Settings on RDMA Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Setting vLAN Priority on RDMA Traffic . . . . . . . . . . . . . . . . . . . . 164
Setting ECN on RDMA Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Setting DSCP on RDMA Traffic . . . . . . . . . . . . . . . . . . . . . . . . . 165
Configuring DSCP-PFC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Enabling DCQCN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Configuring CNP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
DCQCN Algorithm Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
MAC Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Script Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
8 iWARP Configuration
Preparing the Adapter for iWARP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Configuring iWARP on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Configuring iWARP on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Installing the Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Configuring iWARP and RoCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Detecting the Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Supported iWARP Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Running Perftest for iWARP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Configuring NFS-RDMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
iWARP RDMA-Core Support on SLES 12 SP3 and RHEL 7.4 . . . . . . 177
9 iSER Configuration
Before You Begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Configuring iSER for RHEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Configuring iSER for SLES 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Configuring iSER for Ubuntu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Configuring LIO as a Target. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Configuring the Initiator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Optimizing Linux Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Configuring CPUs to Maximum Performance Mode . . . . . . . . . . . . . . 191
Configuring Kernel sysctl Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Configuring IRQ Affinity Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Configuring Block Device Staging . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
x BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring iSER on ESXi 6.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Before You Begin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Configuring iSER for ESXi 6.7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
10 iSCSI Configuration
iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
iSCSI Offload in Windows Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
Installing Cavium QLogic Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Installing the Microsoft iSCSI Initiator . . . . . . . . . . . . . . . . . . . . . . . . . 197
Configuring Microsoft Initiator to Use Cavium’s iSCSI Offload . . . . . . 197
iSCSI Offload FAQs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Windows Server 2012 R2, 2016, and 2019 iSCSI Boot Installation . . 204
iSCSI Crash Dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
iSCSI Offload in Linux Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Differences from bnx2i. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Configuring qedi.ko . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Verifying iSCSI Interfaces in Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
11 FCoE Configuration
Configuring Linux FCoE Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Differences Between qedf and bnx2fc. . . . . . . . . . . . . . . . . . . . . . . . . 210
Configuring qedf.ko . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Verifying FCoE Devices in Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
12 SR-IOV Configuration
Configuring SR-IOV on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Configuring SR-IOV on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Configuring SR-IOV on VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
13 NVMe-oF Configuration with RDMA
Installing Device Drivers on Both Servers . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Configuring the Target Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Configuring the Initiator Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Preconditioning the Target Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Testing the NVMe-oF Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Optimizing Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
.IRQ Affinity (multi_rss-affin.sh). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
CPU Frequency (cpufreq.sh). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
xi BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
14 Windows Server 2016
Configuring RoCE Interfaces with Hyper-V . . . . . . . . . . . . . . . . . . . . . . . . . 241
Creating a Hyper-V Virtual Switch with an RDMA NIC . . . . . . . . . . . . 242
Adding a vLAN ID to Host Virtual NIC . . . . . . . . . . . . . . . . . . . . . . . . . 243
Verifying If RoCE is Enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
Adding Host Virtual NICs (Virtual Ports) . . . . . . . . . . . . . . . . . . . . . . . 245
Mapping the SMB Drive and Running RoCE Traffic . . . . . . . . . . . . . . 245
RoCE over Switch Embedded Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Creating a Hyper-V Virtual Switch with SET and RDMA
Virtual NICs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Enabling RDMA on SET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Assigning a vLAN ID on SET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Running RDMA Traffic on SET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Configuring QoS for RoCE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Configuring QoS by Disabling DCBX on the Adapter . . . . . . . . . . . . . 249
Configuring QoS by Enabling DCBX on the Adapter. . . . . . . . . . . . . . 253
Configuring VMMQ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Enabling VMMQ on the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Creating a Virtual Machine Switch with or Without SR-IOV . . . . . . . . 258
Enabling VMMQ on the Virtual Machine Switch . . . . . . . . . . . . . . . . . 259
Getting the Virtual Machine Switch Capability. . . . . . . . . . . . . . . . . . . 260
Creating a VM and Enabling VMMQ on VMNetworkadapters
in the VM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
Default and Maximum VMMQ Virtual NIC. . . . . . . . . . . . . . . . . . . . . . 262
Enabling and Disabling VMMQ on a Management NIC . . . . . . . . . . . 262
Monitoring Traffic Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
Configuring VXLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
Enabling VXLAN Offload on the Adapter. . . . . . . . . . . . . . . . . . . . . . . 263
Deploying a Software Defined Network. . . . . . . . . . . . . . . . . . . . . . . . 263
Configuring Storage Spaces Direct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Configuring the Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Deploying a Hyper-Converged System . . . . . . . . . . . . . . . . . . . . . . . . 265
Deploying the Operating System . . . . . . . . . . . . . . . . . . . . . . . . 265
Configuring the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
Configuring Storage Spaces Direct . . . . . . . . . . . . . . . . . . . . . .
267
15 Traffic Control Offload
Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Current Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
xii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Ingress Packet Redirection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
Ingress Packet Redirection for SR-IOV VFs . . . . . . . . . . . . . . . . . . . . 272
Ingress Packet Redirection with MAC-vLAN Offloaded
Devices and SR-IOV VFs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
TC Drop Action Support (Ingress Drop) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
TC MQPRIO Offload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
16 Troubleshooting
Troubleshooting Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Verifying that Current Drivers Are Loaded . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Verifying Drivers in Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Verifying Drivers in Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Verifying Drivers in VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Testing Network Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Testing Network Connectivity for Windows . . . . . . . . . . . . . . . . . . . . . 276
Testing Network Connectivity for Linux . . . . . . . . . . . . . . . . . . . . . . . . 277
Microsoft Virtualization with Hyper-V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Linux-specific Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Miscellaneous Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Troubleshooting Windows FCoE and iSCSI Boot from SAN . . . . . . . . . . . . 278
Collecting Debug Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
AAdapter LEDS
B Cables and Optical Modules
Supported Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Tested Cables and Optical Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Tested Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
C Feature Constraints
Glossary
Index
xiii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
List of Figures
Figure Page
3-1 Setting Advanced Adapter Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3-2 Power Management Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5-1 System Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5-2 System Setup: Device Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5-3 Main Configuration Page, Setting Default Partitioning Mode . . . . . . . . . . . . . . . . . . 35
5-4 Main Configuration Page, Setting NPAR Partitioning Mode. . . . . . . . . . . . . . . . . . . 36
5-5 Firmware Information Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5-6 Device Level Configuration Page. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5-7 Port Level Configuration Page: Setting Boot Mode . . . . . . . . . . . . . . . . . . . . . . . . . 42
5-8 FCoE General Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5-9 FCoE Target Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5-10 iSCSI General Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5-11 iSCSI Initiator Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5-12 iSCSI First Target Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5-13 iSCSI Second Target Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5-14 Partitions Configuration Page (No FCoE Offload or iSCSI Offload). . . . . . . . . . . . . 48
5-15 Partitions Configuration Page (with FCoE Offload and iSCSI Offload) . . . . . . . . . . 48
5-16 Global Bandwidth Allocation Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5-17 Partition 1 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5-18 Partition 2 Configuration: FCoE Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5-19 Partition 3 Configuration: iSCSI Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5-20 Partition 4 Configuration: Ethernet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5-21 Changing the Port Mode to 4x25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5-22 Changing the Port Mode to 1x100 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5-23 Changing the Port Mode to 4x10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5-24 Changing the Port Mode to 1x40 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6-1 Systems Utilities at Boot Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6-2 Configuration Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6-3 Selecting Port Level Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6-4 Port Level Configuration, Boot Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6-5 Port Level Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6-6 Systems Utilities at Boot Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6-7 Selecting Port Level Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
P
6-8
6-9 Selecting iSCSI Boot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6-10 Selecting General Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6-11 iSCSI General Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6-12 Selecting iSCSI Initiator Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6-13 iSCSI Initiator Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6-14 iSCSI First Target Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6-15 iSCSI First Target Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6-16 iSCSI Second Target Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6-17 Saving iSCSI Changes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
ort Level Configuration, Boot Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
xiv BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
6-18 iSCSI General Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6-19 iSCSI Initiator Configuration, VLAN ID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6-20 Detecting the iSCSI LUN Using UEFI Shell (Version 2). . . . . . . . . . . . . . . . . . . . . . 84
6-21 Windows Setup: Selecting Installation Destination . . . . . . . . . . . . . . . . . . . . . . . . . 85
6-22 Windows Setup: Selecting Driver to Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6-23 Prompt for Out-of-Box Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
6-24 Red Hat Enterprise Linux 7.4 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6-25 System Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6-26 System Configuration, Port Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6-27 Port Level Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6-28 Boot Mode in Port Level Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6-29 FCoE Offload Enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6-30 Selecting FCoE Boot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6-31 Selecting General Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6-32 FCoE General Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6-33 FCoE Target Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6-34 Selecting the Driver Disk Image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6-35 Selecting the Drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6-36 Prompt for Out-of-Box Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6-37 Red Hat Enterprise Linux 7.4 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6-38 ESXi-Customizer Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6-39 Select a VMware Disk to Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
6-40 VMware Generic USB Boot Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7-1 Configuring RoCE Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7-2 Add Counters Dialog Box. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7-3 Performance Monitor: Cavium FastLinQ Counters . . . . . . . . . . . . . . . . . . . . . . . . . 140
7-4 Switch Settings, Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
7-5 Switch Settings, Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7-6 Configuring RDMA_CM Applications: Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7-7 Configuring RDMA_CM Applications: Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
7-8 Configuring a New Distributed Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
7-9 Assigning a vmknic for PVRDMA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
7-10 Setting the Firewall Rule . . . . . . . . . . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . . . . . . 161
8-1 Windows PowerShell Command: Get-NetAdapterRdma . . . . . . . . . . . . . . . . . . . . . 169
8-2 Windows PowerShell Command: Get-NetOffloadGlobalSetting . . . . . . . . . . . . . . . 170
8-3 Perfmon: Add Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
8-4 Perfmon: Verifying iWARP Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
9-1 RDMA Ping Successful . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9-2 iSER Portal Instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9-3 Iface Transport Confirmed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
9-4 Checking for New iSCSI Device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
10-1 iSCSI Initiator Properties, Configuration Page. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
10-2 iSCSI Initiator Node Name Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
10-3 iSCSI Initiator—Discover Target Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
10-4 Target Portal IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
xv BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
10-5 Selecting the Initiator IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
10-6 Connecting to the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
10-7 Connect To Target Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
12-1 Device Level Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
12-2 Adapter Properties, Advanced: Enabling SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . 215
12-3 Virtual Switch Manager: Enabling SR-IOV. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
12-4 Settings for VM: Enabling SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
12-5 Device Manager: VM with QLogic Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
12-6 Windows PowerShell Command: Get-NetadapterSriovVf . . . . . . . . . . . . . . . . . . . . 219
12-7 Integrated Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
12-8 Editing the grub.conf File for SR-IOV. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
12-9 Command Output for sriov_numvfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
12-10 Command Output for ip link show Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
12-11 RHEL68 Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
12-12 Add New Virtual Hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
12-13 VMware Host Edit Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
13-1 NVMe-oF Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
13-2 Subsystem NQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
13-3 Confirm NVMe-oF Connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
13-4 FIO Utility Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
14-1 Enabling RDMA in Host Virtual NIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
14-2 Hyper-V Virtual Ethernet Adapter Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
14-3 Windows PowerShell Command: Get-VMNetworkAdapter . . . . . . . . . . . . . . . . . . . 244
14-4 Windows PowerShell Command: Get-NetAdapterRdma. . . . . . . . . . . . . . . . . . . . . 244
14-5 Add Counters Dialog Box. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
14-6 Performance Monitor Shows RoCE Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
14-7 Windows PowerShell Command: New-VMSwitch . . . . . . . . . . . . . . . . . . . . . . . . . . 247
14-8 Windows PowerShell Command: Get-NetAdapter. . . . . . . . . . . . . . . . . . . . . . . . . . 248
14-9 Advanced Properties: Enable QoS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
14-10 Advanced Properties: Setting VLAN ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
14-11 Advanced Properties: Enabling QoS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
14-12 Advanced Properties: Setting VLAN ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
rtual Switch RSS. . . . . . . . . . . . . . . . . . . . . . . . . 258
14-13 Advanced Properties: Enabling V
i
14-14 Virtual Switch Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
14-15 Windows PowerShell Command: Get-VMSwitch. . . . . . . . . . . . . . . . . . . . . . . . . . . 260
14-16 Advanced Properties: Enabling VXLAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
14-17 Example Hardware Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
16-1 Windows Setup Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
xvi BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
List of Tables
Table Page
2-1 Host Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2-2 Minimum Host Operating System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3-1 Cavium QLogic 45000 Series Adapters Linux Drivers . . . . . . . . . . . . . . . . . . . . . . . 12
3-2 qede Driver Optional Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3-3 Linux Driver Operation Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3-4 VMware Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3-5 ESXi Driver Packages by Release. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3-6 VMware NIC Driver Optional Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3-7 VMware Driver Parameter Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5-1 Adapter Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6-1 iSCSI Out-of-Box and Inbox Boot from SAN Support . . . . . . . . . . . . . . . . . . . . . . . 59
6-2 iSCSI General Configuration Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6-3 DHCP Option 17 Parameter Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6-4 DHCP Option 43 Sub-option Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6-5 DHCP Option 17 Sub-option Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6-6 FCoE Out-of-Box and Inbox Boot from SAN Support . . . . . . . . . . . . . . . . . . . . . . . 107
7-1 OS Support for RoCE v1, RoCE v2, iWARP, iSER, and OFED . . . . . . . . . . . . . . . . 127
7-2 Advanced Properties for RoCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
7-3 Cavium FastLinQ RDMA Error Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7-4 DCQCN Algorithm Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
13-1 Target Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
16-1 Collecting Debug Data Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
A-1 Adapter Port Link and Activity LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
B-1 Tested Cables and Optical Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
B-2 Switches Tested for Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
xvii BC0154501-00 P
Preface
NOTE
This preface lists the supported products, specifies the intended audience,
explains the typographic conventions used in this guide, lists related documents,
provides technical support information, and describes legal notices.
QConvergeConsole® (QCC) GUI is the only GUI management tool across
all Cavium
longer supported for the FastLinQ
based on 57xx/57xxx controllers, and has been replaced by the QCC GUI
management tool. The QCC GUI provides single-pane-of-glass GUI
management for all Cavium adapters.
In Windows
Agents Installer, it will uninstall the QCS GUI (if installed on the system) and
any related components from your system. To obtain the new GUI, download
QCC GUI for your adapter from the Marvell Web site (see “Downloading
Updates and Documentation” on page xxiii).
®
QLogic® adapters. QLogic Control Suite™ (QCS) GUI is no
®
45000 Series Adapters and adapters
®
environments, when you run QCS CLI and Management
Supported Products
This user’s guide describes the following Cavium FastLinQ products:
25Gb Intelligent Ethernet Adapters:
QL45211HLCU-BK/SP/CK
QL45212HLCU-BK/SP/CK
40Gb Intelligent Ethernet Adapters:
QL45411HLCU-BK/SP/CK
QL45412HLCU-BK/SP/CK
40Gb Converged Network Adapter
QL45462HLCU-BK/SP/CK
100Gb Intelligent Ethernet Adapters:
QL45611HLCU-BK/SP/CK
QL45631HOCU-BK/SP/CK
1
Windows and Linux only. VMware support for FCoE and iSCSI offload is coming soon.
xviii BC0154501-00 P
1
:
Preface
Intended Audience
Intended Audience
This guide is intended for system administrators and other technical staff
members responsible for configuring and managing adapters installed on servers
®
in Windows, Linux
, or VMware® environments.
What Is in This Guide
Following this preface, the remainder of this guide is organized into the following
chapters and appendices:
Chapter 1 Product Overview provides a product functional description, a list
of features, adapter management tool descriptions, and the adapter
specifications.
Chapter 2 Hardware Installation describes how to install the adapter,
including the list of system requirements and a preinstallation checklist.
Chapter 3 Driver Installation describes the installation of the adapter drivers
on Windows, Linux, and VMware.
Chapter 4 Firmware Upgrade Utility describes the use of the utility to
upgrade adapter firmware and boot code.
Chapter 5 Adapter Preboot Configuration describes the preboot adapter
configuration tasks using the Human Infrastructure Interface (HII)
application.
Chapter 6 Boot from SAN Configuration covers boot from SAN configuration
for both iSCSI and FCoE.
Chapter 7 RoCE Configuration describes how to configure the adapter, the
Ethernet switch, and the host to use RDMA over converged Ethernet
(RoCE).
Chapter 8 iWARP Configuration provides procedures for configuring Internet
wide area RDMA protocol (iWARP) on Windows, Linux, and VMware
ESXi 6.7 systems.
Chapter 9 iSER Configuration describes how to configure iSCSI Extensions
for RDMA (iSER) for Linux RHEL, SLES, Ubuntu, and ESXi 6.7.
Chapter 10 iSCSI Configuration describes iSCSI boot and iSCSI offload for
Windows and Linux.
Chapter 11 FCoE Configuration covers configuring Linux FCoE offload.
Chapter 12 SR-IOV Configuration provides procedures for configuring single
root input/output virtualization (SR-IOV) on Windows, Linux, and VMware
systems.
xix BC0154501-00 P
Preface
Related Materials
Chapter 13 NVMe-oF Configuration with RDMA demonstrates how to
configure NVMe-oF on a simple network for 45000 Series Adapters.
Chapter 14 Windows Server 2016 describes the Windows Server 2016
features.
Chapter 15 Traffic Control Offload describes the traffic control (TC) offload
feature.
Chapter 16 Troubleshooting describes a variety of troubleshooting methods
and resources.
Appendix A Adapter LEDS lists the adapter LEDs and their significance.
Appendix B Cables and Optical Modules lists the cables, optical modules,
and switches that the 45000 Series Adapters support.
Appendix C Feature Constraints provides information about feature
constraints implemented in the current release.
At the end of this guide are a glossary of terms and an index to help you quickly
locate the information you need.
Related Materials
For additional information, refer to the following documents that are available on
the Marvell Web Site:
Installation Guide—QConvergeConsole GUI (part number SN0051105-00)
contains detailed information on how to install and use the
QConvergeConsole GUI management tool.
User's Guide—QLogic Control Suite CLI (part number BC0054511-00)
contains detailed information on how to install, start, and use QLogic Control
Suite CLI.
User's Guide—PowerShell (part number BC0054518-00) contains detailed
information on how to install QConvergeConsole FastLinQ PowerKit to
manage the FastLinQ adapters on the system.
User’s Guide—QConvergeConsole Plug-ins for vSphere (part number
SN0054677-00) provides details for using the two QCC plug-ins to extend
the capabilities of VMware vCenter Server and the vSphere Web Client.
User’s Guide—FastLinQ ESXCLI VMware Plug-in (part number
BC0151101-00) describes the plug-in that extends the capabilities of the
®
CLI to manage Cavium QLogic 3400, 8400, 41000, and 45000 Series
ESX
Adapters installed in VMware ESX/ESXi hosts.
For information about downloading documentation from the Marvell Web site, see
“Downloading Updates and Documentation” on page xxiii.
xx BC0154501-00 P
Preface
NOTE
CAUTION
CAUTION
!
!
WARNING
Documentation Conventions
In addition the QConvergeConsole GUI help system provides topics available
while using the QCC GUI.
Documentation Conventions
This guide uses the following documentation conventions:
provides additional information.
without an alert symbol indicates the presence of a hazard
that could cause damage to equipment or loss of data.
with an alert symbol indicates the presence of a hazard that
could cause minor or moderate injury.
indicates the presence of a hazard that could cause serious
injury or death.
Text in blue font indicates a hyperlink (jump) to a figure, table, or section in
this guide, and links to Web sites are shown in underlined blue
example:
Table 9-2 lists problems related to the user interface and remote agent.
See “Installation Checklist” on page 6.
For more information, visit www.marvell.com
Text in bold font indicates user interface elements such as a menu items,
buttons, check boxes, or column headings. For example:
Click the Start button, point to Programs, point to Accessories, and
then click Command Prompt.
Under Notification Options, select the Warning Alarms check box.
Te xt i n
text. For example:
To return to the root directory from anywhere in the file structure:
Issue the following command:
Key names and key strokes are indicated with UPPERCASE:
Courier font indicates a file name, directory path, or command line
Type
cd/ root and press ENTER.
sh ./install.bin.
.
. For
Press CTRL+P.
Press the UP ARROW key.
xxi BC0154501-00 P
Preface
NOTE
Documentation Conventions
Text in italics indicates terms, emphasis, variables, or document titles. For
example:
What are shortcut keys?
To enter the date type mm/dd/yyyy (where mm is the month, dd is the
Topic titles between quotation marks identify related topics either within this
manual or in the online help, which is also referred to as the help system
throughout this document.
Command line interface (CLI) command syntax conventions include the
following:
Plain text indicates items that you must type as shown. For example:
<> (angle brackets) indicate a variable whose value you must
day, and yyyy is the year).
qaucli -pr nic -ei
specify. For example:
<serial_number>
For CLI commands only, variable names are always indicated
using angle brackets instead of italics.
[] (square brackets) indicate an optional parameter. For example:
[<file_name>] means specify a file name, or omit it to select
the default file name.
| (vertical bar) indicates mutually exclusive options; select one option
only. For example:
on|off
1|2|3|4
... (ellipsis) indicates that the preceding item may be repeated. For
example:
x... means one or more instances of x.
[x...] means zero or more instances of x.
⋮ (vertical ellipsis) within command example output indicate where
portions of repetitious output data have been intentionally omitted.
xxii BC0154501-00 P
Preface
Technical Support
() (parentheses) and { } (braces) are used to avoid logical
ambiguity. For example:
a|b c is ambiguous
{(a|b) c} means a or b, followed by c
{a|(b c)}
Technical Support
Customers should contact their authorized maintenance provider for technical
support of their Cavium QLogic products. Technical support for QLogic-direct
products under warranty is available with a Cavium support account.
To set up a support account (if needed) and submit a case:
means either a, or b c
1. Go to www.marvell.com
2. Point to Support. Under Marvell Extranets, click LOGIN NOW.
3. In the Marvell QLogic Extranet box, click SELECT.
4. Do one of the following:
If you do not have an account, click Sign Up. You are directed to a
page to create a support account.
If you have an account, enter your User Name and Password, and
then click Sign In.
5. From the links at the top of the page, click Submit a Case.
.
Downloading Updates and Documentation
The Marvell Web site provides periodic updates to product firmware, software,
and documentation.
To download QLogic firmware, software, and documentation:
1. Go to www.marvell.com
2. Point to Support, and then under Driver Downloads, click Marvell
QLogic/FastLinQ Drivers.
3. On the Cavium Downloads and Documentation page, click Adapters.
.
4. Click the corresponding button to search by Model or by Operating
System.
5. To define a search, click an item in each selection column, and then click
Go.
6. Locate the firmware, software, or document you need, and then click the
item’s name or icon to download or open the item.
xxiii BC0154501-00 P
Preface
CLASS I LASER
Legal Notices
Knowledgebase
The Cavium QLogic knowledgebase is an extensive collection of product
information that you can search for specific solutions. Cavium is constantly adding
to the collection of information in the database to provide answers to your most
urgent questions.
To access the knowledgebase:
1. Go to www.marvell.com
2. Point to Support, and then under Support by Product, click GET
SUPPORT.
3. On the Support page, select the product line for your adapter:
QLogic Fibre Channel Adapters and Controllers
®
FastLinQ
| LiquidIO® Ethernet Adapters and Controllers
4. Under Related Links, click Knowledgebase.
Legal Notices
Legal notices covered in this section include laser safety (FDA notice), agency
certification, and product safety compliance.
Laser Safety—FDA Notice
This product complies with DHHS Rules 21CFR Chapter I, Subchapter J. This
product has been designed and manufactured according to IEC60825-1 on the
safety label of laser product.
.
Class 1
Laser Product
Appareil laser
de classe 1
Produkt der
Laser Klasse 1
Luokan 1 Laserlaite Varoitu s—Luokan 1 lasersäteilyä, kun laite on auki
Caution—Class 1 laser radiation when open
Do not view directly with optical instruments
Attention—Radiation laser de classe 1
Ne pas regarder directement avec des instruments optiques
Vorsi c h t—Laserstrahlung der Klasse 1 bei geöffneter Abdeckung
Direktes Ansehen mit optischen Instrumenten vermeiden
Älä katso suoraan laitteeseen käyttämällä optisia instrumenttej
xxiv BC0154501-00 P
Preface
Legal Notices
Agency Certification
The following sections summarize the EMC and EMI test specifications performed
on the 45000 Series Adapters.
EMI and EMC Requirements
FCC Part 15 compliance: Class A
FCC compliance information statement: This device complies with Part 15 of
the FCC Rules. Operation is subject to the following two conditions: (1) this device
may not cause harmful interference, and (2) this device must accept any
interference received, including interference that may cause undesired operation.
ICES-003 Compliance: Class A
This Class A digital apparatus complies with Canadian ICES-003. Cet appareil
numériqué de la classe A est conformé à la norme NMB-003 du Canada.
CE Mark 2014/30/EU, 2014/35/EU EMC Directive Compliance:
EN55032:2012/ CISPR 32:2015 Class A
EN55024: 2010
EN61000-3-2: Harmonic Current Emission
EN61000-3-3: Voltage Fluctuation and Flicker
Immunity Standards
EN61000-4-2: ESD
EN61000-4-3: RF Electro Magnetic Field
EN61000-4-4: Fast Transient/Burst
EN61000-4-5: Fast Surge Common/ Differential
EN61000-4-6: RF Conducted Susceptibility
EN61000-4-8: Power Frequency Magnetic Field
EN61000-4-11: Voltage Dips and Interrupt
VCCI: 2015-04; Class A
AS/NZS; CISPR 32: 2015 Class A
CNS 13438: 2006 Class A
xxv BC0154501-00 P
Preface
Legal Notices
KCC: Class A
Korea RRA Class A Certified
Product Name/Model: Converged Network Adapters and
Intelligent Ethernet Adapters
Certification holder: QLogic Corporation
Manufactured date: Refer to date code listed on product
Manufacturer/Country of origin: QLogic Corporation/USA
VCCI: Class A
A class equipment
(Business purpose
info/telecommunications
equipment)
As this equipment has undergone EMC registration for business purpose, the seller and/or the buyer is asked to beware
of this point and in case a wrongful sale or purchase has
been made, it is asked that a change to household use be
made.
Korean Language Format—Class A
This is a Class A product based on the standard of the Voluntary Control Council
for Interference (VCCI). If this equipment is used in a domestic environment, radio
interference may occur, in which case the user may be required to take corrective
actions.
Product Safety Compliance
UL, cUL product safety:
UL 60950-1 (2nd Edition) A1 + A2 2014-10-14
CSA C22.2 No.60950-1-07 (2nd Edition) A1 +A2 2014-10
Use only with listed ITE or equivalent.
Complies with 21 CFR 1040.10 and 1040.11, 2014/30/EU, 2014/35/EU.
xxvi BC0154501-00 P
Preface
Legal Notices
2006/95/EC low voltage directive:
TUV EN60950-1:2006+A11+A1+A12+A2 2nd Edition
TUV IEC 60950-1: 2005 2nd Edition Am1: 2009 + Am2: 2013 CB
CB Certified to IEC 60950-1 2nd Edition
xxvii BC0154501-00 P
1 Product Overview
This chapter provides the following information for the 45000 Series Adapters:
Functional Description
Features
“Adapter Management” on page 3
“Adapter Specifications” on page 5
Functional Description
The Cavium FastLinQ 45000 Series Adapters include 10, 25, 40, and 100Gb
Converged Network Adapters and Intelligent Ethernet Adapters that are designed
to perform accelerated data networking for server systems. The 45000 Series
Adapters include a 10/25/40/50/100Gb Ethernet MAC with full-duplex capability.
Features
The 45000 Series Adapters provide the following features. Some features may
not be available on all adapters:
NIC partitioning (NPAR)
Universal RDMA: Remote Direct Memory Access over converged Ethernet,
versions 1 and 2 (RoCE v1/v2) and Internet wide area Protocol (iWARP):
iSCSI Extensions for RDMA (iSER)
NVMe over Fabrics (NVMe-oF) (For details, download the Deployment
Guide: Non-volatile Memory Express over Fabrics with Universal
Remote Direct Memory Access, part number BC0054519-00)
NFS over RDMA (NFSoRDMA)
25G and 40G 45000 Series Adapters support iWARP
iSCSI and FCoE hardware offloads (QL45462 adapter only)
Data center bridging (DCB):
Enhanced transmission selection (ETS; IEEE 802.1Qaz)
Priority-based flow control (PFC; IEEE 802.1Qbb)
1 BC0154501-00 P
1–Product Overview
Features
Single-chip solution:
Performance features:
Data center bridging eXchange protocol (DCBX; CEE version 1.01,
IEEE)
25/40/100Gb MAC
SerDes interface for direct attach copper (DAC) transceiver connection
PCI Express
®
(PCIe®) 3.0 x8
Zero copy capable hardware
TCP, IP, UDP checksum offloads
TCP segmentation offload (TSO)
Large segment offload (LSO)
Generic segment offload (GSO)
HW based Generic Receive Offload (HW-GRO)
Large receive offload (LRO)
Receive segment coalescing (RSC)
Microsoft
®
dynamic virtual machine queue (VMQ), and Linux
Multiqueue
Adaptive interrupts:
Transmit/receive side scaling (TSS/RSS)
Accelerated Receive Flow Steering (ARFS)
Stateless offloads for Network Virtualization using Generic Routing
Encapsulation (NVGRE), Generic Network Virtualization
Encapsulation (GENEVE), and virtual LAN (VXLAN) L2/L3 GRE
tunneled traffic
1
Manageability:
System management bus (SMB) controller
Advanced Configuration and Power Interface (ACPI) 1.1a compliant
(multiple power modes)
Network controller-sideband interface (NC-SI) support
The management applications listed in Adapter Management
1
This feature requires OS or Hypervisor support to use the offloads.
2 BC0154501-00 P
1–Product Overview
Adapter Management
Advanced network features:
Virtualization (SR-IOV Hypervisor)
Jumbo frames (up to 9,600 bytes). The OS and the link partner must
support jumbo frames.
Virtual LANs (vLANs)
Flow control (IEEE Std 802.3x)
VMware NetQueue
Windows Hyper-V
®
Virtual Machine Queue (VMQ)
Linux Multiqueue
Windows, Linux, and VMware switch-independent NPAR
Windows Hyper-V
VMware ESXi
2
, Linux Kernel-based Virtual Machine (KVM), and
™
SR-IOV
Logical link control (IEEE Std 802.2)
High-speed on-chip reduced instruction set computer (RISC) processor
Integrated 96KB frame buffer memory (not applicable to all models)
1,024 classification filters (not applicable to all models)
Support for multicast addresses through 128-bit hashing hardware function
Serial flash NVRAM memory
PCI Power Management Interface (v1.1)
64-bit base address register (BAR) support
EM64T processor support
iSCSI and FCoE boot support
3
Adapter Management
The following applications are available to manage 45000 Series Adapters:
QLogic Control Suite CLI
QLogic QConvergeConsole GUI
QLogic QConvergeConsole vCenter Plug-in
QConvergeConsole PowerKit
FastLinQ ESXCLI VMware Plug-in
2
The QL456x1 (1×100G) adapter does not currently support SR-IOV on Windows Hyper-V.
3
Hardware support limit of SR-IOV VFs varies. The limit may be lower in some OS environments;
refer to the appropriate section for your OS.
3 BC0154501-00 P
Loading...