Qlogic FastLinQ 3400 Series, FastLinQ 8400 Series User Manual

Download

User’s Guide

Converged Network Adapters and Intelligent

Ethernet Adapters

QLogic FastLinQ 3400, 8400 Series

83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

This document is provided for informational purposes only and may contain errors. QLogic reserves the right, without notice, to make changes to this document or in product design or specifications. QLogic disclaims any warranty of any kind, expressed or implied, and does not guarantee that any results or performance described in the document will be achieved by you. All statements regarding QLogic's future direction and intent are subject to change or withdrawal without notice and represent goals and objectives only.

Document Revision History

Revision A, October 20, 2014 Revision B, November 14, 2014 Revision C, April 3, 2015 Revision D, September 16, 2015 Revision E, May 10, 2016

Changes Sections Affected

Updated to replace the QLogic Control Suite GUI

Throughout

with the QConvergeConsole GUI. In the Note, removed Separate licenses are

“Functional Description” on page 1

required for all offloading technologies. Added a QConvergeConsole Plug-ins fo r vSphere

“Features” on page 2

to the list of manageability features. Added the Adapter Management section. “Adapter Management” on page 6 In step 1, removed the reference to the table of

“Connecting the Network Cables” on page 11

10GbE optics. Added the Manually Extracting the Device Drivers

section. In the KMP Packages bullet, added a reference to

“Manually Extracting the Device Drivers” on page 20

“Packaging” on page 26

the QLogic Control Suite CLI User’s Guide. Removed the chapters Installing Management

Applications, Using QLogic Control Suite, and Manageability.

Updated the first sentence to read, The optional

“enable_vxlan_offld” on p age 54

parameter enable_vxlan_ofld can be used to enable or disable . . .

In the Note in step 5, added the option of configuring iSCSI boot parameters with UEFI HII BIOS pages.

Added a procedure to inject adapter drivers into the Windows image files.

In the fourth question, replaced configurations with IP addresses.

“MBA Boot Protocol Configuration” on page 71

“Injecting (Slipstreaming) Adapter Drivers into Windows Image Files” on page 91

“Event Log Messages” on page 105

ii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters

QLogic FastLinQ 3400, 8400 Series

Removed NX2 from the heading. “Bind iSCSI Target to QLogic iSCSI Transport

Name” on page 111

At the end of the first paragraph, added and the

other PFs on that port can support SR-IOV VF connections..

Updated all instances of Advanced Server Program and ASP to QLogic Advanced Server Program and QLASP, respectively.

Added a paragraph at the end of the section describing two traffic classes that can be used by the Windows QoS service.

Removed the table describing the QLogic Teaming Software Component.

In Table 15-4, updated the subheads to read Generic (Static) Trunking and Dynamic LACP.

In the first bullet, added with Auto-Fallback Enabled (SLB).

In the second paragraph, last sentence, updated bnx2 to read bnx2/bnx2x.

Removed sections Running a Cable Length Test and Testing Network Connectivity.

“SR-IOV and Storage” on page 161

Throughout

“Data Center Bridging in Windows Server 2012” on page 172

“Software Components” on page 183

“Teaming Mechanisms” on page 188

“QLASP Overview” on page 233 “Types of Teams” on page 234

“Linux” on page 251

Chapter 18, Troubleshooting

iii 83840-546-00 E

User’s Guide Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

iv 83840-546-00 E

Preface

Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

What Is in This Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

Related Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxii

Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

License Agreements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv

Technical Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv

Downloading Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv

Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi

Contact Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi

Knowledge Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi

Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Warranty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Laser Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

FDA Notice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Agency Certification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

EMI and EMC Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Product Safety Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxviii

1 Product Overview

Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Adaptive Interrupt Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

ASIC with Embedded RISC Processor . . . . . . . . . . . . . . . . . . . . . . . . 5

Adapter Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

QLogic Control Suite CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

QLogic QConvergeConsole Graphical User Interface. . . . . . . . . . . . . 6

QLogic QConvergeConsole vCenter Plug-In. . . . . . . . . . . . . . . . . . . . 6

QLogic FastLinQ ESXCLI VMware Plug-In. . . . . . . . . . . . . . . . . . . . . 6

Supported Operating Environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

v 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

Adapter Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Physical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Standards Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Installing the Hardware

System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Operating System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Safety Precautions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Preinstallation Checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Installation of the Network Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Connecting the Network Cables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Multi-boot Agent (MBA) Driver Software

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Setting Up MBA in a Client Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Enabling the MBA Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Configuring the MBA Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Setting Up the BIOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Setting Up MBA in a Server Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Red Hat Linux PXE Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

MS-DOS UNDI/Intel APITEST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Windows Driver Software

Installing the Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Using the Installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Using Silent Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Manually Extracting the Device Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Removing the Device Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Installing QLogic Management Applications . . . . . . . . . . . . . . . . . . . . . . . . 21

Viewing or Changing the Adapter Properties. . . . . . . . . . . . . . . . . . . . . . . . 21

Setting Power Management Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Linux Driver Software

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

bnx2i Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

bnx2fc Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Packaging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Installing Linux Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

vi 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

Installing the Source RPM Package . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Installing the KMP Package. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Building the Driver from the Source TAR File . . . . . . . . . . . . . . . . . . . 31

Load and Run Necessary iSCSI Software Components . . . . . . . . . . . . . . . 32

Unloading/Removing the Linux Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Unloading/Removing the Driver from an RPM Installation . . . . . . . . . 33

Removing the Driver from a TAR Installation . . . . . . . . . . . . . . . . . . . 33

Uninstalling the QCC GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Patching PCI Files (Optional). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Network Installations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Setting Values for Optional Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

disable_tpa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

int_mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

dropless_fc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

disable_iscsi_ooo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

multi_mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

num_queues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

pri_map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

qs_per_cos. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

cos_min_rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

bnx2i Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

error_mask1 and error_mask2. . . . . . . . . . . . . . . . . . . . . . . . . . 38

en_tcp_dack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

time_stamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

sq_size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

rq_size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

event_coal_div . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

last_active_tcp_port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

ooo_enable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

bnx2fc Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

debug_logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Driver Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

bnx2 Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

vii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Driver Sign On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

CNIC Driver Sign On (bnx2 only) . . . . . . . . . . . . . . . . . . . . . . . . 42

NIC Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Link Up and Speed Indication. . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Link Down Indication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

MSI-X Enabled Successfully . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

bnx2i Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

BNX2I Driver Signon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Network Port to iSCSI Transport Name Binding. . . . . . . . . . . . . 43

Driver Completes handshake with iSCSI Offload-enabled

CNIC Device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Driver Detects iSCSI Offload Is Not Enabled on the CNIC Device 43 Exceeds Maximum Allowed iSCSI Connection Offload Limit. . . 43 Network Route to Target Node and Transport Name Binding

Are Two Different Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Target Cannot Be Reached on Any of the CNIC Devices. . . . . . 43

Network Route Is Assigned to Network Interface, Which Is Down 43

SCSI-ML Initiated Host Reset (Session Recovery) . . . . . . . . . . 43

CNIC Detects iSCSI Protocol Violation - Fatal Errors. . . . . . . . . 44

CNIC Detects iSCSI Protocol Violation - Non-FATAL, Warning . 45

Driver Puts a Session Through Recovery. . . . . . . . . . . . . . . . . . 45

Reject iSCSI PDU Received from the Target . . . . . . . . . . . . . . . 45

Open-iSCSI Daemon Handing Over Session to Driver . . . . . . . 45

bnx2fc Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

BNX2FC Driver Signon: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Driver Compiles Handshake with FCoE Offload Enabled

CNIC Device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Driver Fails Handshake with FCoE Offload Enabled CNIC Device 45

No Valid License to Start FCoE . . . . . . . . . . . . . . . . . . . . . . . . . 46

Session Failures Due to Exceeding Maximum Allowed FCoE

Offload Connection Limit or Memory Limits . . . . . . . . . . . . . . . 46

Session Offload Failures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Session Upload Failures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Unable to Issue ABTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Unable to Recover the IO Using ABTS (Due to ABTS Timeout) 46

Unable to Issue IO Request Due to Session Not Ready . . . . . . 46

Drop Incorrect L2 Receive Frames. . . . . . . . . . . . . . . . . . . . . . . 46

HBA/lport Allocation Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

NPIV Port Creation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

viii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

Teaming with Channel Bonding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6 VMware Driver Software

Packaging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Download, Install, and Update Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Networking Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Driver Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

int_mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

disable_tpa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

num_rx_queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

num_tx_queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

pri_map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

qs_per_cos. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

cos_min_rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

dropless_fc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

RSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

max_vfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

enable_vxlan_offld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Driver Defaults. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Unloading and Removing Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Driver Sign On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

NIC Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

MSI-X Enabled Successfully . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Link Up and Speed Indication. . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Link Down Indication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Memory Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

MultiQueue/NetQueue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

FCoE Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Enabling FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Installation Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Supported Distributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7 Firmware Upgrade

Upgrading Firmware for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Upgrading Firmware for Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

ix 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

8 iSCSI Protocol

iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Supported Operating Systems for iSCSI Boot. . . . . . . . . . . . . . . . . . . 68

iSCSI Boot Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Configuring the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Configuring iSCSI Boot Parameters. . . . . . . . . . . . . . . . . . . . . . 69

MBA Boot Protocol Configuration. . . . . . . . . . . . . . . . . . . . . . . . 71

iSCSI Boot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Enabling CHAP Authentication. . . . . . . . . . . . . . . . . . . . . . . . . . 79

Configuring the DHCP Server to Support iSCSI Boot . . . . . . . . 79

DHCP iSCSI Boot Configurations for IPv4. . . . . . . . . . . . . . . . . 79

DHCP iSCSI Boot Configuration for IPv6. . . . . . . . . . . . . . . . . . 81

Configuring the DHCP Server . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Preparing the iSCSI Boot Image. . . . . . . . . . . . . . . . . . . . . . . . . 83

Booting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Configuring VLANs for iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Other iSCSI Boot Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Changing the Speed and Duplex Settings in

Windows Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Virtual LANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

The 'dd' Method of Creating an iSCSI Boot Image. . . . . . . . . . . 95

Troubleshooting iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

iSCSI Crash Dump. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

iSCSI Offload in Windows Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

iSCSI Offload Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Configuring iSCSI Offload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Installing QLogic Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Installing the Microsoft iSCSI Initiator. . . . . . . . . . . . . . . . . . . . . 99

Configure Microsoft Initiator to Use QLogic’s iSCSI Offload. . . . 99

iSCSI Offload FAQs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Event Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

iSCSI Offload in Linux Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Open iSCSI User Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

User Application - qlgc_iscsiuio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Bind iSCSI Target to QLogic iSCSI Transport Name. . . . . . . . . . . . . . 111

VLAN Configuration for iSCSI Offload (Linux). . . . . . . . . . . . . . . . . . . 112

Modifying the iSCSI iface File. . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Setting the VLAN ID on the Ethernet Interface. . . . . . . . . . . . . . 112

x 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

Making Connections to iSCSI Targets. . . . . . . . . . . . . . . . . . . . . . . . . 113

Add Static Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

iSCSI Target Discovery Using 'SendTargets' . . . . . . . . . . . . . . . 113

List All Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

List All Drives Active in the System . . . . . . . . . . . . . . . . . . . . . . 114

Maximum Offload iSCSI Connections. . . . . . . . . . . . . . . . . . . . . . . . . 114

Linux iSCSI Offload FAQ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

iSCSI Offload on VMware Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

9 Fibre Channel Over Ethernet

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

FCoE Boot from SAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Preparing System BIOS for FCoE Build and Boot. . . . . . . . . . . . . . . . 118

Modify System Boot Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Specify BIOS Boot Protocol (if required) . . . . . . . . . . . . . . . . . . 118

Prepare QLogic Multiple Boot Agent for FCoE Boot. . . . . . . . . . . . . . 119

UEFI Boot LUN Scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Provisioning Storage Access in the SAN. . . . . . . . . . . . . . . . . . . . . . . 125

Pre-Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

CTRL+R Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

One-Time Disabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Windows Server 2008 R2 and Windows Server 2008 SP2

FCoE Boot Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Windows Server 2012/2102 R2 FCoE Boot Installation . . . . . . . . . . . 130

Linux FCoE Boot Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

SLES11 SP2 Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

RHEL6 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Linux: Adding Additional Boot Paths . . . . . . . . . . . . . . . . . . . . . 142

VMware ESXi FCoE Boot Installation . . . . . . . . . . . . . . . . . . . . . . . . . 144

Configuring FCoE Boot from SAN on VMware. . . . . . . . . . . . . . 148

Booting from SAN After Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Driver Upgrade on Linux Boot from SAN Systems . . . . . . . . . . . . . . . 149

Errors During Windows FCoE Boot from SAN Installation . . . . . . . . . 150

Configuring FCoE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

10 NIC Partitioning and Bandwidth Management

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

Supported Operating Systems for NIC Partitioning. . . . . . . . . . . . . . . 152

Configuring for NIC Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

xi 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Number of Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Network MAC Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

iSCSI MAC Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Physical Link Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Relative Bandwidth Weight (%). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Maximum Bandwidth (%). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

11 Virtual LANs in Windows

VLAN Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Adding VLANs to Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

12 SR-IOV

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Enabling SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

SR-IOV and Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

SR-IOV and Jumbo Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

13 Microsoft Virtualization with Hyper-V

Supported Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Single Network Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Windows Server 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Windows Server 2008 R2 and 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Teamed Network Adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Windows Server 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Windows Server 2008 R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Configuring VMQ with SLB Teaming. . . . . . . . . . . . . . . . . . . . . . . . . . 167

Upgrading Windows Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

14 Data Center Bridging (DCB)

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

DCB Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Enhanced Transmission Selection (ETS) . . . . . . . . . . . . . . . . . . . . . . 170

Priority Flow Control (PFC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Data Center Bridging eXchange (DCBX) . . . . . . . . . . . . . . . . . . . . . . 170

Configuring DCB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

DCB Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Data Center Bridging in Windows Server 2012 . . . . . . . . . . . . . . . . . . . . . . 172

xii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

15 QLogic Teaming Services

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Teaming Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Network Addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Teaming and Network Addresses. . . . . . . . . . . . . . . . . . . . . . . . 177

Description of Teaming Types . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Software Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Repeater Hub. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Switching Hub. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Teaming Support by Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Configuring Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Supported Features by Team Type. . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Selecting a Team Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Teaming Mechanisms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Outbound Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Inbound Traffic Flow (SLB Only). . . . . . . . . . . . . . . . . . . . . . . . . 190

Protocol Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Types of Teams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Switch-Independent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Switch-Dependent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

LiveLink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Attributes of the Features Associated with Each Type of Team . . . . . 196

Speeds Supported for Each Type of Team . . . . . . . . . . . . . . . . . . . . . 199

Teaming and Other Advanced Networking Properties . . . . . . . . . . . . . . . . 199

Checksum Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

IEEE 802.1p QoS Tagging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Large Send Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Jumbo Frames. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

IEEE 802.1Q VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Preboot Execution Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

General Network Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Teaming with Microsoft Virtual Server 2005 . . . . . . . . . . . . . . . . . . . . 202

Teaming Across Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Switch-Link Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

xiii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

Spanning Tree Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Topology Change Notice (TCN) . . . . . . . . . . . . . . . . . . . . . . . . . 208

Port Fast/Edge Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Layer 3 Routing/Switching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Teaming with Hubs (for troubleshooting purposes only) . . . . . . . . . . . 209

Hub Usage in Teaming Network Configurations. . . . . . . . . . . . . 209

SLB Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

SLB Team Connected to a Single Hub. . . . . . . . . . . . . . . . . . . . 210

Generic and Dynamic Trunking (FEC/GEC/IEEE 802.3ad) . . . . 210

Teaming with Microsoft NLB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Application Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Teaming and Clustering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Microsoft Cluster Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

High-Performance Computing Cluster . . . . . . . . . . . . . . . . . . . . 213

Oracle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Teaming and Network Backup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Load Balancing and Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Fault Tolerance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Troubleshooting Teaming Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Teaming Configuration Tips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Troubleshooting Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Windows System Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . 224

Base Driver (Physical Adapter/Miniport) . . . . . . . . . . . . . . . . . . . . . . . 224

Intermediate Driver (Virtual Adapter/Team). . . . . . . . . . . . . . . . . . . . . 228

Virtual Bus Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

16 Configuring Teaming in Windows Server

QLASP Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Load Balancing and Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

Types of Teams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

Smart Load Balancing and Failover . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Link Aggregation (802.3ad). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Generic Trunking (FEC/GEC)/802.3ad-Draft Static. . . . . . . . . . . . . . . 236

SLB (Auto-Fallback Disable) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

Limitations of Smart Load Balancing and Failover/SLB

(Auto-Fallback Disable) Types of Teams. . . . . . . . . . . . . . . . . . . . . . 237

Teaming and Large Send Offload/Checksum Offload Support . . . . . . 238

xiv 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

17 User Diagnostics in DOS

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Performing Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Diagnostic Test Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

18 Troubleshooting

Hardware Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

QCC GUI Diagnostic Tests Failures . . . . . . . . . . . . . . . . . . . . . . . . . . 248

QCC Network Test Failures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

Checking Port LEDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

Troubleshooting Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Checking if Current Drivers are Loaded. . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Possible Problems and Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Multi-boot Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

QLASP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

NPAR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

A Adapter LEDS

xv 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

xvi 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters

QLogic FastLinQ 3400, 8400 Series

List of Figures

Figure Page

3-1 MBA Configuration Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4-1 Power Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6-1 Selecting an Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6-2 QLE3442 Driver Versions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6-3 PCI Identifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

6-4 List of Driver Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

6-5 Download Driver Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

8-1 QLogic 577xx/578xx Ethernet Boot Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

8-2 CCM Device List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8-3 Selecting MBA Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8-4 Selecting the iSCSI Boot Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

8-5 Selecting iSCSI Boot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

8-6 Selecting General Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

8-7 Saving the iSCSI Boot Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

8-8 Comprehensive Configuration Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

8-9 Configuring VLANs—CCM Device List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

8-10 Configuring VLANs—Multiboot Agent Configuration . . . . . . . . . . . . . . . . . . . . . . . . 93

8-11 Configuring iSCSI Boot VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

8-12 Saving the iSCSI Boot VLAN Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

8-13 iSCSI Initiator Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

8-14 iSCSI Initiator Node Name Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

8-15 iSCSI Initiator—Add a Target Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

8-16 Target Portal IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

8-17 Selecting the Local Adapter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

8-18 Selecting the Initiator IP Address. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

8-19 Adding the Target Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

8-20 Logging on to the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

8-21 Log On to Target Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

8-22 Assigning a VLAN Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

8-23 Configuring the VLAN on VMKernel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

9-1 FCoE Boot<Variable>—CCM Device List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

9-2 FCoE Boot<Variable>—Enable DCB/DCBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

9-3 FCoE Boot<Variable>—Select FCoE Boot Protocol . . . . . . . . . . . . . . . . . . . . . . . . 120

9-4 FCoE Boot<Variable>—Target Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

9-5 FCoE Boot<Variable>—Specify Target WWPN and Boot LUN . . . . . . . . . . . . . . . . 122

9-6 FCoE Boot Target Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

9-7 FCoE Boot Configuration Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

9-8 FCoE Target Parameters Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

9-9 Selecting an FCoE WWPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

9-10 One-time Disabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

9-11 Load EVBD Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

9-12 Load bxfcoe Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

9-13 Selecting the FCoE Boot LUN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

xvii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

9-14 SLES Boot Options Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

9-15 Choosing Driver Update Medium. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

9-16 FCoE Reboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

11-1 Example of Servers Supporting Multiple VLANs with Tagging. . . . . . . . . . . . . . . . . 156

15-1 Process for Selecting a Team Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

15-2 Intermediate Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

15-3 Teaming Across Switches Without an Interswitch Link . . . . . . . . . . . . . . . . . . . . . . 204

15-4 Teaming Across Switches With Interconnect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

15-5 Failover Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

15-6 Team Connected to a Single Hub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

15-7 Clustering With Teaming Across One Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

15-8 Clustering With Teaming Across Two Switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

15-9 Network Backup without Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

15-10 Network Backup With SLB Teaming Across Two Switches . . . . . . . . . . . . . . . . . . . 218

xviii 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters

QLogic FastLinQ 3400, 8400 Series

List of Tables

Table Page

2-1 100/1000BASE-T and 10GBASE-T Cable Specifications . . . . . . . . . . . . . . . . . . . . 11

4-1 Windows Operating Systems and iSCSI Crash Dump. . . . . . . . . . . . . . . . . . . . . . . 19

5-1 QLogic 8400/3400 Series Linux Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6-1 VMware Driver Packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6-2 QLogic 8400/3400 Series FCoE Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

8-1 Configuration Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

8-2 DHCP Option 17 Parameter Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

8-3 DHCP Option 43 Suboption Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

8-4 DHCP Option 17 Suboption Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

8-5 Offload iSCSI (OIS) Driver Event Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . 105

11-1 Example VLAN Network Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

13-1 Configurable Network Adapter Hyper-V Features . . . . . . . . . . . . . . . . . . . . . . . . . . 163

13-2 Configurable Teamed Network Adapter Hyper-V Features . . . . . . . . . . . . . . . . . . . 165

15-1 Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

15-2 Available Teaming Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

15-3 Comparison of Team Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

15-4 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

15-5 Link Speeds in Teaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

15-6 Advanced Adapter Properties and Teaming Support. . . . . . . . . . . . . . . . . . . . . . . . 199

15-7 Base Driver Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

15-8 Intermediate Driver Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

15-9 VBD Event Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

16-1 Smart Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

17-1 uediag Command Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

17-2 Diagnostic Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

A-1 Network Link and Activity Indicated by the RJ-45 Port LEDs. . . . . . . . . . . . . . . . . . 257

A-2 Network Link and Activity Indicated by the Port LED. . . . . . . . . . . . . . . . . . . . . . . . 257

xix 83840-546-00 E

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters QLogic FastLinQ 3400, 8400 Series

xx 83840-546-00 E

Preface

NOTE

QLogic now supports QConvergeConsole® (QCC) GUI as the only GUI management tool across all QLogic adapters. The QLogic Control Suite (QCS) GUI is no longer supported for the 8400/3400 Series Adapters and adapters based on 57xx/57xxx controllers, and has been replaced by the QCC GUI management tool. The QCC GUI provides single-pane-of-glass GUI management for all QLogic adapters.

In Windows environments, when you run the QCS CLI and the Management Agents Installer, it will uninstall the QCS GUI (if installed on the system) and any related components from your system. To obtain the new GUI, download QCC GUI for your adapter from the QLogic Downloads Web page:

driverdownloads.qlogic.com

Intended Audience

This guide is intended for personnel responsible for installing and maintaining computer networking equipment.

What Is in This Guide

This guide describes the features, installation, and configuration of the QLogic® FastLinQ™ 8400/3400 Series Converged Network Adapters and Intelligent Ethernet Adapters. The guide is organized as follows:

 Chapter 1, Product Overview provides a product functional description, a list

of features, a list of supported operating systems, and the adapter specifications.

 Chapter 2, Installing the Hardware describes how to install the adapter

including the list of system requirements and a preinstallation checklist.

 Chapter 3, Multi-boot Agent (MBA) Driver Software describes the software

module that allows your network computer to boot with the images provided by remote servers across the network.

 Chapter 4, Windows Driver Software describes Windows

and removal, QLogic management application installation, adapter properties management, and power management options.

driver installation

xxi 83840-546-00 E

Preface Related Materials

 Chapter 5, Linux Driver Software describes the Linux® drivers.

 Chapter 6, VMware Driver Software describes the VMware

drivers.

 Chapter 7, Firmware Upgrade describes the installation and use of the

firmware upgrade utility.

 Chapter 8, iSCSI Protocol describes iSCSI boot, iSCSI crash dump, and

iSCSI offload for Windows, Linux, and VMware.

 Chapter 9, Fibre Channel Over Ethernet describes FCoE boot from SAN

and booting from SAN after installation.

 Chapter 10, NIC Partitioning and Bandwidth Management describes the

NPAR operating system requirements and the NPAR configuration parameters.

 Chapter 11, Virtual LANs in Windows describes the use of VLANs to divide

the physical LAN into functional segments.

 Chapter 12, SR-IOV describes the use of Single-Root I/O Virtualization

(SR-IOV) to virtualize network controllers and how to enable SR-IOV.

 Chapter 13, Microsoft Virtualization with Hyper-V describes the use of

Microsoft

Hyper-V® for Windows Server 2008 and 2012.

 Chapter 14, Data Center Bridging (DCB) describes the DCB capabilities

configuration, and requirements.

 Chapter 15, QLogic T eaming Services describes the use of teaming to group

multiple physical devices to provide fault tolerance and load balancing.

 Chapter 16, Configuring T eamin g in Windows Server describ es the teaming

configuration for Windows Server

 Chapter 17, User Diagnostics in DOS describes the MS-DOS based

application that runs diagnostic tests, updates device firmware, and manages adapter properties.

 Chapter 18, Troubleshootingdescribes a variety of troubleshooting methods

and resources.

 Appendix A, Adapter LEDS describes the adapter LEDs and their

significance.

Related Materials

For information about downloading documentation from the QLogic Web site, see

“Downloading Updates” on page xxv.

operating systems.

xxii 83840-546-00 E

Preface

NOTE

CAUTION

WARNING

Documentation Conventions

This guide uses the following documentation conventions:

 provides additional information.

 without an alert symbol indicates the presence of a hazard

that could cause damage to equipment or loss of data.

 with an alert symbol indicates the presence of a hazard that

could cause minor or moderate injury.

 indicates the presence of a hazard that could cause serious

injury or death.

 Text in blue font indicates a hyperlink (jump) to a figure, table, or section in

this guide, and links to Web sites are shown in underlined blue example:

. For

 Table 9-2 lists problems related to the user interface and remote agent.  See “Installation Checklist” on page 6.  For more information, visit www.qlogic.com

 Text in bold font indicates user interface elements such as a menu items,

buttons, check boxes, or column headings. For example:  Click the Start button, point to Programs, point to Accessories, and

then click Command Prompt.

 Under Notification Options, select the Warning Alarms check box.

 Text in Courier font indicates a file name, directory path, or command line

text. For example:  To return to the root directory from anywhere in the file structure:

Type

cd /root and press ENTER.

 Enter the following command: sh ./install.bin

 Key names and key strokes are indicated with UPPERCASE:

 Press CTRL+P.  Press the UP ARROW key.

xxiii 83840-546-00 E

Preface License Agreements

 Text in italics indicates terms, emphasis, variables, or document titles. For

example:

 For a complete listing of license agreements, refer to the QLogic

Software End User License Agreement.

 What are shortcut keys?  To enter the date type mm/dd/yyyy (where mm is the month, dd is the

day, and yyyy is the year).

 Topic titles between quotation marks identify related topics either within this

manual or in the online help, which is also referred to as the help system throughout this document.

License Agreements

Refer to the QLogic Software End User License Agreement for a complete listing of all license agreements affecting this product.

xxiv 83840-546-00 E

Preface Technical Support

Technical Support

Customers should contact their authorized maintenance provider for technical support of their QLogic products. QLogic-direct customers may contact QLogic Technical Support; others will be redirected to their authorized maintenance provider. Visit the QLogic support Web site listed in Cont act Information for the latest firmware and software updates.

For details about available service plans, or for information about renewing and extending your service, visit the Service Program Web page at

http://www.qlogic.com/Support/Pages/ServicePrograms.aspx

Downloading Updates

The QLogic Web site provides periodic updates to product firmware, software, and documentation.

To download firmware, software, and documentation:

1. Go to the QLogic Downloads and Documentation page:

driverdownloads.qlogic.com

2. Type the QLogic model name in the search box.

3. In the search results list, locate and select the firmware, software, or documentation for your product.

4. View the product details Web page to ensure that you have the correct firmware, software, or documentation. For additional information, click Read Me and Release Notes under Support Files.

5. Click Download Now.

6. Save the file to your computer.

7. If you have downloaded firmware, software, drivers, or boot code, follow the installation instructions in the Readme file.

Instead of typing a model name in the search box, you can perform a guided search as follows:

1. Click the product type tab: Adapters, Switches, Routers, or ASICs.

2. Click the corresponding button to search by model or operating system.

3. Click an item in each selection column to define the search, and then click Go.

4. Locate the firmware, software, or document you need, and then click the item’s name or icon to download or open the item.

xxv 83840-546-00 E

Preface Technical Support

Training

QLogic Global Training maintains a Web site at www.qlogictraining.com offering online and instructor-led training for all QLogic products. In addition, sales and technical professionals may obtain Associate and Specialist-level certifications to qualify for additional benefits from QLogic.

Contact Information

QLogic Technical Support for products under warranty is available during local standard working hours excluding QLogic Observed Holidays. For customers with extended service, consult your plan for available hours. For Support phone numbers, see the Contact Support link at support.qlogic.com

Support Headquarters

QLogic Web Site Technical Support Web Site Technical Support E-mail Technical Training E-mail

Knowledge Database

The QLogic knowledge database is an extensive collection of QLogic product information that you can search for specific solutions. QLogic is constantly adding to the collection of information in the database to provide answers to your most urgent questions. Access the database from the QLogic Support Center:

support.qlogic.com.

QLogic Corporation 12701 Whitewater Drive Minnetonka, MN 55343 USA

www.qlogic.com support.qlogic.com support@qlogic.com training@qlogic.com

xxvi 83840-546-00 E

Preface

CLASS I LASER

Legal Notices

Warranty

For warranty details, please check the QLogic Web site:

http://www.qlogic.com/Support/Pages/Warranty.aspx

Laser Safety

FDA Notice

This product complies with DHHS Rules 21CFR Chapter I, Subchapter J. This product has been designed and manufactured according to IEC60825-1 on the safety label of laser product.

Class 1 Laser Product Appareil laser de classe 1 Produkt der Laser Klasse 1 Luokan 1 Laserlaite

Agency Certification

The following sections contain a summary of EMC and EMI test specifications performed on the QLogic adapters to comply with emission and product safety standards.

EMI and EMC Requirements

FCC Rules,CFR Title 47, Part 15, Subpart B:2013 Class A

This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation.

Caution—Class 1 laser radiation when open. Do not view directly with optical instruments

Attention—Radiation laser de classe 1. Ne pas regarder directement avec des instruments optiques.

Vorsicht—Laserstrahlung der Klasse 1 bei geöffneter Abdeckung. Direktes Ansehen mit optischen Instrumenten vermeiden.

Varoitus—Luokan 1 lasersäteilyä, kun laite on auki. Älä katso suoraan laitteeseen käyttämällä optisia instrumenttej.

Industry Canada, ICES-003:2012 Class A

This Class A digital apparatus complies with Canadian ICES-003.Cet appareil numériqué de la classe A est conformé à la norme NMB-003 du Canada.

xxvii 83840-546-00 E

Preface Legal Notices

CE Mark 2004/108/EC EMC Directive Compliance

EN55022:2010 Class A1:2007/CISPR22:2009+A1:2010 Class A EN55024:2010

EN61000-3-2:2006 A1 +A2:2009: Harmonic Current Emission EN61000-3-3:2008: Voltage Fluctuation and Flicker

VCCI

VCCI:2012-04; Class A

AS/NZS CISPR22

AS/NZS; CISPR 22:2009+A1:2010 Class A

KCC

KC-RRA KN22 KN24(2013) Class A

Product Safety Compliance

UL, cUL product safety:

UL60950-1 (2nd Edition), 2007 UL CSA C22.2 60950-1-07 (2nd Edition) 2007

Use only with listed ITE or equivalent. Complies with 21 CFR 1040.10 and 1040.11. 2006/95/EC low voltage directive: TUV EN60950-1:2006+A11+A1+A12 2nd edition

TUV IEC60950-1:2006 2nd Edition Am 1:2009 CB

xxviii 83840-546-00 E

1 Product Overview

NOTE

 Functional De scriptio n  Features  Supported Operating Environments  Adapter Management  Adapter Specifications

Functional Description

The QLogic 8400/3400 Series adapters are based on a new class of Gigabit Ethernet (GbE) and 10GbE converged network interface controller (C-NIC) that can simultaneously perform accelerated data networking and storage networking on a standard Ethernet network. The C-NIC offers acceleration for popular protocols used in the data center, such as:

 Internet Small Computer Systems Interface (iSCSI) offload for accelerating

network storage access featuring centralized boot (iSCSI boot)

 Fibre Channel over Ethernet (FCoE) offload and acceleration for Fibre

Channel block storage

Not all adapters support each listed protocol. For information about supported protocols, refer to the product data sheet at www.qlogic.com under Resources.

Enterprise networks that use multiple protocols and multiple network fabrics benefit from the network adapter’s ability to combine data communications, storage, and clustering over a single Ethernet fabric by boosting server CPU processing performance and memory use while alleviating I/O bottlenecks.

The QLogic 8400/3400 Series adapters include a 100/1000Mbps or 10Gbps Ethernet MAC with both half-duplex and full-duplex capability and a 100/1000Mbps or 10Gbps PHY. The transceiver is fully compatible with the IEEE

802.3 standard for auto-negotiation of speed.

1 83840-546-00 E

1–Product Overview Features

Using the QLogic teaming software, you can split your network into virtual LANs (VLANs) and group multiple network adapters together into teams to provide network load balancing and fault tolerance. See Chapter 15, QLogic Teaming

Services and Chapter 16, Configuring Teaming in Windows Server for detailed

information about teaming. See Chapter 11, Virtual LANs in Windows for a description of VLANs. See “Configuring Teaming” on page 185 for instructions on configuring teaming and creating VLANs on Windows operating systems.

Features

The following is a list of the QLogic 8400/3400 Series adapters features. Some features may not be available on all adapters.

 iSCSI offload  FCoE offload  NIC partitioning (NPAR)  Data center bridging (DCB)

 Enhanced transmission selection (ETS; IEEE 802.1Qaz)  Priority-based flow control (PFC; IEEE 802.1Qbb)  Data center bridging capability exchange protocol (DCBX; CEE

version 1.01)

 Single-chip solution (excluding QLE3442-RJ)  10/100/1000G triple-speed MAC (QLE3442-RJ)  1G/10G triple-speed MAC  SerDes interface for optical transceiver connection

 PCIe

Gen3 x8 (10GE)

 Zero copy capable hardware  Other offload performance features

 TCP, IP, user datagram protocol (UDP) checksum  TCP segmentation  Adaptive interrupts  Receive side scaling (RSS)

2 83840-546-00 E

1–Product Overview Features

 Manageability

 QConvergeConsole GUI. See the QConvergeConsole GUI Installation

Guide, QConvergeConsole GUI online help and the QLogic Control Suite Command Line Interface User’s Guide for more information.

 QConvergeConsole Plug-ins for vSphere

through VMware vCenter™ Server software. For more information, see the QConvergeConsole Plug-ins for vSphere User's Guide.

 Supports the pre-execution environment (PXE) 1.0 and 2.0

specifications

 Universal management port (UMP)  System management bus (SMBus) controller  Advanced configuration and power interface (ACPI) 1.1a compliant

(multiple power modes)

 Intelligent platform management interface (IPMI) support

 Advanced network features

 Jumbo frames (up to 9,600 bytes). The OS and the link partner must

support jumbo frames.

 Virtual LANs  IEEE Std 802.3ad Teaming  Smart Load Balancing™ (SLB) teaming  Flow control (IEEE Std 802.3x)  LiveLink™ (supported in both the 32-bit and 64-bit Windows operating

systems)

 Logical link control (IEEE Std 802.2)  High-speed on-chip reduced instruction set computer (RISC) processor  Integrated 96KB frame buffer memory  Quality of service (QoS)  Serial gigabit media independent interface (SGMII)/

Gigabit media independent interface (GMII)/ Media independent interface (MII)

 256 unique MAC unicast addresses  Support for multicast addresses through the 128 bits hashing hardware

function

 Serial flash NVRAM memory  JTAG support

3 83840-546-00 E

1–Product Overview Features

 PCI power management interface (v1.1)  64-bit base address register (BAR) support  EM64T processor support  iSCSI and FCoE boot support  Virtualization

iSCSI

 Microsoft

 VMware  Linux  XenServer

 Single root I/O virtualization (SR-IOV)

The Internet engineering task force (IETF) has standardized iSCSI. SCSI is a popular protocol that enables systems to communicate with storage devices, using block-level transfer (address data stored on a storage device that is not a whole file). iSCSI maps the SCSI request/response application protocols and its standardized command set over TCP/IP networks.

As iSCSI uses TCP as its sole transport protocol, it greatly benefits from hardware acceleration of the TCP processing. However, iSCSI as a layer 5 protocol has additional mechanisms beyond the TCP layer. iSCSI processing can also be offloaded, thereby reducing CPU use even further.

The QLogic 8400/3400 Series adapters target best-system performance, maintains system flexibility to changes, and supports current and future OS convergence and integration. Therefore, the adapter's iSCSI offload architecture is unique because of the split between hardware and host processing.

4 83840-546-00 E

1–Product Overview Features

FCoE

FCoE allows Fibre Channel protocol to be transferred over Ethernet. FCoE preserves existing Fibre Channel infrastructure and capital investments. The following FCoE features are supported:

 Full stateful hardware FCoE offload  Receiver classification of FCoE and Fibre Channel initialization protocol

 Receiver CRC offload  Transmitter CRC offload  Dedicated queue set for Fibre Channel traffic  DCB provides lossless behavior with PFC  DCB allocates a share of link bandwidth to FCoE traffic with ETS

(FIP) frames. FIP is the FCoE initialization protocol used to establish and maintain connections.

Power Management

Wake on LAN (WOL) is not supported.

Adaptive Interrupt Frequency

The adapter driver intelligently adjusts host interrupt frequency based on traffic conditions to increase overall application throughput. When traffic is light, the adapter driver interrupts the host for each received packet, minimizing latency. When traffic is heavy, the adapter issues one host interrupt for multiple, back-to-back incoming packets, preserving host CPU cycles.

ASIC with Embedded RISC Processor

The core control for QLogic 8400/3400 Series adapters resides in a tightly integrated, high-performance ASIC. The ASIC includes a RISC processor that provides the flexibility to add new features to the card and a dapt to future network requirements through software downloads. In addition, the adapter drivers can exploit the built-in host offload functions on the adapter as host operating systems are enhanced to take advantage of these functions.

5 83840-546-00 E

1–Product Overview Adapter Management

Adapter Management

The following applications are available to manage 8400/3400 Series Adapters:

 QLogic Control Suite CLI  QLogic QConvergeConsole Graphical User Interface  QLogic QConvergeConsole vCenter Plug-In  QLogic FastLinQ ESXCLI VMware Plug-In

QLogic Control Suite CLI

The QCS CLI is a console application that you can run from a Windows command prompt or a Linux terminal console. Use the QCS CLI to manage QLogic 8400/3400 Series Adapters or any QLogic adapter based on 57xx/57xxx controllers on both local and remote computer systems. For information about installing and using the QCS CLI, see the QLogic Control Suite CLI User’s Guide.

QLogic QConvergeConsole Graphical User Interface

The QCC GUI is a Web-based management tool for configuring and managing QLogic Fibre Channel adapters and Intelligent Ethernet adapters. You can use the QCC GUI on Windows and Linux platforms to manage QLogic 8400/3400 Series Adapters on both local and remote computer systems. For information about installing the QCC GUI, see the QConvergeConsole GUI Installation Guide. For information about using the QCC GUI, see the online help.

QLogic QConvergeConsole vCenter Plug-In

The QCC vCenter Plug-In is a Web-based management tool that is integrated into the VMware vCenter Server for configuring and managing QLogic Fibre Channel adapters and Intelligent Ethernet adapters in a virtual environment. You can use the vCenter Plug-in VMware vSph ere clients to manage QLogic 8400/3400 Series Intelligent Ethernet Adapters. For information about installing and using the vCenter Plug-in, see the QConvergeConsole Plug-ins for vSphere User’s Guide.

QLogic FastLinQ ESXCLI VMware Plug-In

The QLogic FastLinQ ESXCLI VMware plug-in extends the capabilities of the

ESX

command line interface to manage QLogic 3400, 8400, and 45000 Series Adapters installed in VMware ESX/ESXi hosts. For information about using the ESXCLI Plug-In, see the QLogic FastLinQ ESXCLI VMware Plug-in User’s Guide.

6 83840-546-00 E

1–Product Overview Supported Operating Environments

Supported Operating Environments

The QLogic 8400/3400 Series adapters support several operating systems including Windows, Linux (RHEL Server

systems and versions, go to driverdownloads.qlogic.com adapter type, model, or operating system.

, and Citrix® XenServer. For a complete list of supported operating

, SUSE®, Ubuntu®, CentOSSM)1, VMware ESXi

Adapter Specifications

Physical Characteristics

The QLogic 8400/3400 Series Adapters are implemented as low-profile PCIe cards. The adapters ship with a full-height bracket for use in a standard PCIe slot or an optional spare low-profile bracket for use in a low-profile PCIe slot. Low-profile slots are typically found in compact servers.

Standards Specifications

and search for your

 IEEE 802.3ae (10Gb Ethernet)  IEEE 802.1q (VLAN)  IEEE 802.3ad (Link Aggregation)  IEEE 802.3x (Flow Control)  IPv4 (RFC 791)  IPv6 (RFC 2460)  IEEE 802.1Qbb (Priority-based Flow Control)  IEEE 802.1Qaz (data center bridging exchange (DCBX) and enhanced

transmission selection [ETS])

 IEEE 802.3an 10GBASE-T  IEEE 802.3ab 1000BASE-T  IEEE 802.3u 100BASE-TX

Ubuntu and CentOS operating systems are supported only on 3400 Series adapters.

3400 Series Adapters only

7 83840-546-00 E

2 Installing the Hardware

 System Requirements  Safety Precautions  Preinstallation Checklist  Installation of the Network Adapter  Connecting the Network Cables

System Requirements

Before you install a QLogic 8400/3400 Series adapter, verify that your system meets the following hardware and operating system requirements:

Hardware Requirements

 IA32- or EMT64-based compute r that meets operatin g system requirements  One open PCI Express slot. Depending on the PCI Express su pport on your

adapter, the slot may be of type

 PCI Express 1.0a x1  PCI Express 1.0a x4  PCI Express Gen2 x8  PCI Express Gen3 x8

Full dual-port 10Gbps bandwidth is supported on PCI Express Gen2 x8 or faster slots.

 128MB RAM (minimum)

Operating System Requirements

For a complete list of supported operating systems and versions, go to

driverdownloads.qlogic.com

system.

and search for your adapter type, model, or operating

8 83840-546-00 E

2–Installing the Hardware

WARNING

NOTE

Safety Precautions

The adapter is being installed in a system that operates with voltages that can be lethal. Before you open the case of your system, observe the following precautions to protect yourself and to prevent damage to the system components.

 Remove any metallic objects or jewelry from your hands and wrists.  Make sure to use only insulated or nonconducting tools.  Verify that the system is powered OFF and is unplugged before you

touch internal components.

 Install or remove adapters in a static-free environment. The use of a

properly grounded wrist strap or other personal antistatic devices and an antistatic mat is strongly recommended.

Preinstallation Checklist

1. Verify that your system meets the hardware and software requirements

listed under System Requirements.

2. Verify that your system is using the latest BIOS.

If you acquired the adapter software on a disk or from the QLogic Web Site driverdownloads.qlogic.com files.

1. If your system is active, shut it down.

2. When system shutdown is complete, turn off the power and unplug the

power cord.

3. Remove the adapter from its shipping package and place it on an antistatic

surface.

4. Check the adapter for visible signs of damage, particularly on the edge

connector. Never attempt to install a damaged adapter.

), verify the path to the adapter driver

9 83840-546-00 E

2–Installing the Hardware

CAUTION

Installation of the Network Adapter

The following instructions apply to installing the QLogic 8400/3400 Series adapters in most systems. Refer to the manuals that were supplied with your system for details about performing these tasks on your particular system.

1. Review “Safety Precautions” on page 9 and “Preinstallation Checklist” on

page 9. Before you install the adapter , ensure that the system power is OFF,

the power cord is unplugged from the power outlet, and that you are following proper electrical grounding procedures.

2. Open the system case and select the slot based on the adapter, which may

be of type PCIe 1.0a x1, PCIe 1.0a x4, PCIe Gen2 x8, PCIe Gen3 x8, or other appropriate slot. A lesser-width adapter can be seated into a greater-width slot (x8 in a x16), but a greater-width adapter cannot be seated into a lesser-width slot (x8 in a x4). If you do not know how to identify a PCI Express slot, refer to your system documentation.

3. Remove the blank cover-plate from the slot that you selected.

4. Align the adapter connector edge with the PCI Express connector slot in the

system.

5. Applying even pressure at both corners of the card, push the adapter card

into the slot until it is firmly seated. When the adapter is properly seate d, the adapter port connectors are aligned with the slot opening, and the adapter faceplate is flush against the system chassis.

Do not use excessive force when seating the card, as this may damage the system or the adapter. If you have difficulty seating the adapter, remove it, realign it, and try again.

6. Secure the adapter with the adapter clip or screw.

7. Close the system case and disconnect any personal antistatic devices.

10 83840-546-00 E

2–Installing the Hardware

NOTE

Connecting the Network Cables

The QLogic 8400/3400 Series adapters have either an RJ-45 connector used for attaching the system to an Ethernet copper-wire segment, or a fiber optic connector for attaching the system to an Ethernet fiber optic segment.

The QLogic 3442-RJ adapter supports Automatic MDI Crossover (MDIX), which eliminates the need for crossover cables when connecting machines back-to-back. A straight-through Category 5/5e/6/6A/7 cable allows the machines to communicate when connected directly together.

1. Select an appropriate cable. Table 2-1 lists the copper cable requirements

for connecting to 100/1000BASE-T and 10GBASE-T ports.

Table 2-1. 100/1000BASE-T and 10GBASE-T Cable Specifications

Port Type Connector Media

100/1000BASE-T 10GBASE-T RJ-45

1000BASE-T signaling requires four twisted pairs of Category 5 balanced cabling, as specified in

ISO/IEC 11801:2002 and ANSI/EIA/TIA-568-B.

Category 5 is the minimum requirement. Categories 5e, 6, 6a, and 7 are fully supported.

10GBASE-T signaling requires four twisted pairs of Category 6 or Category 6A (augmented

Category 6) balanced cabling, as specified in ISO/IEC 11801:2002 and ANSI/TIA/EIA-568-B.

RJ-45

Category 5 b UTP Category 6

Category 6A/7

UTP

Maximum

Distance

100m (328 ft) 40m (131 ft)

100m (328 ft)

2. Connect one end of the cable to the RJ-45 connector on the adapter.

3. Connect the other end of the cable to an RJ-45 Ethernet network port. The 8400/3400 Series Adapters also support direct attach copper cables.

11 83840-546-00 E

3 Multi-boot Agent (MBA)

Driver Software

 Overview  Setting Up MBA in a Client Environment  Setting Up MBA in a Server Environment

Overview

QLogic 8400/3400 Series adapters support Preboot Execution Environment (PXE), Remote Program Load (RPL), iSCSI, and Bootstrap Protocol (BootP). Multi-Boot Agent (MBA) is a software module that allows your network computer to boot with the images provided by remote servers across the network. The QLogic MBA driver complies with the PXE 2.1 specification and is released with split binary images. This provides flexibility to users in different environments where the motherboard may or may not have built-in base code.

The MBA module operates in a client/server environment. A network consists of one or more boot servers that provide boot images to multiple computers through the network. The QLogic implementation of the MBA module has been tested successfully in the following environments:

 Linux Red Hat

and use network resources (NFS mount, and so forth) and to perform Linux installations. In the case of a remote boot, the Linux universal driver binds seamlessly with the QLogic Universal Network Driver Interface (UNDI) and provides a network interface in the Linux remotely-booted client environment.

 Intel

 MS-DOS UNDI. The MS-DOS UNDI seamlessly binds with the QLogic UNDI

APITEST. The QLogic PXE driver passes all API compliance test

suites.

to provide a network adapter driver interface specification (NDIS2) interface to the upper layer protocol stack. This allows computers to connect to network resources in an MS-DOS environment.

PXE Server. QLogic PXE clients are able to remotely boot

12 83840-546-00 E

3–Multi-boot Agent (MBA) Driver Software

NOTE

Setting Up MBA in a Client Environment

Setting up MBA in a client environment involves the following steps:

1. Enabling the MBA driver.

2. Configuring the MBA driver.

3. Setting up the BIOS for the boot order.

Enabling the MBA Driver

To enable or disable the MBA driver:

1. Insert an MS-DOS 6.22 or a Real Mode Kernel bootable disk containing the

uxdiag.exe file (for 10/100/1000Mbps network adapters) or uediag.exe (for 10Gbps network adapters) in the removable disk drive and power up your system.

The uxdiag.exe (or uediag.exe) file is on the installation CD or in the DOS Utilities package available from driverdownloads.qlogic.com/

2. Type:

uxdiag -mba [ 0-disable | 1-enable ] -c devnum (or uediag -mba [ 0-disable | 1-enable ] -c devnum)

where devnum is the specific device(s) number (0,1,2, …) to be programmed.

Configuring the MBA Driver

This section describes the configuration of the MBA driver on add-in NIC models of the QLogic network adapter using the Comprehensive Configuration Management (CCM) utility. To configure the MBA driver on LOM models of the QLogic network adapter, check your system documentation. Both the MBA driver and the CCM utility reside on the adapter Flash memory.

13 83840-546-00 E

3–Multi-boot Agent (MBA) Driver Software Setting Up MBA in a Client Environment

You can use the CCM utility to configure the MBA driver one adapter at a time as described in this section. To simultaneously configure the MBA driver for multiple adapters, use the MS-DOS-based user diagnostics application described in

“Performing Diagnostics” on page 240. For more information about the CCM

utility, see the Comprehensive Configuration Management User’s Guide.

1. Restart your system.

2. Press CTRL+S within four seconds after you are prompted to do so. A list of

adapters displays. a. Select the adapter to configure and press ENTER. The Main Menu

displays.

b. Select MBA Configuration to display the MBA Configuration menu

(Figure 3-1).

Figure 3-1. MBA Configuration Menu

14 83840-546-00 E

3–Multi-boot Agent (MBA) Driver Software

NOTE

Setting Up MBA in a Server Environment

3. Use the up arrow and down arrow keys to move to the Boot Protocol menu

item. Then use the right arrow or left arrow key to select the boot protocol of choice if other boot protocols besides PXE are available. If available, other boot protocols include Remote Program Load (RPL), iSCSI, and BOOTP.

 For iSCSI boot-capable LOMs, the boot protocol is set through the

BIOS. See your system documentation for more information.

 If you have multiple adapters in your system and you are unsure

which adapter you are configuring, press CTRL+F6, which causes the port LEDs on the adapter to start blinking.

4. Use the UP ARROW, DOWN ARROW, LEFT ARROW, and RIGHT ARROW

keys to move to and change the values for other menu items, as desired.

5. Press F4 to save your settings.

6. Press ESC when you are finished.

Setting Up the BIOS

To boot from the network with the MBA, make the MBA enabled adapter the first bootable device under the BIOS. This procedure depends on the system BIOS implementation. Refer to the user manual for the system for instructions.

Setting Up MBA in a Server Environment

Red Hat Linux PXE Server

The Red Hat Enterprise Linux distribution has PXE Server support. It allows users to remotely perform a complete Linux installation over the network. The distribution comes with the boot images boot kernel (vmlinuz) and initial ram disk (initrd), which are located on the Red Hat disk#1:

/images/pxeboot/vmlinuz /images/pxeboot/initrd.img

Refer to the Red Hat documentation for instructions on how to install PXE Server on Linux.

The Initrd.img file distributed with Red Hat Enterprise Linux, however, does not have a Linux network driver for the QLogic 8400/3400 Series adapters. This version requires a driver disk for drivers that are not part of the standard distribution. You can create a driver disk for the QLogic 8400/3400 Series adapters from the image distributed with the installation CD. Refer to the Linux Readme.txt file for more information.

15 83840-546-00 E

3–Multi-boot Agent (MBA) Driver Software Setting Up MBA in a Server Environment

MS-DOS UNDI/Intel APITEST

To boot in MS-DOS mode and connect to a network for the MS-DOS environment, download the Intel PXE PDK from the Intel website. This PXE PDK comes with a TFTP/ProxyDHCP/Boot server. The PXE PDK can be downloaded from Intel at

http://downloadcenter.intel.com/SearchResult.aspx?lang=eng&ProductFamily=N etwork+Connectivity&ProductLine=Boot+Agent+Software&ProductProduct=Intel %c2%ae+Boot+Agent.

16 83840-546-00 E

4 Windows Driver Software

NOTE

 Installing the Driver Software  Removing the Device Drivers  Installing QLogic Management Applications  Viewing or Changing the Adapter Properties  Setting Power Management Options

QLogic now supports QConvergeConsole GUI as the only GUI management tool across all QLogic adapters. The QLogic Control Suite (QCS) GUI is no longer supported for the 8400/3400 Series Adapters and adapters based on 57xx/57xxx controllers, and has been replaced by the QCC GUI management tool. The QCC GUI provides single-pane-of-glass GUI management for all QLogic adapters.

driverdownloads.qlogic.com

Installing the Driver Software

These instructions assume that your QLogic 8400/3400 Series adapter was not factory installed. If your controller was installed at the factory, the driver software has been installed for you.

When Windows first starts after a hardware device has been installed (such as a QLogic 8400/3400 Series adapter), or after the existing device driver has been removed, the operating system automatically detects the hardware and prompts you to install the driver software for that device.

17 83840-546-00 E

4–Windows Driver Software

NOTE

Installing the Driver Software

Both a graphical interactive installation mode (see “Using the Installer” on

page 18) and a command-line silent mode for unattended installation (see “Using Silent Installation” on page 19) are available.

 Before installing the driver software, verify that the Windows operating

system has been upgraded to the latest version with the latest service pack applied.

 A network device driver must be physically installed before the QLogic

8400/3400 Series adapter can be used with your Windows operating system. Drivers are located on the installation CD.

Using the Installer

If supported and if you will use the QLogic iSCSI Crash Dump utility , it is important to follow the installation sequence:

 Run the installer  Install the Microsoft iSCSI Software Initiator along with the patch

To install the QLogic 8400/3400 Series drivers

1. When the Found New Hardware Wizard appears, click Cancel.

2. Insert the installation CD into the CD or DVD drive.

3. On the installation CD, open the folder for your operating system, open the

DrvInst folder, and then double-click Setup.exe to open the InstallShield Wizard.

4. Click Next to continue.

5. After you review the license agreement, click I accept the terms in the

license agreement and then click Next to continue.

6. Click Install.

7. Click Finish to close the wizard.

8. The installer will determine if a system restart is necessary. Follow the

on-screen instructions.

To install the Microsoft iSCSI Software Initiator for iSCSI Crash Dump

If supported and if you will use the QLogic iSCSI Crash Dump utility , it is important to follow the installation sequence:

 Run the installer

18 83840-546-00 E

4–Windows Driver Software

NOTE

Installing the Driver Software

 Install Microsoft iSCSI Software Initiator along with the patch (MS

KB939875)

If performing an upgrade of the device drivers from the installer, re-enable iSCSI Crash Dump from the Advanced section of the QCC Configuration tab.

Perform this procedure after running the installer to install the device drivers.

1. Install Microsoft iSCSI Software Initiator (version 2.06 or later) if not included

in your OS. To determine when you need to install the Microsoft iSCSI Software Initiator, see Table 4-1. To download the iSCSI Software Initiator from Microsoft, go to

http://www.microsoft.com/downloads/en/details.aspx?familyid=12cb3c1a-15 d6-4585-b385-befd1319f825&displaylang=en.

2. Install Microsoft patch for iSCSI crash dump file generation (Microsoft

KB939875) from http://support.microsoft.com/kb/939875. you need to install the Microsoft patch, see Table 4-1.

To determine if

Table 4-1. Windows Operating Systems and iSCSI Crash Dump

Operating System

Windows Server 2008 Yes (included in OS) No Windows Server 2008 R2 Yes (included in OS) No

Windows Server 2008 No No Windows Server 2008 R2 No No

Using Silent Installation

 All commands are case sensitive.  For detailed instructions and information about unattended installs, re fer

to the silent.txt file in the folder.

MS iSCSI Software

Initiator Required

NDIS

OIS

Microsoft Patch (MS

KB939875) Required

19 83840-546-00 E

4–Windows Driver Software

NOTE

Manually Extracting the Device Drivers

To perform a silent install from within the installer source folder

Type the following:

setup /s /v/qn

To perform a silent upgrade from within the installer source folder

Type the following:

setup /s /v/qn

To perform a silent reinstall of the same installer

Type the following:

setup /s /v"/qn REINSTALL=ALL"

The REINSTALL switch should only be used if the same installer is already installed on the system. If upgrading an earlier version of the installer, use setup /s /v/qn as listed above.

To perform a silent install to force a downgrade (default is NO)

setup /s /v” /qn DOWNGRADE=Y”

Manually Extracting the Device Drivers

To manually extract the Windows device drivers, type the following command:

setup /a

This will run the setup utility, extract the drivers, and place them in the designated location.

20 83840-546-00 E

4–Windows Driver Software

NOTE

Removing the Device Drivers

Uninstall the QLogic 8400/3400 Series device drivers from your system only through the InstallShield wizard. Uninstalling the device drivers with Device Manager or any other means may not provide a clean uninstall and may cause the system to become unstable.

Windows Server 2008 and Windows Server 2008 R2 provide the Device Driver Rollback feature to replace a device driver with one that was previously installed. However, the complex software architecture of the 8400/3400 Series device may present problems if the rollback feature is used on one of the individual components. Therefore, we recommend that changes to driver versions be made only through the use of a driver installer .

To remove the device drivers, in Control Panel, double-click Add or Remove

Programs.

Installing QLogic Management Applications

1. Execute the setup file (setup.exe) to open the QLogic Management

Programs installation wizard.

2. Accept the terms of the license agreement, and then click Next.

3. In the Custom Setup dialog box, review the components to be installed,

make any necessary changes, and then click Next.

4. In the Ready to Install the Program dialog box, click Install to proceed with

the installation.

Viewing or Changing the Adapter Properties

To view or change the properties of the QLogic network adapter

1. Open the QCC GUI.

2. Click the Advanced section of the Configurations tab.

21 83840-546-00 E

4–Windows Driver Software

NOTE

Setting Power Management Options

You can set power management options to allow the operating system to turn off the controller to save power. If the device is busy doing something (servicing a call, for example) however, the operating system will not shut down the device. The operating system attempts to shut down every possible device only when the computer attempts to go into hibernation. To have the controller stay on at all times, do not click the Allow the computer to turn off the device to save power check box (Figure 4-1).

Figure 4-1. Power Management

 The Power Management tab is available only for servers that support

power management.

 If you select Only allow management stations to bring the computer

out of standby, the computer can be brought out of standby only by

Magic Packet.

22 83840-546-00 E

4–Windows Driver Software

CAUTION

Setting Power Management Options

Do not select Allow the computer to turn off the device to save power for any adapter that is a member of a team.

23 83840-546-00 E

5 Linux Driver Software

 Introduction  Limitations  Packaging  Installing Linux Driver Software  Unloading/Removing the Linux Driver  Patching PCI Files (Optional)  Network Installations  Setting Values for Optional Properties  Driver Defaults  Driver Messages  Teaming with Channel Bonding  Statistics

24 83840-546-00 E

5–Linux Driver Software Introduction

Introduction

This section discusses the Linux drivers for the QLogic 8400/3400 Series ne twork adapters. Table 5-1 lists the 8400/3400 Series Linux drivers. For information about iSCSI offload in Linux server, see “iSCSI Offload in Linux Server” on

page 110.

Linux Driver Description

bnx2x Linux driver for the 8400/3400 Series 10Gb network adapt-

Table 5-1. QLogic 8400/3400 Series Linux Drivers

ers. This driver directly controls the hardware and is responsible for sending and receiving Ethernet packe t s on behalf o f the Linux host networking stack. This driver also receives and processes device interrupts, both on behalf of itself (for L2 networking) and on behalf of the bnx2fc (FCoE) and cnic drivers.

cnic The cnic driver provides the interface between QLogic’s

bnx2i Linux iSCSI HBA driver to enable iSCSI offload on the

bnx2fc Linux FCoE kernel mode driver used to provide a translation

Limitations

 bnx2x Driver  bnx2i Driver  bnx2fc Driver

upper layer protocol (storage) drivers and QLogic’s 8400/3400 Series 10Gb network adapters. The CNIC module works with the bnx2 and bnx2x network drives in the downstream and the bnx2fc (FCoE) and bnx2i (iSCSI) drivers in the upstream.

8400/3400 Series 10Gb network adapters.

layer between the Linux SCSI stack and the QLogic FCoE firmware/hardware. In addition, the driver interfaces with the networking layer to transmit and receive encap sulated FCoE frames on behalf of open-fcoe’s libfc/libfcoe for FIP/device discovery.

25 83840-546-00 E

5–Linux Driver Software Packaging

bnx2x Driver

The current version of the driver has been tested on 2.6.x kernels starting from

2.6.9. The driver may not compile on kernels older than 2.6.9. Testing is concentrated on i386 and x86_64 architectures. Only limited testing has been done on some other architectures. Minor changes to some source files and Makefile may be needed on some kernels.

bnx2i Driver

The current version of the driver has been tested on 2.6.x kernels, starting from

2.6.18 kernel. The driver may not compile on older kernels. Testing is concentrated on i386 and x86_64 architectures, RHEL 5, RHEL 6, RHEL 7, and SUSE 11 SP1 and later distributions.

bnx2fc Driver

The current version of the driver has been tested on 2.6.x kernels, starting from

2.6.32 kernel, which is included in RHEL 6.1 distribution. This driver may not compile on older kernels. Testing was limited to i386 and x86_64 architectures,

RHEL 6.1, RHEL 7.0, and SLES

11 SP1 and later distributions.

Packaging

The Linux drivers are released in the following packaging formats:

 DKMS Packages

 KMP Packages

 netxtreme2-version.dkms.noarch.rpm  netxtreme2-version.dkms.src.rpm

 SLES

 netxtreme2-kmp-[kernel]-version.i586.rpm

 netxtreme2-kmp-[kernel]-version.x86_64.rpm

 Red Hat

 kmod-kmp-netxtreme2-{kernel]-version.i686.rpm

 kmod-kmp-netxtreme2-{kernel]-version.x86_64.rpm

The QCS CLI management utility is also distributed as an RPM package (QCS-{version}.{arch}.rpm). For information about installing the Linux QCS CLI, see the QLogic Control Suite CLI User’s Guide.

26 83840-546-00 E

5–Linux Driver Software

NOTE

Installing Linux Driver Software

 Source Packages

Identical source files to build the driver are included in both RPM and TAR source packages. The supplemental tar file contains additional utilities such as patches and driver diskette images for network installation.

The following is a list of included files:  netxtreme2-version.src.rpm: RPM package with 8400/3400

Series bnx2/bnx2x/cnic/bnx2fc/bnx2ilibfc/libfcoe driver source.

 netxtreme2-version.tar.gz: tar zipped p ackage with 8400/3400

Series bnx2/bnx2x/cnic/bnx2fc/bnx2i/libfc/libfcoe driver source.

 iscsiuio-version.tar.gz: iSCSI user space management tool

binary.

 open-fcoe-*.qlgc.<subvert>.<arch>.rpm: open-fcoe

userspace management tool binary RPM for SLES11 SP2 and legacy versions.

 fcoe-utils-*.qlgc.<subver>.<arch>.rpm: open-fcoe

userspace management tool binary RPM for RHEL 6.4 and legacy versions.

The Linux driver has a dependency on open-fcoe userspace management tools as the front-end to control FCoE interfaces. The package name of the open-fcoe tool is fcoe-utils for RHEL 6.4 and open-fcoe for SLES11 SP2 and legacy versions.

Installing Linux Driver Software

 Installing the Source RPM Package  Building the Driver from the Source TAR File

If a bnx2x, bnx2i, or bnx2fc driver is loaded and the Linux kernel is updated, the driver module must be recompiled if the driver module was installed using the source RPM or the TAR package.

27 83840-546-00 E

5–Linux Driver Software Installing Linux Driver Software

Installing the Source RPM Package

The following are guidelines for installing the driver source RPM Package. Prerequisites:

 Linux kernel source  C compiler

Procedure:

1. Install the source RPM package:

rpm -ivh netxtreme2-<version>.src.rpm

2. Change the directory to the RPM path and build the binary RPM for your

kernel: For RHEL:

cd ~/rpmbuild rpmbuild -bb SPECS/netxtreme2.spec

For SLES:

cd /usr/src/packages rpmbuild -bb SPECS/netxtreme2.spec

3. Install the newly compiled RPM:

rpm -ivh RPMS/<arch>/netxtreme2-<version>.<arch>.rpm

Note that the --force option may be needed on some Linux distributions if conflicts are reported.

4. For FCoE offload, install the open-fcoe utility.

For RHEL 6.4 and legacy versions, either of the following:

yum install fcoe-utils-<version>.rhel.64.qlgc.<subver>.<arch>.rpm

-or-

rpm -ivh fcoe-utils-<version>.rhel.64.qlgc.<subver>.<arch>.rpm

For SLES11 SP2:

rpm -ivh open-fcoe-<version>.sles.sp2.qlgc.<subver>.<arch>.rpm

28 83840-546-00 E

5–Linux Driver Software

NOTE

Installing Linux Driver Software

For RHEL 6.4 and SLES11 SP2 and legacy versions, the version of fcoe-utils/open-fcoe included in your distribution is sufficient and no out of box upgrades are provided.

Where available, installation with yum will automatically resolve dependencies. Otherwise, required dependencies can be located on your O/S installation media.

5. For SLES, turn on the fcoe and lldpad services for FCoE offload, and just

lldpad for iSCSI-offload-TLV. For SLES11 SP1:

chkconfig lldpad on chkconfig fcoe on

For SLES11 SP2:

chkconfig boot.lldpad on chkconfig boot.fcoe on

6. Inbox drivers are included with all of the supported operating systems. The

simplest means to ensure the newly installed drivers a re loaded is to reboot.

7. For FCoE offload, after rebooting, create configuration files for all FCoE ethX

interfaces:

cd /etc/fcoe cp cfg-ethx cfg-<ethX FCoE interface name>

Note that your distribution might have a different naming scheme for Ethernet devices (pXpX or emX instead of ethX).

8. For FCoE offload or iSCSI-offload-TLV, modify /etc/fcoe/cfg-<interface> by

setting DCB_REQUIRED=yes to DCB_REQUIRED=no.

9. Turn on all ethX interfaces.

ifconfig <ethX> up

10. For SLES, use YaST to configure your Ethernet interfaces to automatically

start at boot by setting a static IP address or enabling DHCP on the interface.

29 83840-546-00 E

5–Linux Driver Software Installing Linux Driver Software

11. For FCoE offload and iSCSI-offload-TLV, disable lldpad on QLogic

converged network adapter interfaces. This is required because QLogic uses an offloaded DCBX client.

lldptool set-lldp –i <ethX> adminStatus=disasbled

12. For FCoE offload and iSCSI-offload-TLV, make sure

/var/lib/lldpad/lldpad.conf is created and each <ethX> block does not specify “adminStatus” or if specified, it is set to 0 (“adminStatus=0”) as below.

lldp : { eth5 : { tlvid00000001 : { info = "04BC305B017B73"; }; tlvid00000002 : { info = "03BC305B017B73"; }; };

13. For FCoE offload and iSCSI-offload-TLV, restart lldpad service to apply new

settings For SLES11 SP1, RHEL 6.4 and legacy versions:

service lldpad restart

For SLES11 SP2:

rclldpad restart

For SLES12:

systemctl restart lldpad

30 83840-546-00 E

5–Linux Driver Software

NOTE

Installing Linux Driver Software

14. For FCOE offload, restart fcoe service to apply new settings

For SLES11 SP1, RHEL 6.4, and legacy versions:

service fcoe restart

For SLES11 SP2:

rcfcoe restart

For SLES12:

systemctl restart fcoe

Installing the KMP Package

The examples in this procedure refer to the bnx2x driver, but also apply to the bxn2fc and bnx2i drivers.

1. Install the KMP package:

rpm -ivh <file> rmmod bnx2x

2. Load the driver:

modprobe bnx2x

Building the Driver from the Source TAR File

The examples used in this procedure refer to the bnx2x driver , but also apply to the bnx2i and bnx2fc drivers.

1. Create a directory and extract the TAR files to the directory:

tar xvzf netxtreme2-<version>.tar.gz

2. Build the driver bnx2x.ko (or bnx2x.o) as a loadable module for the running

kernel:

cd netxtreme2-<version> make

31 83840-546-00 E

5–Linux Driver Software

NOTE

Load and Run Necessary iSCSI Software Components

3. Test the driver by loading it (first unload the existing driver, if necessary):

rmmod bnx2x (or bnx2fc or bnx2i) insmod bnx2x/src/bnx2x.ko (or bnx2fc/src/bnx2fc.ko, or

bnx2i/src/bnx2i.ko)

4. For iSCSI offload and FCoE offload, load the cnic driver (if applicable):

insmod cnic.ko

5. Install the driver and man page:

make install

See the RPM instructions above for the location of the installed driver.

6. Install the user daemon (qlgc_iscsiuio). Refer to “Load and Run Necessary iSCSI Software Components” on page 32 for

instructions on loading the software components required to use the QLog ic iSCSI offload feature.

To configure the network protocol and address after building the driver , refer to the manuals supplied with your operating system.

Load and Run Necessary iSCSI Software Components

The QLogic iSCSI Offload software suite consists of three kernel modules and a user daemon. Required software components can be loaded either manually or through system services.

1. Unload the existing driver, if necessary:

Manual:

rmmod bnx2i

modprobe -r bnx2i

2. Load the iSCSI driver:

Manual: insmod bnx2i.ko or modprobe bnx2i

32 83840-546-00 E

5–Linux Driver Software

NOTE

Unloading/Removing the Linux Driver

 Unloading/Removing the Driver from an RPM Installation  Removing the Driver from a TAR Installation

Unloading/Removing the Driver from an RPM Installation

 The examples used in this procedure refer to the bnx2x driver, but also

apply to the bnx2fc and bnx2i drivers.

 On 2.6 kernels, it is not necessary to bring down the eth# interfaces

before unloading the driver module.

 If the cnic driver is loaded, unload the cnic driver before unloading the

bnx2x driver.

 Prior to unloading the bnx2i driver, disconnect all active iSCSI sessions

to targets.

To unload the driver, use ifconfig to bring down all eth# interfaces opened by the driver, and then type the following:

rmmod bnx2x

The above command will also remove the cnic module.

If the driver was installed using RPM, do the following to remove it:

rpm -e netxtreme2

Removing the Driver from a TAR Installation

The examples used in this procedure refer to the bnx2x driver , but also apply to the bnx2fc and bnx2i drivers.

If the driver was installed using make install from the tar file, the bnx2x.ko driver file has to be manually deleted from the operating system. See “Installing the

Source RPM Package” on page 28 for the location of the installed driver.

33 83840-546-00 E

5–Linux Driver Software

NOTE

Patching PCI Files (Optional)

Uninstalling the QCC GUI

For information about removing the QCC GUI, see QConvergeConsole GUI Installation Guide.

Patching PCI Files (Optional)

The examples used in this procedure refer to the bnx2x driver , but also apply to the bnx2fc and bnx2i drivers.

For hardware detection utilities, such as Red Hat kudzu, to properly identify bnx2x supported devices, a number of files containing PCI vendor and device information may need to be updated.

Apply the updates by running the scripts provided in the supplemental tar file. For example, on Red Hat Enterprise Linux, apply the updates by doing the following:

./patch_pcitbl.sh /usr/share/hwdata/pcitable pci.updates /usr/share/hwdata/pcitable.new bnx2

./patch_pciids.sh /usr/share/hwdata/pci.ids pci.updates /usr/share/hwdata/pci.ids.new

Next, the old files can be backed up and the new files can be renamed for use.

cp /usr/share/hwdata/pci.ids /usr/share/hwdata/old.pci.ids cp /usr/share/hwdata/pci.ids.new /usr/share/hwdata/pci.ids cp /usr/share/hwdata/pcitable /usr/share/hwdata/old.pcitable cp /usr/share/hwdata/pcitable.new /usr/share/hwdata/pcitable

Network Installations

For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the bnx2x driver may be needed. The driver disk images for the most recent Red Hat and SuSE versions are included. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from the Red Hat website,

http://www.redhat.com

34 83840-546-00 E

5–Linux Driver Software Setting Values for Optional Properties

Setting Values for Optional Properties

Optional properties exist for the different drivers:

 bnx2x Driver  bnx2i Driver  bnx2fc Driver

bnx2x Driver

disable_tpa

The disable_tpa parameter can be supplied as a command line argument to disable the Transparent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets. Use disable_tpa to disable the advanced TPA feature.

Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all 8400/3400 Series network adapters in the system. The parameter can also be set in modprobe.conf. See the man page for more information.

int_mode

insmod bnx2x.ko disable_tpa=1

modprobe bnx2x disable_tpa=1

The int_mode parameter is used to force using an interrupt mode. Set the int_mode parameter to 1 to force using the legacy INTx mode on all

8400/3400 Series adapters in the system.

insmod bnx2x.ko int_mode=1

modprobe bnx2x int_mode=1

Set the int_mode parameter to 2 to force using MSI mode on all 8400/3400 Series adapters in the system.

insmod bnx2x.ko int_mode=2

modprobe bnx2x int_mode=2

Set the int_mode parameter to 3 to force using MSI-X mode on all 8400/3400 Series adapters in the system.

35 83840-546-00 E

5–Linux Driver Software Setting Values for Optional Properties

dropless_fc

The dropless_fc parameter can be used to enable a comp lementary flow control mechanism on 8400/3400 Series adapters. The default flow control mechanism is to send pause frames when the on-chip buffer (BRB) is reaching a certain level of occupancy. This is a performance targeted flow control mechanism. On 8400/3400 Series adapters, one can enable another flow control mechanism to send pause frames, where one of the host buffers (when in RSS mode) are exhausted.

This is a zero packet drop targeted flow control mechanism. Set the dropless_fc parameter to 1 to enable the dropless flow control

mechanism feature on all 8400/3400 Series adapters in the system.

insmod bnx2x.ko dropless_fc=1

modprobe bnx2x dropless_fc=1

disable_iscsi_ooo

The disable_iscsi_ooo parameter is to disable the allocation of the iSCSI TCP Out-of-Order (OOO) reception resources, specifically for VMware for low-memory systems.

multi_mode

num_queues

The optional parameter multi_mode is for use on systems that support multi-queue networking. Multi-queue networking on the receive side depends only on MSI-X capability of the system, multi-queue networking on the transmit side is supported only on kernels starting from 2.6.27. By default, multi_mode parameter is set to 1. Thus, on kernels up to 2.6.26, the driver will allocate on the receive side one queue per-CPU and on the transmit side only one queue. On kernels starting from 2.6.27, the driver will allocate on both receive and transmit sides, one queue per-CPU. In any case, the number of allocated queues will be limited by number of queues supported by hardware.

The multi_mode optional parameter can also be used to enable SAFC (Service Aware Flow Control) by differentiating the traffic to up to 3 CoS (Class of Service) in the hardware according to the VLAN PRI value or according to the IP DSCP value (least 3 bits).

The optional parameter num_queues may be used to set the number of queues when multi_mode is set to 1 and interrupt mode is MSI-X. If interrupt mode is different than MSI-X (see int_mode), the number of queues will be set to 1, discarding the value of this parameter.

36 83840-546-00 E

5–Linux Driver Software Setting Values for Optional Properties

pri_map

The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or same CoS in the hardware. This 32-bit parameter is evaluated by the driver as an 8 value of 4 bits each. Each nibble sets the desired hardware queue number for that priority . For example, set pri_map to 0x11 1 10000 to map priority 0 to 3 to CoS 0 and map priority 4 to 7 to CoS 1.

qs_per_cos

The optional parameter qs_per_cos is used to specify how many queues will share the same CoS. This parameter is evaluated by the driver up to 3 values of 8 bits each. Each byte sets the desired number of queues for that CoS. The total number of queues is limited by the hardware limit. For example, set qs_per_cos to 0x10101 to create a total of three queues, one per CoS. In another example, set qs_per_cos to 0x404 to create a total of 8 queues, divided into 2 CoS, 4 queues in each CoS.

cos_min_rate

The optional parameter cos_min_rate is used to determine the weight of each CoS for round-robin scheduling in transmission. This parameter is evaluated by the driver as up to 3 values of 8 bits each. Each byte sets the desired weight for that CoS. The weight ranges from 0 to 100. For example, set cos_min_rate to 0x101 for fair transmission rate between 2 CoS. In another example, set the cos_min_rate to 0x30201 to give CoS the higher rate of transmission. To avoid using the fairness algorithm, omit setting cos_min_rate or set it to 0.

Set the multi_mode parameter to 2 as shown below to differentiate the traffic according to the VLAN PRI value.

insmod bnx2x.ko multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404

modprobe bnx2x multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404

Set the multi_mode parameter to 4, as shown below, to differentiate the traffic according to the IP DSCP value.

insmod bnx2x.ko multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201

modprobe bnx2x multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201

37 83840-546-00 E

5–Linux Driver Software

CAUTION

Setting Values for Optional Properties

bnx2i Driver

Optional parameters en_tcp_dack, error_mask1, and error_mask2 can be supplied as command line arguments to the insmod or modprobe command for bnx2i.

error_mask1 and error_mask2

"Config FW iSCSI Error Mask #", use to configure certain iSCSI protocol violation to be treated either as a warning or a fatal error. All fatal iSCSI protocol violations will result in session recovery (ERL 0). These are bit masks.

Defaults: All violations will be treated as errors.

Do not use error_mask if you are not sure about the consequences. These values are to be discussed with QLogic development team on a case-by-case basis. This is just a mechanism to work around iSCSI implementation issues on the target side. Without proper knowledge of iSCSI protocol details, users are advised not to experiment with these parameters.

en_tcp_dack

time_stamps

"Enable TCP Delayed ACK", enables/disables TCP delayed ACK feature on offloaded iSCSI connections.

Defaults: TCP delayed ACK is ENABLED. For example:

insmod bnx2i.ko en_tcp_dack=0

modprobe bnx2i en_tcp_dack=0

“Enable TCP TimeStamps”, enables/disables TCP time stamp feature on offloaded iSCSI connections.

Defaults: TCP time stamp option is DISABLED. For example:

insmod bnx2i.ko time_stamps=1

modprobe bnx2i time_stamps=1

38 83840-546-00 E

5–Linux Driver Software Setting Values for Optional Properties

sq_size

"Configure SQ size", used to choose send queue size for offloaded connections and SQ size determines the maximum SCSI commands that can be queued. SQ size also has a bearing on the number of connections that can be offloaded; as QP size increases, the number of connections supported will decrease. With the default values, the adapter can offload 28 connections.

Defaults: 128 Range: 32 to 128 Note that QLogic validation is limited to a power of 2; for example, 32, 64, 128.

rq_size

“Configure RQ size”, used to choose the size of asynchronous buffer queue size per offloaded connections. RQ size is not requ ired greater than 16 as it is used to place iSCSI ASYNC/NOP/REJECT messages and SCSI sense data.

Defaults: 16 Range: 16 to 32 Note that QLogic validation is limited to a power of 2; for example, 16, 32.

event_coal_div

"Event Coalescing Divide Factor", performance tuning parameter used to moderate the rate of interrupt generation by the iSCSI firmware.

Defaults: 2 Valid values: 1, 2, 4, 8

last_active_tcp_port

“Last active TCP port used”, status parameter used to indicate the last TCP port number used in the iSCSI offload connection.

Defaults: N/A Valid values: N/A Note: This is a read-only parameter.

ooo_enable

“Enable TCP out-of-order feature”, enables/disables TCP out-of-order rx handling feature on offloaded iSCSI connections.

Defaults: TCP out-of-order feature is ENABLED. For example:

insmod bnx2i.ko ooo_enable=1

modprobe bnx2i ooo_enable=1

39 83840-546-00 E

5–Linux Driver Software Driver Defaults

bnx2fc Driver

Optional parameter debug_logging can be supplied as a command line arguments to the insmod or modprobe command for bnx2fc.

debug_logging

"Bit mask to enable debug logging", enables/disables driver debug logging. Defaults: None. For example:

insmod bnx2fc.ko debug_logging=0xff

modprobe bnx2fc debug_logging=0xff

IO level debugging = 0x1 Session level debugging = 0x2 HBA level debugging = 0x4 ELS debugging = 0x8 Misc debugging = 0x10 Max debugging = 0xff

Driver Defaults

 bnx2 Driver  bnx2x Driver

bnx2 Driver

Speed: Autonegotiation with all speeds advertised Flow Control: Autonegotiation with RX and TX advertised MTU: 1500 (range is 46–9000) RX Ring Size: 255 (range is 0–4080) RX Jumbo Ring Size: 0 (range 0–16320) adjusted by the driver based on MTU

and RX Ring Size TX Ring Size: 255 (range is (MAX_SKB_FRAGS+1)–255). MAX_SKB_FRAGS

varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce RX Microseconds: 18 (range is 0–1023) Coalesce RX Microseconds IRQ: 18 (range is 0–1023) Coalesce RX Frames: 6 (range is 0–255)

40 83840-546-00 E

5–Linux Driver Software Driver Defaults

Coalesce RX Frames IRQ: 6 (range is 0–255) Coalesce TX Microseconds: 80 (range is 0–1023) Coalesce TX Microseconds IRQ: 80 (range is 0–1023) Coalesce TX Frames: 20 (range is 0–255) Coalesce TX Frames IRQ: 20 (range is 0–255) Coalesce Statistics Microseconds: 999936 (approximately 1 second) (range is

0–16776960 in increments of 256)

MSI: Enabled (if supported by the 2.6 kernel and the interrupt test passes) TSO: Enabled (on 2.6 kernels)

bnx2x Driver

Speed: Autonegotiation with all speeds advertised Flow control: Autonegotiation with RX and TX advertised MTU: 1500 (range is 46–9600) RX Ring Size: 4078 (range is 0–4078) TX Ring Size: 4078 (range is (MAX_SKB_FRAGS+4)–4078). MAX_SKB_FRAGS

varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce RX Microseconds: 25 (range is 0–3000) Coalesce TX Microseconds: 50 (range is 0–12288) Coalesce Statistics Microseconds: 999936 (approximately 1 second) (range is

0–16776960 in increments of 256)

MSI-X: Enabled (if supported by the 2.6 kernel and the interrupt test passes) TSO: Enabled

41 83840-546-00 E

5–Linux Driver Software Driver Messages

Driver Messages

The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -n <level> to control the level at which messages appear on the console. Most systems are set to level 6 by default. To see all messages, set the level higher.

 bnx2x Driver  bnx2i Driver  bnx2fc Driver

bnx2x Driver

Driver Sign On

QLogic 8400/3400 Series 10 Gigabit Ethernet Driver bnx2x v1.6.3c (July 23, 20xx)

CNIC Driver Sign On (bnx2 only)

QLogic 8400/3400 Series cnic v1.1.19 (Sep 25, 20xx)

NIC Detected

eth#: QLogic 8400/3400 Series xGb (B1) PCI-E x8 found at mem f6000000, IRQ 16, node addr 0010180476ae

cnic: Added CNIC device: eth0

Link Up and Speed Indication

bnx2x: eth# NIC Link is Up, 10000 Mbps full duplex

Link Down Indication

bnx2x: eth# NIC Link is Down

MSI-X Enabled Successfully

bnx2x: eth0: using MSI-X

bnx2i Driver

BNX2I Driver Signon

QLogic 8400/3400 Series iSCSI Driver bnx2i v2.1.1D (May 12, 20xx)

42 83840-546-00 E

5–Linux Driver Software

NOTE

Driver Messages

Network Port to iSCSI Transport Name Binding

bnx2i: netif=eth2, iscsi=bcm570x-050000 bnx2i: netif=eth1, iscsi=bcm570x-030c00

Driver Completes handshake with iSCSI Offload-enabled CNIC Device

bnx2i [05:00.00]: ISCSI_INIT passed

This message is displayed only when the user attempts to make an iSCSI connection.

Driver Detects iSCSI Offload Is Not Enabled on the CNIC Device

bnx2i: iSCSI not supported, dev=eth3 bnx2i: bnx2i: LOM is not enabled to offload iSCSI connections,

dev=eth0 bnx2i: dev eth0 does not support iSCSI

Exceeds Maximum Allowed iSCSI Connection Offload Limit

bnx2i: alloc_ep: unable to allocate iscsi cid bnx2i: unable to allocate iSCSI context resources

Network Route to Target Node and Transport Name Binding Are Two Different Devices

bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba $USER_CHOSEN_HBA

where:  ROUTE_HBA is the net device on which connection was of floaded based on

route information.

 USER_CHOSEN_HBA is the adapter to which target node is bound (using

iSCSI transport name).

Target Cannot Be Reached on Any of the CNIC Devices

bnx2i: check route, cannot connect using cnic

Network Route Is Assigned to Network Interface, Which Is Down

bnx2i: check route, hba not found

SCSI-ML Initiated Host Reset (Session Recovery)

bnx2i: attempting to reset host, #3

43 83840-546-00 E

5–Linux Driver Software Driver Messages

CNIC Detects iSCSI Protocol Violation - Fatal Errors

bnx2i: iscsi_error - wrong StatSN rcvd bnx2i: iscsi_error - hdr digest err bnx2i: iscsi_error - data digest err bnx2i: iscsi_error - wrong opcode rcvd bnx2i: iscsi_error - AHS len > 0 rcvd bnx2i: iscsi_error - invalid ITT rcvd bnx2i: iscsi_error - wrong StatSN rcvd bnx2i: iscsi_error - wrong DataSN rcvd bnx2i: iscsi_error - pend R2T violation bnx2i: iscsi_error - ERL0, UO bnx2i: iscsi_error - ERL0, U1 bnx2i: iscsi_error - ERL0, U2 bnx2i: iscsi_error - ERL0, U3 bnx2i: iscsi_error - ERL0, U4 bnx2i: iscsi_error - ERL0, U5 bnx2i: iscsi_error - ERL0, U bnx2i: iscsi_error - invalid resi len bnx2i: iscsi_error - MRDSL violation bnx2i: iscsi_error - F-bit not set bnx2i: iscsi_error - invalid TTT bnx2i: iscsi_error - invalid DataSN bnx2i: iscsi_error - burst len violation bnx2i: iscsi_error - buf offset violation bnx2i: iscsi_error - invalid LUN field bnx2i: iscsi_error - invalid R2TSN field bnx2i: iscsi_error - invalid cmd len1 bnx2i: iscsi_error - invalid cmd len2 bnx2i: iscsi_error - pend r2t exceeds MaxOutstandingR2T value bnx2i: iscsi_error - TTT is rsvd bnx2i: iscsi_error - MBL violation bnx2i: iscsi_error - data seg len != 0 bnx2i: iscsi_error - reject pdu len error bnx2i: iscsi_error - async pdu len error bnx2i: iscsi_error - nopin pdu len error bnx2i: iscsi_error - pend r2t in cleanup bnx2i: iscsi_error - IP fragments rcvd bnx2i: iscsi_error - IP options error bnx2i: iscsi_error - urgent flag error

44 83840-546-00 E

5–Linux Driver Software

NOTE

Driver Messages

CNIC Detects iSCSI Protocol Violation - Non-FATAL, Warning

bnx2i: iscsi_warning - invalid TTT bnx2i: iscsi_warning - invalid DataSN bnx2i: iscsi_warning - invalid LUN field

The driver needs to be configured to consider certain violation to treat as warning and not as a critical error.

Driver Puts a Session Through Recovery

conn_err - hostno 3 conn 03fbcd00, iscsi_cid 2 cid a1800

Reject iSCSI PDU Received from the Target

bnx2i - printing rejected PDU contents [0]: 1 ffffffa1 0 0 0 0 20 0 [8]: 0 7 0 0 0 0 0 0 [10]: 0 0 40 24 0 0 ffffff80 0 [18]: 0 0 3 ffffff88 0 0 3 4b [20]: 2a 0 0 2 ffffffc8 14 0 0 [28]: 40 0 0 0 0 0 0 0

Open-iSCSI Daemon Handing Over Session to Driver

bnx2i: conn update - MBL 0x800 FBL 0x800MRDSL_I 0x800 MRDSL_T 0x2000

bnx2fc Driver

BNX2FC Driver Signon:

QLogic NetXtreme II FCoE Driver bnx2fc v0.8.7 (Mar 25, 2011

Driver Compiles Handshake with FCoE Offload Enabled CNIC Device

bnx2fc [04:00.00]: FCOE_INIT passed

Driver Fails Handshake with FCoE Offload Enabled CNIC Device

bnx2fc: init_failure due to invalid opcode bnx2fc: init_failure due to context allocation failure bnx2fc: init_failure due to NIC error bnx2fc: init_failure due to completion status error bnx2fc: init_failure due to HSI mismatch

45 83840-546-00 E

5–Linux Driver Software Driver Messages

No V a lid License to Start FCoE

bnx2fc: FCoE function not enabled <ethX> bnx2fC: FCoE not supported on <ethX>

Session Failures Due to Exceeding Maximum Allowed FCoE Offload Connection Limit or Memory Limits

bnx2fc: Failed to allocate conn id for port_id <remote port id> bnx2fc: exceeded max sessions..logoff this tgt bnx2fc: Failed to allocate resources

Session Offload Failures

bnx2fc: bnx2fc_offload_session - Offload error <rport> not FCP type. not offloading <rport> not FCP_TARGET. not offloading

Session Upload Failures

bnx2fc: ERROR!! destroy timed out bnx2fc: Disable request timed out. destroy not set to FW bnx2fc: Disable failed with completion status <status> bnx2fc: Destroy failed with completion status <status>

Unable to Issue ABTS

bnx2fc: initiate_abts: tgt not offloaded bnx2fc: initiate_abts: rport not ready bnx2fc: initiate_abts: link is not ready bnx2fc: abort failed, xid = <xid>

Unable to Recover the IO Using ABTS (Due to ABTS Timeout)

bnx2fc: Relogin to the target

Unable to Issue IO Request Due to Session Not Ready

bnx2fc: Unable to post io_req

Drop Incorrect L2 Receive Frames

bnx2fc: FPMA mismatch... drop packet bnx2fc: dropping frame with CRC error

HBA/lport Allocation Failures

bnx2fc: Unable to allocate hba bnx2fc: Unable to allocate scsi host

NPIV Port Creation

bnx2fc: Setting vport names, <WWNN>, <WWPN>

46 83840-546-00 E

5–Linux Driver Software Teaming with Channel Bonding

Teaming with Channel Bonding

With the Linux drivers, you can team adapters together using the bonding kernel module and a channel bonding interface. For more information, see the Channel Bonding information in your operating system documentation.

Statistics

Detailed statistics and configuration information can be viewed using the ethtool utility. See the ethtool man page for more information.

47 83840-546-00 E

6 VMware Driver Software

 Packaging  Download, Install, and Update Drivers  Networking Support  FCoE Support

Packaging

The VMware driver is released in the packaging formats shown in Table 6-1. For information about iSCSI offload in VMware server, see “iSCSI Offload on VMware

Server” on page 114.

Table 6-1. VMware Driver Packaging

Format Drivers Package

Compressed zip QLG-NetXtremeII-2.0-version.zip

48 83840-546-00 E

6–VMware Driver Software Download, Install, and Update Drivers

Download, Install, and Update Drivers

To download, install, or update the VMware ESXi driver for 8400/3400 Series 10 GbE network adapters, go to

http://www.vmware.com/resources/compatibility/search.php?deviceCategory=io

and do the following:

1. Type the adapter name (in quotes) in the Keyword window (such as

"QLE3442"), and then click Update and View Results (Figure 6-1).

Figure 6-1. Selecting an Adapter

Figure 6-2 shows the available QLE3442 driver versions.

Figure 6-2. QLE3442 Driver Versions

49 83840-546-00 E

6–VMware Driver Software Download, Install, and Update Drivers

2. Mouse over the QLE3442 link in the results section to show the PCI

identifiers (Figure 6-3).

3. Click the model link to show a listing of all of the driver packages

(Figure 6-4). Click the desired ESXi version, and then click the link to go to the VMware driver download web page.

Figure 6-3. PCI Identifiers

Figure 6-4. List of Driver Packages

50 83840-546-00 E

6–VMware Driver Software

NOTE

Download, Install, and Update Drivers

4. Log in to the VMware driver download page and click Download to

download the desired driver package (Figure 6-5).

Figure 6-5. Download Driver Package

5. This package is double zipped—unzip the package once before copying the

offline bundle zip file to the ESXi host.

6. To install the driver package, issue the following command:

esxcli software vib install -d <path>/<offline bundle name.zip> --maintenance-mode

esxcli software vib install --depot=/<path>/<offline bundle name.zip> --maintenance-mode

 If you do not unzip the outer zipping, the installation will report that it can

not find the drivers.

 Use double dashes (--) before the depot and maintenance-mode

parameters.

 Do not use the -v method of installing individual driver vSphere

installation bundles (VIBs).

 A reboot is required after all driver installations.

51 83840-546-00 E

6–VMware Driver Software Networking Support

Networking Support

This section describes the bnx2x VMware ESXi driver for the QLogic 8400/3400 Series PCIe 10 GbE network adapters.

Driver Parameters

Several optional parameters can be supplied as a command line argument to the vmkload_mod command. These parameters can also be set with the esxcfg-module command. See the man page for more information.

int_mode

The optional parameter int_mode is used to force using an interrupt mode other than MSI-X. By default, the driver will try to enable MSI-X if it is supported by the kernel. If MSI-X is not attainable, then the driver will try to enable MSI if it is supported by the kernel. If MSI is not attainable, then the driver will use the legacy INTx mode.

Set the int_mode parameter to 1 as shown below to force using the legacy INTx mode on all 8400/3400 Series network adapters in the system.

vmkload_mod bnx2x int_mode=1

Set the int_mode parameter to 2 as shown below to force using MSI mode on all 8400/3400 Series network adapters in the system.

vmkload_mod bnx2x int_mode=2

disable_tpa

The optional parameter disable_tpa can be used to disable the Transparent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets, but if you would like to disable this advanced feature, it can be done.

Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all 8400/3400 Series network adapters in the system.

vmkload_mod bnx2x.ko disable_tpa=1

Use ethtool to disable TPA (LRO) for a specific network adapter.

num_rx_queues

The optional parameter num_rx_queues may be used to set the number of Rx queues on kernels starting from 2.6.24 when multi_mode is set to 1 and interr upt mode is MSI-X. Number of Rx queues must be equal to or greater than the number of Tx queues (see num_tx_queues parameter). If the interrupt mode is different than MSI-X (see int_mode parameter), then then the number of Rx queues will be set to 1, discarding the value of this parameter.

52 83840-546-00 E

6–VMware Driver Software Networking Support

num_tx_queues

The optional parameter num_tx_queues may be used to set the number of Tx queues on kernels starting from 2.6.27 when multi_mode is set to 1 and interr upt mode is MSI-X. The number of Rx queues must be equal to or greater than the number of Tx queues (see num_rx_queues parameter). If the interrupt mode is different than MSI-X (see int_mode parameter), then the number of Tx queues will be set to 1, discarding the value of this parameter.

pri_map

The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or the same CoS in the hardware. This 3 2-bit parameter is evaluated by the driver as 8 values of 4 bits each. Each nibble sets the desired hardware queue number for that priority.

For example, set the pri_map parameter to 0x22221 100 to map priority 0 a nd 1 to CoS 0, map priority 2 and 3 to CoS 1, and map priority 4 to 7 to CoS 2. In another example, set the pri_map parameter to 0x11110000 to map priority 0 to 3 to CoS 0, and map priority 4 to 7 to CoS 1.

qs_per_cos

cos_min_rate

The optional parameter qs_per_cos is used to specify the number of q ueues that will share the same CoS. This parameter is evaluated by the driver up to 3 values of 8 bits each. Each byte sets the desired number of queues for that CoS. The total number of queues is limited by the hardware limit.

For example, set the qs_per_cos parameter to 0x10101 to create a total of three queues, one per CoS. In another example, set the qs_per_cos parameter to 0x404 to create a total of 8 queues, divided into only 2 CoS, 4 queues in each CoS.

The optional parameter cos_min_rate is used to determine the weight of each CoS for round-robin scheduling in transmission. This parameter is evaluated by the driver up to three values of eight bits each. Each byte sets the desired weight for that CoS. The weight ranges from 0 to 100.

For example, set the cos_min_rate parameter to 0x101 for fair transmission rate between two CoS. In another example, set the cos_min_rate parameter to 0x30201 to give the higher CoS the higher rate of transmission. To avoid using the fairness algorithm, omit setting the optional parameter cos_min_rate or set it to 0.

53 83840-546-00 E

6–VMware Driver Software Networking Support

dropless_fc

The optional parameter dropless_fc can be used to enable a complement ary flow control mechanism on QLogic network adapters. The default flow control mechanism is to send pause frames when the BRB is reaching a certain level of occupancy. This is a performance targeted flow control mechanism. On QLogic network adapters, you can enable another flow control mechanism to send pause frames if one of the host buffers (when in RSS mode) is exhausted. This is a zero packet drop targeted flow control mechanism.

Set the dropless_fc parameter to 1 as shown below to enable the dropless flow control mechanism feature on all QLogic network adapters in the system.

vmkload_mod bnx2x dropless_fc=1

RSS

The optional parameter RSS can be used to specify the number of receive side scaling queues. For VMware ESXi (5.1, 5.5, 6.0), values for RSS can be from 2 to 4; RSS=1 disables RSS queues.

max_vfs

The optional parameter max_vfs can be used to enable a specific number of virtual functions. V alues for max_vfs can be 1 to 64 , or set max_vfs=0 (default) to disable all virtual functions.

enable_vxlan_offld

The optional parameter enable_vxlan_ofld can be used to enable or disable VMware ESXi (5.5, 6.0) VxLAN task offloads with TX TSO and TX CSO. For VMware ESXi (5.5, 6.0), enable_vxlan_ofld=1 (default) enables VxLAN task offloads; enable_vxlan_offload=0 disables VxLAN task offloads.

Driver Defaults

Speed: Autonegotiation with all speeds advertised Flow Control: Autonegotiation with rx and tx advertised MTU: 1500 (range 46–9600) Rx Ring Size: 4078 (range 0–4078) Tx Ring Size: 4078 (range (MAX_SKB_FRAGS+4) - 4078). MAX_SKB_FRAGS

varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce RX Microseconds: 25 (range 0–3000) Coalesce TX Microseconds: 50 (range 0–12288) MSI-X: Enabled (if supported by 2.6 kernel) TSO: Enabled

54 83840-546-00 E

6–VMware Driver Software Networking Support

Unloading and Removing Driver

To unload the driver, type the following:

vmkload_mod -u bnx2x

Driver Messages

The following are the most common sample messages that may be logged in the file /var/log/messages. Use dmesg -n <level> to control the level at which messages will appear on the console. Most systems are set to level 6 by default. To see all messages, set the level higher.

Driver Sign On

QLogic 8400/3400 Series 10Gigabit Ethernet Driver bnx2x 0.40.15 ($DateTime: 2007/11/22 05:32:40 $)

NIC Detected

eth0: QLogic 8400/3400 Series XGb (A1) PCI-E x8 2.5GHz found at mem e8800000, IRQ 16, node addr

001018360012

MSI-X Enabled Successfully

bnx2x: eth0: using MSI-X

Link Up and Speed Indication

bnx2x: eth0 NIC Link is Up, 10000 Mbps full duplex, receive & transmit flow control ON

Link Down Indication

bnx2x: eth0 NIC Link is Down

Memory Limitation

If you see messages in the log file that look like the following, then the ESXi host is severely strained. To relieve this, disable NetQueue.

Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: Heap: 1435: Heap bnx2x already at its maximumSize. Cannot expand.

Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: Heap: 1645: Heap_Align(bnx2x, 4096/4096 bytes, 4096 align) failed. caller: 0x41800187d654

Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: vmklinux26: alloc_pages: Out of memory

Disable NetQueue by manually loading the bnx2x vmkernel module with the command.

vmkload_mod bnx2x multi_mode=0

55 83840-546-00 E

6–VMware Driver Software FCoE Support

or to persist the settings across reboots with the command

esxcfg-module -s multi_mode=0 bnx2x

Reboot the machine for the settings to take place.

MultiQueue/NetQueue

The optional parameter num_queues may be used to set the number of Rx and Tx queues when multi_mode is set to 1 and interrupt mode is MSI-X. If interrupt mode is different than MSI-X (see int_mode para meter), the number of Rx and Tx queues will be set to 1, discarding the value of this parameter.

If you would like the use of more then 1 queue, force the number of NetQueues to use with the following command:

esxcfg-module -s "multi_mode=1 num_queues=<num of queues>" bnx2x

Otherwise, allow the bnx2x driver to select the number of NetQueues to use with the following command:

esxcfg-module -s "multi_mode=1 num_queues=0" bnx2x

The optimal number is to have the number of NetQueues match the number of CPUs on the machine.

FCoE Support

This section describes the contents and procedures associated with inst allation of the VMware software package for supporting QLogic FCoE C-NICs.

Enabling FCoE

To enable FCoE hardware offload on the C-NIC

1. Determine the ports that are FCoE-capable:

# esxcli fcoe nic list

Output example:

vmnic4 User Priority: 3 Source MAC: FF:FF:FF:FF:FF:FF Active: false Priority Settable: false Source MAC Settable: false VLAN Range Settable: false

56 83840-546-00 E

6–VMware Driver Software

NOTE

FCoE Support

1. Enable the FCoE interface:

# esxcli fcoe nic discover -n vmnicX

Where X is the interface number gained from esxcli fcoe nic list.

2. Verify that the interface is working:

# esxcli fcoe adapter list

Output example:

vmhba34 Source MAC: bc:30:5b:01:82:39 FCF MAC: 00:05:73:cf:2c:ea VNPort MAC: 0e:fc:00:47:04:04 Physical NIC: vmnic7 User Priority: 3 VLAN id: 2008

The output of this command should show valid: FCF MAC, VNPort MAC, Priority, and VLAN id for the Fabric that is connected to the C-NIC.

The following command can also be used to verify that the interface is working properly:

#esxcfg-scsidevs -a

Output example:

vmhba34 bnx2fc link-up fcoe.1000<mac address>:2000<mac address> () Software FCoE

vmhba35 bnx2fc link-up fcoe.1000<mac address>:2000<mac address> () Software FCoE

The label Software FCoE is a VMware term used to describe initiators that depend on the inbox FCoE libraries and utilities. QLogic's FCoE solution is a fully state connection-based hardware offload solution designed to significantly reduce the CPU burden encumbered by a non-offload software initiator.

57 83840-546-00 E

6–VMware Driver Software FCoE Support

Installation Check

To verify the correct installation of the driver and to ensure that the host port is seen by the switch, follow the procedure below.

To verify the correct installation of the driver

1. Verify the host port shows up in the switch FLOGI database using the show

flogi database command for the case of a Cisco

-loginshow command for the case of a Brocade

2. If the Host WWPN does not appear in the FLOGI database, then provide

driver log messages for review.

Limitations

 NPIV is not currently supported with this release on ESXi, due to lack of

supporting inbox components.

 Non-offload FCoE is not supported with offload-capable QLogic devices.

Only the full hardware offload path is supported.

FCF and fcoe

FCF.

Drivers

Table 6-2 lists the 8400/3400 Series FCoE drivers.

Table 6-2. QLogic 8400/3400 Series FCoE Drivers

Driver Description

bnx2x This driver manages all PCI device resources (registers, host inter-

face queues) and also acts as the Layer 2 VMware low-level network driver for QLogic's 8400/3400 Series 10G device. This driver directly controls the hardware and is responsible for sending and receiving Ethernet packets on behalf of the VMware host networking stack. The bnx2x driver also receives and processes device interrupts, both on behalf of itself (for L2 networking) and on be half of the bnx2fc (FCoE protocol) and CNIC drivers.

bnx2fc The QLogic VMware FCoE driver is a kernel mode driver used to

provide a translation layer between the VMware SCSI stack and the QLogic FCoE firmware/hardware. In addition, the driver interfaces with the networking layer to transmit and receive encapsulated FCoE frames on behalf of open-fcoe's libfc/libfcoe for FIP/device discovery.

Supported Distributions

The FCoE/DCB feature set is supported on VMware ESXi 5.0 and later.

58 83840-546-00 E

7 Firmware Upgrade

QLogic provides a Windows and Linux utility for upgrading adapter firmware and bootcode. Each utility executes as a console application that can be run from a command prompt. Upgrade VMware firmware with the VMware vSphere plug-in.

Upgrading Firmware for Windows

To upgrade firmware for Windows:

1. Go to driverdownloads.qlogic.com

upgrade utility for your adapter.

2. Install the firmware upgrade utility.

3. In a DOS command line, type the following command:

C:WinQlgcUpg.bat

****************************************************************************** QLogic Firmware Upgrade Utility for Windows v2.7.14.0 ****************************************************************************** C Brd MAC Drv Name

- ---- ------------ --- -----------------------------------------------------0 16A1 000E1E508E20 Yes [0061] QLogic 57840 10 Gigabit Ethernet #61 1 16A1 000E1E508E22 Yes [0062] QLogic 57840 10 Gigabit Ethernet #62 Upgrading MFW Forced upgrading MFW1 image: from ver MFW1 7.10.39 to ver MFW1 7.12.31 Upgrading MFW2 image to version MFW2 7.12.31 Upgrading SWIM1B image: to version SWIM1 7.12.31 Upgrading SWIM2B image: to version SWIM2 7.12.31 Upgrading SWIM3B image: to version SWIM3 7.12.31 Upgrading SWIM4B image: to version SWIM4 7.12.31 Upgrading SWIM5B image: to version SWIM5 7.12.31 Upgrading SWIM6B image: to version SWIM6 7.12.31 Upgrading SWIM7B image: to version SWIM7 7.12.31 Upgrading SWIM8B image: to version SWIM8 7.12.31

and download the Windows firmware

59 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Windows

Forced upgrading E3_EC_V2 image: from ver N/A to ver N/A Forced upgrading E3_PCIE_V2 image: from ver N/A to ver N/A C Brd MAC Drv Name

Upgrading MFW Forced upgrading MFW1 image: from ver MFW1 7.10.39 to ver MFW1 7.12.31 Upgrading MFW2 image to version MFW2 7.12.31 Upgrading SWIM1B image: to version SWIM1 7.12.31 Upgrading SWIM2B image: to version SWIM2 7.12.31 Upgrading SWIM3B image: to version SWIM3 7.12.31 Upgrading SWIM4B image: to version SWIM4 7.12.31 Upgrading SWIM5B image: to version SWIM5 7.12.31 Upgrading SWIM6B image: to version SWIM6 7.12.31 Upgrading SWIM7B image: to version SWIM7 7.12.31 Upgrading SWIM8B image: to version SWIM8 7.12.31 Forced upgrading E3_EC_V2 image: from ver N/A to ver N/A Forced upgrading E3_PCIE_V2 image: from ver N/A to ver N/A The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95) ****************************************************************************** QLogic Firmware Upgrade Utility for Windows v2.7.14.0 ****************************************************************************** C Brd MAC Drv Name

Upgrading MBA Updating PCI ROM header with Vendor ID = 0x14e4 Device ID = 0x16a1 Updating PCI ROM header with Vendor ID = 0x14e4 Device ID = 0x16a1 Forced upgrading MBA image: from ver PCI30 MBA 7.11.3 ;EFI x64 7.10.54 to ver

PCI30 7.12.4 C Brd MAC Drv Name

- ---- ------------ --- -----------------------------------------------------0 16A1 000E1E508E20 Yes [0061] QLogic 57840 10 Gigabit Ethernet #61

60 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Windows

1 16A1 000E1E508E22 Yes [0062] QLogic 57840 10 Gigabit Ethernet #62

PCI30 7.12.4 The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95) ****************************************************************************** QLogic Firmware Upgrade Utility for Windows v2.7.14.0 ******************************************************************************

C Brd MAC Drv Name

Upgrading L2T Forced upgrading L2T image: from ver L2T 7.10.31 to ver L2T 7.10.31 Forced upgrading L2C image: from ver L2C 7.10.31 to ver L2C 7.10.31 Forced upgrading L2X image: from ver L2X 7.10.31 to ver L2X 7.10.31 Forced upgrading L2U image: from ver L2U 7.10.31 to ver L2U 7.10.31 The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95)

61 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Linux

Upgrading Firmware for Linux

To upgrade firmware for Linux:

1. Go to driverdownloads.qlogic.com

and download the Linux firmware

upgrade utility for your adapter.

2. In a Linux command line window, type the following command:

# ./LnxQlgcUpg.sh

****************************************************************************** QLogic Firmware Upgrade Utility for Linux v2.7.13 ******************************************************************************

C Brd MAC Drv Name

- ---- ------------ --- -----------------------------------------------------0 1639 0026B942B53E Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em1) 1 1639 0026B942B540 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em2) 2 1639 0026B942B542 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em3) 3 1639 0026B942B544 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em4) 4 16A1 000E1E503150 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p1) 5 16A1 000E1E503152 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p2) Forced upgrading MFW1 image: from ver MFW1 7.12.27 to ver MFW1 7.12.31 Upgrading MFW2 image: to version MFW2 7.12.31 Upgrading SWIM1B image: to version SWIM1 7.12.31 Upgrading SWIM2B image: to version SWIM2 7.12.31 Upgrading SWIM3B image: to version SWIM3 7.12.31 Upgrading SWIM4B image: to version SWIM4 7.12.31 Upgrading SWIM5B image: to version SWIM5 7.12.31 Upgrading SWIM6B image: to version SWIM6 7.12.31 Upgrading SWIM7B image: to version SWIM7 7.12.31 Upgrading SWIM8B image: to version SWIM8 7.12.31 Forced upgrading E3_WC_V2 image: from ver N/A to ver N/A Forced upgrading E3_PCIE_V2 image: from ver N/A to ver N/A C Brd MAC Drv Name

62 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Linux

5 16A1 000E1E503152 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p2) Forced upgrading MFW1 image: from ver MFW1 7.12.31 to ver MFW1 7.12.31 Upgrading MFW2 image: to version MFW2 7.12.31 Upgrading SWIM1B image: to version SWIM1 7.12.31 Upgrading SWIM2B image: to version SWIM2 7.12.31 Upgrading SWIM3B image: to version SWIM3 7.12.31 Upgrading SWIM4B image: to version SWIM4 7.12.31 Upgrading SWIM5B image: to version SWIM5 7.12.31 Upgrading SWIM6B image: to version SWIM6 7.12.31 Upgrading SWIM7B image: to version SWIM7 7.12.31 Upgrading SWIM8B image: to version SWIM8 7.12.31 Forced upgrading E3_WC_V2 image: from ver N/A to ver N/A Forced upgrading E3_PCIE_V2 image: from ver N/A to ver N/A The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95) Successfully upgraded mf800v7c.31 ****************************************************************************** QLogic Firmware Upgrade Utility for Linux v2.7.13 ******************************************************************************

C Brd MAC Drv Name

ver PCI30_CLP MBA 7.12.4 C Brd MAC Drv Name

63 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Linux

4 16A1 000E1E503150 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p1) 5 16A1 000E1E503152 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p2) Updating PCI ROM header with Vendor ID = 0x14e4 Device ID = 0x16a1 Updating PCI ROM header with Vendor ID = 0x14e4 Device ID = 0x16a1 Forced upgrading MBA image: from ver PCI30_CLP MBA 7.12.4 to ver PCI30_CLP MBA

7.12.4 The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95) Successfully upgraded evpxe.nic

C Brd MAC Drv Name

64 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Linux

4 16A1 000E1E503150 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p1) 5 16A1 000E1E503152 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p2) NIC is not supported. C Brd MAC Drv Name

65 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Linux

C Brd MAC Drv Name

- ---- ------------ --- -----------------------------------------------------0 1639 0026B942B53E Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em1) 1 1639 0026B942B540 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em2) 2 1639 0026B942B542 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em3) 3 1639 0026B942B544 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em4) 4 16A1 000E1E503150 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p1) 5 16A1 000E1E503152 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p2) Forced upgrading FCOE_B image: from ver v7.12.4 to ver v7.12.4 skipping FCOE boot config block The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95) Successfully upgraded fcbv712.04

C Brd MAC Drv Name

66 83840-546-00 E

7–Firmware Upgrade Upgrading Firmware for Linux

Forced upgrading L2C image: from ver L2C 7.10.31 to ver L2C 7.10.31 Forced upgrading L2X image: from ver L2X 7.10.31 to ver L2X 7.10.31 Forced upgrading L2U image: from ver L2U 7.10.31 to ver L2U 7.10.31 C Brd MAC Drv Name

- ---- ------------ --- -----------------------------------------------------0 1639 0026B942B53E Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em1) 1 1639 0026B942B540 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em2) 2 1639 0026B942B542 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em3) 3 1639 0026B942B544 Yes PowerEdge R710 BCM5709 Gigabit Ethernet rev 20 (em4) 4 16A1 000E1E503150 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p1) 5 16A1 000E1E503152 Yes BCM57840 NetXtreme II 10 Gigabit Ethernet rev 11 (p1p2) Forced upgrading L2T image: from ver L2T 7.10.31 to ver L2T 7.10.31 Forced upgrading L2C image: from ver L2C 7.10.31 to ver L2C 7.10.31 Forced upgrading L2X image: from ver L2X 7.10.31 to ver L2X 7.10.31 Forced upgrading L2U image: from ver L2U 7.10.31 to ver L2U 7.10.31 The System Reboot is required in order for the upgrade to take effect. The System Reboot is required in order for the upgrade to take effect. Quitting program ... Program Exit Code: (95) Successfully upgraded l2fwv710.31

67 83840-546-00 E

8 iSCSI Protocol

 iSCSI Boot  iSCSI Crash Dump  iSCSI Offload in Windows Server  iSCSI Offload in Linux Server  iSCSI Offload on VMware Server

iSCSI Boot

QLogic 8400/3400 Series Gigabit Ethernet adapters support iSCSI boot to enable network boot of operating systems to diskless systems. iSCSI boot allows a Windows, Linux, or VMware operating system boot from an iSCSI target machine located remotely over a standard IP network.

For both Windows and Linux operating systems, iSCSI boot can be configured to boot with two distinctive paths: non-offload (also known as Microsoft/Open-iSCSI initiator) and offload (QLogic’s offload iSCSI driver or HBA). Configuration of the path is set with the HBA Boot Mode option located on the General Parameters screen of the iSCSI Configuration utility. See Table 8-1 for more information on all

General Parameters screen configuration options.

Supported Operating Systems for iSCSI Boot

The QLogic 8400/3400 Series Gigabit Ethernet adapters support iSCSI boot on the following operating systems:

 Windows Server 2008 and later 32-bit and 64-bit (supports offload and

non-offload paths)

 RHEL 5.5 and later, SLES 11.1 and later (supports offload and non-offload

paths)

 SLES 10.x and SLES 11 (only supports non-offload path)  VMware ESXi 5.0 and later (only support non-offload path)

68 83840-546-00 E

8–iSCSI Protocol iSCSI Boot

iSCSI Boot Setup

The iSCSI boot setup consists of:

 Configuring the iSCSI Target  Configuring iSCSI Boot Parameters  Preparing the iSCSI Boot Image  Booting

Configuring the iSCSI Target

Configuring the iSCSI target varies by target vendors. For information on configuring the iSCSI target, refer to the documentation provided by the vendor. The general steps include:

1. Create an iSCSI target (for targets such as SANBlaze

vdisk/volume (for targets such as EqualLogic

2. Create a virtual disk.

or EMC®).

or IET®) or a

3. Map the virtual disk to the iSCSI target created in step 1.

4. Associate an iSCSI initiator with the iSCSI target.

5. Record the iSCSI target name, TCP port number, iSCSI Logical Unit

Number (LUN), initiator Internet Qualified Name (IQN), and CHAP authentication details.

6. After configuring the iSCSI target, obtain the following:

 Target IQN  Target IP address  Target TCP port number  Target LUN  Initiator IQN  CHAP ID and secret

Configuring iSCSI Boot Parameters

Configure the QLogic iSCSI boot software for either static or dynamic configuration. Refer to Table 8-1 for configuration options available from the General Parameters screen.

69 83840-546-00 E

8–iSCSI Protocol

NOTE

iSCSI Boot

Table 8-1 lists parameters for both IPv4 and IPv6. Parameters specific to either

IPv4 or IPv6 are noted.

Availability of IPv6 iSCSI boot is platform/device dependent.

Table 8-1. Configuration Options

Option Description

TCP/IP parameters via DHCP

IP Autoconfiguration This option is specific to IPv6. Controls whether the iSCSI

iSCSI parameters via DHCP

CHAP Authentication Controls whether the iSCSI boot host software uses CHAP

DHCP Vendor ID Controls how the iSCSI boot host software interprets the

This option is specific to IPv4. Controls whether the iSCSI boot host software acquires the IP address information using DHCP (Enabled) or use a static IP configuration (Disabled).

boot host software will configure a stateless link-local address and/or stateful address if DHCPv6 is present and used (Enabled). Router Solicit packets a re sent out up to three times with 4 second intervals in between each retry. Or use a static IP configuration (Disabled).

Controls whether the iSCSI boot host software acquires its iSCSI target parameters using DHCP (Enabled) or through a static configuration (Disabled). The static information is entered through the iSCSI Initiator Parameters Configuration screen.

authentication when connecting to the iSCSI target. If CHAP Authentication is enabled, the CHAP ID and CHAP Secret are entered through the iSCSI Initiator Parameters Configuration screen.

Vendor Class ID field used during DHCP. If the Vendor Class ID field in the DHCP Offer packet matches the value in the field, the iSCSI boot host software looks into the DHCP Option 43 fields for the required iSCSI boot extensions. If DHCP is disabled, this value does not need to be set.

Link Up Delay Time Controls how long the iSCSI boot host sof t ware waits, in sec-

onds, after an Ethernet link is est ablished before sending any data over the network. The valid values are 0 to 255. As an example, a user may need to set a value for this option if a network protocol, such as S panning Tree, is enabled on the switch interface to the client system.

Use TCP Timestamp Controls if the TCP Timestamp option is enabled or disabled.

70 83840-546-00 E

8–iSCSI Protocol iSCSI Boot

Table 8-1. Configuration Options (Continued)

Option Description

Target as First HDD Allows specifying that the iSCSI target drive will appear as

the first hard drive in the system.

LUN Busy Retry Count Controls the number of connection retries the iSCSI Boot ini-

tiator will attempt if the iSCSI target LUN is busy.

IP Version This option specific to IPv6. Toggles between the IPv4 or

IPv6 protocol. All IP settings will be lost when switching from one protocol version to another.

HBA Boot Mode Set to disable when the host OS is configured for software

initiator mode and to enable for HBA mode. This option is available only on 8400 Series adapters. This parameter cannot be changed when the adapter is in Multi-Function mode.

MBA Boot Protocol Configuration

To configure the boot protocol

1. Restart your system.

2. In the QLogic 577xx/578xx Ethernet Boot Agent banner (Figure 8-1), press

CTRL+S.

Figure 8-1. QLogic 577xx/578xx Ethernet Boot Agent

71 83840-546-00 E

8–iSCSI Protocol iSCSI Boot

3. In the CCM device list (Figure 8-2), use the up or down arrow keys to select

a device, and then press ENTER.

Figure 8-2. CCM Device List

4. In the Main menu, select MBA Configuration (Figure 8-3), and then press

ENTER.

Figure 8-3. Selecting MBA Configuration

72 83840-546-00 E

Qlogic FastLinQ 3400 Series, FastLinQ 8400 Series User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Table of Contents

List of Figures

List of Tables

Preface

Intended Audience

What Is in This Guide

Related Materials

Documentation Conventions

License Agreements

Technical Support

Downloading Updates

Training

Contact Information

Knowledge Database

Legal Notices

Warranty

Laser Safety

FDA Notice

Agency Certification

EMI and EMC Requirements

Product Safety Compliance

1 Product Overview

Functional Description

Features

iSCSI

FCoE

Power Management

Adaptive Interrupt Frequency

ASIC with Embedded RISC Processor

Adapter Management

QLogic Control Suite CLI

QLogic QConvergeConsole Graphical User Interface

QLogic QConvergeConsole vCenter Plug-In

QLogic FastLinQ ESXCLI VMware Plug-In

Supported Operating Environments

Adapter Specifications

Physical Characteristics

Standards Specifications

2 Installing the Hardware

System Requirements

Hardware Requirements

Operating System Requirements

Safety Precautions

Preinstallation Checklist

Installation of the Network Adapter

Connecting the Network Cables

Overview

Setting Up MBA in a Client Environment

Enabling the MBA Driver

Configuring the MBA Driver

Setting Up MBA in a Server Environment

Setting Up the BIOS

Red Hat Linux PXE Server

MS-DOS UNDI/Intel APITEST

4 Windows Driver Software

Installing the Driver Software

Using the Installer

Using Silent Installation

Manually Extracting the Device Drivers

Removing the Device Drivers

Installing QLogic Management Applications

Viewing or Changing the Adapter Properties

Setting Power Management Options

5 Linux Driver Software

Introduction

Limitations

bnx2x Driver

bnx2i Driver

bnx2fc Driver

Packaging

Installing Linux Driver Software

Installing the Source RPM Package

Installing the KMP Package

Building the Driver from the Source TAR File

Load and Run Necessary iSCSI Software Components

Unloading/Removing the Linux Driver