Qlogic 8400 Series, 3400 Series User Manual

Download

Page 1

User’s Guide

Converged Network Adapters and Intelligent

Ethernet Adapters

8400, 3400 Series

83840-546-00 D

Page 2

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Information furnished in this manual is believed to be accurate and reliable. However, QLogic Corporation assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. QLogic Corporation reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for illustrative purposes only. QLogic Corporation makes no representation nor warranty that such applications are suitable for the specified use without further testing or modification. QLogic Corporation assumes no responsibility for any errors that may appear in this document.

Document Revision History

Revision A, October 20, 2014 Revision B, November 14, 2014 Revision C, April 3, 2015 Revision D, September 16, 2015

Changes Sections Affected

Updated “Single-chip solution” bullet to exclude

“Features” on page 2

QLE3442-RJ adapters Removed universal management port (UMP) from

the list of manageability features Added the section Adapter Specifications “Adapter Specifications” on page 5 Moved “Network Link and Activity Indication” to

“Adapter LEDS” on page 359

Appendix A

In Table 5-1, removed references to 1Gb adapters. “Introduction” on page 23 In step 1, added modprobe -r bnx2 command

as an option.

“Load and Run Necessary iSCSI Software Compo-

nents” on page 30

Updated instances of QCS4 to QCS. Throughout Added RSS support for VMware ESXi 6.0. “RSS” on page 52 Added VxLAN offload support for VMware ESXi

“enable_vxlan_offld” on p age 52

6.0. Updated the default for enable_vxlan_offld to

enable_vxlan_offld =1. Added missing chapter, “Using QLogic Control

Chapter 8

Suite”. In step 5, removed IP addresses from the first bul-

“iSCSI Boot Configuration” on page 169

let. In step 4b, updated “driver” to “drive”.

“Linux iSCSI Boot Setup” on page 182

In step 7b, updated “with iscsi=1” to “withiscsi=1”. In step 2, updated the bad cross reference to

“Viewing Resource Reservations”.

“Configuring QLogic iSCSI Using QCS” on

page 194

Added iSCSI offload support for VMware ESXi 6.0 “iSCSI Offload on VMware Server” on page 212

ii 83840-546-00 D

Page 3

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters

8400, 3400 Series

Added FCoE boot support for VMware ESXI 5.5 and 6.0

Added NPAR support for VMware ESXi 6.0 “Supported Operating Systems for NIC Partition-

In Table 20-1, removed command option uediag

-fump <ump_image>

Removed Chapter 22 Specifications and Chapter 23 Regulatory Information

“VMware ESXi FCoE Boot Installation” on

page 242

ing” on page 251

“Performing Diagnostics” on page 341

iii 83840-546-00 D

Page 4

User’s Guide Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

iv 83840-546-00 D

Page 5

Preface

Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

What Is in This Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

Related Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv

Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv

License Agreements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi

Technical Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Downloading Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxviii

Contact Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxviii

Knowledge Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxviii

Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix

Warranty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix

Laser Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix

FDA Notice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix

Agency Certification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix

EMI and EMC Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix

Product Safety Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxx

1 Product Overview

Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Adaptive Interrupt Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

ASIC with Embedded RISC Processor . . . . . . . . . . . . . . . . . . . . . . . . 5

QLogic Control Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Supported Operating Environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Adapter Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Physical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Standards Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

v 83840-546-00 D

Page 6

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

2 Installing the Hardware

System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Operating System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Safety Precautions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Preinstallation Checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Installation of the Network Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Connecting the Network Cables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Multi-boot Agent (MBA) Driver Software

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Setting Up MBA in a Client Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Enabling the MBA Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Configuring the MBA Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Setting Up the BIOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Setting Up MBA in a Server Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Red Hat Linux PXE Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

MS-DOS UNDI/Intel APITEST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Windows Driver Software

Installing the Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Using the Installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Using Silent Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Removing the Device Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Installing QLogic Management Applications . . . . . . . . . . . . . . . . . . . . . . . . 18

Viewing or Changing the Adapter Properties. . . . . . . . . . . . . . . . . . . . . . . . 19

Setting Power Management Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5 Linux Driver Software

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

bnx2i Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

bnx2fc Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Packaging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Installing Linux Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Installing the Source RPM Package . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Installing the KMP Package. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Building the Driver from the Source TAR File . . . . . . . . . . . . . . . . . . . 29

Load and Run Necessary iSCSI Software Components . . . . . . . . . . . . . . . 30

vi 83840-546-00 D

Page 7

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Unloading/Removing the Linux Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Unloading/Removing the Driver from an RPM Installation . . . . . . . . . 31

Removing the Driver from a TAR Installation . . . . . . . . . . . . . . . . . . . 31

Uninstalling QCS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

RPM Package. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Patching PCI Files (Optional). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Network Installations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Setting Values for Optional Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

disable_tpa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

int_mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

dropless_fc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

disable_iscsi_ooo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

multi_mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

num_queues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

pri_map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

qs_per_cos. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

cos_min_rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

bnx2i Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

error_mask1 and error_mask2. . . . . . . . . . . . . . . . . . . . . . . . . . 36

en_tcp_dack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

time_stamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

sq_size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

rq_size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

event_coal_div . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

last_active_tcp_port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

ooo_enable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

bnx2fc Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

debug_logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Driver Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

bnx2 Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

vii 83840-546-00 D

Page 8

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

bnx2x Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Driver Sign On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

CNIC Driver Sign On (bnx2 only) . . . . . . . . . . . . . . . . . . . . . . . . 40

NIC Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Link Up and Speed Indication. . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Link Down Indication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

MSI-X Enabled Successfully . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

bnx2i Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

BNX2I Driver Signon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Network Port to iSCSI Transport Name Binding. . . . . . . . . . . . . 41

Driver Completes handshake with iSCSI Offload-enabled

CNIC Device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Driver Detects iSCSI Offload Is Not Enabled on the CNIC Device 41 Exceeds Maximum Allowed iSCSI Connection Offload Limit. . . 41 Network Route to Target Node and Transport Name Binding

Are Two Different Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Target Cannot Be Reached on Any of the CNIC Devices. . . . . . 41

Network Route Is Assigned to Network Interface, Which Is Down 41

SCSI-ML Initiated Host Reset (Session Recovery) . . . . . . . . . . 41

CNIC Detects iSCSI Protocol Violation - Fatal Errors. . . . . . . . . 42

CNIC Detects iSCSI Protocol Violation - Non-FATAL, Warning . 43

Driver Puts a Session Through Recovery. . . . . . . . . . . . . . . . . . 43

Reject iSCSI PDU Received from the Target . . . . . . . . . . . . . . . 43

Open-iSCSI Daemon Handing Over Session to Driver . . . . . . . 43

bnx2fc Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

BNX2FC Driver Signon: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Driver Compiles Handshake with FCoE Offload Enabled

CNIC Device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Driver Fails Handshake with FCoE Offload Enabled CNIC Device 43

No Valid License to Start FCoE . . . . . . . . . . . . . . . . . . . . . . . . . 44

Session Failures Due to Exceeding Maximum Allowed FCoE

Offload Connection Limit or Memory Limits . . . . . . . . . . . . . . . 44

Session Offload Failures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Session Upload Failures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Unable to Issue ABTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Unable to Recover the IO Using ABTS (Due to ABTS Timeout) 44

Unable to Issue IO Request Due to Session Not Ready . . . . . . 44

Drop Incorrect L2 Receive Frames. . . . . . . . . . . . . . . . . . . . . . . 44

HBA/lport Allocation Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

NPIV Port Creation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

viii 83840-546-00 D

Page 9

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Teaming with Channel Bonding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6 VMware Driver Software

Packaging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Download, Install, and Update Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Networking Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Driver Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

int_mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

disable_tpa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

num_rx_queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

num_tx_queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

pri_map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

qs_per_cos. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

cos_min_rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

dropless_fc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

RSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

max_vfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

enable_vxlan_offld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Driver Defaults. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Unloading and Removing Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Driver Sign On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

NIC Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

MSI-X Enabled Successfully . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Link Up and Speed Indication. . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Link Down Indication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Memory Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

MultiQueue/NetQueue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

FCoE Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Enabling FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Installation Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Supported Distributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7 Installing Management Applications

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

QCS Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Communication Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

ix 83840-546-00 D

Page 10

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Installation Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

WS-MAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Windows Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Windows Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Linux Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Linux Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

CIM-XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Linux Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

WMI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Windows Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Windows Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Detailed Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

WS-MAN on Windows Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Install the WinRM Software Component on Server . . . . . . . . . . 61

Perform Basic Configuration on the Server . . . . . . . . . . . . . . . . 62

Perform User Configuration on the Server. . . . . . . . . . . . . . . . . 62

Perform HTTP Configuration on the Server . . . . . . . . . . . . . . . . 62

Perform HTTPS Configuration on the Server

(to use HTTPS Instead of HTTP). . . . . . . . . . . . . . . . . . . . . . . 64

Configure WinRM HTTPS/SSL on the Server . . . . . . . . . . . . . . 68

Additional Server Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 70

WS-MAN—Windows Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Perform HTTP Configuration (if you plan to use HTTP). . . . . . . 71

Perform HTTPS Configuration (if you plan to use HTTPS) . . . . 72

Configure WinRM HTTPS/SSL. . . . . . . . . . . . . . . . . . . . . . . . . . 73

WS-MAN and CIM-XML—Linux Server. . . . . . . . . . . . . . . . . . . . . . . . 73

Install OpenPegasus From the Inbox RPM (Red Hat Only). . . . 74

Install OpenPegasus From Source (Red Hat and SuSE). . . . . . 75

Start CIM Server on the Server. . . . . . . . . . . . . . . . . . . . . . . . . . 77

Configure OpenPegasus on the Server . . . . . . . . . . . . . . . . . . . 77

Install QLogic CMPI Provider . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Perform Linux Firewall Configuration, If Required . . . . . . . . . . . 79

WS-MAN and CIM-XML—Linux Client . . . . . . . . . . . . . . . . . . . . . . . . 80

Configure HTTPS on Linux Client . . . . . . . . . . . . . . . . . . . . . . . 80

Install QCS Management Application. . . . . . . . . . . . . . . . . . . . . 81

WMI—Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Set up Namespace Security Using WMI Control . . . . . . . . . . . . 82

Grant DCOM Remote Launch and Activate Permission. . . . . . . 83

Special Configuration for WMI on Different Systems . . . . . . . . . 84

x 83840-546-00 D

Page 11

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Installing QCS and Related Management Applications . . . . . . . . . . . . . . . . 85

Installing on a Windows System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Installing on a Linux System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Management Applications (Windows) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Modifying the Management Application. . . . . . . . . . . . . . . . . . . . . . . . 86

Repairing Management Applications. . . . . . . . . . . . . . . . . . . . . . . . . . 86

Removing Management Applications . . . . . . . . . . . . . . . . . . . . . . . . . 87

8 Using QLogic Control Suite

QLogic Control Suite Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Starting QLogic Control Suite. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

QCS Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Explorer View Pane. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Context View Selector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Filter View. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Context View Pane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Menu Bar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Discovery Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Manage Targets Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Manage iSNS Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Manage Discovery Portals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Boot Configuration Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Hardware and Resource Configuration Wizard . . . . . . . . . . . . . 93

Description Pane. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Configuring Preferences in Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Connecting to a Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Managing the Host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Information Tab: Host Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Information Tab: iSCSI Initiator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Configuration Tab: System Management . . . . . . . . . . . . . . . . . . . . . . 97

Managing the Network Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Viewing Adapter Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Viewing Resource Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Information Tab: Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Viewing Hardware Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Information Tab: Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Configuring Adapter Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Hardware and Resource Configuration Wizard: Port Configuration . . 99

Managing Ethernet Controller (Port) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Viewing Port Level Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

xi 83840-546-00 D

Page 12

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Viewing Vital Signs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Viewing NIC Partitioning Information. . . . . . . . . . . . . . . . . . . . . . . . . . 103

Testing the Network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Running Diagnostic Tests in Windows. . . . . . . . . . . . . . . . . . . . . . . . . 106

Analyzing Cables in Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Managing the LAN Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Viewing NDIS Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Viewing Driver Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Information Tab: Vital Signs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Information Tab: SR-IOV Switch Information . . . . . . . . . . . . . . . 113

Configuring the NDIS Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Viewing Resource Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Configuring System Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Viewing Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

General Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

IEEE 802.3 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Custom Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Viewing Resource Reservations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Configuring the IP Address for iSCSI Offload . . . . . . . . . . . . . . . . . . . 128

Viewing Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Configuring Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Team Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Smart Load Balance and Failover. . . . . . . . . . . . . . . . . . . . . . 130

Link Aggregation (802.3ad) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Generic Trunking (FEC/GEC)/802.3ad-Draft Static . . . . . . . . . . 130

SLB (Auto-Fallback Disable) . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Standby Team Member and Auto-Fallback Disable Mode . . . . . . . . . 131

LiveLink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Using the QLogic Teaming Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Using Expert Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Creating a Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Modifying a Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

Adding a VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Viewing VLAN Properties and Statistics and Running VLAN Tests 149

Deleting a VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Configuring LiveLink for a Smart Load Balancing and Failover

and SLB (Auto-Fallback Disable) Team . . . . . . . . . . . . . . . . . . 149

Saving and Restoring a Team Configuration . . . . . . . . . . . . . . . 151

Viewing ASP Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

xii 83840-546-00 D

Page 13

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Configuring With the Command Line Interface Utility. . . . . . . . . . . . . . . . . . 152

Troubleshooting QCS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

9 Firmware Upgrade

Upgrading Firmware for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Upgrading Firmware for Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

10 iSCSI Protocol

iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Supported Operating Systems for iSCSI Boot. . . . . . . . . . . . . . . . . . . 164

iSCSI Boot Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Configuring the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Configuring iSCSI Boot Parameters. . . . . . . . . . . . . . . . . . . . . . 165

MBA Boot Protocol Configuration. . . . . . . . . . . . . . . . . . . . . . . . 167

iSCSI Boot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Enabling CHAP Authentication. . . . . . . . . . . . . . . . . . . . . . . . . . 174

Configuring the DHCP Server to Support iSCSI Boot . . . . . . . . 175

DHCP iSCSI Boot Configurations for IPv4. . . . . . . . . . . . . . . . . 175

DHCP iSCSI Boot Configuration for IPv6. . . . . . . . . . . . . . . . . . 177

Configuring the DHCP Server . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Preparing the iSCSI Boot Image. . . . . . . . . . . . . . . . . . . . . . . . . 178

Booting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Configuring VLANs for iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Other iSCSI Boot Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Changing the Speed and Duplex Settings in

Windows Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Virtual LANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

The 'dd' Method of Creating an iSCSI Boot Image. . . . . . . . . . . 190

Troubleshooting iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

iSCSI Crash Dump. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

iSCSI Offload in Windows Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

iSCSI Offload Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Configuring iSCSI Offload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Installing QLogic Drivers and Management Applications. . . . . . 194

Installing the Microsoft iSCSI Initiator. . . . . . . . . . . . . . . . . . . . . 194

Configuring QLogic iSCSI Using QCS . . . . . . . . . . . . . . . . . . . . 194

Configuring a VLAN for the iSCSI Interface . . . . . . . . . . . . . . . . 194

Configure Microsoft Initiator to Use QLogic’s iSCSI Offload. . . . 198

iSCSI Offload FAQs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Event Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

xiii 83840-546-00 D

Page 14

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

iSCSI Offload in Linux Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Open iSCSI User Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

User Application - qlgc_iscsiuio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Bind iSCSI Target to QLogic NX2 iSCSI Transport Name. . . . . . . . . . 209

VLAN Configuration for iSCSI Offload (Linux). . . . . . . . . . . . . . . . . . . 210

Modifying the iSCSI iface File. . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Setting the VLAN ID on the Ethernet Interface. . . . . . . . . . . . . . 210

Making Connections to iSCSI Targets. . . . . . . . . . . . . . . . . . . . . . . . . 211

Add Static Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

iSCSI Target Discovery Using 'SendTargets' . . . . . . . . . . . . . . . 211

List All Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

List All Drives Active in the System . . . . . . . . . . . . . . . . . . . . . . 212

Maximum Offload iSCSI Connections. . . . . . . . . . . . . . . . . . . . . . . . . 212

Linux iSCSI Offload FAQ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

iSCSI Offload on VMware Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

1 1 Fibre Channel Over Ethernet

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

FCoE Boot from SAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Preparing System BIOS for FCoE Build and Boot. . . . . . . . . . . . . . . . 216

Modify System Boot Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Specify BIOS Boot Protocol (if required) . . . . . . . . . . . . . . . . . . 216

Prepare QLogic Multiple Boot Agent for FCoE Boot. . . . . . . . . . . . . . 217

UEFI Boot LUN Scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Provisioning Storage Access in the SAN. . . . . . . . . . . . . . . . . . . . . . . 223

Pre-Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

CTRL+R Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

One-Time Disabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Windows Server 2008 R2 and Windows Server 2008 SP2

FCoE Boot Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Windows Server 2012/2102 R2 FCoE Boot Installation . . . . . . . . . . . 228

Linux FCoE Boot Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

SLES11 SP2 Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

RHEL6 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Linux: Adding Additional Boot Paths . . . . . . . . . . . . . . . . . . . . . 240

VMware ESXi FCoE Boot Installation . . . . . . . . . . . . . . . . . . . . . . . . . 242

Configuring FCoE Boot from SAN on VMware. . . . . . . . . . . . . . 246

Booting from SAN After Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Driver Upgrade on Linux Boot from SAN Systems . . . . . . . . . . . . . . . 247

xiv 83840-546-00 D

Page 15

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Errors During Windows FCoE Boot from SAN Installation . . . . . . . . . 248

Configuring FCoE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

12 NIC Partitioning and Bandwidth Management

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Supported Operating Systems for NIC Partitioning. . . . . . . . . . . . . . . 251

Configuring for NIC Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Number of Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Network MAC Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

iSCSI MAC Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Physical Link Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Relative Bandwidth Weight (%). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Maximum Bandwidth (%). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

13 Virtual LANs in Windows

VLAN Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Adding VLANs to Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

14 SR-IOV

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Enabling SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

SR-IOV and Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

SR-IOV and Jumbo Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

15 Microsoft Virtualization with Hyper-V

Supported Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

Single Network Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

Windows Server 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

Windows Server 2008 R2 and 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . 262

Teamed Network Adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

Windows Server 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

Windows Server 2008 R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

Configuring VMQ with SLB Teaming. . . . . . . . . . . . . . . . . . . . . . . . . . 265

Upgrading Windows Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

16 Data Center Bridging (DCB)

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

DCB Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

Enhanced Transmission Selection (ETS) . . . . . . . . . . . . . . . . . . . . . . 268

xv 83840-546-00 D

Page 16

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Priority Flow Control (PFC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

Data Center Bridging eXchange (DCBX) . . . . . . . . . . . . . . . . . . . . . . 268

Configuring DCB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

DCB Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

Data Center Bridging in Windows Server 2012 . . . . . . . . . . . . . . . . . . . . . . 270

17 QLogic Teaming Services

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

Teaming Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

Network Addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

Teaming and Network Addresses. . . . . . . . . . . . . . . . . . . . . . . . 274

Description of Teaming Types . . . . . . . . . . . . . . . . . . . . . . . . . . 275

Software Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Repeater Hub. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Switching Hub. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

Teaming Support by Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

Configuring Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

Supported Features by Team Type . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

Selecting a Team Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

Teaming Mechanisms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Outbound Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

Inbound Traffic Flow (SLB Only). . . . . . . . . . . . . . . . . . . . . . . . . 288

Protocol Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

Types of Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

Switch-Independent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

Switch-Dependent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

LiveLink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

Attributes of the Features Associated with Each Type of Team . . . . . 294

Speeds Supported for Each Type of Team . . . . . . . . . . . . . . . . . . . . . 298

Teaming and Other Advanced Networking Properties . . . . . . . . . . . . . . . . 298

Checksum Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

IEEE 802.1p QoS Tagging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

Large Send Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

Jumbo Frames. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

IEEE 802.1Q VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

xvi 83840-546-00 D

Page 17

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

Preboot Execution Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

General Network Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

Teaming with Microsoft Virtual Server 2005 . . . . . . . . . . . . . . . . . . . . 301

Teaming Across Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Switch-Link Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Spanning Tree Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

Topology Change Notice (TCN) . . . . . . . . . . . . . . . . . . . . . . . . . 307

Port Fast/Edge Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

Layer 3 Routing/Switching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

Teaming with Hubs (for troubleshooting purposes only) . . . . . . . . . . . 308

Hub Usage in Teaming Network Configurations. . . . . . . . . . . . . 308

SLB Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

SLB Team Connected to a Single Hub. . . . . . . . . . . . . . . . . . . . 309

Generic and Dynamic Trunking (FEC/GEC/IEEE 802.3ad) . . . . 309

Teaming with Microsoft NLB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Application Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Teaming and Clustering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Microsoft Cluster Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

High-Performance Computing Cluster . . . . . . . . . . . . . . . . . . . . 312

Oracle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

Teaming and Network Backup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

Load Balancing and Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

Fault Tolerance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

Troubleshooting Teaming Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

Teaming Configuration Tips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

Troubleshooting Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

Windows System Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . 323

Base Driver (Physical Adapter/Miniport) . . . . . . . . . . . . . . . . . . . . . . . 323

Intermediate Driver (Virtual Adapter/Team). . . . . . . . . . . . . . . . . . . . . 327

Virtual Bus Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

18 Configuring Teaming in Windows Server

ASP Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

Load Balancing and Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

Types of Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

Smart Load Balancing and Failover . . . . . . . . . . . . . . . . . . . . . . . . . . 334

Link Aggregation (802.3ad). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

Generic Trunking (FEC/GEC)/802.3ad-Draft Static. . . . . . . . . . . . . . . 335

xvii 83840-546-00 D

Page 18

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

SLB (Auto-Fallback Disable) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

Limitations of Smart Load Balancing and Failover/SLB

(Auto-Fallback Disable) Types of Teams. . . . . . . . . . . . . . . . . . . . . . 336

Teaming and Large Send Offload/Checksum Offload Support . . . . . . 337

19 Manageability

CIM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

HBA API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

20 User Diagnostics in DOS

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

Performing Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

Diagnostic Test Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

21 T roubleshooting

Hardware Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

QCS Diagnostic Tests Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

QCS Network Test Failures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

Checking Port LEDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

Troubleshooting Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

Checking if Current Drivers are Loaded. . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

Running a Cable Length Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Testing Network Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Possible Problems and Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

Multi-boot Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

ASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

NPAR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

A Adapter LEDS

xviii 83840-546-00 D

Page 19

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters

8400, 3400 Series

List of Figures

Figure Page

3-1 MBA Configuration Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4-1 Power Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6-1 Selecting an Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6-2 QLE3442 Driver Versions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6-3 PCI Identifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6-4 List of Driver Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6-5 Download Driver Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

7-1 MGMTAPP-LAB3 Dialog Box. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

7-2 Launch and Activation Permission. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

8-1 QCS Main Window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

8-2 QCS Host Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

8-3 QCS Host Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

8-4 QCS Adapter Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

8-5 QCS Vital Signs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

8-6 QCS—Network Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

8-7 QCS—Cable Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

8-8 QCS—NDIS-level Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

8-9 QCS—Configure NDIS-level Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

8-10 QCS—General Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

8-11 Teaming Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

8-12 Entering a Team Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

8-13 Configuring the Team Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

8-14 Adding an Adapter to a Team. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

8-15 Configure Auto-Fallback Disable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

8-16 Configure LiveLink. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

8-17 Adding a VLAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

8-18 Naming the VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

8-19 Tagging the VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

8-20 Entering a VLAN Tag Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

8-21 Commit Changes and Exit the Teaming Wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

8-22 Previewing the Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

8-23 Team Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

8-24 Expert Mode—Create a Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

8-25 Expert Mode—Edit a Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

8-26 Expert Mode—Create a Team with a VLAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

10-1 QLogic 577xx/578xx Ethernet Boot Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

10-2 CCM Device List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

10-3 Selecting MBA Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

10-4 Selecting the iSCSI Boot Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

10-5 Selecting iSCSI Boot Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

10-6 Selecting General Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

10-7 Saving the iSCSI Boot Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

10-8 Comprehensive Configuration Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

xix 83840-546-00 D

Page 20

User’s Guide Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

10-9 Configuring VLANs—CCM Device List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

10-10 Configuring VLANs—Multiboot Agent Configuration . . . . . . . . . . . . . . . . . . . . . . . . 188

10-11 Configuring iSCSI Boot VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

10-12 Saving the iSCSI Boot VLAN Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

10-13 iSCSI View. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

10-14 Selecting the iSCSI Adapter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

10-15 Specifying a VLAN ID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

10-16 iSCSI Initiator Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

10-17 iSCSI Initiator Node Name Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

10-18 iSCSI Initiator—Add a Target Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

10-19 Target Portal IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

10-20 Selecting the Local Adapter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

10-21 Selecting the Initiator IP Address. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

10-22 Adding the Target Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

10-23 Logging on to the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

10-24 Log On to Target Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

10-25 Assigning a VLAN Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

10-26 Configuring the VLAN on VMKernel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

11-1 FCoE Boot<Variable>—CCM Device List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

11-2 FCoE Boot<Variable>—Enable DCB/DCBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

11-3 FCoE Boot<Variable>—Select FCoE Boot Protocol . . . . . . . . . . . . . . . . . . . . . . . . 218

11-4 FCoE Boot<Variable>—Target Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

11-5 FCoE Boot<Variable>—Specify Target WWPN and Boot LUN . . . . . . . . . . . . . . . . 220

11-6 FCoE Boot Target Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

11-7 FCoE Boot Configuration Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

11-8 FCoE Target Parameters Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

11-9 Selecting an FCoE WWPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

11-10 One-time Disabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

11-11 Load EVBD Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

11-12 Load bxfcoe Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

11-13 Selecting the FCoE Boot LUN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

11-14 SLES Boot Options Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

11-15 Choosing Driver Update Medium. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

11-16 FCoE Reboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

13-1 Example of Servers Supporting Multiple VLANs with Tagging. . . . . . . . . . . . . . . . . 254

17-1 Process for Selecting a Team Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

17-2 Intermediate Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

17-3 Teaming Across Switches Without an Interswitch Link . . . . . . . . . . . . . . . . . . . . . . 303

17-4 Teaming Across Switches With Interconnect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

17-5 Failover Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

17-6 Team Connected to a Single Hub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

17-7 Clustering With Teaming Across One Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

17-8 Clustering With Teaming Across Two Switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

17-9 Network Backup without Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

17-10 Network Backup With SLB Teaming Across Two Switches . . . . . . . . . . . . . . . . . . . 317

xx 83840-546-00 D

Page 21

User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters

8400, 3400 Series

List of Tables

Table Page

2-1 100/1000BASE-T and 10GBASE-T Cable Specifications . . . . . . . . . . . . . . . . . . . . 10

4-1 Windows Operating Systems and iSCSI Crash Dump . . . . . . . . . . . . . . . . . . . . . . . 17

5-1 QLogic 8400/3400 Series Linux Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6-1 VMware Driver Packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6-2 QLogic 8400/3400 Series FCoE Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7-1 QCS Communication Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7-2 WinRM Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

7-3 OpenPegasus Environment Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

7-4 cimconfig Commands to Configure OpenPegasus . . . . . . . . . . . . . . . . . . . . . . . . . 77

8-1 Maximum Number of Offload Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

10-1 Configuration Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

10-2 DHCP Option 17 Parameter Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

10-3 DHCP Option 43 Suboption Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

10-4 DHCP Option 17 Suboption Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

10-5 Offload iSCSI (OIS) Driver Event Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . 203

13-1 Example VLAN Network Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

15-1 Configurable Network Adapter Hyper-V Features . . . . . . . . . . . . . . . . . . . . . . . . . . 261

15-2 Configurable Teamed Network Adapter Hyper-V Features . . . . . . . . . . . . . . . . . . . 263

17-1 Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

17-2 Available Teaming Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

17-3 QLogic Teaming Software Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

17-4 Comparison of Team Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

17-5 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

17-6 Link Speeds in Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

17-7 Advanced Adapter Properties and Teaming Support. . . . . . . . . . . . . . . . . . . . . . . . 298

17-8 Base Driver Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

17-9 Intermediate Driver Event Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

17-10 VBD Event Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

18-1 Smart Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

20-1 uediag Command Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

20-2 Diagnostic Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

A-1 Network Link and Activity Indicated by the RJ-45 Port LEDs. . . . . . . . . . . . . . . . . . 359

A-2 Network Link and Activity Indicated by the Port LED. . . . . . . . . . . . . . . . . . . . . . . . 359

xxi 83840-546-00 D

Page 22

User’s Guide Converged Network Adapters and Intelligent Ethernet Adapters 8400, 3400 Series

xxii 83840-546-00 D

Page 23

Preface

Intended Audience

This guide is intended for personnel responsible for installing and maintaining computer networking equipment.

What Is in This Guide

This guide describes the features, installation, and configuration of the QLogic® 8400/3400 Series Converged Network Adapters and Intelligent Ethernet Adapters. The guide is organized as follows:

 Chapter 1, Product Overview provides a product functional description, a list

of features, a list of supported operating systems, and the adapter specifications.

 Chapter 2, Installing the Hardware describes how to install the adapter

including the list of system requirements and a preinstallation checklist.

 Chapter 3, Multi-boot Agent (MBA) Driver Software describes the software

module that allows your network computer to boot with the images provided by remote servers across the network.

 Chapter 4, Windows Driver Software describes Windows

and removal, QLogic management application installation, adapter properties management, and power management options.

 Chapter 5, Linux Driver Software describes the Linux  Chapter 6, VMware Driver Software describes the VMware

 Chapter 7, Installing Management Applications describes the installation of

the QLogic Control Suit e.

 Chapter 8, Using QLogic Control Suite describes the QCS application

including how to perform detailed tests and diagnostics, and to view and modify property values and view traffic statistics for network objects.

 Chapter 9, Firmware Upgrade describes the installation and use of the

firmware upgrade utility.

 Chapter 10, iSCSI Protocol describes iSCSI boot, iSCSI crash dump, and

iSCSI offload for Windows, Linux, and VMware.

xxiii 83840-546-00 D

driver installation

drivers.

Page 24

Preface Related Materials

 Chapter 11, Fibre Channel Over Ethernet describes FCoE boot from SAN

and booting from SAN after installation.

 Chapter 12, NIC Partitioning and Bandwidth Management describes the

NPAR operating system requirements and the NPAR configuration parameters.

 Chapter 13, Virtual LANs in Windows describes the use of VLANs to divide

the physical LAN into functional segments.

 Chapter 14, SR-IOV describes the use of Single-Root I/O Virtualization

(SR-IOV) to virtualize network controllers and how to enable SR-IOV.

 Chapter 15, Microsoft Virtualization with Hyper-V describes the use of

Microsoft

Hyper-V® for Windows Server 2008 and 2012.

 Chapter 16, Data Center Bridging (DCB) describes the DCB capabilities

configuration, and requirements.

 Chapter 17, QLogic Teaming Services describes the use of teaming to group

multiple physical devices to provide fault tolerance and load balancing.

 Chapter 18, Configuring Teaming in Windows Server describes the teaming

configuration for Windows Server

 Chapter 19, Manageability describes the implementation of the common

information model (CIM) and the Storage Networking Industry Association (SNIA) Common HBA API on Windows and Linux operating systems.

 Chapter 20, User Diagnostics in DOS describes the MS-DOS based

application that runs diagnostic tests, updates device firmware, and manages adapter properties.

 Chapter 21, Troubleshooting describes a variety of troubleshooting methods

and resources.

 Appendix A, Adapter LEDS describes the adapter LEDs and their

significance.

Related Materials

For information about downloading documentation from the QLogic Web site, see

“Downloading Updates” on page xxvii.

operating systems.

xxiv 83840-546-00 D

Page 25

Preface

NOTE

CAUTION

WARNING

Documentation Conventions

This guide uses the following documentation conventions:

 provides additional information.

 without an alert symbol indicates the presence of a hazard

that could cause damage to equipment or loss of data.

 with an alert symbol indicates the presence of a hazard that

could cause minor or moderate injury.

 indicates the presence of a hazard that could cause serious

injury or death.

 Text in blue font indicates a hyperlink (jump) to a figure, table, or section in

this guide, and links to Web sites are shown in underlined blue example:

. For

 Table 9-2 lists problems related to the user interface and remote agent.  See “Installation Checklist” on page 6.  For more information, visit www.qlogic.com

 Text in bold font indicates user interface elements such as a menu items,

buttons, check boxes, or column headings. For example:  Click the Start button, point to Programs, point to Accessories, and

then click Command Prompt.

 Under Notification Options, select the Warning Alarms check box.

 Text in Courier font indicates a file name, directory path, or command line

text. For example:  To return to the root directory from anywhere in the file structure:

Type

cd /root and press ENTER.

 Enter the following command: sh ./install.bin

 Key names and key strokes are indicated with UPPERCASE:

 Press CTRL+P.  Press the UP ARROW key.

xxv 83840-546-00 D

Page 26

Preface License Agreements

 Text in italics indicates terms, emphasis, variables, or document titles. For

example:

 For a complete listing of license agreements, refer to the QLogic

Software End User License Agreement.

 What are shortcut keys?  To enter the date type mm/dd/yyyy (where mm is the month, dd is the

day, and yyyy is the year).

 Topic titles between quotation marks identify related topics either within this

manual or in the online help, which is also referred to as the help system throughout this document.

License Agreements

Refer to the QLogic Software End User License Agreement for a complete listing of all license agreements affecting this product.

xxvi 83840-546-00 D

Page 27

Preface Technical Support

Technical Support

Customers should contact their authorized maintenance provider for technical support of their QLogic products. QLogic-direct customers may contact QLogic Technical Support; others will be redirected to their authorized maintenance provider. Visit the QLogic support Web site listed in Cont act Information for the latest firmware and software updates.

For details about available service plans, or for information about renewing and extending your service, visit the Service Program Web page at

http://www.qlogic.com/Support/Pages/ServicePrograms.aspx

Downloading Updates

The QLogic Web site provides periodic updates to product firmware, software, and documentation.

To download firmware, software, and documentation:

1. Go to the QLogic Downloads and Documentation page:

driverdownloads.qlogic.com

2. Type the QLogic model name in the search box.

3. In the search results list, locate and select the firmware, software, or documentation for your product.

4. View the product details Web page to ensure that you have the correct firmware, software, or documentation. For additional information, click Read Me and Release Notes under Support Files.

5. Click Download Now.

6. Save the file to your computer.

7. If you have downloaded firmware, software, drivers, or boot code, follow the installation instructions in the Readme file.

Instead of typing a model name in the search box, you can perform a guided search as follows:

1. Click the product type tab: Adapters, Switches, Routers, or ASICs.

2. Click the corresponding button to search by model or operating system.

3. Click an item in each selection column to define the search, and then click Go.

4. Locate the firmware, software, or document you need, and then click the item’s name or icon to download or open the item.

xxvii 83840-546-00 D

Page 28

Preface Technical Support

Training

QLogic Global Training maintains a Web site at www.qlogictraining.com offering online and instructor-led training for all QLogic products. In addition, sales and technical professionals may obtain Associate and Specialist-level certifications to qualify for additional benefits from QLogic.

Contact Information

QLogic Technical Support for products under warranty is available during local standard working hours excluding QLogic Observed Holidays. For customers with extended service, consult your plan for available hours. For Support phone numbers, see the Contact Support link at support.qlogic.com

Support Headquarters

QLogic Web Site Technical Support Web Site Technical Support E-mail Technical Training E-mail

Knowledge Database

The QLogic knowledge database is an extensive collection of QLogic product information that you can search for specific solutions. QLogic is constantly adding to the collection of information in the database to provide answers to your most urgent questions. Access the database from the QLogic Support Center:

support.qlogic.com.

QLogic Corporation 12701 Whitewater Drive Minnetonka, MN 55343 USA

www.qlogic.com support.qlogic.com support@qlogic.com training@qlogic.com

xxviii 83840-546-00 D

Page 29

Preface

CLASS I LASER

Legal Notices

Warranty

For warranty details, please check the QLogic Web site:

http://www.qlogic.com/Support/Pages/Warranty.aspx

Laser Safety

FDA Notice

This product complies with DHHS Rules 21CFR Chapter I, Subchapter J. This product has been designed and manufactured according to IEC60825-1 on the safety label of laser product.

Class 1 Laser Product Appareil laser de classe 1 Produkt der Laser Klasse 1 Luokan 1 Laserlaite

Agency Certification

The following sections contain a summary of EMC and EMI test specifications performed on the QLogic adapters to comply with emission and product safety standards.

EMI and EMC Requirements

FCC Rules,CFR Title 47, Part 15, Subpart B:2013 Class A

This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation.

Caution—Class 1 laser radiation when open. Do not view directly with optical instruments

Attention—Radiation laser de classe 1. Ne pas regarder directement avec des instruments optiques.

Vorsicht—Laserstrahlung der Klasse 1 bei geöffneter Abdeckung. Direktes Ansehen mit optischen Instrumenten vermeiden.

Varoitus—Luokan 1 lasersäteilyä, kun laite on auki. Älä katso suoraan laitteeseen käyttämällä optisia instrumenttej.

Industry Canada, ICES-003:2012 Class A

This Class A digital apparatus complies with Canadian ICES-003.Cet appareil numériqué de la classe A est conformé à la norme NMB-003 du Canada.

xxix 83840-546-00 D

Page 30

Preface Legal Notices

CE Mark 2004/108/EC EMC Directive Compliance

EN55022:2010 Class A1:2007/CISPR22:2009+A1:2010 Class A EN55024:2010

EN61000-3-2:2006 A1 +A2:2009: Harmonic Current Emission EN61000-3-3:2008: Voltage Fluctuation and Flicker

VCCI

VCCI:2012-04; Class A

AS/NZS CISPR22

AS/NZS; CISPR 22:2009+A1:2010 Class A

KCC

KC-RRA KN22 KN24(2013) Class A

Product Safety Compliance

UL, cUL product safety:

UL60950-1 (2nd Edition), 2007 UL CSA C22.2 60950-1-07 (2nd Edition) 2007

Use only with listed ITE or equivalent. Complies with 21 CFR 1040.10 and 1040.11. 2006/95/EC low voltage directive: TUV EN60950-1:2006+A11+A1+A12 2nd edition

TUV IEC60950-1:2006 2nd Edition Am 1:2009 CB

xxx 83840-546-00 D

Page 31

1 Product Overview

NOTE

 Functional Description  Features  Supported Operating Environments  Adapter Specifications

Functional Description

The QLogic 8400/3400 Series adapters are based on a new class of Gigabit Ethernet (GbE) and 10GbE converged network interface controller (C-NIC) that can simultaneously perform accelerated data networking and storage networking on a standard Ethernet network. The C-NIC offers acceleration for popular protocols used in the data center, such as:

 Internet Small Computer Systems Interface (iSCSI) offload for accelerating

network storage access featuring centralized boot (iSCSI boot)

 Fibre Channel over Ethernet (FCoE) offload and acceleration for Fibre

Channel block storage

Not all adapters support each listed protocol. For information about supported protocols, refer to the product data sheet at www.qlogic.com under Resources. Separate licenses are required for all offloading technologies.

Enterprise networks that use multiple protocols and multiple network fabrics benefit from the network adapter’s ability to combine data communications, storage, and clustering over a single Ethernet fabric by boosting server CPU processing performance and memory use while alleviating I/O bottlenecks.

The QLogic 8400/3400 Series adapters include a 100/1000Mbps or 10Gbps Ethernet MAC with both half-duplex and full-duplex capability and a 100/1000Mbps or 10Gbps PHY. The transceiver is fully compatible with the IEEE

802.3 standard for auto-negotiation of speed.

1 83840-546-00 D

Page 32

1–Product Overview Features

Using the QLogic teaming software, you can split your network into virtual LANs (VLANs) and group multiple network adapters together into teams to provide network load balancing and fault tolerance. See Chapter 18 and Chapter 17 for detailed information about teaming. See Chapter 13 for a description of VLANs. See “Configuring Teaming” on page 129 for instructions on configuring teaming and creating VLANs on Windows operating systems.

Features

The following is a list of the QLogic 8400/3400 Series adapters features. Some features may not be available on all adapters.

 iSCSI offload  FCoE offload  NIC partitioning (NPAR)  Data center bridging (DCB)

 Enhanced transmission selection (ETS; IEEE 802.1Qaz)  Priority-based flow control (PFC; IEEE 802.1Qbb)  Data center bridging capability exchange protocol (DCBX; CEE

version 1.01)

 Single-chip solution (excluding QLE3442-RJ)  10/100/1000G triple-speed MAC (QLE3442-RJ)  1G/10G triple-speed MAC  SerDes interface for optical transceiver connection

 PCI Express

Gen3 x8 (10GE)

 Zero copy capable hardware  Other offload performance features

 TCP, IP, user datagram protocol (UDP) checksum  TCP segmentation  Adaptive interrupts  Receive side scaling (RSS)

2 83840-546-00 D

Page 33

1–Product Overview Features

 Manageability

 Advanced network features

 QLogic Control Suite diagnostic and configuration software suite. See

the QLogic Control Suite GUI Installation Guide, QLogic Control Suite GUI online help and the QLogic Control Suite Command Line Interface User’s Guide for more information.

 Supports the pre-execution environment (PXE) 1.0 and 2.0

specifications

 Universal management port (UMP)  System management bus (SMBus) controller  Advanced configuration and power interface (ACPI) 1.1a compliant

(multiple power modes)

 Intelligent platform management interface (IPMI) support

 Jumbo frames (up to 9,600 bytes). The OS and the link partner must

support jumbo frames.

 Virtual LANs  IEEE Std 802.3ad Teaming  Smart Load Balancing™ (SLB) teaming  Flow control (IEEE Std 802.3x)  LiveLink™ (supported in both the 32-bit and 64-bit Windows operating

systems)

 Logical link control (IEEE Std 802.2)  High-speed on-chip reduced instruction set computer (RISC) processor  Integrated 96KB frame buffer memory  Quality of service (QoS)  Serial gigabit media independent interface (SGMII)/

Gigabit media independent interface (GMII)/ Media independent interface (MII)

 256 unique MAC unicast addresses  Support for multicast addresses through the 128 bits hashing hardware

function

 Serial flash NVRAM memory  JTAG support  PCI power management interface (v1.1)

3 83840-546-00 D

Page 34

1–Product Overview Features

 64-bit base address register (BAR) support  EM64T processor support  iSCSI and FCoE boot support  Virtualization

iSCSI

 Microsoft  VMware  Linux  XenServer

 Single root I/O virtualization (SR-IOV)

The Internet engineering task force (IETF) has standardized iSCSI. SCSI is a popular protocol that enables systems to communicate with storage devices, using block-level transfer (address data stored on a storage device that is not a whole file). iSCSI maps the SCSI request/response application protocols and its standardized command set over TCP/IP networks.

As iSCSI uses TCP as its sole transport protocol, it greatly benefits from hardware acceleration of the TCP processing. However, iSCSI as a layer 5 protocol has additional mechanisms beyond the TCP layer. iSCSI processing can also be offloaded, thereby reducing CPU use even further.

The QLogic 8400/3400 Series adapters target best-system performance, maintains system flexibility to changes, and supports current and future OS convergence and integration. Therefore, the adapter's iSCSI offload architecture is unique because of the split between hardware and host processing.

FCoE

FCoE allows Fibre Channel protocol to be transferred over Ethernet. FCoE preserves existing Fibre Channel infrastructure and capital investments. The following FCoE features are supported:

 Full stateful hardware FCoE offload  Receiver classification of FCoE and Fibre Channel initialization protocol

(FIP) frames. FIP is the FCoE initialization protocol used to establish and maintain connections.

 Receiver CRC offload  Transmitter CRC offload  Dedicated queue set for Fibre Channel traffic  DCB provides lossless behavior with PFC

4 83840-546-00 D

Page 35

1–Product Overview Supported Operating Environments

 DCB allocates a share of link bandwidth to FCoE traffic with ETS

Power Management

Wake on LAN (WOL) is not supported.

Adaptive Interrupt Frequency

The adapter driver intelligently adjusts host interrupt frequency based on traffic conditions to increase overall application throughput. When traffic is light, the adapter driver interrupts the host for each received packet, minimizing latency. When traffic is heavy, the adapter issues one host interrupt for multiple, back-to-back incoming packets, preserving host CPU cycles.

ASIC with Embedded RISC Processor

The core control for QLogic 8400/3400 Series adapters resides in a tightly integrated, high-performance ASIC. The ASIC includes a RISC processor that provides the flexibility to add new features to the card and a dapt to future network requirements through software downloads. In addition, the adapter drivers can exploit the built-in host offload functions on the adapter as host operating systems are enhanced to take advantage of these functions.

QLogic Control Suite

QLogic Control Suite (QCS) is an integrated utility that provides useful information about each network adapter that is installed in your system. The QCS utility also enables you to perform detailed tests, diag nostics, and analyses on each a dapter. You can also modify property values and view traffic statistics for each adapter.

Supported Operating Environments

The QLogic 8400/3400 Series adapters support several operating systems including Windows, Linux (RHEL Server

systems and versions, go to driverdownloads.qlogic.com adapter type, model, or operating system.

, and Citrix® XenServer. For a complete list of supported operating

, SUSE®, Ubuntu®, CentOSSM)1, VMware ESXi

Adapter Specifications

Physical Characteristics

The QLogic 8400/3400 Series Adapters are implemented as low-profile PCIe cards. The adapters ship with a full-height bracket for use in a standard PCIe slot or an optional spare low-profile bracket for use in a low-profile PCIe slot. Low-profile slots are typically found in compact servers.

and search for your

Ubuntu and CentOS operating systems are supported only on 3400 Series adapters.

5 83840-546-00 D

Page 36

1–Product Overview Adapter Specifications

Standards Specifications

 IEEE 802.3ae (10Gb Ethernet)  IEEE 802.1q (VLAN)  IEEE 802.3ad (Link Aggregation)  IEEE 802.3x (Flow Control)  IPv4 (RFC 791)  IPv6 (RFC 2460)  IEEE 802.1Qbb (Priority-based Flow Control)  IEEE 802.1Qaz (data center bridging exchange (DCBX) and enhanced

transmission selection [ETS])

 IEEE 802.3an 10GBASE-T  IEEE 802.3ab 1000BASE-T  IEEE 802.3u 100BASE-TX

3400 Series Adapters only

6 83840-546-00 D

Page 37

2 Installing the Hardware

 System Requirements  Safety Precautions  Preinstallation Checklist  Installation of the Network Adapter

System Requirements

Before you install a QLogic 8400/3400 Series adapter, verify that your system meets the following hardware and operating system requirements:

Hardware Requirements

 IA32- or EMT64-based computer that meets operating system requirements  One open PCI Express slot. Depending on the PCI Express support on your

adapter, the slot may be of type

 PCI Express 1.0a x1  PCI Express 1.0a x4  PCI Express Gen2 x8  PCI Express Gen3 x8

Full dual-port 10Gbps bandwidth is supported on PCI Express Gen2 x8 or faster slots.

 128MB RAM (minimum)

Operating System Requirements

For a complete list of supported operating systems and versions, go to

driverdownloads.qlogic.com

system.

and search for your adapter type, model, or operating

7 83840-546-00 D

Page 38

2–Installing the Hardware

WARNING

NOTE

Safety Precautions

The adapter is being installed in a system that operates with voltages that can be lethal. Before you open the case of your system, observe the following precautions to protect yourself and to prevent damage to the system components.

 Remove any metallic objects or jewelry from your hands and wrists.  Make sure to use only insulated or nonconducting tools.  Verify that the system is powered OFF and is unplugged before you

touch internal components.

 Install or remove adapters in a static-free environment. The use of a

properly grounded wrist strap or other personal antistatic devices and an antistatic mat is strongly recommended.

Preinstallation Checklist

1. Verify that your system meets the hardware and software requirements listed under System Requirements.

2. Verify that your system is using the latest BIOS.

If you acquired the adapter software on a disk or from the QLogic Web Site driverdownloads.qlogic.com files.

1. If your system is active, shut it down.

2. When system shutdown is complete, turn off the power and unplug the power cord.

3. Remove the adapter from its shipping package and place it on an antistatic surface.

4. Check the adapter for visible signs of damage, particularly on the edge connector. Never attempt to install a damaged adapter.

), verify the path to the adapter driver

8 83840-546-00 D

Page 39

2–Installing the Hardware

CAUTION

Installation of the Network Adapter

The following instructions apply to installing the QLogic 8400/3400 Series adapters in most systems. Refer to the manuals that were supplied with your system for details about performing these tasks on your particular system.

1. Review “Safety Precautions” on page 8 and “Preinstallation Checklist” on

page 8. Before you install the adapter, ensure that the system power is OFF,

the power cord is unplugged from the power outlet, and that you are following proper electrical grounding procedures.

2. Open the system case and select the slot based on the adapter, which may be of type PCIe

other appropriate slot. A lesser-width adapter can be seated into a greater-width slot (x8 in a x16), but a greater-width adapter cannot be seated into a lesser-width slot (x8 in a x4). If you do not know how to identify a PCI Express slot, refer to your system documentation.

3. Remove the blank cover-plate from the slot that you selected.

1.0a x1, PCIe 1.0a x4, PCIe Gen2 x8, PCIe Gen3 x8, or

4. Align the adapter connector edge with the PCI Express connector slot in the system.

5. Applying even pressure at both corners of the card, push the adapter card into the slot until it is firmly seated. When the adapter is properly seate d, the adapter port connectors are aligned with the slot opening, and the adapter faceplate is flush against the system chassis.

Do not use excessive force when seating the card, as this may damage the system or the adapter. If you have difficulty seating the adapter, remove it, realign it, and try again.

6. Secure the adapter with the adapter clip or screw.

7. Close the system case and disconnect any personal antistatic devices.

9 83840-546-00 D

Page 40

2–Installing the Hardware

NOTE

Connecting the Network Cables

The QLogic 8400/3400 Series adapters have either an RJ-45 connector used for attaching the system to an Ethernet copper-wire segment, or a fiber optic connector for attaching the system to an Ethernet fiber optic segment.

The QLogic 3442-RJ adapter supports Automatic MDI Crossover (MDIX), which eliminates the need for crossover cables when connecting machines back-to-back. A straight-through Category 5/5e/6/6A/7 cable allows the machines to communicate when connected directly together.

1. Select an appropriate cable. Table 2-1 lists the copper cable requirements for connecting to 100/1000BASE-T and 10GBASE-T ports. For a list of tested 10GbE optics for the 3400-CU adapter, see Table 22-3.

Table 2-1. 100/1000BASE-T and 10GBASE-T Cable Specifications

Port Type Connector Media

100/1000BASE-T 10GBASE-T RJ-45

1000BASE-T signaling requires four twisted pairs of Category 5 balanced cabling, as specified in

ISO/IEC 11801:2002 and ANSI/EIA/TIA-568-B.

Category 5 is the minimum requirement. Categories 5e, 6, 6a, and 7 are fully supported.

10GBASE-T signaling requires four twisted pairs of Category 6 or Category 6A (augmented

Category 6) balanced cabling, as specified in ISO/IEC 11801:2002 and ANSI/TIA/EIA-568-B.

RJ-45

Category 5 b UTP Category 6

Category 6A/7

UTP

Maximum

Distance

100m (328 ft) 40m (131 ft)

100m (328 ft)

2. Connect one end of the cable to the RJ-45 connector on the adapter.

3. Connect the other end of the cable to an RJ-45 Ethernet network port.

The 8400/3400 Series Adapters also support direct attach copper cables.

10 83840-546-00 D

Page 41

3 Multi-boot Agent (MBA)

Driver Software

 Overview  Setting Up MBA in a Client Environment  Setting Up MBA in a Server Environment

Overview

QLogic 8400/3400 Series adapters support Preboot Execution Environment (PXE), Remote Program Load (RPL), iSCSI, and Bootstrap Protocol (BootP). Multi-Boot Agent (MBA) is a software module that allows your network computer to boot with the images provided by remote servers across the network. The QLogic MBA driver complies with the PXE 2.1 specification and is released with split binary images. This provides flexibility to users in different environments where the motherboard may or may not have built-in base code.

The MBA module operates in a client/server environment. A network consists of one or more boot servers that provide boot images to multiple computers through the network. The QLogic implementation of the MBA module has been tested successfully in the following environments:

 Linux Red Hat

and use network resources (NFS mount, and so forth) and to perform Linux installations. In the case of a remote boot, the Linux universal driver binds seamlessly with the QLogic Universal Network Driver Interface (UNDI) and provides a network interface in the Linux remotely-booted client environment.

 Intel

 MS-DOS UNDI. The MS-DOS UNDI seamlessly binds with the QLogic UNDI

APITEST. The QLogic PXE driver passes all API compliance test

suites.

to provide a network adapter driver interface specification (NDIS2) interface to the upper layer protocol stack. This allows computers to connect to network resources in an MS-DOS environment.

PXE Server. QLogic PXE clients are able to remotely boot

11 83840-546-00 D

Page 42

3–Multi-boot Agent (MBA) Driver Software

NOTE

Setting Up MBA in a Client Environment

Setting up MBA in a client environment involves the following steps:

1. Enabling the MBA driver.

2. Configuring the MBA driver.

3. Setting up the BIOS for the boot order.

Enabling the MBA Driver

To enable or disable the MBA driver:

1. Insert an MS-DOS 6.22 or a Real Mode Kernel bootable disk containing the uxdiag.exe file (for 10/100/1000-Mbps network adapters) or uediag.exe (for 10Gbps network adapters) in the removable disk drive and power up your system.

The uxdiag.exe (or uediag.exe) file is on the installation CD or in the DOS Utilities package available from driverdownloads.qlogic.com/

2. Type:

uxdiag -mba [ 0-disable | 1-enable ] -c devnum (or uediag -mba [ 0-disable | 1-enable ] -c devnum)

where devnum is the specific device(s) number (0,1,2, …) to be programmed.

Configuring the MBA Driver

This section describes the configuration of the MBA driver on add-in NIC models of the QLogic network adapter using the Comprehensive Configuration Management (CCM) utility. To configure the MBA driver on LOM models of the QLogic network adapter, check your system documentation. Both the MBA driver and the CCM utility reside on the adapter Flash memory.

You can use the CCM utility to configure the MBA driver one adapter at a time as described in this section. To simultaneously configure the MBA driver for multiple adapters, use the MS-DOS-based user diagnostics application described in

“Performing Diagnostics” on page 341. For more information about the CCM

utility, see the Comprehensive Configuration Management User’s Guide.

1. Restart your system.

2. Press CTRL+S within four seconds after you are prompted to do so. A list of adapters displays.

12 83840-546-00 D

Page 43

3–Multi-boot Agent (MBA) Driver Software

NOTE

Setting Up MBA in a Client Environment

a. Select the adapter to configure and press ENTER. The Main Menu

displays.

b. Select MBA Configuration to display the MBA Configuration menu

(Figure 3-1).

Figure 3-1. MBA Configuration Menu

3. Use the up arrow and down arrow keys to move to the Boot Protocol menu item. Then use the right arrow or left arrow key to select the boot protocol of choice if other boot protocols besides PXE are available. If available, other boot protocols include Remote Program Load (RPL), iSCSI, and BOOTP.

 For iSCSI boot-capable LOMs, the boot protocol is set through the

BIOS. See your system documentation for more information.

 If you have multiple adapters in your system and you are unsure

which adapter you are configuring, press CTRL+F6, which causes the port LEDs on the adapter to start blinking.

4. Use the UP ARROW, DOWN ARROW , LEFT ARROW , and RIGHT ARROW keys to move to and change the values for other menu items, as desired.

5. Press F4 to save your settings.

6. Press ESC when you are finished.

13 83840-546-00 D

Page 44

3–Multi-boot Agent (MBA) Driver Software Setting Up MBA in a Server Environment

Setting Up the BIOS

To boot from the network with the MBA, make the MBA enabled adapter the first bootable device under the BIOS. This procedure depends on the system BIOS implementation. Refer to the user manual for the system for instructions.

Setting Up MBA in a Server Environment

Red Hat Linux PXE Server

The Red Hat Enterprise Linux distribution has PXE Server support. It allows users to remotely perform a complete Linux installation over the network. The distribution comes with the boot images boot kernel (vmlinuz) and initial ram disk (initrd), which are located on the Red Hat disk#1:

/images/pxeboot/vmlinuz /images/pxeboot/initrd.img

Refer to the Red Hat documentation for instructions on how to install PXE Server on Linux.

The Initrd.img file distributed with Red Hat Enterprise Linux, however, does not have a Linux network driver for the QLogic 8400/3400 Series adapters. This version requires a driver disk for drivers that are not part of the standard distribution. You can create a driver disk for the QLogic 8400/3400 Series adapters from the image distributed with the installation CD. Refer to the Linux Readme.txt file for more information.

MS-DOS UNDI/Intel APITEST

To boot in MS-DOS mode and connect to a network for the MS-DOS environment, download the Intel PXE PDK from the Intel website. This PXE PDK comes with a TFTP/ProxyDHCP/Boot server. The PXE PDK can be downloaded from Intel at

http://downloadcenter.intel.com/SearchResult.aspx?lang=eng&ProductFamily=N etwork+Connectivity&ProductLine=Boot+Agent+Software&ProductProduct=Intel %c2%ae+Boot+Agent.

14 83840-546-00 D

Page 45

4 Windows Driver Software

NOTE

 Installing the Driver Software  Removing the Device Drivers  Installing QLogic Management Applications  Viewing or Changing the Adapter Properties  Setting Power Management Options

Installing the Driver Software

These instructions assume that your QLogic 8400/3400 Series adapter was not factory installed. If your controller was installed at the factory, the driver software has been installed for you.

When Windows first starts after a hardware device has been installed (such as a QLogic 8400/3400 Series adapter), or after the existing device driver has been removed, the operating system automatically detects the hardware and prompts you to install the driver software for that device.

Both a graphical interactive installation mode (see “Using the Installer” on

page 16) and a command-line silent mode for unattended installation (see “Using Silent Installation” on page 17) are available.

 Before installing the driver software, verify that the Windows operating

system has been upgraded to the latest version with the latest service pack applied.

 A network device driver must be physically installed before the QLogic

8400/3400 Series adapter can be used with your Windows operating system. Drivers are located on the installation CD.

 QCS is not supported on the Server Core installation option for Microsoft

Windows Server 2008 R2.

15 83840-546-00 D

Page 46

4–Windows Driver Software

NOTE

Installing the Driver Software

Using the Installer

If supported and if you will use the QLogic iSCSI Crash Dump utility , it is important to follow the installation sequence:

 Run the installer  Install the Microsoft iSCSI Software Initiator along with the patch

To install the QLogic 8400/3400 Series drivers

1. When the Found New Hardware Wizard appears, click Cancel.

2. Insert the installation CD into the CD or DVD drive.

3. On the installation CD, open the folder for your operating system, open the DrvInst folder, and then double-click Setup.exe to open the InstallShield Wizard.

4. Click Next to continue.

5. After you review the license agreement, click I accept the terms in the license agreement and then click Next to continue.

6. Click Install.

7. Click Finish to close the wizard.

8. The installer will determine if a system restart is necessary. Follow the on-screen instructions.

To install the Microsoft iSCSI Software Initiator for iSCSI Crash Dump

If supported and if you will use the QLogic iSCSI Crash Dump utility , it is important to follow the installation sequence:

 Run the installer  Install Microsoft iSCSI Software Initiator along with the patch (MS

KB939875)

If performing an upgrade of the device drivers from the installer, re-enable iSCSI Crash Dump from the Advanced section of the QCS Configuration tab.

Perform this procedure after running the installer to install the device drivers.

1. Install Microsoft iSCSI Software Initiator (version 2.06 or later) if not included in your OS. To determine when you need to install the Microsoft iSCSI Software Initiator, see Table 4-1. To download the iSCSI Software Initiator from Microsoft, go to

16 83840-546-00 D

Page 47

4–Windows Driver Software

NOTE

Installing the Driver Software

http://www.microsoft.com/downloads/en/details.aspx?familyid=12cb3c1a-15 d6-4585-b385-befd1319f825&displaylang=en.

2. Install Microsoft patch for iSCSI crash dump file generation (Microsoft KB939875) from http://support.microsoft.com/kb/939875. you need to install the Microsoft patch, see Table 4-1.

Table 4-1. Windows Operating Systems and iSCSI Crash Dump

To determine if

Operating System

Windows Server 2008 Yes (included in OS) No Windows Server 2008 R2 Yes (included in OS) No

Windows Server 2008 No No Windows Server 2008 R2 No No

Using Silent Installation

 All commands are case sensitive.  For detailed instructions and information about unattended installs, refer

to the silent.txt file in the folder.

MS iSCSI Software

Initiator Required

NDIS

OIS

Microsoft Patch (MS

KB939875) Required

To perform a silent install from within the installer source folder

Type the following:

setup /s /v/qn

To perform a silent upgrade from within the installer source folder

Type the following:

setup /s /v/qn

17 83840-546-00 D

Page 48

4–Windows Driver Software

NOTE

Removing the Device Drivers

To perform a silent reinstall of the same installer

Type the following:

setup /s /v"/qn REINSTALL=ALL"

The REINSTALL switch should only be used if the same installer is already installed on the system. If upgrading an earlier version of the installer, use setup /s /v/qn as listed above.

To perform a silent install to force a downgrade (default is NO)

setup /s /v” /qn DOWNGRADE=Y”

Removing the Device Drivers

Uninstall the QLogic 8400/3400 Series device drivers from your system only through the InstallShield wizard. Uninstalling the device drivers with Device Manager or any other means may not provide a clean uninstall and may cause the system to become unstable.

Windows Server 2008 and Windows Server 2008 R2 provide the Device Driver Rollback feature to replace a device driver with one that was previously installed. However, the complex software architecture of the 8400/3400 Series device may present problems if the rollback feature is used on one of the individual components. Therefore, we recommend that changes to driver versions be made only through the use of a driver installer .

To remove the device drivers, in Control Panel, double-click Add or Remove

Programs.

Installing QLogic Management Applications

1. Execute the setup file (setup.exe) to open the QLogic Management Programs installation wizard.

2. Accept the terms of the license agreement, and then click Next.

3. In the Custom Setup dialog box, review the components to be installed, make any necessary changes, and then click Next.

4. In the Ready to Install the Program dialog box, click Install to proceed with the installation.

18 83840-546-00 D

Page 49

4–Windows Driver Software Viewing or Changing the Adapter Properties

Viewing or Changing the Adapter Properties

To view or change the properties of the QLogic network adapter

1. In Control Panel, click QLogic Control Suite 4.

2. Click the Advanced section of the Configurations tab.

19 83840-546-00 D

Page 50

4–Windows Driver Software

NOTE

Setting Power Management Options

You can set power management options to allow the operating system to turn off the controller to save power. If the device is busy doing something (servicing a call, for example) however, the operating system will not shut down the device. The operating system attempts to shut down every possible device only when the computer attempts to go into hibernation. To have the controller stay on at all times, do not click the Allow the computer to turn off the device to save power check box (Figure 4-1).

Figure 4-1. Power Management

 The Power Management tab is available only for servers that support

power management.

 If you select Only allow management stations to bring the computer

out of standby, the computer can be brought out of standby only by

Magic Packet.

20 83840-546-00 D

Page 51

4–Windows Driver Software

CAUTION

Setting Power Management Options

Do not select Allow the computer to turn off the device to save power for any adapter that is a member of a team.

21 83840-546-00 D

Page 52

5 Linux Driver Software

 Introduction  Limitations  Packaging  Installing Linux Driver Software  Unloading/Removing the Linux Driver  Patching PCI Files (Optional)  Network Installations  Setting Values for Optional Properties  Driver Defaults  Driver Messages  Teaming with Channel Bonding  Statistics

22 83840-546-00 D

Page 53

5–Linux Driver Software Introduction

Introduction

This section discusses the Linux drivers for the QLogic 8400/3400 Series ne twork adapters. Table 5-1 lists the 8400/3400 Series Linux drivers. For information about iSCSI offload in Linux server, see “iSCSI Offload in Linux Server” on

page 208.

Linux Driver Description

bnx2x Linux driver for the 8400/3400 Series 10Gb network adapt-

Table 5-1. QLogic 8400/3400 Series Linux Drivers

ers. This driver directly controls the hardware and is responsible for sending and receiving Ethernet packe t s on behalf o f the Linux host networking stack. This driver also receives and processes device interrupts, both on behalf of itself (for L2 networking) and on behalf of the bnx2fc (FCoE) and cnic drivers.

cnic The cnic driver provides the interface between QLogic’s

bnx2i Linux iSCSI HBA driver to enable iSCSI offload on the

bnx2fc Linux FCoE kernel mode driver used to provide a translation

Limitations

 bnx2x Driver  bnx2i Driver  bnx2fc Driver

upper layer protocol (storage) drivers and QLogic’s 8400/3400 Series 10Gb network adapters. The CNIC module works with the bnx2 and bnx2x network drives in the downstream and the bnx2fc (FCoE) and bnx2i (iSCSI) drivers in the upstream.

8400/3400 Series 10Gb network adapters.

layer between the Linux SCSI stack and the QLogic FCoE firmware/hardware. In addition, the driver interfaces with the networking layer to transmit and receive encap sulated FCoE frames on behalf of open-fcoe’s libfc/libfcoe for FIP/device discovery.

23 83840-546-00 D

Page 54

5–Linux Driver Software Packaging

bnx2x Driver

The current version of the driver has been tested on 2.6.x kernels starting from

2.6.9. The driver may not compile on kernels older than 2.6.9. Testing is

concentrated on i386 and x86_64 architectures. Only limited testing has been done on some other architectures. Minor changes to some source files and Makefile may be needed on some kernels.

bnx2i Driver

The current version of the driver has been tested on 2.6.x kernels, starting from

2.6.18 kernel. The driver may not compile on older kernels. Testing is

concentrated on i386 and x86_64 architectures, RHEL 5, RHEL 6, RHEL 7, and SUSE 11 SP1 and later distributions.

bnx2fc Driver

The current version of the driver has been tested on 2.6.x kernels, starting from

2.6.32 kernel, which is included in RHEL 6.1 distribution. This driver may not

compile on older kernels. Testing was limited to i386 and x86_64 architectures, RHEL 6.1, RHEL 7.0, and SLES

11 SP1 and later distributions.

Packaging

The Linux drivers are released in the following packaging formats:

 DKMS Packages

 KMP Packages

 netxtreme2-version.dkms.noarch.rpm  netxtreme2-version.dkms.src.rpm

 SLES

 netxtreme2-kmp-[kernel]-version.i586.rpm  netxtreme2-kmp-[kernel]-version.x86_64.rpm

 Red Hat

 kmod-kmp-netxtreme2-{kernel]-version.i686.rpm  kmod-kmp-netxtreme2-{kernel]-version.x86_64.rpm

The QLogic Control Suite CLI management utility is also distributed as an RPM package (QCS-{version}.{arch}.rpm). See “Installing on a Linux

System” on page 85 for information on installing Linux QCS.

24 83840-546-00 D

Page 55

5–Linux Driver Software Packaging

 Source Packages

Identical source files to build the driver are included in both RPM and TAR source packages. The supplemental tar file contains additional utilities such as patches and driver diskette images for network installation.

The following is a list of included files:  netxtreme2-version.src.rpm: RPM package with 8400/3400

Series bnx2/bnx2x/cnic/bnx2fc/bnx2ilibfc/libfcoe driver source.

 netxtreme2-version.tar.gz: tar zipped p ackage with 8400/3400

Series bnx2/bnx2x/cnic/bnx2fc/bnx2i/libfc/libfcoe driver source.

 iscsiuio-version.tar.gz: iSCSI user space management tool

binary.

 open-fcoe-*.qlgc.<subvert>.<arch>.rpm: open-fcoe

userspace management tool binary RPM for SLES11 SP2 and legacy versions.

 fcoe-utils-*.qlgc.<subver>.<arch>.rpm: open-fcoe

userspace management tool binary RPM for RHEL 6.4 and legacy versions.

The Linux driver has a dependency on open-fcoe userspace management tools as the front-end to control FCoE interfaces. The package name of the open-fcoe tool is fcoe-utils for RHEL 6.4 and open-fcoe for SLES11 SP2 and legacy versions.

25 83840-546-00 D

Page 56

5–Linux Driver Software

NOTE

Installing Linux Driver Software

 Installing the Source RPM Package  Building the Driver from the Source TAR File

If a bnx2x, bnx2i, or bnx2fc driver is loaded and the Linux kernel is updated, the driver module must be recompiled if the driver module was installed using the source RPM or the TAR package.

Installing the Source RPM Package

The following are guidelines for installing the driver source RPM Package. Prerequisites:

 Linux kernel source  C compiler

Procedure:

1. Install the source RPM package:

rpm -ivh netxtreme2-<version>.src.rpm

2. Change the directory to the RPM path and build the binary RPM for your kernel:

For RHEL:

cd ~/rpmbuild rpmbuild -bb SPECS/netxtreme2.spec

For SLES:

cd /usr/src/packages rpmbuild -bb SPECS/netxtreme2.spec

3. Install the newly compiled RPM:

rpm -ivh RPMS/<arch>/netxtreme2-<version>.<arch>.rpm

Note that the --force option may be needed on some Linux distributions if conflicts are reported.

4. For FCoE offload, install the open-fcoe utility.

26 83840-546-00 D

Page 57

5–Linux Driver Software

NOTE

Installing Linux Driver Software

For RHEL 6.4 and legacy versions, either of the following:

yum install fcoe-utils-<version>.rhel.64.qlgc.<subver>.<arch>.rpm

-or-

rpm -ivh fcoe-utils-<version>.rhel.64.qlgc.<subver>.<arch>.rpm

For SLES11 SP2:

rpm -ivh open-fcoe-<version>.sles.sp2.qlgc.<subver>.<arch>.rpm

For RHEL 6.4 and SLES11 SP2 and legacy versions, the version of fcoe-utils/open-fcoe included in your distribution is sufficient and no out of box upgrades are provided.

Where available, installation with yum will automatically resolve dependencies. Otherwise, required dependencies can be located on your O/S installation media.

5. For SLES, turn on the fcoe and lldpad services for FCoE offload, and just lldpad for iSCSI-offload-TLV.

For SLES11 SP1:

chkconfig lldpad on chkconfig fcoe on

For SLES11 SP2:

chkconfig boot.lldpad on chkconfig boot.fcoe on

6. Inbox drivers are included with all of the supported operating systems. The simplest means to ensure the newly installed drivers a re loaded is to reboot.

7. For FCoE offload, after rebooting, create configuration files for all FCoE ethX interfaces:

cd /etc/fcoe cp cfg-ethx cfg-<ethX FCoE interface name>

Note that your distribution might have a different naming scheme for Ethernet devices (pXpX or emX instead of ethX).

27 83840-546-00 D

Page 58

5–Linux Driver Software Installing Linux Driver Software

8. For FCoE offload or iSCSI-offload-TLV, modify /etc/fcoe/cfg-<interface> by setting DCB_REQUIRED=yes to DCB_REQUIRED=no.

9. Turn on all ethX interfaces.

ifconfig <ethX> up

10. For SLES, use Y aST to configure your Ethernet interfaces to automatically start at boot by setting a static IP address or enabling DHCP on the interface.

11. For FCoE offload and iSCSI-offload-TLV, disable lldpad on QLogic converged network adapter interfaces. This is required because QLogic uses an offloaded DCBX client.

lldptool set-lldp –i <ethX> adminStatus=disasbled

12. For FCoE offload and iSCSI-offload-TLV, make sure /var/lib/lldpad/lldpad.conf is created and each <ethX> block does not specify “adminStatus” or if specified, it is set to 0 (“adminStatus=0”) as below.

lldp : { eth5 : { tlvid00000001 : { info = "04BC305B017B73"; }; tlvid00000002 : { info = "03BC305B017B73"; }; };

13. For FCoE offload and iSCSI-offload-TLV, restart lldpad service to apply new settings

For SLES11 SP1, RHEL 6.4 and legacy versions:

service lldpad restart

For SLES11 SP2:

rclldpad restart

28 83840-546-00 D

Page 59

5–Linux Driver Software

NOTE

Installing Linux Driver Software

For SLES12:

systemctl restart lldpad

14. For FCOE offload, restart fcoe service to apply new settings For SLES11 SP1, RHEL 6.4, and legacy versions:

service fcoe restart

For SLES11 SP2:

rcfcoe restart

For SLES12:

systemctl restart fcoe

Installing the KMP Package

The examples in this procedure refer to the bnx2x driver, but also apply to the bxn2fc and bnx2i drivers.

1. Install the KMP package:

rpm -ivh <file> rmmod bnx2x

2. Load the driver:

modprobe bnx2x

Building the Driver from the Source TAR File

The examples used in this procedure refer to the bnx2x driver , but also apply to the bnx2i and bnx2fc drivers.

1. Create a directory and extract the TAR files to the directory:

tar xvzf netxtreme2-<version>.tar.gz

2. Build the driver bnx2x.ko (or bnx2x.o) as a loadable module for the running kernel:

cd netxtreme2-<version> make

29 83840-546-00 D

Page 60

5–Linux Driver Software

NOTE

Load and Run Necessary iSCSI Software Components

3. Test the driver by loading it (first unload the existing driver, if necessary):

rmmod bnx2x (or bnx2fc or bnx2i) insmod bnx2x/src/bnx2x.ko (or bnx2fc/src/bnx2fc.ko, or

bnx2i/src/bnx2i.ko)

4. For iSCSI offload and FCoE offload, load the cnic driver (if applicable):

insmod cnic.ko

5. Install the driver and man page:

make install

See the RPM instructions above for the location of the installed driver.

6. Install the user daemon (qlgc_iscsiuio).

Refer to “Load and Run Necessary iSCSI Software Components” on page 30 for instructions on loading the software components required to use the QLog ic iSCSI offload feature.

To configure the network protocol and address after building the driver, refer to the manuals supplied with your operating system.

Load and Run Necessary iSCSI Software Components

The QLogic iSCSI Offload software suite consists of three kernel modules and a user daemon. Required software components can be loaded either manually or through system services.

1. Unload the existing driver, if necessary: Manual:

rmmod bnx2i

modprobe -r bnx2i

2. Load the iSCSI driver: Manual: insmod bnx2i.ko or modprobe bnx2i

30 83840-546-00 D

Page 61

5–Linux Driver Software

NOTE

Unloading/Removing the Linux Driver

 Unloading/Removing the Driver from an RPM Installation  Removing the Driver from a TAR Installation

Unloading/Removing the Driver from an RPM Installation

 The examples used in this procedure refer to the bnx2x driver, but also

apply to the bnx2fc and bnx2i drivers.

 On 2.6 kernels, it is not necessary to bring down the eth# interfaces

before unloading the driver module.

 If the cnic driver is loaded, unload the cnic driver before unloading the

bnx2x driver.

 Prior to unloading the bnx2i driver, disconnect all active iSCSI sessions

to targets.

To unload the driver, use ifconfig to bring down all eth# interfaces opened by the driver, and then type the following:

rmmod bnx2x

The above command will also remove the cnic module.

If the driver was installed using RPM, do the following to remove it:

rpm -e netxtreme2

Removing the Driver from a TAR Installation

The examples used in this procedure refer to the bnx2x driver , but also apply to the bnx2fc and bnx2i drivers.

If the driver was installed using make install from the tar file, the bnx2x.ko driver file has to be manually deleted from the operating system. See “Installing the

Source RPM Package” on page 26 for the location of the installed driver.

31 83840-546-00 D

Page 62

5–Linux Driver Software

NOTE

Patching PCI Files (Optional)

Uninstalling QCS

RPM Package

Use the following command:

% rpm -e QCS

Patching PCI Files (Optional)

The examples used in this procedure refer to the bnx2x driver , but also apply to the bnx2fc and bnx2i drivers.

For hardware detection utilities, such as Red Hat kudzu, to properly identify bnx2x supported devices, a number of files containing PCI vendor and device information may need to be updated.

Apply the updates by running the scripts provided in the supplemental tar file. For example, on Red Hat Enterprise Linux, apply the updates by doing the following:

./patch_pcitbl.sh /usr/share/hwdata/pcitable pci.updates /usr/share/hwdata/pcitable.new bnx2

./patch_pciids.sh /usr/share/hwdata/pci.ids pci.updates /usr/share/hwdata/pci.ids.new

Next, the old files can be backed up and the new files can be renamed for use.

cp /usr/share/hwdata/pci.ids /usr/share/hwdata/old.pci.ids cp /usr/share/hwdata/pci.ids.new /usr/share/hwdata/pci.ids cp /usr/share/hwdata/pcitable /usr/share/hwdata/old.pcitable cp /usr/share/hwdata/pcitable.new /usr/share/hwdata/pcitable

Network Installations

For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the bnx2x driver may be needed. The driver disk images for the most recent Red Hat and SuSE versions are included. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from the Red Hat website,

http://www.redhat.com

32 83840-546-00 D

Page 63

5–Linux Driver Software Setting Values for Optional Properties

Setting Values for Optional Properties

Optional properties exist for the different drivers:

 bnx2x Driver  bnx2i Driver  bnx2fc Driver

bnx2x Driver

disable_tpa

The disable_tpa parameter can be supplied as a command line argument to disable the Transparent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets. Use disable_tpa to disable the advanced TPA feature.

Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all 8400/3400 Series network adapters in the system. The parameter can also be set in modprobe.conf. See the man page for more information.

int_mode

insmod bnx2x.ko disable_tpa=1

modprobe bnx2x disable_tpa=1

The int_mode parameter is used to force using an interrupt mode. Set the int_mode parameter to 1 to force using the legacy INTx mode on all

8400/3400 Series adapters in the system.

insmod bnx2x.ko int_mode=1

modprobe bnx2x int_mode=1

Set the int_mode parameter to 2 to force using MSI mode on all 8400/3400 Series adapters in the system.

insmod bnx2x.ko int_mode=2

modprobe bnx2x int_mode=2

Set the int_mode parameter to 3 to force using MSI-X mode on all 8400/3400 Series adapters in the system.

33 83840-546-00 D

Page 64

5–Linux Driver Software Setting Values for Optional Properties

dropless_fc

The dropless_fc parameter can be used to enable a comp lementary flow control mechanism on 8400/3400 Series adapters. The default flow control mechanism is to send pause frames when the on-chip buffer (BRB) is reaching a certain level of occupancy. This is a performance targeted flow control mechanism. On 8400/3400 Series adapters, one can enable another flow control mechanism to send pause frames, where one of the host buffers (when in RSS mode) are exhausted.

This is a zero packet drop targeted flow control mechanism. Set the dropless_fc parameter to 1 to enable the dropless flow control

mechanism feature on all 8400/3400 Series adapters in the system.

insmod bnx2x.ko dropless_fc=1

modprobe bnx2x dropless_fc=1

disable_iscsi_ooo

The disable_iscsi_ooo parameter is to disable the allocation of the iSCSI TCP Out-of-Order (OOO) reception resources, specifically for VMware for low-memory systems.

multi_mode

num_queues

The optional parameter multi_mode is for use on systems that support multi-queue networking. Multi-queue networking on the receive side depends only on MSI-X capability of the system, multi-queue networking on the transmit side is supported only on kernels starting from 2.6.27. By default, multi_mode parameter is set to 1. Thus, on kernels up to 2.6.26, the driver will allocate on the receive side one queue per-CPU and on the transmit side only one queue. On kernels starting from 2.6.27, the driver will allocate on both receive and transmit sides, one queue per-CPU. In any case, the number of allocated queues will be limited by number of queues supported by hardware.

The multi_mode optional parameter can also be used to enable SAFC (Service Aware Flow Control) by differentiating the traffic to up to 3 CoS (Class of Service) in the hardware according to the VLAN PRI value or according to the IP DSCP value (least 3 bits).

The optional parameter num_queues may be used to set the number of queues when multi_mode is set to 1 and interrupt mode is MSI-X. If interrupt mode is different than MSI-X (see int_mode), the number of queues will be set to 1, discarding the value of this parameter.

34 83840-546-00 D

Page 65

5–Linux Driver Software Setting Values for Optional Properties

pri_map

The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or same CoS in the hardware. This 32-bit parameter is evaluated by the driver as an 8 value of 4 bits each. Each nibble sets the desired hardware queue number for that priority . For example, set pri_map to 0x11110000 to map priority 0 to 3 to CoS 0 and map priority 4 to 7 to CoS 1.

qs_per_cos

The optional parameter qs_per_cos is used to specify how many queues will share the same CoS. This parameter is evaluated by the driver up to 3 values of 8 bits each. Each byte sets the desired number of queues for that CoS. The total number of queues is limited by the hardware limit. For example, set qs_per_cos to 0x10101 to create a total of three queues, one per CoS. In another example, set qs_per_cos to 0x404 to create a total of 8 queues, divided into 2 CoS, 4 queues in each CoS.

cos_min_rate

The optional parameter cos_min_rate is used to determine the weight of each CoS for round-robin scheduling in transmission. This parameter is evaluated by the driver as up to 3 values of 8 bits each. Each byte sets the desired weight for that CoS. The weight ranges from 0 to 100. For example, set cos_min_rate to 0x101 for fair transmission rate between 2 CoS. In another example, set the cos_min_rate to 0x30201 to give CoS the higher rate of transmission. To avoid using the fairness algorithm, omit setting cos_min_rate or set it to 0.

Set the multi_mode parameter to 2 as shown below to differentiate the traffic according to the VLAN PRI value.

insmod bnx2x.ko multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404

modprobe bnx2x multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404

Set the multi_mode parameter to 4, as shown below, to differentiate the traffic according to the IP DSCP value.

insmod bnx2x.ko multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201

modprobe bnx2x multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201

35 83840-546-00 D

Page 66

5–Linux Driver Software

CAUTION

Setting Values for Optional Properties

bnx2i Driver

Optional parameters en_tcp_dack, error_mask1, and error_mask2 can be supplied as command line arguments to the insmod or modprobe command for bnx2i.

error_mask1 and error_mask2

"Config FW iSCSI Error Mask #", use to configure certain iSCSI protocol violation to be treated either as a warning or a fatal error. All fatal iSCSI protocol violations will result in session recovery (ERL 0). These are bit masks.

Defaults: All violations will be treated as errors.

Do not use error_mask if you are not sure about the consequences. These values are to be discussed with QLogic development team on a case-by-case basis. This is just a mechanism to work around iSCSI implementation issues on the target side. Without proper knowledge of iSCSI protocol details, users are advised not to experiment with these parameters.

en_tcp_dack

time_stamps

"Enable TCP Delayed ACK", enables/disables TCP delayed ACK feature on offloaded iSCSI connections.

Defaults: TCP delayed ACK is ENABLED. For example:

insmod bnx2i.ko en_tcp_dack=0

modprobe bnx2i en_tcp_dack=0

“Enable TCP TimeStamps”, enables/disables TCP time stamp feature on offloaded iSCSI connections.

Defaults: TCP time stamp option is DISABLED. For example:

insmod bnx2i.ko time_stamps=1

modprobe bnx2i time_stamps=1

36 83840-546-00 D

Page 67

5–Linux Driver Software Setting Values for Optional Properties

sq_size

"Configure SQ size", used to choose send queue size for offloaded connections and SQ size determines the maximum SCSI commands that can be queued. SQ size also has a bearing on the number of connections that can be offloaded; as QP size increases, the number of connections supported will decrease. With the default values, the adapter can offload 28 connections.

Defaults: 128 Range: 32 to 128 Note that QLogic validation is limited to a power of 2; for example, 32, 64, 128.

rq_size

“Configure RQ size”, used to choose the size of asynchronous buffer queue size per offloaded connections. RQ size is not requ ired greater than 16 as it is used to place iSCSI ASYNC/NOP/REJECT messages and SCSI sense data.

Defaults: 16 Range: 16 to 32 Note that QLogic validation is limited to a power of 2; for example, 16, 32.

event_coal_div

"Event Coalescing Divide Factor", performance tuning parameter used to moderate the rate of interrupt generation by the iscsi firmware.

Defaults: 2 Valid values: 1, 2, 4, 8

last_active_tcp_port

“Last active TCP port used”, status parameter used to indicate the last TCP port number used in the iSCSI offload connection.

Defaults: N/A Valid values: N/A Note: This is a read-only parameter.

ooo_enable

“Enable TCP out-of-order feature”, enables/disables TCP out-of-order rx handling feature on offloaded iSCSI connections.

Defaults: TCP out-of-order feature is ENABLED. For example:

insmod bnx2i.ko ooo_enable=1

modprobe bnx2i ooo_enable=1

37 83840-546-00 D

Page 68

5–Linux Driver Software Driver Defaults

bnx2fc Driver

Optional parameter debug_logging can be supplied as a command line arguments to the insmod or modprobe command for bnx2fc.

debug_logging

"Bit mask to enable debug logging", enables/disables driver debug logging. Defaults: None. For example:

insmod bnx2fc.ko debug_logging=0xff

modprobe bnx2fc debug_logging=0xff

IO level debugging = 0x1 Session level debugging = 0x2 HBA level debugging = 0x4 ELS debugging = 0x8 Misc debugging = 0x10 Max debugging = 0xff

Driver Defaults

 bnx2 Driver  bnx2x Driver

bnx2 Driver

Speed: Autonegotiation with all speeds advertised Flow Control: Autonegotiation with RX and TX advertised MTU: 1500 (range is 46–9000) RX Ring Size: 255 (range is 0–4080) RX Jumbo Ring Size: 0 (range 0–16320) adjusted by the driver based on MTU

and RX Ring Size TX Ring Size: 255 (range is (MAX_SKB_FRAGS+1)–255). MAX_SKB_FRAGS

varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce RX Microseconds: 18 (range is 0–1023) Coalesce RX Microseconds IRQ: 18 (range is 0–1023) Coalesce RX Frames: 6 (range is 0–255)

38 83840-546-00 D

Page 69

5–Linux Driver Software Driver Defaults

Coalesce RX Frames IRQ: 6 (range is 0–255) Coalesce TX Microseconds: 80 (range is 0–1023) Coalesce TX Microseconds IRQ: 80 (range is 0–1023) Coalesce TX Frames: 20 (range is 0–255) Coalesce TX Frames IRQ: 20 (range is 0–255) Coalesce Statistics Microseconds: 999936 (approximately 1 second) (range is

0–16776960 in increments of 256)

MSI: Enabled (if supported by the 2.6 kernel and the interrupt test passes) TSO: Enabled (on 2.6 kernels)

bnx2x Driver

Speed: Autonegotiation with all speeds advertised Flow control: Autonegotiation with RX and TX advertised MTU: 1500 (range is 46–9600) RX Ring Size: 4078 (range is 0–4078) TX Ring Size: 4078 (range is (MAX_SKB_FRAGS+4)–4078). MAX_SKB_FRAGS

varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce RX Microseconds: 25 (range is 0–3000) Coalesce TX Microseconds: 50 (range is 0–12288) Coalesce Statistics Microseconds: 999936 (approximately 1 second) (range is

0–16776960 in increments of 256)

MSI-X: Enabled (if supported by the 2.6 kernel and the interrupt test passes) TSO: Enabled

39 83840-546-00 D

Page 70

5–Linux Driver Software Driver Messages

Driver Messages

The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -n <level> to control the level at which messages appear on the console. Most systems are set to level 6 by default. To see all messages, set the level higher.

 bnx2x Driver  bnx2i Driver  bnx2fc Driver

bnx2x Driver

Driver Sign On

QLogic 8400/3400 Series 10 Gigabit Ethernet Driver bnx2x v1.6.3c (July 23, 20xx)

CNIC Driver Sign On (bnx2 only)

QLogic 8400/3400 Series cnic v1.1.19 (Sep 25, 20xx)

NIC Detected

eth#: QLogic 8400/3400 Series xGb (B1) PCI-E x8 found at mem f6000000, IRQ 16, node addr 0010180476ae

cnic: Added CNIC device: eth0

Link Up and Speed Indication

bnx2x: eth# NIC Link is Up, 10000 Mbps full duplex

Link Down Indication

bnx2x: eth# NIC Link is Down

MSI-X Enabled Successfully

bnx2x: eth0: using MSI-X

bnx2i Driver

BNX2I Driver Signon

QLogic 8400/3400 Series iSCSI Driver bnx2i v2.1.1D (May 12, 20xx)

40 83840-546-00 D

Page 71

5–Linux Driver Software

NOTE

Driver Messages

Network Port to iSCSI Transport Name Binding

bnx2i: netif=eth2, iscsi=bcm570x-050000 bnx2i: netif=eth1, iscsi=bcm570x-030c00

Driver Completes handshake with iSCSI Offload-enabled CNIC Device

bnx2i [05:00.00]: ISCSI_INIT passed

This message is displayed only when the user attempts to make an iSCSI connection.

Driver Detects iSCSI Offload Is Not Enabled on the CNIC Device

bnx2i: iSCSI not supported, dev=eth3 bnx2i: bnx2i: LOM is not enabled to offload iSCSI connections,

dev=eth0 bnx2i: dev eth0 does not support iSCSI

Exceeds Maximum Allowed iSCSI Connection Offload Limit

bnx2i: alloc_ep: unable to allocate iscsi cid bnx2i: unable to allocate iSCSI context resources

Network Route to Target Node and Transport Name Binding Are Two Different Devices

bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba $USER_CHOSEN_HBA

where:  ROUTE_HBA is the net device on which connection was offloaded based on

route information.

 USER_CHOSEN_HBA is the adapter to which target node is bound (using

iSCSI transport name).

Target Cannot Be Reached on Any of the CNIC Devices

bnx2i: check route, cannot connect using cnic

Network Route Is Assigned to Network Interface, Which Is Down

bnx2i: check route, hba not found

SCSI-ML Initiated Host Reset (Session Recovery)

bnx2i: attempting to reset host, #3

41 83840-546-00 D

Page 72

5–Linux Driver Software Driver Messages

CNIC Detects iSCSI Protocol Violation - Fatal Errors

bnx2i: iscsi_error - wrong StatSN rcvd bnx2i: iscsi_error - hdr digest err bnx2i: iscsi_error - data digest err bnx2i: iscsi_error - wrong opcode rcvd bnx2i: iscsi_error - AHS len > 0 rcvd bnx2i: iscsi_error - invalid ITT rcvd bnx2i: iscsi_error - wrong StatSN rcvd bnx2i: iscsi_error - wrong DataSN rcvd bnx2i: iscsi_error - pend R2T violation bnx2i: iscsi_error - ERL0, UO bnx2i: iscsi_error - ERL0, U1 bnx2i: iscsi_error - ERL0, U2 bnx2i: iscsi_error - ERL0, U3 bnx2i: iscsi_error - ERL0, U4 bnx2i: iscsi_error - ERL0, U5 bnx2i: iscsi_error - ERL0, U bnx2i: iscsi_error - invalid resi len bnx2i: iscsi_error - MRDSL violation bnx2i: iscsi_error - F-bit not set bnx2i: iscsi_error - invalid TTT bnx2i: iscsi_error - invalid DataSN bnx2i: iscsi_error - burst len violation bnx2i: iscsi_error - buf offset violation bnx2i: iscsi_error - invalid LUN field bnx2i: iscsi_error - invalid R2TSN field bnx2i: iscsi_error - invalid cmd len1 bnx2i: iscsi_error - invalid cmd len2 bnx2i: iscsi_error - pend r2t exceeds MaxOutstandingR2T value bnx2i: iscsi_error - TTT is rsvd bnx2i: iscsi_error - MBL violation bnx2i: iscsi_error - data seg len != 0 bnx2i: iscsi_error - reject pdu len error bnx2i: iscsi_error - async pdu len error bnx2i: iscsi_error - nopin pdu len error bnx2i: iscsi_error - pend r2t in cleanup bnx2i: iscsi_error - IP fragments rcvd bnx2i: iscsi_error - IP options error bnx2i: iscsi_error - urgent flag error

42 83840-546-00 D

Page 73

5–Linux Driver Software

NOTE

Driver Messages

CNIC Detects iSCSI Protocol Violation - Non-FATAL, Warning

bnx2i: iscsi_warning - invalid TTT bnx2i: iscsi_warning - invalid DataSN bnx2i: iscsi_warning - invalid LUN field

The driver needs to be configured to consider certain violation to treat as warning and not as a critical error.

Driver Puts a Session Through Recovery

conn_err - hostno 3 conn 03fbcd00, iscsi_cid 2 cid a1800

Reject iSCSI PDU Received from the Target

bnx2i - printing rejected PDU contents [0]: 1 ffffffa1 0 0 0 0 20 0 [8]: 0 7 0 0 0 0 0 0 [10]: 0 0 40 24 0 0 ffffff80 0 [18]: 0 0 3 ffffff88 0 0 3 4b [20]: 2a 0 0 2 ffffffc8 14 0 0 [28]: 40 0 0 0 0 0 0 0

Open-iSCSI Daemon Handing Over Session to Driver

bnx2i: conn update - MBL 0x800 FBL 0x800MRDSL_I 0x800 MRDSL_T 0x2000

bnx2fc Driver

BNX2FC Driver Signon:

QLogic NetXtreme II FCoE Driver bnx2fc v0.8.7 (Mar 25, 2011

Driver Compiles Handshake with FCoE Offload Enabled CNIC Device

bnx2fc [04:00.00]: FCOE_INIT passed

Driver Fails Handshake with FCoE Offload Enabled CNIC Device

bnx2fc: init_failure due to invalid opcode bnx2fc: init_failure due to context allocation failure bnx2fc: init_failure due to NIC error bnx2fc: init_failure due to completion status error bnx2fc: init_failure due to HSI mismatch

43 83840-546-00 D

Page 74

5–Linux Driver Software Driver Messages

No V a lid License to Start FCoE

bnx2fc: FCoE function not enabled <ethX> bnx2fC: FCoE not supported on <ethX>

Session Failures Due to Exceeding Maximum Allowed FCoE Offload Connection Limit or Memory Limits

bnx2fc: Failed to allocate conn id for port_id <remote port id> bnx2fc: exceeded max sessions..logoff this tgt bnx2fc: Failed to allocate resources

Session Offload Failures

bnx2fc: bnx2fc_offload_session - Offload error <rport> not FCP type. not offloading <rport> not FCP_TARGET. not offloading

Session Upload Failures

bnx2fc: ERROR!! destroy timed out bnx2fc: Disable request timed out. destroy not set to FW bnx2fc: Disable failed with completion status <status> bnx2fc: Destroy failed with completion status <status>

Unable to Issue ABTS

bnx2fc: initiate_abts: tgt not offloaded bnx2fc: initiate_abts: rport not ready bnx2fc: initiate_abts: link is not ready bnx2fc: abort failed, xid = <xid>

Unable to Recover the IO Using ABTS (Due to ABTS Timeout)

bnx2fc: Relogin to the target

Unable to Issue IO Request Due to Session Not Ready

bnx2fc: Unable to post io_req

Drop Incorrect L2 Receive Frames

bnx2fc: FPMA mismatch... drop packet bnx2fc: dropping frame with CRC error

HBA/lport Allocation Failures

bnx2fc: Unable to allocate hba bnx2fc: Unable to allocate scsi host

NPIV Port Creation

bnx2fc: Setting vport names, <WWNN>, <WWPN>

44 83840-546-00 D

Page 75

5–Linux Driver Software Te aming with Channel Bonding

Teaming with Channel Bonding

With the Linux drivers, you can team adapters together using the bonding kernel module and a channel bonding interface. For more information, see the Channel Bonding information in your operating system documentation.

Statistics

Detailed statistics and configuration information can be viewed using the ethtool utility. See the ethtool man page for more information.

45 83840-546-00 D

Page 76

6 VMware Driver Software

 Packaging  Download, Install, and Update Drivers  Networking Support  FCoE Support

Packaging

The VMware driver is released in the packaging formats shown in Table 6-1. For information about iSCSI offload in VMware server, see “iSCSI Offload on VMware

Server” on page 212.

Table 6-1. VMware Driver Packaging

Format Drivers Package

Compressed zip QLG-NetXtremeII-2.0-version.zip

46 83840-546-00 D

Page 77

6–VMware Driver Software Download, Install, and Update Drivers

Download, Install, and Update Drivers

To download, install, or update the VMware ESXi driver for 8400/3400 Series 10 GbE network adapters, go to

http://www.vmware.com/resources/compatibility/search.php?deviceCategory=io

and do the following:

1. Type the adapter name (in quotes) in the Keyword window (such as "QLE3442"), and then click Update and View Results (Figure 6-1).

Figure 6-1. Selecting an Adapter

Figure 6-2 shows the available QLE3442 driver versions.

Figure 6-2. QLE3442 Driver Versions

47 83840-546-00 D

Page 78

6–VMware Driver Software Download, Install, and Update Drivers

2. Mouse over the QLE3442 link in the results section to show the PCI identifiers (Figure 6-3).

3. Click the model link to show a listing of all of the driver packages (Figure 6-4). Click the desired ESXi version, and then click the link to go to the VMware driver download web page.

Figure 6-3. PCI Identifiers

Figure 6-4. List of Driver Packages

48 83840-546-00 D

Page 79

6–VMware Driver Software

NOTE

Download, Install, and Update Drivers

4. Log in to the VMware driver download page and click Download to download the desired driver package (Figure 6-5).

Figure 6-5. Download Driver Package

5. This package is double zipped—unzip the package once before copying the offline bundle zip file to the ESXi host.

6. To install the driver package, issue the following command:

esxcli software vib install -d <path>/<offline bundle name.zip> --maintenance-mode

esxcli software vib install --depot=/<path>/<offline bundle name.zip> --maintenance-mode

 If you do not unzip the outer zipping, the installation will report that it can

not find the drivers.

 Use double dashes (--) before the depot and maintenance-mode

parameters.

 Do not use the -v method of installing individual driver vSphere

installation bundles (VIBs).

 A reboot is required after all driver installations.

49 83840-546-00 D

Page 80

6–VMware Driver Software Networking Support

Networking Support

This section describes the bnx2x VMware ESXi driver for the QLogic 8400/3400 Series PCIe 10 GbE network adapters.

Driver Parameters

Several optional parameters can be supplied as a command line argument to the vmkload_mod command. These parameters can also be set with the esxcfg-module command. See the man page for more information.

int_mode

The optional parameter int_mode is used to force using an interrupt mode other than MSI-X. By default, the driver will try to enable MSI-X if it is supported by the kernel. If MSI-X is not attainable, then the driver will try to enable MSI if it is supported by the kernel. If MSI is not attainable, then the driver will use the legacy INTx mode.

Set the int_mode parameter to 1 as shown below to force using the legacy INTx mode on all 8400/3400 Series network adapters in the system.

vmkload_mod bnx2x int_mode=1

Set the int_mode parameter to 2 as shown below to force using MSI mode on all 8400/3400 Series network adapters in the system.

vmkload_mod bnx2x int_mode=2

disable_tpa

The optional parameter disable_tpa can be used to disable the Transparent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets, but if you would like to disable this advanced feature, it can be done.

Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all 8400/3400 Series network adapters in the system.

vmkload_mod bnx2x.ko disable_tpa=1

Use ethtool to disable TPA (LRO) for a specific network adapter.

num_rx_queues

The optional parameter num_rx_queues may be used to set the number of Rx queues on kernels starting from 2.6.24 when multi_mode is set to 1 and interr upt mode is MSI-X. Number of Rx queues must be equal to or greater than the number of Tx queues (see num_tx_queues parameter). If the interrupt mode is different than MSI-X (see int_mode parameter), then then the number of Rx queues will be set to 1, discarding the value of this parameter.

50 83840-546-00 D

Page 81

6–VMware Driver Software Networking Support

num_tx_queues

The optional parameter num_tx_queues may be used to set the number of Tx queues on kernels starting from 2.6.27 when multi_mode is set to 1 and interr upt mode is MSI-X. The number of Rx queues must be equal to or greater than the number of Tx queues (see num_rx_queues parameter). If the interrupt mode is different than MSI-X (see int_mode parameter), then the number of Tx queues will be set to 1, discarding the value of this parameter.

pri_map

The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or the same CoS in the hardware. This 3 2-bit parameter is evaluated by the driver as 8 values of 4 bits each. Each nibble sets the desired hardware queue number for that priority.

For example, set the pri_map parameter to 0x22221 100 to map priority 0 a nd 1 to CoS 0, map priority 2 and 3 to CoS 1, and map priority 4 to 7 to CoS 2. In another example, set the pri_map parameter to 0x11110000 to map priority 0 to 3 to CoS 0, and map priority 4 to 7 to CoS 1.

qs_per_cos

cos_min_rate

The optional parameter qs_per_cos is used to specify the number of q ueues that will share the same CoS. This parameter is evaluated by the driver up to 3 values of 8 bits each. Each byte sets the desired number of queues for that CoS. The total number of queues is limited by the hardware limit.

For example, set the qs_per_cos parameter to 0x10101 to create a total of three queues, one per CoS. In another example, set the qs_per_cos parameter to 0x404 to create a total of 8 queues, divided into only 2 CoS, 4 queues in each CoS.

The optional parameter cos_min_rate is used to determine the weight of each CoS for round-robin scheduling in transmission. This parameter is evaluated by the driver up to three values of eight bits each. Each byte sets the desired weight for that CoS. The weight ranges from 0 to 100.

For example, set the cos_min_rate parameter to 0x101 for fair transmission rate between two CoS. In another example, set the cos_min_rate parameter to 0x30201 to give the higher CoS the higher rate of transmission. To avoid using the fairness algorithm, omit setting the optional parameter cos_min_rate or set it to 0.

51 83840-546-00 D

Page 82

6–VMware Driver Software Networking Support

dropless_fc

The optional parameter dropless_fc can be used to enable a complement ary flow control mechanism on QLogic network adapters. The default flow control mechanism is to send pause frames when the BRB is reaching a certain level of occupancy. This is a performance targeted flow control mechanism. On QLogic network adapters, you can enable another flow control mechanism to send pause frames if one of the host buffers (when in RSS mode) is exhausted. This is a zero packet drop targeted flow control mechanism.

Set the dropless_fc parameter to 1 as shown below to enable the dropless flow control mechanism feature on all QLogic network adapters in the system.

vmkload_mod bnx2x dropless_fc=1

RSS

The optional parameter RSS can be used to specify the number of receive side scaling queues. For VMware ESXi (5.1, 5.5, 6.0), values for RSS can be from 2 to 4; RSS=1 disables RSS queues.

max_vfs

The optional parameter max_vfs can be used to enable a specific number of virtual functions. V alues for max_vfs can be 1 to 64 , or set max_vfs=0 (default) to disable all virtual functions.

enable_vxlan_offld

The optional parameter enable_vxlan_ofld can be used to enable VMware ESXi (5.5, 6.0) VxLAN task offloads with TX TSO and TX CSO. For VMware ESXi (5.5, 6.0), enable_vxlan_ofld=1 (default) enables VxLAN task offloads; enable_vxlan_offload=0 disables VxLAN task offloads.

Driver Defaults

Speed: Autonegotiation with all speeds advertised Flow Control: Autonegotiation with rx and tx advertised MTU: 1500 (range 46–9600) Rx Ring Size: 4078 (range 0–4078) Tx Ring Size: 4078 (range (MAX_SKB_FRAGS+4) - 4078). MAX_SKB_FRAGS

varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce RX Microseconds: 25 (range 0–3000) Coalesce TX Microseconds: 50 (range 0–12288) MSI-X: Enabled (if supported by 2.6 kernel) TSO: Enabled

52 83840-546-00 D

Page 83

6–VMware Driver Software Networking Support

Unloading and Removing Driver

To unload the driver, type the following:

vmkload_mod -u bnx2x

Driver Messages

The following are the most common sample messages that may be logged in the file /var/log/messages. Use dmesg -n <level> to control the le vel at which messages will appear on the console. Most systems are set to level 6 by default. To see all messages, set the level higher.

Driver Sign On

QLogic 8400/3400 Series 10Gigabit Ethernet Driver bnx2x 0.40.15 ($DateTime: 2007/11/22 05:32:40 $)

NIC Detected

eth0: QLogic 8400/3400 Series XGb (A1) PCI-E x8 2.5GHz found at mem e8800000, IRQ 16, node addr

001018360012

MSI-X Enabled Successfully

bnx2x: eth0: using MSI-X

Link Up and Speed Indication

bnx2x: eth0 NIC Link is Up, 10000 Mbps full duplex, receive & transmit flow control ON

Link Down Indication

bnx2x: eth0 NIC Link is Down

Memory Limitation

If you see messages in the log file that look like the following, then the ESXi host is severely strained. To relieve this, disable NetQueue.

Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: Heap: 1435: Heap bnx2x already at its maximumSize. Cannot expand.

Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: Heap: 1645: Heap_Align(bnx2x, 4096/4096 bytes, 4096 align) failed. caller: 0x41800187d654

Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: vmklinux26: alloc_pages: Out of memory

Disable NetQueue by manually loading the bnx2x vmkernel module with the command.

vmkload_mod bnx2x multi_mode=0

53 83840-546-00 D

Page 84

6–VMware Driver Software FCoE Support

or to persist the settings across reboots with the command

esxcfg-module -s multi_mode=0 bnx2x

Reboot the machine for the settings to take place.

MultiQueue/NetQueue

The optional parameter num_queues may be used to set the number of Rx and Tx queues when multi_mode is set to 1 and interrupt mode is MSI-X. If interrupt mode is different than MSI-X (see int_mode para meter), the number of Rx and Tx queues will be set to 1, discarding the value of this parameter.

If you would like the use of more then 1 queue, force the number of NetQueues to use with the following command:

esxcfg-module -s "multi_mode=1 num_queues=<num of queues>" bnx2x

Otherwise, allow the bnx2x driver to select the number of NetQueues to use with the following command:

esxcfg-module -s "multi_mode=1 num_queues=0" bnx2x

The optimal number is to have the number of NetQueues match the number of CPUs on the machine.

FCoE Support

This section describes the contents and procedures associated with inst allation of the VMware software package for supporting QLogic FCoE C-NICs.

Enabling FCoE

To enable FCoE hardware offload on the C-NIC

1. Determine the ports that are FCoE-capable:

# esxcli fcoe nic list

Output example:

vmnic4 User Priority: 3 Source MAC: FF:FF:FF:FF:FF:FF Active: false Priority Settable: false Source MAC Settable: false VLAN Range Settable: false

54 83840-546-00 D

Page 85

6–VMware Driver Software

NOTE

FCoE Support

1. Enable the FCoE interface:

# esxcli fcoe nic discover -n vmnicX

Where X is the interface number gained from esxcli fcoe nic list.

2. Verify that the interface is working:

# esxcli fcoe adapter list

Output example:

vmhba34 Source MAC: bc:30:5b:01:82:39 FCF MAC: 00:05:73:cf:2c:ea VNPort MAC: 0e:fc:00:47:04:04 Physical NIC: vmnic7 User Priority: 3 VLAN id: 2008

The output of this command should show valid: FCF MAC, VNPort MAC, Priority, and VLAN id for the Fabric that is connected to the C-NIC.

The following command can also be used to verify that the interface is working properly:

#esxcfg-scsidevs -a

Output example:

vmhba34 bnx2fc link-up fcoe.1000<mac address>:2000<mac address> () Software FCoE

vmhba35 bnx2fc link-up fcoe.1000<mac address>:2000<mac address> () Software FCoE

The label Software FCoE is a VMware term used to describe initiators that depend on the inbox FCoE libraries and utilities. QLogic's FCoE solution is a fully state connection-based hardware offload solution designed to significantly reduce the CPU burden encumbered by a non-offload software initiator.

55 83840-546-00 D

Page 86

6–VMware Driver Software FCoE Support

Installation Check

To verify the correct installation of the driver and to ensure that the host port is seen by the switch, follow the procedure below.

To verify the correct installation of the driver

1. Verify the host port shows up in the switch FLOGI database using the show flogi database command for the case of a Cisco

-loginshow command for the case of a Brocade

2. If the Host WWPN does not appear in the FLOGI database, then provide driver log messages for review.

Limitations

 NPIV is not currently supported with this release on ESXi, due to lack of

supporting inbox components.

 Non-offload FCoE is not supported with offload-capable QLogic devices.

Only the full hardware offload path is supported.

FCF and fcoe

FCF.

Drivers

Table 6-2 lists the 8400/3400 Series FCoE drivers.

Table 6-2. QLogic 8400/3400 Series FCoE Drivers

Driver Description

bnx2x This driver manages all PCI device resources (registers, host inter-

face queues) and also acts as the Layer 2 VMware low-level network driver for QLogic's 8400/3400 Series 10G device. This driver directly controls the hardware and is responsible for sending and receiving Ethernet packets on behalf of the VMware host networking stack. The bnx2x driver also receives and processes device interrupts, both on behalf of itself (for L2 networking) and on be half of the bnx2fc (FCoE protocol) and CNIC drivers.

bnx2fc The QLogic VMware FCoE driver is a kernel mode driver used to

provide a translation layer between the VMware SCSI stack and the QLogic FCoE firmware/hardware. In addition, the driver interfaces with the networking layer to transmit and receive encapsulated FCoE frames on behalf of open-fcoe's libfc/libfcoe for FIP/device discovery.

Supported Distributions

The FCoE/DCB feature set is supported on VMware ESXi 5.0 and later.

56 83840-546-00 D

Page 87

7 Installing Management

Applications

 Overview  Installation Tasks  Detailed Procedures  Installing QCS and Related Management Applications  Management Applications (Windows)

Overview

The QLogic Control Suite (QCS) is a management application for configuring the 8400/3400 Series of converged network adapters and intelligent Ethernet adapters. QCS software operates on Windows and Linux server and client operating systems. This chapter describes how to install the QCS management application. For information about QCS, see the QLogic Control Suite GUI

Installation Guide and the QLogic Control Suite Command Line Interface User’s Guide.

QCS Components

There are two main components of the QCS application: the provider component and the client software. A provider is inst alled on a server, or “managed host”, that contains one or more converged network adapters. The provider collects information on the CNAs and makes it available for retrieval from a management PC on which the client software is installed. The client software enables viewing information from the providers and configuring the converged network adapters.The QCS client software includes a graphical user interface (GUI) and a command line interface (CLI).

57 83840-546-00 D

Page 88

7–Installing Management Applications Installation Tasks

Communication Protocols

A communication protocol enables exchanging information between provider and the client software. These are proprietary or open-source implementations of the Web-Based Enterprise Management (WBEM) and Common Information Model (CIM) standards from the Distributed Management Task Force (DMTF). Network administrators can choose the best option based on the prevailing standard on their network.

Table 7-1 shows the available options based on the operating systems installed

on the managed host and the client.

Table 7-1. QCS Communication Protocols

If the client uses:

Windows Windows WMI

Windows Linux CIM-XML (OpenPegasus)

Linux Windows WS-MAN (WinRM) Linux Linux CIM-XML (OpenPegasus)

 WMI = Windows Management Instrumentation.  WS-MAN = Web Service-Management. WinRM is a Windows-based implementation

and OpenPegasus is an open-source implementation of the that operates on Linux.

 CIM-XML = An XML-based version of OpenPegasus.

If your network includes a mix of Windows and Linux clients accessing Windows and Linux servers, then WS-MAN is a suitable choice. If Linux is the only OS installed on the servers, then CIM-XML is an option. If the network includes only Windows servers and clients, WMI is an option. WMI is very simple to configure but is supported only on the Windows OS.

And the managed

host uses:

WS-MAN (WinRM)

WS-MAN (OpenPegasus)

QCS can use these

communication protocols:

Installation Tasks

QCS installation includes installing the provider component on the managed host and the client software on the management station. The installation process differs based on the combination of operating systems installed on the client and managed host and on the selected communication protocol. The following sections list each task in the overall process and provide links to the specific ste ps for each task, as found in “Detailed Procedures” on page 61.

58 83840-546-00 D

Page 89

7–Installing Management Applications Installation Tasks

WS-MAN

The following steps install the WS-MAN protocol for communication between the client and managed host (server). WS-MAN is supported on both Windows and Linux clients and servers.

Windows Server

On Windows servers, configure the WinRM service as follows:

1. Install the WinRM Software Component on Server.

2. Perform Basic Configuration on the Server.

3. Perform User Configuration on the Server.

4. Perform HTTP Configuration on the Server.

5. Perform HTTPS Configuration on the Server (to use HTTPS Instead of

HTTP)

a. Generate a Self-Signed Certificate for Windows/Linux Server.

6. Configure WinRM HTTPS/SSL on the Server.

7. Perform Additional Server Configuration, if required.

8. Follow the procedure for “Installing QCS and Related Management

Windows Client

On the Windows client, perform following configuration steps.

1. Perform HTTP Configuration (if you plan to use HTTP).

2. Perform HTTPS Configuration (if you plan to use HTTPS).

3. Configure WinRM HTTPS/SSL.

4. Following the procedure for “Installing QCS and Related Management

Linux Server

On Linux server, use the following steps to install OpenPegasus. There are two options available with respect to OpenPegasus: installing from the Inbox RPM or installing from source. The Inbox OpenPegasus is only available on the Red Hat OS. For SLES11, the only option is to use the source RPM. WS-MAN is not supported with the Inbox RPM. If you plan to use WS-MAN then you must install OpenPegasus from source.

b. Install the Self-Signed Certificate on Windows Server.

Applications” on page 85.

1. Install OpenPegasus From the Inbox RPM (Red Hat Only) or Install

OpenPegasus From Source (Red Hat and SuSE).

2. Start CIM Server on the Server.

59 83840-546-00 D

Page 90

7–Installing Management Applications Installation Tasks

3. Configure OpenPegasus on the Server.

4. Install QLogic CMPI Provider.

5. Perform additional configuration, if required, such as firewall configuration. See “Perform Linux Firewall Configuration, If Required” on page 79.

6. Follow the procedure for “Installing QCS and Related Management

Applications” on page 85.

Linux Client

To use HTTP , no specia l configuration is required on the Linux client system. Only the QCS management application must be installed. Perform the following configuration steps:

1. Configure HTTPS on Linux Client.

2. Follow the procedure for “Installing QCS and Related Management

Applications” on page 85.

CIM-XML

CIM-XML is supported only when the server uses the Linux OS. Note that for CIM-XML on the Red Hat Linux OS, two installation options are available with respect to OpenPegasus: installing from the Inbox RPM or installing from source. The Inbox OpenPegasus is only available on the Red Hat OS. For SLES11, the only option is to use the source RPM. WS-MAN is not supported with the Inbox RPM. If you plan to use WS-MAN then you must install OpenPegasus from source.

1. Install OpenPegasus From the Inbox RPM (Red Hat Only) or Install

OpenPegasus From Source (Red Hat and SuSE).

2. Start CIM Server on the Server.

3. Configure OpenPegasus on the Server.

4. Install QLogic CMPI Provider.

5. Perform additional configuration, if required, such as firewall configuration. See “Perform Linux Firewall Configuration, If Required” on page 79.

6. Follow the procedure for “Installing QCS and Related Management

Applications” on page 85.

60 83840-546-00 D

Page 91

7–Installing Management Applications Detailed Procedures

Linux Client

To use HTTP , no specia l configuration is required on the Linux client system. Only the QCS management application must be installed. Perform the following configuration steps:

1. Configure HTTPS on Linux Client.

2. Follow the procedure for “Installing QCS and Related Management

Applications” on page 85.

WMI

The WMI protocol is only supported on Windows OSs. If servers and clients both are running Windows, then WMI can be used.

Windows Server

1. Set up Namespace Security Using WMI Control.

2. Grant DCOM Remote Launch and Activate Permission for a user or group.

3. Perform special configuration if necessary. See “Special Configuration for

WMI on Different Systems” on page 84.

Windows Client

No special configuration is required on the Windows client except installing the QCS management application. See “Installing QCS and Related Management

Applications” on page 85.

Detailed Procedures

This section provides the step-by-step instructions for each installation task. The required tasks for each communication protocol differ, as listed in “Installation

Tasks” on page 58. Refer to the appropriate task list to ensure you complete all

necessary tasks for the chosen protocol.

WS-MAN on Windows Server

Install the WinRM Software Component on Server

On the following operating systems, WinRM 2.0 is preinstalled:

 Windows 7  Windows 8  Windows 8.1  Windows Server 2008 R2  Windows Server 2012  Windows Server 2012 R2

61 83840-546-00 D

Page 92

7–Installing Management Applications

NOTE

Detailed Procedures

For Windows XP and Windows Server 2008, install Windows Management Framework Core, which includes WinRM 2.0 and Windows Powershell 2.0, from the following link:

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=11829

Perform Basic Configuration on the Server

The Windows firewall must be enabled for WinRM to work properly. For detailed information about firewall configuration, see “Additional Server Configuration” on

page 70. After the firewall is configured, open a command prompt and run the

following command to enable the remote management on the Windows server:

winrm quickconfig

You can use the following command to view the configuration information for the service:

winrm get winrm/config

Perform User Configuration on the Server

To connect to WinRM, the account must be a member of the local administrators group on the local or remote computer. The output of the get winrm/config command will be as follows:

RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)

BA stands for BUILTIN\Administrators. To add another user group to the WinRM allowed connect list, you can modify the

RootSDDL to include the new user group. You will need the SSDL ID for the new group. For example, the following command adds the new user group with SDDL ID S-1-5-21-1866529496-2433358402-1775838904-1021.

winrm set winrm/config/Service @{RootSDDL="O:NSG:BAD:P(A;GA;;;BA)(A;;GA;;;

S-1-5-21-1866529496-2433358402-1775838904-1021)S:P(AU;FA;GA;; WD)(AU;SA;GWGX;;;WD)"}

Perform HTTP Configuration on the Server

To use the QCS GUI, you must configure the HTTP protocol, as follows:

The default HTTP port is 5985 for WinRM 2.0.

1. Click Start (or press the Windows logo key) and select Run.

2. Enter gpedit.msc to open the local Group Policy editor.

62 83840-546-00 D

Page 93

7–Installing Management Applications Detailed Procedures

3. Under Computer Configuration, open the Administrative Templates folder and then open the Windows Components folder.

4. Select Windows Remote Management (WinRM).

5. Under Windows Remote Management (WinRM), select WinRm Client.

6. Under WinRM Client, double-click Trusted Hosts.

7. In the TrustedHostsList, enter the host names of the clients. If all clients are trusted then enter an asterisk (*) only.

8. Select WinRM Service.

9. Enable Allow Basic Authentication.

10. Enable Allow unencrypted traffic.

11. Close the Group Policy wIndow.

12. From the command prompt, run the following command to configure WinRM with default settings:

winrm qc or winrm quickconfig

13. When the tool displays “Make these changes[y/n]?“, enter “y”.

14. Enter one of the following commands to check whether an HTTP listener is created:

winrm enumerate winrm/confg/listener

winrm e winrm/config/Listener

15. Enter the following command from the command prompt to test locally.

winrm id

63 83840-546-00 D

Page 94

7–Installing Management Applications

NOTE

Detailed Procedures

Perform HTTPS Configuration on the Server (to use HTTPS Instead of HTTP)

This step consists of two distinct processes: generating a self-signed certificate, if certificate does not exist, and importing it to a Windows server. If one does not already exist, you must configure a self-signed certificate on the Windows server to enable HTTPS/SSL communication with the QCS GUI on the Windows or Linux client. The Windows and Linux client also must be configured with the self-signed certificate. See “Perform HTTPS Configuration (if you plan to use HTTPS)” on

page 72 to configure Windows and “Configure HTTPS on Linux Client” on page 80 to configure Linux client.

The self-signed certificate can be created on any Windows or Linux server. The server does not require QCS to be installed. The self-signed certificate generated on any Windows/Linux server should be copied on the local d rive of client.

1. Click Start (or press the Windows logo key) and select Run.

2. Enter gpedit.msc to open the local Group Policy editor.

3. Under Computer Configuration, open the Administrative Templates folder and then open the Windows Components folder.

4. Select Windows Remote Management (WinRM).

5. Under Windows Remote Management (WinRM), select WinRm Client.

6. Under WinRM Client, double-click Trusted Hosts.

7. In the TrustedHostsList, enter the host names of the clients. If all clients are trusted then enter an asterisk (*) only.

8. Select WinRM Service.

9. Enable Allow Basic Authentication.

Generate a Self-Signed Certificate for Windows/Linux Server

Openssl on Linux or Windows can be used to generate the self-signed certificate, as follows:

1. Enter the following command to generate a private key:

openssl genrsa -des3 -out server.key 1024

2. You are prompted to enter a passphrase. Be sure to remember the passphrase.

3. Use the following steps to generate a Certificate Signing Request (CSR).

64 83840-546-00 D

Page 95

7–Installing Management Applications Detailed Procedures

During the generation of the CSR, you are prompted for several pieces of information. When prompted for the “Common Name”, enter the Windows Server host name or IP address.

Enter the following command (sample responses are shown):

openssl req -new -key server.key -out server.csr

If this command does not work, try the following:

openssl req –new –key server.key –out server.csr –config openssl.cnf

The openssl.cnf file should be placed in the same directory whe re openssl is placed. Openssl.cnf is located in the folder C:\Program Files (x86)\GnuWin32\share.

The following information is requested:

 Country Name (2 letter code) []:US  State or Province Name (full name) []: California  Locality Name (city) []: Irvine  Organization Name (company) []: QLogic Corporation  Organizational Unit Name (section) []: Engineering  Common Name (your name) []: Enter the host name or IP address of

the Windows server. For iPv6, enter the Common Name in the format

[xyxy:xxx:….::xxx], including the brackets [ ].  (Optional) Email Address []: Enter the following additional attributes to be sent with your certificate

request:

 A challenge password []:linux1  An optional company name []:

4. Remove the passphrase from the key. Enter the following commands:

cp server.key server.key.org openssl rsa -in server.key.org -out server.key

5. Generate a self-signed certificate:

65 83840-546-00 D

Page 96

7–Installing Management Applications Detailed Procedures

To generate a self-signed certificate that is active for 365 days, enter the following command:

openssl x509 -req -days 365 -in server.csr -signkey server.key

-out server.crt

The following output appears:

Signature ok subject=/C=US/ST=California/L=Irvine/O=QLogic

Corporation/OU=Engineering/CN=MGMTAPP- LAB3/emailAddress= Getting Private key

6. Enter the following command to verify the generated self-signed certificate.

openssl verify server.crt

The following output appears:

server.crt:/C=US/ST=California/L=Irvine/O=QLogic Corporation/OU=Engineering/CN=MGMTAPP- LAB3/emailAddress=

error 18 at 0 depth lookup:self signed certificate OK

Ignore the error message “error 18 at 0 depth lookup:self signed certificate”. This error indicates that this is a self-signed certificate.

7. Convert the certificate from "crt" to "pkcs12" format, as follows: For a Windows server, the certificate should be in pkcs12 format. Enter the

following command:

openssl pkcs12 -export -in server.crt -inkey server.key -out hostname.pfx

You will be prompted for the following:

Enter Export Password: Verifying - Enter Export Password:

Enter the password and be sure to remember it. The password is required when importing the certificate on the Windows server and client.

8. Make a copy of the certificate file server.crt and place it on the server where QCS will be installed, so that it can be imported. If you plan to use a Windows or Linux client to connect to the server running QCS, then the certificate also needs to be transferred (copied and pasted) to the client system.

66 83840-546-00 D

Page 97

7–Installing Management Applications

NOTE

Detailed Procedures

In Linux, the certificate should have the extension “.pem”. The extension “.crt” and “.pem” are the same, so there is no need to use the openssl command to convert from .crt to .pem. You can simply copy the file as-is.

A separate certificate must be generated for an IPv4 address, IPv6 address, and Hostname.

Install the Self-Signed Certificate on Windows Server

Transfer the file hostname.pfx you generated on the Windows server before you install the certificate:

1. Click Start (or press the Windows logo key) and select Run.

2. Enter MMC and click OK.

3. Click File > Add/Remove Snap-in.

4. Click Add.

5. Select Certificates and click Add.

6. Select Computer account.

7. Click Next and then click Finish.

8. Click Close, then click OK.

9. Open the Certificates (Local Computer) folder and then open the Personal folder.

10. Right-click Certificates, select All Tasks and then click Import.

11. Click Next to begin the Certificate Import Wizard.

12. Browse to select hostname.pfx.

13. When you are prompted for the password for the private key, enter the same password you created in “Generate a Self-Signed Certificate for

Windows/Linux Server” on page 64.

14. Follow the instructions, select the defaults, and continue. The certificate is shown as installed on the right side of the window. The

name will be the name you specified while creating a self-signed certificate.

67 83840-546-00 D

Page 98

7–Installing Management Applications

NOTE

Detailed Procedures

15. Right-click on the certificate, and then select Properties to open the MGMTAPP-LAB3 Properties dialog box (Figure 7-1).

Figure 7-1. MGMTAPP-LAB3 Dialog Box

16. Ensure that only Server Authentication is enabled, as shown in the figure.

17. Open Trusted Root Certification Authorities and then open Certificates.

18. Follow the instructions from Step 1 1 to Step 17

See “Perform HTTPS Configuration (if you plan to use HTTPS)” on page 72 for instructions on importing the self-signed certificate on a client.

Configure WinRM HTTPS/SSL on the Server

1. Create WinRM Listener, as follows: a. Click Start (or press the Windows logo key) and select Run. b. Enter MMC and click OK. c. Select the self-signed certificate from the Personal store.

68 83840-546-00 D

Page 99

7–Installing Management Applications

NOTE

Detailed Procedures

For example, if the certificate is created with a host name, the host

name will appear. d. Double-click the certificate to open it. e. Click the Details tab. f. Scroll down and select the Thumbprint field. g. Select and copy the thumbprint in the Details window so you can

insert it in the next step. h. Return to the command prompt. i. Enter the following command:

winrm create winrm/config/Listener?Address=*+Transport=

HTTPS @{Hostname="<HostName or IPAddress>";

CertificateThumbprint="<paste from the previous step and

remove the spaces>"}

 If the certificate was generated using the host name, enter

the host name. If it was generated using the IP address, enter the IP address. For an IPv6 address, use brackets [ ] around the address.

 If HTTPS is configured in your system, the listener must be

deleted before creating a new HTTPS listener. Use the following command:

winrm delete winrm/config/Listener?Address=*+Transport=HTT PS

j. The above command creates a listener on the HTTPS port (5986)

using any/all network address of the server, and my SelfSSL

generated certificate. k. You can use the winrm co mmand to modify or set the HTTPS listener,

as WinRM listeners can be configured on any user defined port. l. From command prompt, run the following command to verify that the

listener(s) that have been configured:

winrm e winrm/config/listener

69 83840-546-00 D

Page 100

7–Installing Management Applications Detailed Procedures

2. Test HTTPS/SSL connection on the server. a. At the command prompt on the server, enter the following command:

winrs -r:https://yourserver:5986 -u:username -p:password hostname

b. If setup correctly, the output of the command shows the server host

name.

c. To check WinRM Service Configuration, run the following command:

winrm get winrm/config/service

Additional Server Configuration

If necessary, modify the firewall rules as follows:

Windows Server 2008 R2

1. From the Administrative Tools menu, open Windows Firewall with Advanced Security.

2. Right-click Inbound Rules and select New Rule. The new rule wizard opens.

3. Select Port and click Next.

4. On the Protocol and Ports screen, select TCP and enter the specific port, for example, 5985 for HTTP or 5986 for HTTPS.

5. Click Next.

6. On the Action screen, select Allow the connection and click Next.

7. For Profile, you can select all three profiles if your server is in a workgroup.

8. Specify a name for the rule and click Finish.

9. Ensure that the new rule and is enabled (the green check box is selected).

Windows XP

1. Click Start > Control Panel, and then double-click Windows Firewall.

2. Click the Exceptions tab

3. Click Add Port.

4. Enter a meaningful Name, for example “WinRM rule” and port number, for example, 5985 for HTTP or 5986 for HTTPS.

5. Click OK.

70 83840-546-00 D

Qlogic 8400 Series, 3400 Series User Manual

Table of Contents

List of Figures

List of Tables

Preface

Intended Audience

What Is in This Guide

Related Materials

Documentation Conventions

License Agreements

Technical Support

Downloading Updates

Training

Contact Information

Knowledge Database

Legal Notices

Warranty

Laser Safety

FDA Notice

Agency Certification

EMI and EMC Requirements

Product Safety Compliance

1 Product Overview

Functional Description

Features

iSCSI

FCoE

Power Management

Adaptive Interrupt Frequency

ASIC with Embedded RISC Processor

QLogic Control Suite

Supported Operating Environments

Adapter Specifications

Physical Characteristics

Standards Specifications

2 Installing the Hardware

System Requirements

Hardware Requirements

Operating System Requirements

Safety Precautions

Preinstallation Checklist

Installation of the Network Adapter

Connecting the Network Cables

Overview

Setting Up MBA in a Client Environment

Enabling the MBA Driver

Configuring the MBA Driver

Setting Up the BIOS

Setting Up MBA in a Server Environment

Red Hat Linux PXE Server

MS-DOS UNDI/Intel APITEST

4 Windows Driver Software

Installing the Driver Software

Using the Installer

Using Silent Installation

Removing the Device Drivers

Installing QLogic Management Applications

Viewing or Changing the Adapter Properties

Setting Power Management Options

5 Linux Driver Software

Introduction

Limitations

bnx2x Driver

bnx2i Driver

bnx2fc Driver

Packaging

Installing Linux Driver Software

Installing the Source RPM Package

Installing the KMP Package

Building the Driver from the Source TAR File

Load and Run Necessary iSCSI Software Components

Unloading/Removing the Linux Driver

Unloading/Removing the Driver from an RPM Installation

Removing the Driver from a TAR Installation

Patching PCI Files (Optional)

Uninstalling QCS

RPM Package

Network Installations

Setting Values for Optional Properties

bnx2x Driver