Chelsio Communications Terminator Series Installation And User Manual

Chelsio Unified Wire for Windows ii

This document and related products are distributed under licenses restricting their use, copying, distribution,
and reverse-engineering.

No part of this document may be reproduced in any form or by any means without prior written permission
by Chelsio Communications.

All third-party trademarks are copyright of their respective owners.

THIS DOCUMENTATION IS PROVIDED “AS IS” AND WITHOUT ANY EXPRESS OR IMPLIED

WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

THE USE OF THE SOFTWARE AND ANY ASSOCIATED MATERIALS (COLLECTIVELY THE

“SOFTWARE”) IS SUBJECT TO THE SOFTWARE LICENSE TERMS OF CHELSIO COMMUNICATIONS,

INC.

Sales

For all sales inquiries please send email to [e-mail]

Support

For all support related questions please send email to [e-mail]
Copyright © 2019. Chelsio Communications. All Rights Reserved.

Chelsio ® is a registered trademark of Chelsio Communications.
All other marks and names mentioned herein may be trademarks of their respective companies.

Chelsio Communications (Headquarters)
209 North Fair Oaks Avenue,
Sunnyvale, CA 94085
U.S.A

www.chelsio.com

Tel: [phone]
Fax: [phone]

Chelsio (India) Private Limited
Subramanya Arcade, Floor 3, Tower B
No. 12, Bannerghatta Road,
Bangalore-560029
Karnataka,
India

Tel: [phone]

Chelsio KK (Japan)

Yamato Building 8F,
5-27-3 Sendagaya,
Shibuya-ku,
Tokyo 151-0051,
Japan

Chelsio Unified Wire for Windows iii

Document History

Version

Revision Date

1.5.2

01/05/2017

1.5.3

02/07/2017

1.5.4

02/24/2017

1.5.5

03/10/2017

1.5.6

03/31/2017

1.5.7

04/11/2017

1.5.8

05/09/2017

1.5.9

05/29/2017

1.6.0

06/26/2017

1.6.1

06/30/2017

1.6.2

08/18/2017

1.6.3

08/24/2017

1.6.4

09/28/2017

1.6.5

11/27/2017

1.6.6

12/04/2017

1.6.7

12/08/2017

1.6.8

12/27/2017

1.6.9

02/20/2018

1.7.0

02/26/2018

1.7.1

02/28/2018

1.7.2

07/06/2018

1.7.3

07/17/2018

1.7.4

08/13/2018

1.7.5

09/26/2018

1.7.6

12/27/2018

1.7.7

01/31/2019

1.7.8

03/29/2019

1.7.9

04/24/2019

Chelsio Unified Wire for Windows iv

TABLE OF CONTENTS

I. CHELSIO UNIFIED WIRE 5

1. Introduction 6

1.1. Features 6

1.2. Hardware Requirements 6

1.3. Software Requirements 7

1.4. Package Contents 7

2. Hardware Installation 8

3. Software/Driver Installation 9

3.1. Prerequisites 9

3.2. Unified Wire Installer 9

3.3. Zip Package 13

4. Firmware Update 16

5. cxgbtool help 17

6. Software/Driver Uninstallation 36

6.1. Installer 36

6.2. Zip Package 39

II. FCOE STORPORT MINIPORT 40

1. Introduction 41

1.1. Hardware Requirements 41

1.2. Software Requirements 41

2. Configuration and Fine-tuning 42

2.1. Configuring Cisco Nexus 5010 switch 42

2.2. FCoE fabric discovery verification 45

III. APPENDIX 50

1. Troubleshooting 51

2. Chelsio End-User License Agreement (EULA) 52

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 5

I. Chelsio Unified Wire

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 6

1. Introduction

Thank you for choosing Chelsio Unified Wire adapters. These high speed, single chip, single
firmware cards provide enterprises and data centers with high performance solutions for various
Network and Storage related requirements.

The Terminator series is Chelsio’s next generation of highly integrated, hyper-virtualized
10/25/40/50/100GbE controllers. The adapters are built around a programmable protocol­processing engine, with full offload of a complete Unified Wire solution comprising NIC, TOE,
iWARP RDMA, iSCSI, FCoE and NAT support. It scales to true 100Gb line rate operation from a
single TCP connection to thousands of connections and allows simultaneous low latency and high
bandwidth operation thanks to multiple physical channels through the ASIC.

Ideal for all data, storage and high-performance clustering applications, the Unified wire
adapters enable a unified fabric over a single wire by simultaneously running all unmodified IP
sockets, Fibre Channel and InfiniBand applications over Ethernet at line rate.

Designed for deployment in virtualized datacenters, cloud service installations and high­performance computing environments, Chelsio Terminator adapters bring a new level of
performance metrics and functional capabilities to the computer networking industry.

1.1. Features

Chelsio Unified Wire for Windows is an easy to use utility developed to provide installation of 64­bit Windows based drivers and tools for Chelsio's Unified Wire adapters.

It consists of the following component(s):

• FCoE Storport Miniport Driver

• Virtual Bus Driver (VBD)

1.2. Hardware Requirements

The Chelsio Unified Wire supports all x64 architectures supporting PCIE (x4, x8, x16) slots.

AMD CPUs, 64-bit (x86_64/amd64)
Intel CPUs, 64-bit (x86_64)

The Chelsio Unified Wire supports 3.3v PCI bus only. Running an adapter on a
PCI x4 slot is not recommended as performance will be significantly reduced by
the limitations of PCI.

Note

T4 family of adapters are not supported.

Note

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 7

1.3. Software Requirements

The Chelsio Unified Wire software has been developed to run on Windows based platforms. To
know more about the complete list of versions supported by each driver/software, please refer
their respective sections.

1.4. Package Contents

Unified Wire Installer

Chelsio Unified Wire comes with an interactive installer and support documentation. The
documentation, consisting of README, Release Notes and User’s Guide (this document), can
be found in the <system_drive>\ChelsioUwire\docs\ directory after installing Chelsio Unified Wire.

Zip Package

The zip package contains driver files, firmware binaries, adapter configuration binaries and
support documentation (docs folder).

Drivers are WHQL certified for Windows Server 2016, 2012 R2 and 10 Client.

Note

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 8

2. Hardware Installation

i. Shutdown/power off your system.
ii. Power off all remaining peripherals attached to your system.
iii. Unpack the Chelsio adapter and place it on an anti-static surface.
iv. Remove the system case cover according to the system manufacturer’s instructions.
v. Remove the PCI filler plate from the slot where you will install the Ethernet adapter.
vi. For maximum performance, it is highly recommended to install the adapter into a PCIE x8/x16

slot.

vii. Holding the Chelsio adapter by the edges, align the edge connector with the PCI connector

on the motherboard. Apply even pressure on both edges until the card is firmly seated. It may

be necessary to remove the SFP (transceiver) modules prior to inserting the adapter.
viii. Secure the Chelsio adapter with a screw, or other securing mechanism, as described by the

system manufacturer’s instructions. Replace the case cover.
ix. After securing the card, ensure that the card is still fully seated in the PCIE x8/x16 slot as

sometimes the process of securing the card causes the card to become unseated.
x. Connect a fiber/twinax cable, multi-mode for short range (SR) optics or single-mode for long

range (LR) optics, to the Ethernet adapter or regular Ethernet cable for the 1Gb Ethernet

adapter.
xi. Power on your system.
xii. Verify if the adapter was installed successfully. To do so, open Device Manager in Control

Panel.
xiii. Under Other devices section, Chelsio adapter should be listed as Ethernet Controller. If the

adapter is not listed, right-click on the system name or click on the Actions menu and select

Scan for hardware changes
For Chelsio adapters, the physical functions are currently assigned as:

Physical functions 0 - 3: for the SR-IOV functions
Physical function 4: for all NIC functions of the card
Physical function 5: for iSCSI
Physical function 6: for FCoE
Physical function 7: Currently not assigned

xiv. Once the Unified Wire package is installed, open Device Manager again. Expand Network

adapters section and now Chelsio adapter should be listed.

Network device names for Chelsio’s physical ports are assigned using the

following convention: the port farthest from the motherboard will appear as the
first Ethernet interface. However, for T5 40G adapters, the association of physical
Ethernet ports and their corresponding network device names is opposite. For
these adapters, the port nearest to the motherboard will appear as the first

network interface.

Note

All 4-ports of T6425-CR adapter will be functional only if PCIe x8 -> 2x PCIe x4 slot
bifurcation is supported by the system and enabled in BIOS. Otherwise, only 2-ports
will be functional.

Note

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 9

3. Software/Driver Installation

Chelsio Unified Wire can be installed using the Installer or the zip package. Refer the relevant
section below depending on the method of installation selected.

3.1. Prerequisites

Please make sure that the following requirements are met before installation:

Latest Microsoft Visual C++ Redistributable packages for Visual Studio (x86 and x64) are

required if you are planning to use installer.

Windows update KB2685811 should be installed on Windows 7 Client and 2008 R2 Server. It

will be installed if Automatic Updates is configured or can be installed manually from

https://www.catalog.update.microsoft.com/Search.aspx?q=KB2685811

3.2. Unified Wire Installer

Chelsio Unified Wire Installer provides two methods of installation: GUI or CLI mode. GUI mode
provides an interactive GUI installer with customizable options. Whereas, CLI mode enables
unattended installation of Chelsio drivers and software thereby relieving the user from monitoring
the installation process and providing input via dialog boxes.

Enabling Test Signing

In case the drivers in the package are not WHQL certified, follow the steps mentioned below to
enable test signing on your system:

i. Goto Start->Run command option, enter "cmd" and press OK. This will open the command

prompt utility.
ii. Run the following command:

C:\Users\Administrator> bcdedit /set testsigning on

iii. Reboot the machine for the changes to take effect.

GUI mode (Installer)

i. Run the ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe installer application.
ii. Click Next for the Chelsio End User License Agreement Window.

Important

If you are planning to install Unified Wire in a cluster, it should be done on all
the members. Using different versions of Unified Wire is not supported.

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 10

Figure 1 - Unified Wire installer welcome window

iii. Select the radio button I accept the terms of the license agreement and click Next

Figure 2 - Chelsio EULA window

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 11

iv. Select Complete for complete package installation.

Figure 3 - Select setup (installation) type

v. Click Install to start the installation.

Figure 4 - Start installation

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 12

vi. FCoE Storport Miniport and NIC drivers will now be installed.

Figure 5 - Installation in progress

vii. Click Finish to exit from the Unified Wire Installer.

Figure 6 - Finishing Unified Wire installation

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 13

CLI mode (Silent Installation)

To install the driver using this feature, open command prompt and execute the following
command:

C:\Users\Administrator> ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe –in all

The above command will install NIC and FCoE Storport Miniport drivers.
To know more about other parameters and options, execute the following command:

C:\Users\Administrator> ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -help

The above command will create a help file, ChelsioUwire_x.x.x.x_WIN_yyy.y.yy_help.txt, in the
present working directory which contains the complete list of command line syntax required for
performing all the necessary CLI operations.

3.3. Zip Package

Chelsio Unified Wire zip package provides two methods of installation: Manual and using Offline
Windows Image. In Manual method, the driver components must be individually installed using

the Device Manager. Using the second method, you can add Chelsio driver components to an
offline OS image. The image can then be used to install Windows on your system.

Enabling Test Signing

In case the drivers in the package are not WHQL certified, please follow the steps mentioned
below to enable test signing before installing Unified Wire:

i. Open command prompt and run the following command to enable test signing:

C:\Users\Administrator>bcdedit /set testsigning on

ii. Change your working directory to ChelsioUwire_x.x.x.x_WIN_yyy.y.yy/Selfsign/ and run the

following commands:

C:\ ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign> .\certmgr.exe -add
.\chelsiocert1.cer -s -r LocalMachine root -all
C:\ ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign> .\certmgr.exe -add
.\chelsiocert1.cer -s -r LocalMachine trustedpublisher -all

iii. Reboot the machine for the changes to take effect.

A log file, ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.log, is created in the same directory
which keeps a record of all the commands executed and their results.

Note

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 14

Drivers location

The driver files should be manually picked from:

• ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\drivers\Win10 for Server 2016 and 10 Client.

• ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\drivers\Win7 for Server 2012 R2.

Manual

Virtual Bus Driver

i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),

click on System Devices, right click on Chelsio Bus Enumerator and select Update

Driver Software.
ii. Select Browse my computer for driver software.
iii. Select Let me pick from a list of device drivers on my computer and click Next.
iv. Click on Have Disk Button and on the next screen browse for chvbdx64.inf file in the zip

package. Click Open and then OK.
v. Click Next and driver installation will progress. Click Close once the installation is complete.

FCoE Storport Miniport driver

i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),

click on System Devices, right click on Chelsio Generic function with function 6 and select

Properties. In case the adapter is not listed, it may appear as Fibre Channel Controller

under Other devices.

Important

• The driver components need to be installed strictly in the order described
here. Using any other order is not supported.

• Before proceeding, open Device Manager and disable NDIS Miniport driver
(Network adapters->Chelsio Network Adapter) followed by Virtual Bus Driver
(System devices->Chelsio Bus Enumerator).

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 15

ii. Under Driver tab, select Update Driver…
iii. Select Browse my computer for driver software.
iv. Select Let me pick from a list of device drivers on my computer and click Next.
v. Click on Have Disk Button and on the next screen, browse for csiofcoe.inf file in the zip

package. Click Open and then OK.

vi. Click Next and driver installation will progress. Click Close once the installation is complete.

NDIS Miniport Driver

i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),

click on Network Adapters, right click on the Chelsio network adapter and select Update

Driver Software.
ii. Select Browse my computer for driver software.
iii. Select Let me pick from a list of device drivers on my computer.
iv. Click on Have Disk Button and on the next screen browse for chnetx64.inf file in the zip

package. Click Open and then OK.
v. Click Next and driver installation will progress. Click Close once the installation is complete.

Repeat the above steps for the other adapters.

Generic Function

i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),

click on System Devices, right click on Chelsio Generic Function and select Update

Driver Software.
ii. Select Browse my computer for driver software.
iii. Select Let me pick from a list of device drivers on my computer.
iv. Click on Have Disk Button and on the next screen browse for chnulx64.inf file in the zip

package. Click Open and then OK.
v. Click Next and driver installation will progress. Click Close once the installation is complete.

Offline Windows Image

You can use Windows Deployment Image Servicing and Management (DISM) tool to add Chelsio
driver components (.inf files) to an offline Windows image. For more information, visit the following
links:

• http://technet.microsoft.com/en-us/library/hh825070.aspx

• http://blogs.technet.com/b/heyscriptingguy/archive/2012/09/27/use-the-powershell-dism-

cmdlets-to-manage-windows-8.aspx

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 16

4. Firmware Update

The driver will auto-load the firmware if an update is required. The firmware version can be verified
using cxgbtool:

C:\Users\Administrator>cxgbtool nic0 debug version

In case of installer, the firmware binaries will be copied to <system_drive>\ChelsioUwire
\Firmware\ folder during installation.

For zip package, the binaries will be present in ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\firmware\
folder.

Chapter I. Chelsio Unified Wire

Chelsio Unified Wire for Windows 17

5. cxgbtool help

The cxgbtool command queries or sets various aspects of Chelsio network interface cards. It
complements standard tools used to configure network settings and provides functionality not
available through such tools.

Some of the commands provided can be used to query running statistics to aid in debugging.

Syntax

cxgbtool <vbdIface|nicIface|iSCSIIface|rdma> <category> <command> <options>

Definitions

nicIface is the name of the network device to work on, given in the format nic0, nic1.
vbdIface is the name of the Chelsio Bus Enumerator instance to work on, given in the format

vbd0, vbd1
iscsIface is the name of the Chelsio iSCSI interface to work on, given in the format iSCSI0,

iSCSI1
rdma is the name of the Chelsio RDMA interface to work on.
fcoeIface is the name of the Chelsio FCoE interface to work on, given in the format

Scsi1,Scsi2
category is one of the cxgbtool categories, i.e., boot, cudbg, debug, firmware, hardware
command is the action you wish to perform on the adapter or property you wish to view/change.
option is the optional parameter to be provided with command.

Help

o Displaying cxgbtool help: cxgbtool

o Displaying list of available cxgbtool commands: cxgbtool -h all

o Displaying category specific help:

cxgbtool <vbdIface|nicIface|iSCSIIface> <category> -h

o Displaying list of cudbg entities: cxgbtool <vbdIface> cudbg dbg -h

o Displaying RDMA help: cxgbtool rdma -h

o Displaying FCoE help: cxgbtool fcoe -h