Chelsio Communications Terminator Series Installation And User Manual

Chelsio Unified Wire for Windows ii
This document and related products are distributed under licenses restricting their use, copying, distribution, and reverse-engineering.
No part of this document may be reproduced in any form or by any means without prior written permission by Chelsio Communications.
All third-party trademarks are copyright of their respective owners.
WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
THE USE OF THE SOFTWARE AND ANY ASSOCIATED MATERIALS (COLLECTIVELY THE
“SOFTWARE”) IS SUBJECT TO THE SOFTWARE LICENSE TERMS OF CHELSIO COMMUNICATIONS,
INC.
Sales
For all sales inquiries please send email to sales@chelsio.com
Support
For all support related questions please send email to support@chelsio.com Copyright © 2019. Chelsio Communications. All Rights Reserved.
Chelsio ® is a registered trademark of Chelsio Communications. All other marks and names mentioned herein may be trademarks of their respective companies.
Chelsio Communications (Headquarters) 209 North Fair Oaks Avenue, Sunnyvale, CA 94085 U.S.A
www.chelsio.com
Tel: 408.962.3600 Fax: 408.962.3661
Chelsio (India) Private Limited Subramanya Arcade, Floor 3, Tower B No. 12, Bannerghatta Road, Bangalore-560029 Karnataka, India
Tel: +1-91-80-4039-6800
Chelsio KK (Japan)
Yamato Building 8F, 5-27-3 Sendagaya, Shibuya-ku, Tokyo 151-0051, Japan
Chelsio Unified Wire for Windows iii
Document History
Version
Revision Date
1.5.2
01/05/2017
1.5.3
02/07/2017
1.5.4
02/24/2017
1.5.5
03/10/2017
1.5.6
03/31/2017
1.5.7
04/11/2017
1.5.8
05/09/2017
1.5.9
05/29/2017
1.6.0
06/26/2017
1.6.1
06/30/2017
1.6.2
08/18/2017
1.6.3
08/24/2017
1.6.4
09/28/2017
1.6.5
11/27/2017
1.6.6
12/04/2017
1.6.7
12/08/2017
1.6.8
12/27/2017
1.6.9
02/20/2018
1.7.0
02/26/2018
1.7.1
02/28/2018
1.7.2
07/06/2018
1.7.3
07/17/2018
1.7.4
08/13/2018
1.7.5
09/26/2018
1.7.6
12/27/2018
1.7.7
01/31/2019
1.7.8
03/29/2019
1.7.9
04/24/2019
Chelsio Unified Wire for Windows iv
TABLE OF CONTENTS
I. CHELSIO UNIFIED WIRE 5
1. Introduction 6
1.1. Features 6
1.2. Hardware Requirements 6
1.3. Software Requirements 7
1.4. Package Contents 7
2. Hardware Installation 8
3. Software/Driver Installation 9
3.1. Prerequisites 9
3.2. Unified Wire Installer 9
3.3. Zip Package 13
4. Firmware Update 16
5. cxgbtool help 17
6. Software/Driver Uninstallation 36
6.1. Installer 36
6.2. Zip Package 39
II. FCOE STORPORT MINIPORT 40
1. Introduction 41
1.1. Hardware Requirements 41
1.2. Software Requirements 41
2. Configuration and Fine-tuning 42
2.1. Configuring Cisco Nexus 5010 switch 42
2.2. FCoE fabric discovery verification 45
III. APPENDIX 50
1. Troubleshooting 51
2. Chelsio End-User License Agreement (EULA) 52
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 5
I. Chelsio Unified Wire
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 6
1. Introduction
Thank you for choosing Chelsio Unified Wire adapters. These high speed, single chip, single firmware cards provide enterprises and data centers with high performance solutions for various Network and Storage related requirements.
The Terminator series is Chelsio’s next generation of highly integrated, hyper-virtualized 10/25/40/50/100GbE controllers. The adapters are built around a programmable protocol­processing engine, with full offload of a complete Unified Wire solution comprising NIC, TOE, iWARP RDMA, iSCSI, FCoE and NAT support. It scales to true 100Gb line rate operation from a single TCP connection to thousands of connections and allows simultaneous low latency and high bandwidth operation thanks to multiple physical channels through the ASIC.
Ideal for all data, storage and high-performance clustering applications, the Unified wire adapters enable a unified fabric over a single wire by simultaneously running all unmodified IP sockets, Fibre Channel and InfiniBand applications over Ethernet at line rate.
Designed for deployment in virtualized datacenters, cloud service installations and high­performance computing environments, Chelsio Terminator adapters bring a new level of performance metrics and functional capabilities to the computer networking industry.
1.1. Features
Chelsio Unified Wire for Windows is an easy to use utility developed to provide installation of 64­bit Windows based drivers and tools for Chelsio's Unified Wire adapters.
It consists of the following component(s):
FCoE Storport Miniport Driver
Virtual Bus Driver (VBD)
1.2. Hardware Requirements
The Chelsio Unified Wire supports all x64 architectures supporting PCIE (x4, x8, x16) slots.
AMD CPUs, 64-bit (x86_64/amd64) Intel CPUs, 64-bit (x86_64)
The Chelsio Unified Wire supports 3.3v PCI bus only. Running an adapter on a PCI x4 slot is not recommended as performance will be significantly reduced by the limitations of PCI.
Note
T4 family of adapters are not supported.
Note
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 7
1.3. Software Requirements
The Chelsio Unified Wire software has been developed to run on Windows based platforms. To know more about the complete list of versions supported by each driver/software, please refer their respective sections.
1.4. Package Contents
Unified Wire Installer
Chelsio Unified Wire comes with an interactive installer and support documentation. The documentation, consisting of README, Release Notes and User’s Guide (this document), can be found in the <system_drive>\ChelsioUwire\docs\ directory after installing Chelsio Unified Wire.
Zip Package
The zip package contains driver files, firmware binaries, adapter configuration binaries and support documentation (docs folder).
Drivers are WHQL certified for Windows Server 2016, 2012 R2 and 10 Client.
Note
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 8
2. Hardware Installation
i. Shutdown/power off your system. ii. Power off all remaining peripherals attached to your system. iii. Unpack the Chelsio adapter and place it on an anti-static surface. iv. Remove the system case cover according to the system manufacturer’s instructions. v. Remove the PCI filler plate from the slot where you will install the Ethernet adapter. vi. For maximum performance, it is highly recommended to install the adapter into a PCIE x8/x16
slot.
vii. Holding the Chelsio adapter by the edges, align the edge connector with the PCI connector
on the motherboard. Apply even pressure on both edges until the card is firmly seated. It may
be necessary to remove the SFP (transceiver) modules prior to inserting the adapter. viii. Secure the Chelsio adapter with a screw, or other securing mechanism, as described by the
system manufacturer’s instructions. Replace the case cover. ix. After securing the card, ensure that the card is still fully seated in the PCIE x8/x16 slot as
sometimes the process of securing the card causes the card to become unseated. x. Connect a fiber/twinax cable, multi-mode for short range (SR) optics or single-mode for long
range (LR) optics, to the Ethernet adapter or regular Ethernet cable for the 1Gb Ethernet
adapter. xi. Power on your system. xii. Verify if the adapter was installed successfully. To do so, open Device Manager in Control
Panel. xiii. Under Other devices section, Chelsio adapter should be listed as Ethernet Controller. If the
adapter is not listed, right-click on the system name or click on the Actions menu and select
Scan for hardware changes For Chelsio adapters, the physical functions are currently assigned as:
Physical functions 0 - 3: for the SR-IOV functions Physical function 4: for all NIC functions of the card Physical function 5: for iSCSI Physical function 6: for FCoE Physical function 7: Currently not assigned
xiv. Once the Unified Wire package is installed, open Device Manager again. Expand Network
adapters section and now Chelsio adapter should be listed.
Network device names for Chelsio’s physical ports are assigned using the
following convention: the port farthest from the motherboard will appear as the first Ethernet interface. However, for T5 40G adapters, the association of physical Ethernet ports and their corresponding network device names is opposite. For these adapters, the port nearest to the motherboard will appear as the first
network interface.
Note
All 4-ports of T6425-CR adapter will be functional only if PCIe x8 -> 2x PCIe x4 slot bifurcation is supported by the system and enabled in BIOS. Otherwise, only 2-ports will be functional.
Note
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 9
3. Software/Driver Installation
Chelsio Unified Wire can be installed using the Installer or the zip package. Refer the relevant section below depending on the method of installation selected.
3.1. Prerequisites
Please make sure that the following requirements are met before installation:
Latest Microsoft Visual C++ Redistributable packages for Visual Studio (x86 and x64) are
required if you are planning to use installer.
Windows update KB2685811 should be installed on Windows 7 Client and 2008 R2 Server. It
will be installed if Automatic Updates is configured or can be installed manually from
https://www.catalog.update.microsoft.com/Search.aspx?q=KB2685811
3.2. Unified Wire Installer
Chelsio Unified Wire Installer provides two methods of installation: GUI or CLI mode. GUI mode provides an interactive GUI installer with customizable options. Whereas, CLI mode enables unattended installation of Chelsio drivers and software thereby relieving the user from monitoring the installation process and providing input via dialog boxes.
Enabling Test Signing
In case the drivers in the package are not WHQL certified, follow the steps mentioned below to enable test signing on your system:
i. Goto Start->Run command option, enter "cmd" and press OK. This will open the command
prompt utility. ii. Run the following command:
C:\Users\Administrator> bcdedit /set testsigning on
iii. Reboot the machine for the changes to take effect.
GUI mode (Installer)
i. Run the ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe installer application. ii. Click Next for the Chelsio End User License Agreement Window.
Important
If you are planning to install Unified Wire in a cluster, it should be done on all the members. Using different versions of Unified Wire is not supported.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 10
Figure 1 - Unified Wire installer welcome window
iii. Select the radio button I accept the terms of the license agreement and click Next
Figure 2 - Chelsio EULA window
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 11
iv. Select Complete for complete package installation.
Figure 3 - Select setup (installation) type
v. Click Install to start the installation.
Figure 4 - Start installation
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 12
vi. FCoE Storport Miniport and NIC drivers will now be installed.
Figure 5 - Installation in progress
vii. Click Finish to exit from the Unified Wire Installer.
Figure 6 - Finishing Unified Wire installation
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 13
CLI mode (Silent Installation)
To install the driver using this feature, open command prompt and execute the following command:
C:\Users\Administrator> ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe –in all
The above command will install NIC and FCoE Storport Miniport drivers. To know more about other parameters and options, execute the following command:
C:\Users\Administrator> ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -help
The above command will create a help file, ChelsioUwire_x.x.x.x_WIN_yyy.y.yy_help.txt, in the present working directory which contains the complete list of command line syntax required for performing all the necessary CLI operations.
3.3. Zip Package
Chelsio Unified Wire zip package provides two methods of installation: Manual and using Offline Windows Image. In Manual method, the driver components must be individually installed using
the Device Manager. Using the second method, you can add Chelsio driver components to an offline OS image. The image can then be used to install Windows on your system.
Enabling Test Signing
In case the drivers in the package are not WHQL certified, please follow the steps mentioned below to enable test signing before installing Unified Wire:
i. Open command prompt and run the following command to enable test signing:
C:\Users\Administrator>bcdedit /set testsigning on
ii. Change your working directory to ChelsioUwire_x.x.x.x_WIN_yyy.y.yy/Selfsign/ and run the
following commands:
C:\ ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign> .\certmgr.exe -add .\chelsiocert1.cer -s -r LocalMachine root -all C:\ ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign> .\certmgr.exe -add .\chelsiocert1.cer -s -r LocalMachine trustedpublisher -all
iii. Reboot the machine for the changes to take effect.
A log file, ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.log, is created in the same directory which keeps a record of all the commands executed and their results.
Note
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 14
Drivers location
The driver files should be manually picked from:
ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\drivers\Win10 for Server 2016 and 10 Client.
ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\drivers\Win7 for Server 2012 R2.
Manual
Virtual Bus Driver
i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),
click on System Devices, right click on Chelsio Bus Enumerator and select Update
Driver Software. ii. Select Browse my computer for driver software. iii. Select Let me pick from a list of device drivers on my computer and click Next. iv. Click on Have Disk Button and on the next screen browse for chvbdx64.inf file in the zip
package. Click Open and then OK. v. Click Next and driver installation will progress. Click Close once the installation is complete.
FCoE Storport Miniport driver
i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),
click on System Devices, right click on Chelsio Generic function with function 6 and select
Properties. In case the adapter is not listed, it may appear as Fibre Channel Controller
under Other devices.
Important
The driver components need to be installed strictly in the order described here. Using any other order is not supported.
Before proceeding, open Device Manager and disable NDIS Miniport driver (Network adapters->Chelsio Network Adapter) followed by Virtual Bus Driver (System devices->Chelsio Bus Enumerator).
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 15
ii. Under Driver tab, select Update Driver… iii. Select Browse my computer for driver software. iv. Select Let me pick from a list of device drivers on my computer and click Next. v. Click on Have Disk Button and on the next screen, browse for csiofcoe.inf file in the zip
package. Click Open and then OK.
vi. Click Next and driver installation will progress. Click Close once the installation is complete.
NDIS Miniport Driver
i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),
click on Network Adapters, right click on the Chelsio network adapter and select Update
Driver Software. ii. Select Browse my computer for driver software. iii. Select Let me pick from a list of device drivers on my computer. iv. Click on Have Disk Button and on the next screen browse for chnetx64.inf file in the zip
package. Click Open and then OK. v. Click Next and driver installation will progress. Click Close once the installation is complete.
Repeat the above steps for the other adapters.
Generic Function
i. Open Device Manager (Control Panel -> System & Security-> System -> Device Manager),
click on System Devices, right click on Chelsio Generic Function and select Update
Driver Software. ii. Select Browse my computer for driver software. iii. Select Let me pick from a list of device drivers on my computer. iv. Click on Have Disk Button and on the next screen browse for chnulx64.inf file in the zip
package. Click Open and then OK. v. Click Next and driver installation will progress. Click Close once the installation is complete.
Offline Windows Image
You can use Windows Deployment Image Servicing and Management (DISM) tool to add Chelsio driver components (.inf files) to an offline Windows image. For more information, visit the following links:
http://technet.microsoft.com/en-us/library/hh825070.aspx
http://blogs.technet.com/b/heyscriptingguy/archive/2012/09/27/use-the-powershell-dism-
cmdlets-to-manage-windows-8.aspx
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 16
4. Firmware Update
The driver will auto-load the firmware if an update is required. The firmware version can be verified using cxgbtool:
C:\Users\Administrator>cxgbtool nic0 debug version
In case of installer, the firmware binaries will be copied to <system_drive>\ChelsioUwire \Firmware\ folder during installation.
For zip package, the binaries will be present in ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\firmware\ folder.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 17
5. cxgbtool help
The cxgbtool command queries or sets various aspects of Chelsio network interface cards. It complements standard tools used to configure network settings and provides functionality not available through such tools.
Some of the commands provided can be used to query running statistics to aid in debugging.
Syntax
cxgbtool <vbdIface|nicIface|iSCSIIface|rdma> <category> <command> <options>
Definitions
nicIface is the name of the network device to work on, given in the format nic0, nic1. vbdIface is the name of the Chelsio Bus Enumerator instance to work on, given in the format
vbd0, vbd1 iscsIface is the name of the Chelsio iSCSI interface to work on, given in the format iSCSI0,
iSCSI1 rdma is the name of the Chelsio RDMA interface to work on. fcoeIface is the name of the Chelsio FCoE interface to work on, given in the format
Scsi1,Scsi2 category is one of the cxgbtool categories, i.e., boot, cudbg, debug, firmware, hardware command is the action you wish to perform on the adapter or property you wish to view/change. option is the optional parameter to be provided with command.
Help
o Displaying cxgbtool help: cxgbtool
o Displaying list of available cxgbtool commands: cxgbtool -h all
o Displaying category specific help:
cxgbtool <vbdIface|nicIface|iSCSIIface> <category> -h
o Displaying list of cudbg entities: cxgbtool <vbdIface> cudbg dbg -h
o Displaying RDMA help: cxgbtool rdma -h
o Displaying FCoE help: cxgbtool fcoe -h
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 18
FCoE
Description: Display FCoE information. Syntax: cxgbtool fcoe -a [FCoEAdapterIndex] <options> Examples:
o Scanning for available Chelsio adapters.
o Printing adapter status and parameters.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 19
o Printing adapter statistics.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 20
o Printing physical port statistics of adapter.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 21
o Printing DCB parameters.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 22
o Printing driver parameters.
o Printing SCSI statistics.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 23
o Printing FCFs information connected to the adapter.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 24
o Printing local nodes information.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 25
o Printing local nodes statistics.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 26
o Printing remote nodes information.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 27
o Printing remote nodes statistics.
o Displaying FCoE statistics.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 28
o Displaying internal memory information.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 29
o Displaying firmware debug logs.
o Creating NPIV ports for the parent WWN.
o Deleting NPIV ports for the parent WWN.
o Printing NPIV ports for the parent WWPN.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 30
Debug
version Description: Display adapter information like part number, serial number, device ID, firmware
version, TP, etc.
Syntax: cxgbtool <vbdIface|nicIface|iscsiIface> debug version Example:
inst Description: Install driver package specified in the setup information file (.inf).
Syntax: cxgbtool debug inst <file> Example:
update Description: Update driver package.
Syntax: cxgbtool debug update <file> Example:
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 31
uninst Description: Uninstall driver package specified in the setup information file (.inf).
Syntax: cxgbtool debug uninst <file> Example:
rescan Description: Scan for hardware changes in the device manager.
Syntax: cxgbtool debug rescan Example:
qstats Description: Display statistics for each Tx & Rx queue.
Syntax: cxgbtool <nicIface|iscsiIface> debug qstats <queueType|clr>
clr: Clear queue statistics.
Queue Type
Description
txeth
Tx tunnel queue statistics.
rxeth
Rx tunnel queue statistics.
txvirt
Tx VM queue statistics.
rxvirt
Rx VM queue statistics.
txtoe
Tx Chimney queue statistics.
rxtoe
Rx Chimney queue statistics.
txrdma
Tx RDMA queue statistics.
rxrdma
Rx RDMA queue statistics.
txctrl
Chimney control queue statistics.
txfwd
Chimney forwarding queue statistics.
txnvgre *
Tx NVGRE statistics.
rxnvgre *
Rx NVGRE statistics.
txiscsi
Tx iSCSI queue statistics.
rxiscsi
Rx iSCSI queue statistics.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 32
txpd
Tx PacketDirect queue statistics.
rxpd
Rx PacketDirect queue statistics.
* not supported on T6 adapters
Qstats Parameter
Description
BcBytes
Broadcast packet size in bytes.
BcPkts
Number of broadcast packets.
CoalBytes
Coalesced packet size in bytes.
CoalPkts
Number of coalesced packets.
DropPkts
Number of dropped packets.
EncapBytes
#
NVGRE/VXLAN packet size in bytes.
EncapPkts
#
Number of NVGRE/VXLAN packets.
VlanMismatch
Number of packets with VLAN mismatch.
Fw4Ack
Number of SGE update requests.
LargeRxDropPkts
Dropped packets due to large size counter.
LSO
Number of large send offloaded packets.
McBytes
Multicast packet size in bytes.
McPkts
Number of multicast packets.
NblsRecv
Total Nbls received.
NblsSent
Total Nbls sent.
NoRxBufs
Packets couldn't process because of lack of memory.
NvPkts
Send NVGRE packets.
PeerAbort
Peer abort request.
CqComp
Number of CQ completions.
PktsDefrag
Number of defragged packets.
PktsDrop
Total packets dropped.
PktsQued
Total packets queued.
PktsSent
Total packets sent.
Quefull
Number of time queue full happened.
RecvNbls
Total Nbls received.
RssGroup
RSS group number allocated to queue.
VPortId
Vport ID of queue.
RssProc
RSS processor number allocated to queue.
RxCsumErr
Number of received packets with checksum error.
RxCsumErrIp
Number of received IP packets with checksum error.
RxCsumErrTcp
Number of received TCP packets with checksum error.
SysThreadEvts
Number of system thread events.
TxCsumOfld
Number of checksum offloaded Tx packets.
UcBytes
Unicast packets size in bytes.
UcPkts
Number of unicast packets.
VlanEx
Number of VLAN extracted packets.
VLANin
Number of VLAN inserted packets.
#
supported only on T6 adapters
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 33
Example of rxeth qstats
hardware
sensor Description: Display sensor data.
Syntax: cxgbtool <nicIface> hardware sensor Example:
fec Description: Set/Display Forward Error Correction (FEC) parameters.
Syntax:
cxgbtool <nicIface> hardware fec [<config params>]
Example:
o Displaying currently set FEC mode.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 34
o Setting FEC mode:
link_cfg Description: Set/Display link configuration parameters.
Syntax:
cxgbtool <nicIface> hardware link_cfg <options>
Example:
o Displaying link configuration information:
o Enabling/Disabling auto-negotiation:
Auto-negotiation is enabled by default. To disable it, use the autoneg off parameter on all
connected machines of the link.
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 35
o Changing link speed:
25G port can be set to 10G and 100G port can be set to 25G/40G/50G. Auto-negotiation
should be disabled on all connected machines while changing link speed.
cudbg
collect Description: Collect Chelsio adapter debug logs to a compressed file.
Syntax: cxgbtool <vbdIface> cudbg collect <entities> <outputFile> <options> Examples:
o Collecting debug log for all entities
o Collecting debug log skipping specific debug entities
o Collecting debug log avoiding entities that can affect running traffic
Before setting 40G or 10G speeds, FEC should be disabled on the port using:
C:\Users\Administrator>cxgbtool nic0 hardware fec off
Important
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 36
6. Software/Driver Uninstallation
Similar to installation, Chelsio Unified Wire can be uninstalled using the Installer or zip package. Refer the relevant section depending on the method of installation used to install drivers.
6.1. Installer
Chelsio Unified Wire Installer can be uninstalled using two methods: GUI or CLI mode. GUI mode requires user interaction and uninstallation occurs with options specified by the user. Whereas, CLI mode does not require any user input.
GUI mode (Installer)
i. Run the ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe application. ii. Select Modify to add or remove features. Select Repair to repair the previous installation.
Select Remove to uninstall the application. After you have selected the appropriate option,
click Next.
Figure 7 - Adding or removing features using Modify option
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 37
Figure 8 - Repairing previous installation using Repair option
Figure 9 – Uninstalling Unified Wire using ‘Remove’ option
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 38
iii. Click on the Finish button to exit from the installer.
Figure 10 - Finishing uninstallation
CLI Mode
To uninstall FCoE Storport Miniport driver, execute the following command:
C:\Users\Administrator> ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe –un all
This method of uninstallation is possible only if the drivers were installed using Unified Wire Installer.
Note
Chapter I. Chelsio Unified Wire
Chelsio Unified Wire for Windows 39
6.2. Zip Package
If the driver components were installed using the zip package, they will have to be uninstalled manually. The following section lists the various components and their respective methods of uninstallation.
NDIS Miniport driver
i. Open the Device Manager (Control Panel -> System & Security-> System -> Device
Manager), click on the Network Adapters, right click on the Chelsio Network Adapter and
select Uninstall. ii. Select the Delete the driver software for this device checkbox when Confirm Device
Uninstall is prompted and click OK. Repeat the same procedure for the other Chelsio Network Adapters.
FCoE Storport Miniport
Open the Device Manager (Control Panel -> System & Security-> System -> Device Manager), click on Storage controllers, right-click on Chelsio [FCoE] function and select Uninstall.
Virtual Bus Driver
i. Open the Device Manager (Control Panel -> System & Security-> System -> Device
Manager), click on the System Devices, right click on the Chelsio Bus Enumerator and
select Uninstall. ii. Select the Delete the driver software for this device checkbox when Confirm Device
Uninstall is prompted and click OK.
Generic Function
i. Open the Device Manager (Control Panel -> System & Security-> System -> Device
Manager), click on the Network Adapters, right click on the Chelsio Generic Function and
select Uninstall. ii. Select the Delete the driver software for this device checkbox when Confirm Device
Uninstall is prompted and click OK. Repeat the same procedure for the other Generic Functions.
Important
The driver components need to be uninstalled strictly in the order described here. Using any other order is not supported.
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 40
II. FCoE Storport Miniport
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 41
1. Introduction
Fibre Channel over Ethernet (FCoE) is a mapping of Fibre Channel over selected full duplex IEEE
802.3 networks. The goal is to provide I/O consolidation over Ethernet, reducing network complexity in the Datacenter.
Chelsio FCoE initiator maps Fibre Channel directly over Ethernet while being independent of the Ethernet forwarding scheme. The FCoE protocol specification replaces the FC0 and FC1 layers of the Fibre Channel stack with Ethernet. By retaining the native Fibre Channel constructs, FCoE will integrate with existing Fibre Channel networks and management software.
1.1. Hardware Requirements
Supported Adapters
The following are the currently shipping Chelsio adapters that are compatible with Chelsio FCoE storport miniport driver:
T6225-CR T6225-LL-CR T520-CR T520-LL-CR T540-CR
1.2. Software Requirements
Windows Requirement
Currently Chelsio FCoE Storport Miniport driver is available for following Windows versions:
Server 2016 10 Client Server 2012 R2
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 42
2. Configuration and Fine-tuning
2.1. Configuring Cisco Nexus 5010 switch
Configuring the DCBX parameters
In this procedure, you may need to adjust some of the parameters to suit your environment, such as VLAN IDs, Ethernet interfaces, and virtual Fibre Channel interfaces.
To enable PFC, ETS, and DCB functions on a Cisco Nexus 5000 series switch: i. Open a terminal configuration setting.
Switch# config terminal switch(config)#
ii. Configure qos class-maps and set the traffic priorities: NIC uses priority 0 and FCoE uses
priority 3.
Switch(config)#class-map type qos class-nic Switch(config-cmap-qos)# match cos 0 Switch(config-cmap-qos)# class-map type qos class-fcoe
Switch(config-cmap-qos)# match cos 3
iii. Configure queuing class-maps.
Switch(config)#class-map type queuing class-nic
Switch(config-cmap-que)#match qos-group 2
By default, the Cisco Nexus switch enables DCBX functionality and configures PFC for FCoE traffic making it no drop with bandwidth of 50% assigned to FCoE class of traffic and another 50% for the rest (like NIC). If you wish to configure custom bandwidth, then follow the procedure below.
Note
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 43
iv. Configure network-qos class-maps.
Switch(config)#class-map type network-qos class-nic
Switch(config-cmap-nq)#match qos-group
v. Configure qos policy-maps.
Switch(config)#policy-map type qos policy-test Switch(config-pmap-qos)#class type qos class-nic
Switch(config-pmap-c-qos)#set qos-group 2
vi. Configure queuing policy-maps and assign network bandwidth. Divide the network
bandwidth between FCoE and NIC traffic.
Switch(config)#policy-map type queuing policy-test Switch(config-pmap-que)#class type queuing class-nic Switch(config-pmap-c-que)#bandwidth percent 50 Switch(config-pmap-c-que)#class type queuing class-fcoe Switch(config-pmap-c-que)#bandwidth percent 50 Switch(config-pmap-c-que)#class type queuing class-default
Switch(config-pmap-c-que)#bandwidth percent 0
vii. Configure network-qos policy maps and set up the PFC for no-drop traffic class.
Switch(config)#policy-map type network-qos policy-test Switch (config-pmap-nq)#class type network-qos class-nic
Switch(config-pmap-nq-c)#pause no-drop
viii. Apply the new policy (PFC on NIC and FCoE traffic) to the entire system.
Switch(config)#system qos Switch(config-sys-qos)#service-policy type qos input policy-test Switch(config-sys-qos)#service-policy type queuing output policy-test Switch(config-sys-qos)#service-policy type queuing input policy-test
Switch(config-sys-qos)#service-policy type network-qos policy-test
By default, FCoE is set to pause no drop. In such a tradeoff, one may want to set NIC to drop instead.
Note
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 44
Configuring the FCoE/FC ports
In this procedure, you may need to adjust some of the parameters to suit your environment, such as VLAN IDs, Ethernet interfaces, and virtual Fibre Channel interfaces
i. Following steps will enable FCoE services on a particular VLAN and does a VSAN-VLAN
mapping. These steps are not required to be done every time, unless a new mapping is
required.
Switch(config)# vlan 2 Switch(config-vlan)# fcoe vsan 2
Switch(config-vlan)#exit
ii. Following steps help in creating a virtual fibre channel(VFC) and binds that VFC to an
Ethernet interface so that the Ethernet port begins functioning as a FCoE port.
Switch(config)# interface vfc 13 Switch(config-if)#bind interface ethernet 1/13 Switch(config-if)# no shutdown Switch(config-if)#exit Switch(config)#vsan database Switch(config-vsan-db)# vsan 2 Switch(config-vsan-db)#vsan 2 interface vfc 13
Switch(config-vsan-db)#exit
iii. Assign VLAN ID to the Ethernet port on which FCoE service was enabled in step1.
Switch(config)# interface ethernet 1/13 Switch(config-if)#switchport mode trunk Switch(config-if)#switchport trunk allowed vlan 2 Switch(config-if)#no shutdown
Switch(config)#exit
If you are binding the VFC to a MAC address instead of an Ethernet port then make sure the Ethernet port is part of both default VLAN and FCoE VLAN
Note
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 45
iv. On the FC Ports, if a FC target is connected then perform the following steps -
Switch(config)#vsan database Switch(config-vsan-db)#vsan 2 Switch(config-vsan-db)# vsan 2 interface fc 2/2 Switch(config-vsan-db)#exit Switch(config)interface fc 2/2 Switch(config-if)# switchport mode auto Switch(config-if)# switchport speed auto
Switch(config-if)# no shutdown.
v. If you have not created a zone then make sure the default-zone permits the VSAN created,
otherwise the initiator and the target on that particular VSAN although FLOGI’d into the
switch will not talk to each other. To enable it, execute the below command.
Switch(config)# zone default-zone permit vsan 2
2.2. FCoE fabric discovery verification
Verifying Target Discovery
To check the summary of targets being discovered from FCoE ports, open device manger and follow the steps given below:
i. Go to Start->Control Panel->Device Manager or launch device manager directly from
Start->Run->devmgmt.msc and hit [Enter]. ii. Go to Disk drives section under which you will find the list of all the LUNS that were
discovered.
Figure 11 - Discovered Target LUNs
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 46
iii. To bring the LUNs online, launch disk management by going to Start->Run-
>diskmgmt.msc. Right click on the LUN and select online.
Figure 12 - Bringing LUNs online
iv. Now format the LUN by right clicking on it and choosing any of the volume types. In this
example, we will choose New Simple Volume. This will invoke the New Simple Volume
Wizard.
Figure 13 - Formatting LUNs
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 47
v. Enter the volume size (the entire volume size is selected by default), and in the subsequent
windows assign a driver letter to the volume and choose the file system.
Figure 14 - Specifying Volume size
Figure 15 - Assigning drive letter or path
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 48
Figure 16 - Formatting the partition
vi. Click Finish to close the wizard.
Figure 17 - Completing Simple Volume Wizard
Chapter II. FCoE Storport Miniport
Chelsio Unified Wire for Windows 49
vii. The volume should appear in My Computer.
Figure 18 - The newly created volume in My Computer
Troubleshooting
i. Always enable LLDP on the interfaces as there won’t be FCoE link up until and unless a
successful LLDP negotiation happens. ii. On switch, make sure priority-flow-control mode is always set to auto and flowcontrol is
disabled. iii. Always configure the Ethernet interfaces on cisco switch in trunk mode. iv. If you are binding the VFC to MAC address in case of cisco nexus, then make sure you
make the Ethernet interface part of both Ethernet VLAN and FCoE VLAN. v. If in any case the switch-port on the cisco nexus switch is reporting
“pauseRateLimitErrDisable”, then perform an Ethernet port shut/no shut. Also, note that only one FCoE function will be displayed under the storage controllers of the
device manager, irrespective of the number of ports a card possesses. All ports will be using the same interface.
Chapter III. Appendix
Chelsio Unified Wire for Windows 50
III. Appendix
Chapter III. Appendix
Chelsio Unified Wire for Windows 51
1. Troubleshooting
Installer issues
In case of any failures while running the Chelsio Unified Wire Installer, please collect the below:
Screenshot of the failure
Installer Log file from C:\Windows\Temp\ChelsioUwire_x.x.x.x_WIN_y.y.y_devlog.log
Logs collection
In case of any driver/firmware issues, follow the below procedure to collect the standard logs:
Download the script from https://github.com/Microsoft/SDN/blob/master/Diagnostics/Get-
NetView.PS1
Run the script and the logs will be collected in a zip package.
PS C:\Users\Administrator\Desktop> .\Get-NetView.PS1
Please contact Chelsio support at support@chelsio.com with relevant logs for any issues.
Chapter III. Appendix
Chelsio Unified Wire for Windows 52
2. Chelsio End-User License Agreement (EULA)
Installation and use of the driver/software implies acceptance of the terms in the Chelsio End­User License Agreement (EULA).
IMPORTANT: PLEASE READ THIS SOFTWARE LICENSE CAREFULLY BEFORE DOWNLOADING OR OTHERWISE USING THE SOFTWARE OR ANY ASSOCIATED DOCUMENTATION OR OTHER MATERIALS (COLLECTIVELY, THE "SOFTWARE"). BY CLICKING ON THE "OK" OR "ACCEPT" BUTTON YOU AGREE TO BE BOUND BY THE TERMS OF THIS AGREEMENT. IF YOU DO NOT AGREE TO THE TERMS OF THIS AGREEMENT, CLICK THE "DO NOT ACCEPT" BUTTON TO TERMINATE THE INSTALLATION PROCESS.
1. License. Chelsio Communications, Inc. ("Chelsio") hereby grants you, the Licensee, and you hereby accept, a limited, non-exclusive, non-transferable license to install and use the Software with one or more Chelsio network adapters on a single server computer for use in communicating with one or more other computers over a network. You may also make one copy of the Software in machine readable form solely for back-up purposes, provided you reproduce Chelsio's copyright notice and any proprietary legends included with the Software or as otherwise required by Chelsio.
2. Restrictions. This license granted hereunder does not constitute a sale of the Software or any copy thereof. Except as expressly permitted under this Agreement, you may not:
(i) reproduce, modify, adapt, translate, rent, lease, loan, resell, distribute, or create derivative works of or based upon, the Software or any part thereof; or
(ii) make available the Software, or any portion thereof, in any form, on the Internet. The Software contains trade secrets and, in order to protect them, you may not decompile, reverse engineer, disassemble, or otherwise reduce the Software to a human-perceivable form. You assume full responsibility for the use of the Software and agree to use the Software legally and responsibly.
3. Ownership of Software. As Licensee, you own only the media upon which the Software is recorded or fixed, but Chelsio retains all right, title and interest in and to the Software and all subsequent copies of the Software, regardless of the form or media in or on which the Software may be embedded.
4. Confidentiality. You agree to maintain the Software in confidence and not to disclose the Software, or any information or materials related thereto, to any third party without the express written consent of Chelsio. You further agree to take all reasonable precautions to limit access of the Software only to those of your employees who reasonably require such access to perform their employment obligations and who are bound by confidentiality agreements with you.
5. Term. This license is effective in perpetuity, unless terminated earlier. You may terminate the license at any time by destroying the Software (including the related documentation), together with all copies or modifications in any form. Chelsio may terminate this license, and this license shall be deemed to have automatically terminated, if you fail to comply with any term or condition of this Agreement. Upon any termination, including termination by you, you must destroy the Software (including the related documentation), together with all copies or modifications in any form.
6. Limited Warranty. If Chelsio furnishes the Software to you on media, Chelsio warrants only that the media upon which the Software is furnished will be free from defects in
Chapter III. Appendix
Chelsio Unified Wire for Windows 53
material or workmanship under normal use and service for a period of thirty (30) days from the date of delivery to you.
CHELSIO DOES NOT AND CANNOT WARRANT THE PERFORMANCE OR RESULTS YOU MAY OBTAIN BY USING THE SOFTWARE OR ANY PART THEREOF. EXCEPT FOR THE FOREGOING LIMITED WARRANTY, CHELSIO MAKES NO OTHER WARRANTIES, EXPRESS OR IMPLIED, AND HEREBY DISCLAIMS ALL OTHER WARRANTIES, INCLUDING, BUT NOT LIMITED TO, NON-INFRINGEMENT OF THIRD PARTY RIGHTS, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow the exclusion of implied warranties or limitations on how long an implied warranty may last, so the above limitations may not apply to you. This warranty gives you specific legal rights and you may also have other rights which vary from state to state.
7. Remedy for Breach of Warranty. The sole and exclusive liability of Chelsio and its distributors, and your sole and exclusive remedy, for a breach of the above warranty, shall be the replacement of any media furnished by Chelsio not meeting the above limited warranty and which is returned to Chelsio. If Chelsio or its distributor is unable to deliver replacement media which is free from defects in materials or workmanship, you may terminate this Agreement by returning the Software.
8. Limitation of Liability. IN NO EVENT SHALL CHELSIO HAVE ANY LIABILITY TO YOU OR ANY THIRD PARTY FOR ANY INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR PUNITIVE DAMAGES, HOWEVER CAUSED, AND ON ANY THEORY OF LIABILITY, ARISING OUT OF OR RELATED TO THE LICENSE OR USE OF THE SOFTWARE, INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR LOSS OF ANTICIPATED PROFITS, EVEN IF CHELSIO HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. IN NO EVENT SHALL CHELSIO'S LIABILITY ARISING OUT OF OR RELATED TO THE LICENSE OR USE OF THE SOFTWARE EXCEED THE AMOUNTS PAID BY YOU FOR THE LICENSE GRANTED HEREUNDER. THESE LIMITATIONS SHALL APPLY NOTWITHSTANDING ANY FAILURE OF ESSENTIAL PURPOSE OF ANY LIMITED REMEDY.
9. High Risk Activities. The Software is not fault-tolerant and is not designed, manufactured or intended for use or resale as online equipment control equipment in hazardous environments requiring fail-safe performance, such as in the operation of nuclear facilities, aircraft navigation or communication systems, air traffic control, direct life support machines, or weapons systems, in which the failure of the Software could lead directly to death, personal injury, or severe physical or environmental damage. Chelsio specifically disclaims any express or implied warranty of fitness for any high risk uses listed above.
10. Export. You acknowledge that the Software is of U.S. origin and subject to U.S. export jurisdiction. You acknowledge that the laws and regulations of the United States and other countries may restrict the export and re-export of the Software. You agree that you will not export or re-export the Software or documentation in any form in violation of applicable United States and foreign law. You agree to comply with all applicable international and national laws that apply to the Software, including the U.S.
Export Administration Regulations, as well as end-user, end-use, and destination restrictions issued by U.S. and other governments.
11. Government Restricted Rights. The Software is subject to restricted rights as follows. If the Software is acquired under the terms of a GSA contract: use, reproduction or disclosure is subject to the restrictions set forth in the applicable ADP Schedule contract. If the Software is acquired under the terms of a DoD or civilian agency contract, use, duplication or disclosure by the Government is subject to the restrictions of this Agreement in accordance with 48 C.F.R. 12.212 of the Federal
Chapter III. Appendix
Chelsio Unified Wire for Windows 54
Acquisition Regulations and its successors and 49 C.F.R. 227.7202-1 of the DoD FAR Supplement and its successors.
12. General. You acknowledge that you have read this Agreement, understand it, and that by using the Software you agree to be bound by its terms and conditions. You further agree that it is the complete and exclusive statement of the agreement between Chelsio and you, and supersedes any proposal or prior agreement, oral or written, and any other communication between Chelsio and you relating to the subject matter of this Agreement. No additional or any different terms will be enforceable against Chelsio unless Chelsio gives its express consent, including an express waiver of the terms of this Agreement, in writing signed by an officer of Chelsio. This Agreement shall be governed by California law, except as to copyright matters, which are covered by Federal law. You hereby irrevocably submit to the personal jurisdiction of, and irrevocably waive objection to the laying of venue (including a waiver of any argument of forum non conveniens or other principles of like effect) in, the state and federal courts located in Santa Clara County, California, for the purposes of any litigation undertaken in connection with this Agreement. Should any provision of this Agreement be declared unenforceable in any jurisdiction, then such provision shall be deemed severable from this Agreement and shall not affect the remainder hereof. All rights in the Software not specifically granted in this Agreement are reserved by Chelsio. You may not assign or transfer this Agreement (by merger, operation of law or in any other manner) without the prior written consent of Chelsio and any attempt to do so without such consent shall be void and shall constitute a material breach of this Agreement.
Should you have any questions concerning this Agreement, you may contact Chelsio by writing to:
Chelsio Communications, Inc. 209 North Fair Oaks Avenue, Sunnyvale, CA 94085 U.S.A
Loading...