warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP
shall not be liable for technical or editorial errors or omissions contained herein.
Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212,
Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S.
Government under vendor’s standard commercial license.
Intended audience
This document is for the person who installs, administers, and troubleshoots servers and storage systems.
HP assumes you are qualified in the servicing of computer equipment and trained in recognizing hazards
in products with hazardous energy levels.
The HP iSCSI protocol transports block-level storage requests of TCP connections over IPv4 and IPv6.
Using the iSCSI protocol, systems can connect to remote storage and use it as a physical disk although
the remote storage provider or target may actually be providing virtual physical disks. HP Accelerated
iSCSI supports all targets that are certified and supported by Linux software.
HP Gigabit Ethernet Multifunction Server Adapters can be enabled as iSCSI devices or virtual iSCSI host
bus adapters (HBAs), a feature commonly referred to as accelerated iSCSI. iSCSI HBAs or accelerated
adapters, provide the capability to run iSCSI protocol over offloaded TCP connections. By offloading the
IPv4 or IPv6 TCP connection processing to an HP Gigabit Ethernet Multifunction Server Adapter, CPU
utilization is reduced allowing additional processing capacity for applications. The HP accelerated iSCSI
solution supports all targets that are certified and supported by Linux software.
Supported operating systems
• Red Hat Enterprise Linux 5 AS/ES for AMD64/EM64T
• Red Hat Enterprise Linux 5 AS/ES for x86
• SUSE LINUX Enterprise Server 10 (SP1, SP2) for x86
• SUSE LINUX Enterprise Server 10 (SP1, SP2) for AMD64
Supported hardware
Servers. Server support includes all ProLiant servers that include an iLO or RiLO management device with
any of the supported operating systems.
Adapters. HP adapter support includes the following:
• HP NC370T PCI-X Multifunction Gigabit Server Adapter
• HP NC370F PCI-X Multifunction Gigabit Server Adapter
• HP NC370i PCI-X Multifunction Gigabit Server Adapter *
• HP NC371i PCI-X Multifunction Gigabit Server Adapter*
• HP NC373F PCI Express Multifunction Gigabit Server Adapter
• HP NC373T PCI Express Multifunction Gigabit Server Adapter
• HP NC373m PCI Express Dual Port Multifunction Gigabit Server Adapter for c-Class BladeSystem
• HP NC373i Integrated Multifunction Gigabit Server Adapter*
• HP NC374m PCI Express Dual Port Multifunction Gigabit Server Adapter
• HP NC380T PCI Express Dual Port Multifunction Gigabit Server Adapter
Overview 4
• HP NC382T PCI Express Dual Port Multifunction Gigabit Server Adapter
• HP NC382m Dual Port 1GbE Multifunction BL-c Adapter
• HP NC382i Integrated Quad Port PCI Express Gigabit Server Adapter
• HP NC532m Dual Port Flex-10 10GbE Multifunction BL-c Adapter
• HP NC532i Dual Port 10GbE Multifunction BL-c Adapter*
* Requires an HP ProLiant Accelerated iSCSI license to enable acceleration
iSCSI targets. All iSCSI targets supported by the Linux software initiator are supported by HP.
Overview 5
Installation and configuration
Downloading files
1. Go to the HP website (http://www.hp.com/#Support).
2. Select Download drivers and software (and firmware).
3. Type the adapter name in the For product box, and then click Go. For example, NC382T.
4. Select an operating system.
5. Click HP NC-Series iSCSI Offload Driver for Linux.
6. Click download and save the HP SoftPaq (sp#####.exe) file to a directory on your hard drive. The
SoftPaq file is a self-extracting executable with a file name based on the SoftPaq number.
7. Click the SoftPaq file to extract the files.
Installing the BRCM Multifunction drivers for Linux
The HP NC-Series Broadcom Multifunction Drivers for Linux are released in a source RPM format and the
file name for the package is netxtreme2-<version>.sri.rpm. The package contains the source code and
build instructions for bnx2, bnx2x, and cnic drivers.
1. Build Environment Setup. The netxtreme2 package requires the presence of packages that provide
the right build environment. These packages provide kernel headers, makefiles, and symbol files
(suggested only on SLES distributions).
Red Hat installations:
For each kernel flavor you want to build the driver make sure you have the associated kernel-devel
package. For example, kernel-2.6.18.92.el5 requires kernel-devel-2.6.18-92.el5 and kernel-PAE-
2.6.18.92.el5 requires kernel-PAE-devel-2.6.18-92.el5.
SUSE SLES installations:
Make sure that kernel-source and kernel-syms packages are installed. The kernel-syms package
contains symbol versions for all of the iterations that the distribution supports.
2. Install the source RPM package.
# rpm -ivh netxtreme2-<version>.src.rpm
3. Build the binary RPM for the netxtreme2.
Red Hat installations:
# cd /usr/src/redhat
# rpmbuild -bb SPECS/netxtreme2.spec
SLES installations:
# cd /usr/src/packages
# rpmbuild -bb SPECS/netxtreme2.spec
If you get an error during the build process, go back to step one above.
4. Check for the existence of a current version of the netxtreme2 package as follows:
# rpm -q netxtreme2
Installation and configuration 6
If an old version of the package exists, the RPM package should be removed.
# rpm -e netxtreme2
5. Verify that the old netxtreme2 package has been removed as follows:
# rpm -q netxtreme2
6. Verify if a old version of bnx2 or bnx2x is loaded into the kernel and remove it, if needed, as
follows:
If you are installing the package over the network, it is not necessary to remove the modules. This
step is only necessary if you are doing the upgrade on the physical console and you want the new
drivers to take over immediately.
#lsmod | grep bnx2x
#lsmod | grep bnx2
7. Before removing the drivers, bring down the interfaces controlled by bnx2x and bnx2.
#ifconfig eth<n> down
8. Remove the drivers.
#rmmod bnx2x
#rmmod bnx2
9. Install the new RPM package. This installs bnx2/bnx2x and cnic drivers. Do not use the U (Upgrade)
flag in the RPM command below--previous netxtreme2 packages must be removed as described
above before installing this version.
The bnx2.ko, bnx2x.ko and cnic.ko modules are installed in the following directory:
Red Hat EL 4/5 installations:
/lib/modules/<kernel_version>/updates
SLES 9/10 installations:
/lib/modules/<kernel_version>/updates
10. Configure your network setting and address. See your Linux vendor documentation for more
information. Helpful network configuration tools such as "yast2" in SLES or linuxconf/redhat-confignetwork/netconfig in Red Hat exist for easy configuration.
11. Ensure that the /etc/modules.conf file is configured similar to the example listed below. The example
below is presented as if more than one adapter is present. If so, one eth# instance should exist for
each ethernet port. See the modules.conf man page for more information.
If you have a 1Gb Network card use bnx2 below otherwise use bnx2x for 10Gb.
alias eth0 bnx2
alias eth1 bnx2
For SLES, the configuration file is /etc/modprobe.conf or /etc/modprobe.conf.local
12. You can now reboot your server or restart the network services. Upon reboot the network should start
with either bnx2 or bnx2x driver loaded.
To verify that the driver is loaded use the following command.
# lsmod
You should find bnx2 or bnx2x listed. You can also verify that the correct bnx2 or bnx2x driver is
loaded through any of the following methods. Note that version of the driver loaded should be same
as that of the package version.
Installing the Linux iSCSI offload bnx2i driver
The Linux iSCSI offload bnx2i driver is released in a source RPM format.
Installation and configuration 7
1. To install the source RPM package, change to the linux\iscsi-offload directory on your hard drive
and use the following command:
# rpm -ivh bnx2i-<version>.src.rpm
2. Build the binary RPM for the bnx2i driver.
For Red Hat EL 5 installations enter the following commands:
# cd /usr/src/redhat
# rpmbuild -bb SPECS/bnx2i.spec
For SLES installations enter the following commands:
# cd /usr/src/packages
# rpmbuild -bb SPECS/bnx2i.spec
3. Check for the existence of a current version of the bnx2i package using the following command:
# rpm -q bnx2i
If an older version of the package exists, remove the old RPM package using the following
command:
# rpm -e bnx2i
Remove the loaded bnx2i module using the following command:
# modprobe -r bnx2i
This command removes the bnx2i module along with its dependencies.
Verify that the old bnx2i package has been removed using the following command:
# rpm -q bnx2i
4. Install the bnx2i binary RPM package using the following command:
# rpm -ivh RPMS/<arch>/bnx2i-<version>.<arch>.rpm
The bnx2i.ko module is installed in the following directory:
• The current version of the driver has been tested on 2.6.x kernels beginning with the 2.6.18 kernel.
The driver may not compile on kernels older than the 2.6.18 kernel. SLES10 SP1 is an exception. It
also runs on the 2.6.16.46 kernel because SUSE upgraded the iscsi_transport kernel module to
support offload initiators in SLES10 SP1.
• When installing SLES 10 SP2 select to enable IPv6 so that autoload is enabled for the IPv6 kernel
module. The cnic driver which is delivered in the NetXtreme2 RPM and is required for iSCSI offload
will fail to resolve symbols if IPv6 is disabled because it has a dependancy on the IPv6 module.
Uninstalling the HP Linux iSCSI offload bnx2i driver RPM
package
If you need to uninstall the HP bnx2 RPM, use the following command:
# rpm -e bnx2i-<version>
Linux iSCSI offload bnx2i driver messages
Linux iSCSI offload bnx2i driver messages are logged in the file /var/log/messages.
Use the following command to control the level that messages appear on the console:
dmesg -n <level>
Installation and configuration 8
Most systems are set to level 6 by default. To include debug-level messages, set the level to 7.
The following sample messages are the most common bnx2i driver messages that may be logged in the
file /var/log/messages.
•bnx2i driver signon
Broadcom NetXtreme II iSCSI Driver bnx2i v1.0.30 (Sep 29, 2007)
• Driver completes handshake with iSCSI offload enabled CNIC device
bnx2i [05:00.00]: ISCSI_INIT passed
This message displays only when a user attempts to make an iSCSI connection.
•Driver detects iSCSI Offload is not enabled on the CNIC device
bnx2i: iSCSI not supported, dev=eth3
bnx2i: bnx2i: LOM is not enabled to offload iSCSI connections, dev=eth0
• Driver unable to allocate TCP port for iSCSI connection
bnx2i: run 'bnx2id' daemon to alloc TCP ports
• Exceeds maximum allowed iSCSI connection offload limit
bnx2i: unable to allocate iSCSI context resources
• Network route to target node and transport name binding are 2 different devices
bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba
$USER_CHOSEN_HBA
where ROUTE_HBA --> net device on which connection was offloaded based
on route information
USER_CHOSEN_HBA --> HBA to which target node is bound (using iSCSI
transport name)
• Target cannot be reached on any of CNIC devices
bnx2i: check route, can't connect using cnic
• Network route is assigned to network interface that is down
The HP Accelerated iSCSI capability for Linux systems is available as a licensable feature for most
embedded HP Multifunction Gigabit Server Adapters through the purchase of an HP ProLiant Accelerated
iSCSI port-based license. License keys are installed on a per-port basis for each embedded adapter. See
the list of supported adapters to see if your adapter requires a license.
This license is not required for standup and mezzanine HP Multifunction Gigabit Server Adapters and HP
Multifunction Gigabit Server Adapters for HP BladeSystem servers.
Installation and configuration 10
Red Hat
Installing the license key
To install the license key on Red Hat Enterprise Linux systems,
1. Navigate to the \linux\hp-pel\hp-pel-<version>.rhel5.linux.rpm file.
2. Use the following command to install the HP ProLiant Licensing RPM:
rpm -Uvh hp-pel-<version>.rhel5.linux.rpm
3. Add the license key using the following command:
/opt/hp/hp-pel/nalicense -a license_string
If you have an old license and want to add a new license, just add the new license. You cannot
delete the old license.
4. Check the status of all currently installed licenses on the system using the following command:
/opt/hp/hp-pel/nalicense -d [-l <logfile>]
Include the -l parameter if you want all the information that normally prints to stdout to be redirected
Installing the port-based license utility
to the file specified with <logfile>.
To install the port-based Linux iSCSI Offload Utility on Red Hat Enterprise Linux systems,
1. Navigate to the \linux\hp-pbl\hp-pbl-<version>.rhel5.linux.rpm file.
2. Use the following command to install the utility:
rpm -Uvh hp-pbl-<version>.rhel5.linux.rpm
3. The hp-pbl utility is configured to start on system boot-up; however, you can start it immediately after
installation without rebooting using the following command:
/etc/init.d/hp-pbl start
4. The hp-pbl configuration file is a text file located at etc/opt/hp/hp-pbl/hp-pbl.conf. The format of
the configurable MAC parameter is documented in the hp-pbl.conf file. If changes are made to the
hp-pbl.conf configuration file, you must restart hp-pbl using the following command:
/etc/init.d/hp-pbl restart
5. To manually stop the hp-pbl daemon, use the following command:
/etc/init.d/hp-pbl stop
SLES
Installing the license key
To install the license key on SUSE Linux Enterprise Server systems,
1. Navigate to the \linux\hp-pel\hp-pel-<version>.sles10.linux.rpm file.
2. Use the following command to install the HP ProLiant Licensing RPM:
rpm -Uvh hp-pel-<version>.sles10.linux.rpm
3. Add the license key using the following command:
/opt/hp/hp-pel/nalicense -a license_string
If you have an old license and want to add a new license, just add the new license. You cannot
delete the old license.
4. Check the status of all currently installed licenses on the system using the following command:
Installation and configuration 11
/opt/hp/hp-pel/nalicense -d [-l <logfile>]
Include the -l parameter if you want all the information that normally prints to stdout to be redirected
to the file specified with <logfile>.
Installing the port-based license utility
To install the HP Linux iSCSI Offload Utility on SUSE Linux Enterprise Server systems,
1. Navigate to the \linux\hp-pbl\hp-pbl-<version>.sles10.linux.rpm file.
2. Use the following command to install the utility:
rpm -Uvh hp-pbl-<version>.<distro>.linux.rpm
3. The hp-pbl utility is configured to start on system boot-up; however, you can start it immediately after
installation without rebooting using the following command:
/etc/init.d/hp-pbl start
4. The hp-pbl configuration file is a text file located at etc/opt/hp/hp-pbl/hp-pbl.conf. The format of
the configurable MAC parameter is documented in the hp-pbl.conf file. If changes are made to the
hp-pbl.conf configuration file, you must restart hp-pbl using the following command:
/etc/init.d/hp-pbl restart
5. To manually stop the hp-pbl daemon, use the following command:
/etc/init.d/hp-pbl stop
Reporting iSCSI Offload status
To obtain the iSCSI offload status of embedded adapters, use the following command:
o iSCSI licenses available reports the number of iSCSI offload licenses available.
o iSCSI licenses used reports the number of iSCSI offload licenses enabled.
o LOM is the iSCSI capable LOM MACs. If the MAC is not an iSCSI capable LOM, it is not
displayed with the status command. Error messages are entered in the hp-pbl.log file.
o Configured reports YES when the iSCSI capable LOM MAC is entered in the hp-p bl.conf file
and is reported as NO if it was not entered in the hp-pbl.conf file. An error message is entered in
the hp-pbl.log file if the iSCSI capable LOM MAC is not entered correctly in the hp-pbl.conf file.
o Enabled reports YES when the iSCSI capable LOM MAC is currently enabled for iSCSI offload
and reports NO when it is not enabled.
HP ProLiant Licensing Log and Event File
All messages are logged to /var/opt/hp/hp-pbl/hp-pbl.log.
Log messages are in the following format:
[TAG] SP [DATE] SP TEXT TAG
Where TAG is one of:
ERROR indicates an error such as a missing hp-pbl.conf file.
Installation and configuration 12
WARNING indicates a warning such as too many MACs configured for iSCSI offload.
INFO indicates an informative event such as hp-pbl starting and stopping.
SP is one or more spaces.
DATE is the current date stamp in the format Sat Feb 17 12:54:35 CST 2007.
TEXT is free form text that may or may not exist in every message. Lines that do not start with a tag
are a continuation of the previous line.
The following represent sample text lines.
[INFO] [Thu Feb 10 10:34:15 CST 2007] hp-pbl started - v1.0.0-1.rhel5
[INFO] [Thu Feb 10 12:54:36 CST 2007]
first text line of second info message
second text line of second info message
Troubleshoot known issues for HP port-based Linux iSCSI offload
Complete the following steps to troubleshoot the hp-pbl licensing.
1. Run the following command in a separate window to monitor the hp-pbl.log file for messages.
tail -f /var/opt/hp/hp-pbl/hp-pbl.log
2. Run the following command in a separate window to monitor the messages file.
tail -f /var/log/messages
Potential problems, causes and solutions
Problem:
Does not appear to create an iSCSI offloaded connection when configured for offload.
Possible cause:
The initiator needs to be restarted after the transport name is changed for offload, and the bnx2i driver
needs to be reloaded so that the configuration changes are realized. Both of these events occur when hppbl is restarted.
Possible solution:
Run the following command to restart hp-pbl:
/opt/hp/hp-pbl restart
Problem:
Does not appear to create an iSCSI offloaded connection when configured for offload and
/var/log/messages lists the following information:
bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba
$USER_CHOSEN_HBA
Possible cause:
A potential problem with the link layer address to IP address mapping occurs when a server with multiple
network cards is connected to the same network segment. This issue is commonly known as the Arp Flux
Problem.
Possible solution:
Set up the network so that network cards are on separate network segments or broadcast domains.
Installation and configuration 13
Uninstalling the license for HP port-based Linux iSCSI
offload
To uninstall the licensing for HP port-based Linux iSCSI offload utility, remove the HP ProLiant licensing
Removing the HP ProLiant Licensing RPM
RPM and the Linux iSCSI offload utility package.
The database containing ProLiant licenses is deleted if the HP ProLiant Licensing RPM package is
removed. Make sure your hard-copy ProLiant packaging license strings are available before removing this
package.
You can obtain the license strings by displaying all licenses on the system using the check the status of all
currently installed licenses on the system command:
/opt/hp/hp-pel/nalicense -d [-l <logfile>]
Include the -l parameter if you want all the information that normally prints to stdout to be redirected
to the file specified with <logfile>.
Record the license strings and keep them for reference.
To remove the HP PEL package, run the following command:
rpm -e hp-pel
Removing the hp-pbl RPM package
Remove the hp-pbl RPM package using the following command:
rpm -e hp-pbl
HP NC-Series iSCSI offload initiator utilities
The HP NC-Series iSCSI Offload Initiator Utilities for Red Hat and SLES are released in a source RPM
Red Hat
Install HP NC-Series iSCSI Offload Initiator Utility for RH
format.
Install the HP NC-Series iSCSI Offload Initiator Utilities for Red Hat RPM package:
1. Change to the linux\iscsi-offload directory on your hard drive and use the following command:
6. Start the open-iscsi daemon using the following command:
#/etc/init.d/open-iscsi start
Uninstalling the HP NC-Series iSCSI offload initiator utilities for SLES
If you need to uninstall the HP NC-Series iSCSI Offload Initiator Utilities for SLES RPM, use the following
command:
# rpm -e open-iscsi-<version>
Installation and configuration 15
Loading the Linux iSCSI offload bnx2i module
After installing the Linux iSCSI Offload bnx2i driver and the iSCSI Offload Initiator utilities for Red Hat or
SLES, use the following command to load the Linux iSCSI Offload bnx2i module:
# modprobe bnx2i
Executing the bnx2id application
After the Linux iSCSI offload bnx2i RPM package is installed, the bnx2id application is installed under
/sbin. The driver cannot establish an iSCSI connection until the bnx2id application is executed.
Execute the bnx2id application using the following command:
# bnx2id
Installation and configuration 16
Binding the iSCSI target
Changing the transport binding of the iSCSI node
By default the open-iscsi daemon connects to discovered targets using the software initiator with transport
name = tcp.
To offload iSCSI connections, you must change the transport binding of the iSCSI node by completing the
following steps.
1. Discover targets at a specified IP address using the following command: