warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP
shall not be liable for technical or editorial errors or omissions contained herein.
Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212,
Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S.
Government under vendor’s standard commercial license.
Audience assumptions
This document is for the person who installs, administers, and troubleshoots servers and storage systems.
HP assumes you are qualified in the servicing of computer equipment and trained in recognizing hazards
in products with hazardous energy levels.
System requirements ..................................................................................................................................4
Supported operating systems ............................................................................................................ 4
The HP iSCSI protocol transports block-level storage requests over TCP connections. Using the iSCSI
protocol, systems can connect to remote storage and use it as a physical disk although the remote storage
provider or target may actually be providing virtual physical disks. HP Accelerated iSCSI supports all
targets that are certified and supported by Linux software.
HP Gigabit Ethernet Multifunction Server Adapters can be enabled as iSCSI devices or virtual iSCSI host
bus adapters (HBAs), a feature commonly referred to as accelerated iSCSI. iSCSI HBAs, or accelerated
adapters, provide the capability to run iSCSI protocol over offloaded TCP connections. By offloading the
TCP connection processing to an HP Gigabit Ethernet Multifunction Server Adapter, CPU utilization is
reduced allowing additional processing capacity for applications. The HP accelerated iSCSI solution
supports all targets that are certified and supported by Linux software.
System requirements
Supported operating systems
Supported hardware
The following software and hardware support HP Accelerated iSCSI implementations for Linux.
• Red Hat Enterprise Linux 5 ES/AS for x86_64
• Red Hat Enterprise Linux 5 ES/AS for x86
• SUSE Linux Enterprise Server 10 for x86 SP1
• SUSE Linux Enterprise Server 10 for x86_64 SP1
Servers. Server support includes all ProLiant servers that include an iLO or RiLO management device with
any of the supported operating systems.
Adapters. HP adapter support includes the following:
• HP NC370T PCI-X Multifunction Gigabit Server Adapter
• HP NC370F PCI-X Multifunction Gigabit Server Adapter
• HP NC370i PCI-X Multifunction Gigabit Server Adapter
• HP NC371i PCI-X Multifunction Gigabit Server Adapter
• HP NC373T PCI Express Multifunction Gigabit Server Adapter
• HP NC373F PCI Express Multifunction Gigabit Server Adapter
• HP NC373i PCI Express Multifunction Gigabit Server Adapter
Overview 4
• HP NC373m PCI Express Dual Port Multifunction Gigabit Server Adapter for c-Class BladeSystem
• HP NC374m PCI Express Dual Port Multifunction Gigabit Server Adapter
• HP NC380T PCI Express Dual Port Multifunction Gigabit Server Adapter
Download lates
docu
iSCSI targets. All iSCSI targets supported by the Linux software initiator are supported by HP.
t drivers, firmware, and
mentation
HP updates networking software frequently to include new functionality and
firmware, and documentation updates go to the
2. Click Software & Driver Downloads from the left menu bar.
3. Type the product name in the For product box and press Enter. For example, type NC373T.
4. Select an operating system.
5. Click HP ProLiant Networking Software.
6.
Click download and save the HP SoftPaq (sp#####.exe) file to a directory on your hard drive. The
SoftPaq file is a self-extracting executable with a file name based on the SoftPaq number.
Overview 5
Installation and configuration
Summary of installation and configuration
Complete the following steps to install and configure Linux open-iSCSI offload files on your server and
connect to an iSCSI target.
1. Download the open-iSCSI offload files.
2. Install the bnx2i driver.
3. For Linux systems with embedded HP Gigabit Multifunction Server Adapters, HP accelerated iSCSI is
available as a port-based licensable option. Install the HP ProLiant Licensing RPM and the HP
licensing utility to enable HP accelerated iSCSI capability.
For Linux systems with standup and mezzanine HP Gigabit Multifunction Server Adapters and HP
Gigabit Multifunction Server Adapters for HP BladeSystem servers, omit step 3. HP accelerated iSCSI
capability is available as a standard feature for these adapters so no license is required.
4. Install the iSCSI initiator utilities for either Red Hat or SLES.
5. Load the Linux iSCSI offload bnx2i module.
6. Execute the bnx2id application.
7. Change the transport binding of the iSCSI node to offload the iSCSI connection.
Installing the Linux iSCSI offload bnx2i driver
The Linux iSCSI offload bnx2i driver is released in a source RPM format. Install the Linux iSCSI offload
bnx2i driver RPM package by completing the following steps:
1. To install the source RPM package, change to the linux\iscsi-offload directory on your hard drive
and use the following command:
# rpm -ivh bnx2i-<version>.src.rpm
2. Build the binary RPM for the bnx2i driver.
For Red Hat EL 5 installation enter the following commands:
# cd /usr/src/redhat
# rpmbuild -bb SPECS/bnx2i.spec
For SLES 10 SP1 installation enter the following commands:
# cd /usr/src/packages
# rpmbuild -bb SPECS/bnx2i.spec
3. Check for the existence of a current version of the bnx2i package using the following command:
# rpm -q bnx2i
If an older version of the package exists, remove the old RPM package using the following
command:
# rpm -e bnx2i
Remove the loaded bnx2i module using the following command:
# modprobe -r bnx2i
Installation and configuration 6
This command removes the bnx2i module along with its dependencies.
Verify that the old bnx2i package has been removed using the following command:
# rpm -q bnx2i
4. Install the bnx2i binary RPM package using the following command:
# rpm -ivh RPMS/<arch>/bnx2i-<version>.<arch>.rpm
The bnx2i.ko module is installed in the following directory:
/lib/modules/<kernel_version>/kernel/drivers/net
Linux iSCSI offload bnx2i driver messages
Linux iSCSI offload bnx2i driver messages are logged in the file /var/log/messages.
Use the following command to control the level that messages appear on the console:
dmesg -n <level>
Most systems are set to level 6 by default. To include debug-level messages, set the level to 7.
The following sample messages are the most common bnx2i driver messages that may be logged in the
file /var/log/messages.
•bnx2i driver signon
Broadcom NetXtreme II iSCSI Driver bnx2i v1.0.30 (Sep 29, 2007)
• bled CNIC deviceDriver completes handshake with iSCSI offload ena
bnx2i [05:00.00]: ISCSI_INIT passed
This message displays only when a user attempts to make an iSCSI connection.
•IC deviceDriver detects iSCSI Offload is not enabled on the CN
bnx2i: iSCSI not supported, dev=eth3
bnx2i: bnx2i: LOM is not enabled to offload iSCSI connections, dev=eth0
• Driver unable to allocate TCP port for iSCSI connection
bnx2i: run 'bnx2id' daemon to alloc TCP ports
•
Exceeds maximum allowed iSCSI connection offload limit
bnx2i: unable to allocate iSCSI context resources
•
Network route to target node and transport name binding are 2 different devices
ep=0x... ($ROUTE_HBA) does not belong to hba bnx2i: conn bind,
$USER_CHOSEN_HBA
where ROUTE_HBA --> n
on route information
USER_CHOSEN_HBA transport name)
•
Target cannot be reached on any of CNIC devices
bnx2i: check route, can't connect using cnic
et device on which connection was offloaded based
-> HBA to which target node is bound (using iSCSI
• hat is down
Network route is assigned to network interface t
bnx2i: check route, hba not found
• abled device
Attempting to offload iSCSI connection onto Jumbo frame
bnx2i: eth# network i/f mtu is set to #mtu
bnx2i: iSCSI HBA can support mtu of 1500
en
Installation and configuration 7
The user must change mtu to < 1500 using ethtool and restart the interface to offload iSCSI
connections.
• Open-iSCSI daemon handing over session to driver
bnx2i: conn update - MBL 0x800 FBL 0x800MRDSL_
Linux iSCSI offload bnx2i driver limitations
I 0x800 MRDSL_T 0x2000
The current version of the driver has been tested on 2.6.x kernels beginning with the 2.6.18 ker
driver may not compile on kernels older than the 2.6.18 kernel.
SLES10 SP1 is an exception. It also runs on the 2.6.16.46 kernel because SUSE upgraded the
iscsi_transport kernel module to support offload initiators in SLES10 SP1.
Uninstallin
package
If you need to uninstall the HP bnx2 RPM, use the following command:
# rpm -e bnx2i-<version>
Installing
offlo
ad
The HP Accelerated iSCSI capability for port-based Linux systems is available as a standard feature w
standup and mezzanine HP Gigabit Multifunction Server Adapters and HP Gigabit Multifunction Server
ers for HP BladeSystem servers. It is available as a licensable option, through the purchase of an HP
Adapt
nt Accelerated iSCSI licensing RPM for embedded HP Gigabit Multifunction Server Adapters.
ProLia
nel. The
g the HP Linux iSCSI offload bnx2i driver RPM
the license for HP port-based Linux iSCSI
ith
Red Hat
Red Ha
t Enterprise Linux 5
To install the HP ProLiant licensing for Red Hat Enterprise Linux 5 AS/ES [x86/AMD64/EM64T],
comp
1. After you extract the SoftPaq from the previous "Download
2.
3. Use the following command to install the HP ProLiant Licensin
4.
5.
lete the following steps:
ing the SoftPaq and extracting (on page
5)" section, locate the hp-pel-<version>.rhel5.linux.rpm stored in the li
Change to the linux\hp-pel directory on your hard drive.
g RPM:
rpm -Uvh hp-pel-<version>.rhel5.linux.rpm
Add the license key using the following command:
/opt/hp/hp-pel/na
If you have an old license and want to add a new license, ju
delete the old license.
Check the status of all currently installed licenses on the system using the following command:
/opt/hp/hp-pel/nalicense -d [-l <logfile>]
license -a license_string
st add the new license. You canno
nux\hp-pel folder.
Installation and configuration 9
t
Include the -l parameter if you want all the information that normally prints to st
InstalliniSCSI offload utility
to the file specified with <logfile>.
g the HP Licensing for port-based Linux
in
To stall the HP Licensing for port-based Linux iSCSI offload utility for Red Hat Enterprise Linux 5 AS/ES
[x86/AMD64/EM64T], complete the following steps:
1. After you extract the SoftPaq from the previous Downloading the SoftPaq and extrac
section, locate the hp-pbl-<version>.rhel5.linux.rpm stored
2. Change to the linux\hp-pbl directory on your hard drive.
3.
Use the following command to install the utility:
rpm -Uvh hp-pbl-<version>.
4. The hp-pbl utility is configured to start on system boot-up; however, you can start it immediately afte
rhel5.linux.rpm
in the linux\hp-pbl folder.
installation without rebooting using the following command:
/etc/init.d/hp-pbl start
5. ile located at etc/opt/hp/hp-pbl/hp-pbl.conf. The format of
The hp-pbl configuration file is a text f
the configurable MAC parameter is documented in the hp-pbl.conf
hp-pbl.conf configuration file, you
must restart hp-pbl using the following command:
file. If changes are made to the
/etc/init.d/hp-pbl restart
6. To manually stop the hp-pbl daemon, use the following command:
/etc/init.d/hp-pbl stop
dout to be redirected
ting (on page 5)
r
SLES
SUSE Lx
Installin port-based Linux iSCSI offload utility
inu Enterprise Server 10 SP1
To install the HP ProLiant licensing for SUSE Linux Enterprise Server
1. After you extract the SoftPaq from the previous "Downloading the SoftPaq and extrac
rive. 2. Change to the linux\hp-pel directory on your hard d
3. g RPM:
Use the following command to install the HP ProLiant Licensin
rpm -Uvh hp-pel-<version>.sles10.linux.rpm
4. he following command:
Add the license key using t
/opt/hp/hp-pel/nalicense -a license_string
delete the old license.
5. alled licenses on the system using the following command:
Check the status of all currently inst
/opt/hp/hp-pel/nalicense -d [-l <logfile>]
Include the -l parameter if you want all the information that normally prints to st
to the file specified with <logfile>.
10 SP1 [x86/AMD64/EM64T]:
red in the5)" section, locate the hp-pel-<version>.sles10.linux.rpm sto
linux\hp-pel folder.
st add the new license. You cannot If you have an old license and want to add a new license, ju
g the HP Licensing for
ting (on page
dout to be redirected
in
To stall the HP Licensing for port-based Linux iSCSI offload utility for SUSE Linux Enterprise Server 10
SP1 [x86/AMD64/EM64T]:
1. After you extract the SoftPaq from the previous "
5)" section, locate the hp-pbl-<version>.sles10.linux.rpm stored
Downloading the SoftPaq and extracting (on page
in the linux\hp-pbl folder.
Installation and configuration 10
2.
Change to the linux\hp-pbl directory on your hard drive.
Use the following command to install the utility: 3.
rpm -Uvh hp-pbl-<version>.
<distro>.linux.rpm
4. The hp-pbl utility is configured to start on system boot-up; however, you can start it immediately afte
installation without rebooting using the following command:
/etc/init.d/hp-pbl start
5. ile located at etc/opt/hp/hp-pbl/hp-pbl.c
The hp-pbl configuration file is a text f
the configurable MAC parameter is documented in the hp-pbl.conf
hp-pbl.conf configuration file, you
must restart hp-pbl using the following command:
/etc/init.d/hp-pbl restart
6. To manually stop the hp-pbl daemon, use the followi
/etc/init.d/hp-pbl stop
HP port-based licensing status report
To obd:
tain the status of hp-pbl, use the following comman
Where:
The iSCSI licenses available output reports the number of iSCSI offload licens
The iSCSI licenses used output reports the number of iSCSI offload licenses enabled.
ffload licenses available: 4
r
onf. The format of
file. If changes are made to the
ng command:
es available.
LOM is the iSCSI capable LOM MACs. If the MAC is not an iSCSI capable LOM, it is not displayed
with the status command. Error messages are entered in the hp-pbl.log file.
The Configured output reports YES when the iSCSI capable LOM MAC is entered in the hp-p bl.con
file and is reported as NO if it was not entered in
the hp-pbl.conf file. An error message is entered in
the hp-pbl.log file if the iSCSI capable LOM MAC is not entered correctly in the hp-pbl.conf file.
The Enabled output reports YES when the iSCSI capable LO
offload and reports NO when it is not enabled.
HP ProLiant Licensing Log and Event File
All m
essages are logged to /var/opt/hp/hp-pbl/hp-pbl.log.
Log m
essages are in the following format:
[TAG] SP [DATE] SP TEXT TAG
Where TAG is one of:
ERROR indicates an error
WARNING indicates a warning such as too many MACs configured for iSCSI
INFO indicates an informative event such as hp-pbl starting and stopping.
SP is one or more spaces.
DATE is the current date stamp in the
such as a missing hp-pbl.conf file.
format Sat Feb 17 12:54:35 CST 2007.
f
M MAC is currently enabled for iSCSI
offload.
Installation and configuration 11
TEXT is free form text that may or may not exist in every message. Lines that do not start with a tag
are a continuation of the previous line.
The following represent sample text lines.
[INFO] [Thu Feb 10 10:34:15 CST 2007] hp-pbl started - v1.0.0-1.rhel5
[INFO] [Thu Feb 10 12:54:36 CST 2007]
first text line of second info message
second text line of second info message
TroubsSI offload
le hoot known issues for HP port-based Linux iSC
Com
plete the following steps to troubleshoot the hp-pbl licen
1. to monitor the hp-pbl.log file for messages.
Run the following command in a separate window
-f /var/opt/hp/hp-pbl/hp-pbl.log
tail
2. Run the following command in a separate window to monitor the messages file.
tail -f
Potential problems, causes and solutions
Problem:
Does not appear
Possible cause:
The initiator needs to be restarted after the tra
needguration changes are realized. Both of these events occur when hp-
s to be reloaded so that the confi
pbl is resta
Possible solution:
Run the following command to restart hp-pbl:
Problem:
Does not appear to create an iSCSI offloaded connection when configured for offload and
/var/log/messages lists the following information:
bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba
$USE
/var/log/messages
to create an iSCSI offloaded connection when configured for offload.
nsport name is changed for offload, and the bnx2i driver
rted.
start /opt/hp/hp-pbl re
R_CHOSEN_HBA
sing.
Possible cause:
A potential problem with the link layer address to IP address mapping occurs when a server with mu
network cards is connected to the same network segment. This issue is commonly known as the Arp Flux
Problem.
Possible solution:
Set up t
he network so that network cards are on separate network segments or broadcast domains.
ltiple
Uninstalling the license for HP port-based Linux iSCSI
offload
To uninstall the licensing for HP port-based Linux iSCSI offload utility, remove the HP ProLiant licen
RPM and the Linux iSCSI offload utility package.
Installation and configuration 12
sing
Remo
ving the HP ProLiant Licensing RPM
The diant Licensing RPM package is
atabase containing ProLiant licenses is deleted if the HP ProL
remo
ved. Make sure your hard-copy ProLiant packaging license strings are available before removing this
pack
age.
You n the system using the check the status of all
can obtain the license strings by displaying all licenses o
currently installed licenses on the system command:
/opt/hp/hp-pel/
Include the -l parameter if you want all the information that normally prints to stdout to be redirected
to the file specified with <logfile>.
Record the license strings and keep them for referen
To rekage, run the following command: move the HP PEL pac
rpm -e hp-pel
nalicense -d [-l <logfile>]
Removing the hp-pbl RPM package
Remove the hp-pbl RPM package using the following command:
-e hp-pbl
HP NC-
rpm
Series iSCSI offload initiator utilities
ce.
The HP NC-Series iSCSI Offload Initiator Utilities for Red Hat and SLES
Red Ht
Install H N
format.
a
Pd Initiator Utility for RH C-Series iSCSI Offloa
Instapackage:
1. rectory on your hard drivChange to the linux\iscsi-offload di
2. Build the binary RPM for the open-iSCSI initator utility using the following commands:
3.
4. the following
are released in a source RPM
ll the HP NC-Series iSCSI Offload Initiator Utilities for Red Hat RPM
6. Start the open-iscsi daemon using the following command:
Uninstalling the HP NC-Series iSCSI of
#/etc/init.d/open-iscsi start
fload initiator utilities for SLES 10 SP1
If you need to uninstall the HP NC-Series iSCSI Offload Initiator Utilities for SLES 10 SP1 RPM, use the
following command:
# rpm -e open-iscsi-<version>
Loading the Linux iSCSI offload bnx2i module
After installing the Linux iSCSI offload bnx2i driver and the SCSI offload initiator utilities for Red Hat or
SLES 10 SP1, use the following command to load the Linux iSCSI
# modprobe bnx2i
offload bnx2i module:
Installation and configuration 14
Executing the bnx2id application
After the Linux iSCSI offload bnx2i RPM package is installed, the bnx2id application is installed under
/sbin. The driver cannot establish an iSCSI connection until the bnx2id application is executed.
Execute the bnx2id application using the following command:
# bnx2id
Installation and configuration 15
Binding the iSCSI target
Changing the transport binding of the iSCSI node
By default the open-iscsi daemon connects to discovered targets using the software initiator with transport
name = tcp.
To offload iSCSI connections, users must change the transport binding of the iSCSI node by completing
the following steps.
1. Discover targets at a specified IP address using the following command:
XPORT_NAME=bcm570#d-xxyyzz
#d — '8' for 5708 device and '6' for 5706 device
xx — pci bus number of the NX2 device
yy — pci device number of the NX2 device
zz — pci function number of the NX2 device
Executing the following command to obtain the iSCSI transport name binding:
# dmesg | grep "bnx2i: netif"
The following is a sample output of a system with 2 NetXtreme II devices: