Broadcom NetXtreme II® Network Adapter User Guide
Broadcom NetXtreme II® Network Adapter User Guide
The information below is provided by the supplier of the referenced device without independent verification by Dell and is
subject to the Restrictions and Disclaimers noted below.
Introduction
Functionality and Features
Teaming
Virtual LANs (VLANs)
Manageability
Installing the Hardware
Installing the Driver Software
Broadcom Boot Agent Driver Software
NDIS2 Driver Software
Linux Driver Software
Solaris Driver Software
VMware Driver Software
Installing Windows Drivers and Management Applications
Linux Management Application Installation
iSCSI
Advanced Teaming Concepts
NIC Partitioning
Fibre Channel Over Ethernet
Data Center Bridging
SR-IOV
Using Broadcom Advanced Control Suite
User Diagnostics
Specifications
Regulatory Information
Troubleshooting
Information in this document is subject to change without notice.
© 2014 Broadcom Corporation. All rights reserved.
Trademarks used in this text: Broadcom, NetXtreme II, Ethernet@Wirespeed, LiveLink, and Smart Load Balancing are among
the trademarks of Broadcom Corporation and/or its affiliates in the United States, certain other countries, and/or the EU. Dell
and the DELL logo are trademarks of Dell Inc. Microsoft and Windows are trademarks of Microsoft Corporation. Linux is a
trademark of Linus Torvalds. Intel is a trademark of Intel Corporation. Magic Packet is a trademark of Advanced Micro
Devices, Inc. Red Hat is a trademark of Red Hat, Inc. PCI Express is a trademark of PCI-SIG. Any other trademarks or trade
names mentioned are the property of their respective owners.
Restrictions and Disclaimers
The information contained in this document, including all instructions, cautions, and regulatory approvals and certifications, is
provided by the supplier and has not been independently verified or tested by Dell, except where specifically noted. Dell
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/index.htm[9/5/2014 3:44:56 PM]
Broadcom NetXtreme II® Network Adapter User Guide
cannot be responsible for damage caused as a result of either following or failing to follow these instructions. All statements
or claims regarding the properties, capabilities, speeds or qualifications of the part referenced in this document are made by
the supplier and not by Dell. Dell specifically disclaims knowledge of the accuracy, completeness or substantiation for any
such statements. All questions or comments relating to such statements or claims should be directed to the supplier.
Export Regulations
Customer acknowledges that these Products, which may include technology and software, are subject to the customs and
export control laws and regulations of the United States ("U.S.") and may also be subject to the customs and export laws and
regulations of the country in which the Products are manufactured and/or received. Customer agrees to abide by those laws
and regulations. Further, under U.S. law, the Products may not be sold, leased or otherwise transferred to restricted endusers or to restricted countries. In addition, the Products may not be sold, leased or otherwise transferred to, or utilized by an
end-user engaged in activities related to weapons of mass destruction, including without limitation, activities related to the
design, development, production or use of nuclear weapons, materials, or facilities, missiles or the support of missile projects,
and chemical or biological weapons.
Initial release: December 2005
Last revised: March 2014
2CSINGSRVT710-CDUM100-R
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/index.htm[9/5/2014 3:44:56 PM]
Functionality and Features: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Functionality and Features: Broadcom NetXtreme II® Network
Adapter User Guide
Functional Description
Features
Functional Description
The Broadcom NetXtreme II adapter is a new class of Gigabit Ethernet (GbE) and 10 GbE converged network interface
controller (C-NIC) that can simultaneously perform accelerated data networking and storage networking on a standard
Ethernet network. The C-NIC offers acceleration for popular protocols used in the data center, such as:
TCP Offload Engine (TOE) for accelerating TCP over 1 GbE and 10 GbE
Internet Small Computer Systems Interface (iSCSI) offload for accelerating network storage access featuring
centralized boot functionality (iSCSI boot)
Fibre Channel over Ethernet (FCoE) offload and acceleration for fibre channel block storage
NOTE: Not all adapters support each listed protocol. Refer to the specific product data sheet for protocol support.
NOTE: Separate licences are required for all offloading technologies.
Enterprise networks that use multiple protocols and multiple network fabrics benefit from the C-NICs ability to combine data
communications, storage, and clustering over a single Ethernet fabric by boosting server CPU processing performance and
memory utilization while alleviating I/O bottlenecks.
The Broadcom NetXtreme II adapter includes a 10/100/1000-Mbps or 10-Gbps Ethernet MAC with both half-duplex and fullduplex capability and a 10/100/1000-Mbps or 10-Gbps PHY. The transceiver is fully compatible with the IEEE 802.3 standard
for auto-negotiation of speed.
Using the Broadcom teaming software, you can split your network into virtual LANs (VLANs) as well as group multiple network
adapters together into teams to provide network load balancing and fault tolerance functionality. See Configuring Teaming
and Broadcom Gigabit Ethernet Teaming Services for detailed information about teaming. See Virtual LANs, for a description
of VLANs. See Configuring Teaming for instructions on configuring teaming and creating VLANs on Windows operating
systems.
Features
The following is a list of the Broadcom NetXtreme II adapter features. Some features may not be available on all adapters.
TCP Offload Engine (TOE)
Internet Small Computer Systems Interface (iSCSI) offload
Fibre Channel over Ethernet (FCoE)
NIC Partitioning
Data Center Bridging (DCB)
Enhanced Transmission Selection (ETS; IEEE 802.1Qaz)
Priority-based Flow Control (PFC; IEEE 802.1Qbb)
Data Center Bridging Capability eXchange Protocol (DCBX; CEE version 1.01)
Single-chip solution
Integrated 10/100/1000BASE-T transceivers
Integrated 10GBASE-T transceivers
10/100/1000 triple-speed MAC
SerDes interface for optical transceiver connection
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/features.htm[9/5/2014 3:44:57 PM]
Functionality and Features: Broadcom NetXtreme II® Network Adapter User Guide
PCI Express 1.0a x4 (Gigabit Ethernet)
PCI Express Gen2 x8 (10 Gigabit Ethernet)
Full fast-path TCP offload
Zero copy capable hardware
Other performance features
TCP, IP, UDP checksum
TCP segmentation
Adaptive interrupts
Receive Side Scaling (RSS)
Manageability
Broadcom Advanced Control Suite diagnostic and configuration software suite
Supports PXE 2.0 specification (Linux Red Hat PXE Server, SUSE Linux Enterprise Server, Windows Server 2008,
Windows Server 2008 R2, Windows Server 2012, Intel APITEST, DOS UNDI)
Wake on LAN support
Universal Management Port (UMP) support
Statistics for SNMP MIB II, Ethernet-like MIB, and Ethernet MIB (IEEE Std 802.3z, Clause 30)
SMBus controller
ACPI 1.1a compliant (multiple power modes)
IPMI support
Advanced network features
Jumbo frames (up to 9 KB). The OS and the link partner must support jumbo frames.
Virtual LANs
IEEE Std 802.3ad Teaming
Smart Load Balancing Teaming
Smart Load Balancing TOE Teaming (with the correct configuration)
Flow Control (IEEE Std 802.3x)
LiveLink™ (supported in both the 32-bit and 64-bit Windows operating systems)
Logical Link Control (IEEE Std 802.2)
Layer-2 Priority Encoding (IEEE Std 802.1p)
High-speed on-chip RISC processor
Up to 4 classes of service (CoS)
Up to 4 send rings and receive rings
Integrated 96 KB frame buffer memory
Quality of Service (QoS)
GMII/MII Management Interface
Four unique MAC unicast addresses
Support for multicast addresses via 128 bits hashing hardware function
Serial flash NVRAM memory
JTAG support
PCI Power Management Interface (v1.1)
64-bit BAR support
EM64T processor support
1.2 V core voltage, 0.13 µm process
iSCSI Boot support
Virtualization
Microsoft
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/features.htm[9/5/2014 3:44:57 PM]
Functionality and Features: Broadcom NetXtreme II® Network Adapter User Guide
VMware
Single Root I/O Virtualization (SRIOV)
TCP Offload Engine (TOE)
The TCP/IP protocol suite is used to provide transport services for a wide range of applications for the Internet, LAN, and for
file transfer. Without the TCP Offload Engine, the TCP/IP protocol suite runs on the host CPU, consuming a very high
percentage of its resources and leaving little resources for the applications. With the use of the Broadcom NetXtreme II
adapter, the TCP/IP processing can be moved to hardware, freeing the CPU for more important tasks such as application
processing.
The Broadcom NetXtreme II adapter's TOE functionality allows simultaneous operation of up to 1024 fully offloaded TCP
connections for 1-Gbps network adapters and 1880 fully offloaded TCP connections for 10-Gbps network adapters. The TOE
support on the adapter significantly reduces the host CPU utilization while preserving the implementation of the operating
system stack.
Internet Small Computer Systems Interface (iSCSI)
The IETF has standardized the Internet Small Computer Systems Interface (iSCSI). SCSI is a popular protocol that enables
systems to communicate with storage devices, using block-level transfer (i.e., address data stored on a storage device that is
not a whole file). iSCSI maps the SCSI request/response application protocols and its standardized command set over TCP/IP
networks.
As iSCSI utilizes TCP as its sole transport protocol, it greatly benefits from hardware acceleration of the TCP processing (i.e.,
use of a TOE). However, iSCSI as a Layer 5 protocol has additional mechanisms beyond the TCP layer. iSCSI processing can
also be offloaded, thereby reducing CPU utilization even further.
The Broadcom NetXtreme II adapter targets best-system performance, maintains system flexibility to changes, and supports
current and future OS convergence and integration. Therefore, the adapter's iSCSI offload architecture is unique as evident by
the split between hardware and host processing.
NOTES: The iSCSI offload feature is not available for all Broadcom network adapters.
Fibre Channel over Ethernet
FCoE (Fibre Channel Backbone-5 (FC-BB-5)) allows Fibre Channel protocol to be transferred over Ethernet. FCoE preserves
existing Fibre Channel infrastructure and capital investments. The following FCoE features are supported:
Full stateful hardware FCoE offload
Receiver classification of FCoE and FIP frames. FIP is the FCoE Initialization Protocol used to establish and maintain
connections.
Receiver CRC offload
Transmitter CRC offload
Dedicated queue set for Fibre Channel traffic
Data Center Bridging (DCB) provides lossless behavior with Priority Flow Control (PFC)
DCB allocates a share of link bandwidth to FCoE traffic with Enhanced Transmission Selection (ETS)
NOTES: FCoE is not available for all Broadcom network adapters.
Power Management
The adapter speed setting will link at the configured speed for WOL when the system is powered down.
NOTES:
Dell supports WOL on only one adapter in the system at a time.
For specific systems, see your system documentation for WOL support.
WOL is supported in Broadcom NetXtreme II BCM5708 devices with silicon revisions of B2 or later. For more
information, see Limitations.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/features.htm[9/5/2014 3:44:57 PM]
Functionality and Features: Broadcom NetXtreme II® Network Adapter User Guide
Adaptive Interrupt Frequency
The adapter driver intelligently adjusts host interrupt frequency based on traffic conditions to increase overall application
throughput. When traffic is light, the adapter driver interrupts the host for each received packet, minimizing latency. When
traffic is heavy, the adapter issues one host interrupt for multiple, back-to-back incoming packets, preserving host CPU
cycles.
ASIC with Embedded RISC Processor
The core control for Broadcom NetXtreme II adapters resides in a tightly integrated, high-performance ASIC. The ASIC
includes a RISC processor. This functionality provides the flexibility to add new features to the card and adapts it to future
network requirements through software downloads. This functionality also enables the adapter drivers to exploit the built-in
host offload functions on the adapter as host operating systems are enhanced to take advantage of these functions.
Broadcom Advanced Control Suite
Broadcom Advanced Control Suite (BACS) is an integrated utility that provides useful information about each network adapter
that is installed in your system. The BACS utility also enables you to perform detailed tests, diagnostics, and analyses on
each adapter, as well as to modify property values and view traffic statistics for each adapter.
Supported Operating Environments
The Broadcom NetXtreme II adapter has software support for the following operating systems:
Microsoft® Windows® (32-bit and 64-bit extended)
Linux® (32-bit and 64-bit extended)
MS-DOS
ESX and ESXi Server (VMware)
Oracle Solaris
SCO® UnixWare
SCO OpenServer
®
®
®
Network Link and Activity Indication
For copper-wire Ethernet connections, the state of the network link and activity is indicated by the LEDs on the RJ-45
connector, as described in Table 1. For fiber optic Ethernet connections and SFP+, the state of the network link and activity is
indicated by a single LED located adjacent to the port connector, as described in Table 2. Broadcom Advanced Control Suite
also provides information about the status of the network link and activity (see Viewing Vital Signs).
Table 1: Network Link and Activity Indicated by the RJ-45
Port LEDs
Port LED LED Appearance Network State
Link LED
Activity LED
Off No link (cable disconnected)
Continuously illuminated Link
Off No network activity
Blinking Network activity
Table 2: Network Link and Activity Indicated by
the Port LED
LED Appearance Network State
Off No link (cable disconnected)
Continuously illuminated Link
Blinking Network activity
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/features.htm[9/5/2014 3:44:57 PM]
Functionality and Features: Broadcom NetXtreme II® Network Adapter User Guide
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/features.htm[9/5/2014 3:44:57 PM]
Configuring Teaming in Windows Server: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Configuring Teaming in Windows Server: Broadcom NetXtreme
®
II
technology on Linux operating systems (called "Channel Bonding"), refer to your operating system documentation.
Broadcom Advanced Server Program Overview
Broadcom Advanced Server Program (BASP) is the Broadcom teaming software for the Windows family of operating systems.
BASP settings are configured by Broadcom Advanced Control Suite (BACS) utility.
BASP provides heterogeneous support for adapter teaming to include all of the Broadcom NetXtreme and NetXtreme II
adapters as well as Dell-shipping Intel NIC adapters/LOMs.BASP provides support for TOE teaming only for NetXtreme II
adapters.
BASP supports four types of teams for Layer 2 teaming:
Network Adapter User Guide
Broadcom Advanced Server Program Overview
Load Balancing and Fault Tolerance
NOTE: This chapter describes teaming for adapters in Windows Server systems. For more information on a similar
Smart Load Balancing and Failover
Link Aggregation (802.3ad)
Generic Trunking (FEC/GEC)/802.3ad-Draft Static
SLB (Auto-Fallback Disable)
BASP supports two types of teams for TOE teaming:
Smart Load Balancing and Failover
SLB (Auto-Fallback Disable)
For more information on network adapter teaming concepts, see Broadcom Gigabit Ethernet Teaming Services.
NOTE: Windows Server 2012 provides built-in teaming support, called NIC Teaming. It is not recommended that users
enable teams through NIC Teaming and BASP at the same time on the same adapters.
Load Balancing and Fault Tolerance
Teaming provides traffic load balancing and fault tolerance (redundant adapter operation in the event that a network
connection fails). When multiple Gigabit Ethernet network adapters are installed in the same system, they can be grouped
into teams, creating a virtual adapter.
A team can consist of two to eight network interfaces, and each interface can be designated as a primary interface or a
standby interface (standby interfaces can be used only in a Smart Load Balancing™ and Failover type of team, and only one
standby interface can be designated per SLB team). If traffic is not identified on any of the adapter team member connections
due to failure of the adapter, cable, switch port, or switch (where the teamed adapters are attached to separate switches),
the load distribution is reevaluated and reassigned among the remaining team members. In the event that all of the primary
adapters are down, the hot standby adapter becomes active. Existing sessions are maintained and there is no impact on the
user.
NOTE: Although a team can be created with one adapter, it is not recommended since this defeats the purpose of
teaming. A team consisting of one adapter is automatically created when setting up VLANs on a single adapter, and this
should be the only time when creating a team with one adapter.
Types of Teams
The available types of teams for the Windows family of operating systems are:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/teaming.htm[9/5/2014 3:44:58 PM]
Configuring Teaming in Windows Server: Broadcom NetXtreme II® Network Adapter User Guide
Smart Load Balancing and Failover
Link Aggregation (802.3ad) (TOE is not applicable)
Generic Trunking (FEC/GEC)/802.3ad-Draft Static (TOE is not applicable)
SLB (Auto-Fallback Disable)
Smart Load Balancing™ and Failover
Smart Load Balancing™ and Failover is the Broadcom implementation of load balancing based on IP flow. This feature
supports balancing IP traffic across multiple adapters (team members) in a bidirectional manner. In this type of team, all
adapters in the team have separate MAC addresses. This type of team provides automatic fault detection and dynamic failover
to other team member or to a hot standby member. This is done independently of Layer 3 protocol (IP, IPX, NetBEUI);
rather, it works with existing Layer 2 and 3 switches. No switch configuration (such as trunk, link aggregation) is necessary
for this type of team to work.
NOTES:
If you do not enable LiveLink™ when configuring SLB teams, disabling Spanning Tree Protocol (STP) or enabling
Port Fast at the switch or port is recommended. This minimizes the downtime due to spanning tree loop
determination when failing over. LiveLink mitigates such issues.
TCP/IP is fully balanced and IPX balances only on the transmit side of the team; other protocols are limited to
the primary adapter.
If a team member is linked at a higher speed than another, most of the traffic is handled by the adapter with the
higher speed rate.
Link Aggregation (802.3ad)
This mode supports link aggregation and conforms to the IEEE 802.3ad (LACP) specification. Configuration software allows
you to dynamically configure which adapters you want to participate in a given team. If the link partner is not correctly
configured for 802.3ad link configuration, errors are detected and noted. With this mode, all adapters in the team are
configured to receive packets for the same MAC address. The outbound load-balancing scheme is determined by our BASP
driver. The team link partner determines the load-balancing scheme for inbound packets. In this mode, at least one of the
link partners must be in active mode.
NOTE: Link Aggregation team type is not supported for TOE teaming.
Generic Trunking (FEC/GEC)/802.3ad-Draft Static
The Generic Trunking (FEC/GEC)/802.3ad-Draft Static type of team is very similar to the Link Aggregation (802.3ad) type of
team in that all adapters in the team are configured to receive packets for the same MAC address. The Generic Trunking
(FEC/GEC)/802.3ad-Draft Static) type of team, however, does not provide LACP or marker protocol support. This type of team
supports a variety of environments in which the adapter link partners are statically configured to support a proprietary
trunking mechanism. For instance, this type of team could be used to support Lucent's OpenTrunk or Cisco's Fast
EtherChannel (FEC). Basically, this type of team is a light version of the Link Aggregation (802.3ad) type of team. This
approach is much simpler, in that there is not a formalized link aggregation control protocol (LACP). As with the other types of
teams, the creation of teams and the allocation of physical adapters to various teams is done statically through user
configuration software.
The Generic Trunking (FEC/GEC/802.3ad-Draft Static) type of team supports load balancing and failover for both outbound
and inbound traffic.
NOTE: Generic Trunking (FEC/GEC/802.3ad-Draft Static) team type is not supported for TOE teaming.
SLB (Auto-Fallback Disable)
The SLB (Auto-Fallback Disable) type of team is identical to the Smart Load Balancing and Failover type of team, with the
following exception—when the standby member is active, if a primary member comes back on line, the team continues using
the standby member, rather than switching back to the primary member.
All primary interfaces in a team participate in load-balancing operations by sending and receiving a portion of the total traffic.
Standby interfaces take over in the event that all primary interfaces have lost their links.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/teaming.htm[9/5/2014 3:44:58 PM]
Configuring Teaming in Windows Server: Broadcom NetXtreme II® Network Adapter User Guide
Failover teaming provides redundant adapter operation (fault tolerance) in the event that a network connection fails. If the
primary adapter in a team is disconnected because of failure of the adapter, cable, or switch port, the secondary team
member becomes active, redirecting both inbound and outbound traffic originally assigned to the primary adapter. Sessions
will be maintained, causing no impact to the user.
Limitations of Smart Load Balancing and Failover/SLB (Auto-Fallback Disable) Types
of Teams
Smart Load Balancing™ (SLB) is a protocol-specific scheme. The level of support for IP, IPX, and NetBEUI protocols is listed in
Table 1.
Table 1: Smart Load Balancing
Operating System Failover/Fallback — All Broadcom Failover/Fallback — Multivendor
Protocol IP IPX NetBEUI IP IPX NetBEUI
Windows Server 2008 Y Y N/S Y N N/S
Windows Server 2008 R2 Y Y N/S Y N N/S
Windows Server 2012 Y Y N/S Y N N/S
Operating System Load Balance — All Broadcom Load Balance — Multivendor
Protocol IP IPX NetBEUI IP IPX NetBEUI
Windows Server 2008 Y Y N/S Y N N/S
Windows Server 2008 R2 Y Y N/S Y N N/S
Windows Server 2012 Y Y N/S Y N N/S
Legend Y = yes
N = no
N/S = not supported
The Smart Load Balancing type of team works with all Ethernet switches without having to configure the switch ports to any
special trunking mode. Only IP traffic is load-balanced in both inbound and outbound directions. IPX traffic is load-balanced in
the outbound direction only. Other protocol packets are sent and received through one primary interface only. Failover for
non-IP traffic is supported only for Broadcom network adapters. The Generic Trunking type of team requires the Ethernet
switch to support some form of port trunking mode (for example, Cisco's Gigabit EtherChannel or other switch vendor's Link
Aggregation mode). The Generic Trunking type of team is protocol-independent, and all traffic should be load-balanced and
fault-tolerant.
NOTE: If you do not enable LiveLink™ when configuring SLB teams, disabling Spanning Tree Protocol (STP) or enabling
Port Fast at the switch is recommended. This minimizes the downtime due to the spanning tree loop determination when
failing over. LiveLink mitigates such issues.
Teaming and Large Send Offload/Checksum Offload Support
Large Send Offload (LSO) and Checksum Offload are enabled for a team only when all of the members support and are
configured for the feature.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/teaming.htm[9/5/2014 3:44:58 PM]
Virtual LANs in Windows: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Virtual LANs in Windows: Broadcom NetXtreme II® Network
Adapter User Guide
VLAN Overview
Adding VLANs to Teams
VLAN Overview
Virtual LANs (VLANs) allow you to split your physical LAN into logical parts, to create logical segmentation of workgroups, and
to enforce security policies for each logical segment. Each defined VLAN behaves as its own separate network with its traffic
and broadcasts isolated from the others, increasing bandwidth efficiency within each logical group. Up to 64 VLANs (63
tagged and 1 untagged) can be defined for each Broadcom adapter on your server, depending on the amount of memory
available in your system.
VLANs can be added to a team to allow multiple VLANs with different VLAN IDs. A virtual adapter is created for each VLAN
added.
Although VLANs are commonly used to create individual broadcast domains and/or separate IP subnets, it is sometimes
useful for a server to have a presence on more than one VLAN simultaneously. Broadcom adapters support multiple VLANs on
a per-port or per-team basis, allowing very flexible network configurations.
Figure 1: Example of Servers Supporting Multiple VLANs with Tagging
Figure 1 shows an example network that uses VLANs. In this example network, the physical LAN consists of a switch, two
servers, and five clients. The LAN is logically organized into three different VLANs, each representing a different IP subnet.
The features of this network are described in Table 1.
Table 1: Example VLAN Network Topology
Component Description
VLAN #1 An IP subnet consisting of the Main Server, PC #3, and PC #5. This subnet represents an engineering group.
VLAN #2
VLAN #3 Includes the Main Server, the Accounting Server and PC #4. This VLAN is an accounting group.
Main Server
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vlan.htm[9/5/2014 3:44:58 PM]
Includes the Main Server, PCs #1 and #2 via shared media segment, and PC #5. This VLAN is a software
development group.
A high-use server that needs to be accessed from all VLANs and IP subnets. The Main Server has a Broadcom
adapter installed. All three IP subnets are accessed via the single physical adapter interface. The server is
attached to one of the switch ports, which is configured for VLANs #1, #2, and #3. Both the adapter and the
connected switch port have tagging turned on. Because of the tagging VLAN capabilities of both devices, the
Virtual LANs in Windows: Broadcom NetXtreme II® Network Adapter User Guide
server is able to communicate on all three IP subnets in this network, but continues to maintain broadcast
separation between all of them.
Accounting
Server
PCs #1 and
#2
PC #3
PC #4
PC #5
NOTE: VLAN tagging is only required to be enabled on switch ports that create trunk links to other switches, or on ports
connected to tag-capable end-stations, such as servers or workstations with Broadcom adapters.
Available to VLAN #3 only. The Accounting Server is isolated from all traffic on VLANs #1 and #2. The switch
port connected to the server has tagging turned off.
Attached to a shared media hub that is then connected to the switch. PCs #1 and #2 belong to VLAN #2 only,
and are logically in the same IP subnet as the Main Server and PC #5. The switch port connected to this
segment has tagging turned off.
A member of VLAN #1, PC #3 can communicate only with the Main Server and PC #5. Tagging is not enabled
on PC #3 switch port.
A member of VLAN #3, PC #4 can only communicate with the servers. Tagging is not enabled on PC #4 switch
port.
A member of both VLANs #1 and #2, PC #5 has an Broadcom adapter installed. It is connected to switch port
#10. Both the adapter and the switch port are configured for VLANs #1 and #2 and have tagging enabled.
Adding VLANs to Teams
Each team supports up to 64 VLANs (63 tagged and 1 untagged). Note that only Broadcom adapters and Alteon® AceNIC
adapters can be part of a team with VLANs. With multiple VLANs on an adapter, a server with a single adapter can have a
logical presence on multiple IP subnets. With multiple VLANs in a team, a server can have a logical presence on multiple IP
subnets and benefit from load balancing and failover. For instructions on adding a VLAN to a team, see Adding a VLAN for
Windows operating systems.
NOTE: Adapters that are members of a failover team can also be configured to support VLANs. Because VLANs are not
supported for an Intel LOM, if an Intel LOM is a member of a failover team, VLANs cannot be configured for that team.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vlan.htm[9/5/2014 3:44:58 PM]
Manageability: Broadcom NetXtreme II® Network Adapter User Guide
participating in a team, and the virtual NIC adapters created as the result of teaming. Non-teamed NIC adapters are not
Back to Contents Page
Manageability: Broadcom NetXtreme II® Network Adapter User
Guide
CIM
SNMP
HBA API
CIM
The Common Information Model (CIM) is an industry standard defined by the Distributed Management Task Force (DMTF).
Microsoft implements CIM on Windows server platforms. Broadcom support CIM on Windows Server and Linux platforms.
NOTE: For information on installing a CIM provider on Linux-based systems, see Linux Management Application
Installation.
Broadcom's implementation of CIM will provide various classes to provide information to users through CIM client applications.
Note that Broadcom CIM data provider will provide data only, and users can choose their preferred CIM client software to
browse the information exposed by Broadcom CIM provider.
Broadcom CIM provider provides information through BRCM_NetworkAdapter and BRCM_ExtraCapacityGroup classes.
BRCM_NetworkAdapter class provides network adapter information pertaining to a group of adapters including Broadcom and
other vendors' controllers. BRCM_ExtraCapacityGroup class provides team configuration for the Broadcom Advanced Server
Program. Current implementation will provide team information and information of physical network adapters in the team.
Broadcom Advanced Server Program provides events through event logs. Users can use the "Event Viewer" provided by
Windows server platforms, or use CIM to inspect or monitor these events. Broadcom CIM provider will also provide event
information through the CIM generic event model. These events are __InstanceCreationEvent, __InstanceDeletionEvent and
__InstanceModificationEvent, and are defined by CIM. CIM requires the client application to register the events from the client
application, using queries as examples shown below in order to receive events properly.
SELECT * FROM __InstanceModificationEvent
where TargetInstance ISA "BRCM_NetworkAdapter"
SELECT * FROM __InstanceModificationEvent
where TargetInstance ISA "BRCM_ExtraCapacityGroup"
SELECT * FROM __InstanceCreationEvent
where TargetInstance ISA "BRCM_NetworkAdapter"
SELECT * FROM __InstanceDeletionEvent
where TargetInstance ISA "BRCM_NetworkAdapter"
SELECT * FROM __InstanceCreationEvent
where TargetInstance ISA "BRCM_ActsAsSpare"
SELECT * FROM __InstanceDeletionEvent
where TargetInstance ISA "BRCM_ActsAsSpare"
For detailed information about these events, see the CIM documentation at
http://www.dmtf.org/sites/default/files/standards/documents/DSP0004V2.3_final.pdf.
Broadcom also implements the Storage Management Initiative-Specification (SMI-S), which defines CIM management profiles
for storage systems.
SNMP
BASP Subagent
The BASP subagent, baspmgnt.dll, is designed for the Windows Server 2008 and Windows Server 2008 R2 SNMP service. It is
required to install the SNMP service before installing the BASP subagent.
The BASP subagent allows an SNMP manager software to actively monitor the configurations and performance of the
Broadcom Advanced Server features. The subagent also provides an alarm trap to an SNMP manager to inform the manager
of any changes to the conditions of the BASP component.
The BASP subagent allows monitoring of the configurations and statistics for the BASP teams, the physical NIC adapters
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/manage.htm[9/5/2014 3:44:59 PM]
Manageability: Broadcom NetXtreme II® Network Adapter User Guide
monitored at this time. The BASP configuration data includes information such as team IDs, physical/virtual/VLAN/team
adapter IDs, physical/virtual/VLAN/team/ adapter descriptions, and MAC addresses of the adapters.
The statistics include detailed information such as data packets transmitted and received for the physical/virtual/VLAN/team
adapters.
The alarm trap forwards information about the changes in configuration of the physical adapters participating in a team, such
as physical adapter link up/down, and adapter installed/removed events.
To monitor this information, an SNMP manager must load the Broadcom BASP MIB database files to allow monitoring of the
information described above. These files, which are shown below, are included with the driver source media.
baspcfg.mib
baspstat.mib
basptrap.mib
HBA API
Broadcom supports the Storage Networking Industry Association (SNIA) Common HBA API on Windows and Linux operating
systems. The Common HBA API is an application program interface for the management of Fibre Channel Host Bus Adapters.
BASP Extensible-Agent
The Broadcom NetXtreme II Gigabit Ethernet Controller Extended Information SNMP extensible-agent (bcmif.dll) is designed
for Windows Server 2008 SNMP service.
The extensible-agent allows the SNMP manager software to actively monitor the configurations of the Broadcom NetXtreme II
adapter. It is intended to supplement the information already provided by the standard SNMP Management Network Interface
information.
The extensible-agent provides in-depth information about a Broadcom NetXtreme II adapter such as:
MAC address
Bound IP address
IP subnet mask
Physical link status
Adapter state
Line speed
Duplex mode
Memory range
Interrupt setting
Bus number
Device number
Function number
To monitor this information, a SNMP manager needs to load the Broadcom Extended information MIB file to allow monitoring
of the information described above. This file, bcmif.mib, is included on the installation CD.
The monitored workstation requires the installation of the Broadcom Extended Information SNMP extensible-agent, bcmif.dll,
and requires the Microsoft Windows Server 2008 SNMP service to be installed and loaded.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/manage.htm[9/5/2014 3:44:59 PM]
Installing the Hardware: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Installing the Hardware: Broadcom NetXtreme II® Network
Adapter User Guide
Overview
System Requirements
Safety Precautions
Preinstallation Checklist
Installation of the Add-In NIC
NOTE: Service Personnel: This product is intended only for installation in a Restricted Access Location (RAL).
Overview
This section applies to Broadcom NetXtreme II add-in network interface cards.
System Requirements
Before you install a Broadcom NetXtreme II adapter, verify that your system meets the following hardware and operating
system requirements:
Hardware Requirements
IA32- or EMT64-based computer that meets operating system requirements
One open PCI Express slot. Depending on the PCI Express support on your adapter, the slot may be of type PCI Express
1.0a x1, PCI Express 1.0a x4, or PCI Express Gen2 x8.
128-MB RAM (minimum)
Operating System Requirements
General
PCI Express v1.0a, x1 (or greater) Host Interface
Microsoft Windows
One of the following versions of Microsoft Windows:
Windows Server 2008 family
Windows Server 2008 R2 family
Windows Server 2012 family
Linux
Although the adapter driver should work with many Linux kernel versions and distributions, it has only been tested on 2.4x
kernels (starting from 2.4.24) and 2.6.x kernels. The driver may not compile on kernels older than 2.4.24. Testing is
concentrated on i386 and x86_64 architectures. Only limited testing has been done on other architectures. Minor changes to
some source files and Makefile may be needed on some kernels.
VMware ESX
VMware ESX
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/install.htm[9/5/2014 3:44:59 PM]
Installing the Hardware: Broadcom NetXtreme II® Network Adapter User Guide
VMware ESX 3.5
6. Secure the adapter with the adapter clip or screw.
VMware ESX 4.0
VMware ESX 4.1
VMware ESXi 5.0
VMware ESXi 5.1
Safety Precautions
CAUTION! The adapter is being installed in a system that operates with voltages that can be lethal. Before
you open the case of your system, observe the following precautions to protect yourself and to prevent damage
to the system components.
Remove any metallic objects or jewelry from your hands and wrists.
Make sure to use only insulated or nonconducting tools.
Verify that the system is powered OFF and is unplugged before you touch internal components.
Install or remove adapters in a static-free environment. The use of a properly grounded wrist strap or other
personal antistatic devices and an antistatic mat is strongly recommended.
Preinstallation Checklist
1. Verify that your system meets the hardware and software requirements listed under System Requirements.
2. Verify that your system is using the latest BIOS.
NOTE: If you acquired the adapter software on a disk or from the Dell support website (http://support.dell.com),
verify the path to the adapter driver files.
3. If your system is active, shut it down.
4. When system shutdown is complete, turn off the power and unplug the power cord.
5. Remove the adapter from its shipping package and place it on an antistatic surface.
6. Check the adapter for visible signs of damage, particularly on the edge connector. Never attempt to install a damaged
adapter.
Installation of the Add-In NIC
The following instructions apply to installing the Broadcom NetXtreme II adapter (add-in NIC) in most systems. Refer to the
manuals that were supplied with your system for details about performing these tasks on your particular system.
Installing the Add-In NIC
1. Review Safety Precautions and Preinstallation Checklist. Before you install the adapter, ensure that the system power is
OFF, the power cord is unplugged from the power outlet, and that you are following proper electrical grounding
procedures.
2. Open the system case and select the slot based on the adapter, which may be of type PCIe 1.0a x1, PCIe 1.0a x4, PCIe
Gen2 x8, or other appropriate slot. A lesser width adapter can be seated into a greater width slot (x1 in a x4), but a
greater width adapter cannot be seated into a lesser width slot (x4 in a x1). If you do not know how to identify a PCI
Express slot, refer to your system documentation.
3. Remove the blank cover-plate from the slot that you selected.
4. Align the adapter connector edge with the PCI Express connector slot in the system.
5. Applying even pressure at both corners of the card, push the adapter card into the slot until it is firmly seated. When
the adapter is properly seated, the adapter port connectors are aligned with the slot opening, and the adapter faceplate
is flush against the system chassis.
CAUTION! Do not use excessive force when seating the card, as this may damage the system or the
adapter. If you have difficulty seating the adapter, remove it, realign it, and try again.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/install.htm[9/5/2014 3:44:59 PM]
Installing the Hardware: Broadcom NetXtreme II® Network Adapter User Guide
7. Close the system case and disconnect any personal antistatic devices.
Connecting the Network Cables
The Broadcom NetXtreme II adapter has either an RJ-45 connector used for attaching the system to an Ethernet copper-wire
segment or a fiber optic connector for attaching the system to an Ethernet fiber optic segment.
NOTE: This section does not apply to blade servers.
Copper Wire
NOTE: The Broadcom NetXtreme II adapter supports Automatic MDI Crossover (MDIX), which eliminates the need for
crossover cables when connecting machines back-to-back. A straight-through Category 5 cable allows the machines to
communicate when connected directly together.
1. Select an appropriate cable. Table 1 lists the copper cable requirements for connecting to 10/100/1000BASE-T and
10GBASE-T ports:
Table 1: 10/100/1000BASE-T and 10GBASE-T Cable Specifications
Port Type Connector Media Maximum Distance
10BASE-T RJ-45 Category 3, 4, or 5 unshielded twisted pairs (UTP) 100m (328 ft)
100/1000BASE-T
1
RJ-45
Category 5
2
UTP
100m (328 ft)
10GBASE-T RJ-45
Category 63 UTP
Category 6A3 UTP
1
1000BASE-T signaling requires four twisted pairs of Category 5 balanced cabling, as specified in ISO/IEC
11801:2002 and ANSI/EIA/TIA-568-B.
2
Category 5 is the minimum requirement. Category 5e and Category 6 are fully supported.
3
10GBASE-T signaling requires four twisted pairs of Category 6 or Category 6A (augmented Category 6) balanced
cabling, as specified in ISO/IEC 11801:2002 and ANSI/TIA/EIA-568-B.
2. Connect one end of the cable to the RJ-45 connector on the adapter.
3. Connect the other end of the cable to an RJ-45 Ethernet network port.
Please read all Restrictions and Disclaimers.
Back to Contents Page
50m (164 ft)
100m (328 ft)
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/install.htm[9/5/2014 3:44:59 PM]
Broadcom Boot Agent Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Broadcom Boot Agent Driver Software: Broadcom NetXtreme II® Network
Adapter User Guide
Overview
Setting Up MBA in a Client Environment
Setting Up MBA in a Server Environment
Overview
Broadcom NetXtreme II adapters support Preboot Execution Environment (PXE), Remote Program Load (RPL), iSCSI, and Bootstrap Protocol (BootP). MultiBoot Agent (MBA) is a software module that allows your network computer to boot with the images provided by remote servers across the network. The
Broadcom MBA driver complies with the PXE 2.1 specification and is released with both monolithic and split binary images. This provides flexibility to users in
different environments where the motherboard may or may not have built-in base code.
The MBA module operates in a client/server environment. A network consists of one or more boot servers that provide boot images to multiple computers
through the network. The Broadcom implementation of the MBA module has been tested successfully in the following environments:
Linux Red Hat PXE Server. Broadcom PXE clients are able to remotely boot and use network resources (NFS mount, and so forth) and to perform
Linux installations. In the case of a remote boot, the Linux universal driver binds seamlessly with the Broadcom Universal Network Driver Interface
(UNDI) and provides a network interface in the Linux remotely-booted client environment.
Intel APITEST. The Broadcom PXE driver passes all API compliance test suites.
MS-DOS UNDI. The MS-DOS Universal Network Driver Interface (UNDI) seamlessly binds with the Broadcom UNDI to provide a network adapter driver
interface specification (NDIS2) interface to the upper layer protocol stack. This allows computers to connect to network resources in an MS-DOS
environment.
Windows Deployment Service (WDS). To extend functionalities beyond basic network connectivity when loading an operating system through WDS,
see Using the NetXtreme II Monolithic Driver.
Automated Deployment Service (ADS). To extend functionalities beyond basic network connectivity when loading an operating system through
ADS, see Using the NetXtreme II Monolithic Driver.
Setting Up MBA in a Client Environment
Setting up MBA in a client environment involves the following steps:
1. Enabling the MBA driver.
2. Configuring the MBA driver.
3. Setting up the BIOS for the boot order.
Enabling the MBA Driver
To enable or disable the MBA driver:
1. Insert an MS-DOS 6.22 or Dell Real Mode Kernel bootable disk containing the uxdiag.exe file (for 10/100/1000-Mbps network adapters) or uediag.exe
(for 10-Gbps network adapters) in the removable disk drive and power up your system.
NOTE: The uxdiag.exe (or uediag.exe) file is on the installation CD or in the DOS Utilities package available from http://support.dell.com/.
1. Type:
uxdiag -mba [ 0-disable | 1-enable ] -c devnum
(or uediag -mba [ 0-disable | 1-enable ] -c devnum)
where
devnum is the specific device(s) number (0,1,2, ...) to be programmed.
Configuring the MBA Driver
This section pertains to configuring the MBA driver on add-in NIC models of the Broadcom network adapter. For configuring the MBA driver on LOM models of
the Broadcom network adapter, check your system documentation.
NOTE: You can use Broadcom's Comprehensive Configuration Management (CCM) utility or the uEFI to configure the MBA driver one adapter at a time as
described below. Or you can use the MS-DOS based User Diagnostics application to simultaneously configure the MBA driver for multiple adapters.
Using CCM
1. Restart your system.
2. Press CTRL+s within 4 seconds after you are prompted to do so. A list of adapters displays.
a. Select the adapter to configure and press Enter. The Main Menu displays.
b. Select MBA Configuration to display the MBA Configuration menu.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/pxe.htm[9/5/2014 3:45:00 PM]
Broadcom Boot Agent Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
3. Use the UP ARROW and DOWN ARROW keys to move to the Boot Protocol menu item. Then use the RIGHT ARROW or LEFT ARROW key to select the
boot protocol of choice if other boot protocols besides Preboot Execution Environment (PXE) are available. If available, other boot protocols include
Remote Program Load (RPL), iSCSI, and Bootstrap Protocol (BOOTP).
NOTE: For iSCSI boot-capable LOMs, the boot protocol is set via the BIOS. See your system documentation for more information.
NOTE: If you have multiple adapters in your system and you are unsure which adapter you are configuring, press CTRL+F6, which causes the port
LEDs on the adapter to start blinking.
4. Use the UP ARROW, DOWN ARROW, LEFT ARROW, and RIGHT ARROW keys to move to and change the values for other menu items, as desired.
5. Press F4 to save your settings.
6. Press ESC when you are finished.
Using uEFI
1. Restart your system.
2. Enter the System Setup or Device Setting configuration menu.
3. Select the device on which you want to change MBA settings.
4. Select MBA Configuration Menu.
5. Use the drop-down menu to select the boot protocol of choice, if boot protocols other than Preboot Execution Environment (PXE) are available. If
available, other boot protocols include iSCSI, FCoE, and Bootstrap Protocol (BOOTP).
NOTE: For iSCSI boot-capable LOMs, the boot protocol is set via the BIOS. See your system documentation for more information.
6. Use the UP ARROW, DOWN ARROW, LEFT ARROW, and RIGHT ARROW keys to move to and change the values for other menu items, as desired.
7. Select Back to go to Main menu
8. Select Finish to save and exit.
Setting Up the BIOS
To boot from the network with the MBA, make the MBA enabled adapter the first bootable device under the BIOS. This procedure depends on the system BIOS
implementation. Refer to the user manual for the system for instructions.
Setting Up MBA in a Server Environment
Red Hat Linux PXE Server
The Red Hat Enterprise Linux distribution has PXE Server support. It allows users to remotely perform a complete Linux installation over the network. The
distribution comes with the boot images boot kernel (vmlinuz) and initial ram disk (initrd), which are located on the Red Hat disk#1:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/pxe.htm[9/5/2014 3:45:00 PM]
Broadcom Boot Agent Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
/images/pxeboot/vmlinuz
/images/pxeboot/initrd.img
Refer to the Red Hat documentation for instructions on how to install PXE Server on Linux.
The Initrd.img file distributed with Red Hat Enterprise Linux, however, does not have a Linux network driver for the Broadcom NetXtreme II adapters. This
version requires a driver disk for drivers that are not part of the standard distribution. You can create a driver disk for the Broadcom NetXtreme II adapter
from the image distributed with the installation CD. Refer to the Linux Readme.txt file for more information.
MS-DOS UNDI/Intel APITEST
To boot in MS-DOS mode and connect to a network for the MS-DOS environment, download the Intel PXE PDK from the Intel website. This PXE PDK comes
with a TFTP/ProxyDHCP/Boot server. The PXE PDK can be downloaded from Intel at http://downloadcenter.intel.com/SearchResult.aspx?
lang=eng&ProductFamily=Network+Connectivity&ProductLine=Boot+Agent+Software&ProductProduct=Intel%c2%ae+Boot+Agent.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/pxe.htm[9/5/2014 3:45:00 PM]
NDIS2 Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
NDIS2 Driver Software: Broadcom NetXtreme II® Network
Adapter User Guide
Overview
Preinstallation Requirements
Installing the NDIS2 Driver Software for Use on MS-DOS Platforms
Using Keywords for the Drivers
Overview
Two drivers are discussed in this section:
BXND20X: Broadcom NetXtreme II Gigabit Ethernet driver
BNX2EV: Broadcom NetXtreme II 10 Gigabit Ethernet driver
The examples used in this section refer to the BXND20X driver, but also apply to the BNX2EV driver.
Preinstallation Requirements
Before you can successfully install the NDIS2 driver software, the Broadcom network adapter must be physically installed in
the server. Networking software that is appropriate to the operating system (such as Microsoft LAN Manager 2.2 for MS-DOS)
must already be running on your server.
Installing the NDIS2 Driver Software for Use on MS-DOS Platforms
The NDIS2 driver software can be run from an MS-DOS startup disk using Microsoft Network Client 3.0 or from the hard disk
using Microsoft LAN Manager 2.2.
Creating a Startup Disk to Run Microsoft Network Client
To perform this installation you must have the following items
Windows NT Server 4.0 CD-ROM
A blank MS-DOS system disk (3.5" high-density floppy disk)
Access to the Broadcom NDIS2 driver file (BXND20X.dos). This file is located on the driver source media.
NOTES:
Windows NT Server 4.0 users. When running Setup for Microsoft Network Client v3.0 for MS-DOS, click any
network card from the list (NE2000 Compatible, for example) to create the startup disk.
After creating the startup disk, follow the instructions in Modifying the Startup Disk.
To create a startup disk
1. Create a folder called NCADMIN in the root of the C drive.
2. Copy the NCADMIN.CN_, NCADMIN.EX_, and NCADMIN.HL_ files from the I386 folder on the Windows NT Server 4.0
CD-ROM.
3. Open a command prompt window and change the directory to C:\NCADMIN.
4. Type expand -r ncadmin.* and press ENTER.
5. Close the command prompt window by typing exit and then pressing ENTER.
6. Start Windows Explorer.
7. Open the NCADMIN folder and double-click ncadmin.exe.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/ndis2.htm[9/5/2014 3:45:00 PM]
NDIS2 Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
8. Follow the on-screen instructions to make the network startup disk (choose NE2000 Compatible from the list of
adapters).
Modifying the Startup Disk
To modify the startup disk
1. Edit A:\Net\Protocol.ini with Notepad or a similar text editor.
a. Change DriverName=$ to DriverName=BXND20X$.
b. Remove all other parameter entries under the [MS$NE2CLONE] or equivalent section such as IOBASE=0x300 or
INTERRUPT=3, and so on.
Example Protocol.ini file for IP
[network.setup]
version=0x3110
netcard=ms$ne2clone,1,MS$NE2CLONE,1
transport=tcpip,TCPIP
lana0=ms$ne2clone,1,tcpip
[MS$NE2CLONE]
DriverName=BXND20X$
[protman]
DriverName=PROTMAN$
PRIORITY=MS$NDISHLP
[tcpip]
NBSessions=6
DefaultGateway=0
SubNetMask=255 0 0 0
IPAddress=192 168 0 1
DisableDHCP=0
DriverName=TCPIP$
BINDINGS=MS$NE2CLONE
LANABASE=0
Example Protocol.ini file for IPX
[network.setup]
version=0x3110
netcard=ms$ne2clone,1,MS$NE2CLONE,1
transport=ms$ndishlp,MS$NDISHLP
transport=ms$nwlink,MS$NWLINK
lana0=ms$ne2clone,1,ms$nwlink
lana1=ms$ne2clone,1,ms$ndishlp
[MS$NE2CLONE]
DriverName=BXND20X$
[protman]
DriverName=PROTMAN$
PRIORITY=MS$NDISHLP
[MS$NDISHLP]
DriverName=ndishlp$
BINDINGS=ms$ne2clone
[ms$nwlink]
DriverName=nwlink$
FRAME=Ethernet_802.2
BINDINGS=MS$NE2CLONE
LANABASE=0
Example Protocol.ini file for NetBEUI
[network.setup]
version=0x3110
netcard=ms$ne2clone,1,MS$NE2CLONE,1
transport=ms$ndishlp,MS$NDISHLP
transport=ms$netbeui,MS$NETBEUI
lana0=ms$ne2clone,1,ms$ndishlp
lana1=ms$ne2clone,1,ms$netbeui
[MS$NE2CLONE]
DriverName=BXND20X$
[protman]
DriverName=PROTMAN$
PRIORITY=MS$NDISHLP
[MS$NDISHLP]
DriverName=ndishlp$
BINDINGS=MS$NE2CLONE
[MS$NETBEUI]
DriverName=netbeui$
SESSIONS=10
NCBS=12
BINDINGS=MS$NE2CLONE
LANABASE=0
2. Edit A:\Net\System.ini.
a. Change netcard= to netcard=BXND20X.dos.
b. Check for references to C:\NET and change C:\NET to A:\NET if necessary.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/ndis2.htm[9/5/2014 3:45:00 PM]
NDIS2 Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
DriverName = PROTMAN$
Example System.ini file
[network]
sizworkbuf=1498
filesharing=no
printsharing=no
autologon=yes
computername=MYPC
lanroot=A:\NET
username=USER1
workgroup=WORKGROUP
reconnect=yes
dospophotkey=N
lmlogon=0
logondomain=
preferredredir=basic
autostart=basic
maxconnections=8
[network drivers]
netcard=BXND20X.dos
transport=ndishlp.sys,*netbeui
devdir=A:\NET
LoadRMDrivers=yes
3. Copy BXND20X.dos to A:\Net.
4. Create the appropriate Autoexec.bat file in drive A for the chosen protocol as shown below.
For TCP/IP
path=a:\net
a:\net\net initialize
a:\net\netbind.com
a:\net\umb.com
a:\net\tcptsr.exe
a:\net\tinyrfc.exe
a:\net\nmtsr.exe
a:\net\emsbfr.exe
a:\net\net start basic
net use z: \\SERVERNAME\SHARENAME
For IPX
SET PATH=A:\NET
A:\NET\net initialize
A:\NET\nwlink
A:\NET\NET START BASIC
net use z: \\SERVERNAME\SHARENAME
For NetBEUI
SET PATH=A:\NET
A:\NET\NET START BASIC
net use z: \\SERVERNAME\SHARENAME
5. Create a Config.sys file on the startup disk in drive A as shown below.
files=30
device=a:\net\ifshlp.sys
lastdrive=z
Installing the DOS NDIS2 Driver Software on the Hard Disk
To install the DOS NDIS2 Driver Software on the hard disk
1. Verify that the system has Microsoft LAN Manager 2.2 installed, with a protocol such as NetBEUI configured.
2. Create a folder on your hard disk to store the NDIS 2.01 driver.
Example: C:\LANMAN
3. Copy the BXND20X.dos file to this folder.
Edit the Config.sys file by adding the following lines:
DEVICE = C:\LANMAN\PROTMAN.DOS
DEVICE = C:\LANMAN\BXND20X.DOS
DEVICE = C:\LANMAN\NETBEUI.DOS
4. Edit the Autoexec.bat file by adding the following lines:
C:\ LANMAN\NETBIND.EXE
C:\LANMAN\NET START WORKSTATION
C:\LANMAN\NET USE drive letter: \\server name\resource name
5. Edit the Protocol.ini file (located in C:\LANMAN) to configure the driver to bind with NetBEUI or any other protocols.
Example:
[PROTOCOL MANAGER]
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/ndis2.htm[9/5/2014 3:45:00 PM]
NDIS2 Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
[NETBEUI_XIF]
DriverName = netbeui$
BINDINGS = BXND20X
[BXND20X]
DriverName = "BXND20X$"
6. Restart the computer to complete the installation.
NOTE: The driver loads during system configuration and displays the Broadcom banner, controller name, MAC
address, IRQ number, detected line speed, and the controller BusNum and DevNum. If the driver fails to load, an
initialization fail message is displayed.
Using Keywords for the Drivers
The Protocol.ini file contains certain keywords that are used by the BXND20X.dos AND BXND20X.dos drivers. These keywords
are listed below:
BusNum. Specifies the number of the PCI bus on which the network adapter is located. Requires a decimal number having a
value ranging from 0 to 255.
DevNum. Specifies the device number assigned to the network adapter when it is configured by the PCI BIOS. Requires a
decimal number having a value ranging from 0 to 255.
FuncNum or PortNum. Specifies the PCI function or port number assigned to the network controller. Requires a decimal
number having a value ranging from 0 to 7.
NOTE: These keywords, BusNum, DevNum, and FuncNum (or PortNum) are needed when multiple adapters are
installed in the server and when a specific controller must be loaded in a certain order. These keywords are used concurrently
and are included for manufacturing purposes. Do not use them unless you are familiar with how to configure PCI devices. A
PCI device scan utility is needed to find this information.
LineSpeed. Specifies the speed of the network connection in Mbit/s. Requires the decimal number 10, 100, or 1000.
Technically, a line speed of 1000 Mbit/s cannot be forced and is achievable only through auto-negotiation. For the sake of
simplicity, the driver performs auto-negotiation when the line speed is set to a value of 1000.
NOTE: LineSpeed is not available with the Broadcom NetXtreme II 10 Gigabit Ethernet driver.
Duplex. Specifies the duplex mode of the network adapter. Requires a setting of either Half or Full. When this keyword is
used, the LineSpeed keyword must also be used. If neither keyword is used, the network adapter defaults to autonegotiation mode.
NOTE: LineSpeed is not available with the Broadcom NetXtreme II 10 Gigabit Ethernet driver.
NodeAddress. Specifies the network address used by the network adapter. If a multicast address or a broadcast address is
specified, the adapter uses the default MAC address.
Example:
[BXND20X]
DriverName = "BXND20X$"
BusNum = 3
DevNum = 14
PortNum = 2
LineSpeed = 1000
Duplex = Full
NodeAddress = 001020304050
FixCheckSumOff. Turns off the driver's workaround for the TCP/IP stack to recognize the 1s complemented version of the
checksum.
AcceptAllMC. Informs the driver to deliver all multicast packets to the upper protocol.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/ndis2.htm[9/5/2014 3:45:00 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Linux Driver Software: Broadcom NetXtreme II® Network
Adapter User Guide
Introduction
Limitations
Packaging
Installing Linux Driver Software
Unloading/Removing the Linux Driver
Patching PCI Files (Optional)
Network Installations
Setting Values for Optional Properties
Driver Defaults
Driver Messages
Teaming with Channel Bonding
Remote PHY Support
Statistics
Linux iSCSI Offload
Introduction
This section discusses the Linux drivers for the Broadcom NetXtreme II network adapters.
Table 1: Broadcom NetXtreme II Linux Drivers
Linux
Driver
bnx2 Linux driver for the NetXtreme II 1 Gb network adapters.
bnx2x
cnic
bnx2i Linux iSCSI HBA driver to enable iSCSI offload on the NetXtreme II 1 Gb and 10 Gb network adapters.
bnx2fc
Description
Linux driver for the NetXtreme II 10 Gb network adapters. This driver directly controls the hardware and is
responsible for sending and receiving Ethernet packets on behalf of the Linux host networking stack. This driver also
receives and processes device interrupts, both on behalf of itself (for L2 networking) and on behalf of the bnx2fc
(FCoE) and cnic drivers.
The cnic driver provides the interface between Broadcom's upper layer protocol (e.g., storage) drivers and
Broadcom's NetXtreme II 1 Gb and 10 Gb network adapters. The CNIC module works with the bnx2 and bnx2x
network drives in the downstream and the bnx2fc (FCoE) and bnx2i (iSCSI) drivers in the upstream.
Linux FCoE kernel mode driver used to provide a translation layer between the Linux SCSI stack and the Broadcom
FCoE firmware/hardware. In addition, the driver interfaces with the networking layer to transmit and receive
encapsulated FCoE frames on behalf of open-fcoe's libfc/libfcoe for FIP/device discovery.
Limitations
bnx2 Driver
bnx2x Driver
bnx2i Driver
bnx2 Driver
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
The current version of the driver has been tested on all 2.6.x kernels. Testing is concentrated on i386 and x86_64
architectures. Only limited testing has been done on other architectures. Minor changes to some source files and Makefile may
be needed on some kernels. Additionally, the Makefile will not compile the cnic driver on kernels older than 2.6.16. iSCSI
offload is only supported on 2.6.16 and newer kernels.
NOTE: For Broadcom NetXtreme II BCM5708 devices with a silicon revision prior to B2, the open source bnx2 driver does
not support the reporting and configuration of NetXtreme II WOL settings via ethtool. For silicon revisions of B2 or later, the
bnx2 driver reports support for Magic Packet WOL via ethtool. Enabling support via ethtool is mandatory to successfully wake
the system. To determine the silicon revision of your Broadcom NetXtreme II device, use the lspci command, where "10" =
revision B0, "11" = revision B1, and "12" = revision B2.
bnx2x Driver
The current version of the driver has been tested on 2.6.x kernels starting from 2.6.9. The driver may not compile on kernels
older than 2.6.9. Testing is concentrated on i386 and x86_64 architectures. Only limited testing has been done on some other
architectures. Minor changes to some source files and Makefile may be needed on some kernels.
bnx2i Driver
The current version of the driver has been tested on 2.6.x kernels, starting from 2.6.18 kernel. The driver may not compile
on older kernels. Testing is concentrated on i386 and x86_64 architectures, Red Hat EL5, and SUSE 11 SP1 distributions.
Packaging
The Linux drivers are released in the following packaging formats:
DKMS Packages
netxtreme2-version.dkms.noarch.rpm
netxtreme2-version.dkms.src.rpm
KMP Packages
SLES
broadcom-netxtreme2-kmp-[kernel]-version.i586.rpm
broadcom-netxtreme2-kmp-[kernel]-version.x86_64.rpm
Red Hat
kmod-kmp-netxtreme2-{kernel]-version.i686.rpm
kmod-kmp-netxtreme2-{kernel]-version.x86_64.rpm
The Broadcom Advanced Control Suite management utility is also distributed as an RPM package (BACS{version}.{arch}.rpm). See Linux BACS Installation for information on installing Linux BACS.
Source Packages
Identical source files to build the driver are included in both RPM and TAR source packages. The supplemental tar file contains
additional utilities such as patches and driver diskette images for network installation.
The following is a list of included files:
netxtreme2-version.src.rpm: RPM package with NetXtreme II bnx2/bnx2x/cnic/bnx2fc/bnx2ilibfc/libfcoe driver
source.
netxtreme2-version.tar.gz: tar zipped package with NetXtreme II bnx2/bnx2x/cnic/bnx2fc/bnx2i/libfc/libfcoe driver
source.
iscsiuio-version.tar.gz: iSCSI user space management tool binary.
open-fcoe-*.brcm.<subvert>.<arch>.rpm: open-fcoe userspace management tool binary RPM for SLES11 SP2 and
legacy versions.
fcoe-utils-*.brcm.<subver>.<arch>.rpm: open-fcoe userspace management tool binary RPM for RHEL 6.4 and
legacy versions.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
The Linux driver has a dependency on open-fcoe userspace management tools as the front-end to control FCoE interfaces.
chkconfig lldpad on
The package name of the open-fcoe tool is fcoe-utils for RHEL 6.4 and open-fcoe for SLES11 SP2 and legacy versions.
Installing Linux Driver Software
Installing the Source RPM Package
Building the Driver from the Source TAR File
Installing the Binary DKMS RPM Driver Package
Installing the Binary DKMS RPM Driver Package
NOTE: If a bnx2/bnx2x/bnx2i driver is loaded and the Linux kernel is updated, the driver module must be recompiled if
the driver module was installed using the source RPM or the TAR package. This does not apply to the source DKMS RPM.
Installing the Source RPM Package
The following are guidelines for installing the driver source RPM Package.
Prerequisites:
Linux kernel source
C compiler
Procedure:
1. Install the source RPM package:
rpm -ivh netxtreme2-<version>.src.rpm
2. Change the directory to the RPM path and build the binary RPM for your kernel:
For RHEL:
cd ~/rpmbuild
rpmbuild -bb SPECS/netxtreme2.spec
For SLES:
cd /usr/src/packages
rpmbuild -bb SPECS/netxtreme2.spec
3. Install the newly compiled RPM:
rpm -ivh RPMS/<arch>/netxtreme2-<version>.<arch>.rpm
Note that the --force option may be needed on some Linux distributions if conflicts are reported.
1. Install the open-fcoe utility.
For RHEL 6.4 and legacy versions, either of the following:
yum install fcoe-utils-<version>.rhel.64.brcm.<subver>.<arch>.rpm
-or-
rpm -ivh fcoe-utils-<version>.rhel.64.brcm.<subver>.<arch>.rpm
For SLES11 SP2:
rpm -ivh open-fcoe-<version>.sles.sp2.brcm.<subver>.<arch>.rpm
For RHEL 6.4 and SLES11 SP2 and legacy versions, the version of fcoe-utils/open-fcoe included in your distribution
is sufficient and no out of box upgrades are provided.
Where available, installation with yum will automatically resolve dependencies. Otherwise, required dependencies
can be located on your O/S installation media.
2. For SLES, turn on the fcoe and lldpad services.
For SLES11 SP1:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
chkconfig fcoe on
1. Create a directory and extract the TAR files to the directory:
For SLES11 SP2:
chkconfig boot.lldpad on
chkconfig boot.fcoe on
3. Inbox drivers are included with all of the supported operating systems. The simplest means to ensure the newly
installed drivers are loaded is to reboot.
4. After rebooting, create configuration files for all FCoE ethX interfaces:
cd /etc/fcoe
cp cfg-ethx cfg-<ethX FCoE interface name>
NOTE: Note that your distribution might have a different naming scheme for Ethernet devices. (i.e., pXpX or emX
instead of ethX).
5. Modify /etc/fcoe/cfg-<interface> by setting DCB_REQUIRED=yes to DCB_REQUIRED=no.
6. Turn on all ethX interfaces.
ifconfig <ethX> up
7. For SLES, use YaST to configure your Ethernet interfaces to automatically start at boot by setting a static IP address or
enabling DHCP on the interface.
8. Disable lldpad on Broadcom CNA interfaces. This is required because Broadcom utilizes an offloaded DCBX client.
lldptool set-lldp –i <ethX> adminStatus=disasbled
9. Make sure /var/lib/lldpad/lldpad.conf is created and each <ethX> block does not specify "adminStatus" or if
specified, it is set to 0 ("adminStatus=0") as below.
lldp :
{
eth5 :
{
tlvid00000001 :
{
info = "04BC305B017B73";
};
tlvid00000002 :
{
info = "03BC305B017B73";
};
};
10. Restart lldpad service to apply new settings
For SLES11 SP1, RHEL 6.4 and legacy versions:
service lldpad restart
For SLES11 SP2:
rclldpad restart
11. Restart fcoe service to apply new settings
For SLES11 SP1, RHEL 6.4, and legacy versions:
service fcoe restart
For SLES11 SP2:
rcfcoe restart
Installing the KMP Package
NOTE: The examples in this procedure refer to the bnx2x driver, but also apply to the bnx2 and bnx2i drivers.
1. Install the KMP package:
rpm -ivh <file>
rmmod bnx2x
2. Load the driver:
modprobe bnx2x
Building the Driver from the Source TAR File
NOTE: The examples used in this procedure refer to the bnx2 driver, but also apply to the bnx2x driver.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
tar xvzf netxtreme2-version.tar.gz
2. Build the driver bnx2.ko (or bnx2i.ko) as a loadable module for the running kernel:
cd netxtreme2-version
make
3. Test the driver by loading it (first unload the existing driver, if necessary):
rmmod bnx2 (or bnx2x, or bnx2i)
insmod bnx2/src/bnx2.ko (or bnx2x/src/bnx2x.ko, or bnx2i/src/bnx2i.ko)
Verify that your network adapter supports iSCSI by checking the message log. If the message "bnx2i: dev eth0
does not support iSCSI" appears in the message log after loading the bnx2i driver, then iSCSI is not supported. This
message may not appear until the interface is opened, as with:
ifconfig eth0 up
4. Load the cnic driver (if applicable):
insmod cnic.ko
5. Install the driver and man page:
make install
NOTE: See the RPM instructions above for the location of the installed driver.
6. Install the user daemon (brcm_iscsiuio).
Refer to Load and Run Necessary iSCSI Software Components for instructions on loading the software components required to
use the Broadcom iSCSI offload feature.
To configure the network protocol and address after building the driver, refer to the manuals supplied with your operating
system.
Installing the Binary DKMS RPM Driver Package
Dynamic Kernel Module Support (DKMS) is designed to simplify the rebuilding of modules whenever you upgrade the kernel.
This is accomplished by creating a framework where a kernel-dependent module source can reside.
To install the binary DKMS RPM driver package
1. Download the binary DKMS RPM (dkms-version.noarch.rpm) from http://linux.dell.com/dkms/.
2. Install the binary DKMS RPM package:
rpm -ivh dkms-version.noarch.rpm
3. Install the DKMS RPM driver package:
rpm -ivh netxtreme2-version dkms.noarch.rpm
Verify that your network adapter supports iSCSI by checking the message log. If the message "bnx2i: dev eth0
does not support iSCSI" appears in the message log after loading the bnx2i driver, then iSCSI is not supported. This
message may not appear until the interface is opened, as with:
ifconfig eth0 up
4. To use Broadcom iSCSI, refer to Load and Run Necessary iSCSI Software Components to load the necessary software
components.
For more information, go to http://linux.dell.com.
Installing the Binary KMOD/KMP Driver Package
To install the binary KMOD/KMP driver package
1. Install the KMOD/KMP RPM driver package:
SUSE: rpm -ivh broadcom-netxtreme2-kmp-[kernel]-version.x86_64.rpm
Red Hat: kmod-kmp-netxtreme2-{kernel]-version.x86_64.rpm
Verify that your network adapter supports iSCSI by checking the message log. If the message "bnx2i: dev
eth0 does not support iSCSI" appears in the message log after loading the bnx2i driver, then iSCSI is not
supported. This message may not appear until the interface is opened, as with:
ifconfig eth0 up
2. To use Broadcom iSCSI, refer to Load and Run Necessary iSCSI Software Components to load the necessary software
components.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
For more information, go to http://linux.dell.com.
Load and Run Necessary iSCSI Software Components
The Broadcom iSCSI Offload software suite consists of three kernel modules and a user daemon. Required software
components can be loaded either manually or through system services.
1. Unload the existing driver, if necessary:
Manual:
rmmod bnx2i
2. Load the iSCSI driver:
Manual:
insmod bnx2i.ko
or
modprobe bnx2i
Unloading/Removing the Linux Driver
Unloading/Removing the Driver from an RPM Installation
Removing the Driver from a TAR Installation
Unloading/Removing the Driver from an RPM Installation
NOTES:
The examples used in this procedure refer to the bnx2 driver, but also apply to the bnx2x driver.
On 2.6 kernels, it is not necessary to bring down the eth# interfaces before unloading the driver module.
If the cnic driver is loaded, unload the cnic driver before unloading the bnx2 driver.
Prior to unloading the bnx2i driver, disconnect all active iSCSI sessions to targets.
To unload the driver, use ifconfig to bring down all eth# interfaces opened by the driver, and then type the following:
rmmod bnx2
NOTE: The above command will also remove bnx2, bnx2x, and cnic modules.
If the driver was installed using RPM, do the following to remove it:
rpm -e netxtreme2
Removing the Driver from a TAR Installation
NOTE: The examples used in this procedure refer to the bnx2 driver, but also apply to the bnx2x and bnx2i drivers.
If the driver was installed using make install from the tar file, the bnx2.ko driver file has to be manually deleted from the
operating system. See Installing the Source RPM Package for the location of the installed driver.
Uninstalling BACS
RPM Package
Use the following command:
% rpm -e BACS
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Patching PCI Files (Optional)
NOTE: The examples used in this procedure refer to the bnx2 driver, but also apply to the bnx2x and bnx2i drivers.
For hardware detection utilities such as Red Hat kudzu to properly identify bnx2 supported devices, a number of files
containing PCI vendor and device information may need to be updated.
Apply the updates by running the scripts provided in the supplemental tar file. For example, on Red Hat Enterprise Linux,
apply the updates by doing the following:
./patch_pcitbl.sh /usr/share/hwdata/pcitable pci.updates
/usr/share/hwdata/pcitable.new bnx2
./patch_pciids.sh /usr/share/hwdata/pci.ids pci.updates
/usr/share/hwdata/pci.ids.new
Next, the old files can be backed up and the new files can be renamed for use.
cp /usr/share/hwdata/pci.ids /usr/share/hwdata/old.pci.ids
cp /usr/share/hwdata/pci.ids.new /usr/share/hwdata/pci.ids
cp /usr/share/hwdata/pcitable /usr/share/hwdata/old.pcitable
cp /usr/share/hwdata/pcitable.new /usr/share/hwdata/pcitable
Network Installations
For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the
bnx2/bnx2x driver may be needed. The driver disk images for the most recent Red Hat and SuSE versions are included. Boot
drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is
available from the Red Hat website, http://www.redhat.com.
Setting Values for Optional Properties
Optional properties exist for the different drivers:
bnx2 Driver
bnx2x Driver
bnx2i Driver
bnx2 Driver
disable_msi
The disable_msi optional property can be supplied as a command line argument to the insmod or modprobe command. The
property can also be set in modprobe.conf. See the man page for more information. All other driver settings can be queried
and changed using the ethtool utility. See the ethtool man page for more information. The ethtool settings do not persist
across a reboot or module reload. The ethtool commands can be put in a startup script such as /etc/rc.local to preserve the
settings across a reboot.
NOTE: Some combinations of property values may conflict and result in failures. The driver cannot detect all such
conflicting combinations.
This property is used to disable Message Signal Interrupts (MSI), and the property is valid only on 2.6 kernels that support
MSI. By default, the driver enables MSI if it is supported by the kernel. It runs an interrupt test during initialization to
determine if MSI is working. If the test passes, the driver enables MSI. Otherwise, it uses legacy INTx mode.
insmod bnx2.ko disable_msi=1
or
modprobe bnx2 disable_msi=1
bnx2x Driver
disable_tpa
The disable_tpa parameter can be supplied as a command line argument to disable the Transparent Packet Aggregation
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
(TPA) feature. By default, the driver will aggregate TCP packets. Use disable_tpa to disable the advanced TPA feature.
Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all NetXtreme II network adapters in the
system. The parameter can also be set in modprobe.conf. See the man page for more information.
insmod bnx2x.ko disable_tpa=1
or
modprobe bnx2x disable_tpa=1
int_mode
The int_mode parameter is used to force using an interrupt mode.
Set the int_mode parameter to 1 to force using the legacy INTx mode on all NetXtreme II adapters in the system.
insmod bnx2x.ko int_mode=1
or
modprobe bnx2x int_mode=1
Set the int_mode parameter to 2 to force using MSI mode on all NetXtreme II adapters in the system.
insmod bnx2x.ko int_mode=2
or
modprobe bnx2x int_mode=2
Set the int_mode parameter to 3 to force using MSI-X mode on all NetXtreme II adapters in the system.
dropless_fc
The dropless_fc parameter can be used to enable a complementary flow control mechanism on BCM57711/BCM57712
adapters. The default flow control mechanism is to send pause frames when the on-chip buffer (BRB) is reaching a certain
level of occupancy. This is a performance targeted flow control mechanism. On BCM57711/BCM57712 adapters, one can
enable another flow control mechanism to send pause frames, where one of the host buffers (when in RSS mode) are
exhausted.
This is a "zero packet drop" targeted flow control mechanism.
Set the dropless_fc parameter to 1 to enable the dropless flow control mechanism feature on all BCM57711/BCM57712
NetXtreme II adapters in the system.
insmod bnx2x.ko dropless_fc=1
or
modprobe bnx2x dropless_fc=1
disable_iscsi_ooo
The disable_iscsi_ooo parameter is to disable the allocation of the iSCSI TCP Out-of-Order (OOO) reception resources,
specifically for VMware for low-memory systems.
multi_mode
The optional parameter multi_mode is for use on systems that support multi-queue networking. Multi-queue networking on
the receive side depends only on MSI-X capability of the system, multi-queue networking on the transmit side is supported
only on kernels starting from 2.6.27. By default, multi_mode parameter is set to 1. Thus, on kernels up to 2.6.26, the driver
will allocate on the receive side one queue per-CPU and on the transmit side only one queue. On kernels starting from 2.6.27,
the driver will allocate on both receive and transmit sides, one queue per-CPU. In any case, the number of allocated queues
will be limited by number of queues supported by hardware.
The multi_mode optional parameter can also be used to enable SAFC (Service Aware Flow Control) by differentiating the
traffic to up to 3 CoS (Class of Service) in the hardware according to the VLAN PRI value or according to the IP DSCP value
(least 3 bits).
num_queues
The optional parameter num_queues may be used to set the number of queues when multi_mode is set to 1 and interrupt
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
mode is MSI-X. If interrupt mode is different than MSI-X (see int_mode), the number of queues will be set to 1, discarding
the value of this parameter.
pri_map
The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or same CoS in the
hardware. This 32-bit parameter is evaluated by the driver as an 8 value of 4 bits each. Each nibble sets the desired hardware
queue number for that priority. For example, set pri_map to 0x11110000 to map priority 0 to 3 to CoS 0 and map priority 4
to 7 to CoS 1.
qs_per_cos
The optional parameter qs_per_cos is used to specify how many queues will share the same CoS. This parameter is
evaluated by the driver up to 3 values of 8 bits each. Each byte sets the desired number of queues for that CoS. The total
number of queues is limited by the hardware limit. For example, set qs_per_cos to 0x10101 to create a total of three
queues, one per CoS. In another example, set qs_per_cos to 0x404 to create a total of 8 queues, divided into 2 CoS, 4
queues in each CoS.
cos_min_rate
The optional parameter cos_min_rate is used to determine the weight of each CoS for round-robin scheduling in
transmission. This parameter is evaluated by the driver as up to 3 values of 8 bits each. Each byte sets the desired weight for
that CoS. The weight ranges from 0 to 100. For example, set cos_min_rate to 0x101 for fair transmission rate between 2
CoS. In another example, set the cos_min_rate to 0x30201 to give CoS the higher rate of transmission. To avoid using the
fairness algorithm, omit setting cos_min_rate or set it to 0.
Set the multi_mode parameter to 2 as shown below to differentiate the traffic according to the VLAN PRI value.
insmod bnx2x.ko multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404
or
modprobe bnx2x multi_mode=2 pri_map=0x11110000 qs_per_cos=0x404
Set the multi_mode parameter to 4 as shown below to differentiate the traffic according to the IP DSCP value.
insmod bnx2x.ko multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201
or
modprobe bnx2x multi_mode=4 pri_map=0x22221100 qs_per_cos=0x10101 cos_min_rate=0x30201
bnx2i Driver
Optional parameters en_tcp_dack, error_mask1, and error_mask2 can be supplied as command line arguments to the
insmod or modprobe command for bnx2i.
error_mask1 and error_mask2
"Config FW iSCSI Error Mask #", use to configure certain iSCSI protocol violation to be treated either as a warning or a fatal
error. All fatal iSCSI protocol violations will result in session recovery (ERL 0). These are bit masks.
Defaults: All violations will be treated as errors.
CAUTION! Do not use error_mask if you are not sure about the consequences. These values are to be discussed with
Broadcom development team on a case-by-case basis. This is just a mechanism to work around iSCSI implementation issues
on the target side and without proper knowledge of iSCSI protocol details, users are advised not to experiment with these
parameters.
en_tcp_dack
"Enable TCP Delayed ACK", enables/disables TCP delayed ACK feature on offloaded iSCSI connections.
Defaults: TCP delayed ACK is ENABLED. For example:
insmod bnx2i.ko en_tcp_dack=0
or
modprobe bnx2i en_tcp_dack=0
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
time_stamps
bnx2x Driver
"Enable TCP TimeStamps", enables/disables TCP time stamp feature on offloaded iSCSI connections.
Defaults: TCP time stamp option is DISABLED. For example:
insmod bnx2i.ko time_stamps=1
or
modprobe bnx2i time_stamps=1
sq_size
"Configure SQ size", used to choose send queue size for offloaded connections and SQ size determines the maximum SCSI
commands that can be queued. SQ size also has a bearing on the number of connections that can be offloaded; as QP size
increases, the number of connections supported will decrease. With the default values, the BCM5708 adapter can offload 28
connections.
Defaults: 128
Range: 32 to 128
Note that Broadcom validation is limited to a power of 2; for example, 32, 64, 128.
rq_size
"Configure RQ size", used to choose the size of asynchronous buffer queue size per offloaded connections. RQ size is not
required greater than 16 as it is used to place iSCSI ASYNC/NOP/REJECT messages and SCSI sense data.
Defaults: 16
Range: 16 to 32
Note that Broadcom validation is limited to a power of 2; for example, 16, 32.
event_coal_div
"Event Coalescing Divide Factor", performance tuning parameter used to moderate the rate of interrupt generation by the
iscsi firmware.
Defaults: 1
Valid values: 1, 2, 4, 8
last_active_tcp_port
"Last active TCP port used", status parameter used to indicate the last TCP port number used in the iSCSI offload connection.
Defaults: N/A
Valid values: N/A
Note: This is a read-only parameter.
ooo_enable
"Enable TCP out-of-order feature", enables/disables TCP out-of-order rx handling feature on offloaded iSCSI connections.
Defaults: TCP out-of-order feature is ENABLED. For example:
insmod bnx2i.ko ooo_enable=1
or
modprobe bnx2i ooo_enable=1
Driver Defaults
bnx2 Driver
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
bnx2 Driver
Speed: Autonegotiation with all speeds advertised
Flow Control: Autonegotiation with RX and TX advertised
MTU: 1500 (range is 46–9000)
RX Ring Size: 255 (range is 0–4080)
RX Jumbo Ring Size: 0 (range 0–16320) adjusted by the driver based on MTU and RX Ring Size
TX Ring Size: 255 (range is (MAX_SKB_FRAGS+1)–255). MAX_SKB_FRAGS varies on different kernels and different
architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.
Coalesce RX Microseconds: 18 (range is 0–1023)
Coalesce RX Microseconds IRQ: 18 (range is 0–1023)
Coalesce RX Frames: 6 (range is 0–255)
Coalesce RX Frames IRQ: 6 (range is 0–255)
Coalesce TX Microseconds: 80 (range is 0–1023)
Coalesce TX Microseconds IRQ: 80 (range is 0–1023)
Coalesce TX Frames: 20 (range is 0–255)
Coalesce TX Frames IRQ: 20 (range is 0–255)
Coalesce Statistics Microseconds: 999936 (approximately 1 second) (range is 0–16776960 in increments of 256)
MSI: Enabled (if supported by the 2.6 kernel and the interrupt test passes)
TSO: Enabled (on 2.6 kernels)
WoL: Initial setting based on NVRAM's setting
bnx2x Driver
Speed: Autonegotiation with all speeds advertised
Flow control: Autonegotiation with RX and TX advertised
MTU: 1500 (range is 46–9000)
RX Ring Size: 4078 (range is 0–4078)
TX Ring Size: 4078 (range is (MAX_SKB_FRAGS+4)–4078). MAX_SKB_FRAGS varies on different kernels and different
architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.
Coalesce RX Microseconds: 25 (range is 0–3000)
Coalesce TX Microseconds: 50 (range is 0–12288)
Coalesce Statistics Microseconds: 999936 (approximately 1 second) (range is 0–16776960 in increments of 256)
MSI-X: Enabled (if supported by the 2.6 kernel and the interrupt test passes)
TSO: Enabled
WoL: Disabled
Driver Messages
The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -n
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
<level> to control the level at which messages appear on the console. Most systems are set to level 6 by default. To see all
messages, set the level higher.
bnx2 and bnx2x Driver
bnx2i Driver
bnx2 and bnx2x Driver
NOTE: The examples used in this procedure refer to the bnx2 driver, but also apply to the bnx2x driver.
Driver Sign on
Broadcom NetXtreme II Gigabit Ethernet Driver
bnx2 v1.6.3c (July 23, 2007)
CNIC Driver Sign on (bnx2 only)
Broadcom NetXtreme II cnic v1.1.19 (Sep 25, 2007)
NIC Detected
eth#: Broadcom NetXtreme II BCM5708 1000Base-T (B0)
PCI-X 64-bit 133MHz found at mem f6000000, IRQ 16, node addr 0010180476ae
cnic: Added CNIC device: eth0
Link Up and Speed Indication
bnx2: eth# NIC Link is Up, 1000 Mbps full duplex
Link Down Indication
bnx2: eth# NIC Link is Down
MSI enabled successfully (bnx2 only)
bnx2: eth0: using MSI
MSI-X enabled successfully (bnx2x only)
bnx2x: eth0: using MSI-X
bnx2i Driver
BNX2I Driver signon
Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.1D (May 12, 2010)
Network port to iSCSI transport name binding
bnx2i: netif=eth2, iscsi=bcm570x-050000
bnx2i: netif=eth1, iscsi=bcm570x-030c00
Driver completes handshake with iSCSI offload-enabled CNIC device
bnx2i [05:00.00]: ISCSI_INIT passed
NOTE: This message is displayed only when the user attempts to make an iSCSI connection.
Driver detects iSCSI offload is not enabled on the CNIC device
bnx2i: iSCSI not supported, dev=eth3
bnx2i: bnx2i: LOM is not enabled to offload iSCSI connections, dev=eth0
bnx2i: dev eth0 does not support iSCSI
Exceeds maximum allowed iSCSI connection offload limit
bnx2i: alloc_ep: unable to allocate iscsi cid
bnx2i: unable to allocate iSCSI context resources
Network route to target node and transport name binding are two different devices
bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba $USER_CHOSEN_HBA
where ROUTE_HBA --> net device on which connection was offloaded based on route information USER_CHOSEN_HBA -->
HBA to which target node is bound (using iscsi transport name)
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Target cannot be reached on any of the CNIC devices
bnx2i: check route, cannot connect using cnic
Network route is assigned to network interface, which is down
bnx2i: check route, hba not found
Attempting to offload iSCSI connection onto a Jumbo Frame-enabled device
bnx2i: eth# network i/f mtu is set to #mtu
bnx2i: iSCSI HBA can support mtu of 1500
NOTE: Change mtu to 1500 using ifconfig and restart the interface in order to offload iSCSI connections.
SCSI-ML initiated host reset (session recovery)
bnx2i: attempting to reset host, #3
CNIC detects iSCSI protocol violation - Fatal errors
bnx2i: iscsi_error - wrong StatSN rcvd
bnx2i: iscsi_error - hdr digest err
bnx2i: iscsi_error - data digest err
bnx2i: iscsi_error - wrong opcode rcvd
bnx2i: iscsi_error - AHS len > 0 rcvd
bnx2i: iscsi_error - invalid ITT rcvd
bnx2i: iscsi_error - wrong StatSN rcvd
bnx2i: iscsi_error - wrong DataSN rcvd
bnx2i: iscsi_error - pend R2T violation
bnx2i: iscsi_error - ERL0, UO
bnx2i: iscsi_error - ERL0, U1
bnx2i: iscsi_error - ERL0, U2
bnx2i: iscsi_error - ERL0, U3
bnx2i: iscsi_error - ERL0, U4
bnx2i: iscsi_error - ERL0, U5
bnx2i: iscsi_error - ERL0, U
bnx2i: iscsi_error - invalid resi len
bnx2i: iscsi_error - MRDSL violation
bnx2i: iscsi_error - F-bit not set
bnx2i: iscsi_error - invalid TTT
bnx2i: iscsi_error - invalid DataSN
bnx2i: iscsi_error - burst len violation
bnx2i: iscsi_error - buf offset violation
bnx2i: iscsi_error - invalid LUN field
bnx2i: iscsi_error - invalid R2TSN field
bnx2i: iscsi_error - invalid cmd len1
bnx2i: iscsi_error - invalid cmd len2
bnx2i: iscsi_error - pend r2t exceeds MaxOutstandingR2T value
bnx2i: iscsi_error - TTT is rsvd
bnx2i: iscsi_error - MBL violation
bnx2i: iscsi_error - data seg len != 0
bnx2i: iscsi_error - reject pdu len error
bnx2i: iscsi_error - async pdu len error
bnx2i: iscsi_error - nopin pdu len error
bnx2i: iscsi_error - pend r2t in cleanup
bnx2i: iscsi_error - IP fragments rcvd
bnx2i: iscsi_error - IP options error
bnx2i: iscsi_error - urgent flag error
CNIC detects iSCSI protocol violation - non-FATAL, warning
bnx2i: iscsi_warning - invalid TTT
bnx2i: iscsi_warning - invalid DataSN
bnx2i: iscsi_warning - invalid LUN field
NOTE: The driver needs to be configured to consider certain violation to treat as warning and not as a critical error.
Driver puts a session through recovery
conn_err - hostno 3 conn 03fbcd00, iscsi_cid 2 cid a1800
Reject iSCSI PDU received from the target
bnx2i - printing rejected PDU contents
[0]: 1 ffffffa1 0 0 0 0 20 0
[8]: 0 7 0 0 0 0 0 0
[10]: 0 0 40 24 0 0 ffffff80 0
[18]: 0 0 3 ffffff88 0 0 3 4b
[20]: 2a 0 0 2 ffffffc8 14 0 0
[28]: 40 0 0 0 0 0 0 0
Open-iSCSI daemon handing over session to driver
bnx2i: conn update - MBL 0x800 FBL 0x800MRDSL_I 0x800 MRDSL_T 0x2000
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Teaming with Channel Bonding
With the Linux drivers, you can team adapters together using the bonding kernel module and a channel bonding interface. For
more information, see the Channel Bonding information in your operating system documentation.
Remote PHY Support
The bnx2 driver supports Remote PHY on blade servers that use the NetXtreme II BCM5708S or BCM5709s device, support
Remote PHY, and have Remote PHY enabled.
On Remote PHY-enabled systems, the bnx2 driver enables the NetXtreme II BCM5708S or BCM5709S device to take
advantage of the features available in the blade chassis' copper twisted pair PHY. The bnx2 driver will indicate that it is using
Remote PHY twisted pair mode in ethtool output.
NOTE: bnx2 driver versions prior to 1.63d do not support Remote PHY and operating systems using these older drivers
may behave differently than operating systems with Remote PHY driver support. Refer to your system and operating system
documentation for the status of Remote PHY support.
Configure the adapter using the standard ethtool commands.
Statistics
Detailed statistics and configuration information can be viewed using the ethtool utility. See the ethtool man page for more
information.
Linux iSCSI Offload
Open iSCSI User Applications
User Application - brcm_iscsiuio
Bind iSCSI Target to Broadcom NX2 iSCSI Transport Name
VLAN Configuration for iSCSI Offload (Linux)
Making Connections to iSCSI Targets
Maximum Offload iSCSI Connections
Linux iSCSI Offload FAQ
Open iSCSI User Applications
Install and run the inbox open-iscsi initiator programs from the DVD. Refer to Packaging for details.
User Application - brcm_iscsiuio
Install and run the brcm_iscsiuio daemon before attempting to create iSCSI connections. The driver will not be able to
establish connections to the iSCSI target without the daemon's assistance.
1. Install the brcm_iscsiuio source package
# tar -xvzf iscsiuio-<version>.tar.gz
2. CD to the directory where iscsiuio is extracted
# cd iscsiuio-<version>
3. Compile and install
# ./configure
# make
# make install
4. Check the iscsiuio version matches with the source package
# brcm_iscsiuio -v
5. Start brcm_iscsiuio
# brcm_iscsiuio
Bind iSCSI Target to Broadcom NX2 iSCSI Transport Name
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
"BOOTPROTO=static
By default, the open-iscsi daemon connects to discovered targets using software initiator (transport name = 'tcp'). Users who
wish to offload iSCSI connection onto CNIC device should explicitly change transport binding of the iSCSI iface. This can be
done using the iscsiadm CLI utility as follows,
iscsiadm -m iface -I <iface_file_name> -n iface.transport_name -v bnx2i -o update
where the iface file includes the following information for RHEL 5.4, RHEL 5.5, and SLES 11 SP1:
iface.net_ifacename = ethX
iface.iscsi_ifacename = <name of the iface file>
iface.hwaddress = XX:XX:XX:XX:XX:XX
iface.ipaddress = XX.XX.XX.XX
iface.transport_name = bnx2i
Ensure that the iface.hwaddress is in lower case format.
If you wish to switch back to use the software initiator, use the following:
iscsiadm -m iface -I <iface_file_name> -n iface.transport_name -v tcp -o update
where the iface file includes the following information:
iface.net_ifacename = ethX
iface.iscsi_ifacename = <name of the iface file>
iface.transport_name = tcp
VLAN Configuration for iSCSI Offload (Linux)
iSCSI traffic on the network may be isolated in a VLAN to segregate it from other traffic. When this is the case, you must
make the iSCSI interface on the adapter a member of that VLAN.
Modifying the iSCSI iface File
To configure the iSCSI VLAN add the VLAN ID in the iface file for iSCSI. In the following example, the VLAN ID is set to 100.
#Begin Record 6.2.0-873.2.el6
Iface.iscsi_ifacefile name = <>
Iface.ipaddress = 0.0.0.0
Iface.hwaddress = <>
Iface.trasport_name = bnx2i
Iface.vlan_id = 100
Iface.vlan_priority = 0
Iface.iface_num = 100
Iface.mtu = 0
Iface.port = 0
#END Record
NOTE: Although not strictly required, Broadcom recommends configuring the same VLAN ID on the iface.iface_num field
for iface file identification purposes.
Setting the VLAN ID on the Ethernet Interface
If using RHEL5.X versions of Linux, it is recommended that you configure the iSCSI VLAN on the Ethernet interface. In
RHEL6.3, and sles11sp3, it is not necessary to set the VLAN on the Ethernet driver.
Execute the following commands to set the VLAN ID:
Vconfig add ethx <vlan number> — Creates an L2 VLAN interface.
Ifconfig eth.<VLANID> <static ip> up — Assigns and IP address to the VLAN interface.
Use the following command to get detailed information about VLAN interface:
# cat /proc/net/vlan/ethx.<vlanid>
Preserve the VLAN configuration across reboots by adding it to configuration files. Configure the VLAN interface configuration
in /etc/sysconfig/network-scripts. The configuration filename has a specific format that includes the physical interface, a
character, and the VLAN ID.
For example, if the VLAN ID is 100, and the physical interface is eth0, then the configuration filename should be ifcfg-
eth0.100. The following are example settings in the configuration file.
"DEVICE=ethX.100
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Linux Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
"ONBOOT=yes
"IPADDR=<>
"NETMASK=<>
"USERCTL=no
"NETWORK=<>
"VLAN=yes
Restart the networking service, in order for the changes to take effect, as follows:
"Service network restart"
Making Connections to iSCSI Targets
Refer to open-iscsi documentation for a comprehensive list of iscsiadm commands. This is a sample list of commands to
discovery targets and to create iscsi connections to a target.
Add static entry
iscsiadm -m node -p <ipaddr[:port]> -T iqn.2007-05.com.broadcom:target1 -o new -I <iface_file_name>
iSCSI target discovery using 'SendTargets'
iscsiadm -m discovery --type sendtargets -p <ipaddr[:port]> -I <iface_file_name>
Login to target using 'iscsiadm' command
iscsiadm --mode node --targetname <iqn.targetname> --portal <ipaddr[:port]> --login
List all drives active in the system
fdisk -l
Maximum Offload iSCSI Connections
With default driver parameters set, which includes 128 outstanding commands, bnx2i can offload the following number of
connections:
BCM5708: 28
BCM5709: 43
BCM5771x: 128
This is not a hard limit, but just a simple on-chip resource allocation math. bnx2i will be able to offload > 28 connections on
1G devices by reducing the shared queue size, which in turn limits the maximum outstanding tasks on a connection. See
Setting Values for Optional Properties for information on sq_size and rq_size. The driver logs the following message to syslog
when the maximum allowed connection offload limit is reached - "bnx2i: unable to allocate iSCSI context resources".
Linux iSCSI Offload FAQ
Not all Broadcom NetXtreme II adapters support iSCSI offload.
The iSCSI session will not recover after a hot remove and hot plug.
For MPIO to work properly, iSCSI nopout should be enabled on each iSCSI session. Refer to open-iscsi documentation
for procedures on setting up noop_out_interval and noop_out_timeout values.
In the scenario where multiple CNIC devices are in the system and the system is booted via Broadcom's iSCSI boot
solution, ensure that the iscsi node under /etc/iscsi/nodes for the boot target is bound to the NIC that is used for
booting.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/linux.htm[9/5/2014 3:45:01 PM]
Solaris Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Solaris Driver Software: Broadcom NetXtreme II® Network
Adapter User Guide
Overview
Installing the Driver
Upgrading the Driver
Uninstalling Driver
Configuring the Driver
Memory Usage
Interrupt Management
FCoE Support
Overview
This file describes how to install the Solaris driver for Broadcom's NetXtreme II 10 Gigabit Ethernet network adapters. Refer to
the 'bnxe' manual page for details on how to configure the driver.
The Solaris driver is released in two formats:
BRCMbnxe-version.pkg: Datastream format
BRCMbnxe-version.tar.Z: Compressed TAR file system format.
NOTE: A DU image does not exist at this time because of driver size limitations. Solaris DU diskettes can be used to
install the driver into the system both during system installation and/or after the system has been installed and
booted.
This driver only works with the GLDv3 Streams interface as it appears in Solaris 10 (Update 4) and later.
Installing the Driver
1. Change directory to where BRCMbnxe-version.pkg resides.
2. pkgadd -d BRCMbnxe-version.pkg
or
1. Copy BRCMbnxe-X.Y.Z.tar.Z to /tmp.
2. cd /tmp
uncompress BRCMbnxe-version.tar.Z
tar -xvf BRCMbnxe-version.tar
pkgadd -d /tmp
3. Execute prtconf to determine instance numbers of the NIC.
4. ifconfig bnxe[instance_number] plumb
5. ifconfig bnxe[instance_number] ip_address netmask .... up
To make these changes permanent:
1. Use your favorite text editor and create a file named hostname.bnxe[instance_number] in the /etc directory. Add the
IP address of the interface to this file, save, and exit.
2. Add a proper subnet mask to the file /etc/netmasks.
Upgrading the Driver
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/solaris.htm[9/5/2014 3:45:02 PM]
Solaris Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
To upgrade the Broadcom driver package to the current version, you must first uninstall the previous driver version from the
system. See Uninstalling Driver. Once the previous driver has been removed, you can follow any of the installation methods
in this document to install the new driver version.
NOTE: Do not install multiple instances of the driver on a single system.
Uninstalling Driver
1. ifconfig bnxe[instance_number] down
2. ifconfig bnxe[instance_number] unplumb
3. pkgrm BRCMbnxe
Configuring the Driver
The bnxe driver can be configured via the bnxe.conf file installed under /kernel/drv. When this config file is modified, the
system must be either rebooted or the driver unloaded and reconfigured using the update_drv admin command.
All configuration can be specified per-instance. The format used is as follows and each line must end with a semicolon:
bnxe<#>_<config_item>=X;
So for adv_autoneg_cap, you would use the following:
bnxe0_adv_autoneg_cap=1;
bnxe1_adv_autoneg_cap=0;
bnxe2_adv_autoneg_cap=1;
bnxe3_adv_autoneg_cap=1;
If a configuration item is not specified for a specific instance, then the default value will be used. The default value used by all
instances can be overridden using:
default_<config_item>=X;
For boolean values, 1 = TRUE and 0 = FALSE.
Memory Usage
The number of RX/TX buffer descriptors specified in the configuration file can have a detrimental affect on memory usage. If
the counts are too high, DMA allocations can fail, thereby affecting other drivers loaded on the system. If DMA allocations fail
during system initialization and/or boot, then there is a chance the system will not boot. This behavior is an implementation
constraint of the Solaris OS. Additionally, it has been seen that the amount of DMA allocation space available on a system
running in 32-bit mode is less than when running as 64-bit.
For a single RX descriptor, the following is allocated:
1 DMA handle
1 DMA memory buffer that is MTU in size
1K memory overhead
For a single TX descriptor, the following is allocated:
9 DMA handles for sending chained mblks
1 DMA memory buffer that is MTU in size
1K memory overhead
NOTE: The number of DMA handles available in the system scales with the amount of RAM. With more RAM, the
descriptor counts can be safely increased.
The default number of RX/TX buffer descriptors is 2048 for each. When using a Broadcom BCM57711 network adapter in
multifunction mode, the number of configured descriptors is divided by four, ending up at 512. This is to keep the number of
DMA allocations at a minimum. After installation, it is suggested these descriptor counts be increased until stability is
guaranteed and the desired performance is reached.
For example, using the default setting of 2048 for the number of both RX and TX descriptors, the approximate amount of
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/solaris.htm[9/5/2014 3:45:02 PM]
Solaris Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
memory a single interface would consume is:
Single Function Mode
RX: 2048 DMA handles and 5M (MTU=1500) or 21M (MTU=9216) of memory
TX: 20480 DMA handles and 5M (MTU=1500) or 21M (MTU=9216) of memory
Total: 22528 DMA handles and 10M (MTU=1500) or 42M (MTU=9216) of memory
Multifunction Mode (#descs / 4)
RX: 512 DMA handles and 1M (MTU=1500) or 5M (MTU=9216) of memory
TX: 5120 DMA handles and 1M (MTU=1500) or 5M (MTU=9216) of memory
Total: 5335 DMA handles and 2M (MTU=1500) or 10M (MTU=9216) of memory
Interrupt Management
If you have a system with many interfaces, it is possible to reach the allocation limit of MSIX interrupts. By default, Solaris
limits each driver to 2 MSIX allocations, and there is an issue with the pcplusmp module where only a maximum of 31 MSIX
interrupts are available per interrupt priority level.
If your system has four Broadcom BCM57711 network adapter ports, each running in multifunction mode, Solaris will
enumerate 16 bnxe interfaces. The last interface attached will fail to allocate its second MSIX interrupt and revert to Fixed.
This in turn can eventually expose an issue in the system regarding interrupt management resulting in interrupts never being
received on the interface that reverted back to Fixed.
To ensure all interfaces are able to allocate their two MSIX interrupts, the workaround is to change the priority levels of
specific interfaces. Network drivers are automatically assigned an interrupt priority level of 6, so changing an interface's
priority level to 5 is common.
1. First read the driver.conf man page for a background primer.
2. Find out the driver instance paths assigned on your system.
% grep bnxe /etc/path_to_inst
"/pci@0,0/pci8086,2779@1/pci14e4,1650@0" 0 "bnxe"
"/pci@0,0/pci8086,2779@1/pci14e4,1650@0,1" 1 "bnxe"
3. Normally, the name of the driver is the last portion of the path, but you should use the most appropriate PCI ID found
in /etc/driver_aliases. Depending on how the hardware is layered, there are cases where the name identified in
path_to_inst will not work. To figure out which name to use, examine the output from prtconf -v and match against
the IDs specified in the driver_aliases file.
% grep bnxe /etc/driver_aliases
bnxe "pci14e4,164e"
bnxe "pci14e4,164f"
bnxe "pci14e4,1650"
bnxe "pciex14e4,164e"
bnxe "pciex14e4,164f"
bnxe "pciex14e4,1650"
4. The parent of the driver is the entire path leading up to the name.
5. The unit-address is located after the final @ in the path.
6. Therefore, change both of the bnxe interfaces found in path_to_inst to interrupt priority 5 and use the following config
lines to bnxe.conf:
name = "pciex14e4,1650" parent = "/pci@0,0/pci8086,2779@1" unit-address = "0" interrupt-priorities =
5;
name = "pciex14e4,1650" parent = "/pci@0,0/pci8086,2779@1" unit-address = "0,1" interrupt-priorities
= 5;
7. After modifying the config, either reboot the system or unplumb all interfaces and run the update_drv command.
8. When the system has been reconfigured and the interfaces plumbed back up, verify the new interrupt priority settings
by running the following command as root:
% echo "::interrupts -d" | mdb -k
FCoE Support
Overview
FCoE is supported on Solaris 11 with limited support on Solaris 10, Update 9. The following features are the differences in
Solaris 10, Update 9 when compared to Solaris 11:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/solaris.htm[9/5/2014 3:45:02 PM]
Solaris Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Support does not exist for NPIV in the Solaris 10 Update 9.
Some of the fcinfo(1M) options, which are available in Solaris 11, are not be available in Solaris 10, Update 9. For more
information, read the man page fcinfo(1M).
brcmfcoeadm(1M) feature is supported in both Solaris 10 Update 9 and Solaris 11. However, when "delete-fcoe-port" is
complete, you need to issue the following two commands to unload the bnxef driver before you can re-issue "createfcoe-port". There is a reaper thread in Solaris 11 that aggressively looks for unused driver modules and unloads the
driver. That thread does not exist in Solaris 10 Update 9. Therefore, you have to explicitly look for the driver module ID
of the bnxef driver by issuing the following command.
# modinfo | grep bnxef
249 fffffffff8d63000 486b8 54 1 bnxef (6.4.13)
Then issue the modunload command to unload the module before "create-fcoe-port" is issued to create a new FCoE
port.
# modunload -i 249
Any time "create-fcoe-port" needs to be issued, the driver must be unloaded, if it is already loaded. If not, the "create-fcoeport" will fail indicating the driver is busy. This is true when you have two or more instances of bnxef loaded, in which case,
you should first delete all FCoE ports and then unload the driver. Unloading will occur only when all the instances are deleted.
Supported FC/FCoE Devices
The bnxef Broadcom 10 Gb FCoE driver works with all the major FCoE fabric devices. It is also compatible with all known FC
disk devices and tape devices working through the FCoE fabric.
Unloading FCoE Driver
Delete all FCoE ports created across the various bnxe instances.
1. Delete all the NPIV ports created before deleting FCoE ports.
2. brcmfcoeadm delete-fcoe-port bnxe<instance_number>
3. modinfo | grep bnxef
The first column for the above command will give the module ID for the bnxef driver.
4. modunload -i <module id>
The procedure should unload the driver. However, if there are many instances of the FCoE ports created, all the FCoE ports
must be deleted before the unload can be attempted.
Configuring the FCoE Driver
The bnxef driver can be configured via the bnxef.conf file installed under /kernel/drv. When this config file is modified, the
system must be either rebooted or use the update_drv(1M) command to update the driver configuration.
The details of the configurations parameters are detailed in the bnxef(7D) man page. The default parameters should work for
all conditions.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/solaris.htm[9/5/2014 3:45:02 PM]
VMware Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
VMware Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Packaging
Networking Support
Drivers
FCoE Support
Packaging
The VMware driver is released in the packaging formats shown in Table 1.
Table 1: VMware Driver Packaging
Format Drivers
Compressed tar bnx2x-version.tar.gz
VMware VIB vmware-esx-drivers-net-bnx2x-version.x86_64.vib
Networking Support
This section describes the bnx2x VMware ESX driver for the Broadcom NetXtreme II PCIE 10 GbE network adapters.
Drivers
Download, Install, and Update Drivers
To download, install, or update the VMware ESX/ESXi driver for NetXtreme II 10 GbE network adapters, see http://www.vmware.com/support.
Driver Parameters
Several optional parameters can be supplied as a command line argument to the vmkload_mod command. These parameters can also be set via the esxcfg-module
command. See the man page for more information.
int_mode
The optional parameter int_mode is used to force using an interrupt mode other than MSI-X. By default, the driver will try to enable MSI-X if it is supported by the
kernel. If MSI-X is not attainable, then the driver will try to enable MSI if it is supported by the kernel. If MSI is not attainable, then the driver will use the legacy
INTx mode.
Set the int_mode parameter to 1 as shown below to force using the legacy INTx mode on all NetXtreme II network adapters in the system.
vmkload_mod bnx2x int_mode=1
Set the int_mode parameter to 2 as shown below to force using MSI mode on all NetXtreme II network adapters in the system.
vmkload_mod bnx2x int_mode=2
disable_tpa
The optional parameter disable_tpa can be used to disable the Transparent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets,
but if you would like to disable this advanced feature, it can be done.
Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all NetXtreme II network adapters in the system.
vmkload_mod bnx2x.ko disable_tpa=1
Use ethtool to disable TPA (LRO) for a specific network adapter.
num_rx_queues
The optional parameter num_rx_queues may be used to set the number of Rx queues on kernels starting from 2.6.24 when multi_mode is set to 1 and interrupt
mode is MSI-X. Number of Rx queues must be equal to or greater than the number of Tx queues (see num_tx_queues parameter). If the interrupt mode is
different than MSI-X (see int_mode parameter), then then the number of Rx queues will be set to 1, discarding the value of this parameter.
num_tx_queues
The optional parameter num_tx_queues may be used to set the number of Tx queues on kernels starting from 2.6.27 when multi_mode is set to 1 and interrupt
mode is MSI-X. The number of Rx queues must be equal to or greater than the number of Tx queues (see num_rx_queues parameter). If the interrupt mode is
different than MSI-X (see int_mode parameter), then the number of Tx queues will be set to 1, discarding the value of this parameter.
pri_map
The optional parameter pri_map is used to map the VLAN PRI value or the IP DSCP value to a different or the same CoS in the hardware. This 32-bit parameter is
evaluated by the driver as 8 values of 4 bits each. Each nibble sets the desired hardware queue number for that priority.
For example, set the pri_map parameter to 0x22221100 to map priority 0 and 1 to CoS 0, map priority 2 and 3 to CoS 1, and map priority 4 to 7 to CoS 2. In
another example, set the pri_map parameter to 0x11110000 to map priority 0 to 3 to CoS 0, and map priority 4 to 7 to CoS 1.
qs_per_cos
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vmware.htm[9/5/2014 3:45:03 PM]
VMware Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
The optional parameter qs_per_cos is used to specify the number of queues that will share the same CoS. This parameter is evaluated by the driver up to 3 values
vmkload_mod bnx2x multi_mode=0
of 8 bits each. Each byte sets the desired number of queues for that CoS. The total number of queues is limited by the hardware limit.
For example, set the qs_per_cos parameter to 0x10101 to create a total of three queues, one per CoS. In another example, set the qs_per_cos parameter to
0x404 to create a total of 8 queues, divided into only 2 CoS, 4 queues in each CoS.
cos_min_rate
The optional parameter cos_min_rate is used to determine the weight of each CoS for Round-robin scheduling in transmission. This parameter is evaluated by the
driver up to 3 values of 8 bits each. Each byte sets the desired weight for that CoS. The weight ranges from 0 to 100.
For example, set the cos_min_rate parameter to 0x101 for fair transmission rate between two CoS. In another example, set the cos_min_rate parameter to
0x30201 to give the higher CoS the higher rate of transmission. To avoid using the fairness algorithm, omit setting the optional parameter cos_min_rate or set it
to 0.
dropless_fc
The optional parameter dropless_fc can be used to enable a complementary flow control mechanism on Broadcom network adapters. The default flow control
mechanism is to send pause frames when the on-chip buffer (BRB) is reaching a certain level of occupancy. This is a performance targeted flow control mechanism.
On Broadcom network adapters, you can enable another flow control mechanism to send pause frames if one of the host buffers (when in RSS mode) is exhausted.
This is a "zero packet drop" targeted flow control mechanism.
Set the dropless_fc parameter to 1 as shown below to enable the dropless flow control mechanism feature on all Broadcom network adapters in the system.
vmkload_mod bnx2x dropless_fc=1
Driver Defaults
Speed: Autonegotiation with all speeds advertised
Flow Control: Autonegotiation with rx and tx advertised
MTU: 1500 (range 46–9000)
Rx Ring Size: 4078 (range 0–4078)
Tx Ring Size: 4078 (range (MAX_SKB_FRAGS+4) - 4078). MAX_SKB_FRAGS varies on different kernels and different architectures. On a 2.6 kernel for x86,
MAX_SKB_FRAGS is 18.
Coalesce RX Microseconds: 25 (range 0–3000)
Coalesce TX Microseconds: 50 (range 0–12288)
MSI-X: Enabled (if supported by 2.6 kernel)
TSO: Enabled
WoL: Disabled
Unloading and Removing Driver
To unload the driver, type the following:
vmkload_mod -u bnx2x
Driver Messages
The following are the most common sample messages that may be logged in the file /var/log/messages. Use dmesg -n <level> to control the level at which
messages will appear on the console. Most systems are set to level 6 by default. To see all messages, set the level higher.
Driver Sign On
Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver
bnx2x 0.40.15 ($DateTime: 2007/11/22 05:32:40 $)
NIC Detected
eth0: Broadcom NetXtreme II BCM57710 XGb (A1)
PCI-E x8 2.5GHz found at mem e8800000, IRQ 16, node addr 001018360012
MSI-X Enabled Successfully
bnx2x: eth0: using MSI-X
Link Up and Speed Indication
bnx2x: eth0 NIC Link is Up, 10000 Mbps full duplex, receive & transmit flow control ON
Link Down Indication
bnx2x: eth0 NIC Link is Down
Memory Limitation
If you see messages in the log file that look like the following, then the ESX host is severely strained. To relieve this, disable NetQueue.
Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: Heap: 1435: Heap bnx2x already at its maximumSize. Cannot expand.
Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: Heap: 1645: Heap_Align(bnx2x, 4096/4096 bytes, 4096 align) failed.
caller: 0x41800187d654
Dec 2 18:24:20 ESX4 vmkernel: 0:00:00:32.342 cpu2:4142)WARNING: vmklinux26: alloc_pages: Out of memory
Disable NetQueue by manually loading the bnx2x vmkernel module via the command.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vmware.htm[9/5/2014 3:45:03 PM]
VMware Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
or to persist the settings across reboots via the command
esxcfg-module -s multi_mode=0 bnx2x
Reboot the machine for the settings to take place.
MultiQueue/NetQueue
The optional parameter num_queues may be used to set the number of Rx and Tx queues when multi_mode is set to 1 and interrupt mode is MSI-X. If interrupt
mode is different than MSI-X (see int_mode parameter), the number of Rx and Tx queues will be set to 1, discarding the value of this parameter.
If you would like the use of more then 1 queue, force the number of NetQueues to use via the following command:
esxcfg-module -s "multi_mode=1 num_queues=<num of queues>" bnx2x
Otherwise, allow the bnx2x driver to select the number of NetQueues to use via the following command:
esxcfg-module -s "multi_mode=1 num_queues=0" bnx2x
The optimal number is to have the number of NetQueues match the number of CPUs on the machine.
FCoE Support
This section describes the contents and procedures associated with installation of the VMware software package for supporting Broadcom FCoE C-NICs.
Drivers
Table 2: Broadcom NetXtreme II FCoE Drivers
Driver Description
This driver manages all PCI device resources (registers, host interface queues, etc.) and also acts as the Layer 2 VMware low-level network driver for
Broadcom's NetXtreme II 10G device. This driver directly controls the hardware and is responsible for sending and receiving Ethernet packets on behalf of
bnx2x
the VMware host networking stack. The bnx2x driver also receives and processes device interrupts, both on behalf of itself (for L2 networking) and on
behalf of the bnx2fc (FCoE protocol) and CNIC drivers.
The Broadcom VMware FCoE driver is a kernel mode driver used to provide a translation layer between the VMware SCSI stack and the Broadcom FCoE
bnx2fc
firmware/hardware. In addition, the driver interfaces with the networking layer to transmit and receive encapsulated FCoE frames on behalf of open-fcoe's
libfc/libfcoe for FIP/device discovery.
Supported Distributions
The FCoE/DCB feature set is supported on VMware ESXi 5.0 and above.
iSCSI Support
This following driver is provided to support iSCSI.
Table 3: Broadcom NetXtreme II iSCSI Driver
Driver Description
The bnx2i driver is Broadcom VMware iSCSI HBA driver. Similar to bnx2fc, bnx2i is a kernel mode driver used to provide a translation layer between the
bnx2i
VMware SCSI stack and the Broadcom iSCSI firmware/hardware. Bnx2i functions under the open-iscsi framework.
VLAN Configuration for iSCSI Offload (Linux)
iSCSI traffic on the network may be isolated in a VLAN to segregate it from other traffic. When this is the case, you must make the iSCSI interface on the adapter a
member of that VLAN.
To configure the VLAN using the V-Sphere client (GUI):
1. Click the ESXi/ESX host.
2. Click the Configuration tab.
3. Click the Networking link and click Properties.
4. Click the virtual switch / portgroups in the Ports tab and click Edit.
5. Click the General tab.
6. Assign a VLAN number in VLAN ID (optional).
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vmware.htm[9/5/2014 3:45:03 PM]
VMware Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
7. Configure the VLAN on VMKernel.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vmware.htm[9/5/2014 3:45:03 PM]
VMware Driver Software: Broadcom NetXtreme II® Network Adapter User Guide
Enabling FCoE
To enable FCoE hardware offload on the C-NIC
1. Determine the ports that are FCoE-capable:
# esxcli fcoe nic list
Output example:
vmnic4
User Priority: 3
Source MAC: FF:FF:FF:FF:FF:FF
Active: false
Priority Settable: false
Source MAC Settable: false
VLAN Range Settable: false
1. Enable the FCoE interface:
# esxcli fcoe nic discover -n vmnicX
Where X is the interface number gained from esxcli fcoe nic list.
2. Verify that the interface is working:
# esxcli fcoe adapter list
Output example:
vmhba34
Source MAC: bc:30:5b:01:82:39
FCF MAC: 00:05:73:cf:2c:ea
VNPort MAC: 0e:fc:00:47:04:04
Physical NIC: vmnic7
User Priority: 3
VLAN id: 2008
The output of this command should show valid:
FCF MAC, VNPort MAC, Priority, and VLAN id for the Fabric that is connected to the C-NIC.
The following command can also be used to verify that the interface is working properly:
#esxcfg-scsidevs -a
Output example:
vmhba34 bnx2fc link-up fcoe.1000<mac address>:2000<mac address> () Software FCoE
vmhba35 bnx2fc link-up fcoe.1000<mac address>:2000<mac address> () Software FCoE
NOTE: The label "Software FCoE" is a VMware term used to describe initiators that depend on the inbox FCoE libraries and utilities. Broadcom's FCoE solution is
a fully state connection-based hardware offload solution designed to significantly reduce the CPU burden encumbered by a non-offload software initiator.
Installation Check
To verify the correct installation of the driver and to ensure that the host port is seen by the switch, follow the procedure below.
To verify the correct installation of the driver
1. Verify the host port shows up in the switch FLOGI database using the "show flogi database" command for the case of a Cisco FCF and "fcoe -loginshow"
command for the case of a Brocade FCF.
2. If the Host WWPN does not appear in the FLOGI database, then provide driver log messages for review.
Limitations
NPIV is not currently supported with this release on ESX, due to lack of supporting inbox components.
Non-offload FCoE is not supported with offload-capable Broadcom devices. Only the full hardware offload path is supported.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/vmware.htm[9/5/2014 3:45:03 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Windows Driver and Management Application Installation:
Broadcom NetXtreme II
Installing the Driver Software
Modifying the Driver Software
Repairing or Reinstalling the Driver Software
Removing the Device Drivers
Using the NetXtreme II Monolithic Driver
Inserting the NetXtreme II Monolithic Driver in a in a WinPE 2.0 or 3.1 Image
Configuring the Speed/Duplex Setting for the NetXtreme II Monolithic Driver
Viewing or Changing the Properties of the Adapter
Setting Power Management Options
Configuring the Communication Protocol To Use With BACS4
®
Network Adapter User Guide
Installing the Driver Software
NOTE: These instructions are based on the assumption that your Broadcom NetXtreme II adapter was not factory installed.
If your controller was installed at the factory, the driver software has been installed for you.
When Windows first starts after a hardware device (such as a Broadcom NetXtreme II Adapter) has been installed, or after
the existing device driver has been removed, the operating system automatically detects the hardware and prompts you to
install the driver software for that device.
Both a graphical interactive installation mode (see Using the Installer) and a command-line silent mode for unattended
installation (see Using Silent Installation) are available.
NOTES:
Before installing the driver software, verify that the Windows operating system has been upgraded to the latest
version with the latest service pack applied.
A network device driver must be physically installed before the Broadcom NetXtreme II Controller can be used
with your Windows operating system. Drivers are located on the driver source media as well as at the Dell
website at http://support.dell.com.
To use the TCP/IP Offload Engine (TOE), you must have Windows Server 2008, Windows Server 2008 R2, or
Windows Server 2012. You must also have a license key installed on the motherboard (for LOMs). For add-in
NICs, the license key is preprogrammed in the hardware.
BACS is not supported on the Server Core installation option for Microsoft Windows Server 2008 R2.
Using the Installer
In addition to the Broadcom device drivers, the installer installs the management applications. The following are installed
when running the installer:
Broadcom Device Drivers. Installs the Broadcom device drivers.
Control Suite. Broadcom Advanced Control Suite (BACS).
BASP. Installs Broadcom Advanced Server Program.
SNMP. Installs the Simple Network Management Protocol subagent.
CIM Provider. Installs the Common Information Model provider.
iSCSI Crash Dump Driver. Installs the driver needed for the iSCSI Crash Dump utility.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
NOTE: Although installing the BACS software and related management applications is optional, the Broadcom device
drivers must be installed when you use the installer.
NOTE: BASP is not available on Windows Small Business Server (SBS) 2008.
To install the Broadcom NetXtreme II drivers and management applications
1. When the Found New Hardware Wizard appears, click Cancel.
2. Insert the Dell-provided CD into the CD or DVD drive or download the software driver package from the Dell website at
http://support.dell.com/.
3. On the driver source media, or from the location to which you downloaded the software driver package, open the folder
for your operating system, open the Driver_Management_Apps_Installer folder, and then double-click Setup.exe to
open the InstallShield Wizard.
4. Click Next to continue.
5. After you review the license agreement, click I accept the terms in the license agreement and then click Next to
continue.
6. Select the features you want installed.
7. Select how you want to install the NetXtreme II drivers and then click Next.
8. Click Install.
9. Click Finish to close the wizard.
10. The installer will determine if a system restart is necessary. Follow the on-screen instructions.
To install the Microsoft iSCSI Software Initiator for iSCSI Crash Dump
If supported and if you will use the Broadcom iSCSI Crash Dump utility, it is important to follow the installation sequence:
Run the installer
Install the Microsoft iSCSI Software Initiator along with the patch (MS KB939875)
NOTE: If performing an upgrade of the device drivers from the installer, re-enable iSCSI Crash Dump from the
Advanced section of the BACS Configuration tab.
Perform this procedure after running the installer to install the device drivers and the management applications.
1. Install Microsoft iSCSI Software Initiator (version 2.06 or later) if not included in your OS. To determine when you need
to install the Microsoft iSCSI Software Initiator, see Table 1. To download the iSCSI Software Initiator from Microsoft,
go to http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=18986.
2. Install Microsoft patch for iSCSI crash dump file generation (Microsoft KB939875) from
http://support.microsoft.com/kb/939875. To determine if you need to install the Microsoft patch, see Table 1.
Table 1: Windows Operating Systems and iSCSI Crash Dump
Operating System MS iSCSI Software Initiator Required Microsoft Patch (MS KB939875) Required
NDIS
Windows Server 2008 Yes (included in OS) No
Windows Server 2008 R2 Yes (included in OS) No
Windows Server 2012 Yes (included in OS) No
OIS
Windows Server 2008 No No
Windows Server 2008 R2 No No
Windows Server 2012 No No
Using Silent Installation
NOTES:
All commands are case sensitive.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
For detailed instructions and information about unattended installs, refer to the Silent.txt file in the
Driver_Management_Apps_Installer folder.
To perform a silent install from within the installer source folder
Type the following:
setup /s /v/qn
To perform a silent upgrade from within the installer source folder
Type the following:
setup /s /v/qn
To perform a silent reinstall of the same installer
Type the following:
setup /s /v"/qn REINSTALL=ALL"
NOTE: The REINSTALL switch should only be used if the same installer is already installed on the system. If upgrading an
earlier version of the installer, use setup /s /v/qn as listed above.
To perform a silent install by feature
Use the ADDSOURCE to include any of the features listed below.
Type the following according to platform:
IA32 platforms: setup /s /v"/qn ADDSOURCE=Driversi32,BACSi32,BASPi32,SNMPi32,CIMi32"
AMD/EM64T platforms: setup /s /v"/qn ADDSOURCE=Driversa64,BACSa64,BASPa64,SNMPa64,CIMa64"
The following command-line statement installs only the Broadcom drivers according to platform:
IA32 platforms: setup /s /v"/qn ADDSOURCE=Driversi32"
AMD64 platforms: setup /s /v"/qn ADDSOURCE=Driversa64"
NOTE: The Broadcom device drivers are a required feature and are always installed, even if you do not specify
ADDSOURCE.
To perform a silent install from within a batch file
To perform a silent install from within a batch file and to wait for the install to complete before continuing with the next
command line, type the following:
start /wait setup /s /w /v/qn
To perform a silent install to force a downgrade (default is NO)
setup /s /v" /qn DOWNGRADE=Y"
Modifying the Driver Software
To modify the driver software
1. In Control Panel, double-click Add or Remove Programs.
2. Click Broadcom Drivers and Management Applications, and then click Change.
3. Click Next to continue.
4. Click Modify, Add, or Remove to change program features. This option does not install drivers for new adapters. For
information on installing drivers for new adapters, see Repairing or Reinstalling the Driver Software.
5. Click Next to continue.
6. Click on an icon to change how a feature is installed.
7. Click Next.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
8. Click Install.
9. Click Finish to close the wizard.
10. The installer will determine if a system restart is necessary. Follow the on-screen instructions.
Repairing or Reinstalling the Driver Software
To repair or reinstall the driver software
1. In Control Panel, double-click Add or Remove Programs.
2. Click Broadcom Drivers and Management Applications, and then click Change.
3. Click Next to continue.
4. Click Repair or Reinstall to repair errors or install drivers for new adapters.
5. Click Next to continue.
6. Click Install.
7. Click Finish to close the wizard.
8. The installer will determine if a system restart is necessary. Follow the on-screen instructions.
Removing the Device Drivers
When removing the device drivers, any management application that is installed is also removed.
NOTE: Windows Server 2008 and Windows Server 2008 R2 provide the Device Driver Rollback feature to replace a device
driver with one that was previously installed. However, the complex software architecture of the NetXtreme II device may
present problems if the rollback feature is used on one of the individual components. Therefore, we recommend that changes
to driver versions be made only through the use of a driver installer.
To remove the device drivers
1. In Control Panel, double-click Add or Remove Programs.
2. Click Broadcom Drivers and Management Applications, and then click Remove. Follow the on-screen prompts.
3. Reboot your system to completely remove the drivers. If you fail to reboot your system, you will not be able to
successfully install the drivers.
Using the NetXtreme II Monolithic Driver
The NetXtreme II, based on its advanced functionalities, uses a software architecture that includes a Virtual Bus Device (VBD)
to extend functionalities beyond basic network connectivity. Microsoft, however, does not currently support this architecture
when loading an operating system through its Windows Deployment Services (WDS), which was previously known as Remote
Installation Services (RIS), or for the deployment agent used in the Automated Deployment Services (ADS). Therefore, a
separate driver was created to accommodate these Microsoft deficiencies. This driver is known as the NetXtreme II monolithic
driver, but it is sometimes referred to as the "RIS" driver.
The NetXtreme II monolithic driver was developed to work only for the text mode portion of a WDS legacy installation and to
establish connectivity with a deployment agent for ADS. It is not intended to be used as a driver loaded in the running state
of an operating system. The exception to this would be when used for the Windows Preinstallation Environment (WinPE).
For WDS, this driver is used similarly to any other network adapter driver for supporting network connectivity after the PXE
boot to the WDS server. When placed in the I386 or AMD64 directory (depending on the version of the operating system
being deployed), the monolithic driver is called to establish that there is driver support for the NetXtreme II adapter included
in the WDS legacy image.
For ADS, the driver is placed in the PreSystem directory on the server running ADS to establish connectivity with the
deployment agent on remote systems with NetXtreme II adapters when booting from PXE.
While Windows PE 2005 natively supports the VBD architecture, it was found that using the "minint" switch in the
startnet.cmd file does not. The minint switch performs a limited scan of the system bus to identify network devices only and,
therefore, does not support the VBD architecture. Since only network connectivity is required in Windows PE, the only
supported driver is the monolithic driver for the NetXtreme II adapter in this environment as well. Place the b06nd.inf file in
the INF directory within the Windows PE image, and place the appropriate driver file (b06nd51a.sys for x64-based builds or
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
b06nd51.sys for x86-based builds) in the driver's directory. If Windows PE is deployed as a flat image from a RIS or WDS
server, you must also place both the b06nd.inf and the appropriate driver file in the I386 or AMD64 directory containing the
image. If the RIS or WDS server is running Windows 2000 Server and deploying an x86 WinPE image, you may need to apply
the following modification to the b06nd.inf file located in the I386 directory as follows:
1. Locate [Manufacturer] header within the file.
2. Review the line below it which reads: %brcm% = broadcom, ntx86, ntamd64, ntia64 or equivalent.
3. Modify that line to read: %brcm% = broadcom.ntx86, ntamd64, ntia64. The change made replaces the comma and
space after "broadcom" with a period.
4. Save the file.
5. Restart the RIS service (binlsvc) or WDS services (wdsserver).
Inserting the NetXtreme II Monolithic Driver in a in a WinPE 2.0 or 3.1
Image
Follow these procedures for inserting the NetXtreme II monolithic driver into WinPE images. The instructions differ depending
on the WinPE version and the Windows Server OS version system being used.
WinPE 2.0
The Microsoft Windows Server 2008 method of inserting the NetXtreme II monolithic driver in a WinPe 2.0 image is different
from the Windows Server 2008 R2 method, as discussed below.
By default, the monolithic driver is not included in the boot.wim and install.wim files that come with either the Microsoft
Windows Server 2008 CD or the Windows Server 2008 R2 CD. Microsoft's Windows Automated Installation Kit (AIK) allows
you to modify the default boot.wim and install.wim files, and create WinPE 2.0 images to include the NetXtreme II monolithic
driver in the Windows Server installation.
To insert the monolithic driver into a WinPE 2.0 boot image (Windows Server 2008)
To insert Broadcom's NetXtreme II monolithic driver in a WinPE 2.0 image, download AIK from http://www.microsoft.com/en-
us/download/default.aspx and install.
After installing AIK, copy the latest monolithic driver to a directory on the local hard drive of the system you installed the AIK.
Follow the procedure below to insert the monolithic driver into a WinPE 2.0 boot image.
1. From All Programs, open Windows AIK and select Windows PE Tools Command prompt.
2. At the command prompt, run the copype.cmd script. The script requires two arguments: hardware architecture and
destination location.
copype.cmd <arch> <destination>
For example: copype x86 c:\VistaPEx86
NOTE: The directory structure c:\VistaPEx86 is used throughout this procedure.
3. Mount the base image to a local directory so that you can add or remove packages by typing:
imagex /mountrw c:\VistaPEx86\winpe.wim 1 c:\VistaPEx86\mount
4. Place the monolithic driver and inf file in c:\drivers\x32\ by typing:
peimg /inf=c:\Drivers\x32\b06nd.inf c:\VistaPEx86\mount\windows
AIK inserts the driver into the WinPE 2.0 image.
5. To complete the customization of the image, prepare the image for deployment, type:
peimg /prep c:\VistaPEx86\mount\windows
6. When asked to continue and have the program prepare the image for deployment, type:
yes
7. To commit the changes to the original image file (Winpe.wim), type:
imagex /unmount c:\VistaPEx86\mount /commit
8. To replace the default Boot.wim file in the \ISO directory with your new custom image, type:
copy c:\VistaPex86\winpe.wim c:\VistaPEx86\ISO\sources\boot.wim
To add a device driver to an offline Windows PE image (Windows Server 2008 R2)
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
This procedure demonstrates how to use the Deployment Image Servicing and Management (DISM) tool to add a device
driver (.inf) to an offline Windows PE image. Before running a DISM command, first mount the Windows PE image.
1. Mount the base image by using the DISM tool to a local Windows PE directory. For example:
Dism /Mount-WIM /WimFile:c:\winpe_x86\winpe.wim /index:1 /MountDir:c:\winpe_x86\mount
NOTE: The directory structure c:\winpe_x86 is used throughout this procedure.
2. Add the .inf file to the base image by using the dism command with the /Add-Driver option. For example Driver.inf is
the Broadcom driver, evnd.inf is the driver for the 10 Gbps devices, and b06nd.inf is the driver for the 1 Gbps devices.
Dism /image:<path_to_image> /Add-Driver /Driver:c:\winpe_x86\mount\Windows\driver.inf
3. Repeat steps 1 and 2 for each additional device driver.
4. Unmount the image after modifying it.
dism /unmount-wim /Mountdir:c:\winpe_x86\mount /commit
5. After unmounting the image, burn it to the designated media.
To create a bootable CD-ROM
1. On your technician computer, at the command prompt, create an .iso file by typing:
oscdimg -n -bc:\VistaPEx86\etfsboot.com c:\VistaPEx86\ISO C:\VistaPEx86\VistaPEx86.iso
2. Burn the iso image to a CD.
WinPE 3.1
To insert the NetXtreme II monolithic driver into a WinPE 3.1 boot image (Windows server 2008 R2 SP1)
1. Open the WinPE image and mount it:
Dism /Mount-WIM /WimFile:c:\WinPEx64\winpe.wim /index:1 /MountDir:c:\WinPEx64\mount
2. Use the following commands to insert the Broadcom NetXtreme II drivers into the WinPE 3.1 image:
eVBD driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\evbd.inf
NetXtreme I NDIS 5.1 driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\b57amd64.inf
NetXtreme II NDIS driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\bxnd.inf
NetXtreme I NDIS 6.0 driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\b57nd60a.inf
NetXtreme II VBD driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\bxvbd.inf
3. Close the WinPE image and unmount it:
Dism /unmount-wim /Mountdir:c:\WinPEx64\mount /commit
WinPE 4.0
To insert the NetXtreme II monolithic driver into a WinPE 4.0 boot image (Windows server 2008 R2 SP1)
Configuring the Speed/Duplex Setting for the NetXtreme II Monolithic
Driver
Since the typical environment where the NetXtreme II monolithic driver is used does not provide the means to configure
advanced network adapter properties, the driver file (b06nd.inf) was modified to include a section that allows it to be
configured for a specific speed and/or duplex. This provides a more robust connection to the network as it allows the adapter
to match the settings of its link partner (e.g., a switch, router, etc.).
To manually configure the speed and duplex
1. Open the b06nd.inf file with a text editor like Microsoft Notepad or WordPad.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
2. Perform a search on the file for "Registry parameters" to locate the section that will allow you to configure the adapter
speed/duplex.
3. Once located, notice the following information shown.
[params_utp]
hkr, , req_medium, 2, "0"
[params_fiber]
hkr, , req_medium, 2, "65283"
These make up two separate sections that can be configured: one for standard RJ-45 copper interfaces (params_utp) and one
for fiber devices (params_fiber).
1. As described in the file, replace the value above in quotation marks under the correct section, depending upon the
network adapter in your system. The available values are shown below.
Options for copper interfaces:
Auto (1 Gbps is enabled when that speed is supported) = "0"
10 Mbps Half Duplex = "65794"
10 Mbps Full Duplex = "258"
100 Mbps Half Duplex = "66050"
100 Mbps Full Duplex = "514"
Options for fiber interfaces:
Auto (1 Gbps is enabled when that speed is supported) = "0"
1 Gbps Full Duplex = "771"
Auto with 1 Gbps Fallback = "33539"
Hardware default = "65283"
An example is provided in the file showing how to configure a copper interface for a 10 Mbps Full Duplex connection. The
example is shown below.
hkr, , req_medium, 2, "258"
Viewing or Changing the Properties of the Adapter
To view or change the properties of the Broadcom network adapter
1. In Control Panel, click Broadcom Control Suite 4.
2. Click the Advanced section of the Configurations tab.
Setting Power Management Options
You can set power management options to allow the operating system to turn off the controller to save power or to allow the
controller to wake up the computer. If the device is busy doing something (servicing a call, for example) however, the
operating system will not shut down the device. The operating system attempts to shut down every possible device only
when the computer attempts to go into hibernation. To have the controller stay on at all times, do not click the Allow the
computer to turn off the device to save power check box.
NOTE: Power management options are not available on blade servers.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
NOTES:
The Power Management tab is available only for servers that support power management.
To enable Wake on LAN (WOL) when the computer is on standby, click Allow the device to bring the
computer out of standby box.
If you select Only allow management stations to bring the computer out of standby, the computer can
be brought out of standby only by Magic Packet.
CAUTION! Do not select Allow the computer to turn off the device to save power for any adapter
that is a member of a team.
Configuring the Communication Protocol To Use With BACS4
There are two main components of the BACS4 management application: the provider component and the client software. A
provider is installed on a server, or managed host, that contains one or more CNAs. The provider collects information on the
CNAs and makes it available for retrieval from a management PC on which the client software is installed. The client software
enables viewing information from the providers and configuring the CNAs.The BACS client software includes a graphical user
interface (GUI) and a command line interface (CLI).
A communication protocol enables communication between the provider and the client software. Depending on the mix of
operating systems (Linux, Windows, or both) on the clients and managed hosts in your network, you can choose an
appropriate communication protocol to use. See Installing the Broadcom Advanced Control Suite in the "Linux Management
Application Installation" topic for a description of the available communication protocols for each network configuration.
The instructions in this chapter address only the scenario where Windows managed hosts are communicating
with Windows clients. In these scenarios, you can use either the WMI or the WS-MAN (WinRM) communication protocols.
When you use the driver installer described in this chapter to install both the driver and the management applications, the
provider for both WMI and WS-MAN is installed on the managed host. Additionally, the BACS4 utility is installed on the client.
The following sections provide additional configuration steps for the communication protocol you select.
For Linux installations, the driver is installed separately from the management applications. See Linux Driver Software and
see Linux Management Application Installation for related instructions.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
Using WS-MAN
To use the WS-MAN communication protocol, follow the instructions in the following sections:
WS-MAN Windows Server Configuration
WS-MAN Windows Client Installation
WS-MAN Windows Server Configuration
Step 1: Install the WinRM Software Component on Server
On the following operating systems, WinRM 2.0 is preinstalled:
Windows 7
Windows 8
Windows 8.1
Windows Server 2008 R2
Windows Server 2012
Windows 2012 R2
For Windows XP and Windows Server, 2008, install Windows Management Framework Core, which includes WinRM 2.0 and
Windows Powershell 2.0, from the following link:
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=11829
Step 2: Perform Basic Configuration on the Server
The Windows firewall must be enabled for WinRM to work properly. For detailed information about firewall configuration, see
Step 7: Additional Server Configuration. After the firewall is configured, open a command prompt and run the following
command to enable the remote management on the Windows server:
winrm quickconfig
You can use the following command to view the configuration information for the service:
winrm get winrm/config
Step 3: Perform User Configuration on the Server
To connect to WinRM, the account must be a member of the local administrators group on the local or remote computer. The
output of the get winrm/config command will be as follows:
RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)
BA stands for BUILTIN\Administrators.
To add another user group to the WinRM allowed connect list, you can modify the RootSDDL to include the new user group.
You will need the SSDL ID for the new group. For example, the following command adds the new user group with SDDL ID S1-5-21-1866529496-2433358402-1775838904-1021.
winrm set winrm/config/Service @{RootSDDL="O:NSG:BAD:P(A;GA;;;BA)(A;;GA;;;
S-1-5-21-1866529496-2433358402-1775838904-1021)S:P(AU;FA;GA;;
WD)(AU;SA;GWGX;;;WD)"}
Step 4: Perform HTTP Configuration on the Server
To use the BACS GUI, you must configure the HTTP protocol, as follows:
NOTE: The default HTTP port is 5985 for WinRM 2.0.
1. Click Start (or press the Windows logo key) and select Run.
2. Enter gpedit.msc to open the local Group Policy editor.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
3. Under Computer Configuration, open the Administrative Templates folder and then open the Windows
Components folder.
4. Select Windows Remote Management (WinRM).
5. Under Windows Remote Management (WinRM), select WinRm Client.
6. Under WinRM Client, double-click Trusted Hosts.
7. In the TrustedHostsList, enter the host names of the clients. If all clients are trusted then enter an asterisk (*) only.
8. Select WinRM Service.
9. Enable Allow Basic Authentication.
10. Enable Allow unencrypted traffic.
11. Close the Group Policy wIndow.
12. From the command prompt, run the following command to configure WinRM with default settings:
winrm qc or winrm quickconfig
13. When the tool displays "Make these changes[y/n]?", enter "y".
14. Enter one of the following commands to check whether an HTTP listener is created:
winrm enumerate winrm/confg/listener
or
winrm e winrm/config/Listener
15. Enter the following command from the command prompt to test locally.
winrm id
Step 5: Perform HTTPS Configuration on the Server (to use HTTPS rather than HTTP)
This step consists of two distinct processes: generating a self-signed certificate, if certificate does not exist, and importing it
to a Windows server. If one does not already exist, you must configure a self-signed certificate on the Windows server to
enable HTTPS/SSL communication with the BACS GUI on the Windows client. The Windows client also must be configured with
the self-signed certificate. See Perform HTTPS Configuration (if you plan to use HTTPS).
NOTE: The self-signed certificate can be created on any Windows server. The server does not require BACS to be installed.
The self-signed certificate generated on any Windows server should be copied on the local drive of client.
1. Click Start (or press the Windows logo key) and select Run.
2. Enter gpedit.msc to open the local Group Policy editor.
3. Under Computer Configuration, open the Administrative Templates folder and then open the Windows
Components folder.
4. Select Windows Remote Management (WinRM).
5. Under Windows Remote Management (WinRM), select WinRm Client.
6. Under WinRM Client, double-click Trusted Hosts.
7. In the TrustedHostsList, enter the host names of the clients. If all clients are trusted then enter an asterisk (*) only.
8. Select WinRM Service.
9. Enable Allow Basic Authentication.
To generate a self-signed certificate for the Windows Server:
Openssl on Windows can be used to generate the self-signed certificate, as follows:
NOTE: You can download and install openssl from http://gnuwin32.sourceforge.net/packages/openssl.htm.
1. Enter the following command to generate a private key:
openssl genrsa -des3 -out server.key 1024
2. You are prompted to enter a passphrase. Be sure to remember the passphrase.
3. Use the following steps to generate a Certificate Signing Request (CSR).
During the generation of the CSR, you are prompted for several pieces of information. When prompted for the
"Common Name", enter the Windows Server host name or IP address.
Enter the following command (sample responses are shown):
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
openssl req -new -key server.key -out server.csr
If this command does not work, try the following:
openssl req –new –key server.key –out server.csr –config openssl.cnf
The openssl.cnf file should be placed in the same directory where openssl is placed. Openssl.cnf is located in the
folder C:\Program Files (x86)\GnuWin32\share.
The following information is requested:
Country Name (2 letter code) []:US
State or Province Name (full name) []: California
Locality Name (e.g., city) []: Irvine
Organization Name (e.g., company) []: Broadcom Corporation
Organizational Unit Name (e.g., section) []: Engineering
Common Name (e.g., YOUR name) []: Enter the host name or IP address of the Windows server. For iPv6, enter
the Common Name in the format [xyxy:xxx:....::xxx], including the brackets [ ].
(Optional) Email Address []:
Enter the following additional attributes to be sent with your certificate request:
A challenge password []:password1
An optional company name []:
4. Remove the passphrase from the key.
Enter the following commands:
cp server.key server.key.org
openssl rsa -in server.key.org -out server.key
5. Generate a self-signed certificate:
To generate a self-signed certificate which is active for 365 days, enter the following command:
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
The following output displays:
Signature ok
subject=/C=US/ST=California/L=Irvine/O=Broadcom Corporation/OU=Engineering/CN=MGMTAPP-
LAB3/emailAddress=
Getting Private key
6. Enter the following command to verify the generated self-signed certificate.
openssl verify server.crt
The following output displays:
server.crt:/C=US/ST=California/L=Irvine/O=Broadcom Corporation/OU=Engineering/CN=MGMTAPP-
LAB3/emailAddress=
error 18 at 0 depth lookup:self signed certificate
OK
Ignore the error message "error 18 at 0 depth lookup:self signed certificate". This error indicates that this is a selfsigned certificate.
7. Convert the certificate from "crt" to "pkcs12" format, as follows:
For a Windows server, the certificate should be in pkcs12 format. Enter the following command:
openssl pkcs12 -export -in server.crt -inkey server.key -out hostname.pfx
You will be prompted for the following:
Enter Export Password:
Verifying - Enter Export Password:
Enter the password and be sure to remember it. The password is required when importing the certificate on the
Windows server and client.
8. Make a copy of the certificate file server.crt and place it on the server where BACS will be installed, so that it can be
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
imported. If you plan to use a Windows client to connect to the server running BACS, then the certificate also needs to
be transferred (copied and pasted) to the client system.
To instal the self-signed certificate on Windows server:
Transfer the file hostname.pfx you generated on the Windows server before you install the certificate:
1. Click Start (or press the Windows logo key) and select Run.
2. Enter MMC and click OK.
3. Click File > Add/Remove Snap-in.
4. Click Add.
5. Select Certificates and click Add.
6. Select Computer account.
7. Click Next and then click Finish.
8. Click Close, then click OK.
9. Open the Certificates (Local Computer) folder and then open the Personal folder.
10. Right-click Certificates, select All Tasks and then click Import.
11. Click Next to begin the Certificate Import Wizard.
12. Browse to select hostname.pfx.
13. When you are prompted for the password for the private key, enter the same password you created in To generate a
self-signed certificate for the Windows Server:.
14. Follow the instructions, select the defaults, and continue.
The certificate is shown as installed on the right side of the window. The name will be the name you specified while
creating a self-signed certificate.
15. Right-click on the certificate and select Properties.
A dialog box displays, as follows:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
16. Ensure that only Server Authentication is enabled, as shown in the figure.
17. Open Trusted Root Certification Authorities and then open Certificates.
18. Follow the instructions from Step 11. to Step 17.
NOTE: See Perform HTTPS Configuration (if you plan to use HTTPS) for instructions on importing the self-signed
certificate on a client.
Step 6: Configure WinRM HTTPS/SSL on the Server
1. Create WinRM Listener, as follows:
a. Click Start (or press the Windows logo key) and select Run.
b. Enter MMC and click OK.
c. Select the self-signed certificate from the Personal store.
For example, if the certificate is created with a host name, the host name will appear.
d. Double-click the certificate to open it.
e. Click the Details tab.
f. Scroll down and select the Thumbprint field.
g. Select and copy the thumbprint in the Details window so you can insert it in the next step.
h. Return to the command prompt.
i. Enter the following command:
winrm create winrm/config/Listener?Address=*+Transport=
HTTPS @{Hostname="<HostName or IPAddress>";
CertificateThumbprint="<paste from the previous step and remove the spaces>"}
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
NOTES:
If the certificate was generated using the host name, enter the host name. If it was generated
using the IP address, enter the IP address. For an IPv6 address, use brackets [ ] around the
address.
If HTTPS is configured in your system, the listener must be deleted before creating a new HTTPS
listener. Use the following command:
winrm delete winrm/config/Listener?Address=*+Transport=HTTPS
j. The above command creates a listener on the HTTPS port (5986) using any/all network address of the server,
and my SelfSSL generated certificate.
k. You can use the winrm command to modify or set the HTTPS listener, as WinRM listeners can be configured on
any user defined port.
l. From command prompt, run the following command to verify that the listener(s) that have been configured:
winrm e winrm/config/listener
2. Test HTTPS/SSL connection on the server.
a. At the command prompt on the server, enter the following command:
winrs -r:https://yourserver:5986 -u:username -p:password hostname
b. If setup correctly, the output of the command shows the server host name.
c. To check WinRM Service Configuration, run the following command:
winrm get winrm/config/service
Step 7: Additional Server Configuration
If necessary, modify the firewall rules as follows:
Windows Server 2008 R2
1. From the Administrative Tools menu, open Windows Firewall with Advanced Security.
2. Right-click Inbound Rules and select New Rule.
The new rule wizard opens.
3. Select Port and click Next.
4. On the Protocol and Ports screen, select TCP and enter the specific port, for example, 5985 for HTTP or 5986 for
HTTPS.
5. Click Next.
6. On the Action screen, select Allow the connection and click Next.
7. For Profile, you can select all three profiles if your server is in a workgroup.
8. Specify a name for the rule and click Finish.
9. Ensure that the new rule and is enabled (the green check box is selected).
Windows XP
1. Click Start > Control Panel, and then double-click Windows Firewall.
2. Click the Exceptions tab
3. Click Add Port.
4. Enter a meaningful Name, for example "WinRM rule" and port number, for example, 5985 for HTTP or 5986 for HTTPS.
5. Click OK.
Useful WinRM Commands
Command Description
winrm quickconfig or winrm qc
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Configures WinRM with default settings
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
winrm enumerate winrm/config/Listener or winrm e
winrm/config/Listener
winrm get winrm/config/Service
winrm delete winrm/config/Listener?
Address=*+Transport=HTTPS
Helps to check which service listener are enabled and listening on
which port and IP Address.
Checks WinRM Service Configuration.
Deletes a Listener (in this case deleting a HTTPS listener).
Useful WinRM Websites
http://msdn.microsoft.com/en-us/library/aa384372%28v=vs.85%29.aspx
http://technet.microsoft.com/en-us/library/cc782312%28WS.10%29.aspx
http://msdn.microsoft.com/en-us/library/aa384295%28v=VS.85%29.aspx
The following articles on "http://support.microsoft.com:
"Configuring WINRM for HTTPS"
"Windows Management Framework (Windows PowerShell 2.0, WinRM 2.0, and BITS 4.0)"
WS-MAN Windows Client Installation
On the Windows client, perform following configuration steps.
1. Perform HTTP Configuration (if you plan to use HTTP)
a. Click Start (or press the Windows logo key) and select Run.
b. Enter gpedit.msc to open the local Group Policy editor.
c. Under Computer Configuration, open the Administrative Templates folder and then open the Windows
Components folder.
d. Select Windows Remote Management (WinRM).
e. Under Windows Remote Management (WinRM), select WinRm Client.
f. Under WinRM Client, double-click Trusted Hosts.
g. In the TrustedHostsList, enter the host names of the clients and click OK. If all clients are trusted then enter
an asterisk (*) only.
h. Select WinRM Service.
i. Enable Allow Basic Authentication and click OK.
j. Run the following command from the command prompt to test the connection:
winrm id -remote:<remote machine Hostname or IP Address>
2. Perform HTTPS Configuration (if you plan to use HTTPS)
After you generate a self-signed certificate, as described in To generate a self-signed certificate for the Windows
Server:, you can import the certificate on the client to facilitate a connection between server and client. Ensure that
all steps mentioned in section To generate a self-signed certificate for the Windows Server: are completed, including
copying hostname.pfx at the location from where client can access it, before you proceed with the following steps.
a. Click Start (or press the Windows logo key) and select Run.
b. Enter MMC and click OK.
c. Click File and select Add/Remove Snap-in.
d. Click Add.
e. Select Certificates and click Add.
f. Select Computer account and click Next.
g. Click Finish.
h. Click Close and then click OK.
i. Under Certificates (Local Computer), right-click on Trusted Root Certification Authorities, select All
Tasks, and select Import.
j. Click Next to begin the Certificate Import Wizard.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
k. Browse to select the .pfx file you generated in To generate a self-signed certificate for the Windows Server:.
Change the selection in the Files of type list to Personal Information Exchange (*.pfxas, *.p12), select the
hostname.pfx file and click Open.
l. Enter the password you assigned to the private key and click Next.
3. Configure WinRM HTTPS/SSL
You can run winrm from a client to retrieve information from the server using WinRM HTTPS connection. Use the
following steps to test the WinRM HTTPS/SSL connection from client:
a. To retrieve the server operating system information, enter the following command.
winrm e wmi/root/cimv2/Win32_OperatingSystem -r:https://yourservername u:username
-p:password -skipCAcheck
b. To retrieve the server WinRM identity information, enter the following command.
winrm id -r:https://yourservername -u:username -p:password -skipCAcheck
c. To enumerate Windows services on the server, enter the following command.
winrm e wmicimv2/Win32_service -r:https://yourservername -u:username p:password skipCAcheck
NOTE: It is important to use -skipCAcheck switch in the winrm command line testing, as the certificate is
self-generated and not imported on the client. Otherwise, the following error message displays: WSManFault.
Using WMI
No special configuration is required to use WMI on the Windows client. Perform the steps in the following sections to configure
WMI on the Windows server.
Step 1: Set up Namespace Security Using WMI Control
The WMI Control provides one way to manage namespace security. You can start the WMI Control from the command prompt
using this command:
wmimgmt
On Windows 9x or Windows NT4 computers that have WMI installed, use this command instead:
wbemcntl.exe
Alternatively, you can access the WMI Control and the Security tab as follows:
1. Right-click on My Computer and click Manage.
2. Double-click Services and Applications and then double-click WMI Control.
3. Right-click WMI Control and then click Properties.
4. In WMI Control Properties, click the Security tab.
5. A folder named Root with a plus sign (+) next to it should now be visible. Expand this tree as necessary to locate the
namespace for which you want to set permissions.
6. Click Security.
A list of users and their permissions appears. If the user is on the list, modify the permissions as appropriate. If the
user is not on the list, click Add and add the user from the location (local machine, domain, etc.) where the account
resides.
NOTES: You can add these exports at the end of the .bash_profile. This file is located in the /root directory.
In order to view and set namespace security, the user must have Read Security and Edit Security permissions.
Administrators have these permissions by default, and can assign the permissions to other user accounts as
required.
If this user needs to access the namespace remotely, you must select the Remote Enable permission.
By default, user permissions set on a namespace apply only to that namespace. If you want the user to have
access to a namespace and all subnamespaces in the tree below it, or in subnamespaces only, click Advanced.
Click Edit and specify the scope of access in the dialog box that displays.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
Step 2: Grant DCOM Remote Launch and Activate Permission
In the Windows domain environment, the Domain Administrator account has the necessary privilege level to access the WMI
component for BACS management and, therefore, no special configuration is needed. In a large enterprise, however, a user
who is accessing the local or remote host using the BACS4 client GUI may not always have the domain administrator account
privilege. It is necessary to configure WMI security access on the remote host to allow the user to connect to it using the
BACS4 client GUI.
This configuration can be easily done using the following procedure. If you do not have sufficient privileges to configure
security for WMI access, contact your Network Administrator.
1. Click Start (or press the Windows logo key) and select Run.
2. Enter DCOMCNFG, and then click OK.
3. The Component Services dialogue box displays.
4. Open Component Services and then open Computers.
5. Right-click My Computer and click Properties.
6. In My Computer Properties, click the COM Security tab.
7. Under Launch and Activation Permissions, click Edit Limits.
8. Follow these steps if your name or your group does not appear in the Groups or user names list.
a. In the Launch Permission dialog box, click Add.
b. In the Select Users, Computers, or Groups dialog box, add your name and the group in the Enter the object
names to select box, and then click OK.
c. In the Launch Permission dialog box, select your user and group in the Group or user names list.
d. In the Permissions for User area, select Allow for Remote Launch and Remote Activation, and then click
OK.
Figure 1: Launch and Activation Permission
For more information, see Securing a Remote WMI Connection on the Microsoft Developer Network site.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Windows Driver and Management Application Installation: Broadcom NetXtreme II® Network Adapter User Guide
Special Configuration for WMI on Different Systems
On a Windows XP Pro computer, ensure that remote logons are not being coerced to the GUEST account (referred to as
"ForceGuest", which is enabled by default on computers that are not attached to a domain). Open the Local Security
Policy editor by clicking Start > Run and entering secpol.msc. Open the Local Policies node and select Security
Options. Then, scroll down to the setting titled Network access: Sharing and security model for local accounts.
If this is set to Guest only, change it to Classic and restart the computer.
In Windows Vista and Windows 7, in order to let all users in the administrator group connect using the WMI
namespace, the user might need to change the LocalAccountTokenFilterPolicy as needed.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/umwindrv.htm[9/5/2014 3:45:04 PM]
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
Back to Contents Page
Linux BACS Installation: Broadcom NetXtreme II® Network
Adapter User Guide
Overview
Installing WS-MAN or CIM-XML on Linux Server
Installing WS-MAN or CIM-XML on Linux Client
Installing the Broadcom Advanced Control Suite
Overview
The Broadcom Advanced Control Suite version 4 (BACS4) is a management application for configuring the NetXtreme II family
of adapters, also known as Converge Network Adapters (CNAs). BACS4 software operates on Windows and Linux server and
client operating systems. This chapter describes how to install the BACS4 management application.
This chapter describes how to install the BACS4 management application on Linux systems. For Windows systems,
an installation program is provided which installs both the Windows drivers and the management applications, including
BACS4 (see Windows Driver and Management Application Installation for instructions).
There are two main components of the BACS4 utility: the provider component and the client software. A provider is installed
on a server, or "managed host", that contains one or more CNAs. The provider collects information on the CNAs and makes it
available for retrieval from a management PC on which the client software is installed. The client software enables viewing
information from the providers and configuring the CNAs.The BACS client software includes a graphical user interface (GUI)
and a command line interface (CLI).
Communication Protocols
A communication protocol enables exchanging information between provider and the client software. These are proprietary or
open-source implementations of the Web-Based Enterprise Management (WBEM) and Common Information Model (CIM)
standards from the Distributed Management Task Force (DMTF). Network administrators can choose the best option based on
the prevailing standard on their network.
The following table shows the available options based on the operating systems installed on the managed host and the client.
If the client uses: And the managed host uses: BACS can use these communication protocols:
Windows Windows
Windows Linux
Linux Windows WS-MAN (WinRM)
Linux Linux
WMI
WS-MAN (WinRM)
CIM-XML (OpenPegasus)
WS-MAN (OpenPegasus)
CIM-XML (OpenPegasus)
WS-MAN (OpenPegasus)
WMI = Windows Management Instrumentation.
WS-MAN = Web Service-Management. WinRM is a Windows-based implementation and OpenPegasus is an open-
source implementation of the that operates on Linux.
CIM-XML = An XML-based version of OpenPegasus.
If your network includes a mix of Windows and Linux clients accessing Windows and Linux servers, then WS-MAN is a suitable
choice. If Linux is the only OS installed on the servers, then CIM-XML is an option. If the network includes only Windows
servers and clients, WMI is an option. WMI is very simple to configure but is supported only on the Windows OS. (See
Windows Driver and Management Application Installation for instructions on installing and configuring the Windows protocols.)
BACS installation includes installing the provider component on the managed host and the client software on the management
station. The installation process differs based on the combination of operating systems installed on the client and managed
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
host and on the selected communication protocol.
Installing WS-MAN or CIM-XML on Linux Server
Step 1: Install OpenPegasus
On the Red Hat Linux OS, two installation options are available:
From the Inbox RPM (Red Hat Only)
From Source (Red Hat and SuSE)
On the SUSE Linux Enterprise Server 11 (SLES11) OS, you must use the source RPM.
NOTE: The Inbox RPM does not support the WS-MAN communication protocol. To use WS-MAN, you must install
OpenPegasus from source.
From the Inbox RPM (Red Hat Only)
In Red Hat Linux, an Inbox OpenPegasus RPM is available as tog-pegasus-<version>.<arch>.rpm.
1. Use the following command to install tog-pegasus:
rpm -ivh tog-openpegasus-<version>.<arch>.rpm
2. Use the following command to start Pegasus:
/etc/init.d/tog-pegasus start
NOTE: If your system has "Red Hat Security Enhancement for tog-pegasus" enabled, disable it before connecting to
BACS. See /usr/share/doc/tog-pegasus-2.5.2/README.RedHat.Security for details. To disable it, remove the line from
/etc/pam.d/wbem.
NOTE: On SuSE Linux, the Inbox OpenPegasus RPM is not available. OpenPegasus must be installed form source, as
described in the following section.
Note that in inbox Pegasus, HTTP is not enabled by default. After Inbox OpenPegasus is installed successfully, if no further
configuration is required, then follow the instructions in Step 4: Install Broadcom CMPI Provider. To enable HTTP, see Enable
HTTP.
NOTE: After the server is rebooted, the CIM server must be manually restarted to enable the client to reconnect to the
server. This is a known limitation of Red Hat v6.2 Inbox RPM.
From Source (Red Hat and SuSE)
The OpenPegasus source can be downloaded from www.openpegasus.org.
NOTES:
If not already installed, download and install the openssl and libopenssl-devel rpm. This step is optional and
required only if you are planning to use HTTPS to connect the client to the managed host.
In some cases, if OpenPegasus installation fails, openssl must be installed with the -fPIC option:
./config no-threads --fPIC. This installs openssl and the include files to /usr/local/ssl. Set the
OPENSSL_HOME path to /usr/local/ssl and continue with the OpenPegasus installation.
Set the Environment Variables
Set the environment variables for building OpenPegasus as follows.
Environment Variable Description
PEGASUS_ROOT The location of the Pegasus source tree
The location for the built executable, repository; e.g.,
PEGASUS_HOME
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
$PEGASUS_HOME/bin, PEGASUS_HOME/lib, $PEGAUS_HOME/repository,
and $PEGASUS_HOME/mof subdirectories.
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
PATH $PATH:$PEGASUS_HOME/bin
CIM server will not load the repository properly. Consider setting PEGASUS_HOME in the ".bash_profile" file.
PEGASUS_ENABLE_CMPI_PROVIDER_MANAGER True
PEGASUS_CIM_SCHEMA "CIM222"
PEGASUS_PLATFORM
PEGASUS_HAS_SSL Optional. Set to "true" for HTTPS support.
PEGASUS_ENABLE_PROTOCOL_WSMAN Optional. Set to "true" for WSMAN protocol support.
Additional Settings
The $PEGASUS_HOME variable must be set up in the shell environment, and $PEGASUS_HOME/bin needs to be appended to
the $PATH environment.
Examples
export PEGASUS_PLATFORM="LINUX_X86_64_GNU"
export PEGASUS_CIM_SCHEMA="CIM222"
export PEGASUS_ENABLE_CMPI_PROVIDER_MANAGER=true
export PEGASUS_ROOT="/share/pegasus-2.10-src"
export PEGASUS_HOME="/pegasus"
export PATH=$PATH:$PEGASUS_HOME/bin
For Linux 32 bit systems: "LINUX_IX86_GNU"
For Linux 64 bit systems: "LINUX_X86_64_GNU"
For SSL Support, add the following environment variable:
export PEGASUS_HAS_SSL=true
For WS-MAN Support, add the following environment variable:
export PEGASUS_ENABLE_PROTOCOL_WSMAN=true
CIM-XML and WSMAN in OpenPegasus use the same ports for HTTP or HTTPS. The default port numbers for HTTP and HTTPS
are 5989 and 5989, respectively.
NOTE: You can add these exports at the end of the .bash_profile. This file is located in the /root directory.
The environment variables will be set when a user logs in using PuTTY.
On the Linux system itself, for each terminal where the environment variables are not set, run the following
command:
source /root/.bash_profile
When you logout and login, the environment variables will be set.
Build and install OpenPegasus
From $PEGASUS_ROOT (the location of the Pegasus source root directory), run the following:
make clean
make
make repository
NOTE: Whenever OpenPegasus is built from source, all configurations are reset to the default values. If you are rebuilding
OpenPegasus, you must redo the configuration as described in Step 3: Configure OpenPegasus on the Server.
Step 2: Start CIM Server on the Server
Use the cimserver command to start CIM server. To stop CIM server, use the command cimserver -s.
To check whether OpenPegasus has been installed properly, enter the following command:
cimcli ei -n root/PG_Interop PG_ProviderModule
NOTE: For OpenPegasus compiled from source, PEGASUS_HOME must be defined when you start CIM server. Otherwise,
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
Step 3: Configure OpenPegasus on the Server
Use the cimconfig command to configure OpenPegasus, as shown in the following table:
Command Description
cimconfig -l
cimconfig -l -c
cimconfig -g <property name>
cimconfig -s <property name>=<value> -p
cimconfig --help
CIM server must be started before running cimconfig, and must be restarted for configuration changes to take effect.
Enable Authentication
The following OpenPegasus properties have to be set as described in this section. Otherwise, the Broadcom CIM Provider will
not work properly. Ensure the following are set before launching BACS and connecting to the provider.
Start CIM server if it is not already started. Then, set the following:
List all valid property names.
List all valid property names and its value
Query a particular property.
Set a particular property.
Find out more about the command.
cimconfig -s enableAuthentication=true -p
cimconfig -s enableNamespaceAuthorization=false -p
cimconfig -s httpAuthType=Basic -p
cimconfig -s passwordFilePath=cimserver.passwd -p
cimconfig -s forceProviderProcesses=false -p
If you want root user to connect remotely:
cimconfig -s enableRemotePrivilegedUserAccess=true -p
User configuration with privilege: The Linux system users are used for OpenPegasus authentication. The systems users have
to be added to OpenPegasus using cimuser to connect via BACS:
cimuser -a -u <username> -w <password>
Example: cimuser -a -u root -w linux1
Enable HTTP
1. If CIM server is not started, start it.
2. Use the following command to set up an HTTP port (optional):
cimconfig -s httpPort=5988 -p
This property is not available for Inbox OpenPegasus.
3. Use the following command to enable HTTP connection:
cimconfig -s enableHttpConnection=true -p
4. Use the cimserver -s and cimserver commands, respectively, to stop and restart CIM server for the new
configuration to take effect.
Enable HTTPS
1. If CIM server is not started, start it.
2. Set up HTTPS port with the following command (optional):
cimconfig -s httpsPort=5989 -p
This property is not available for inbox OpenPegasus.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
1. Enable HTTPS connection with 'the following command:
cimconfig -s enableHttpsConnection=true -p
2. Use the cimserver -s and cimserver commands, respectively, to stop and restart CIM server for the new
configuration to take effect.
Step 4: Install Broadcom CMPI Provider
Ensure that OpenPegasus is installed properly before installing CMPI Provider.
Install
Enter following command to install Broadcom CMPI Provider.
% rpm -i BRCM_CMPIProvider-{version}.{arch}.rpm
Uninstall
Enter following command to uninstall Broadcom CMPI Provider:
% rpm -e BRCM_CMPIProvider
Step 5: Perform Linux Firewall Configuration, If Required
Follow these procedures to open the appropriate ports in the firewall:
RedHat
1. Click System, select Administration, and then select Firewall.
2. Select Other Ports.
3. In the Port and Protocol Dialog box, select User Defined.
4. In the Port/Port Range field, add the port number.
5. In the Protocol field, add the protocol as TCP or UDP, etc.
6. Click Apply for the firewall rules to take effect.
Example:
For CIM-XML over HTTP, the port number is 5988 and protocol is TCP.
For CIM-XML over HTTPS, the port number is 5989 and protocol is TCP.
SuSE
1. Click Compute and then click YaST.
2. Select Security & Users on the left pane.
3. On the right pane, double-click Firewall.
4. Select Custom Rules on the left pane.
5. On the right pane click Add.
6. Enter the following values:
Source Network: 0/0 (means all)
Protocol: TCP (or the appropriate protocol)
Destination Port: <Port Number> or <Range of Port Numbers>
Source Port: Leave blank.
7. Click Next and then click Finish for the firewall rules to take effect.
Example:
For CIM-XML, use the following values:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
Source Network: 0/0 (means all)
Protocol: TCP
Destination Port: 5988:5989
Source Port: Leave blank.
Step 6: Install BACS and Related Management Applications
See Installing the Broadcom Advanced Control Suite.
Installing WS-MAN or CIM-XML on Linux Client
No special software components are required on the Linux client system to use the HTTP except installing the BACS
management application. However, for WS-MAN installations, you can optionally configure the HTTPS protocol for use with
BACS.
Configure HTTPS on Linux Client
Import Self-Signed Certificate on Linux Client
On Linux distributions, make note of the following certificate directory:
For all SuSE versions, the certificate directory is /etc/ssl/certs.
For RedHat, the certificate directory can be different for each version. For some versions, it is /etc/ssl/certs or
/etc/pki/tls/certs. For other versions, find out the certificate directory.
Copy the self-signed certificate filehostname.pem into the certificate directory of the Linux client. For example, if the
certificate directory is /etc/ssl/certs, copy hostname.pem to /etc/ssl/certs.
1. Change directory to /etc/ssl/certs.
2. Create a hash value by running the following command.
openssl x509 -noout -hash -in hostname.pem
A value such as the following will be returned.
100940db
3. Create a symbolic link to the hash value by running the following command:
ln -s hostname.pem 100940db.0
Test HTTPS/SSL Connection from Linux Client
Use the following command to test whether the certificate is installed correctly on Linux:
# curl -v --capath /etc/ssl/certs https://Hostname or IPAddress:5986/wsman
If this fails, then the certificate is not installed correctly and an error message displays, indicating to take corrective action.
Installing the Broadcom Advanced Control Suite
The Broadcom Advanced Control Suite (BACS) software can be installed on a Linux system using the Linux RPM package. This
installation includes a BACS GUI and a CLI client.
Before you begin:
Ensure that the Broadcom network adapter(s) is physically installed and the appropriate device driver for the NIC is
installed on the system to be managed by this utility.
Ensure that the CIM provider is installed properly on the system that is to be managed by this utility.
For managing iSCSI on Linux hosts, ensure that the open-iscsi and sg utilities are installed on the Linux host.
To install BACS
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
Linux BACS Installation: Broadcom NetXtreme II® Network Adapter User Guide
1. Download the latest BACS management application RPM package.
2. Install the RPM package using the following command:
% rpm -i BACS-{version}.{arch}.rpm
To Use BACS
To use the GUI, on XWindow, double-click the BACS4 desktop icon, or access the BACS program from the task bar
under System Tools.
To use BACS CLI, refer to the file BACSCLI_Readme.txt provided with the release files.
To remove BACS
To uninstall the RPM package, use the following command:
% rpm -e BACS
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/lnx_bacs.htm[9/5/2014 3:45:05 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Target LUN
Back to Contents Page
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User
Guide
iSCSI Boot
iSCSI Crash Dump
iSCSI Offload in Windows Server
iSCSI Boot
Broadcom NetXtreme II Gigabit Ethernet adapters support iSCSI boot to enable network boot of operating systems to diskless
systems. iSCSI boot allows a Windows, Linux, or VMware operating system boot from an iSCSI target machine located
remotely over a standard IP network.
For both Windows and Linux operating systems, iSCSI boot can be configured to boot with two distinctive paths: non-offload
(also known as Microsoft/Open-iSCSI initiator) and offload (Broadcom's offload iSCSI driver or HBA). Configuration of the
path is set with the HBA Boot Mode option located on the General Parameters screen of the iSCSI Configuration utility.
See Table 1 for more information on all General Parameters screen configuration options.
Supported Operating Systems for iSCSI Boot
The Broadcom NetXtreme II Gigabit Ethernet adapters support iSCSI boot on the following operating systems:
Windows Server 2008 and later 32-bit and 64-bit (supports offload and non-offload paths)
Linux RHEL 5.5 and later, SLES 11.1 and later (supports offload and non-offload paths)
SLES 10.x and SLES 11 (only supports non-offload path)
iSCSI Boot Setup
The iSCSI boot setup consists of:
Configuring the iSCSI Target
Configuring iSCSI Boot Parameters
Preparing the iSCSI Boot Image
Booting
Configuring the iSCSI Target
Configuring the iSCSI target varies by target vendors. For information on configuring the iSCSI target, refer to the
documentation provided by the vendor. The general steps include:
1. Create an iSCSI target.
2. Create a virtual disk.
3. Map the virtual disk to the iSCSI target created in step 1.
4. Associate an iSCSI initiator with the iSCSI target.
5. Record the iSCSI target name, TCP port number, iSCSI Logical Unit Number (LUN), initiator Internet Qualified Name
(IQN), and CHAP authentication details.
6. After configuring the iSCSI target, obtain the following:
Target IQN
Target IP address
Target TCP port number
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Initiator IQN
CHAP ID and secret
Configuring iSCSI Boot Parameters
Configure the Broadcom iSCSI boot software for either static or dynamic configuration. Refer to Table 1 for configuration
options available from the General Parameters screen.
Table 1 lists parameters for both IPv4 and IPv6. Parameters specific to either IPv4 or IPv6 are noted.
NOTE: Availability of IPv6 iSCSI boot is platform/device dependent.
Table 1: Configuration Options
Option Description
TCP/IP
parameters via
DHCP
IP
Autoconfiguration
iSCSI
parameters via
DHCP
CHAP
Authentication
DHCP Vendor ID
Link Up Delay
Time
Use TCP
Timestamp
Target as First
HDD
LUN Busy Retry
Count
IP Version
HBA Boot Mode
This option is specific to IPv4. Controls whether the iSCSI boot host software acquires the IP address
information using DHCP (Enabled) or use a static IP configuration (Disabled).
This option is specific to IPv6. Controls whether the iSCSI boot host software will configure a stateless
link-local address and/or stateful address if DHCPv6 is present and used (Enabled). Router Solicit packets
are sent out up to three times with 4 second intervals in between each retry. Or use a static IP
configuration (Disabled).
Controls whether the iSCSI boot host software acquires its iSCSI target parameters using DHCP (Enabled)
or through a static configuration (Disabled). The static information is entered through the iSCSI Initiator
Parameters Configuration screen.
Controls whether the iSCSI boot host software uses CHAP authentication when connecting to the iSCSI
target. If CHAP Authentication is enabled, the CHAP ID and CHAP Secret are entered through the iSCSI
Initiator Parameters Configuration screen.
Controls how the iSCSI boot host software interprets the Vendor Class ID field used during DHCP. If the
Vendor Class ID field in the DHCP Offer packet matches the value in the field, the iSCSI boot host
software looks into the DHCP Option 43 fields for the required iSCSI boot extensions. If DHCP is disabled,
this value does not need to be set.
Controls how long the iSCSI boot host software waits, in seconds, after an Ethernet link is established
before sending any data over the network. The valid values are 0 to 255. As an example, a user may
need to set a value for this option if a network protocol, such as Spanning Tree, is enabled on the switch
interface to the client system.
Controls if the TCP Timestamp option is enabled or disabled.
Allows specifying that the iSCSI target drive will appear as the first hard drive in the system.
Controls the number of connection retries the iSCSI Boot initiator will attempt if the iSCSI target LUN is
busy.
This option specific to IPv6. Toggles between the IPv4 or IPv6 protocol. All IP settings will be lost when
switching from one protocol version to another.
Set to disable when the host OS is configured for software initiator mode and to enable for HBA mode.
This option is available on NetXtreme II adapters. (Note: This parameter cannot be changed when the
adapter is in Multi-Function mode.)
MBA Boot Protocol Configuration
To configure the boot protocol
1. Restart your system.
2. From the PXE banner, select CTRL+S. The MBA Configuration Menu appears (see Broadcom Boot Agent).
3. From the MBA Configuration Menu, use the UP ARROW or DOWN ARROW to move to the Boot Protocol option. Use
the LEFT ARROW or RIGHT ARROW to change the Boot Protocol option to iSCSI.
NOTE: For iSCSI boot-capable LOMs, the boot protocol is set via the BIOS. See your system documentation for
more information.
4. Select iSCSI Boot Configuration from Main Menu.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
NOTE: If iSCSI boot firmware is not programmed in the NetXtreme II network adapter, selecting iSCSI Boot
Configuration will not have any effect.
iSCSI Boot Configuration
Static iSCSI Boot Configuration
Dynamic iSCSI Boot Configuration
Static iSCSI Boot Configuration
In a static configuration, you must enter data for the system's IP address, the system's initiator IQN, and the target
parameters obtained in Configuring the iSCSI Target. For information on configuration options, see Table 1.
To configure the iSCSI boot parameters using static configuration
1. From the General Parameters Menu screen, set the following:
TCP/IP parameters via DHCP: Disabled. (For IPv4.)
IP Autoconfiguration: Disabled. (For IPv6, non-offload.)
iSCSI parameters via DHCP: Disabled
CHAP Authentication: Disabled
DHCP Vendor ID: BRCM ISAN
Link Up Delay Time: 0
Use TCP Timestamp: Enabled (for some targets such as the Dell/EMC AX100i, it is necessary to enable Use
TCP Timestamp)
Target as First HDD: Disabled
LUN Busy Retry Count: 0
IP Version: IPv6. (For IPv6, non-offload.)
HBA Boot Mode: Disabled (Note: This parameter cannot be changed when the adapter is in Multi-Function
mode.)
2. Select ESC to return to the Main menu.
3. From the Main menu, select Initiator Parameters.
4. From the Initiator Parameters screen, type values for the following:
IP Address (unspecified IPv4 and IPv6 addresses should be "0.0.0.0" and "::", respectively)
Subnet Mask Prefix
Default Gateway
Primary DNS
Secondary DNS
iSCSI Name (corresponds to the iSCSI initiator name to be used by the client system)
NOTE: Carefully enter the IP address. There is no error-checking performed against the IP address to check
for duplicates or incorrect segment/network assignment.
5. Select ESC to return to the Main menu.
6. From the Main menu, select 1st Target Parameters.
NOTE: For the initial setup, configuring a second target is not supported.
7. From the 1st Target Parameters screen, enable Connect to connect to the iSCSI target. Type values for the
following using the values used when configuring the iSCSI target:
IP Address
TCP Port
Boot LUN
iSCSI Name
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
8. Select ESC to return to the Main menu.
9. Select ESC and select Exit and Save Configuration.
10. Select F4 to save your MBA configuration.
Dynamic iSCSI Boot Configuration
In a dynamic configuration, you only need to specify that the system's IP address and target/initiator information are
provided by a DHCP server (see IPv4 and IPv6 configurations in Configuring the DHCP Server to Support iSCSI Boot). For
IPv4, with the exception of the initiator iSCSI name, any settings on the Initiator Parameters, 1st Target Parameters, or 2nd
Target Parameters screens are ignored and do not need to be cleared. For IPv6, with the exception of the CHAP ID and
Secret, any settings on the Initiator Parameters, 1st Target Parameters, or 2nd Target Parameters screens are ignored and do
not need to be cleared. For information on configuration options, see Table 1.
NOTE: When using a DHCP server, the DNS server entries are overwritten by the values provided by the DHCP server.
This occurs even if the locally provided values are valid and the DHCP server provides no DNS server information. When the
DHCP server provides no DNS server information, both the primary and secondary DNS server values are set to 0.0.0.0. When
the Windows OS takes over, the Microsoft iSCSI initiator retrieves the iSCSI Initiator parameters and configures the
appropriate registries statically. It will overwrite whatever is configured. Since the DHCP daemon runs in the Windows
environment as a user process, all TCP/IP parameters have to be statically configured before the stack comes up in the iSCSI
Boot environment.
If DHCP Option 17 is used, the target information is provided by the DHCP server, and the initiator iSCSI name is retrieved
from the value programmed from the Initiator Parameters screen. If no value was selected, then the controller defaults to the
name:
iqn.1995-05.com.broadcom.<11.22.33.44.55.66>.iscsiboot
where the string 11.22.33.44.55.66 corresponds to the controller's MAC address.
If DHCP option 43 (IPv4 only) is used, then any settings on the Initiator Parameters, 1st Target Parameters, or 2nd Target
Parameters screens are ignored and do not need to be cleared.
To configure the iSCSI boot parameters using dynamic configuration
1. From the General Parameters Menu screen, set the following:
TCP/IP parameters via DHCP: Enabled. (For IPv4.)
IP Autoconfiguration: Enabled. (For IPv6, non-offload.)
iSCSI parameters via DHCP: Enabled
CHAP Authentication: Disabled
DHCP Vendor ID: BRCM ISAN
Link Up Delay Time: 0
Use TCP Timestamp: Enabled (for some targets such as the Dell/EMC AX100i, it is necessary to enable Use
TCP Timestamp)
Target as First HDD: Disabled
LUN Busy Retry Count: 0
IP Version: IPv6. (For IPv6, non-offload.)
HBA Boot Mode: Disabled. (Note: This parameter cannot be changed when the adapter is in Multi-Function
mode.)
2. Select ESC to return to the Main menu.
NOTE: Information on the Initiator Parameters, and 1st Target Parameters screens are ignored and do not
need to be cleared.
3. Select Exit and Save Configurations.
Enabling CHAP Authentication
Ensure that CHAP authentication is enabled on the target.
To enable CHAP authentication
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
1. From the General Parameters screen, set CHAP Authentication to Enabled.
two iSCSI target IQNs that can be used for booting. The format for the iSCSI target IQN is the same as that of DHCP
2. From the Initiator Parameters screen, type values for the following:
3. Select ESC to return to the Main menu.
4. From the Main menu, select 1st Target Parameters.
5. From the 1st Target Parameters screen, type values for the following using the values used when configuring the
6. Select ESC to return to the Main menu.
7. Select ESC and select Exit and Save Configuration.
CHAP ID (up to 128 bytes)
CHAP Secret (if authentication is required, and must be 12 characters in length or longer)
iSCSI target:
CHAP ID (optional if two-way CHAP)
CHAP Secret (optional if two-way CHAP, and must be 12 characters in length or longer)
Configuring the DHCP Server to Support iSCSI Boot
The DHCP server is an optional component and it is only necessary if you will be doing a dynamic iSCSI Boot configuration
setup (see Dynamic iSCSI Boot Configuration).
Configuring the DHCP server to support iSCSI boot is different for IPv4 and IPv6.
DHCP iSCSI Boot Configurations for IPv4
DHCP iSCSI Boot Configuration for IPv6
DHCP iSCSI Boot Configurations for IPv4
The DHCP protocol includes a number of options that provide configuration information to the DHCP client. For iSCSI boot,
Broadcom adapters support the following DHCP configurations:
DHCP Option 17, Root Path
DHCP Option 43, Vendor-Specific Information
DHCP Option 17, Root Path
Option 17 is used to pass the iSCSI target information to the iSCSI client.
The format of the root path as defined in IETC RFC 4173 is:
"iscsi:"<servername>":"<protocol>":"<port>":"<LUN>":"<targetname>"
The parameters are defined below.
Table 2: DHCP Option 17 Parameter Definition
Parameter Definition
"iscsi:"
<servername>
":"
<protocol>
<port>
<LUN>
<targetname>
A literal string
The IP address or FQDN of the iSCSI target
Separator
The IP protocol used to access the iSCSI target. Currently, only TCP is supported so the protocol is 6.
The port number associated with the protocol. The standard port number for iSCSI is 3260.
The Logical Unit Number to use on the iSCSI target. The value of the LUN must be represented in
hexadecimal format. A LUN with an ID OF 64 would have to be configured as 40 within the option 17
parameter on the DHCP server.
The target name in either IQN or EUI format (refer to RFC 3720 for details on both IQN and EUI formats). An
example IQN name would be "iqn.1995-05.com.broadcom:iscsi-target".
DHCP Option 43, Vendor-Specific Information
DHCP option 43 (vendor-specific information) provides more configuration options to the iSCSI client than DHCP option 17. In
this configuration, three additional suboptions are provided that assign the initiator IQN to the iSCSI boot client along with
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
option 17, while the iSCSI initiator IQN is simply the initiator's IQN.
NOTE: DHCP Option 43 is supported on IPv4 only.
The suboptions are listed below.
Table 3: DHCP Option 43 Suboption Definition
Suboption Definition
201
203 iSCSI initiator IQN
Using DHCP option 43 requires more configuration than DHCP option 17, but it provides a richer environment and provides
more configuration options. Broadcom recommends that customers use DHCP option 43 when performing dynamic iSCSI boot
configuration.
Configuring the DHCP Server
Configure the DHCP server to support option 17 or option 43.
NOTE: If using Option 43, you also need to configure Option 60. The value of Option 60 should match the DHCP Vendor
ID value. The DHCP Vendor ID value is BRCM ISAN, as shown in General Parameters of the iSCSI Boot Configuration
menu.
First iSCSI target information in the standard root path format
"iscsi:"<servername>":"<protocol>":"<port>":"<LUN>":"<targetname>"
DHCP iSCSI Boot Configuration for IPv6
The DHCPv6 server can provide a number of options, including stateless or stateful IP configuration, as well s information to
the DHCPv6 client. For iSCSI boot, Broadcom adapters support the following DHCP configurations:
DHCPv6 Option 16, Vendor Class Option
DHCPv6 Option 17, Vendor-Specific Information
NOTE: The DHCPv6 standard Root Path option is not yet available. Broadcom suggests using Option 16 or Option 17
for dynamic iSCSI Boot IPv6 support.
DHCPv6 Option 16, Vendor Class Option
DHCPv6 Option 16 (vendor class option) must be present and must contain a string that matches your configured DHCP
Vendor ID parameter. The DHCP Vendor ID value is BRCM ISAN, as shown in General Parameters of the iSCSI Boot
Configuration menu.
The content of Option 16 should be <2-byte length> <DHCP Vendor ID>.
DHCPv6 Option 17, Vendor-Specific Information
DHCPv6 Option 17 (vendor-specific information) provides more configuration options to the iSCSI client. In this configuration,
three additional suboptions are provided that assign the initiator IQN to the iSCSI boot client along with two iSCSI target
IQNs that can be used for booting.
The suboptions are listed below.
Table 4: DHCP Option 17 Suboption Definition
Suboption Definition
201
203 iSCSI initiator IQN
NOTE: In Table 4, the brackets [ ] are required for the IPv6 addresses.
The content of option 17 should be <2-byte Option Number 201|202|203> <2-byte length> <data>.
First iSCSI target information in the standard root path format
"iscsi:"[<servername>]":"<protocol>":"<port>":"<LUN>":"<targetname>"
Configuring the DHCP Server
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Configure the DHCP server to support Option 16 and Option 17.
NOTE: The format of DHCPv6 Option 16 and Option 17 are fully defined in RFC 3315.
Preparing the iSCSI Boot Image
Windows Server 2008 R2 and SP2 iSCSI Boot Setup
Windows Server 2012 iSCSI Boot Setup
Linux iSCSI Boot Setup
Removing Inbox Drivers from Windows 2008/R2 OS Image
Removing Inbox Drivers from Windows 2012/R2 OS Image
Injecting (Slipstreaming) Broadcom Drivers into Windows Image Files
Windows Server 2008 R2 and SP2 iSCSI Boot Setup
Windows Server 2008 R2 and Windows Server 2008 SP2 support booting as well as installing in either the offload or nonoffload paths.
The following procedure prepares the image for installation and booting in either the offload or non-offload path. The following
procedure references Windows Server 2008 R2 but is common to both the Windows Server 2008 R2 and SP2.
Required CD/ISO image:
Windows Server 2008 R2 x64 with the Broadcom drivers injected. See Injecting (Slipstreaming) Broadcom Drivers into
Windows Image Files. Also refer to the Microsoft knowledge base topic KB974072 at support.microsoft.com.
NOTES:
The Microsoft procedure injects only the eVBD and NDIS drivers. Broadcom recommends that all
drivers (eVBD, VBD, BXND, OIS, FCoE, and NetXtreme I NDIS) be injected.
Refer to the silent.txt file for the specific driver installer application for instructions on how to extract
the individual Windows NetXtreme II drivers.
Other software required:
Bindview.exe (Windows Server 2008 R2 only; see KB976042)
Procedure:
1. Remove any local hard drives on the system to be booted (the "remote system").
2. Load the latest Broadcom MBA and iSCSI boot images onto NVRAM of the adapter.
3. Configure the BIOS on the remote system to have the Broadcom MBA as the first bootable device, and the CDROM as
the second device.
4. Configure the iSCSI target to allow a connection from the remote device. Ensure that the target has sufficient disk
space to hold the new O/S installation.
5. Boot up the remote system. When the Preboot Execution Environment (PXE) banner displays, press Ctrl+S to enter
the PXE menu.
6. At the PXE menu, set Boot Protocol to iSCSI.
7. Enter the iSCSI target parameters.
8. Set HBA Boot Mode to Enabled or Disabled. (Note: This parameter cannot be changed when the adapter is in
Multi-Function mode.)
9. Save the settings and reboot the system.
The remote system should connect to the iSCSI target and then boot from the DVDROM device.
10. Boot to DVD and begin installation.
11. Answer all the installation questions appropriately (specify the Operating System you want to install, accept the license
terms, etc.).
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
When the Where do you want to install Windows? window appears, the target drive should be visible. This is a
drive connected via the iSCSI boot protocol, located in the remote iSCSI target.
12. Select Next to proceed with Windows Server 2008 R2 installation.
A few minutes after the Windows Server 2008 R2 DVD installation process starts, a system reboot will follow. After
the reboot, the Windows Server 2008 R2 installation routine should resume and complete the installation.
13. Following another system restart, check and verify that the remote system is able to boot to the desktop.
14. After Windows Server 2008 R2 is booted up, load all drivers and run Bindview.exe.
a. Select All Services.
b. Under WFP Lightweight Filter you should see Binding paths for the AUT. Right-click and disable them. When
done, close out of the application.
15. Verify that the OS and system are functional and can pass traffic by pinging a remote system's IP, etc.
Windows Server 2012 iSCSI Boot Setup
Windows Server 2012 supports booting as well as installing in either the offload or non-offload paths. Broadcom requires the
use of a "slipstream" DVD with the latest Broadcom drivers injected. See Injecting (Slipstreaming) Broadcom Drivers into
Windows Image Files. Also refer to the Microsoft knowledge base topic KB974072 at support.microsoft.com.
NOTE: The Microsoft procedure injects only the eVBD and NDIS drivers. Broadcom recommends that all drivers
(eVBD, VBD, BXND, OIS, FCoE, and NetXtreme I NDIS) be injected.
The following procedure prepares the image for installation and booting in either the offload or non-offload path:
1. Remove any local hard drives on the system to be booted (the "remote system").
2. Load the latest Broadcom MBA and iSCSI boot images into the NVRAM of the adapter.
3. Configure the BIOS on the remote system to have the Broadcom MBA as the first bootable device and the CDROM as
the second device.
4. Configure the iSCSI target to allow a connection from the remote device. Ensure that the target has sufficient disk
space to hold the new O/S installation.
5. Boot up the remote system. When the Preboot Execution Environment (PXE) banner displays, press Ctrl+S to enter
the PXE menu.
6. At the PXE menu, set Boot Protocol to iSCSI.
7. Enter the iSCSI target parameters.
8. Set HBA Boot Mode to Enabled or Disabled. (Note: This parameter cannot be changed when the adapter is in
Multi-Function mode.)
9. Save the settings and reboot the system.
The remote system should connect to the iSCSI target and then boot from the DVDROM device.
10. Boot from DVD and begin installation.
11. Answer all the installation questions appropriately (specify the Operating System you want to install, accept the license
terms, etc.).
When the Where do you want to install Windows? window appears, the target drive should be visible. This is a
drive connected via the iSCSI boot protocol, located in the remote iSCSI target.
12. Select Next to proceed with Windows Server 2012 installation.
A few minutes after the Windows Server 2012 DVD installation process starts, a system reboot will occur. After the
reboot, the Windows Server 2012 installation routine should resume and complete the installation.
13. Following another system restart, check and verify that the remote system is able to boot to the desktop.
14. After Windows Server 2012 boots to the OS, Broadcom recommends running the driver installer to complete the
Broadcom drivers and application installation.
Linux iSCSI Boot Setup
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Linux iSCSI boot is supported on Red Hat Enterprise Linux 5.5 and later and SUSE Linux Enterprise Server 11 SP1 and later in
both the offload and non-offload paths. Note that SLES 10.x and SLES 11 have support only for the non-offload path.
1. For driver update, obtain the latest Broadcom Linux driver CD.
2. Configure the iSCSI Boot Parameters for DVD direct install to target by disabling the Boot from target option on the
network adapter.
3. Configure to install via the non-offload path by setting HBA Boot Mode to Disabled in the NVRAM Configuration.
(Note: This parameter cannot be changed when the adapter is in Multi-Function mode.). Note that, for RHEL6.2 and
SLES11SP2 and newer, installation via the offload path is supported. For this case, set the HBA Boot Mode to Enabled
in the NVRAM Configuration.
4. Change the boot order as follows:
a. Boot from the network adapter.
b. Boot from the CD/DVD driver.
5. Reboot the system.
6. System will connect to iSCSI target, then will boot from CD/DVD drive.
7. Follow the corresponding OS instructions.
a. RHEL 5.5 — Type "linux dd" at "boot:" prompt and press enter
b. SuSE 11.X — Choose installation and type withiscsi=1 netsetup=1 at the boot option. If driver update is
desired, choose YES for the F6 driver option.
8. If driver update is desired, follow the instructions to load the driver CD; otherwise skip this step.
9. At the "networking device" prompt, choose the desired network adapter port and press OK.
10. At "configure TCP/IP prompt", configure the way the system acquire IP address and press OK.
11. If static IP was chosen, you need to enter IP information for iscsi initiator.
12. (RHEL) Choose to "skip" media testing.
13. Continue installation as desired. A drive will be available at this point. After file copying is done, remove CD/DVD and
reboot the system.
14. When the system reboots, enable "boot from target" in iSCSI Boot Parameters and continue with installation until it is
done.
At this stage, the initial installation phase is complete. The rest of the procedure pertains to creating a new customized initrd
for any new components update:
1. Update iscsi initiator if desired. You will first need to remove the existing initiator using rpm -e.
2. Make sure all runlevels of network service are on:
chkconfig network on
3. Make sure 2,3 and 5 runlevels of iscsi service are on.
chkconfig -level 235 iscsi on
4. For Red Hat 6.0, make sure Network Manager service is stopped and disabled.
5. Install iscsiuio if desired (not required for SuSE 10).
6. Install linux-nx2 package if desired.
7. Install bibt package.
8. Remove ifcfg-eth*.
9. Reboot.
10. For SUSE 11.1, follow the remote DVD installation workaround shown below.
11. After the system reboots, log in, change to the /opt/bcm/bibt folder, and run iscsi_setup.sh script to create the offload
and/or the non-offload initrd image.
12. Copy the initrd image(s), offload and/or non-offload, to the /boot folder.
13. Change the grub menu to point to the new initrd image.
14. To enable CHAP, you need to modify iscsid.conf (Red Hat only).
15. Reboot and change CHAP parameters if desired.
16. Continue booting into the iSCSI Boot image and select one of the images you created (non-offload or offload). Your
choice should correspond with your choice in the iSCSI Boot parameters section. If HBA Boot Mode was enabled in the
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
iSCSI Boot Parameters section, you have to boot the offload image. SLES 10.x and SLES 11 do not support offload.
17. For IPv6, you can now change the IP address for both the initiator and the target to the desired IPv6 address in the
NVRAM configuration.
SUSE 11.1 Remote DVD installation workaround
1. Create a new file called boot.open-iscsi with the content shown below.
2. Copy the file you just created to /etc/init.d/ folder and overwrite the existing one.
Content of the new boot.open-iscsi file:
#!/bin/bash
#
# /etc/init.d/iscsi
#
### BEGIN INIT INFO
# Provides: iscsiboot
# Required-Start:
# Should-Start: boot.multipath
# Required-Stop:
# Should-Stop: $null
# Default-Start: B
# Default-Stop:
# Short-Description: iSCSI initiator daemon root-fs support
# Description: Starts the iSCSI initiator daemon if the
# root-filesystem is on an iSCSI device
#
### END INIT INFO
ISCSIADM=/sbin/iscsiadm
ISCSIUIO=/sbin/iscsiuio
CONFIG_FILE=/etc/iscsid.conf
DAEMON=/sbin/iscsid
ARGS="-c $CONFIG_FILE"
# Source LSB init functions
. /etc/rc.status
#
# This service is run right after booting. So all targets activated
# during mkinitrd run should not be removed when the open-iscsi
# service is stopped.
#
iscsi_load_iscsiuio()
{
TRANSPORT=`$ISCSIADM -m session 2> /dev/null | grep "bnx2i"`
if [ "$TRANSPORT" ] ; then
echo -n "Launch iscsiuio "
startproc $ISCSIUIO
fi
}
iscsi_mark_root_nodes()
{
$ISCSIADM -m session 2> /dev/null | while read t num i target ; do
ip=${i%%:*}
STARTUP=`$ISCSIADM -m node -p $ip -T $target 2> /dev/null | grep "node.conn\[0\].startup" | cut d' ' -f3`
if [ "$STARTUP" -a "$STARTUP" != "onboot" ] ; then
$ISCSIADM -m node -p $ip -T $target -o update -n node.conn[0].startup -v onboot
fi
done
}
# Reset status of this service
rc_reset
# We only need to start this for root on iSCSI
if ! grep -q iscsi_tcp /proc/modules ; then
if ! grep -q bnx2i /proc/modules ; then
rc_failed 6
rc_exit
fi
fi
case "$1" in
start)
echo -n "Starting iSCSI initiator for the root device: "
iscsi_load_iscsiuio
startproc $DAEMON $ARGS
rc_status -v
iscsi_mark_root_nodes
;;
stop|restart|reload)
rc_failed 0
;;
status)
echo -n "Checking for iSCSI initiator service: "
if checkproc $DAEMON ; then
rc_status -v
else
rc_failed 3
rc_status -v
fi
;;
*)
echo "Usage: $0 {start|stop|status|restart|reload}"
exit 1
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
;;
esac
rc_exit
Removing Inbox Drivers from Windows 2008/R2 OS Image
1. Create a temporary folder, such as D:\temp.
2. Create the following two subfolders in the temporary folder:
Win2008R2Copy
Win2008R2Mod
3. Copy all the contents from the DVD installation media into the Win2008R2Copy folder.
4. Open the Windows AIK command prompt in elevated mode from All program, and then run the following command:
attrib -r D:\Temp\Win2008R2Copy\sources\boot.wim
5. Run the following command to mount the boot.wim image:
dism /Mount-WIM /WimFile:D:\Temp\Win2008R2Copy\sources\boot.wim /index:1
/MountDir:D:\Temp\Win2008R2Mod
6. The Boot.wim image was mounted in the Win2008R2Mod folder. Find all the instances of the files listed below in the
subfolders of the Win2008R2Mod folder, and then delete them.
List of files to be deleted.
netevbda.inf
netevbda.pnf
evbda.sys
netbxnda.inf
netbxnda.pnf
bxnd60a.sys
bxvbda.sys
netbvbda.inf
netbvbda.pnf
To easily find all the instances of the files to be deleted, run the following command:
dir /s D:\Temp\Win2008R2Mod\filename
7. Run the following command to unmount the Boot.wim image:
dism /unmount-wim /Mountdir:D:\Temp\Win2008R2Mod /commit
8. Repeat steps 5 to 7, but set the index = 2 for the command in step 5.
In this example, index 2 is specified for the standard edition. For other editions, change the index accordingly.
Removing Inbox Drivers from Windows 2012/R2 OS Image
1. Create a temporary folder, such as D:\temp.
2. Create the following two subfolders in the temporary folder:
Win2012R2Copy
Win2012R2Mod
3. Copy all the contents from the DVD installation media into the Win2012R2Copy folder.
4. Open the Windows AIK command prompt in elevated mode from All program, and then run the following command:
attrib -r D:\Temp\Win2012R2Copy\sources\boot.wim
5. Run the following command to mount the boot.wim image:
dism /Mount-WIM /WimFile:D:\Temp\Win2012R2Copy\sources\boot.wim /index:2
/MountDir:D:\Temp\Win2012R2Mod
NOTE: You must always use "2" for the index value.
6. The Boot.wim image was mounted in the Win2012R2Mod folder.
7. Delete the files listed below, in the subfolders of the Win2012R2Mod folder.
List of files to be deleted in these two folders.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
INFs (e.g. D:\Temp\Win2012R2Mod\Windows\INF)
netevbda.inf
netbxnda.inf
netbvbda.inf
bxfcoe.inf
bxois.inf
Drivers (e.g. D:\Temp\Win2012R2Mod\Windows\System32\drivers)
bxfcoe.sys
bxois.sys
bnxcd.sys
bnxfcd.sys
bxvbda.sys
bxnd60a.sys
evbda.sys
8. Run the following command to unmount the Boot.wim image:
dism /unmount-wim /Mountdir:D:\Temp\Win2012R2Mod /commit
Injecting (Slipstreaming) Broadcom Drivers into Windows Image Files
To inject Broadcom drivers into the Windows image files, you must obtain the following correct Broadcom driver packages for
the applicable Windows Server version (2008R2, 2008SP2, 2012, or 2012R2) from the following location:
http://www.broadcom.com/support/ethernet_nic/netxtremeii10.php
NOTE: Refer to the silent.txt file for the specific driver installer application for instructions on how to extract the individual
Windows NetXtreme II drivers.
EVBDa
BXND60a
BXOIS
BXFCOE
Then, you place these driver packages to a working directory. For example, copy the driver packages to the following
directories:
C:\Temp\evbda
C:\Temp\bxnd60a
C:\Temp\bxois
C:\Temp\bxfcoe
Finally, you inject these drivers into the Windows Image (WIM) files and install the applicable Windows Server version from
the updated images.
The detailed steps are provided below:
NOTE: The file and folder names used in this procedure are examples only. You can specify your own file and folder
names for your slipstream project.
1. For Windows Server 2008 R2 and SP2, install the Windows Automated Installation Kit (AIK).
—or—
For Windows Server 2012 and 2012 R2, install the Windows Assessment and Deployment Kit (ADK).
2. Use the following commands to create a temporary directory and set it as the current directory for all later steps:
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
md C:\Temp\x
cd /d C:\Temp\x
3. Use the following commands to create two subdirectories:
md src
md mnt
4. Use the following command to copy the original DVD into the src subdirectory.
xcopy N:\ .\src /e /c /i /f /h /k /y /q
Note that in this example, the installation DVD is in the N: drive.
5. Open a Deployment and Imaging Tools command prompt in elevated mode. Then, set c:\Temp\x as the current
directory.
Note that you will use this command prompt window in all subsequent steps.
6. Enter the following commands:
attrib -r .\src\sources\boot.wim
attrib -r .\src\sources\install.wim
7. Enter run the following command to mount the boot.wim image:
dism /mount-wim /wimfile:.\src\sources\boot.wim /index:2 /mountdir:.\mnt
Note that you must always use "2" for the index value.
8. Enter the following commands to add the below drivers to the currently mounted image:
dism /image:.\mnt /add-driver /driver:C:\Temp\evbda\evbda.inf
dism /image:.\mnt /add-driver /driver:C:\Temp\bxnd60a\bxnd60a.inf
dism /image:.\mnt /add-driver /driver:C:\Temp\bxfcoe\bxfcoe.inf
dism /image:.\mnt /add-driver /driver:C:\Temp\bxois\bxois.inf
9. Enter the following command to unmount the boot.wim image:
dism /unmount-wim /mountdir:.\mnt /commit
10. Enter the following command to determine the index of the desired SKU in the install.wim image:
dism /get-wiminfo /wimfile:.\src\sources\install.wim
For example, in Windows Server 2012, index 2 is identified as "Windows Server 2012 SERVERSTANDARD."
11. Enter the following command to mount the install.wim image:
dism /mount-wim /wimfile:.\src\sources\install.wim /index:X /mountdir:.\mnt
Note that X is a placeholder for the index value that you obtained in step 10.
12. Enter the following commands to add these drivers to the currently mounted image:
dism /image:.\mnt /add-driver /driver:C:\Temp\evbd\evbd.inf
dism /image:.\mnt /add-driver /driver:C:\Temp\bxnd60a\bxnd60a.inf
dism /image:.\mnt /add-driver /driver:C:\Temp\bxfcoe\bxfcoe.inf
dism /image:.\mnt /add-driver /driver:C:\Temp\bxois\bxois.inf
13. Enter the following command to unmount the install.wim image:
dism /unmount-wim /mountdir:.\mnt /commit
14. Enter the following command to create an .iso file:
oscdimg -e -h -m -n -lslipstream -bootdata:2#p0,e,b"c:\Program Files\Windows
AIK\Tools\PETools\amd64\boot\etfsboot.com"#pEF,e,b"c:\Program Files\Windows
AIK\Tools\PETools\amd64\boot\efisys.bin" c:\temp\x\src c:\temp\Win20xxMOD.iso
Note that Platform is a placeholder for the architecture of the operating system that you want to install, such as
amd64 or x86. Also, xx in the file names is a placeholder for the Windows Server OS version (2012, 2008R2,
2008SP2.)
15. Using a DVD-burning application, burn the .iso file you created to a DVD.
16. Use the DVD that you created in step 15 to install the applicable Windows Server version.
Booting
After that the system has been prepared for an iSCSI boot and the operating system is present on the iSCSI target, the last
step is to perform the actual boot. The system will boot to Windows or Linux over the network and operate as if it were a
local disk drive.
1. Reboot the server.
2. Select CTRL+S.
3. To boot through an offload path, set the HBA Boot Mode to Enabled. To boot through a non-offload path, set the HBA
Boot Mode to Disabled. (Note: This parameter cannot be changed when the adapter is in Multi-Function mode.)
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
If CHAP authentication is needed, enable CHAP authentication after determining that booting is successful (see Enabling CHAP
Authentication).
Other iSCSI Boot Considerations
There are several other factors that should be considered when configuring a system for iSCSI boot.
Changing the Speed & Duplex Settings in Windows Environments
Changing the Speed & Duplex settings on the boot port using Windows Device Manager when performing iSCSI boot via the
offload path is not supported. Booting via the NDIS path is supported. The Speed & Duplex settings can be changed using the
BACS management utility for iSCSI boot via the offload and NDIS paths.
Locally Administered Address
A user-defined MAC address assigned through the Locally Administered Address property of the Advanced section of the BACS
Configurations tab is not supported on iSCSI boot-enabled devices.
Virtual LANs
Virtual LAN (VLAN) tagging is not supported for iSCSI boot with the Microsoft iSCSI Software Initiator.
The 'dd' method of creating an iSCSI boot image
In the case when installation directly to a remote iSCSI target is not an option, an alternate way to create such an image is to
use the `dd' method. With this method, you install the image directly to a local hard drive and then create an iSCSI boot
image for the subsequent boot:
1. Install Linux OS on your local hard drive and ensure that the Open-iSCSI initiator is up to date.
2. Ensure that all Runlevels of network service are on.
3. Ensure that the 2, 3, and 5 Runlevels of iSCSI service are on.
4. Update iscsiuio. You can get the iscsiuio package from the Broadcom CD. This step is not needed for SuSE 10.
5. Install the linux-nx2 package on your Linux system. You can get this package from Broadcom CD.
6. Install bibt package on you Linux system. You can get this package from Broadcom CD.
7. Delete all ifcfg-eth* files.
8. Configure one port of the network adapter to connect to iSCSI Target (for instructions, see Configuring the iSCSI
Target).
9. Connect to the iSCSI Target.
10. Use the DD command to copy from the local hard drive to iSCSI Target.
11. When DD is done, execute the sync command a couple of times, log out, and then log in to iSCSI Target again.
12. Run the fsck command on all partitions created on the iSCSI Target.
13. Change to the /OPT/bcm/bibt folder and run the iscsi_setup.sh script to create the initrd images. Option 0 will create a
non-offload image and option 1 will create an offload image. The Iscsi_script.sh script will create the non-offload image
only on SuSE 10 as offload is not supported on SuSE 10.
14. Mount the /boot partition on the iSCSI Target.
15. Copy the initrd images you created in step 13 from your local hard drive to the partition mounted in step 14.
16. On the partition mounted in step 14, edit the grub menu to point to the new initrd images.
17. Unmount the /boot partition on the iSCSI Target.
18. (Red Hat Only) To enable CHAP, you need to modify the CHAP section of the iscsid.conf file on the iSCSI Target. Edit
the iscsid.conf file with one-way or two-way CHAP information as desired.
19. Shut down the system and disconnect the local hard drive. Now you are ready to iSCSI boot the iSCSI Target.
20. Configure iSCSI Boot Parameters, including CHAP parameters if desired (see Configuring the iSCSI Target).
21. Continue booting into the iSCSI Boot image and choose one of the images you created (non-offload or offload). Your
choice should correspond with your choice in the iSCSI Boot parameters section. If HBA Boot Mode was enabled in the
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
iSCSI Boot Parameters section, you have to boot the offload image. SuSE 10.x and SLES 11 do not support offload.
Troubleshooting iSCSI Boot
The following troubleshooting tips are useful for iSCSI boot.
Problem: A system blue screen occurs when iSCSI boots Windows Server 2008 R2 through the adapter's NDIS path with the
initiator configured using a link-local IPv6 address and the target configured using a router-configured IPv6 address.
Solution: This is a known Windows TCP/IP stack issue.
Problem: The Broadcom iSCSI Crash Dump utility will not work properly to capture a memory dump when the link speed for
iSCSI boot is configured for 10 Mbps or 100 Mbps.
Solution: The iSCSI Crash Dump utility is supported when the link speed for iSCSI boot is configured for 1 Gbps or 10 Gbps.
10 Mbps or 100 Mbps is not supported.
Problem: An iSCSI target is not recognized as an installation target when you try to install Windows Server 2008 by using an
IPv6 connection.
Solution: This is a known third-party issue. See Microsoft Knowledge Base KB 971443,
http://support.microsoft.com/kb/971443.
Problem: When switching iSCSI boot from the Microsoft standard path to Broadcom iSCSI offload, the booting fails to
complete.
Solution: Install or upgrade the Broadcom Virtual Bus Device (VBD) driver to 5.0.x, along with the OIS driver, prior to
switching the iSCSI boot path.
Problem: The iSCSI configuration utility will not run.
Solution: Ensure that the iSCSI Boot firmware is installed in the NVRAM.
Problem: A system blue screen occurs when installing the Broadcom drivers through Windows Plug-and-Play (PnP).
Solution: Install the drivers through the Setup installer.
Problem: For static IP configuration when switching from Layer 2 iSCSI boot to Broadcom iSCSI HBA, then you will receive
an IP address conflict.
Solution: Change the IP address of the network property in the OS.
Problem: After configuring the iSCSI boot LUN to 255, a system blue screen appears when performing iSCSI boot.
Solution: Although Broadcom's iSCSI solution supports a LUN range from 0 to 255, the Microsoft iSCSI software initiator
does not support a LUN of 255. Configure a LUN value from 0 to 254.
Problem: NDIS miniports with Code 31 yellow-bang after L2 iSCSI boot install.
Solution: Run the T7.4 installer.
Problem: Unable to update inbox driver if a non-inbox hardware ID present.
Solution: Create a custom slipstream DVD image with supported drivers present on the install media.
Problem: In Windows Server 2012, toggling between iSCSI HBA offload mode and iSCSI software initiator boot can leave the
machine in a state where the HBA offload miniport bxois will not load.
Solution: Manually edit [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\bxois\StartOverride] from 3 to 0.
Modify the registry key before toggling back from NDIS to HBA path in CCM.
NOTE: Microsoft recommends against this method. Toggling the boot path from NDIS to HBA or vice versa
after installation is completed is not recommended.
Problem: The Xen hypervisor will not start when booting from an iSCSI image created with the RHEL 5.4 Xen kernel.
Solution: This is a known third-party issue. To work around this issue, disable the Xen hypervisor's EDD feature by editing
the grub.conf file in the boot/grub folder to add the edd=off switch to the end of the kernel line. For example,
kernel /xen.gz edd=off.
Problem: Unable to connect to an EqualLogic target using Windows Server 2008 and higher.
Solution: Add an exception to your firewall to allow ICMP echo requests.
Problem: Installing Windows onto an iSCSI target via iSCSI boot fails when connecting to a 1 Gbps switch port.
Solution: This is a limitation relating to adapters that use SFP+ as the physical connection. SFP+ defaults to 10 Gbps
operation and does not support autonegotiation.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Problem: SLES 11 SP3 - iSCSI remote installation on 1 Gbps port of the 57800 NIC adapter fails with inbox driver. Loading
the external driver does not unload inbox bnx2x driver as well.
Solution: Enter or append the following syntax bnx2x.num_vfs=0 in the command prompt at the initial installation in order to
prevent the inbox bnx2x driver from loading. Example: at step 7b in the Linux iSCSI Boot Setup section, enter:
withiscsi=1 netsetup=1 bnx2x.num_vfs=0
iSCSI Crash Dump
If you will use the Broadcom iSCSI Crash Dump utility, it is important to follow the installation procedure to install the iSCSI
Crash Dump driver. See Using the Installer for more information.
iSCSI Offload in Windows Server
iSCSI traffic may be segregated offload is a technology that offloads iSCSI protocol processing overhead from host processors
to the iSCSI host bus adapter to increase network performance and throughput while helping to optimize server processor
utilization.
This section covers Broadcom's iSCSI offload feature for the NetXtreme II family of network adapters on Windows Server
systems. For Linux iSCSI offload, see Linux iSCSI Offload.
iSCSI Offload Limitations
The bnx2i driver for iSCSI does not operate on a stand-alone PCI device. It shares the same PCI device with the networking
driver (bnx2 and bnx2x). The networking driver alone supports layer 2 networking traffic. Offloaded iSCSI operations require
both the networking driver and the bnx2i driver.
iSCSI operations will be interrupted when the networking driver brings down or resets the device. This scenario requires
proper handling by the networking and bnx2i drivers, as well as the userspace iscsid daemon that keeps track of all iSCSI
sessions. Offloaded iSCSI connections take up system and on-chip resources that must be freed up before the device can be
reset. iscsid running in userspace is generally less predictable, as it can run slowly and take a long time to disconnect and
reconnect iSCSI sessions during network reset, especially when the number of connections is large. Broadcom cannot
guarantee that iSCSI sessions will always recover in every conceivable scenario when the networking device is repeatedly
being reset. Broadcom recommends that administrator-administered network device resets, such as MTU change, ring size
change, device shutdown, hot-unplug, and so forth, be kept at a minimum while there are active offloaded iSCSI sessions
running on that shared device. On the other hand, link-related changes do not require device reset and are safe to be
performed at any time.
To help alleviate some of the above issues, install the latest open-iscsi utilities by upgrading your Red Hat Network
subscription.
Configuring iSCSI Offload
With the proper iSCSI offload licensing, you can configure your iSCSI-capable NetXtreme II network adapter to offload iSCSI
processing from the host processor. The following process enables your system to take advantage of Broadcom's iSCSI
offload feature.
Installing Broadcom Drivers and Management Applications
Installing the Microsoft iSCSI Initiator
Configuring Broadcom iSCSI Using BACS
Configure Microsoft Initiator to Use Broadcom's iSCSI Offload
Installing Broadcom Drivers and Management Applications
Install the Windows drivers and management applications. See Installing Windows Drivers and Management Applications.
Installing the Microsoft iSCSI Initiator
For Windows Server 2008 and later, the iSCSI initiator is included inbox. To download the iSCSI initiator from Microsoft, go to
http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=18986 and locate the direct link for your system.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Configuring Broadcom iSCSI Using BACS
The Broadcom Advanced Control Suite (BACS) is used to manage all of Broadcom's network adapters and advanced features.
For more information, see Using Broadcom Advanced Control Suite 4.
1. Open BACS.
2. Select the Broadcom NetXtreme II C-NIC iSCSI adapter. If the C-NIC iSCSI adapter is not present, then select the VBD
device and enable iSCSI offload by selecting iSCSI Offload Engine from the Resource Reservations area of the
Configuration tab. See Viewing and Configuring Resource Reservations.
3. Select the Configuration tab.
4. DHCP is the default for IP address assignment, but this can be changed to static IP address assignment, if this is the
preferred method of IP address assignment.
NOTE: The IP address assignment method cannot be changed if the adapter was used for boot.
5. Select Apply and close BACS.
Configure Microsoft Initiator to Use Broadcom's iSCSI Offload
Now that the IP address has been configured for the iSCSI adapter, you need to use Microsoft Initiator to configure and add a
connection to the iSCSI target using Broadcom iSCSI adapter. See Microsoft's user guide for more details on Microsoft
Initiator.
1. Open Microsoft Initiator.
2. Configure the initiator IQN name according to your setup. To change, click on Change.
3. Enter the initiator IQN name.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
4. Select the Discovery tab and click Add to add a target portal.
5. Enter the IP address of the target and click Advanced.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
6. From the General tab, select Broadcom NetXtreme II C-NIC iSCSI Adapter from Local adapter.
7. Select the IP address for the adapter from Source IP.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
8. Click OK to close Advanced setting and then OK to add the target portal.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
9. From the Targets tab, select the target and click Log On to log into your iSCSI target using the Broadcom iSCSI
adapter.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
10. Click on Advanced.
11. On the General tab, select the Broadcom NetXtreme Ii C-NIC iSCSI adapter from Local adapter.
12. Click OK to close Advanced settings.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
13. Click OK to close the Microsoft Initiator.
14. To format your iSCSI partition, use Disk Manager.
NOTES:
Teaming does not support iSCSI adapters.
Teaming does not support NDIS adapters that are in the boot path.
Teaming supports NDIS adapters that are not in the iSCSI boot path, but only for the SLB team type.
iSCSI Offload FAQs
Q: How do I assign an IP address for iSCSI offload?
A: Use the Configurations tab in Broadcom Advanced Control Suite (BACS).
Q: What tools should be used to create the connection to the target?
A: Use Microsoft iSCSI Software Initiator (version 2.08 or later).
Q: How do I know that the connection is offloaded?
A: Use Microsoft iSCSI Software Initiator. From a command line, type iscsicli sessionlist. From Initiator Name, an
iSCSI offloaded connection will display an entry beginning with "B06BDRV...". A non-offloaded connection will display an entry
beginning with "Root...".
Q: What configurations should be avoided?
A: The IP address should not be the same as the LAN.
Q: Why does the install fail when attempting to complete an iSCSI offload install using Windows Server 2008 R2 for BCM5709
(1 GbE) adapters?
A: There is a conflict with the internal inbox driver.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
Event Log Messages
Table 5 lists the offload iSCSI driver event log messages.
Offload iSCSI (OIS) Driver
Table 5: Offload iSCSI (OIS) Driver Event Log Messages
Message
Number
1 Error
2 Error The initiator could not allocate resources for an iSCSI session.
3 Error
4 Error
5 Error Failed to setup initiator portal. Error status is given in the dump data.
6 Error The initiator could not allocate resources for an iSCSI connection
7 Error The initiator could not send an iSCSI PDU. Error status is given in the dump data.
8 Error
9 Error Target did not respond in time for a SCSI request. The CDB is given in the dump data.
10 Error Login request failed. The login response packet is given in the dump data.
11 Error
12 Error Target provided invalid data for login redirect. Dump data contains the data returned by the target.
13 Error Target offered an unknown AuthMethod. Dump data contains the data returned by the target.
14 Error
15 Error
16 Error An invalid key was received during CHAP negotiation. The key=value pair is given in the dump data.
17 Error
18 Error Header Digest is required by the initiator, but target did not offer it.
19 Error Data Digest is required by the initiator, but target did not offer it.
20 Error Connection to the target was lost. The initiator will attempt to retry the connection.
21 Error
22 Error Header digest error was detected for the given PDU. Dump data contains the header and digest.
23 Error Target sent an invalid iSCSI PDU. Dump data contains the entire iSCSI header.
24 Error Target sent an iSCSI PDU with an invalid opcode. Dump data contains the entire iSCSI header.
25 Error
26 Error Target trying to send more data than requested by the initiator.
27 Error
28 Error Initiator received an invalid R2T packet. Dump data contains the entire iSCSI header.
29 Error Target rejected an iSCSI PDU sent by the initiator. Dump data contains the rejected PDU.
30 Error Initiator could not allocate a work item for processing a request.
31 Error Initiator could not allocate resource for processing a request.
Severity Message
Initiator failed to connect to the target. Target IP address and TCP Port number are given in dump
data.
Maximum command sequence number is not serially greater than expected command sequence
number in login response. Dump data contains Expected Command Sequence number followed by
Maximum Command Sequence number.
MaxBurstLength is not serially greater than FirstBurstLength. Dump data contains FirstBurstLength
followed by MaxBurstLength.
Target or discovery service did not respond in time for an iSCSI request sent by the initiator. iSCSI
Function code is given in the dump data. For details about iSCSI Function code please refer to iSCSI
User's Guide.
Target returned an invalid login response packet. The login response packet is given in the dump
data.
Target offered an unknown digest algorithm for CHAP. Dump data contains the data returned by the
target.
CHAP challenge given by the target contains invalid characters. Dump data contains the challenge
given.
CHAP Response given by the target did not match the expected one. Dump data contains the CHAP
response.
Data Segment Length given in the header exceeds MaxRecvDataSegmentLength declared by the
target.
Data digest error was detected. Dump data contains the calculated checksum followed by the given
checksum.
Initiator could not find a match for the initiator task tag in the received PDU. Dump data contains the
entire iSCSI header.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
32 Information Initiator received an asynchronous logout message. The Target name is given in the dump data.
33 Error Challenge size given by the target exceeds the maximum specified in iSCSI specification.
34 Information
A connection to the target was lost, but Initiator successfully reconnected to the target. Dump data
contains the target name.
35 Error Target CHAP secret is smaller than the minimum size (12 bytes) required by the specification.
36 Error
Initiator CHAP secret is smaller than the minimum size (12 bytes) required by the specification.
Dump data contains the given CHAP secret.
37 Error FIPS service could not be initialized. Persistent logons will not be processed.
38 Error Initiator requires CHAP for logon authentication, but target did not offer CHAP.
39 Error
Initiator sent a task management command to reset the target. The target name is given in the
dump data.
40 Error Target requires logon authentication via CHAP, but Initiator is not configured to perform CHAP.
41 Error Target did not send AuthMethod key during security negotiation phase.
42 Error
Target sent an invalid status sequence number for a connection. Dump data contains Expected Status
Sequence number followed by the given status sequence number.
43 Error Target failed to respond in time for a login request.
44 Error Target failed to respond in time for a logout request.
45 Error
Target failed to respond in time for a login request. This login request was for adding a new
connection to a session.
46 Error Target failed to respond in time for a SendTargets command.
47 Error Target failed to respond in time for a SCSI command sent through a WMI request.
48 Error Target failed to respond in time to a NOP request.
49 Error Target failed to respond in time to a Task Management request.
50 Error Target failed to respond in time to a Text Command sent to renegotiate iSCSI parameters.
51 Error
52 Error
53 Error
Target failed to respond in time to a logout request sent in response to an asynchronous message
from the target.
Initiator Service failed to respond in time to a request to configure IPSec resources for an iSCSI
connection.
Initiator Service failed to respond in time to a request to release IPSec resources allocated for an
iSCSI connection.
54 Error Initiator Service failed to respond in time to a request to encrypt or decrypt data.
55 Error Initiator failed to allocate resources to send data to target.
56 Error Initiator could not map an user virtual address to kernel virtual address resulting in I/O failure.
57 Error Initiator could not allocate required resources for processing a request resulting in I/O failure.
58 Error Initiator could not allocate a tag for processing a request resulting in I/O failure.
59 Error Target dropped the connection before the initiator could transition to Full Feature Phase.
60 Error
Target sent data in SCSI Response PDU instead of Data_IN PDU. Only Sense Data can be sent in
SCSI Response.
61 Error Target set DataPduInOrder to NO when initiator requested YES. Login will be failed.
62 Error Target set DataSequenceInOrder to NO when initiator requested YES. Login will be failed.
63 Error Cannot reset the target or LUN. Will attempt session recovery.
64 Information Attempt to bootstrap Windows using iSCSI NIC Boot (iBF).
65 Error Booting from iSCSI, but could not set any NIC in Paging Path.
66 Error Attempt to disable the Nagle Algorithm for iSCSI connection failed.
67 Information If Digest support selected for iSCSI Session, will use Processor support for Digest computation.
68 Error
After receiving an async logout from the target, attempt to relogin the session failed. Error status is
given in the dump data.
69 Error Attempt to recover an unexpected terminated session failed. Error status is given in the dump data.
70 Error
71 Information
Error occurred when processing iSCSI logon request. The request was not retried. Error status is
given in the dump data.
Initiator did not start a session recovery upon receiving the request. Dump data contains the error
status.
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
iSCSI Protocol: Broadcom NetXtreme II® Network Adapter User Guide
72 Error Unexpected target portal IP types. Dump data contains the expected IP type.
Please read all Restrictions and Disclaimers.
Back to Contents Page
file:///C|/Users/Nalina_N_S/Documents/NetXtremeII/English/iscsi.htm[9/5/2014 3:45:06 PM]
Loading...