For more information.......................................................................................................................... 29
Abstract
This HOWTO provides best practice guidelines and configuration examples for installation of the
ProLiant BL p-Class GbE2 Interconnect Switch into a Cisco-based network. This guide is meant to be a
tool to help direct decisions in planning, optimization, and securing the GbE2 Interconnect Switch
environment. While the best practices and configurations examples in this document could be used in
real world environments, they are to be used only as guidelines. This HOWTO does not serve as a
replacement for the GbE2 Interconnect Switch user guides; rather it is meant to serve as a supplement
to this documentation.
The intended audience for this paper includes engineers and system administrators familiar with the
ProLiant BL p-Class GbE2 Interconnect Switch. For readers not familiar with GbE2 Interconnect Switch,
please see the ProLiant BL p-Class GbE2 Interconnect Switch Overview white paper as well as the user
documentation that shipped with the GbE2 Interconnect Switch. To obtain these documents, go to the
HP website (http://www.hp.com/support
), and search for GbE2.
Introduction
This HOWTO identifies best practice guidelines and configuration examples for installation of the
ProLiant BL p-Class GbE2 Interconnect Switch into a Cisco-based network consisting of redundant
Catalyst 6509 switches with the Catalyst switch operating system (CatOS). However, the examples in
this document can be used as general guidelines appropriate for network infrastructures consisting of
other Cisco switches, with the CatOS or Internetwork Operating System Software (IOS), and network
devices from other vendors including Nortel, Extreme, Foundry, 3Com, etc.
The GbE2 Interconnect Switch is intended for applications that require up to 1000 megabits per
second (Mb/s) Gigabit Ethernet network adapter (NIC) consolidation, advanced network feature
support (including future planned options for layer 3 and 4-7 switching), server blade Fibre Channel
pass-through, and future upgradeability for 10 Gigabit Ethernet bandwidth connectivity to the
network. For additional information on the GbE2 Interconnect Switch, please see the ProLiant BL p-Class GbE2 Interconnect Switch Overview white paper.
For best practice guidelines for the entire p-Class system, see the HP ProLiant BL System Best Practices Guide and the HP ProLiant BL System Common Procedures Guide.
Terminology
The terminology that differs between the Cisco Catalyst 6509 switch and the GbE2 Interconnect
Switch documentation is identified in Table 1.
Table 1. Network terminology cross reference
HP ProLiant GbE2 Interconnect Switch Cisco Catalyst 6509 Switch
VLAN tagging, 802.1Q tagging trunking, VLAN or 802.1Q encapsulation
port VLAN identification (PVID) VLAN identification (VLANID)
link aggregation, multi-link trunking (MLT) EtherChannel, channeling
spanning tree protocol group (STG) spanning tree instance
IEEE 802.1d, Spanning Tree Protocol per VLAN Spanning Tree Plus (PVST+)
3
Typographical conventions
The following table describes the switch command typographic styles used in this guide:
Table 2. Switch command typographical conventions
HP typeface
AaBbCc123
<AaBbCc12
3>
To distinguish between ProLiant BL p-Class GbE2 Interconnect Switch and Catalyst 6509 commands,
each command will be preceded by a GbE2>> and 6509#, respectively.
MeaningExample
This type displays in command examples and
shows text that must be typed in exactly as shown.
This italicized type displays in command examples
as a parameter placeholder. Replace the
indicated text with the appropriate real name or
value when using the command. Do not type the
brackets.
/cfg/vlan
/cfg/vlan <vlan number>
Critical features for successful deployment
Understanding VLANs and VLAN tagging (VLAN trunking), spanning tree protocol, and multi-link
trunking (channeling) is critical to the successful deployment of the GbE2 Interconnect switch. Each of
these topics is covered providing a high-level primer inclusive of GbE2 Interconnect Switch command
introduction and general configuration guidelines. Specific commands and configuration steps follow
in the section titled “Common topological examples”. For additional information, refer to the HP ProLiant BL p-Class GbE2 Interconnect Switch Application Guide.
Virtual local area network
A virtual local area network (VLAN) is a network topology configured according to a logical scheme
rather than the physical layout. VLANs are used to logically segment traffic into different broadcast
domains allowing packets to be forwarded only between ports within the VLAN. This enhances
performance by conserving bandwidth and improves security by limiting traffic to specific domains.
The standard practice of configuring VLANs on an Ethernet switch is by assigning each port to a
specific VLAN. In this port-based VLAN implementation, the switch identifies the specific VLAN
membership of a packet per the port on which it was received. Individual VLANs are defined via a
configurable VLAN number. The VLAN number is known as port VLAN identification (PVID) on GbE2
Interconnect Switches and VLAN identification (VLANID) on Cisco Catalyst switches. The GbE2
Interconnect Switch allows any PVID value from 2 to 4095 with PVID 1 reserved as the default VLAN.
The default GbE2 Interconnect switch configuration has all ports assigned to PVID 1.
The IEEE industry standard for VLANs is 802.1Q. Each GbE2 Interconnect Switch supports 255
port-based IEEE 802.1Q VLANs. The GbE2 Interconnect Switch VLAN menu can be found under:
GbE2>> /cfg/vlan <vlan number>
VLAN tagging
VLAN tagging (often called VLAN trunking or encapsulation by Cisco) is the process of inserting into
a data frame a tag identifying its VLAN membership. VLAN tagging allows each switch port to
belong to multiple VLANs and provides the information switches need to create VLANs across the
network.
Switch ports may be configured as tagged or untagged. A tagged port may receive tagged or
untagged frames and is capable of forwarding the frames appropriately. When a VLAN tagged
frame arrives at a tagged port, the switch looks at the PVID in the tag to determine its VLAN
4
membership before switching the packet to the correct port. If an untagged frame arrives on a tagged
port, the switch will tag the frame with the PVID of that port. If a frame exits the switch via a tagged
port, any tag will remain on the frame unchanged as it exits.
An untagged port is only capable of switching untagged frames. Therefore, an untagged port will
only see and accept incoming untagged frames. Frames received by the untagged port will be
forwarded without any changes to the frame. For frames exiting the switch via an untagged port, any
tag will be stripped from the frame before its forwarded.
GbE2 Interconnect Switch ports may be individually configured as tagged or untagged using the
following command:
GbE2>> /cfg/port <port number>/tag ena
When implementing VLAN tagging on the GbE2 Interconnect Switch, the PVID values must be
established correctly between devices communicating in the VLAN. This option is found under:
GbE2>> /cfg/port <port number>/pvid <PVID number>
IP management interface
The IP management interface provides management access to the GbE2 Interconnect Switch over an
IP network. By default, the IP management interface is configured to request its IP address from a
bootstrap protocol (BOOTP) server, but the IP address may also be assigned manually resulting in
BOOTP being disabled.
Carefully consider how VLANs are configured within the GbE2 Interconnect Switch to ensure remote
communication to the switch remains possible. In order to access the GbE2 Interconnect Switch for
remote configuration, SNMP trap messages, and other remote management functions, confirm at least
one IP management interface on the switch has a VLAN defined.
It is possible to inadvertently disable access to management functions if the port associated with the IP
management interface is excluded from VLAN membership. Likewise, if all IP interfaces remain within
the default VLAN (VLAN 1) and all ports are configured for a different VLAN, such as VLAN 2, then
GbE2 Interconnect Switch management features are effectively disabled. To avoid these situations, it
is suggested that all ports used for remote GbE2 Interconnect Switch management remain on the
default VLAN and that an IP management interface be assigned to the default VLAN.
On the GbE2 Interconnect Switch, assign the IP management interface to a VLAN using the
commands:
Spanning tree protocol (STP) is used to ensure that redundant paths within a layer 2 network do not
result in broadcast loops. For a layer 2 Ethernet network to function correctly, only one active path
may forward frames between any two switches at a given time.
Redundant connections between network switches can create loops or multiple forwarding paths. In
layer 2 networks, these loops cause duplicate packets to be forwarded to the same destination over
and over again until the network is completely saturated, which in turn prevents valid traffic from
traversing the network. STP configures the network by allowing a switch to use the most efficient path
while forcing the remaining redundant paths into a standby (blocked) state. If the forwarding path
fails, STP automatically activates a standby path to sustain network operations.
Spanning tree groups
STP examines the network topology and defines a tree structure spanning all switches in a given layer
2 network domain. These layer 2 network domains are called spanning tree groups (STG). STGs are
5
created by assigning a group of layer 2 switches to be part of a separate layer 2 network domain.
When STP examines the network topology it only considers eliminating loops within a single STG.
Within a layer 2 domain, there may be multiple STGs each operating its own individual STP
algorithm to eliminate layer 2 loops.
The IEEE industry standard for STP is defined in 802.1D. The GbE2 Interconnect Switch meets the IEEE
802.1D standard and further provides interoperability with Cisco’s Per VLAN Spanning Tree Plus
(PVST+) via the use of STGs; refer to the “Multiple spanning tree groups” section for more information
on PVST+.
NOTE: The GbE2 Interconnect Switch does not support Cisco’s Per VLAN Spanning Tree (PVST),
only Per VLAN Spanning Tree Plus (PVST+). Interoperability with Cisco’s proprietary
MSTP/RSTP implementation is not supported.
Bridging protocol data unit
All network devices that are members of a spanning tree send out packets called bridging protocol
data units (BPDU). A BPDU is a 64-byte packet sent by all switches participating in the spanning tree
protocol providing information about each other. The BPDU includes information known as switch or
bridge priority, port cost, and port priority used to establish a spanning tree root switch and which
paths to designate as forwarding and blocking.
Root bridge
The STP root switch (or root bridge) is the base of the spanning tree topology much like the roots of a
tree. All redundant paths to the root bridge within the spanning tree network are placed in the
blocked mode. The root bridge is chosen by all the switches based on the results of the BPDU
exchange process.
Bridge priority
The bridge priority is used to determine what switch is the root bridge. Bridge priority is a numerical
value that may be configured on a switch. The lower a bridges priority value, the greater the chance
it has of becoming the root bridge. If all switches are configured with the same default bridge priority
setting, the switch with the lowest MAC address in the STP network becomes the root switch. Bridge
priority is automatically assigned by the STP process, or may be manually configured on the GbE2
Interconnect Switch using the following command:
The port cost is a value assigned to each switch port. The port cost information is exchanged within
the BPDU to help determine the lowest cost path to the root switch. The port with the lowest cost path
is used as the forwarding port between two segments in the STG. All remaining paths within each
segment are placed in a blocked state.
The objective is to use the fastest links ensuring the route with the lowest cost is chosen. The spanning
tree protocol assigns lower values to high-bandwidth ports, such as Gigabit Ethernet, to encourage
their use. The cost of a port also depends on whether the port operates at full-duplex (lower cost) or
half-duplex (higher cost). For example, a 100-Mb/s (Fast Ethernet) link has a STP assigned “cost” of
10 in half-duplex mode, and a cost of 5 in full-duplex mode. Port cost is automatically assigned by the
STP process, or manually set on the GbE2 Interconnect Switch using the following command:
The port priority is yet another STP value assigned to each switch port. In case of identical port costs,
the port priority is used as a tie breaker to determine the lowest path cost to the root switch and the
resulting forwarding port for each segment. Therefore, in a network topology segment that has
multiple paths with the same port cost, the port with the lowest port priority becomes the designated
6
port for the segment. It is also possible for the ports to have identical port priorities. If this is the case,
the port number becomes the final decision criteria. Port priority is automatically assigned by the STP
process, or manually set on the GbE2 Interconnect Switch using the following command:
The IEEE 802.1D standard considers the network topology of all the switches participating in the
spanning tree network as one broadcast domain or one spanning tree group (STG). It does not
consider the logical VLAN implementation. Ports within different VLANs are logically separated
broadcast domains. With the 802.1D implementation, paths that form physical loops within the
network may be placed in a blocking state even though the VLAN topology would have not caused a
layer 2 broadcast storm.
To prevent this, the IEEE standard 802.1s was adopted as an extension to the original 802.1D
standard. It allows multiple STGs within a network switch taking into consideration the VLAN logical
topology. Forwarding and blocking decisions are now made according to the BPDU information
within its own broadcast domain. IEEE 802.1s utilizes the 802.1Q VLAN tagging method in its
implementation. Prior to the adoption of 802.1s, Cisco developed a similar protocol known as Per
VLAN Spanning Tree (PVST). PVST uses the Cisco proprietary Intra Switch Link (ISL) method of VLAN
tagging. A more recent update to the protocol known as PVST+ provides the same functionality as
PVST, but utilizes the 802.1Q VLAN tagging method.
The GbE2 Interconnect Switch integrates into a PVST+ environment through the use of STGs. In the
GbE2 implementation, an administrator creates an STG and then assigns a VLAN to it. This differs
from the Cisco implementation where an administrator creates a VLAN and then a spanning tree
instance (i.e. STG) is automatically assigned to it. The PVST+ interoperability feature on the GbE2
Interconnect Switch includes the following:
• Tagged ports may belong to more than one STG, but untagged ports can belong to only one STG.
• When a tagged port belongs to more than one STG, egress BPDUs are tagged to identify their STG
membership.
• An untagged port cannot span multiple STGs.
• Sixteen STGs are supported per GbE2 Interconnect Switch.
• The default STG 1 can hold multiple VLANs, all other STGs (groups 2–16) can hold one VLAN.
On each GbE2 Interconnect Switch, the six external ports (ports19-24) and the crosslink ports (ports
17-18) are by default in STG 1. The STG can be changed for each port using the following
command:
GbE2>> /cfg/stp <stg number>/port <port number>
VLAN and STG configuration guidelines
When creating a VLAN on the GbE2 Interconnect Switch, that VLAN automatically belongs to the
default STG 1. To add the VLAN in another STG, it must be assigned to another STG. Keep the
following rules in mind when creating VLANs and assigning STGs:
• The default VLAN (VLAN 1) cannot be removed from the default STG 1.
• VLANs must be contained within a single STG; a VLAN cannot span multiple STGs.
• When a VLAN spans multiple switches, the VLAN must be within the same STG (have the same STG
ID) across all the switches.
• If ports are tagged, all trunked ports can belong to multiple STGs.
• A port that is not a member of any VLAN cannot be added to a STG. The port must be added to a
VLAN, and that VLAN added to the desired STG.
7
• Tagged ports can belong to more than one STG, but untagged ports can belong to only one STG.
• When a tagged port belongs to more than one STG, the egress BPDUs are tagged to distinguish the
BPDUs of one STG from those of another STG.
• An untagged port cannot span multiple STGs.
• When a port is removed from a VLAN that belongs to an STG, that port will also be removed from
the STG. However, if that port belongs to another VLAN in the same STG, the port remains in the
STG.
• An STG cannot be deleted, only disabled. If you disable the STG while it contains VLAN members,
STP will be off on all ports belonging to that VLAN.
• If any port in a trunk is set to forwarding (STP), the remaining ports in the trunk will also be set to
forwarding.
Multi-link trunking
Multi-link trunking (MLT), also know as link aggregation and port trunking (and EtherChannel by
Cisco), combines multiple physical switch ports into a single logical port called a trunk. The
bandwidth of the trunk is the multiple of the bandwidth of the individual links. An algorithm
automatically applies load balancing to the ports in the trunk. A port failure within the group causes
the network traffic to be directed to the remaining ports. Load balancing is maintained whenever a
link in a trunk is lost or returned to service.
The industry standard for multi-link trunking is IEEE 802.3ad. Cisco has developed a similar multi-link
trunking method known as EtherChannel. The GbE2 Interconnect Switch supports twelve IEEE
802.3ad (without LACP
two to six ports providing a 12-Gbps aggregate throughput full duplex.
1
) trunks per switch interoperable with EtherChannel. Each trunk may contain
Load balancing
Within the trunk, the load distribution is determined by information embedded within the data frame.
For traffic that does not contain IP information, the GbE2 Interconnect Switch elects the port with the
lowest port number in the trunk to be the designated port for forwarding traffic. For traffic that
contains IP addresses, the GbE2 Interconnect Switch will calculate the designated trunk port for
forwarding traffic by using the statistical load balancing algorithm that considers the packet's source
and destination IP addresses.
Multi-link trunking and spanning tree
A typical network is designed with multiple links between switches to provide increased bandwidth
and redundant connections. In layer 2 networks, redundant links between switches create loops or
multiple forwarding paths resulting in broadcast storms. The spanning tree protocol will identify these
loops and place ports in a blocked state to eliminate the possibly of multiple forwarding paths.
However, this defeats the purpose of using multiple connects between switches for increased
bandwidth. MLT can be used to provide redundant links while ensuring that STP does not block this
redundancy. Within a multi-link trunk, all the individual ports are seen as one logical by the spanning
tree protocol.
1
Link aggregation control protocol (LACP) is an enhancement over EtherChannel and other static multi-link trunking methods. LACP dynamically
learns about the link status and makes decisions on which links to use for load balancing and failback in case of link failure. As a result, IEEE
802.3ad with LACP is often called dynamic trunking.
8
Multi-link trunking configuration guidelines
When creating trunks, consider the following configuration rules that determine how a trunk reacts in
the network topology.
• Confirm the GbE2 Interconnect Switch ports to be trunked are set to enabled.
• All trunks must originate from one device, and lead to one destination device. For example, it is not
possible to combine a port from two different switches into one trunk.
• Any physical switch port can belong to only one trunk.
• Trunking from non-HP devices must comply with Cisco EtherChannel technology.
• All ports within a trunk (trunk members) must be assigned to the same VLAN configuration before
the trunk can be enabled.
• All ports within the trunk must be configured to full duplex.
• If the VLAN settings of any one trunk member are modified, the change cannot be applied until the
VLAN settings of all trunk members are modified.
• When an active GbE2 Interconnect Switch port is configured in a trunk, the port becomes a trunk
member using the following trunk command:
The spanning tree parameters for the port will change to reflect the new trunk settings.
• All trunk members must be in the same STG. If all ports are tagged, then all the ports within trunk
can belong to multiple STGs; otherwise, only one STG membership is allowed.
• When a trunk is enabled, the spanning tree participation setting of the trunk takes precedence over
that of any individual trunk member.
• If the spanning tree protocol participation of any trunk member is changed to enabled or disabled,
the spanning tree participation of all members of that trunk changes similarly.
• A trunk member cannot be a monitoring port in a port mirroring configuration.
• Trunks act as a single logical port, but cannot be monitored by a monitor port; however, individual
trunk members can.
• The port speeds of each trunk member must be the same.
Uplink Failure Detection
Uplink Failure Detection (UFD) is designed to provide High Availability in “straight-through”
topologies. A straight through topology is one that does not provide any redundancy either through
STP or Virtual Router Redundancy Protocol (VRRP). Uplink Failure detection is designed to work with
Network Adapter Teaming on HP server blades.
For details about Network Adapter Teaming on HP ProLiant server blades, refer to the white paper at
the following location:
The main components of UFD are as follows:
• Uplinks (external ports)
• Downlinks (internal ports)
• Server network adapters (NICs)
When UFD is configured, it enables the switch to monitor uplink ports. Once the switch detects an
uplink failure or state change to blocking, it automatically disables the corresponding downlink ports.
The Network Adaptor Teaming driver detects that the downlink port has been disabled and triggers a