Cisco Nexus B22 Design And Deployment Manual

Cisco Nexus B22 Blade Fabric

Extender for IBM

Design and Deployment Guide

Contents

Introduction .............................................................................................................................................................. 3

Network Diagram ...................................................................................................................................................... 3

Hardware Installation ............................................................................................................................................... 5

Fabric Extender Management Model ..................................................................................................................... 6

Fabric Connectivity Options ................................................................................................................................... 7

Statically Pinned Fabric Interface Connection ....................................................................................................... 7

Port Channel Fabric Interface Connection ............................................................................................................ 9

Configuring a Fabric Port Channel ...................................................................................................................... 10

Virtual Port Channel Connection .......................................................................................................................... 13

Configuring a vPC ................................................................................................................................................ 14

Server Network Teaming ....................................................................................................................................... 16

Creating Host-Side vPC for Server Links with LACP .......................................................................................... 17

Fibre Channel over Ethernet ................................................................................................................................. 21

Configuring FCoE ................................................................................................................................................ 21

Configuring the Cisco Nexus 5000 Series and B22 for IBM for FCoE ................................................................ 22

iSCSI Configuration ............................................................................................................................................... 27

Virtual Network Adapter Partitioning ................................................................................................................... 30

Debug Commands ................................................................................................................................................. 35

show fex ............................................................................................................................................................... 35

show fex detail ..................................................................................................................................................... 36

show interface brief .............................................................................................................................................. 37

show interface ethernet 191/1/1 .......................................................................................................................... 42

show vlan ............................................................................................................................................................. 43

show interface fex-fabric ...................................................................................................................................... 44

Cisco Nexus Configurations ................................................................................................................................. 45

Cisco Nexus 5000 Series Switch 1 Configuration ............................................................................................... 45

Cisco Nexus 5000 Series Switch 2 Configuration ............................................................................................... 56

Conclusion .............................................................................................................................................................. 70

For More Information ............................................................................................................................................. 70

Introduction

The Cisco Nexus® B22 Blade Fabric Extender for IBM® extends the Cisco Nexus switch fabric to the server edge. Logically, it behaves like a remote line card to a parent Cisco Nexus 5000 or 6000 Series Switch. The fabric extender and the parent Cisco Nexus 5000 or 6000 Series Switch together form a distributed modular system. The Cisco Nexus B22 for IBM forwards all traffic to the parent Cisco Nexus 5000 or 6000 Series Switch over eight 10 Gigabit Ethernet uplinks. Low-cost uplink connections of up to 10 meters can be made with copper Twinax cable, and longer connections of up to 100 meters can use the Cisco® 10-Gbps fabric extender transceiver (FET10G). Standard 10-Gbps optics such as short reach (SR), long reach (LR), and extended reach (ER) are also supported. Downlinks to each server are 10 Gigabit Ethernet and work with all Ethernet and converged network adapter (CNA) mezzanine cards, allowing customers a choice of Ethernet, Fibre Channel over Ethernet (FCoE), or Small Computer System Interface over IP (iSCSI) connections. Because the Cisco Nexus B22 for IBM is a transparent extension of a Cisco Nexus switch, traffic can be switched according to policies established by the Cisco Nexus switch using a single point of management.

The Cisco Nexus B22 for IBM provides the following benefits: Highly scalable, consistent server access: This distributed modular system creates a scalable server access

environment with no reliance on Spanning Tree Protocol and with consistent features and architecture between blade and rack servers.

Simplified operations: The availability of one single point of management and policy enforcement using upstream Cisco Nexus 5000 Series Switches eases the commissioning and decommissioning of blades through zero-touch installation and automatic configuration of fabric extenders.

Increased business benefits: Consolidation, reduced cabling, investment protection through feature inheritance from the parent switch, and the capability to add functions without the need for a major equipment upgrade of server-attached infrastructure all contribute to reduced operating expenses (OpEx) and capital expenditures (CapEx).

The Cisco Nexus B22 for IBM integrates into the I/O module slot of a third-party blade chassis, drawing both power and cooling from the blade chassis itself.

Network Diagram

Figure 1 presents a sample network topology that can be built using the Cisco Nexus B22 for IBM, 2000 Series Fabric Extenders, and 5000 or 6000 Series Switches. In this topology, the Cisco Nexus 5000 or 6000 Series Switch serves as the parent switch, performing all packet switching and policy enforcement for the entire distributed modular system. The Cisco Nexus switch also serves as the only point of management for both configuration and monitoring within the domain, making it simple to manage blade server and rack server connections together.

Figure 1: Cisco Nexus Virtual Chassis Topology

The Cisco Nexus switches, along with the Cisco Nexus 2000 Series and B22 for IBM, create a distributed modular system that unifies the data center architecture. Within this distributed modular system, both IBM Flex System® computing nodes and rack servers are managed identically. This approach allows the use of the same business and technical processes and procedures.

The left-most blade chassis in Figure 1 contains dual Cisco Nexus B22 for IBM fabric extenders. Each Cisco Nexus B22 for IBM is singlely attached to a parent Cisco Nexus 5500 platform switch, a connection mode referred to as straight-through mode. The fabric links can be either statically pinned or put into a Port Channel. This connection mode helps ensure that all data packets from a particular Cisco Nexus B22 for IBM enter the same parent Cisco Nexus switch. This approach may be necessary when certain types of traffic must be restricted to either the left or right Cisco Nexus 5500 platform switch: for instance, to maintain SAN A and SAN B separation. Also, in this example the connections to individual computing nodes are in active-standby mode, which helps ensure traffic flow consistency but does not make full use of the server network interface card (NIC) bandwidth.

The second IBM Flex System chassis from the left in Figure 1 improves on the first with the creation of an Ethernet virtual Port Channel (vPC) from the computing node to the Cisco Nexus parent switch. This vPC places the Ethernet portion of the NICs in an active-active configuration, giving increased bandwidth to each host. The FCoE portion of the CNA is also configured as active-active but maintains SAN A and SAN B separation because each virtual Fibre Channel (vFC) interface is bound to a particular link at the server. This configuration also achieves high availability through redundancy, and it can withstand a failure of a Cisco Nexus 5500 platform switch, a Cisco Nexus B22 for IBM, or any connecting cable. This topology is widely used in FCoE deployments.

The third blade chassis from the left in Figure 1 contains Cisco Nexus B22 for IBM fabric extenders that connect to both Cisco Nexus 5500 platform switches through vPC for redundancy. In this configuration, active-active load balancing using vPC from the blade server to the Cisco Nexus 5500 platform switch cannot be enabled. However, the servers can still be dual-homed with active-standby or active-active transmit-load-balancing (TLB) teaming. This topology is only for Ethernet traffic because SAN A and SAN B separation between the fabric extender and the parent switch is necessary.

The fourth blade chassis from the left in Figure 1 contains Cisco Nexus B22 for IBM fabric extenders that connect

Card

Connection Blades

LAN on motherboard (LoM) plus mezzanine card in slot 1

I/O module bays 1 and 2 Mezzanine card in slot 2

I/O module bays 3 and 4

Card

Connection Blades

Mezzanine 1 ports 1 to 4

I/O module bays 1 and 2

Mezzanine 2 ports 1 to 4

I/O module bays 3 and 4

Card

Connection Blades

Mezzanine 1 ports 1 to 4

I/O module bays 1 and 2

Mezzanine 2 ports 1 to 4

I/O module bays 3 and 4

Mezzanine 3 ports 1 to 4

I/O module bays 1 and 2

Mezzanine 4 ports 1 to 4

I/O module bays 3 and 4

Blade Chassis

Server Manager Firmware

IBM PureFlex™ System Model 8721HC1

DSA:9.41, IMM2:2.6, UEFI:1.31, and CMM: 2PET12E

to both Cisco Nexus 5500 platform switches with enhanced vPC (EvPC) technology. This configuration allows active-active load balancing from the fabric extenders and the computing nodes.

The last two configurations show how rack-mount servers can connect to the same Cisco Nexus parent switch using rack-mount Cisco Nexus 2000 Series Fabric Extenders. The topology for blade servers and rack-mount servers can be identical if desired.

Hardware Installation

Installation of the Cisco Nexus B22 for IBM in the rear of the blade server chassis is similar to the installation of other connection blades. The layout of the blade server chassis, as well as the server types and mezzanine cards used, determines the slots that should be populated with the Cisco Nexus B22 for IBM for 10 Gigabit Ethernet connectivity. Tables 1 through 3 summarize the typical options for servers using dual-port 10 Gigabit Ethernet devices.

Table 1 Mapping of Third-Party Half-Wide Server Dual-Port Mezzanine Card to I/O Module

Table 2 Mapping of Third-Party Half-Wide Server Quad-Port Mezzanine Card to I/O Module

Table 3 Mapping of Third-Party Full-Wide Server Quad-Port Mezzanine Card to I/O Module

After the Cisco Nexus B22 for IBM fabric extenders are installed, the chassis management module (CMM) should be updated to at least the minimum version shown in Table 4.

Table 4 Management Blade Minimum Firmware Versions

No configuration is required from the chassis MMB. Only the minimum CMM firmware is required to properly detect and enable the Cisco Nexus B22 for IBM in the blade chassis (Figure 2).

Figure 2: Cisco Nexus B22 for IBM Fabric Extenders as Seen in the CMM

Fabric Extender Management Model

The Cisco Nexus fabric extenders are managed by a parent switch through the fabric interfaces using a zerotouch configuration model. The switch discovers the fabric extender by using a detection protocol.

After discovery, if the fabric extender has been correctly associated with the parent switch, the following operations are performed:

1. The switch checks the software image compatibility and upgrades the fabric extender if necessary.

2. The switch and fabric extender establish in-band IP connectivity with each other. The switch assigns an IP address in the range of loopback addresses (127.15.1.0/24) to the fabric extender to avoid conflicts with IP addresses that might be in use on the network.

3. The switch pushes the configuration data to the fabric extender. The fabric extender does not store any configuration locally.

4. The fabric extender updates the switch with its operating status. All fabric extender information is displayed using the switch commands for monitoring and troubleshooting.

This management model allows fabric extender modules to be added without adding management points or complexity. Software image and configuration management is also handled automatically, without the need for user intervention.

Fabric Connectivity Options

Interface

Fabric Link

1, 2, 3, 4, 5, 6, and 7

Fabric link 1

8, 9, 10, 11, 12, 13, and 14

Fabric link 2

The Cisco Nexus B22 for IBM creates a distributed, modular chassis with the Cisco Nexus parent switch after a fabric connection has been made over standard 10-Gbps cabling. This connection can be accomplished using any of the following types of interconnects:

 Cisco passive direct-attach cables (1m, 3m, or 5m)  Cisco active direct-attach cables (7m or 10m)  Cisco standard Enhanced Small Form-Factor Pluggable (SFP+) optics (SR, LR, and ER)  Cisco Fabric Extender Transceivers (FET modules)

After the fabric links have been physically established, the logical configuration of the links must be established. The fabric links to the Cisco Nexus B22 for IBM can use either of two connection methods:

 Statically pinned fabric interface connection  Port Channel fabric interface connection

Statically Pinned Fabric Interface Connection

Static pinning is the default method of connection between the fabric extender and the Cisco Nexus parent switch. In this mode of operation, a deterministic relationship exists between the host interfaces and the upstream parent; up to eight fabric interfaces can be connected. These fabric interfaces are equally divided among the 16 serverside host ports. If fewer fabric ports are allocated, more server ports are assigned to a single fabric link. The advantage of this configuration is that the traffic path and the amount of allocated bandwidth are always known for a particular set of servers.

Since static pinning will group host-side ports into individual fabric links, you should understand how ports are grouped. The size of the port groups is determined by the number of host ports divided by the max link parameter value. For example, if the max link parameter is set to 2, eight host ports would be assigned to each link. The interfaces will be grouped in ascending order starting from interface 1. Thus, interfaces 1 to 8 will be pinned to one fabric link, and interfaces 9 to 16 will be pinned to a different interface (Table 5).

Table 5 Interface Assignment with Two Fabric Links

Table 6 summarizes the assignment with four fabric links: With the max link parameter set to 4, the interfaces are

Interface

Fabric Link

1, 2, 3, and 4

Fabric link 1

5, 6, 7, and 8

Fabric link 2

9, 10, and 11

Fabric link 3

12, 13, and 14

Fabric link 4

Interface

Fabric Link

1 and 2

Fabric link 1

3 and 4

Fabric link 2

5 and 6

Fabric link 3

7 and 8

Fabric link 4

9 and 10

Fabric link 5

11 and 12

Fabric link 6

Fabric link 7

Fabric link 8

divided into four groups. Table 6 Interface Assignment with Four Fabric Links

Table 7 summarizes the assignment of eight fabric links: With the max link parameter set to 8, the interfaces are divided into eight groups.

Table 7 Interface Assignment with Eight Fabric Links

Note: The assignment of the host-side ports is always based on the configured max link parameter and not the actual

physical number of fabric ports connected. Be sure to match the max link parameter with the actual number of physical links used.

The relationship of host-side ports to parent switch fabric ports is static. If a fabric interface fails, all its associated host interfaces are brought down and will remain down until the fabric interface is restored. Figure 3 shows static port mappings.

Figure 3: Static Port Mapping Based on Max Link Parameter

Port Channel Fabric Interface Connection

The Port Channel fabric interface provides an alternative way of connecting the parent switch and the Cisco Nexus B22 for IBM fabric extender. In this mode of operation, the physical fabric links are bundled into a single logical channel. This approach prevents a single fabric interconnect link loss from disrupting traffic to any one server. The total bandwidth of the logical channel is shared by all the servers, and traffic is spread across the members through the use of a hash algorithm.

 For a Layer 2 frame, the switch uses the source and destination MAC addresses.  For a Layer 3 frame, the switch uses the source and destination MAC addresses and the source and

destination IP addresses.

Since both redundancy and increased bandwidth are possible, configuration of the fabric links on a Port Channel is the most popular connection option.

Figure 4 shows Port Channel designs.

Note: A fabric interface that fails in the Port Channel does not trigger a change to the host interfaces. Traffic is automatically

redistributed across the remaining links in the Port Channel fabric interface.

Figure 4: Port Channel Designs

Configuring a Fabric Port Channel

Follow these steps to configure a fabric Port Channel.

1. Log into the first parent switch and enter configuration mode:

Nexus 5000 Switch

Password:

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at

http://www.opensource.org/licenses/gpl-2.0.php and

http://www.opensource.org/licenses/lgpl-2.1.php

N5548-Bottom# configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

N5548-Bottom(config)#

2. Enable the fabric extender feature:

N5548-Bottom(config)#

N5548-Bottom(config)# feature fex

N5548-Bottom(config)#

3. Logically create the fabric extender:

N5548-Bottom(config)#

N5548-Bottom(config)# fex 191

N5548-Bottom(config-fex)#

4. Create the Port Channel, change the port mode, and associate the fabric extender with the Port Channel:

N5548-Bottom(config-if)# interface port-channel 191

N5548-Bottom(config-if)# switchport mode fex-fabric

N5548-Bottom(config-if)# fex associate 191

N5548-Bottom(config-if)#

5. Assign the Cisco Nexus parent switch ports to the Port Channel:

N5548-Bottom(config-if)# interface ethernet 1/17

N5548-Bottom(config-if)# switchport mode fex-fabric

N5548-Bottom(config-if)# fex associate 191

N5548-Bottom(config-if)# channel-group 191

N5548-Bottom(config-if)# interface ethernet 1/18

N5548-Bottom(config-if)# switchport mode fex-fabric

N5548-Bottom(config-if)# fex associate 191

N5548-Bottom(config-if)# channel-group 191

6. Repeat the steps on the second Cisco Nexus 5000 Series Switch connected to the fabric extender in interconnect bay 4:

N5548-Top# configure terminal

N5548-Top(config)# feature fex

N5548-Top(config)# fex 192

N5548-Top(config-if)# interface port-channel 192

N5548-Top(config-if)# switchport mode fex-fabric

N5548-Top(config-if)# fex associate 192

N5548-Top(config-if)# interface ethernet 1/17

N5548-Top(config-if)# switchport mode fex-fabric

N5548-Top(config-if)# fex associate 192

N5548-Top(config-if)# channel-group 192

N5548-Top(config-if)# interface ethernet 1/18

N5548-Top(config-if)# switchport mode fex-fabric

N5548-Top(config-if)# fex associate 192

N5548-Top(config-if)# channel-group 192

7. Verify that the Cisco Nexus B22 for IBM is up and running:

switch(config-if)# show fex

FEX FEX FEX FEX

Number Description State Model Serial

------------------------------------------------------------------------

192 FEX0192 Online N2K-B22IBM-P FOC1730R0XQ

A pair of fabric extenders now is configured in straight-through mode, also known as a single-attached configuration, and each is communicating with its respective Cisco Nexus switch. The links between the two Cisco Nexus switches and the Cisco Nexus B22 fabric extenders use Port Channels for connectitivity.

Virtual Port Channel Connection

vPCs allow links that are physically connected to two different Cisco Nexus switches to form a Port Channel to a downstream device. The downstream device can be a switch, a server, or any other networking device that supports IEEE 802.3ad Port Channels. vPC technology enables networks to be designed with multiple links for redundancy while also allowing those links to connect to different endpoints for added resiliency (Figure 5).

More information about vPC technology can be found at

http://www.cisco.com/en/US/products/ps9670/products_implementation_design_guides_list.html.

Figure 5: Blade Server Configuration Options

Configuring a vPC

The high-level steps for enabling vPC are listed here. This configuration should be implemented on both switches in parallel.

1. Enable the vPC feature.

2. Create the vPC domain.

3. Configure the peer keepalive link.

4. Configure the vPC peer link Port Channel.

5. Check the status of the vPC.

1. Enable the vPC feature:

N5548-Bottom# configure terminal

N5548-Bottom(config)# feature vpc

N5548-Top# configure terminal

N5548-Top(config)# feature vpc

2. Create the vPC domain (the domain should be unique within network):

N5548-Bottom(config)# vpc domain 5

N5548-Bottom(config)# role priority 1

N5548-Bottom(config)# system-priority 1

N5548-Top(config)# vpc domain 5

N5548-Top(config)# role priority 2

N5548-Top(config)# system-priority 1

3. Configure the peer keepalive link over the management network:

N5548-Bottom(config-vpc-domain)# peer-keepalive destination 172.25.182. 109 source

172.25.182. 108

Note:

--------:: Management VRF will be used as the default VRF ::--------

N5548-Top(config-vpc-domain)# peer-keepalive destination 172.25.182. 108 source

172.25.182. 109

Note:

--------:: Management VRF will be used as the default VRF ::--------

4. Configure the vPC peer link:

N5548-Bottom# interface port-channel 20

N5548-Bottom(config-if)# interface ethernet 1/9

N5548-Bottom(config-if)# channel-group 20

N5548-Bottom(config-if)# interface ethernet 1/10

N5548-Bottom(config-if)# channel-group 20

N5548-Bottom(config-if)# interface port-channel 20

N5548-Bottom(config-if)# vpc peer-link

Please note that spanning tree port type is changed to "network" port type on vPC peerlink.

This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance(which is enabled by default) is not disabled.

N5548-Bottom(config-if)#

N5548-Top# interface port-channel 20

N5548-Top (config-if)# interface ethernet 1/9

N5548-Top(config-if)# channel-group 20

N5548-Top(config-if)# interface ethernet 1/10

N5548-Top(config-if)# channel-group 20

N5548-Top(config-if)# interface port-channel 20

N5548-Top(config-if)# vpc peer-link

Please note that spanning tree port type is changed to "network" port type on vPC peerlink.

This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance(which is enabled by default) is not disabled.

N5548-Bottom(config-if)#

5. Check the vPC status:

N5548-Bottom(config-if)# show vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 5

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status: success

Per-vlan consistency status : success

Type-2 consistency status : success

vPC role : primary

Number of vPCs configured : 0

Peer Gateway : Disabled

Dual-active excluded VLANs : -

Graceful Consistency Check : Enabled

vPC Peer-link status

---------------------------------------------------------------------

id Port Status Active vlans

-- ---- ------ --------------------------------------------------

1 Po20 up 1,182

N5548-Bottom(config-if)#

Now the two switches have been configured to support vPC links to other devices. These connections can be used for upstream links to the data center core. These vPC links can be used for connections to hosts in the data center, allowing additional bandwidth and redundant links.

Server Network Teaming

Server NIC teaming provides an additional layer of redundancy to servers. It makes it possible for multiple links to be available, for redundancy. In the blade server environment, server network teaming typically is limited to active-standby configurations and cannot provide active-active links, because active-active links require an EtherChannel or Link Aggregation Control Protocol (LACP) connection to a single switch. However, because the Cisco Nexus B22 for IBM fabric extender is an extension of the parent switch, EtherChannel or LACP connections can be created between the blade server and the virtual chassis. Dual Cisco Nexus switches can be used with vPC for additional switch redundancy while providing active-active links to servers, thus enabling aggregate 40Gbps bandwidth with dual links (Figure 6).

Figure 6: Fabric Link and Server Topologies

Figure 7: Enhanced vPC Configuration

Creating Host-Side vPC for Server Links with LACP

1. Enable LACP on both parent switches.

5548-Bottom(config)# feature lacp

5548-Top(config)# feature lacp

2. Define and configure the left diagram FEX 191 for enhanced vPC on the left Nexus parent

5548-Bottom(config)# fex 191

5548-Bottom(config-fex)# interface ethernet 1/21-22

5548-Bottom(config-if)# channel-group 191

5548-Bottom(config-if)# no shutdown

5548-Bottom(config-if)# interface port-channel 191

5548-Bottom(config-if)# switchport mode fex-fabric

5548-Bottom(config-if)# fex associate 191

5548-Bottom(config-if)# vpc 191

5548-Bottom(config-if)# no shutdown

3. Define and configure the right diagram FEX 192 for enhanced vPC on the left Nexus parent

5548-Bottom(config)# fex 192

5548-Bottom(config-fex)# interface ethernet 1/23-24

5548-Bottom(config-if)# channel-group 192

5548-Bottom(config-if)# no shutdown

5548-Bottom(config-if)# interface port-channel 192

5548-Bottom(config-if)# switchport mode fex-fabric

5548-Bottom(config-if)# fex associate 192

5548-Bottom(config-if)# vpc 192

5548-Bottom(config-if)# no shutdown

4. Define and configure the left diagram FEX 191 for enhanced vPC on the right Nexus parent

5548-Top(config)# fex 191

5548-Top(config-fex)# interface ethernet 1/21-22

5548-Top(config-if)# channel-group 191

5548-Top(config-if)# no shutdown

5548-Top(config-if)# interface port-channel 191

5548-Top(config-if)# switchport mode fex-fabric

5548-Top(config-if)# fex associate 191

5548-Top(config-if)# vpc 191

5548-Top(config-if)# no shutdown

5. Define and configure the right diagram FEX 192 for enhanced vPC on the right Nexus parent

5548-Top(config)# fex 192

5548-Top(config-fex)# interface ethernet 1/23-24

5548-Top(config-if)# channel-group 192

5548-Top(config-if)# no shutdown

5548-Top(config-if)# interface port-channel 192

5548-Top(config-if)# switchport mode fex-fabric

5548-Top(config-if)# fex associate 192

5548-Top(config-if)# vpc 192

5548-Top(config-if)# no shutdown

6. Create the port channel between the blade server and the FEX

5548-Bottom# configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

5548-Bottom#(config)# interface port-channel 201

5548-Bottom#(config-if)# switchport mode trunk

5548-Bottom#(config-if)# switchport trunk native vlan 182

5548-Bottom#(config-if)# switchport trunk allowed vlan 182-184,200

5548-Bottom#(config-if)# no shutdown

5548-Top# configure terminal

5548-Top#(config)# interface port-channel 201

5548-Top#(config)# switchport mode trunk

5548-Top#(config)# switchport trunk native vlan 182

5548-Top#(config)# switchport trunk allowed vlan 182-182,201

5548-Top#(config)# no shutdown

7. Add the member interfaces to the vPC port channel and permit VLAN or desired VLANs for L2 Trunk links.

Note: If a Native VLAN besides 1 is desired ensure it is correctly defined before use.

5548-Bottom(config-if)# interface ethernet 191/1/7

5548-Bottom(config-if)# switchport mode trunk

5548-Bottom(config-if)# switchport trunk native vlan 182

5548-Bottom(config-if)# switchport trunk allowed vlan 182-184,200

5548-Bottom(config-if)# channel-group 201 mode active

5548-Bottom(config-if)# no shutdown

5548-Bottom(config-if)# interface ethernet 192/1/7

5548-Bottom(config-if)# switchport mode trunk

5548-Bottom(config-if)# switchport trunk native vlan 182

5548-Bottom(config-if)# switchport trunk allowed vlan 182-184,200

5548-Bottom(config-if)# channel-group 201 mode active

5548-Bottom(config-if)# no shutdown

8. Repeat the vlan configuration on the second Nexus parent switch for the vPC port channel and permit VLAN or desired VLANs for L2 Trunk links.

5548-Top(config-if)# interface ethernet 191/1/7

5548-Top(config-if)# switchport mode trunk

5548-Top(config-if)# switchport trunk native vlan 182

5548-Top(config-if)# switchport trunk allowed vlan 182-184,201

5548-Top(config-if)# channel-group 201 mode active

5548-Top(config-if)# no shutdown

5548-Top(config-if)# interface ethernet 192/1/7

5548-Top(config-if)# switchport mode trunk

5548-Top(config-if)# switchport trunk native vlan 182

5548-Top(config-if)# switchport trunk allowed vlan 182-184,201

5548-Top(config-if)# channel-group 201 mode active

5548-Top(config-if)# no shutdown

Note: With EvPC when you configure a Port Channel from the Cisco Nexus 2000 Series to the server, do not include the

vpc x configuration under the Port Channel. vPC should be assigned automatically by the Cisco NX-OS Software. For more information, see

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/layer2/6x/b_5500_Layer2_Config_602N12_cha pter_0101.html.

To verify that the vPC is formed, go to one of the Cisco Nexus switches to check the status of the server Port Channel interface. The pair of Cisco Nexus switches is in a vPC configuration, so each has a single port in the Port Channel. A check of the status of the Port Channel on each parent switch shows that channel group 201 is in the “P - Up in port-channel” state on each switch. A check from the OneCommand utility will show that the status is “Active” for each link that is up in the Port Channel.

5548-Bottom# show port-channel summary

Flags: D - Down P - Up in port-channel (members)

I - Individual H - Hot-standby (LACP only)

s - Suspended r - Module-removed

S - Switched R - Routed

U - Up (port-channel)

M - Not in use. Min-links not met

--------------------------------------------------------------------------------

Group Port- Type Protocol Member Ports

Channel

--------------------------------------------------------------------------------

20 Po20(SU) Eth NONE Eth1/9(P) Eth1/10(D)

191 Po191(SU) Eth NONE Eth1/17(P)

192 Po192(SU) Eth NONE Eth1/18(P)

193 Po193(SU) Eth NONE Eth1/19(P)

194 Po194(SD) Eth NONE Eth1/20(D)

201 Po201(SU) Eth NONE Eth191/1/1(P)

202 Po202(SU) Eth NONE Eth192/1/1(P)

5548-Bottom #

N5548-Top# show port-channel summary

show port-channel summary

Flags: D - Down P - Up in port-channel (members)

I - Individual H - Hot-standby (LACP only)

s - Suspended r - Module-removed

S - Switched R - Routed

U - Up (port-channel)

M - Not in use. Min-links not met

--------------------------------------------------------------------------------

Group Port- Type Protocol Member Ports

Channel

--------------------------------------------------------------------------------

10 Po10(SD) Eth NONE --

20 Po20(SU) Eth NONE Eth1/9(P) Eth1/10(D)

61 Po61(SU) Eth NONE Eth1/5(P) Eth1/6(P)

191 Po191(SU) Eth NONE Eth1/17(P)

192 Po192(SU) Eth NONE Eth1/18(P)

193 Po193(SU) Eth NONE Eth1/19(P)

194 Po194(SD) Eth NONE Eth1/20(D)

201 Po201(SU) Eth NONE Eth191/1/1(P)

202 Po202(SU) Eth NONE Eth192/1/1(P)

N5548-Top#

Fibre Channel over Ethernet

FCoE combines LAN and storage traffic on a single link, eliminating the need for dedicated adapters, cables, and devices for each type of network, resulting in savings that can extend the life of the data center. The Cisco Nexus B22 for IBM is the building block that enables FCoE traffic to travel outside the blade chassis.

Best practices for unified fabric are listed in the Cisco NX-OS operations guide for the Cisco Nexus 5000 Series at

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/operations/n5k_ops_guide.html.

Configuring FCoE

Follow these steps to configure FCoE:

1. Enable the FCoE personality on the CNA.

2. Verify and, if necessary, install the FCoE drivers in the server OS.

3. Enable FCoE on the parent switches.

4. Configure quality of service (QoS) to support FCoE on the Cisco Nexus parent switch.

5. Enable the FCoE feature on the Cisco Nexus switch.

6. Create the SAN A and SAN B VLANs.

7. Create vFC interfaces.

1. Enable FCoE on the CNA.

The CNA personality should be set to FCoE according to the CNA documentation.

2. Verify and, if necessary, install the FCoE drivers in the server OS.

Verify that the latest FCoE drivers and firmware are loaded for the operating system. The latest versions can be obtained from the third-party support website. The FCoE drivers are separate from the Ethernet NIC drivers. Generally, the latest versions of the CNA drivers and the CNA firmware should be used.

+ 49 hidden pages

Cisco Nexus B22 Design And Deployment Manual

Specifications and Main Features

Frequently Asked Questions

User Manual