Dell M8024-k, 8024, 8024F User Manual

Authors

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024 and 8024F

A Dell Deployment Guide

Network Enabled Solutions Team, Kevin Locklear

Contributor Kili Land

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

This document is for informational purposes only and may contain typographical errors and technical inaccuracies. The content is p rovided as is, without express or implied warranties of any kind.

© 2011 - 2012 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions in typography or p h otography. Dell, the Dell logo, PowerConnect and PowerEdge are trademarks of Dell Inc. Intel and Xeon are registered trademarks of Intel Corporation in the U.S. and other countries. Microsoft, Windows, and Windows Server are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Other trademarks and trade names may be used in this document to refer to either the e n tities claiming the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others.

March 2012| Rev 2.0

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Contents

Introduction ........................................................................................................... 6

Basic Terminology .................................................................................................... 6

FIP snooping ........................................................................................................ 6

FIP snooping bridge (FSB) ........................................................................................ 6

FCF ................................................................................................................... 6

PFC ................................................................................................................... 7

NPIV .................................................................................................................. 7

NPV................................................................................................................... 7

VSAN ................................................................................................................. 7

Configuration scenarios ............................................................................................. 8

Important notes prior to deployment .......................................................................... 9

Scenario 1: Deploying the Dell PowerConnect 8024 Series FSB in a Cisco 5000 Series Switch (NPIV )

environment .......................................................................................................... 10

Configuring the Dell PowerConnect M8024-k,8024, and 8024F for FIP Snooping ....................... 13

Configuring the Cisco 5000 series switch with firmware ver 5. x for a single connection from the

Dell PowerConnect M8024-k or 8024()(F) ..................................................................... 17

Basic Validation for the Pow e r Connect M8024-k configuration ........................................... 23

Basic Troubleshooting Areas .................................................................................... 28

Scenario 2: Configuring Mult ip le Uplinks into LAG for Cisco Nexus 5000 Series Switch (NPIV)

Environment .......................................................................................................... 30

Configuring the Cisco Nexus 50 00 series switch with firmware ver 5.x for a multiple link LAG (link

aggregation) connection at the Top-of-Rack. ................................................................ 31

Configuring the M8024-k,8024, and 8024F for FIP Snoop in g ............................................... 33

Scenario 3: Configuring Mult ip le Uplinks into LAG for Cisco Nexus 5000 Series Switch (NPV mode)

Environment .......................................................................................................... 35

Configuring the Cisco Nexus 50 00 series switch with firmware ver 5.0(3)N2(2a) in NPV mode for a multiple link LAG (link aggregation) connection from the Dell PowerConnect M8024-k or 8024()(F)

...................................................................................................................... 36

Configuring the Dell PowerConnect M8024-k,8024,and 8024F for FIP Snooping with Cisco Nexus

5000 series switch in NPV mode. ............................................................................... 37

ETS Behavior and CoS configur ations on the PowerConnect 8024 series switches ..................... 37

Updating firmware .................................................................................................. 39

Command-line interface method ............................................................................... 39

Web interface method ........................................................................................... 41

Appendix A – Full CLI examples ................................................................................... 45

M8024-k CLI example ............................................................................................. 45

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Cisco Nexus 5548UP CLI example .............................................................................. 52

Appendix B - Network Switch Versions ........................................................................... 56

References ............................................................................................................ 56

About Dell ............................................................................................................ 56

Figures

Figure 1. Dell PowerConnect™ M8024-k Switch (10G Ethernet) .............................................. 6

Figure 2. Dell PowerConnect™ 8024F (10G Ethernet) ......................................................... 6

Figure 3. General overview of deployment ...................................................................... 8

Figure 4. Disabling simple mode .................................................................................. 9

Figure 5. Simple 1-link connection between devices ......................................................... 10

Figure 6. General Overview of the whole configuration and planning procedure ........................ 11

Figure 7. Overview of parallel configurati on ................................................................... 12

Figure 8. Example commands for Dell PowerConnect M8024-k (can be copied and pasted) ........... 13

Figure 9. Configuration overview of Dell PowerConnect M8024-k .......................................... 16

Figure 10. Sample CLI for Cisco Nexus 5020 ( can be copied and pasted) ................................... 17

Figure 11. Sample CLI for Cisco Nexus 5548 ( can be copied and pasted) ................................... 18

Figure 12. Cisco Nexus 5000 series configuration sequence ................................................... 20

Figure 13. M1000e Chassis Management Controller -> Server Overview -> Properties -> WWN/MAC

information for Blade Serv e r 3’ s “B” fabric CNA port 2 (B2). ............................................ 21

Figure 14. Example of show interface brief command ......................................................... 22

Figure 15. show spanning-tree summary co mmand showing current configuration with ports states .. 23

Figure 16. show flogi database comman d sh owing devices that have compl e ted fabric login .......... 23

Figure 17. Example of show zoneset active command ......................................................... 23

Figure 18. Show interface status results ......................................................................... 24

Figure 19. show spanning-tree blockedports command ........................................................ 25

Figure 20. Show fip-snooping command which gives a brief status on available ENode’s, and FCF’s .. 25

Figure 21. show lldp dcbx interface all ........................................................................... 26

Figure 22. Show lldp dcbx interface te1/0/20 detail .......................................................... 27

Figure 23. Multiple port link (LAG) configuration between switches and storage ......................... 30

Figure 24. Multiple port link (LAG) Cisco 5020 configuration (can be copied and pasted) ............... 31

Figure 25. Multiple port uplink (LAG) M8024-k configuration (can be copied and pasted) ............... 33

Figure 26. Multiple link configuration between switches and storage ....................................... 35

Figure 27. Multiple port link Cisco 5020 configuration (can be copied and pasted) ....................... 36

Figure 28. CoS settings to establish min imum bandwidth for FCoE qu e ue .................................. 37

Figure 29. Fabric separation as preferred method for management of networks and storage .......... 38

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Introduction

The PowerConnect™ M8024-k, 8024 and 8024F switches are now DCB/DCBx capable with a downloadable update. Starting with firmware 4.2, the latest PowerConnect™ 10 Gigabit switches can now be used as an FCoE Transit Switch (FIP Snooping Bridge, T11, BB-5). With this new firmware implementation Converged Network Adapters (CNAs) can be used in the rack-mount or blade server to enable access to Fibre Channel networks and their storage.

NOTE: The PowerConnect™ M8024 (predecessor to th e M 8024-k) does not support the FIP Snooping

capability and will not be suppo r t e d for any of the described scenarios. This document provides an easy t o use guide for configuring FIP Snooping on the Dell

PowerConnect™ M8024-k Blade Switch (Figure 1), and the PowerConnect™ 8024F

Dell PowerConnect™ M8024-k Switch (10G Ethernet) Figure 1.

(Figure 2).

Dell PowerConnect™ 8024F (10G Ethernet) Figure 2.

Basic Terminology

FIP snooping

With FIP snooping enabled on the PowerConnect™ 8024 model switches, FIP logins, solicitations, and advertisements are monitor e d . In this monitoring or snoopin g p rocess the switch gathers information pertaining to the ENode and FCF ad d r esses. With this information the switch will then place filters that only allow access to ENode devices that have logged-in successfully. This enables the FCoE VLAN to deny all other traffic except this lossless FCoE storage traffic.

The filtering process also secures the end-to-end path between the ENode device and the FCF. The ENode will only be able to talk with the FCF in which it has logged into.

FIP snooping bridge (FSB)

With a switch configured to per forming FIP snooping the industry term for this switch is FSB or FIP snooping bridge. It is perfo r ming FIP snooping as described in th e pr e vious term.

FCF

FCoE forwarders (FCFs) act as an Ethernet and FC switch combine d . All typical termination functions that would occur on a FC switch occur on the FCF. FCF’s give VF_Ports and VE_Ports for their virtual FC interfaces.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

PFC

Priority Flow Control (PFC), or Per-Priority Pause is d e fined in the IEEE 802.1Qbb standar d . PFC is flow control based on priority settings and adds additional information to the standard pause frame. The additional fields added to the p ause frame allow devices to pause traffic on a specific priority instead of pausing all traffic. (IEEE, 2009) Pause frames will be initiated by the FCF in most cases when its receive buffers are starting to reach a congested point. With PFC traffic is paused instead of dropped and retransmitted. This provides the lossless network behavior necessary for FC packets to be encapsulated and passed along the Ethernet paths.

NPIV

N-port identifier virtualization which enables multiple N-port fabric logins at the same time on the same physical FC link (Cisco Systems, Inc., 2011).This term is in reference to t he Cisco Nexus 5000 series switches implementation of NPIV .

NPV

N-port virtualizer is a FC aggregation method which passes tr affic through to end devices, w hile eliminating the need to use a domain ID for this device (Cisco Systems, Inc., 2011). This term is also in reference to configuration settings on the Cisco Nexus 5000 series switches.

VSAN

Virtual SAN is a logical partitioning of physical connections to provide for fabric or SAN separation.

Note: The Dell M 100 0e Server Chassis includes a console redirect feature that allows you to manage

each PowerConnect M8024-k module from a single serial c onnection to the chassis. For mo r e information about console r e d ir ect, see the Dell Blade Server CMC User's Guide at http://support.dell.com/support/edocs/software/smdrac3/cmc/index.htm

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Configuration scenarios

The following sections will present very basic examples of deploying the 10G switches for FIP Snooping and will provide step-by-step explanations of the CLI commands as a guide. The GUI d oe s n ot currently support configurations for FIP Snooping. Consult the table of contents above for a list of examples covered in this document.

General overview of deployment Figure 3.

The following suggested co nfigurations used to deploy t his solution is done in a sequential order for reading but as Figure 3 represents this is more of a simultaneous process. There are dependencies that will be occuring during the configuration that will rely on other parts of the process. Storage configuration is not covered in any depth due to the possibilit y for various supported storage devices as part of the whole solution.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

configure

Important notes prior to deployment

No Simple Mode

Each of the following scenarios in this document assume that the PowerConnect™ 8024 model switch being used is in normal Switch Mode (not Simple Mode) and is using firmware version 4.2.x.x or later.

NOTE: If Simple Mode is enabled it will need to be disabled prior to impl ementing the deployment

covered in this document. FCoE is not supported with the PowerConnect 8024 model switches in Simple Mode. The CLI command in the example may be used for disabling Simple Mode, but please consult the User Guide for more information on specifics of Simple Mode.

Disabling simple mode Figure 4.

no mode simple

_________________________________________________________________________________________

Non-FIP-aware switches

If a Non-FIP-Aware switch is introduced anywhere in the data path FCoE will not be supported and can’t be expected to work as designed. The Dell PowerConnect M8024-k and 8024F are considered nonFIP-aware switches until they have t h e 4.2 or greater firmware installed. See updating firmware section for instructions on p e r forming this update.

_________________________________________________________________________________________

Stacking

Stacking is not recommended in an FCoE environment with the Dell PowerConnect 8024 Model Switches. If the switches are st acked the configuration should be changed to disable stacking . Please refer to the Dell PowerConnect 8024 4.2 firmware user’s guid e for further details on disabli ng or changing stacking ports. If th e configuration is used in this manner lossless Ethernet and reliability can not be guaranteed.

Dell PowerConnect 4.2 or greater firmware on M8024-k or 8024F

As mentioned in the non-FIP-aware bullet the Dell PowerConnect Switches will not support FCoE or FIP snooping without 4.2 or great e r firmware. See updating firmware

________________________________________________________________________________________

section to perform this update.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Scenario 1: Deploying the Dell PowerCo nnect 8024 Series FSB in a Cisco 5000 Series Switch (NPIV) environment

This first example is a basic, single connection between devices example using the Dell PowerConnect M8024-k. This configuration is being shown for the purposes of simplification and potentially easing into the progression of a more in-dep th setup. It is also easier to use a simple configuration such as this setup to aid in troubleshooting of the initial install. In a typical business environment most configurations will be scaled to include several connections between servers and storage . The scenarios following this one will show some of these larger configurations. Note that this configuration will also work in the rack server environment with Dell PowerConnect 8024F switch

Simple 1-link connection between devices Figure 5.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

The flowchart in Figure 8 is a general overview of how the deployment will occur. This inclu d e s t he basic planning that will need to take place in order for most of t he steps in the rest of the document to fall into place.

General Overview of the whole configuration and planning procedure Figure 6.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Figure 9 is a graphical represe n tation of how many of the configuration pieces are considered parallel settings. Most of the configuration will depend heavily on configurations being comple ted in more than just one place.

Overview of parallel configuration Figure 7.

In many of the business environments where this configuration will be installed there w ill be different administrators for the diffe r e nt areas of the infrastructure. In other words there may be a LAN infrastructure administrator, a storage or SAN administrator, and potentially a server administrator. These different team member s will have to work together for a successful deployment of al l the involved parts. In an M1000e bladeserver environment it may be the server admin that deploys t he blade servers, operatings systems, net work adapter drivers, and very p ossibly configures the blade IOM networking switches. If differ e n t admins are involved as described these tasks can be done in par allel to enable a quicker deployment.

It is important to understand ce r tain checks or validations al on g the way may rely on configurat ions being completed in a different part of the infrastructure.

configure

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Configuring the Dell PowerConnect M8024-k,8024, and 8024F for FIP Snooping

The Dell PowerConnect 8024 model switches will monitor FIP packets and will establish the proper filtering, and priorities f or the FCoE traffic that is passed t h rough the configured links. To see an example of the full configuration see Appendix-A

Command-Line Interface Method

Example commands for Dell PowerConnect M8024-k (can be copied and pasted) Figure 8.

no mode simple vlan database vlan 20,1000 exit hostname "mySwitch" vlan database vlan 20,1000 exit feature fip-snooping vlan 20,1000 fip-snooping enable exit interface out-of-band ip address 192.168.10.1 255.255.255.0 192.168.10.254 exit classofservice dot1p-mapping 1 1 classofservice dot1p-mapping 2 2 classofservice dot1p-mapping 3 3 classofservice dot1p-mapping 4 4 classofservice dot1p-mapping 5 5 classofservice dot1p-mapping 6 6 exit

interface Te1/0/1 switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 tagged switchport general allowed vlan remove 1 switchport mode general lldp dcbx port-role auto-down spanning-tree portfast exit

interface Te1/0/20 switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 tagged switchport general allowed vlan remove 1 spanning-tree cost 0 spanning-tree port-priority 0 switchport mode general lldp dcbx port-role auto-up fip-snooping port-mode fcf exit

- M8024-K Example.

CAUTION: The “copy running-configuration startup-configuration” command should be issued after

several impacting steps so that the switch will retain the configuration settings put into place on the next boot.

Routed VLAN’s can’t have FIP-snooping enabled. VLAN 1 may be set for routing and this must be

changed in the VLAN database if it is going to be used as the native VLAN or PVID.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Step by Step explanation of CLI example

o Configure – this brings the prompt into the configuration interface

o no mode simple – puts switch into normal mode o vlan database - moves down into the VLAN dat ab ase interface

 vlan 20 – add VLAN 20 to u se d for untagged traffic or as the nat ive VLAN  vlan 1000 - add VLAN 1000 to the VLAN database, this will be the FCoE VLAN  exit – exit the current level of the inte r face configuration

o hostname “mySwitch” – set the hostname of the switch in this example “mySwitch” o feature fip-snooping - this turns on the fip-snooping capab ility of the switch o vlan 20,1000 – this moves the interface into vlan 20,1000

 fip-snooping enable – this enables the fip-snooping capabilities on these p ar t icular

VLAN’s. Both must be included for the initial TLV negotiation to establish the FCoE VLAN

 exit – exit interfa ce configuration

o interface out-of-band – move into the interface out-of-band configuration inte rfa ce

 ip address 192.168. 100.1 255.255.255.0 192.168.100.254 – this sets the out-of-

band management interface IP address, subnet, and gateway for the switch

 exit – exit the interface configuration

o classofservice dot1p-mapping x x – establishes dire ct C oS mapping for the priorities (must be

in place for certain CNA’s

o interface te1/0/1 – this moves into the interface te1/0/1 configuration

 switchport general pvid 20 – establishes the native VLAN as 20, you must r emove VLAN

1 in order for this to function correctly

 switchport general allowed vlan add 20 - adds VLAN 20 the trunk as an untagged VLAN  switchport general allowed vlan add 1000 tagged – this sets up a trunk with a tagged

VLAN of 1000 (the FCoE VLAN), and includes the native VLAN as untagged if general mode is enabled.

 switchport general allowed remove vlan 1 – this removes vlan 1 which would typically

be the native vlan otherwise.

 switchport mode general – this enables the port for general mode  lldp dcbx port-role auto-down – sets t h e DCBx port-role to be auto-down for an ENode

connection

 spanning-tree portfast – sets the ports to a portfast b e h av ior since these are internal-

facing server ports.

 exit – exits the interfa ce conf igu r ati on

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

o interface te1/0/20 – this moves into the interface te1/0/20 configuration

 switchport general pvid 20 – establishes the native VLAN as 20, you must r emove VLAN

1 in order for this to function correctly

 switchport general allowed vlan add 20 - adds VLAN 20 the trunk as an untagged VLAN  switchport general allowed vlan add 1000 tagged - – this sets up a trunk with a

tagged VLAN of 1000 (the FCoE V LAN), and includes the native VL AN as untagged if general mode is enabled.

 switchport general allowed remove vlan 1 – this removes vlan 1 which would typically

be the native vlan otherwise.

 switchport mode general – this enables the port for mode general  spanning-tree cost 0 – sets spanning tree cost to 0  spanning-tree port-priority 0 – sets this ports priority t o 0 so t hat it has the lowest

spanning tree priority in ca se a loop is created elsewhere on th e switch

 lldp dcbx port-role auto-up – sets the DCBx port-role to be auto-up which dynamically

sets the configuration-source for an FCF connection

 fip-snooping port-mode fcf – enables the port for fip-snooping fr om an FCF connection  exit - exits the interfa ce conf igu r ati on  exit – exits from con figuration mode

Critical steps: The “copy running-configuration startup-configuration” command should be issued

after important steps so that the switch will retain the configuration settings when the switch is next rebooted or if a power los s occurs. It is also a good practice to copy a well-validated working configuration to a separate location such as the management station for the networks, and have a backup-configuration saved local to the switch.

Further explanation of key points:

• The spanning-tree settings in this example are established to keep the port from being potentially blocked by spanning-tree. This could occur because anothe r cable is plugged into a port with a lower priority, causing a loop. When the uplink port is set to 0 it will have the lowest priority and therefore most likely not end up in a blocked state.

• A second key se tting to note is “switchport ge n e ral allowed vlan remove 1”. T his command must be entered if you are choosing to use a different PVID or n at iv e V LAN. A port cannot have two native VLANs. In this example the configuration is set to use VLAN 20 since typically the recommendation is to have regular untagged traffic on a different VLAN other than just 1 for segregation of the network. In addition when the FCF sends information to the fip-snooping bridge (FSB) or M8024-k in this case, the M8024-k is receiving the initial information for negotiation on its untagged vlan (vlan 20 in this case). Once the initial negotiations have occurred properly the FCoE t r af fic will traverse the FCoE VLAN ( in this case VLAN 1000).

• The last configuration line “fip-snooping port-mod e fcf” is also ke y to this configuration. This line establishes where the FCF is attached to the switch. With this setting the port is configured to make Fibre Channel aware of the conne ct ion via this port to the forwarde r . The previous line “lldp dcbx port-role auto-up“, is setting this port to be aware of DCBx T LV ’s, he difference being the fip-snooping configurati on line points to the port for using fip-snooping tohe FCF, and the lldp configur ation points to the point for doin g DCBx negotiations.

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Configuration overview of Dell PowerConnect M8024-k Figure 9.

feature fcoe

Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F

Configuring the Cisco 5000 series switch with firmware ver 5.x for a single connection from the Dell PowerConnect M8024-k or 8024()(F)

The CLI commands below are necessary for an un-configured Cisco 5020. The CLI will show additional lines that are either default or can’t be changed and are not added f or t his example. The CLI will also show the lines in a different order after they have been entered.

example will have a copy of the f u ll configuration for a 5548UP for reference.

Command-Line Interface Method

Sample CLI for Cisco Nexus 5020 (can be copied and pasted) Figure 10.

feature npiv feature telnet feature lacp feature lldp

system default switchport trunk mode auto

vlan 20 vlan 1000 fcoe vsan 2

vsan database vsan 2 vsan 2 interface vfc1 vsan 2 interface fc2/1

interface vfc1 bind interface Ethernet1/1 no shutdown

interface fc2/1 switchport trunk mode auto no shutdown

interface Ethernet1/1 switchport mode trunk switchport trunk native vlan 20 switchport trunk allowed vlan 1000

zone name blade1 vsan 2 member interface fc2/1 member pwwn xx:xx:xx:xx:xx:xx:xx:xx

zoneset name set1 vsan 2 member blade1

zoneset activate name set1 vsan 2

Appendix-A Cisco Nexus 5548UP CLI

+ 39 hidden pages