Cisco SD2008T-NA, SGE2000, SLM2024T-NA, Nexus 2000 Series Configuration Manual

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

First Published: May 03, 2012

Last Modified: August 22, 2012

Americas Headquarters

Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883

Text Part Number: OL-25816-02

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright©1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://

www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership

relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.

CONTENTS

Preface

CHAPTER 1

Preface v

Audience v

Document Conventions v

Related Documentation for Cisco Nexus 7000 Series NX-OS Software vii

Documentation Feedback ix

Obtaining Documentation and Submitting a Service Request ix

Overview 1

Information About the Cisco Nexus 2000 Series Fabric Extender 2

Fabric Extender Terminology 2

Fabric Interface Features 3

Host Interfaces 3

Layer 3 Host Interfaces 3

Layer 2 Host Interfaces 3

Host Interface Port Channels 4

Layer 3 Host Interface Port Channels 4

Layer 2 Host Interface Port Channels 4

VLANs 5

Protocol Offload 5

Quality of Service 5

Access Control Lists 5

IGMP Snooping 6

Switched Port Analyzer 6

Oversubscription 6

Management Model 7

Forwarding Model 8

Port Channel Fabric Interface Connection 9

OL-25816-02 iii

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Contents

Port Numbering Convention 9

Fabric Extender Image Management 10

Licensing Requirements for the Fabric Extender 10

Guidelines and Limitations 10

Configuration Limits 12

Default Settings 12

CHAPTER 2

Configuring the Fabric Extender 13

Configuring the Cisco Nexus 2000 Series Fabric Extender 13

Managing the Fabric Extender Feature Set 13

Installing the Fabric Extender Feature Set 14

Enabling the Fabric Extender Feature Set 15

Disallowing the Fabric Extender Feature Set 15

Associating a Fabric Extender to a Fabric Interface 16

Associating a Fabric Extender to a Port Channel 17

Disassociating a Fabric Extender From an Interface 19

Associating a Fabric Extender to an F2 Module 19

Configuring Fabric Extender Global Features 21

Configuration Examples 23

Configuring a FEX with a Layer 3 Host Interface 24

Configuring a Host Interface in a vPC Topology Connected to Two FEXs 24

Dual-homing of a Server to a FEX with FabricPath 25

Verifying the Configuration 26

Verifying the Fabric Extender Configuration 26

Verifying the Chassis Management Information 29

Additional References 33

Other Software Documents

You can locate these documents starting at the following landing page:

http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html

•

Cisco Nexus 7000 Series NX-OS MIB Quick Reference

•

Cisco Nexus 7000 Series NX-OS Software Upgrade and Downgrade Guide

•

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide

•

Cisco NX-OS Licensing Guide

•

Cisco NX-OS System Messages Reference

•

Cisco NX-OS XML Interface User Guide

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

viii OL-25816-02

Preface

Documentation Feedback

To provide technical feedback on this document, or to report an error or omission, please send your comments to nexus7k-docfeedback@cisco.com. We appreciate your feedback.

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.

OL-25816-02 ix

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Obtaining Documentation and Submitting a Service Request

Preface

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

x OL-25816-02

CHAPTER 1

Overview

This chapter provides an architectural overview of the Cisco Nexus 2000 Series Fabric Extender and includes the following sections:

• Information About the Cisco Nexus 2000 Series Fabric Extender, page 2

• Fabric Extender Terminology, page 2

• Fabric Interface Features , page 3

• Host Interfaces, page 3

• Host Interface Port Channels, page 4

• VLANs, page 5

• Protocol Offload, page 5

• Quality of Service, page 5

• Access Control Lists, page 5

• IGMP Snooping, page 6

• Switched Port Analyzer, page 6

• Oversubscription, page 6

• Management Model, page 7

• Forwarding Model, page 8

• Port Channel Fabric Interface Connection, page 9

• Port Numbering Convention, page 9

• Fabric Extender Image Management, page 10

• Licensing Requirements for the Fabric Extender, page 10

• Guidelines and Limitations, page 10

• Default Settings, page 12

OL-25816-02 1

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Overview

Information About the Cisco Nexus 2000 Series Fabric Extender

The Cisco Nexus 2000 Series Fabric Extender, also known as FEX, is a highly scalable and flexible server networking solution that works with Cisco Nexus Series devices to provide high-density, low-cost connectivity for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and blade server environments, the Fabric Extender is designed to simplify data center architecture and operations.

The Fabric Extender integrates with its parent switch, a Cisco Nexus Series device, to allow automatic provisioning and configuration taken from the settings on the parent device. This integration allows large numbers of servers and hosts to be supported using the same feature set as the parent device, including security and quality-of-service (QoS) configuration parameters, with a single management domain. The Fabric Extender and its parent switch enable a large multipath, loop-free, active-active data center topology without the use of Spanning Tree Protocol (STP).

The Cisco Nexus 2000 Series Fabric Extender forwards all traffic to its parent Cisco Nexus Series device over 10-Gigabit Ethernet fabric uplinks, which allows all traffic to be inspected by policies established on the Cisco Nexus Series device.

Note

You must connect the Fabric Extender to its parent Cisco Nexus 7000 Series device equipped with a 32-port, 10-Gigabit M1 module (N7K-M132XP-12), a 32-port 10-Gigabit M1-XL module (N7K-M132XP-12L), an M2 module, or an F2 module.

No software is included with the Fabric Extender. Software is automatically downloaded and upgraded from its parent device.

Fabric Extender Terminology

Some terms used in this document are as follows:

• Fabric interface—A 10-Gigabit Ethernet uplink port designated for connection from the Fabric Extender to its parent switch. A fabric interface cannot be used for any other purpose. It must be directly connected to the parent switch.

Note

• Port channel fabric interface—A port channel uplink connection from the Fabric Extender to its parent switch. This connection consists of fabric interfaces bundled into a single logical channel.

• Host interface—An Ethernet host interface for connection to a server or host system.

A fabric interface includes the corresponding interface on the parent switch. This interface is enabled when you enter the switchport mode fex-fabric command.

Note

• Port channel host interface—A port channel host interface for connection to a server or host system.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

2 OL-25816-02

Do not connect a bridge or switch to a host interface. These interfaces are designed to provide end host or server connectivity.

Overview

Fabric Interface Features

The FEX fabric interfaces support static port channels. During the initial discovery and association process, SFP+ validation and digital optical monitoring (DOM) are performed as follows:

• The FEX performs a local check on the uplink SFP+ transceiver. If it fails the security check, the LED flashes but the link is still allowed to come up.

• The FEX local check is bypassed if it is running its backup image.

• The parent switch performs SFP validation again when the fabric interface is brought up. It keeps the fabric interface down if SFP validation fails.

Once an interface on the parent switch is configured in fex-fabric mode, all other features that were configured on that port and are not relevant to this mode are deactivated. If the interface is reconfigured to remove fex-fabric mode, the previous configurations are reactivated.

For more information about PFC, see the Cisco Nexus 7000 Series NX-OS Quality of Service Configuration Guide.

Fabric Interface Features

Host Interfaces

Layer 3 Host Interfaces

Beginning with Cisco NX-OS Release 5.2, by default, all host interfaces on a Fabric Extender connected to a Cisco Nexus 7000 Series parent switch run in Layer 3 mode.

Note

Layer 2 Host Interfaces

If you have updated the parent switch to Cisco Nexus Release 5.2, previously configured Fabric Extender host interfaces retain their default port mode, Layer 2. You can change these ports to Layer 3 mode with the no switchport command.

The host interfaces also support subinterfaces. You can create up to 32 subinterfaces on a Fabric Extender host interface.

For information about interfaces and subinterfaces, see the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide.

In Cisco NX-OS Release 5.1 and earlier releases, the default port mode is Layer 2.

To run a host interface in Layer 2 mode, use the switchport command. For Cisco NX-OS Release 5.2 and later releases, to change the port mode to Layer 3, use the no switchport command.

The Fabric Extender provides connectivity for computer hosts and other edge devices in the network fabric. The following guidelines should be followed when connecting devices to Fabric Extender host interfaces:

• All Fabric Extender host interfaces run as spanning tree edge ports with BPDU Guard enabled and cannot be configured as spanning tree network ports.

OL-25816-02 3

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Host Interface Port Channels

• Servers utilizing active/standby teaming, 802.3ad port channels, or other host-based link redundancy mechanisms can be connected to Fabric Extender host interfaces.

• Any device running spanning tree connected to a Fabric Extender host interface results in that host interface being placed in an error-disabled state when a BPDU is received.

• Any edge switch that leverages a link redundancy mechanism not dependent on spanning tree such as Cisco FlexLink or vPC (with the BPDU Filter enabled) can be connected to a Fabric Extender host interface. Because spanning tree is not used to eliminate loops, you should ensure a loop-free topology below the Fabric Extender host interfaces.

Ingress and egress packet counters are provided on each host interface.

For more information about BPDU Guard, see the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide.

Host Interface Port Channels

Overview

Layer 3 Host Interface Port Channels

The Fabric Extender supports host interface port channel configurations. You can combine up to 8 interfaces in a standard mode port channel and 16 interfaces when configured with the Link Aggregation Control Protocol (LACP).

Port channel resources are allocated when the port channel has one or more members.Note

All members of the port channel must be Fabric Extender host interfaces and all host interfaces must be from the same Fabric Extender. You cannot mix interfaces from the Fabric Extender and the parent switch.

Layer 3 mode is supported on host interface port channels.

A host interface port channel also supports subinterfaces. You can create up to 1000 subinterfaces on a Fabric Extender host interface port channel.

For more information about port channels, see the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide.

Layer 2 Host Interface Port Channels

Port channel resources are allocated when the port channel has one or more members.Note

Layer 2 mode is supported on host interface port channels.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

4 OL-25816-02

Overview

VLANs

You can configure Layer 2 port channels as access or trunk ports.

Beginning with Cisco NX-OS Release 5.2(1), Fabric Extenders support the host vPC feature where a server can be dual-attached to two different FEXs through a port channel. You must configure parent switches that connect each Fabric Extender (one parent switch per FEX) in a vPC domain.

The Fabric Extender supports Layer 2 VLAN trunks and IEEE 802.1Q VLAN encapsulation.

For more information about VLANs, see the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide.

Note: The Fabric Extender does not support Private VLANs (PVLANs).Note

Protocol Offload

To reduce the load on the control plane of the Cisco Nexus Series device, Cisco NX-OS allows you to offload link-level protocol processing to the Fabric Extender CPU. The following protocols are supported:

• Link Layer Discovery Protocol (LLDP) and Data Center Bridging Exchange (DCBX)

• Cisco Discovery Protocol (CDP)

• Link Aggregation Control Protocol (LACP)

Quality of Service

The Fabric Extender uses IEEE 802.1p class of service (CoS) values to associate traffic with the appropriate class. Per-port QoS configuration is also supported.

Host interfaces support pause frames, which are implemented using IEEE 802.3x link-level flow control (LLC). By default, flow control send is on and flow control receive is off on all host interfaces. Autonegotiation is enabled on the host interfaces. Per-class flow control is set according to the QoS classes.

For more information about LLC and quality-of-service, see the Cisco Nexus 7000 Series NX-OS Quality of Service Configuration Guide.

Access Control Lists

The Fabric Extender supports the full range of ingress access control lists (ACLs) that are available on its parent Cisco Nexus Series device.

For more information about ACLs, see the Cisco Nexus 7000 Series NX-OS Security Configuration Guide.

OL-25816-02 5

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

IGMP Snooping

IGMP snooping is supported on all host interfaces of the Fabric Extender.

The Fabric Extender and its parent switch support IGMPv2 and IGMPv3 snooping based only on the destination multicast MAC address. It does not support snooping based on the source MAC address or on proxy reports.

Overview

Note

For more information about IGMP snooping, see http://tools.ietf.org/wg/magma/draft-ietf-magma-snoop/

rfc4541.txt. Also, see the Cisco Nexus 7000 Series NX-OS Multicast Routing Configuration Guide.

Switched Port Analyzer

You can configure the host interfaces on the Fabric Extender as Switched Port Analyzer (SPAN) source ports. You cannot configure Fabric Extender ports as a SPAN destination. Only one SPAN session is supported for all the host interfaces on the same Fabric Extender. Ingress source (Rx), egress source (Tx), or both ingress and egress monitoring are supported.

Note

All IP multicast traffic on the VLANs that a Fabric Extender host interface belongs to is captured in the SPAN session. You cannot separate the traffic by IP multicast group membership.

If ingress monitoring and egress monitoring is configured for host interfaces on the same Fabric Extender, you might see a packet twice: once as the packet ingresses on an interface with Rx configured, and again as the packet egresses on an interface with Tx configured.

For more information about SPAN, see the Cisco Nexus 7000 Series NX-OS System Management Configuration Guide.

Oversubscription

In a switching environment, oversubscription is the practice of connecting multiple devices to the same interface to optimize port usage. An interface can support a connection that runs at its maximum speed. Because most interfaces do not run at their maximum speeds, you can take advantage of unused bandwidth by sharing ports. Oversubscription, which is a function of the available fabric interfaces to active host interfaces, provides cost-effective scalability and flexibility for Ethernet environments.

The Cisco Nexus 2248TP Fabric Extender has 4 10-Gigabit Ethernet fabric interfaces and 48 100/1000BASE-T (100-Mb/1-Gigabit) Ethernet host interfaces. When its host interfaces are running in Gigabit Ethernet mode, it offers the following configurations:

• No oversubscription (40 host interfaces for four fabric interfaces)

• 1.2 to 1 oversubscription (48 host interfaces for four fabric interfaces)

• 4.8 to 1 oversubscription (48 host interfaces for one fabric interface)

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

6 OL-25816-02

Overview

Management Model

The Cisco Nexus 2248TP can be run with no oversubscription when its host interfaces are running in 100-Mb mode.

The Cisco Nexus 2248TP-E Fabric Extender has 4 10-Gigabit Ethernet fabric interfaces and 48 100/1000BASE-T (100-Mb/1-Gigabit) Ethernet host interfaces. When its host interfaces are running in Gigabit Ethernet mode, it offers 1.2 to 1 oversubscription (48 host interfaces for four fabric interfaces).

The Cisco Nexus 2232PP Fabric Extender has 8 10-Gigabit Ethernet fabric interfaces and 32 10-Gigabit Ethernet host interfaces. All host interfaces use all of the available fabric interfaces. (Static pinning is not supported. Port-channel mode is supported only on fabric interfaces.) When all host interfaces are sending traffic to all fabric interfaces, the maximum oversubscription ratio for the Cisco Nexus 2232PP is 4:1.

The Cisco Nexus 2232TM Fabric Extender has 8 10-Gigabit Ethernet fabric interfaces and 32 Gigabit and 10-Gigabit Ethernet host interfaces. All host interfaces use all of the available fabric interfaces. (Static pinning is not supported. Port-channel mode is supported only on fabric interfaces.) When all host interfaces are sending traffic to all fabric interfaces, the maximum oversubscription ratio for the Cisco Nexus 2232TM is 4:1.

The Cisco Nexus 2224TP Fabric Extender has 2 10-Gigabit Ethernet fabric interfaces and 24 100/1000BASE-T (100-Mb/1-Gigabit) Ethernet host interfaces. With this system, you can configure a 1.2 to 1 oversubscription (24 host interfaces for 2 fabric interfaces) or higher.

Management Model

The Cisco Nexus 2000 Series Fabric Extender is managed by its parent switch over the fabric interfaces through a zero-touch configuration model. The switch discovers the Fabric Extender by detecting the fabric interfaces of the Fabric Extender.

After discovery, if the Fabric Extender has been correctly associated with the parent switch, the following operations are performed:

The switch checks the software image compatibility and upgrades the Fabric Extender if necessary.

The switch and Fabric Extender establish in-band IP connectivity with each other. The switch assigns an IP address in the range of loopback addresses (127.15.1.0/24) to the Fabric Extender to avoid conflicts with IP addresses that might be in use on the network.

The switch pushes the configuration data to the Fabric Extender. The Fabric Extender does not store any configuration locally.

The Fabric Extender updates the switch with its operational status. All Fabric Extender information is displayed using the switch commands for monitoring and troubleshooting.

OL-25816-02 7

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Forwarding Model

The Cisco Nexus 2000 Series Fabric Extender does not perform any local switching. All traffic is sent to the parent switch that provides central forwarding and policy enforcement, including host-to-host communications between two systems that are connected to the same Fabric Extender as shown in the following figure.

Figure 1: Forwarding Model

Overview

Note

The forwarding model facilitates feature consistency between the Fabric Extender and its parent Cisco Nexus Series device.

The Fabric Extender provides end-host connectivity into the network fabric. As a result, BPDU Guard is enabled on all its host interfaces. If you connect a bridge or switch to a host interface, that interface is placed in an error-disabled state when a BPDU is received.

You cannot disable BPDU Guard on the host interfaces of the Fabric Extender.

The Fabric Extender supports egress multicast replication from the network to the host. Packets sent from the parent switch for multicast addresses attached to the Fabric Extender are replicated by the Fabric Extender ASICs and are then sent to corresponding hosts.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

8 OL-25816-02

Overview

Port Channel Fabric Interface Connection

To provide load balancing between the host interfaces and the parent switch, you can configure the Fabric Extender to use a port channel fabric interface connection. This connection bundles 10-Gigabit Ethernet fabric interfaces into a single logical channel as shown in the following figure.

Figure 2: Port Channel Fabric Interface Connection

Port Channel Fabric Interface Connection

When you configure the Fabric Extender to use a port channel fabric interface connection to its parent switch, the switch load balances the traffic from the hosts that are connected to the host interface ports by using the following load-balancing criteria to select the link:

• For a Layer 2 frame, the switch uses the source and destination MAC addresses.

• For a Layer 3 frame, the switch uses the source and destination MAC addresses and the source and destination IP addresses.

Note

A fabric interface that fails in the port channel does not trigger a change to the host interfaces. Traffic is automatically redistributed across the remaining links in the port channel fabric interface. If all links in the fabric port channel go down, all host interfaces on the FEX are set to the down state.

Port Numbering Convention

The following port numbering convention is used for the Fabric Extender:

interface ethernet chassis/slot/port

where

•

chassis is configured by the administrator. A Fabric Extender must be directly connected to its parent Cisco Nexus Series device via a port channel fabric interface. You configure a chassis ID on a port channel on the switch to identify the Fabric Extender that is discovered through those interfaces.

OL-25816-02 9

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Fabric Extender Image Management

The chassis ID ranges from 101 to 199.

Overview

Note

•

slot identifies the slot number on the Fabric Extender.

•

port identifies the port number on a specific slot and chassis ID.

The chassis ID is required only to access a host interface on the Fabric Extender. A value of less than 101 indicates a slot on the parent switch. The following port numbering convention is used for the interfaces on the switch:

interface ethernet slot/port

Fabric Extender Image Management

No software ships with the Cisco Nexus 2000 Series Fabric Extender. The Fabric Extender image is bundled into the system image of the parent switch. The image is automatically verified and updated (if required) during the association process between the parent switch and the Fabric Extender.

When you enter the install all command, it upgrades the software on the parent Cisco Nexus Series switch and also upgrades the software on any attached Fabric Extender. To minimize downtime as much as possible, the Fabric Extender remains online while the installation process loads its new software image. Once the software image has successfully loaded, the parent switch and the Fabric Extender both automatically reboot.

This process is required to maintain version compatibility between the parent switch and the Fabric Extender.

Licensing Requirements for the Fabric Extender

The following table shows the licensing requirements for the Cisco Nexus 2000 Series Fabric Extender:

License RequirementProduct

Cisco NX-OS

The Cisco Nexus 2000 Series Fabric Extender requires no license. Any feature not included in a license package is bundled with the Cisco NX-OS system images and is provided at no extra charge to you. For an explanation of the licensing scheme, see the Cisco NX-OS Licensing Configuration Guide.

Guidelines and Limitations

The Cisco Nexus 2000 Series Fabric Extender has the following configuration guidelines and limitations:

• Beginning with Cisco NX-OS Release 5.2(1), the default port mode is Layer 3. Before Cisco NX-OS Release 5.2(1), the default port mode was Layer 2.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

10 OL-25816-02

Overview

Guidelines and Limitations

• You must enable the Fabric Extender feature set in the default virtual device context (VDC). After you enable the feature set in the default VDC, the FEX can belong to any VDC and can be configured from those VDCs.

• All the uplinks and host ports of a Fabric Extender belong to a single VDC. The ports cannot be allocated or split among multiple VDCs.

• You must connect the Fabric Extender to its parent Cisco Nexus 7000 Series device equipped with a 32-port 10-Gigabit M1 module (N7K-M132XP-12), a 32-port, 10-Gigabit M1-XL module (N7K-M132XP-12L), an M2 module, or an F2 module.

• The Fabric Extender feature set operation might cause the standby supervisor to reload if it is in an unstable state, such as following a service failure or powering up. You can check whether the standby supervisor is stable with the show modules command. When the standby supervisor is stable, it is indicated as ha-standby.

• You can configure the Fabric Extender host interfaces as edge ports only. The interface is placed in an error disabled state if a downstream switch is detected.

• When you connect a FEX is to a Cisco Nexus 7000 series device, the queuing capability on the FEX host interface is limited. A router connected to an Layer 2 (using SVI interfaces) or Layer 3 FEX interface cannot participate in routing protocol adjacency. The FEX cannot be used as a peer because when congestion occurs on the FEX host interface, the control plane traffic is not prioritized. This limitation also applies to the FEX when it is connected to other Layer 3 devices, such as an ASA firewall, an ACE load balancer, or other Layer 3 networking devices running a dynamic routing protocol. Static routes to routers, ASA firewalls, ACE load balancers, and other Layer 3 network devices are supported.

• The Fabric Extender does not support PVLANs.

Associating with F2 Series modules

• F2 modules are only supported by the following FEX devices:

◦ 2248TP

◦ 2248TP-E

◦ 2232TP

◦ 2232PP

◦ 2232TM

◦ 2224TP

• Each port in the ASIC has an index. Allow only ports with similar indices across ASICs to be added to a port channel.

For example, if port 1 has an index of 1 and port 2 has an index of 2, then

◦ Supported: Port 1 of ASIC 1 and port 1 of ASIC 2 are added to a port channel.

◦ Not supported: Port 1 of ASIC 1 and port 2 of ASIC 2 to form a port channel.

In general, a set of ports from an ASIC that has an index sub-set S, such as {1,2,4}, is allowed to be added to a port-channel only if the port-channel has an equivalent or an empty set.

OL-25816-02 11

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Configuration Limits

The configuration limits are documented in the Cisco Nexus 7000 Series NX-OS Verified Scalability Guide.

Default Settings

This table lists the default settings for the Fabric Extender parameters.

Table 1: Default Cisco Nexus 2000 Series Fabric Extender Parameter Settings

Overview

DefaultParameters

Disabledfeature-set fex command

Port mode

Layer 3 (Cisco NX-OS Release 5.2 and later releases).

Layer 2 (Cisco NX-OS Release 5.1 and earlier releases).

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

12 OL-25816-02

CHAPTER 2

Configuring the Fabric Extender

This chapter describes how to configure a Cisco Nexus 2000 Series Fabric Extender using a parent Cisco Nexus 7000 Series device and includes the following sections:

• Configuring the Cisco Nexus 2000 Series Fabric Extender, page 13

• Verifying the Configuration, page 26

• Additional References, page 33

Configuring the Cisco Nexus 2000 Series Fabric Extender

This section describes how to configure the Fabric Extender.

Managing the Fabric Extender Feature Set

You can install and manage the Fabric Extender feature set.

SUMMARY STEPS

Installing the Fabric Extender Feature Set, on page 14

Enabling the Fabric Extender Feature Set, on page 15

(Optional) Disallowing the Fabric Extender Feature Set, on page 15

DETAILED STEPS

Step 1

Step 2

Step 3

OL-25816-02 13

Installing the Fabric Extender Feature Set, on page 14

Enabling the Fabric Extender Feature Set, on page 15

(Optional) Disallowing the Fabric Extender Feature Set, on page 15

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Managing the Fabric Extender Feature Set

Installing the Fabric Extender Feature Set

Configuring the Fabric Extender

Note

SUMMARY STEPS

DETAILED STEPS

Step 1

You must enable the Fabric Extender feature set in the default VDC. Once enabled in the default VDC, the FEX can belong to any VDC and can be configured from those VDCs. A single Fabric Extender belongs exclusively to a single VDC.

Before You Begin

Ensure that you are in the default VDC.

Ensure that you have disabled the IDS reserved addresses check (it is disabled by default). Use the show hardware ip verify command and look for the string "address reserved" in the output. If the IDS reserved addresses check is enabled, disable it with the no hardware ip verify address reserved command.

configure terminal

install feature-set fex

exit

PurposeCommand or Action

Enters configuration mode.configure terminal

Step 2

Step 3

Example:

switch# configure terminal switch(config)#

Example:

switch(config)# install feature-set fex

Example:

switch(config)# exit switch#

Installs the Fabric Extender feature set in the default VDC.install feature-set fex

To uninstall the Fabric Extender feature set, use the no install feature-set fex command. Before you can uninstall the feature set, you must ensure the following:

• The feature set must be installed in the default VDC.

• The feature set must not be enabled in any VDC.

Exits configuration mode.exit

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

14 OL-25816-02

Configuring the Fabric Extender

Enabling the Fabric Extender Feature Set

You can enable the installed Fabric Extender feature set in any VDC on the device.

Before You Begin

Ensure that you have installed the Fabric Extender feature set in the default VDC.

Ensure that you are in the correct VDC or use the switchto vdc command.

SUMMARY STEPS

configure terminal

feature-set fex

exit

DETAILED STEPS

Managing the Fabric Extender Feature Set

Step 1

Step 2

Step 3

Example:

switch# configure terminal switch(config)#

feature-set fex

Example:

switch(config)# feature-set fex

Example:

switch(config)# exit switch#

PurposeCommand or Action

Enters configuration mode.configure terminal

Enables the Fabric Extender feature set. The feature set must be installed before it shows as an option to this command.

To disable the Fabric Extender feature set, use the no feature-set fex command. Before you can disable a feature set, you must install the feature set in the default VDC.

Note

The no feature-set fex command might take some time to complete if the size of the configuration is very large. The command cleans up all of the configurations associated with the Fabric Extender feature set.

Exits configuration mode.exit

Disallowing the Fabric Extender Feature Set

By default, when you install the Fabric Extender feature set, it is allowed in all VDCs. You can disallow the installed Fabric Extender feature set in a specific VDC on the device.

Before You Begin

Ensure that you have installed the feature set in the default VDC.

OL-25816-02 15

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Associating a Fabric Extender to a Fabric Interface

Ensure that you have not enabled the feature set in the specified VDC.

SUMMARY STEPS

configure terminal

vdc vdc_ID

no allow feature-set fex

end

DETAILED STEPS

Configuring the Fabric Extender

PurposeCommand or Action

Step 1

Step 2

Step 3

Step 4

Example:

switch# configure terminal switch(config)#

vdc vdc_ID

Example:

switch(config)# vdc 1

no allow feature-set fex

Example:

switch(config-vdc)# no allow feature-set

fex

Example:

switch(config-vdc)# end switch#

Enters configuration mode.configure terminal

Specifies a VDC and enters VDC configuration mode.

Disallows the feature set in the VDC. You cannot disallow a feature set that is enabled in the specified VDC.

By default, the installed Fabric Extender feature set is allowed in all VDCs on the device. You can disallow a feature set in a specific VDC. Subsequently, you can change the status back to allowed with the allow feature-set fex command.

Exits VDC configuration mode and returns to EXEC mode.end

Associating a Fabric Extender to a Fabric Interface

A FEX is connected to its parent device through a port channel. By default, the parent device does not allow the attached Fabric Extender to connect until it has been assigned a FEX-number and is associated with the connected interface.

Note

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

16 OL-25816-02

You must have installed and enabled the Fabric Extender features with the install feature-set fex and feature-set fex commands before you can configure and use a Fabric Extender that is connected to the

parent switch.

Configuring the Fabric Extender

Associating a Fabric Extender to a Port Channel

Before You Begin

Ensure that you have installed and enabled the Fabric Extender feature set.

Ensure that you are in the correct VDC (or use the switchto vdc command).

SUMMARY STEPS

configure terminal

interface port-channel channel

switchport

switchport mode fex-fabric

fex associate FEX-number

(Optional) show interface port-channel channel fex-intf

Associating a Fabric Extender to a Fabric Interface

DETAILED STEPS

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

Example:

switch# configure terminal switch(config)#

interface port-channel channel

Example:

switch(config)# interface port-channel 4 switch(config-if)#

Example:

switch(config-if)# switchport

switchport mode fex-fabric

Example:

switch(config-if)# switchport mode fex-fabric

fex associate FEX-number

Example:

switch(config-if)# fex associate 101

show interface port-channel channel fex-intf

Example:

switch# show interface port-channel 4 fex-intf

PurposeCommand or Action

Enters configuration mode.configure terminal

Specifies a port channel to configure.

Note

All interfaces in the port channel must be in the same VDC.

Sets the interface as a Layer 2 switching port.switchport

Sets the port channel to support an external Fabric Extender.

Associates a FEX number to the Fabric Extender unit attached to the interface. The range is from 101 to 199.

(Optional) Displays the association of a Fabric Extender to a port channel interface.

OL-25816-02 17

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Associating a Fabric Extender to a Fabric Interface

Examples

This example shows how to associate the Fabric Extender to a port channel interface on the parent device:

switch# configure terminal switch(config)# interface ethernet 1/28 switch(config-if)# channel-group 4 switch(config-if)# no shutdown switch(config-if)# exit switch(config)# interface ethernet 1/29 switch(config-if)# channel-group 4 switch(config-if)# no shutdown switch(config-if)# exit switch(config)# interface ethernet 1/30 switch(config-if)# channel-group 4 switch(config-if)# no shutdown switch(config-if)# exit switch(config)# interface ethernet 1/31 switch(config-if)# channel-group 4 switch(config-if)# no shutdown switch(config-if)# exit switch(config)# interface port-channel 4 switch(config-if)# switchport switch(config-if)# switchport mode fex-fabric switch(config-if)# fex associate 101

Configuring the Fabric Extender

Tip

Note

As a best practice, only enter the fex associate command from the port channel interface, not from the physical interface.

If you try to associate a physical port to a FEX before that physical port is joined to a port channel, the physical port moves to the err-disable state and the Cisco Nexus 7000 series device does not communicate with the FEX on that link. You must enter the shutdown command and the no shutdown command on the Ethernet interface (not the port channel interface) to clear the err-disable state and bring the link up. (Note that this does not apply if the configuration is performed prior to cabling.)

When adding physical interfaces to port channels, all configurations on the port channel and physical interface must match.

This example shows how to display the association of the Fabric Extender and the parent device:

switch# show interface port-channel 4 fex-intf Fabric FEX Interface Interfaces

--------------------------------------------------Po4 Eth101/1/48 Eth101/1/47 Eth101/1/46 Eth101/1/45

Eth101/1/44 Eth101/1/43 Eth101/1/42 Eth101/1/41 Eth101/1/40 Eth101/1/39 Eth101/1/38 Eth101/1/37 Eth101/1/36 Eth101/1/35 Eth101/1/34 Eth101/1/33 Eth101/1/32 Eth101/1/31 Eth101/1/30 Eth101/1/29 Eth101/1/28 Eth101/1/27 Eth101/1/26 Eth101/1/25 Eth101/1/24 Eth101/1/23 Eth101/1/22 Eth101/1/21 Eth101/1/20 Eth101/1/19 Eth101/1/18 Eth101/1/17 Eth101/1/16 Eth101/1/15 Eth101/1/14 Eth101/1/13 Eth101/1/12 Eth101/1/11 Eth101/1/10 Eth101/1/9 Eth101/1/8 Eth101/1/7 Eth101/1/6 Eth101/1/5 Eth101/1/4 Eth101/1/3 Eth101/1/2 Eth101/1/1

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

18 OL-25816-02

Configuring the Fabric Extender

Disassociating a Fabric Extender From an Interface

Before You Begin

Ensure that you have installed and enabled the Fabric Extender feature set.

Ensure that you are in the correct VDC (or use the switchto vdc command).

SUMMARY STEPS

configure terminal

interface {ethernet slot/port | port-channel channel}

no fex associate

DETAILED STEPS

Associating a Fabric Extender to a Fabric Interface

PurposeCommand or Action

Step 1

Example:

switch# configure terminal switch(config)#

Step 2

Step 3

interface {ethernet slot/port | port-channel channel}

Example:

switch(config)# interface port-channel 4 switch(config-if)#

no fex associate

Example:

switch(config-if)# no fex associate

Associating a Fabric Extender to an F2 Module

You can associate the Fabric Extender to an F2 module.

Before You Begin

Ensure that you have installed and enabled the Fabric Extender feature set.

Enters configuration mode.configure terminal

Specifies the interface to configure. The interface can be an Ethernet interface or a port channel.

Disassociates the Fabric Extender unit attached to the interface.

• switch# configure terminal

• switch(config)# install feature-set fex

• switch(config)# feature-set fex

OL-25816-02 19

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Associating a Fabric Extender to a Fabric Interface

F2 modules are only supported by the following FEX devices:Note

• 2248TP

• 2248TP-E

• 2232TP

• 2232PP

• 2232TM

• 2224TP

Configuring the Fabric Extender

Note

SUMMARY STEPS

Each port in the ASIC has an index. Allow only ports with similar indices across ASICs to be added to a port channel.

For example, if port 1 has an index of 1 and port 2 has an index of 2, then

• Supported: Port 1 of ASIC 1 and port 1 of ASIC 2 are added to a port channel.

• Not supported: Port 1 of ASIC 1 and port 2 of ASIC 2 to form a port channel.

In general, a set of ports from an ASIC that has an index sub-set S, such as {1,2,4}, is allowed to be added to a port-channel only if the port-channel has an equivalent or an empty set.

vdc switch

limit-resource module-type f2

interface ethernet <mod-number><port-range>

allocate interface ethernet <slot-port>

switchport mode fex -fabric

fex associate <FEX chassis number>

channel-group <port-channel number>

no shutdown

int po <port-channel number>

no shutdown

10.

DETAILED STEPS

Step 1

Example:

switch(config)# vdc switch

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

20 OL-25816-02

PurposeCommand or Action

Specifies the VDC.vdc switch

Configuring the Fabric Extender

Configuring Fabric Extender Global Features

PurposeCommand or Action

Step 2

Step 3

Step 4

Step 5

Step 6

Step 7

Step 8

Specifies the module type.limit-resource module-type f2

Example:

switch(config)# limit-resource module-type f2

Specifies the interface.interface ethernet <mod-number><port-range>

Example:

switch(config)# interface ethernet 1/1

Allocates the F2 interfaces to VDC.allocate interface ethernet <slot-port>

Example:

switch(config)# allocate interface ethernet 1

Specifies the FEXswitchport mode fex -fabric

Example:

switch(config-if)# switchport mode fex -fabric

Specifies the chassis.fex associate <FEX chassis number>

Example:

switch(config-if)# fex associate 101

Specifies the port channel number.channel-group <port-channel number>

Example:

switch(config-if)# channel-group 1

Brings up port.no shutdown

Example:

switch(config-if)# no shutdown

Step 9

Example:

switch(config-if)# int po 1

Step 10

Example:

switch(config-if)# no shutdown

Configuring Fabric Extender Global Features

You can configure global features on the Fabric Extender.

Specifies the port channel.int po <port-channel number>

Brings up the port channel.no shutdown

OL-25816-02 21

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Configuring Fabric Extender Global Features

Before You Begin

Ensure that you have enabled the Fabric Extender feature set.

Ensure that you are in the correct VDC (or use the switchto vdc command).

SUMMARY STEPS

Configuring the Fabric Extender

configure terminal

fex FEX-number

(Optional) description desc

(Optional) no description

(Optional) type FEX-type

(Optional) no type

(Optional) serial serial

(Optional) no serial

DETAILED STEPS

Step 1

Example:

switch# configure terminal switch(config)#

Step 2

Step 3

Step 4

Step 5

fex FEX-number

Example:

switch(config)# fex 101 switch(config-fex)#

description desc

Example:

switch(config-fex)# description Rack7A-N2K

no description

Example:

switch(config-fex)# no description

type FEX-type

Example:

switch(config-fex)# type N2248T

PurposeCommand or Action

Enters configuration mode.configure terminal

Enters configuration mode for the specified Fabric Extender. The range of the FEX-number is from 101 to 199.

(Optional) Specifies the description. The default is the string FEXxxxx where xxxx is the FEX-number. If the FEX-number is 123, the description is FEX0123.

(Optional) Deletes the description.

(Optional) Specifies the type of Fabric Extender. FEX-type is one of the following:

• N2224TP—24 100Base-T/1000Base-T Ethernet host interfaces and 2 10-Gigabit SFP+ Ethernet fabric interfaces module

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

22 OL-25816-02

• N2232P and N2232TM—32 10-Gigabit SFP+ Ethernet host interfaces and 8 10-Gigabit SFP+ Ethernet fabric interfaces module

Configuring the Fabric Extender

Configuration Examples

PurposeCommand or Action

• N2248T and N2248TP-E—48 100Base-T/1000Base-T Ethernet host interfaces and 4 10-Gigabit SFP+ Ethernet fabric interfaces module.

The parent Cisco Nexus Series device remembers the type of the Fabric Extender in its binary configuration. When this feature is configured, the Fabric Extender is only allowed to come online if its type matches the configured FEX-type.

Step 6

Step 7

Step 8

no type

Example:

switch(config-fex)# no type

serial serial

Example:

switch(config-fex)# serial JAF1339BDSK

no serial

Example:

switch(config-fex)# no serial

Configuration Examples

This section contains examples of FEX configurations.

(Optional) Deletes the FEX type. When a Fabric Extender is connected to the fabric interfaces and does not match the configured type saved in the binary configuration on the parent switch, all configurations for all interfaces on the Fabric Extender are deleted.

(Optional) Defines a serial number string. If this command is configured, a switch only allows the corresponding chassis ID to associate (using the fex associate command) if the Fabric Extender reports a matching serial number string.

Caution

Configuring a serial number that does not match the specified Fabric Extender forces the Fabric Extender offline.

(Optional) Deletes the serial number string.

OL-25816-02 23

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Configuration Examples

Configuring a FEX with a Layer 3 Host Interface

This example shows how to configure a Fabric Extender with a Layer 3 host interface (at the interface level, subinterface level, port channel level, and port channel subinterface level):

config t interface ethernet 101/1/1 no switchport ip address 192.0.1.1/24 Mtu 9000 no shutdown

Configuring the Fabric Extender

Layer 3 Sub-interface ConfigurationLayer 3 Configuration

config t interface ethernet 101/1/1.12 ip address 192.0.2.1/24 encapsulation dot1Q 12 mtu 850 no shutdown

Layer 3 Host Interface Port Channel Configuration

Layer 3 Host Interface Port Channel Sub-interface Configuration

config t interface ethernet 101/1/1-2 no switchport channel-group 12 no shutdown

interface port-channel 12.14 ip address 192.0.4.1/24 encapsulation dot1Q 14 mtu 1700 no shutdown

Note

config t interface ethernet 101/1/1-2 no switchport channel-group 12 no shutdown

interface port-channel 12 ip address 192.0.3.1/24 mtu 2000 no shutdown

The VLAN used in the Layer 3 host interface (HIF) or host interface port channel (HIFPC) subinterface has only local significance to its parent interface. The same VLAN ID can be reused between Layer 3 subinterfaces in the same switch or VDC.

Configuring a Host Interface in a vPC Topology Connected to Two FEXs

This example shows how to configure a host vPC with a FEX (host vPC attached to 2 different FEXs):

Switch 2 ConfigurationSwitch 1 Configuration

config t feature lacp int e101/1/1-2 channel-group 12 mode active no shutdown

Int port-channel10 switchport

switchport mode trunk switchport trunk allowed vlan 1-20

vpc 10

config t feature lacp int e101/1/1-2 channel-group 12 mode active no shutdown

Int port-channel10 switchport

switchport mode trunk switchport trunk allowed vlan 1-20

vpc 10

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

24 OL-25816-02

Configuring the Fabric Extender

Dual-homing of a Server to a FEX with FabricPath

To configure FabricPath interfaces for two Cisco Nexus 7000 switches that are connected with FabricPath, perform the following tasks on each switch:

• Enable FabricPath on each switch.

• Configure the interfaces that you want to designate as FabricPath interfaces.

• Set the STP priority device to 8192 on all FabricPath Layer 2 gateway devices.

• (Optional) Set the STP domain ID for each of the separate STP domains that are connected to the FabricPath network.

• (Optional) Configure a FEX switch ID.

To configure FabricPath interfaces, follow these steps:

(Optional) Enable FabricPath on each switch.

switch# config terminal switch(config)# feature fabricpath switch(config-lldp)# exit switch(config)#

After you enable FabricPath on the switch, configure the specified interface as FabricPath interfaces.

switch(config)# interface ethernet 1/2 switch(config-if)# switchport mode fabricpath switch(config-if)# exit switch(config)#

Configure the STP priority for all Rapid PVST+ VLANs as 8192.

switch# config terminal switch(config)# spanning-tree vlan 11-20 priority 8192 switch(config)#

Configure the STP priority for all MST instances as 8192.

switch# config terminal switch(config)# spanning-tree mst 1-5 priority 8192 switch(config)#

(Optional) Configure the STP domain ID on each FabricPath Layer 2 gateway switch attached to the FabricPath network.

switch# config terminal switch(config)# spanning-tree domain 5 switch(config)

(Optional) Configure the FEX switch ID.

Configuration Examples

Note

See the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide for information on configuring FEX.

FEX VPC+ configurations are only supported on F2 modules.Note

If you are setting up an initial FEX VPC+ configuration on an F Series module, follow these steps:

In the VPC domain configuration mode, enable partial DF mode with the fabricpath multicast load-balance command.

OL-25816-02 25

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Verifying the Configuration

Configuring the Fabric Extender

If disabled, enable TRILL style mac-address learning with the mac address-table core-port-learning command.

In the VPC domain configuration mode, configure the emulated switch ID with the fabricpath switchid <switchid#> command.

On each of the VPC/VPC+ peer link interfaces in interface configuration mode, enter the switchport mode fabricpath command.

On each VPC/VPC+ peer link port channel, enter the VPC peer-link command.

Configure the VPC ID with the vpc vpcid command.

If you are changing an existing FEX VPC configuration to a FEX VPC+ configuration on an F Series module, follow these steps:

In the VPC domain configuration mode, enable partial DF mode with the fabricpath multicast load-balance command.

If disabled, enable trill style mac-address learning with the mac address-table core-port-learning command.

In the VPC domain configuration mode, configure the emulated switch ID with the fabricpath switchid <switchid#> command.

Copy the configuration.

switch(config)# copy running-config startup-config switch(config)#

Verifying the Configuration

This section describes how to display the configuration of the Fabric Extender and verify the chassis hardware status.

Verifying the Fabric Extender Configuration

To display configuration information about the defined interfaces on a Fabric Extender, perform one of the following tasks:

show fex [FEX-number] [detail]

show interface type number fex-intf

PurposeCommand or Action

Displays information about a specific Fabric Extender or all attached units.

Displays the Fabric Extender ports that are pinned to a specific switch interface.

show interface fex-fabric

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

26 OL-25816-02

Displays the switch interfaces that have detected a Fabric Extender uplink.

Configuring the Fabric Extender

Verifying the Fabric Extender Configuration

PurposeCommand or Action

show interface ethernet number transceiver

[fex-fabric]

Displays the SFP+ transceiver and diagnostic optical monitoring (DOM) information for the Fabric Extender uplinks.

Displays the status of the feature sets on the device.show feature-set

Configuration Examples for the Fabric Extender

This example shows how to display all the attached Fabric Extender units:

switch# show fex

FEX FEX FEX FEX

Number Description State Model Serial

-----------------------------------------------------------------------101 FEX0101 Online N2K-C2248TP-1GE JAF1418AARL

This example shows how to display the detailed status of a specific Fabric Extender:

switch# show fex 101 detail FEX: 101 Description: FEX0101 state: Online

FEX version: 5.1(1) [Switch version: 5.1(1)] FEX Interim version: 5.1(0.159.6) Switch Interim version: 5.1(1) Extender Model: N2K-C2248TP-1GE, Extender Serial: JAF1418AARL Part No: 73-12748-05 Card Id: 99, Mac Addr: 54:75:d0:a9:49:42, Num Macs: 64 Module Sw Gen: 21 [Switch Sw Gen: 21]

pinning-mode: static Max-links: 1

Fabric port for control traffic: Po101 Fabric interface state:

Po101 - Interface Up. State: Active Eth2/1 - Interface Up. State: Active Eth2/2 - Interface Up. State: Active Eth4/1 - Interface Up. State: Active Eth4/2 - Interface Up. State: Active

Fex Port State Fabric Port Primary Fabric

Eth101/1/1 Up Po101 Po101 Eth101/1/2 Up Po101 Po101 Eth101/1/3 Down Po101 Po101 Eth101/1/4 Down Po101 Po101 Eth101/1/5 Down Po101 Po101 Eth101/1/6 Down Po101 Po101 Eth101/1/7 Down Po101 Po101 Eth101/1/8 Down Po101 Po101

Eth101/1/9 Down Po101 Po101 Eth101/1/10 Down Po101 Po101 Eth101/1/11 Down Po101 Po101 Eth101/1/12 Down Po101 Po101 Eth101/1/13 Down Po101 Po101 Eth101/1/14 Down Po101 Po101 Eth101/1/15 Down Po101 Po101 Eth101/1/16 Down Po101 Po101 Eth101/1/17 Down Po101 Po101 Eth101/1/18 Down Po101 Po101 Eth101/1/19 Down Po101 Po101 Eth101/1/20 Down Po101 Po101 Eth101/1/21 Down Po101 Po101 Eth101/1/22 Down Po101 Po101 Eth101/1/23 Down Po101 Po101 Eth101/1/24 Down Po101 Po101 Eth101/1/25 Down Po101 Po101 Eth101/1/26 Down Po101 Po101 Eth101/1/27 Down Po101 Po101 Eth101/1/28 Down Po101 Po101 Eth101/1/29 Down Po101 Po101

OL-25816-02 27

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Verifying the Fabric Extender Configuration

Eth101/1/30 Down Po101 Po101 Eth101/1/31 Down Po101 Po101 Eth101/1/32 Down Po101 Po101 Eth101/1/33 Down Po101 Po101 Eth101/1/34 Down Po101 Po101 Eth101/1/35 Down Po101 Po101 Eth101/1/36 Down Po101 Po101 Eth101/1/37 Down Po101 Po101 Eth101/1/38 Down Po101 Po101 Eth101/1/39 Down Po101 Po101 Eth101/1/40 Down Po101 Po101 Eth101/1/41 Down Po101 Po101 Eth101/1/42 Down Po101 Po101 Eth101/1/43 Down Po101 Po101 Eth101/1/44 Down Po101 Po101 Eth101/1/45 Down Po101 Po101 Eth101/1/46 Down Po101 Po101 Eth101/1/47 Down Po101 Po101

Eth101/1/48 Down Po101 Po101 Logs: 09/21/2010 21:14:26.843850: Module register received 09/21/2010 21:14:26.845778: Registration response sent 09/21/2010 21:14:27.663073: Module Online Sequence 09/21/2010 21:14:30.191121: Module Online

This example shows how to display the Fabric Extender interfaces pinned to a specific switch interface:

switch# show interface port-channel 101 fex-intf Fabric FEX Interface Interfaces

---------------------------------------------------

Po101 Eth101/1/2 Eth101/1/1

Configuring the Fabric Extender

This example shows how to display the switch interfaces that are connected to a Fabric Extender uplink:

switch# show interface fex-fabric

Fabric Fabric Fex FEX

Fex Port Port State Uplink Model Serial

--------------------------------------------------------------101 Eth2/1 Active 1 N2K-C2248TP-1GE JAF1418AARL 101 Eth2/2 Active 2 N2K-C2248TP-1GE JAF1418AARL 101 Eth4/1 Active 3 N2K-C2248TP-1GE JAF1418AARL 101 Eth4/2 Active 4 N2K-C2248TP-1GE JAF1418AARL

This example shows how to display the SFP+ transceiver and diagnostic optical monitoring (DOM) information for Fabric Extender uplinks for an SFP+ transceiver that is plugged into the parent switch interface:

switch# show interface ethernet 1/40 transceiver Ethernet1/40

sfp is present name is CISCO-MOLEX INC part number is 74752-9026 revision is A0 serial number is MOC13321057 nominal bitrate is 12000 MBits/sec Link length supported for copper is 3 m(s) cisco id is -cisco extended id number is 4

This example shows how to display the SFP+ transceiver and DOM information for Fabric Extender uplinks for an SFP+ transceiver that is plugged into the uplink port on the Fabric Extender:

switch# show interface ethernet 1/40 transceiver fex-fabric Ethernet1/40

sfp is present name is CISCO-MOLEX INC part number is 74752-9026 revision is A0 serial number is MOC13321057 nominal bitrate is 12000 MBits/sec Link length supported for 50/125mm fiber is 0 m(s) Link length supported for 62.5/125mm fiber is 0 m(s)

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

28 OL-25816-02

Configuring the Fabric Extender

cisco id is -cisco extended id number is 4

The following example shows how to display the status of the feature sets on the device:

switch# show feature-set Feature Set Name ID State

-------------------- -------- -------fcoe 1 installed fabricpath 2 enabled fex 3 enabled switch#

The following example shows how to display the services used by the Fabric Extender feature set:

switch# show feature-set services fex vntag_mgr fex 2 services in feature set fex switch#

Verifying the Chassis Management Information

To display configuration information used on the switch supervisor to manage the Fabric Extender, perform one of the following commands:

PurposeCommand or Action

show environment fex {all | FEX-number}

Displays the environmental sensor status.

[temperature | power | fan]

show inventory fex FEX-number

show module fex [ FEX-number ]

show sprom fex FEX-number {all | backplane | powersupply ps-num} | all

Displays inventory information for a Fabric Extender.

Displays module information about a Fabric Extender.

Displays the contents of the serial PROM (SPROM) on the Fabric Extender.

Configuration Examples for Chassis Management

This example shows how to display the module information about all connected Fabric Extender units:

switch# show module fex

FEX Mod Ports Card Type Model Status.

--- --- ----- ---------------------------------- ------------------ ----------101 1 48 Fabric Extender 48x1GE + 4x10G M N2K-C2248TP-1GE ok

FEX Mod Sw Hw World-Wide-Name(s) (WWN)

--- --- -------------- ------ ----------------------------------------------101 1 5.1(1) 3.5 -FEX Mod MAC-Address(es) Serial-Num

--- --- -------------------------------------- ---------101 1 5475.d0a9.4940 to 5475.d0a9.496f JAF1418AARL

This example shows how to display the inventory information about a specific Fabric Extender:

switch# show inventory fex 101 NAME: "FEX 101 CHASSIS", DESCR: "N2K-C2248TP-1GE CHASSIS" PID: N2K-C2248TP-1GE , VID: V00 , SN: SSI13380FSM

OL-25816-02 29

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Verifying the Chassis Management Information

NAME: "FEX 101 Module 1", DESCR: "Fabric Extender Module: 48x1GE, 4x10GE Supervisor" PID: N2K-C2248TP-1GE , VID: V00 , SN: JAF1339BDSK

NAME: "FEX 101 Fan 1", DESCR: "Fabric Extender Fan module" PID: N2K-C2248-FAN , VID: N/A , SN: N/A

NAME: "FEX 101 Power Supply 2", DESCR: "Fabric Extender AC power supply" PID: NXK-PAC-400W , VID: 000, SN: LIT13370QD6

This example shows how to display the environment status for a specific Fabric Extender:

switch# show environment fex 101

Temperature Fex 101:

----------------------------------------------------------------Module Sensor MajorThresh MinorThres CurTemp Status

----------------------------------------------------------------1 Outlet-1 60 50 33 ok 1 Outlet-2 60 50 38 ok 1 Inlet-1 50 40 35 ok 1 Die-1 100 90 44 ok

Fan Fex: 101:

-----------------------------------------------------Fan Model Hw Status

-----------------------------------------------------Chassis N2K-C2148-FAN -- ok PS-1 -- -- absent PS-2 NXK-PAC-400W -- ok

Configuring the Fabric Extender

(Celsius) (Celsius) (Celsius)

Power Supply Fex 101:

--------------------------------------------------------------------------Voltage: 12 Volts

----------------------------------------------------PS Model Power Power Status

----------------------------------------------------1 -- -- -- -2 NXK-PAC-400W 4.32 0.36 ok

Mod Model Power Power Power Power Status

--- ------------------- ------- ---------- --------- ---------- ---------1 N2K-C2248TP-1GE 0.00 0.00 0.00 0.00 powered-up

Power Usage Summary:

-------------------Power Supply redundancy mode: redundant

Total Power Capacity 4.32 W

Power reserved for Supervisor(s) 0.00 W Power currently used by Modules 0.00 W

Total Power Available 4.32 W

(Watts) (Amp)

Requested Requested Allocated Allocated (Watts) (Amp) (Watts) (Amp)

-------------

This example shows how to display the SPROM for a specific Fabric Extender:

switch# show sprom fex 101 all DISPLAY FEX 101 SUP sprom contents Common block:

Block Signature : 0xabab Block Version : 3 Block Length : 160

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

30 OL-25816-02

Configuring the Fabric Extender

Block Checksum : 0x1a1e EEPROM Size : 65535 Block Count : 3 FRU Major Type : 0x6002 FRU Minor Type : 0x0 OEM String : Cisco Systems, Inc. Product Number : N2K-C2248TP-1GE Serial Number : JAF1339BDSK Part Number : 73-12748-01 Part Revision : 11 Mfg Deviation : 0 H/W Version : 0.103 Mfg Bits : 0 Engineer Use : 0 snmpOID : 9.12.3.1.9.78.3.0 Power Consump : 1666 RMA Code : 0-0-0-0 CLEI Code : XXXXXXXXXTBDV00 VID : V00

Supervisor Module specific block:

Block Signature : 0x6002 Block Version : 2 Block Length : 103 Block Checksum : 0x2686 Feature Bits : 0x0 HW Changes Bits : 0x0 Card Index : 11016 MAC Addresses : 00-00-00-00-00-00 Number of MACs : 0 Number of EPLD : 0 Port Type-Num : 1-48;2-4 Sensor #1 : 60,50 Sensor #2 : 60,50 Sensor #3 : -128,-128 Sensor #4 : -128,-128 Sensor #5 : 50,40 Sensor #6 : -128,-128 Sensor #7 : -128,-128 Sensor #8 : -128,-128 Max Connector Power: 4000 Cooling Requirement: 65 Ambient Temperature: 40

DISPLAY FEX 101 backplane sprom contents: Common block:

Block Signature : 0xabab Block Version : 3 Block Length : 160 Block Checksum : 0x1947 EEPROM Size : 65535 Block Count : 5 FRU Major Type : 0x6001 FRU Minor Type : 0x0 OEM String : Cisco Systems, Inc. Product Number : N2K-C2248TP-1GE Serial Number : SSI13380FSM Part Number : 68-3601-01 Part Revision : 03 Mfg Deviation : 0 H/W Version : 1.0 Mfg Bits : 0 Engineer Use : 0 snmpOID : 9.12.3.1.3.914.0.0 Power Consump : 0 RMA Code : 0-0-0-0 CLEI Code : XXXXXXXXXTDBV00 VID : V00

Chassis specific block:

Block Signature : 0x6001 Block Version : 3 Block Length : 39 Block Checksum : 0x2cf Feature Bits : 0x0

Verifying the Chassis Management Information

OL-25816-02 31

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

Verifying the Chassis Management Information

HW Changes Bits : 0x0 Stackmib OID : 0 MAC Addresses : 00-0d-ec-e3-28-00 Number of MACs : 64 OEM Enterprise : 0 OEM MIB Offset : 0 MAX Connector Power: 0

WWN software-module specific block:

Block Signature : 0x6005 Block Version : 1 Block Length : 0 Block Checksum : 0x66

wwn usage bits:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

License software-module specific block:

Block Signature : 0x6006 Block Version : 1 Block Length : 16 Block Checksum : 0x86f

lic usage bits:

ff ff ff ff ff ff ff ff

DISPLAY FEX 101 power-supply 2 sprom contents: Common block:

Block Signature : 0xabab Block Version : 3 Block Length : 160 Block Checksum : 0x1673 EEPROM Size : 65535 Block Count : 2 FRU Major Type : 0xab01 FRU Minor Type : 0x0 OEM String : Cisco Systems Inc NXK-PAC-400W Product Number : NXK-PAC-400W Serial Number : LIT13370QD6 Part Number : 341 Part Revision : -037 CLEI Code : 5-01 01 000 VID : 000 snmpOID : 12336.12336.12336.12336.12336.12336.12374.12336 H/W Version : 43777.2 Current : 36

Configuring the Fabric Extender

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide

32 OL-25816-02

Configuring the Fabric Extender

RMA Code : 200-32-32-32

Power supply specific block:

Block Signature : 0x0 Block Version : 0 Block Length : 0 Block Checksum : 0x0 Feature Bits : 0x0 Current 110v : 36 Current 220v : 36 Stackmib OID : 0

Additional References

This section includes additional information that is related to configuring the Cisco Nexus 2000 Series Fabric Extender.

Cisco SD2008T-NA, SGE2000, SLM2024T-NA, Nexus 2000 Series Configuration Manual

Specifications and Main Features

Frequently Asked Questions

User Manual