Cisco SD2008T-NA, SGE2000, SLM2024T-NA, Nexus 2000 Series Configuration Manual
Cisco Nexus 2000 Series NX-OS Fabric Extender Software
Configuration Guide
First Published: May 03, 2012
Last Modified: August 22, 2012
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
Text Part Number: OL-25816-02
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version
of the UNIX operating system. All rights reserved. Copyright©1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://
www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown
for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
©
2012 Cisco Systems, Inc. All rights reserved.
CONTENTS
Preface
CHAPTER 1
Preface v
Audience v
Document Conventions v
Related Documentation for Cisco Nexus 7000 Series NX-OS Software vii
Documentation Feedback ix
Obtaining Documentation and Submitting a Service Request ix
Overview 1
Information About the Cisco Nexus 2000 Series Fabric Extender 2
Fabric Extender Terminology 2
Fabric Interface Features 3
Host Interfaces 3
Layer 3 Host Interfaces 3
Layer 2 Host Interfaces 3
Host Interface Port Channels 4
Layer 3 Host Interface Port Channels 4
Layer 2 Host Interface Port Channels 4
VLANs 5
Protocol Offload 5
Quality of Service 5
Access Control Lists 5
IGMP Snooping 6
Switched Port Analyzer 6
Oversubscription 6
Management Model 7
Forwarding Model 8
Port Channel Fabric Interface Connection 9
OL-25816-02 iii
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
Contents
Port Numbering Convention 9
Fabric Extender Image Management 10
Licensing Requirements for the Fabric Extender 10
Guidelines and Limitations 10
Configuration Limits 12
Default Settings 12
CHAPTER 2
Configuring the Fabric Extender 13
Configuring the Cisco Nexus 2000 Series Fabric Extender 13
Managing the Fabric Extender Feature Set 13
Installing the Fabric Extender Feature Set 14
Enabling the Fabric Extender Feature Set 15
Disallowing the Fabric Extender Feature Set 15
Associating a Fabric Extender to a Fabric Interface 16
Associating a Fabric Extender to a Port Channel 17
Disassociating a Fabric Extender From an Interface 19
Associating a Fabric Extender to an F2 Module 19
Configuring Fabric Extender Global Features 21
Configuration Examples 23
Configuring a FEX with a Layer 3 Host Interface 24
Configuring a Host Interface in a vPC Topology Connected to Two FEXs 24
Dual-homing of a Server to a FEX with FabricPath 25
Verifying the Configuration 26
Verifying the Fabric Extender Configuration 26
Verifying the Chassis Management Information 29
Additional References 33
Related Documents 33
Feature History 33
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
iv OL-25816-02
Preface
This preface describes the audience, organization, and conventions of the Cisco Nexus 2000 Series Fabric
Extender Software Configuration Guide. It also provides information on how to obtain related documentation.
This chapter includes the following sections:
• Audience, page v
• Document Conventions, page v
• Related Documentation for Cisco Nexus 7000 Series NX-OS Software, page vii
• Documentation Feedback , page ix
• Obtaining Documentation and Submitting a Service Request, page ix
Audience
This publication is for experienced network administrators who configure and maintain Cisco Nexus Series
devices.
Document Conventions
Command descriptions use the following conventions:
bold
Italic
[x | y]
OL-25816-02 v
DescriptionConvention
Bold text indicates the commands and keywords that you enter literally
as shown.
Italic text indicates arguments for which the user supplies the values.
Square brackets enclose an optional element(keyword or argument).[x]
Square brackets enclosing keywords or arguments separated by a vertical
bar indicate an optional choice.
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
Document Conventions
Preface
DescriptionConvention
{x | y}
Braces enclosing keywords or arguments separated by a vertical bar
indicate a required choice.
[x {y | z}]
Nested set of square brackets or braces indicate optional or required
choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.
variable
Indicates a variable for which you supply values, in context where italics
cannot be used.
string
A nonquoted set of characters. Do not use quotation marks around the
string or the string will include the quotation marks.
Examples use the following conventions:
DescriptionConvention
Terminal sessions and information the switch displays are in screen font.screen font
Information you must enter is in boldface screen font.boldface screen font
italic screen font
Arguments for which you supply values are in italic screen font.
Note
Caution
Nonprinting characters, such as passwords, are in angle brackets.< >
Default responses to system prompts are in square brackets.[ ]
!, #
An exclamation point (!) or a pound sign (#) at the beginning of a line
of code indicates a comment line.
This document uses the following conventions:
Means reader take note. Notes contain helpful suggestions or references to material not covered in the
manual.
Means reader be careful. In this situation, you might do something that could result in equipment damage
or loss of data.
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
vi OL-25816-02
Preface
Related Documentation for Cisco Nexus 7000 Series NX-OS Software
Related Documentation for Cisco Nexus 7000 Series NX-OS
Software
The entire Cisco Nexus 7000 Series NX-OS documentation set is available at the following URL:
http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html
Release Notes
The release notes are available at the following URL:
http://www.cisco.com/en/US/products/ps9402/prod_release_notes_list.html
Configuration Guides
These guides are available at the following URL:
http://www.cisco.com/en/US/products/ps9402/products_installation_and_configuration_guides_list.html
The documents in this category include:
•
Cisco Nexus 7000 Series NX-OS Configuration Examples
•
Cisco Nexus 7000 Series NX-OS FabricPath Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Fundamentals Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide
•
Cisco Nexus 7000 Series NX-OS IP SLAs Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide
•
Cisco Nexus 7000 Series NX-OS LISP Configuration Guide
•
Cisco Nexus 7000 Series NX-OS MPLS Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Multicast Routing Configuration Guide
•
Cisco Nexus 7000 Series NX-OS OTV Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Quality of Service Configuration Guide
•
Cisco Nexus 7000 Series NX-OS SAN Switching Guide
•
Cisco Nexus 7000 Series NX-OS Security Configuration Guide
•
Cisco Nexus 7000 Series NX-OS System Management Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Verified Scalability Guide
•
Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Virtual Device Context Quick Start
•
Cisco Nexus 7000 Series NX-OS OTV Quick Start Guide
•
Cisco NX-OS FCoE Configuration Guide for Cisco Nexus 7000 and Cisco MDS 9500
OL-25816-02 vii
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
Related Documentation for Cisco Nexus 7000 Series NX-OS Software
•
Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide
Command References
These guides are available at the following URL:
http://www.cisco.com/en/US/products/ps9402/prod_command_reference_list.html
The documents in this category include:
•
Cisco Nexus 7000 Series NX-OS Command Reference Master Index
•
Cisco Nexus 7000 Series NX-OS FabricPath Command Reference
•
Cisco Nexus 7000 Series NX-OS Fundamentals Command Reference
•
Cisco Nexus 7000 Series NX-OS High Availability Command Reference
•
Cisco Nexus 7000 Series NX-OS Interfaces Command Reference
•
Cisco Nexus 7000 Series NX-OS Layer 2 Switching Command Reference
•
Cisco Nexus 7000 Series NX-OS LISP Command Reference
Preface
•
Cisco Nexus 7000 Series NX-OS MPLS Configuration Guide
•
Cisco Nexus 7000 Series NX-OS Multicast Routing Command Reference
•
Cisco Nexus 7000 Series NX-OS OTV Command Reference
•
Cisco Nexus 7000 Series NX-OS Quality of Service Command Reference
•
Cisco Nexus 7000 Series NX-OS SAN Switching Command Reference
•
Cisco Nexus 7000 Series NX-OS Security Command Reference
•
Cisco Nexus 7000 Series NX-OS System Management Command Reference
•
Cisco Nexus 7000 Series NX-OS Unicast Routing Command Reference
•
Cisco Nexus 7000 Series NX-OS Virtual Device Context Command Reference
•
Cisco NX-OS FCoE Command Reference for Cisco Nexus 7000 and Cisco MDS 9500
Other Software Documents
You can locate these documents starting at the following landing page:
http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html
•
Cisco Nexus 7000 Series NX-OS MIB Quick Reference
•
Cisco Nexus 7000 Series NX-OS Software Upgrade and Downgrade Guide
•
Cisco Nexus 7000 Series NX-OS Troubleshooting Guide
•
Cisco NX-OS Licensing Guide
•
Cisco NX-OS System Messages Reference
•
Cisco NX-OS XML Interface User Guide
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
viii OL-25816-02
Preface
Documentation Feedback
Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments
to nexus7k-docfeedback@cisco.com. We appreciate your feedback.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional information,
see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco
technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed
and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free
service and Cisco currently supports RSS version 2.0.
OL-25816-02 ix
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
Obtaining Documentation and Submitting a Service Request
Preface
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
x OL-25816-02
CHAPTER 1
Overview
This chapter provides an architectural overview of the Cisco Nexus 2000 Series Fabric Extender and includes
the following sections:
• Information About the Cisco Nexus 2000 Series Fabric Extender, page 2
• Fabric Extender Terminology, page 2
• Fabric Interface Features , page 3
• Host Interfaces, page 3
• Host Interface Port Channels, page 4
• VLANs, page 5
• Protocol Offload, page 5
• Quality of Service, page 5
• Access Control Lists, page 5
• IGMP Snooping, page 6
• Switched Port Analyzer, page 6
• Oversubscription, page 6
• Management Model, page 7
• Forwarding Model, page 8
• Port Channel Fabric Interface Connection, page 9
• Port Numbering Convention, page 9
• Fabric Extender Image Management, page 10
• Licensing Requirements for the Fabric Extender, page 10
• Guidelines and Limitations, page 10
• Default Settings, page 12
OL-25816-02 1
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
Overview
Information About the Cisco Nexus 2000 Series Fabric Extender
Information About the Cisco Nexus 2000 Series Fabric Extender
The Cisco Nexus 2000 Series Fabric Extender, also known as FEX, is a highly scalable and flexible server
networking solution that works with Cisco Nexus Series devices to provide high-density, low-cost connectivity
for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and blade
server environments, the Fabric Extender is designed to simplify data center architecture and operations.
The Fabric Extender integrates with its parent switch, a Cisco Nexus Series device, to allow automatic
provisioning and configuration taken from the settings on the parent device. This integration allows large
numbers of servers and hosts to be supported using the same feature set as the parent device, including security
and quality-of-service (QoS) configuration parameters, with a single management domain. The Fabric Extender
and its parent switch enable a large multipath, loop-free, active-active data center topology without the use
of Spanning Tree Protocol (STP).
The Cisco Nexus 2000 Series Fabric Extender forwards all traffic to its parent Cisco Nexus Series device
over 10-Gigabit Ethernet fabric uplinks, which allows all traffic to be inspected by policies established on the
Cisco Nexus Series device.
Note
You must connect the Fabric Extender to its parent Cisco Nexus 7000 Series device equipped with a
32-port, 10-Gigabit M1 module (N7K-M132XP-12), a 32-port 10-Gigabit M1-XL module
(N7K-M132XP-12L), an M2 module, or an F2 module.
No software is included with the Fabric Extender. Software is automatically downloaded and upgraded from
its parent device.
Fabric Extender Terminology
Some terms used in this document are as follows:
• Fabric interface—A 10-Gigabit Ethernet uplink port designated for connection from the Fabric Extender
to its parent switch. A fabric interface cannot be used for any other purpose. It must be directly connected
to the parent switch.
Note
• Port channel fabric interface—A port channel uplink connection from the Fabric Extender to its parent
switch. This connection consists of fabric interfaces bundled into a single logical channel.
• Host interface—An Ethernet host interface for connection to a server or host system.
A fabric interface includes the corresponding interface on the parent switch. This interface
is enabled when you enter the switchport mode fex-fabric command.
Note
• Port channel host interface—A port channel host interface for connection to a server or host system.
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
2 OL-25816-02
Do not connect a bridge or switch to a host interface. These interfaces are designed to
provide end host or server connectivity.
Overview
Fabric Interface Features
The FEX fabric interfaces support static port channels. During the initial discovery and association process,
SFP+ validation and digital optical monitoring (DOM) are performed as follows:
• The FEX performs a local check on the uplink SFP+ transceiver. If it fails the security check, the LED
flashes but the link is still allowed to come up.
• The FEX local check is bypassed if it is running its backup image.
• The parent switch performs SFP validation again when the fabric interface is brought up. It keeps the
fabric interface down if SFP validation fails.
Once an interface on the parent switch is configured in fex-fabric mode, all other features that were configured
on that port and are not relevant to this mode are deactivated. If the interface is reconfigured to remove
fex-fabric mode, the previous configurations are reactivated.
For more information about PFC, see the Cisco Nexus 7000 Series NX-OS Quality of Service Configuration
Guide.
Fabric Interface Features
Host Interfaces
Layer 3 Host Interfaces
Beginning with Cisco NX-OS Release 5.2, by default, all host interfaces on a Fabric Extender connected to
a Cisco Nexus 7000 Series parent switch run in Layer 3 mode.
Note
Layer 2 Host Interfaces
If you have updated the parent switch to Cisco Nexus Release 5.2, previously configured Fabric Extender
host interfaces retain their default port mode, Layer 2. You can change these ports to Layer 3 mode with
the no switchport command.
The host interfaces also support subinterfaces. You can create up to 32 subinterfaces on a Fabric Extender
host interface.
For information about interfaces and subinterfaces, see the Cisco Nexus 7000 Series NX-OS Interfaces
Configuration Guide.
In Cisco NX-OS Release 5.1 and earlier releases, the default port mode is Layer 2.
To run a host interface in Layer 2 mode, use the switchport command. For Cisco NX-OS Release 5.2 and
later releases, to change the port mode to Layer 3, use the no switchport command.
The Fabric Extender provides connectivity for computer hosts and other edge devices in the network fabric.
The following guidelines should be followed when connecting devices to Fabric Extender host interfaces:
• All Fabric Extender host interfaces run as spanning tree edge ports with BPDU Guard enabled and cannot
be configured as spanning tree network ports.
OL-25816-02 3
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
Host Interface Port Channels
• Servers utilizing active/standby teaming, 802.3ad port channels, or other host-based link redundancy
mechanisms can be connected to Fabric Extender host interfaces.
• Any device running spanning tree connected to a Fabric Extender host interface results in that host
interface being placed in an error-disabled state when a BPDU is received.
• Any edge switch that leverages a link redundancy mechanism not dependent on spanning tree such as
Cisco FlexLink or vPC (with the BPDU Filter enabled) can be connected to a Fabric Extender host
interface. Because spanning tree is not used to eliminate loops, you should ensure a loop-free topology
below the Fabric Extender host interfaces.
Ingress and egress packet counters are provided on each host interface.
For more information about BPDU Guard, see the Cisco Nexus 7000 Series NX-OS Layer 2 Switching
Configuration Guide.
Host Interface Port Channels
Overview
Layer 3 Host Interface Port Channels
The Fabric Extender supports host interface port channel configurations. You can combine up to 8 interfaces
in a standard mode port channel and 16 interfaces when configured with the Link Aggregation Control Protocol
(LACP).
Port channel resources are allocated when the port channel has one or more members.Note
All members of the port channel must be Fabric Extender host interfaces and all host interfaces must be from
the same Fabric Extender. You cannot mix interfaces from the Fabric Extender and the parent switch.
Layer 3 mode is supported on host interface port channels.
A host interface port channel also supports subinterfaces. You can create up to 1000 subinterfaces on a Fabric
Extender host interface port channel.
For more information about port channels, see the Cisco Nexus 7000 Series NX-OS Interfaces Configuration
Guide.
Layer 2 Host Interface Port Channels
The Fabric Extender supports host interface port channel configurations. You can combine up to 8 interfaces
in a standard mode port channel and 16 interfaces when configured with the Link Aggregation Control Protocol
(LACP).
Port channel resources are allocated when the port channel has one or more members.Note
All members of the port channel must be Fabric Extender host interfaces and all host interfaces must be from
the same Fabric Extender. You cannot mix interfaces from the Fabric Extender and the parent switch.
Layer 2 mode is supported on host interface port channels.
Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide
4 OL-25816-02
Loading...