Cisco Nexus 2000, Nexus 2248TP, Nexus 2248TP-E, Nexus 2248PQ, Nexus 2232PP Series Manual

Overview

This chapter provides an architectural overview of the Cisco Nexus 2000 Series Fabric Extender and includes
the following sections:

Information About the Cisco Nexus 2000 Series Fabric Extender, page 2

•

Fabric Extender Terminology, page 2

•

Fabric Interface Features , page 3

•

Host Interfaces, page 3

•

Host Interface Port Channels, page 4

•

VLANs, page 6

•

Protocol Offload, page 6

•

Quality of Service, page 6

•

Access Control Lists, page 6

•

IGMP Snooping, page 7

•

Switched Port Analyzer, page 7

•

Oversubscription, page 7

•

Management Model, page 8

•

Forwarding Model, page 9

•

Port Channel Fabric Interface Connection, page 10

•

Port Numbering Convention, page 10

•

Fabric Extender Image Management, page 11

•

Licensing Requirements for the Fabric Extender, page 11

•

Guidelines and Limitations for the Fabric Extender, page 11

•

Default Settings, page 19

•

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series

Switches, Release 5.2

1

Overview

Information About the Cisco Nexus 2000 Series Fabric Extender

Information About the Cisco Nexus 2000 Series Fabric Extender

The Cisco Nexus 2000 Series Fabric Extender, also known as FEX, is a highly scalable and flexible server
networking solution that works with Cisco Nexus Series devices to provide high-density, low-cost connectivity
for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and blade
server environments, the Fabric Extender is designed to simplify data center architecture and operations.

The Fabric Extender integrates with its parent switch, which is a Cisco Nexus Series device, to allow automatic
provisioning and configuration taken from the settings on the parent device. This integration allows large
numbers of servers and hosts to be supported by using the same feature set as the parent device, including
security and quality-of-service (QoS) configuration parameters, with a single management domain. The Fabric
Extender and its parent switch enable a large multipath, loop-free, active-active data center topology without
the use of the Spanning Tree Protocol (STP).

The Cisco Nexus 2000 Series Fabric Extender forwards all traffic to its parent Cisco Nexus Series device
over 10-Gigabit Ethernet fabric uplinks, which allows all traffic to be inspected by policies established on the
Cisco Nexus Series device.

Note

You must connect the Fabric Extender to its parent Cisco Nexus 7000 Series device with a 32-port,
10-Gigabit M1 module (N7K-M132XP-12), a 32-port 10-Gigabit M1-XL module (N7K-M132XP-12L),
an M2 module, or an F2 module.

No software is included with the Fabric Extender. The software is automatically downloaded and upgraded
from its parent device.

Fabric Extender Terminology

Some terms used in this document are as follows:

• Fabric interface—A 10-Gigabit Ethernet uplink port that is designated for connection from the Fabric

Extender to its parent switch. A fabric interface cannot be used for any other purpose. It must be directly
connected to the parent switch.

Note

• Port channel fabric interface—A port channel uplink connection from the Fabric Extender to its parent

switch. This connection consists of fabric interfaces that are bundled into a single logical channel.

• Host interface—An Ethernet host interface for connection to a server or host system.

A fabric interface includes the corresponding interface on the parent switch. This interface
is enabled when you enter the switchport mode fex-fabric command.

Note

• Port channel host interface—A port channel host interface for connection to a server or host system.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series

Switches, Release 5.2

2

Do not connect a bridge or switch to a host interface. These interfaces are designed to
provide end host or server connectivity.

Overview

Fabric Interface Features

The FEX fabric interfaces support static port channels. During the initial discovery and association process,
SFP+ validation and digital optical monitoring (DOM) are performed as follows:

The FEX performs a local check on the uplink SFP+ transceiver. If it fails the security check, the LED

•

flashes but the link is still allowed to come up.

The FEX local check is bypassed if it is running its backup image.

•

The parent switch performs SFP validation again when the fabric interface is brought up. It keeps the

•

fabric interface down if SFP validation fails.

After an interface on the parent switch is configured in fex-fabric mode, all other features that were configured
on that port and are not relevant to this mode are deactivated. If the interface is reconfigured to remove
fex-fabric mode, the previous configurations are reactivated.

For more information about PFC, see the Cisco Nexus 7000 Series NX-OS Quality of Service Configuration
Guide.

Fabric Interface Features

Host Interfaces

Layer 3 Host Interfaces

Beginning with Cisco NX-OS Release 5.2, by default, all host interfaces on a Fabric Extender that are connected
to a Cisco Nexus 7000 Series parent switch run in Layer 3 mode.

Note

Layer 2 Host Interfaces

If you have updated the parent switch to Cisco Nexus Release 5.2, previously configured fabric Extender
host interfaces retain their default port mode, Layer 2. You can change these ports to Layer 3 mode with
the no switchport command.

The host interfaces also support subinterfaces. You can create up to 32 subinterfaces on a Fabric Extender
host interface.

Beginning with Cisco NX-OS Release 6.2, port profiles are supported on the host interfaces of a Fabric
Extender.

For information about interfaces, see the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide.

In Cisco NX-OS Release 5.1 and earlier releases, the default port mode is Layer 2.

To run a host interface in Layer 2 mode, use the switchport command. For Cisco NX-OS Release 5.2 and
later releases, to change the port mode to Layer 3, use the no switchport command.

The Fabric Extender provides connectivity for computer hosts and other edge devices in the network fabric.

Follow these guidelines when connecting devices to Fabric Extender host interfaces:

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series

Switches, Release 5.2

3

Host Interface Port Channels

•

•

•

•

Ingress and egress packet counters are provided on each host interface.

For more information about BPDU Guard, see the Cisco Nexus 7000 Series NX-OS Layer 2 Switching
Configuration Guide.

Overview

All Fabric Extender host interfaces run as spanning tree edge ports with BPDU Guard enabled and you
cannot configure them as spanning tree network ports.

You can connect servers that use active/standby teaming, 802.3ad port channels, or other host-based
link redundancy mechanisms to Fabric Extender host interfaces.

Any device that is running spanning tree connected to a Fabric Extender host interface results in that
host interface being placed in an error-disabled state when a BPDU is received.

You can connect any edge switch that leverages a link redundancy mechanism not dependent on spanning
tree such as Cisco FlexLink or vPC (with the BPDU Filter enabled) to a Fabric Extender host interface.
Because spanning tree is not used to eliminate loops, you should ensure a loop-free topology below the
Fabric Extender host interfaces.

Host Interface Port Channels

Layer 3 Host Interface Port Channels

The Fabric Extender (FEX) supports host interface port channel configurations. You can combine up to 8
interfaces in a standard mode port channel and 16 interfaces when configured with the Link Aggregation
Control Protocol (LACP).

Port channel resources are allocated when the port channel has one or more members.Note

All members of the port channel must be FEX host interfaces and all host interfaces must be from the same
FEX. You cannot mix interfaces from the FEX and the parent switch.

Layer 3 mode is supported on host interface port channels.

A host interface port channel also supports subinterfaces. You can create up to 1000 subinterfaces on a FEX
host interface port channel.

For more information about port channels, see the Cisco Nexus 7000 Series NX-OS Interfaces Configuration
Guide.

Layer 2 Host Interface Port Channels

The Fabric Extender supports host interface port channel configurations. You can combine up to 8 interfaces
in a standard mode port channel and 16 interfaces when configured with the Link Aggregation Control Protocol
(LACP).

Port channel resources are allocated when the port channel has one or more members.Note

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series

Switches, Release 5.2

4

Overview

Minimum Number of Links on a Fabric Port Channel

All members of the port channel must be Fabric Extender host interfaces and all host interfaces must be from
the same Fabric Extender. You cannot mix interfaces from the Fabric Extender and the parent switch.

Layer 2 mode is supported on host interface port channels.

You can configure Layer 2 port channels as access or trunk ports.

Beginning with Cisco NX-OS Release 5.2(1), Fabric Extenders support the host vPC feature where a server
can be dual-attached to two different FEXs through a port channel. You must configure parent switches that
connect each Fabric Extender (one parent switch per FEX) in a vPC domain.

Minimum Number of Links on a Fabric Port Channel

In a network configuration of dual-homed hosts (active/standby), you can configure the Fabric Extender to
support a minimum number of links for fabric port channels (FPCs) with the port-channel min-links command.

When the number of FPC links falls below the specified threshold, the host-facing Cisco Nexus 2000 interfaces
are brought down. This process allows for a NIC switchover on the connection between the host and the FEX.

The automatic recovery of Cisco Nexus 2000 Series interfaces to the standby FEX is triggered when the
number of FPC links reaches the specified threshold.

Load Balancing Using Host Interface Port Channels

The Cisco NX-OS software allows for load balancing traffic across all operational interfaces on a FEX host
interface port-channel by hashing the addresses in the frame to a numerical value that selects one of the links
in the channel. Port-channels provide load balancing by default.

You can configure the type of load-balancing algorithm used. You can choose the load-balancing algorithm
that determines which member port to select for egress traffic by looking at the fields in the frame.

You can configure the load-balancing mode to apply to all Fabric Extenders or to specified ones. If
load-balancing mode is not configured, Fabric Extenders use the default system configuration. The per-FEX
configuration takes precedence over the load-balancing configuration for the entire system. You cannot
configure the load-balancing method per port channel.

Note

The default load-balancing mode for Layer 3 interfaces is the source and destination IP address, and the
default load-balancing mode for non-IP interfaces is the source and destination MAC address. For more
details, see the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide, Release 6.x.

You can configure the device to use one of the following methods to load balance across the port channel:

Destination MAC address

•

Source MAC address

•

Source and destination MAC address

•

Destination IP address

•

Source IP address

•

Source and destination IP address

•

Source TCP/UDP port number

•

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series

Switches, Release 5.2

5

VLANs

Destination TCP/UDP port number

•

Source and destination TCP/UDP port number

•

Dot1Q VLAN number

•

Overview

Note

You must be in the default virtual device context (VDC) to configure load-balancing method for FEX; if
you attempt to configure this feature from another VDC, the system displays an error.

VLANs

The Fabric Extender supports Layer 2 VLAN trunks and IEEE 802.1Q VLAN encapsulation.

For more information about VLANs, see the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration
Guide.

The Fabric Extender does not support private VLANs (PVLANs).Note

Protocol Offload

To reduce the load on the control plane of the Cisco Nexus Series device, Cisco NX-OS allows you to offload
link-level protocol processing to the Fabric Extender CPU. The following protocols are supported:

Link Layer Discovery Protocol (LLDP)

•

Link Aggregation Control Protocol (LACP)

•

Quality of Service

The Fabric Extender uses IEEE 802.1p class of service (CoS) values to associate traffic with the appropriate
class. Per-port quality of service (QoS) configuration is also supported.

Host interfaces support pause frames, which are implemented using IEEE 802.3x link-level flow control
(LLC). By default, flow control send is on and flow control receive is off on all host interfaces. Autonegotiation
is enabled on the host interfaces. Per-class flow control is set according to the QoS classes.

For more information about LLC and quality-of-service, see the Cisco Nexus 7000 Series NX-OS Quality of
Service Configuration Guide.

Access Control Lists

The Fabric Extender supports the full range of ingress access control lists (ACLs) that are available on its
parent Cisco Nexus Series device.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series

Switches, Release 5.2

6