Cisco Systems DSC9148D8G48PK9 User Manual

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

Cisco MDS NX-OS Release 6.2(1) April 2013

Americas Headquarters

Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000

800 553-NETS (6387)

Fax: 408 527-0883

Text Part Number: OL-29284-01

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL:

www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership

relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

IMPLIED, INCLUDING, WITHOUT

New and Changed Information xiii

Preface xv

Audience xv

Organization xv

Document Conventions xvi

Related Documentation xvii

Release Notes xvii Regulatory Compliance and Safety Information xvii Compatibility Information xvii Hardware Installation xvii Software Installation and Upgrade xvii Cisco NX-OS xvii Command-Line Interface xviii Intelligent Storage Networking Services Configuration Guides xviii Troubleshooting and Reference xviii

Obtaining Documentation and Submitting a Service Request xviii

CHAPTER

OL-29284-01, Release 6.x

1 Interfaces Overview 1-1

2 Configuring Interfaces 2-1

Trunks and PortChannels 1-1

Fibre Channel Port Rate Limiting 1-1

Extended Credits 1-2

N Port Virtualization 1-2

FlexAttach 1-2

Information About Interfaces 2-1

Interface Description 2-2 Interface Modes 2-2

E Port 2-3 F Port 2-3 FL Port 2-3 NP Ports 2-3 TL Port 2-4

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

iii

Contents

TE Port 2-4 TF Port 2-4 TNP Port 2-5 SD Port 2-5 ST Port 2-5 Fx Port 2-5 B Port 2-5 Auto Mode 2-5

Interface States 2-6

Administrative States 2-6 Operational States 2-6

Reason Codes 2-6 Graceful Shutdown 2-9 Port Administrative Speeds 2-10

Autosensing 2-10 Frame Encapsulation 2-10 Beacon LEDs 2-11 Speed LEDs 2-11 Bit Error Thresholds 2-11 SFP Transmitter Types 2-12 TL Ports 2-13 TL Port ALPA Caches 2-14 Port Guard 2-14 Port Monitor 2-15 Port Monitor Port Guard 2-16 Port Group Monitor 2-16 Local Switching 2-16 Slow Drain Device Detection and Congestion Avoidance 2-17 Management Interfaces 2-17 VSAN Interfaces 2-18

Prerequisites for Interfaces 2-18

Guidelines and Limitations 2-18

Generation 1 Interface Configuration Guidelines 2-18 Private Loop Configuration Guidelines 2-19 VSAN Interface Configuration Guidelines 2-19

Default Settings 2-20

Configuring Interfaces 2-20

Configuring Fibre Channel Interfaces 2-21 Setting the Interface Administrative State 2-22

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Configuring Interface Modes 2-22 Configuring System Default Port Mode F 2-23 Configuring ISL between Two Switches 2-24 Configuring 10-Gbps FC Mode 2-24 Configuring Port Administrative Speeds 2-25 Configuring Port Speed Group 2-26 Configuring the Interface Description 2-26 Specifying a Port Owner 2-27 Configuring Beacon Mode 2-27 Disabling Bit Error Threshold 2-28 Configuring Switch Port Attribute Default Values 2-28 Configuring TL Ports 2-29 Manually Inserting Entries into the ALPA Cache 2-29 Clearing the ALPA Cache 2-29 Configuring Port Guard 2-30 Configuring Port Monitor 2-31

Enabling Port Monitor 2-31 Configuring a Port Monitor Policy 2-32

Activating a Port Monitor Policy 2-34 Configuring a Port Monitor Port Guard 2-34 Configuring Port Group Monitor 2-34

Enabling Port Group Monitor 2-35

Configuring a Port Group Monitor Policy 2-35

Reverting to the Default Policy for a Specific Counter 2-36

Turning Off the Monitoring of Specific Counter 2-36

Activating a Port Group Monitor Policy 2-37 Configuring Management Interfaces 2-37 Creating VSAN Interfaces 2-38 Configuring Slow Drain Device Detection and Congestion Avoidance 2-38

Configuring Congestion Frame Timeout Value 2-39

Configuring Stuck Frame Timeout Value 2-39

Configuring No-Credit Timeout Value 2-39

Configuring Credit Loss Recovery Threshold and Action 2-40

Configuring Average Credit Nonavailable Duration Threshold and Action 2-41

Contents

OL-29284-01, Release 6.x

Verifying Interfaces Configuration 2-41

Displaying Interface Information 2-42 Displaying TL Port Information 2-50 Displaying the ALPA Cache Contents 2-51 Displaying Port Monitor Status and Policies 2-51 Displaying Port Group Monitor Status and Policies 2-53

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

Contents

Displaying Management Interface Configuration 2-55 Displaying VSAN Interface Information 2-55

CHAPTER

3 Configuring Fibre Channel Interfaces 3-1

Information About Fibre Channel Interfaces 3-1

Generations of Modules and Switches 3-1 Port Groups 3-3 Port Rate Modes 3-5

Dedicated Rate Mode 3-7 Shared Rate Mode 3-8

Dedicated Rate Mode Configurations for the 8-Gbps Modules 3-9 Port Speed 3-10 Dynamic Bandwidth Management 3-10 Out-of-Service Interfaces 3-11 Oversubscription Ratio Restrictions 3-11 Bandwidth Fairness 3-17

Upgrade or Downgrade Scenario 3-17

Guidelines and Limitations 3-18

Combining Generation 1, Generation 2, Generation 3, and Generation 4 Modules 3-18 Local Switching Limitations 3-19 Port Index Limitations 3-19 PortChannel Limitations 3-21

Default Settings 3-25

Configuring Fibre Channel Interfaces 3-26

Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode 3-26 Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode 3-27 Task Flow for Configuring 12-Port 4-Gbps Module Interfaces 3-28 Task Flow for Configuring 4-Port 10-Gbps Module Interfaces 3-28 Configuring Port Speed 3-29 Configuring Rate Mode 3-30 Displaying the Rate Mode Configuration for Interfaces 3-31 Configuring Local Switching 3-36 Disabling Restrictions on Oversubscription Ratios 3-37 Enabling Restrictions on Oversubscription Ratios 3-39 Enabling Bandwidth Fairness 3-40 Disabling Bandwidth Fairness 3-41 Taking Interfaces Out of Service 3-41 Releasing Shared Resources in a Port Group 3-42 Disabling ACL Adjacency Sharing for System Image Downgrade 3-43

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Verifying Fibre Channel Interfaces Configuration 3-43

Displaying Interface Capabilities 3-44 Displaying SFP Diagnostic Information 3-45

Configuration Examples for Fibre Channel Interfaces 3-45

Configuration Example for 48-Port 8-Gbps Module Interfaces 3-46 Configuration Example for 24-Port 8-Gbps Module Interfaces 3-46 Configuration Example for 4/44-Port 8-Gbps Module Interfaces 3-47 Configuration Example for 48-Port 4-Gbps Module Interfaces 3-48 Configuration Example for 24-Port 4-Gbps Module Interfaces 3-49

Contents

CHAPTER

4 Configuring Interface Buffers 4-1

Information About Interface Buffers 4-1

Buffer-to-Buffer Credits 4-1 Performance Buffers 4-2 Buffer Pools 4-2 BB_Credit Buffers for Switching Modules 4-5

Configuring Buffer Credits on a Generation 2, Generation 3 or Generation 4 Module 4-5 48-Port 8-Gbps Advanced Fibre Channel Module BB_Credit Buffers 4-6 48-Port 8-Gbps Fibre Channel Module BB_Credit Buffers 4-7 24-Port 8-Gbps Fibre Channel Module BB_Credit Buffers 4-8 4/44-Port 8-Gbps Host-Optimized Fibre Channel Module BB_Credit Buffers 4-9 48-Port 4-Gbps Fibre Channel Module BB_Credit Buffers 4-10 24-Port 4-Gbps Fibre Channel Module BB_Credit Buffers 4-12 18-Port Fibre Channel/4-Port Gigabit Ethernet Multiservice Module BB_Credit Buffers 4-13 12-Port 4-Gbps Switching Module BB_Credit Buffers 4-13 4-Port 10-Gbps Switching Module BB_Credit Buffers 4-15

BB_Credit Buffers for Fabric Switches 4-16

Cisco MDS 9148 Fabric Switch BB_Credit Buffers 4-16 Cisco MDS 9134 Fabric Switch BB_Credit Buffers 4-16 Cisco MDS 9124 Fabric Switch BB_Credit Buffers 4-17 Cisco MDS 9222i Multiservice Modular Switch BB_Credit Buffers 4-17

Extended BB_Credits 4-17

Extended BB_credits on Generation 1 Switching Modules 4-18

Extended BB_credits on Generation 2 and Generation 3 Switching Modules 4-19 Buffer-to-Buffer Credit Recovery 4-20 Buffer-to-Buffer State Change Number 4-20 Receive Data Field Size 4-21

OL-29284-01, Release 6.x

Configuring Interface Buffers 4-21

Configuring Buffer-to-Buffer Credits 4-21

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

vii

Contents

Configuring Performance Buffers 4-22 Configuring Extended BB_credits 4-23 Enabling Buffer-to-Buffer Credit Recovery 4-24 Enabling the Buffer-to-Buffer State Change Number 4-24 Configuring Receive Data Field Size 4-24

Verifying BB_Credit Configuration 4-25

CHAPTER

5 Configuring Trunking 5-1

Information About Trunking 5-1

Trunking E Ports 5-1 Trunking F Ports 5-2 Key Concepts 5-3 Trunking Protocols 5-3 Trunk Modes 5-4 Trunk-Allowed VSAN Lists and VF_IDs 5-5

Guidelines and Limitations 5-7

General Guidelines and Limitations 5-7 Upgrade and Downgrade Limitations 5-8 Difference Between TE Ports and TF-TNP Ports 5-8 Trunking Misconfiguration Examples 5-10

Default Settings 5-11

Configuring Trunking 5-11

Enabling the Cisco Trunking and Channeling Protocols 5-11 Enabling the F Port Trunking and Channeling Protocol 5-12 Configuring Trunk Mode 5-12 Configuring an Allowed-Active List of VSANs 5-12

CHAPTER

viii

Verifying Trunking Configuration 5-13

Configuration Example for F Port Trunking 5-14

6 Configuring PortChannels 6-1

Information About PortChannels 6-1

PortChannels Overview 6-2 E PortChannels 6-2 F and TF PortChannels 6-3 PortChanneling and Trunking 6-3 Load Balancing 6-4 PortChannel Modes 6-6 PortChannel Deletion 6-7 Interfaces in a PortChannel 6-7

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Interface Addition to a PortChannel 6-8

Forcing an Interface Addition 6-9

Interface Deletion from a PortChannel 6-9 PortChannel Protocols 6-9 Channel Group Creation 6-10 Autocreation 6-11 Manually Configured Channel Groups 6-12

Prerequisites for PortChannels 6-12

Guidelines and Limitations 6-13

General Guidelines and Limitations 6-13 Generation 1 PortChannel Limitations 6-14 F and TF PortChannel Limitations 6-14 Valid and Invalid PortChannel Examples 6-15

Default Settings 6-16

Configuring PortChannels 6-17

Configuring PortChannels Using the WizardCreating a PortChannel 6-17 Configuring the PortChannel Mode 6-17 Deleting PortChannels 6-18 Adding an Interface to a PortChannel 6-18 Forcing an Interface Addition 6-19 Deleting an Interface from a PortChannel 6-19 Enabling and Configuring Autocreation 6-20 Converting to Manually Configured Channel Groups 6-20

Contents

CHAPTER

OL-29284-01, Release 6.x

7 Configuring N Port Virtualization 7-1

Verifying PortChannel Configuration 6-20

Configuration Examples for F and TF PortChannels 6-24

Information About N Port Virtualization 7-1

NPV Overview 7-1 N Port Identifier Virtualization 7-2 N Port Virtualization 7-2 NPV Mode 7-4 NP Ports 7-5 NP Links 7-5

Internal FLOGI Parameters 7-5 Default Port Numbers 7-6 NPV CFS Distribution over IP 7-7 NPV Traffic Management 7-7

Auto 7-7

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

Contents

Traffic Map 7-7 Disruptive 7-8

Multiple VSAN Support 7-8

Guidelines and Limitations 7-8

NPV Guidelines and Requirements 7-8 NPV Traffic Management Guidelines 7-9 DPVM Configuration Guidelines 7-9 NPV and Port Security Configuration Guidelines 7-10

Configuring N Port Virtualization 7-10

Enabling N Port Identifier Virtualization 7-10 Configuring NPV 7-10 Configuring NPV Traffic Management 7-12

Configuring List of External Interfaces per Server Interface 7-12 Enabling the Global Policy for Disruptive Load Balancing 7-13

Verifying NPV Configuration 7-13

Verifying NPV 7-14 Verifying NPV Traffic Management 7-15

CHAPTER

8 Configuring FlexAttach Virtual pWWN 8-1

Information About FlexAttach Virtual pWWN 8-1

FlexAttach Virtual pWWN 8-1 Difference Between San Device Virtualization and FlexAttach Port Virtualization 8-2 FlexAttach Virtual pWWN CFS Distribution 8-2 Security Settings for FlexAttach Virtual pWWN 8-3

Guidelines and Limitations 8-3

Configuring FlexAttach Virtual pWWN 8-3

Automatically Assigning FlexAttach Virtual pWWN 8-3 Manually Assigning FlexAttach Virtual pWWN 8-4 Mapping pWWN to Virtual pWWN 8-4

Verifying FlexAttach Virtual pWWN Configuration 8-5

Verifying the End Device 8-6

Monitoring FlexAttach Virtual pWWN 8-6

9 Configuring Port Tracking 9-1

Information About Port Tracking 9-1

Guidelines and Limitations 9-2

Default Settings 9-2

Configuring Port Tracking 9-3

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Enabling Port Tracking 9-3 Information About Configuring Linked Ports 9-3 Binding a Tracked Port Operationally 9-4 Information About Tracking Multiple Ports 9-4 Tracking Multiple Ports 9-5 Information About Monitoring Ports in a VSAN 9-5 Monitoring Ports in a VSAN 9-5 Information AboutForceful Shutdown 9-6 Forcefully Shutting Down a Tracked Port 9-6

Displaying Port Tracking Information 9-6

9-8

Contents

OL-29284-01, Release 6.x

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

Contents

xii

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

New and Changed Information

As of Cisco MDS NX-OS Release 4.2(1), software configuration information is available in new feature-specific configuration guides for the following information:

• System management

• Interfaces

• Fabric

• Quality of service

• Security

• IP services

• High availability and redundancy

The information in these new guides previously existed in the Cisco MDS 9000 Family CLI Configuration Guide and in the Cisco MDS 9000 Family Fabric Manager Configuration Guide. Those

configuration guides remain available on Cisco.com and should be used for all software releases prior to Fabric Manager Release 5.0(1a). Each guide addresses the features introduced in or available in a particular release. Select and view the configuration guide that pertains to the software installed in your switch.

For a complete list of document titles, see the list of Related Documentation in the “Preface.”

http://www.cisco.com/en/US/products/ps5989/prod_release_notes_list.htm

About This Guide

Table 1 lists the New and Changed features for this guide, starting with MDS NX-OS Release 5.2(1).

Ta b l e 1 New and Changed Features

Feature GUI Change Description

Port Monitor Enhancements

OL-29284-01

Configuring Port Monitor Policy

Added information about the feature Port Monitor Port Guard and three new counters for the port monitor command.

Changed in Release

5.2(2a) Chapter 2, “Configuring

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

Where Documented

Interfaces”

-xiii

Table 1 New and Changed Features (continued)

Feature GUI Change Description

FlexAttach Disable FlexAttach Added information about disabling

FlexAttach.

Port Group Monitoring

Check Oversubscription > Monitor

Added information about monitoring a selected port group.

Enhancements

Changed in Release Where Documented

5.0(1a) Chapter 8, “Configuring

FlexAttach Virtual pWWN”

5.0(1a) Chapter 2, “Configuring

Interfaces”

-xiv

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01

Preface

This preface describes the audience, organization, and conventions of the Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide. It also provides information on how to obtain related

documentation.

Audience

This guide is for experienced network administrators who are responsible for configuring and maintaining the Cisco MDS 9000 Family of multilayer directors and fabric switches.

Organization

The Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide is organized as follows:

Chapter Title Description

Chapter 1 Interfaces Overview Provides an overview of all the features in this

Chapter 2 Configuring Interfaces Explains Generation 1 and Generation 2

Chapter 3 Configuring Fibre Channel Interfaces Explains configuration concepts for Fibre

Chapter 4 Configuring Interface Buffers Explains configuration concepts for Interface

Chapter 5 Configuring Trunking Explains TE ports and trunking concepts.

Chapter 6 Configuring PortChannels Explains PortChannels and load balancing

guide.

module port and operational state concepts in Cisco MDS 9000 Family switches and provides details on configuring ports and interfaces.

Channel module ports and interfaces.

Buffers.

concepts and provides details on configuring PortChannels, adding ports to PortChannels, and deleting ports from PortChannels.

OL-29284-01

Cisco MDS 9000 Family I/O Accelerator Configuration Guide

-xv

Chapter Title Description

Chapter 7 Configuring N Port Virtualization Provides an overview of N Port Virtualization

Chapter 8 Configuring FlexAttach Virtual

pWWN

Document Conventions

Command descriptions use these conventions:

boldface font Commands and keywords are in boldface.

italic font Arguments for which you supply values are in italics.

[ ] Elements in square brackets are optional.

[ x | y | z ] Optional alternative keywords are grouped in brackets and separated by

and includes quidelines and requirements for configuring and verifying NPV.

FlexAttach virtual pWWN feature facilitates server and configuration management. In a SAN environment, the server installation or replacement, requires interaction and coordination among the SAN and server administrators.

vertical bars.

Screen examples use these conventions:

screen font

boldface screen font

Terminal sessions and information the switch displays are in screen font.

Information you must enter is in boldface screen font.

italic screen font Arguments for which you supply values are in italic screen font.

< >

[ ]

!, #

Nonprinting characters, such as passwords, are in angle brackets.

Default responses to system prompts are in square brackets.

An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line.

This document uses the following conventions:

Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the

manual.

Caution Means reader be careful. In this situation, you might do something that could result in equipment

damage or loss of data.

-xvi

Cisco MDS 9000 Family I/O Accelerator Configuration Guide

OL-29284-01

Interfaces Overview

• Trunks and PortChannels, page 1-1

• Fibre Channel Port Rate Limiting, page 1-1

• Extended Credits, page 1-2

• N Port Virtualization, page 1-2

• FlexAttach, page 1-2

Trunks and PortChannels

Trunking, also known as VSAN trunking, is a feature specific to switches in the Cisco MDS 9000 Family. Trunking enables interconnect ports to transmit and receive frames in more than one VSAN, over the same physical link. Trunking is supported on E ports and F ports.

PortChannels aggregate multiple physical ISLs into one logical link with higher bandwidth and port resiliency for both Fibre Channel and FICON traffic. With this feature, up to 16 expansion ports (E-ports) or trunking E-ports (TE-ports) can be bundled into a PortChannel. ISL ports can reside on any switching module, and they do not need a designated master port. If a port or a switching module fails, the PortChannel continues to function properly without requiring fabric reconfiguration.

Cisco NX-OS software uses a protocol to exchange PortChannel configuration information between adjacent switches to simplify PortChannel management, including misconfiguration detection and autocreation of PortChannels among compatible ISLs. In the autoconfigure mode, ISLs with compatible parameters automatically form channel groups; no manual intervention is required.

PortChannels load balance Fibre Channel traffic using a hash of source FC-ID and destination FC-ID, and optionally the exchange ID. Load balancing using PortChannels is performed over both Fibre Channel and FCIP links. Cisco NX-OS software also can be configured to load balance across multiple same-cost FSPF routes.

CHAP T E R

Fibre Channel Port Rate Limiting

The Fibre Channel port rate-limiting feature for the Cisco MDS 9100 Series controls the amount of bandwidth available to individual Fibre Channel ports within groups of four host-optimized ports. Limiting bandwidth on one or more Fibre Channel ports allows the other ports in the group to receive a greater share of the available bandwidth under high-utilization conditions. Port rate limiting is also beneficial for throttling WAN traffic at the source to help eliminate excessive buffering in Fibre Channel and IP data network devices.

OL-29284-01, Release 6.x

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

1-1

Extended Credits

Full line-rate Fibre Channel ports provide at least 255 buffer credits standard. Adding credits lengthens distances for Fibre Channel SAN extension. Using extended credits, up to 4095 buffer credits from a pool of more than 6000 buffer credits for a module can be allocated to ports as needed to greatly extend the distance for Fibre Channel SANs.

N Port Virtualization

Cisco NX-OS software supports industry-standard N port identifier virtualization (NPIV), which allows multiple N port fabric logins concurrently on a single physical Fibre Channel link. HBAs that support NPIV can help improve SAN security by enabling zoning and port security to be configured independently for each virtual machine (OS partition) on a host. In addition to being useful for server connections, NPIV is beneficial for connectivity between core and edge SAN switches.

N port virtualizer (NPV) is a complementary feature that reduces the number of Fibre Channel domain IDs in core-edge SANs. Cisco MDS 9000 family fabric switches operating in the NPV mode do not join a fabric; they only pass traffic between core switch links and end devices, which eliminates the domain IDs for these switches. NPIV is used by edge switches in the NPV mode to log in to multiple end devices that share a link to the core switch. This feature is available only for Cisco MDS Blade Switch Series, the Cisco MDS 9124 Multilayer Fabric Switch, and the Cisco MDS 9134 Multilayer Fabric Switch.

Chapter 1 Interfaces Overview

FlexAttach

Cisco NX-OS supports the FlexAttach feature. One of the main problems in a SAN environment is the time and effort required to install and replace servers. The process involves both SAN and server administrators, and the interaction and coordination between them can make the process time consuming. To alleviate the need for interaction between SAN and server administrators, the SAN configuration should not be changed when a new server is installed or an existing server is replaced. FlexAttach addresses these problems, reducing configuration changes and the time and coordination required by SAN and server administrators when installing and replacing servers. This feature is available only for Cisco MDS 9000 Blade Switch Series, the Cisco MDS 9124, and the Cisco MDS 9134 when NPV mode is enabled.

1-2

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Chapter 1 Interfaces Overview

FlexAttach

OL-29284-01, Release 6.x

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

1-3

FlexAttach

Chapter 1 Interfaces Overview

1-4

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Configuring Interfaces

• Information About Interfaces, page 2-1

• Prerequisites for Interfaces, page 2-18

• Guidelines and Limitations, page 2-18

• Default Settings, page 2-20

• Configuring Interfaces, page 2-20

• Verifying Interfaces Configuration, page 2-41

Information About Interfaces

The main function of a switch is to relay frames from one data link to another. To relay the frames, the characteristics of the interfaces through which the frames are received and sent must be defined. The configured interfaces can be Fibre Channel interfaces, Gigabit Ethernet interfaces, the management interface (mgmt0), or VSAN interfaces.

This section includes the following topics:

• Interface Description, page 2-2

CHAP T E R

OL-29284-01, Release 6.x

• Interface Modes, page 2-2

• Interface States, page 2-6

• Graceful Shutdown, page 2-9

• Port Administrative Speeds, page 2-10

• Frame Encapsulation, page 2-10

• Beacon LEDs, page 2-11

• Speed LEDs, page 2-11

• Bit Error Thresholds, page 2-11

• SFP Transmitter Types, page 2-12

• TL Ports, page 2-13

• TL Port ALPA Caches, page 2-14

• Port Guard, page 2-14

• Port Monitor, page 2-15

• Port Monitor Port Guard, page 2-16

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

2-1

Information About Interfaces

• Port Group Monitor, page 2-16

• Local Switching, page 2-16

• Slow Drain Device Detection and Congestion Avoidance, page 2-17

• Management Interfaces, page 2-17

• VSAN Interfaces, page 2-18

Interface Description

For the Fibre Channel interfaces, you can configure the description parameter to provide a recognizable name for the interface. Using a unique name for each interface allows you to quickly identify the interface when you are looking at a listing of multiple interfaces. You can also use the description to identify the traffic or the use for that interface.

Interface Modes

Each physical Fibre Channel interface in a switch may operate in one of several port modes: E port, F

port, FL port, TL port, TE port, SD port, ST port, and B port (see

interface may be configured in auto or Fx port modes. These two modes determine the port type during interface initialization.

Chapter 2 Configuring Interfaces

Figure 2-1). Besides these modes, each

Figure 2-1 Cisco MDS 9000 Family Switch Port Modes

NL port NL port

N port

ISL link

F port

FL port

Public

loop

NL port NL port

Note Interfaces are created in VSAN 1 by default. See the Cisco MDS 9000 Family NX-OS Fabric

E port E port

Private

loop

TL port

79528

Configuration Guide.

2-2

Each interface has an associated administrative configuration and an operational status:

• The administrative configuration does not change unless you modify it. This configuration has

various attributes that you can configure in administrative mode.

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Chapter 2 Configuring Interfaces

• The operational status represents the current status of a specified attribute like the interface speed.

Note When a module is removed and replaced with the same type of module, the configuration is retained. If

a different type of module is inserted, then the original configuration is no longer retained.

Each interface is briefly described in the sections that follow.

E Port

In expansion port (E port) mode, an interface functions as a fabric expansion port. This port may be connected to another E port to create an Inter-Switch Link (ISL) between two switches. E ports carry frames between switches for configuration and fabric management. They serve as a conduit between switches for frames destined to remote N ports and NL ports. E ports support class 2, class 3, and class F service.

An E port connected to another switch may also be configured to form a PortChannel (see Chapter 6,

“Configuring PortChannels”).

Information About Interfaces

This status cannot be changed and is read-only. Some values may not be valid when the interface is down (for example, the operational speed).

F Port

FL Port

NP Ports

Note We recommend that you configure E ports on 16-port modules. If you must configure an E port on a

32-port oversubscribed module, then you can only use the first port in a group of four ports (for example, ports 1 through 4, 5 through 8, and so forth). The other three ports cannot be used.

In fabric port (F port) mode, an interface functions as a fabric port. This port may be connected to a peripheral device (host or disk) operating as an N port. An F port can be attached to only one N port. F ports support class 2 and class 3 service.

In fabric loop port (FL port) mode, an interface functions as a fabric loop port. This port may be connected to one or more NL ports (including FL ports in other switches) to form a public arbitrated loop. If more than one FL port is detected on the arbitrated loop during initialization, only one FL port becomes operational and the other FL ports enter nonparticipating mode. FL ports support class 2 and class 3 service.

Note FL port mode is not supported on 4-port 10-Gbps switching module interfaces.

OL-29284-01, Release 6.x

An NP port is a port on a device that is in NPV mode and connected to the core switch via an F port. NP ports function like N ports except that in addition to providing N port operations, they also function as proxies for multiple, physical N ports.

For more details about NP ports and NPV, see Chapter 7, “Configuring N Port Virtualization.”

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

2-3

Information About Interfaces

TL Port

Tip We recommend configuring devices attached to TL ports in zones that have up to 64 zone members.

Chapter 2 Configuring Interfaces

In translative loop port (TL port) mode, an interface functions as a translative loop port. It may be connected to one or more private loop devices (NL ports). TL ports are specific to Cisco MDS 9000 Family switches and have similar properties as FL ports. TL ports enable communication between a private loop device and one of the following devices:

• A device attached to any switch on the fabric

• A device on a public loop anywhere in the fabric

• A device on a different private loop anywhere in the fabric

• A device on the same private loop

TL ports support class 2 and class 3 services.

Private loop devices refer to legacy devices that reside on arbitrated loops. These devices are not aware of a switch fabric because they only communicate with devices on the same physical loop (see the

“TL

Port ALPA Caches” section on page 2-14).

TE Port

TF Port

Note TL port mode is not supported on Generation 2 switching module interfaces.

In trunking E port (TE port) mode, an interface functions as a trunking expansion port. It may be connected to another TE port to create an extended ISL (EISL) between two switches. TE ports are specific to Cisco MDS 9000 Family switches. They expand the functionality of E ports to support the following:

• VSAN trunking

• Transport quality of service (QoS) parameters

• Fibre Channel trace (fctrace) feature

In TE port mode, all frames are transmitted in EISL frame format, which contains VSAN information. Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANs across the same physical link. This feature is referred to as trunking in the Cisco MDS 9000 Family switches (see

Chapter 5, “Configuring Trunking”). TE ports support class 2, class 3, and class F service.

In trunking F port (TF port) mode, an interface functions as a trunking expansion port. It may be connected to another trunked N port (TN port) or trunked NP port (TNP port) to create a link between a core switch and an NPV switch or an HBA to carry tagged frames. TF ports are specific to Cisco MDS 9000 Family switches. They expand the functionality of F ports to support VSAN trunking.

In TF port mode, all frames are transmitted in EISL frame format, which contains VSAN information. Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANs across the same physical link. This feature is referred to as trunking in the Cisco MDS 9000 Family (see

“Configuring Trunking”). TF ports support class 2, class 3, and class F service.

Chapter 5,

2-4

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Chapter 2 Configuring Interfaces

TNP Port

In trunking NP port (TNP port) mode, an interface functions as a trunking expansion port. It may be connected to a trunked F port (TF port) to create a link to a core NPIV switch from an NPV switch to carry tagged frames.

SD Port

In SPAN destination port (SD port) mode, an interface functions as a switched port analyzer (SPAN). The SPAN feature is specific to switches in the Cisco MDS 9000 Family. It monitors network traffic that passes though a Fibre Channel interface. This monitoring is done using a standard Fibre Channel analyzer (or a similar switch probe) that is attached to an SD port. SD ports do not receive frames, they only transmit a copy of the source traffic. The SPAN feature is nonintrusive and does not affect switching of network traffic for any SPAN source ports (see the Cisco MDS 9000 Family NX-OS System Management Configuration Guide).

ST Port

In the SPAN tunnel port (ST port) mode, an interface functions as an entry point port in the source switch for the RSPAN Fibre Channel tunnel. The ST port mode and the remote SPAN (RSPAN) feature are specific to switches in the Cisco MDS 9000 Family. When configured in ST port mode, the interface cannot be attached to any device, and thus cannot be used for normal Fibre Channel traffic (see the Cisco MDS 9000 Family NX-OS System Management Configuration Guide).

Information About Interfaces

Fx Port

B Port

Auto Mode

Note ST port mode is not supported on the Cisco MDS 9124 Fabric Switch, the Cisco Fabric Switch for HP

c-Class BladeSystem, and the Cisco Fabric Switch for IBM BladeCenter.

Interfaces configured as Fx ports can operate in either F port or FL port mode. The Fx port mode is determined during interface initialization depending on the attached N port or NL port. This administrative configuration disallows interfaces to operate in any other mode—for example, preventing an interface to connect to another switch.

While E ports typically interconnect Fibre Channel switches, some SAN extender devices, such as the Cisco PA-FC-1G Fibre Channel port adapter, implement a bridge port (B port) model to connect geographically dispersed fabrics. This model uses B ports as described in the T11 Standard FC-BB-2.

If an FCIP peer is a SAN extender device that only supports Fibre Channel B ports, you need to enable the B port mode for the FCIP link. When a B port is enabled, the E port functionality is also enabled and they coexist. If the B port is disabled, the E port functionality remains enabled (see the Cisco MDS 9000 Family NX-OS IP Services Configuration Guide).

OL-29284-01, Release 6.x

Interfaces configured in auto mode can operate in one of the following modes: F port, FL port, E port, TE port, or TF port. The port mode is determined during interface initialization. For example, if the interface is connected to a node (host or disk), it operates in F port or FL port mode depending on the N

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

2-5

Information About Interfaces

port or NL port mode. If the interface is attached to a third-party switch, it operates in E port mode. If the interface is attached to another switch in the Cisco MDS 9000 Family, it may become operational in TE port mode (see

TL ports and SD ports are not determined during initialization and are administratively configured.

Note Fibre Channel interfaces on Storage Services Modules (SSMs) cannot be configured in auto mode.

Interface States

The interface state depends on the administrative configuration of the interface and the dynamic state of the physical link.

Administrative States

The administrative state refers to the administrative configuration of the interface as described in

Table 2-1.

Chapter 2 Configuring Interfaces

Chapter 5, “Configuring Trunking”).

Operational States

Reason Codes

Ta b l e 2-1 Administrative States

Administrative State Description

Up Interface is enabled.

Down Interface is disabled. If you administratively disable an interface by shutting

down that interface, the physical link layer state change is ignored.

The operational state indicates the current operational state of the interface as described in Tab le 2-2.

Ta b l e 2-2 Operational States

Operational State Description

Up Interface is transmitting or receiving traffic as desired. To be in this state, an

interface must be administratively up, the interface link layer state must be up, and the interface initialization must be completed.

Down Interface cannot transmit or receive (data) traffic.

Trunking Interface is operational in TE or TF mode.

2-6

Reason codes are dependent on the operational state of the interface as described in Table 2-3.

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

Chapter 2 Configuring Interfaces

Ta b l e 2-3 Reason Codes for Interface States

Information About Interfaces

Administrative Configuration

Operational Status

Reason Code

Up Up None.

Down Down Administratively down—If you administratively configure an interface

as down, you disable the interface. No traffic is received or transmitted.

Up Down See Tab le 2-4.

Note Only some of the reason codes are listed in Table 2-4.

If the administrative state is up and the operational state is down, the reason code differs based on the nonoperational reason code as described in

Table 2-4.

OL-29284-01, Release 6.x

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

2-7

Information About Interfaces

Ta b l e 2-4 Reason Codes for Nonoperational States

Reason Code (long version) Description

Link failure or not connected The physical layer link is not operational. All

SFP not present The small form-factor pluggable (SFP) hardware is not

Initializing The physical layer link is operational and the protocol

Reconfigure fabric in progress The fabric is currently being reconfigured.

Offline The Cisco NX-OS software waits for the specified

Inactive The interface VSAN is deleted or is in a suspended

Hardware failure A hardware failure is detected.

Error disabled Error conditions require administrative attention.

Chapter 2 Configuring Interfaces

Applicable Modes

plugged in.

initialization is in progress.

R_A_TOV time before retrying initialization.

state.

To make the interface operational, assign that port to a configured and active VSAN.

Interfaces may be error-disabled for various reasons. For example:

• Configuration failure.

• Incompatible buffer-to-buffer credit configuration.

To make the interface operational, you must first fix the error conditions causing this state; and next, administratively shut down or enable the interface.

FC redirect failure A port is isolated because a Fibre Channel redirect is

unable to program routes.

No port activation license available

A port is not active because it does not have a port license.

SDM failure A port is isolated because SDM is unable to program

routes.

2-8

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

OL-29284-01, Release 6.x

+ 206 hidden pages

Cisco Systems DSC9148D8G48PK9 User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

CONTENTS

New and Changed Information

Preface

Interfaces Overview

Trunks and PortChannels

Fibre Channel Port Rate Limiting

Extended Credits

N Port Virtualization

FlexAttach

Configuring Interfaces

Information About Interfaces

Interface Description

Interface Modes

E Port

F Port

FL Port

NP Ports

TL Port

TE Port

TF Port

TNP Port

SD Port

ST Port

Fx Port

B Port

Auto Mode

Interface States

Administrative States

Operational States

Reason Codes