Page 1
Contents
Introduction
Prerequisites
Requirements
Components Used
Configure
Network Diagram
System QoS Policy
Unified Ports for Fibre Channel
Breakout Ports 4x10GE Connectivity
Save the Configuration
Verify
Troubleshoot
Introduction
This document describes how the 3rd generation Cisco Unified Computing System Fabric
Interconnect (UCS-FI-6332-16UP) contains 16x10GE unified ports,18x40GE breakout capable
ports, and 6x40GE ports. Certain configuration changes require a reboot of the fabric interconnect.
Multiple reboots consume valuable time during initial system setup. It explains a process to
complete all configuration changes with a single reboot of each fabric interconnect. Specifically
this document covers configuring System QoS, fibre channel interfaces, and 4x10GE breakout
interfaces. You may as well wish to add additional steps of setting Ethernet and Fibre Channel
(FC) switching modes.
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
Cisco Unified Computing System (UCS) Manager
●
Cisco Unified Computing System (UCS) Manager Command Line Interface (CLI)
●
Components Used
The information in this document is based on these software and hardware versions:
UCS Manager version 3.1(1) or later
●
UCS-FI-6332-16UP
●
UCS-FI-6332 (Does not support Unified Ports)
●
The information in this document was created from the devices in a specific lab environment. All of
the devices used in this document started with a cleared (default) configuration. If your network is
live, make sure that you understand the potential impact of any command.
Page 2
Configure
Network Diagram
These are the configurations of the document,
Set the best-effort class for MTU 9216
●
Configure ports 1/1-1/6 as Fibre Channel
●
Configure ports 1/31-1/34 as 4x10GE Breakout Ports
●
System QoS Policy
Configure the best-effort system QoS policy for jumbo framing. Unlike previous fabric interconnect
generations the 6300 series requires a reload when modifying global QoS policies.
UCS-6332-A# scope eth-server
UCS-6332-A /eth-server # scope qos
UCS-6332-A /eth-server/qos # scope eth-best-effort
UCS-6332-A /eth-server/qos/eth-best-effort # set mtu 9216
UCS-6332-A /eth-server/qos/eth-best-effort *# top
Unified Ports for Fibre Channel
Unified Port are number 1/1-16 on the 6332-16UP. Fibre Channel port modes must be configured
such that the last FC port occupies port id 6,12,16.This results in a minimum of six FC ports and a
maximum of sixteen.
Valid port combinations are:
Ethernet: 1/1-1/16
●
FC: 1/1-1/6 Ethernet: 1/7-1/16
●
FC: 1/1-1/12 Ethernet: 1/13-1/16
●
FC: 1/1-16
●
UCS-6332-A *# scope fc-uplink
UCS-6332-A /fc-uplink *# scope fabric a
UCS-6332-A /fc-uplink/fabric* # create interface 1 1
UCS-6332-A /fc-uplink/fabric/interface* # up
UCS-6332-A /fc-uplink/fabric* # create interface 1 2
UCS-6332-A /fc-uplink/fabric/interface* # up
UCS-6332-A /fc-uplink/fabric* # create interface 1 3
UCS-6332-A /fc-uplink/fabric/interface* # up
UCS-6332-A /fc-uplink/fabric* # create interface 1 4
UCS-6332-A /fc-uplink/fabric/interface* # up
UCS-6332-A /fc-uplink/fabric* # create interface 1 5
UCS-6332-A /fc-uplink/fabric/interface* # up
UCS-6332-A /fc-uplink/fabric* # create interface 1 6
Page 3
UCS-6332-A /fc-uplink/fabric/interface* # top
Breakout Ports 4x10GE Connectivity
Breakout ports are used to connect 40GE interfaces to 10GE capable equipment.
Ethernet 1/17-1/34 are capable of breakout configuration
●
Ethernet 1/35-1/40 are fixed at 40GE
●
A individual 1x10GE interface in a breakout port is capable of FCoE
●
Note: If jumbo QoS is configured globally then only 4 physical ports can be configured for
breakout.
UCS-6332-A *# scope cabling
UCS-6332-A /cabling *# scope fabric a
UCS-6332-A /cabling/fabric *# create breakout 1 31
Warning: Port breakout create action reboots FI and any existing configurations on 40G port will
be erased.!
UCS-6332-A /cabling/fabric/breakout* # up
UCS-6332-A /cabling/fabric # create breakout 1 32
UCS-6332-A /cabling/fabric/breakout* # up
UCS-6332-A /cabling/fabric # create breakout 1 33
UCS-6332-A /cabling/fabric/breakout* # up
UCS-6332-A /cabling/fabric # create breakout 1 34
UCS-6332-A /fc-uplink/fabric/interface* # top
Save the Configuration
Once the changes are committed reboot Fabric Interconnect A.
UCS-6332-A* # commit-buffer
*The switch will now reboot.
Repeat the changes on Fabric Interconnect B.
Verify
Confirm ports FC 1/1-6 are configured for Fibre Channel and ports Ethernet 1/31-34 are in
Breakout mode. Notice the breakout ports now have 4 subinterfaces. In this example, ports 1/33/1
and 1/33/2 are configured for FCoE and 1/33/3-4 as Uplink interfaces.
UCS-6332-A# scope fabric-interconnect a
UCS-6332-A /fabric-interconnect # show port
Ether Port:
Slot Aggr Port Port Oper State Mac Role Xcvr
----- ---------- ----- ---------------- -------------------- ------- ---[...]
1 0 30 Sfp Not Present 8C:60:4F:BC:C4:D0 Unknown N/A
1 0 35 Sfp Not Present 8C:60:4F:BC:C4:E4 Unknown N/A
1 0 36 Sfp Not Present 8C:60:4F:BC:C4:E5 Unknown N/A
1 0 37 Sfp Not Present 8C:60:4F:BC:C4:E6 Unknown N/A
1 0 38 Sfp Not Present 8C:60:4F:BC:C4:E7 Unknown N/A
1 0 39 Sfp Not Present 8C:60:4F:BC:C4:E8 Unknown N/A
1 0 40 Sfp Not Present 8C:60:4F:BC:C4:E9 Unknown N/A
1 31 1 Sfp Not Present 8C:60:4F:BC:C4:D4 Unknown N/A
1 31 2 Sfp Not Present 8C:60:4F:BC:C4:D5 Unknown N/A
Page 4
1 31 3 Sfp Not Present 8C:60:4F:BC:C4:D6 Unknown N/A
1 31 4 Sfp Not Present 8C:60:4F:BC:C4:D7 Unknown N/A
1 32 1 Sfp Not Present 8C:60:4F:BC:C4:D8 Unknown N/A
1 32 2 Sfp Not Present 8C:60:4F:BC:C4:D9 Unknown N/A
1 32 3 Sfp Not Present 8C:60:4F:BC:C4:DA Unknown N/A
1 32 4 Sfp Not Present 8C:60:4F:BC:C4:DB Unknown N/A
1 33 1 Up 8C:60:4F:BC:C4:DC Fcoe Uplink
QSFP 40G SR4
1 33 2 Up 8C:60:4F:BC:C4:DD Fcoe Uplink
QSFP 40G SR4
1 33 3 Up 8C:60:4F:BC:C4:DE Network N/A
1 33 4 Up 8C:60:4F:BC:C4:DF Network N/A 1 34 1 Sfp Not Present
8C:60:4F:BC:C4:E0 Unknown N/A 1 34 2 Sfp Not Present 8C:60:4F:BC:C4:E1 Unknown N/A 1 34 3 Sfp
Not Present 8C:60:4F:BC:C4:E2 Unknown N/A 1 34 4 Sfp Not Present 8C:60:4F:BC:C4:E3 Unknown N/A
FC Port:
Slot Port Oper State Wwn
----- ----- ---------------- -- 1 1 Up 20:01:8C:60:4F:BC:C4:80
1 2 Up 20:02:8C:60:4F:BC:C4:80
1 3 Sfp Not Present 20:03:8C:60:4F:BC:C4:80
1 4 Sfp Not Present 20:04:8C:60:4F:BC:C4:80
1 5 Sfp Not Present 20:05:8C:60:4F:BC:C4:80
1 6 Sfp Not Present 20:06:8C:60:4F:BC:C4:80
In NXOS, breakout ethernet ports are named as br-ethernet x/y/z.
UCS-6332-A# # connect nxos a
UCS-6332-A(nxos)# show int br-ethernet 1/33/1
Br-Ethernet1/33/1 is up
Dedicated Interface
Hardware: 10000 Ethernet, address: 8c60.4fbc.c4dc (bia 8c60.4fbc.c4dc)
Description: C: FcoeUplink
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
[...]
Troubleshoot
There is currently no specific troubleshooting information available for this configuration.
Loading...