Cisco Nexus 9336PQ, N9K-C9336PQ Hardware Installation Manual

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

First Published: August 13, 2014

Last Modified: February 01, 2016

Americas Headquarters

Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.

The following information is for FCC compliance of Class B devices: This equipment has been tested and found to comply with the limits for a Class B digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiate radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If the equipment causes interference to radio or television reception, which can be determined by turning the equipment off and on, users are encouraged to try to correct the interference by using one or more of the following measures:

Reorient or relocate the receiving antenna.

•

Increase the separation between the equipment and receiver.

•

Connect the equipment into an outlet on a circuit different from that to which the receiver is connected.

•

Consult the dealer or an experienced radio/TV technician for help.

•

Modifications to this product not authorized by Cisco could void the FCC approval and negate your authority to operate the product

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright©1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED "AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://

www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership

relationship between Cisco and any other company. (1110R)

Preface

CHAPTER 1

CHAPTER 2

Preface vii

Audience vii

Documentation Conventions vii

Preface

Audience, page vii

•

Documentation Conventions, page vii

•

Audience

This publication is for hardware installers and network administrators who install, configure, and maintain Cisco Nexus switches.

Documentation Conventions

Command descriptions use the following conventions:

DescriptionConvention

bold

Italic

[x | y]

{x | y}

Bold text indicates the commands and keywords that you enter literally as shown.

Italic text indicates arguments for which the user supplies the values.

Square brackets enclose an optional element (keyword or argument).[x]

Square brackets enclosing keywords or arguments separated by a vertical bar indicate an optional choice.

Braces enclosing keywords or arguments separated by a vertical bar indicate a required choice.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

vii

Documentation Feedback

Cisco Application Centric Infrastructure (ACI) Simulator Documentation

The following Cisco ACI Simulator documentation is available at http://www.cisco.com/c/en/us/support/cloud-systems-management/application-centric-infrastructure-simulator/tsd-products-support-series-home.html.

Cisco Nexus 9000 Series Switches Documentation

The Cisco Nexus 9000 Series Switches documentation is available at http://www.cisco.com/c/en/us/support/switches/nexus-9000-series-switches/tsd-products-support-series-home.html.

Cisco Application Virtual Switch Documentation

The Cisco Application Virtual Switch (AVS) documentation is available at http://www.cisco.com/c/en/us/support/switches/application-virtual-switch/tsd-products-support-series-home.html.

Cisco ACI Simulator Release Notes

•

Cisco ACI Simulator Installation Guide

•

Cisco ACI Simulator Getting Started Guide

•

Preface

Documentation Feedback

To provide technical feedback on this document, or to report an error or omission, please send your comments to apic-docfeedback@cisco.com. We appreciate your feedback.

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What's New in Cisco Product Documentation, at: http://

www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.

Subscribe to What's New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Overview

CHAPTER 1

Overview

Overview, page 1

•

The Cisco Nexus 9336PQ ACI Spine Switch (N9K-C9336PQ) is a 2-rack-unit (RU) switch for the Cisco Application-Centric Infrastructure (ACI). This switch includes the following components:

36 fixed 40-Gigabit Quad Small Form-factor Pluggable (QSFP+) ports

•

Console and out-of-band-management ports (one each)

•

• Power supplies (two—one required for operations and an optional one for redundancy)

Fan modules (two)

•

The following figure shows the hardware features seen on the power supply side of the chassis.

Figure 1: Hardware Features on the Power Supply Side of the Chassis

Fan modules (2)3Grounding Pad1

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Ports

Overview

Power supplies (2) (AC power supply shown)2

The following figure shows the hardware features seen from the port side of the chassis.

Figure 2: Hardware Features on the Port Side of the Chassis

Ports

36 40-Gigabit QSFP+ ports5Chassis LEDs1

Screw holes (2) for attaching a front-mount bracket for four-post racks

6USB ports2

(one bracket on each of two sides)

Notch in both sides of the chassis for locking the power supply end of

port (RJ-45 port)

7Out-of-band management

the chassis to the bottom support brackets

Console port (RJ232 port)4

The switch chassis includes the following types of ports:

40-Gigabit QSFP+ interface ports (36)

•

Console port (1)

•

Out-of-band management port (1)

•

USB flash-memory ports (2)

•

The interface ports support QSFP+ transceivers and are numbered from left to right. You use these ports to connect the switch to Cisco Nexus 9300 switches in the ACI network that this switch belongs to. These ports are numbered 1 to 36 from left to right.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Overview

The console port enables you to perform the initial setup of the switch.

The out-of-band management port enables you to manage switch operations after you set up the switch.

If you need to download or upload switch configurations on a flash drive, you can use the flash drive in one of the two USB ports.

Supported Transceivers and Cables

To determine which transceivers and cables are supported by this switch, see the Cisco Transceiver Modules

Compatibility Information document.

Fan Modules

The switch supports two fan modules, which provide port-side intake or port-side exhaust airflow for cooling. You can hot swap one of the fan modules during operations but must replace it within one minute. If you cannot replace the fan module within one minute, then it is best to leave the failed fan module in the chassis until you are ready to replace it.

The fan modules are labeled FAN 1 on the left and FAN 2 on the right.

Supported Transceivers and Cables

Note

The switch must run with all of its power supply and fan modules taking in cooling air from a cold aisle and exhausting to a hot aisle. If they take in air from the hot aisle, the switch can overheat and shut down. All fan and power supply modules must use the same direction of airflow.

Power Supply Modules

The switch has two power supply slots labeled PS1 on the left and PS2 on the right. You can mix any of the following power supplies in those slots:

1200-W HVAC/HVDC power supplies with white coloring for dual-directional airflow

•

(N9K-PUV-1200W)

1200-W port-side exhaust AC power supply with blue coloring and NEBS compliance

•

(NXA-PAC-1200W-PE)

1200-W port-side intake AC power supply with burgundy coloring and NEBS compliance

•

(NXA-PAC-1200W-PI)

1200-W port-side exhaust AC power supply with blue coloring (N9K-PAC-1200W-B)

•

1200-W port-side intake AC power supply with burgundy coloring (N9K-PAC-1200W)

•

930-W DC power supplies with green coloring for port-side intake airflow (UCSC-PSU-930WDC)

•

930-W DC power supplies with gray coloring for port-side exhaust airflow (UCS-PSU-6332-DC)

•

The switch requires one power supply for its operations and a second power supply for power redundancy. You can hot swap one of the power supplies during operations but otherwise must always have two power supplies (or one power supply and one blank faceplate) installed in the chassis to maintain the designed airflow.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Power Supply Modules

Overview

Caution

The switch must run with all of its power supply and fan modules taking in cooling air from a cold aisle and exhausting to a hot aisle. If they take in air from the hot aisle, the switch can overheat and shut down. All power supply and fan modules must use the same direction of airflow.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

CHAPTER 2

Preparing the Site

Temperature Requirements, page 5

•

Humidity Requirements, page 5

•

Altitude Requirements, page 6

•

Dust and Particulate Requirements, page 6

•

Minimizing Electromagnetic and Radio Frequency Interference, page 6

•

Shock and Vibration Requirements, page 7

•

Grounding Requirements, page 7

•

Planning for Power Requirements, page 7

•

Airflow Requirements, page 8

•

Rack and Cabinet Requirements, page 9

•

Clearance Requirements, page 10

•

Temperature Requirements

The switch requires an operating temperature of 32 to 104° F (0 to 40° C). If the switch is not operating, the temperature must be between –40 to 158° F (–40 to 70° C).

Humidity Requirements

High humidity can cause moisture to enter the switch. Moisture can cause corrosion of internal components and degradation of properties such as electrical resistance, thermal conductivity, physical strength, and size. The switch is rated to operate at 8 to 80 percent relative humidity, with a humidity gradation of 10 percent per hour. For nonoperating conditions, the switch can withstand from 5 to 95 percent relative humidity. Buildings in which the climate is controlled by air-conditioning in the warmer months and by heat during the colder months usually maintain an acceptable level of humidity for the switch equipment. However, if the switch is located in an unusually humid location, you should use a dehumidifier to maintain the humidity within an acceptable range.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Altitude Requirements

If you operate a switch at a high altitude (low pressure), the efficiency of forced and convection cooling is reduced and can result in electrical problems that are related to arcing and corona effects. This condition can also cause sealed components with internal pressure, such as electrolytic capacitors, to fail or to perform at a reduced efficiency. This switch is rated to operate at altitudes from 0 to 13,123 feet (0 to 4,000 meters).

Dust and Particulate Requirements

Exhaust fans cool power supplies and system fans cool switches by drawing in air and exhausting air out through various openings in the chassis. However, fans also ingest dust and other particles, causing contaminant buildup in the switch and increased internal chassis temperature. A clean operating environment can greatly reduce the negative effects of dust and other particles, which act as insulators and interfere with the mechanical components in the switch.

In addition to regular cleaning, follow these precautions to avoid contamination of your switch:

Preparing the Site

Do not permit smoking near the switch.

•

Do not permit food or drink near the switch.

•

Minimizing Electromagnetic and Radio Frequency Interference

Electromagnetic interference (EMI) and radio frequency interference (RFI) from the switch can adversely affect other devices such as radio and television (TV) receivers operating near the switch. Radio frequencies that emanate from the switch can also interfere with cordless and low-power telephones. Conversely, RFI from high-power telephones can cause spurious characters to appear on the switch monitor.

RFI is defined as any EMI with a frequency above 10 kHz. This type of interference can travel from the switch to other devices through the power cable and power source or through the air as transmitted radio waves. The Federal Communications Commission (FCC) publishes specific regulations to limit the amount of EMI and RFI that can be emitted by computing equipment. Each switch meets these FCC regulations.

To reduce the possibility of EMI and RFI, follow these guidelines:

Cover all open expansion slots with a blank filler plate.

•

Always use shielded cables with metal connector shells for attaching peripherals to the switch.

•

When wires are run for any significant distance in an electromagnetic field, interference can occur between the field and the signals on the wires with the following implications:

Bad wiring can result in radio interference emanating from the plant wiring.

•

Strong EMI, especially when it is caused by lightning or radio transmitters, can destroy the signal drivers

•

and receivers in the chassis and even create an electrical hazard by conducting power surges through lines into equipment.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Preparing the Site

Shock and Vibration Requirements

Note

Caution

To predict and prevent strong EMI, you might need to consult experts in radio frequency interference (RFI).

The wiring is unlikely to emit radio interference if you use twisted-pair cable with a good distribution of grounding conductors. If you exceed the recommended distances, use a high-quality twisted-pair cable with one ground conductor for each data signal when applicable.

If the wires exceed the recommended distances, or if wires pass between buildings, give special consideration to the effect of a lightning strike in your vicinity. The electromagnetic pulse caused by lightning or other high-energy phenomena can easily couple enough energy into unshielded conductors to destroy electronic switches. You might want to consult experts in electrical surge suppression and shielding if you had similar problems in the past.

Shock and Vibration Requirements

The switch has been shock- and vibration-tested for operating ranges, handling, and earthquake standards.

Grounding Requirements

The switch is sensitive to variations in voltage supplied by the power sources. Overvoltage, undervoltage, and transients (or spikes) can erase data from the memory or cause components to fail. To protect against these types of problems, ensure that there is an earth-ground connection for the switch. You can connect the grounding pad on the switch either directly to the earth-ground connection or to a fully bonded and grounded rack.

You must provide the grounding cable to make this connection, but you can connect the grounding wire to the switch using a grounding lug that ships with the switch. Size the grounding wire to meet local and national installation requirements. Depending on the power supply and system, a 12-AWG to 6-AWG copper conductor is required for U.S. installations (for those installations, we recommend that you use commercially available 6-AWG wire). The length of the grounding wire depends on the proximity of the switch to proper grounding facilities.

Note

You automatically ground the power supplies when you connect them to power sources. You must also connect the chassis to the facility earth ground.

Planning for Power Requirements

The switch includes two power supplies (1-to-1 redundancy with current sharing) in one of the following combinations (all power supplies must have the same airflow direction as the fan modules):

Two 1200-W HVAC/HVDC power supplies

•

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Airflow Requirements

Two 1200-W NEBS-compliant AC power supplies

•

Two 1200-W standard AC power supplies

•

Two 930-W DC power supplies

•

Any combination of the above power supplies

•

Preparing the Site

Note

For n+1 redundancy, you must provide power to both power supplies. For n+n redundancy, you must provide power to both power supplies and each power supply must be connected to a different power source.

The power supplies are rated to output up to 1200 W (AC, NEBS-compliant AC, and HVAC/HVDC power supplies) or up to 930 W (DC power supplies), but the switch requires less than those amounts of power from the power supply. To operate the switch you must provision enough power from the power source to cover the requirements of both the switch and a power supply. Typically, this switch and a power supply require about 400 W of power input from the power source, but you must provision as much as 660 W power input from the power source to cover peak demand.

To minimize the possibility of circuit failure, make sure that each power-source circuit used by the switch is dedicated to the switch.

For the AC power cables that you can use with this switch, see Power Cord Specifications, on page 47. For DC power cables, the recommended wire gauge is 8 AWG and the minimum wire gauge is 10 AWG.

Airflow Requirements

The switch is designed to be positioned with its ports in either the front or the rear of the rack depending on your cabling and maintenance requirements. Depending on which side of the switch faces the cold aisle, you must have fan and power supply modules that move the coolant air from the cold aisle to the hot aisle in one of the following ways:

• Port-side exhaust airflow—Coolant air enters the chassis through the fan and power supply modules in

the cold aisle and exhausts through the port end of the chassis in the hot aisle.

• Port-side intake airflow—Coolant air enters the chassis through the port end in the cold aisle and exhausts

through the fan and power supply modules in the hot aisle.

• Dual-directional airflow—The airflow direction of the power supplies is determined by the airflow

direction of the fan modules.

You can identify the airflow direction of each fan and power supply module by its coloring as follows:

Blue coloring indicates port-side exhaust airflow.

•

Burgundy coloring indicates port-side intake airflow.

•

Gray coloring on DC power supplies indicates port-side exhaust airflow.

•

Green coloring on DC power supplies indicates port-side intake airflow.

•

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Preparing the Site

Rack and Cabinet Requirements

White coloring on HVAC/HVDC power supplies indicates dual direction airflow.

•

Note

To prevent the switch from overheating and shutting down, you must position the air intake for the switch in a cold aisle, and all of the fan and power supply modules must have the same direction of airflow (even if their coloring is different).

Rack and Cabinet Requirements

You can install the following types of racks or cabinets for your switch:

Standard perforated cabinets

•

Solid-walled cabinets with a roof fan tray (bottom-to-top cooling)

•

Standard open four-post Telco racks

•

To correctly install the switch in a cabinet that is located in a hot-aisle/cold-aisle environment, you should fit the cabinet with baffles to prevent exhaust air from recirculating into the chassis air intake.

Work with your cabinet vendors to determine which of their cabinets meet the following requirements or see the Cisco Technical Assistance Center (TAC) for recommendations:

Use a standard 19-inch (48.3-cm), four-post Electronic Industries Alliance (EIA) cabinet or rack with

•

mounting rails that conform to English universal hole spacing per section 1 of the ANSI/EIA-310-D-1992 standard.

The depth of a four-post rack must be 24 to 32 inches (61.0 to 81.3 cm) between the front and rear

•

mounting brackets.

Warning

Required clearances between the chassis and the edges of its rack or the interior of its cabinet are as

•

follows:

4.5 inches (11.4 cm) between the front of the chassis and the interior of the cabinet (required for

◦

cabling).

3.0 inches (7.6 cm) between the rear of the chassis and the interior of the cabinet (required for

◦

airflow in the cabinet if used).

No clearance is required between the chassis and the sides of the rack or cabinet (no side airflow).

◦

Additionally, you must have power receptacles located within reach of the power cords used with the switch. For the power cord specifications, see the Power Cord Specifications.

Statement 1048—Rack Stabilization

Stability hazard. The rack stabilizing mechanism must be in place, or the rack must be bolted to the floor before you slide the unit out for servicing. Failure to stabilize the rack can cause the rack to tip over.

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Clearance Requirements

You must provide the chassis with adequate clearance between the chassis and any other rack, device, or structure so that you can properly install the chassis, route cables, provide airflow, and maintain the switch. For the clearances required for an installation of this chassis in a four-post rack, see the following figure.

Figure 3: Clearances Required for a Four-Post Rack Installation

Preparing the Site

Width of the front clearance area (this equals

the width of the chassis with two rack-mount brackets attached to it)

Both the front and rear of the chassis must be open to both aisles for airflow.Note

Depth of the chassis5Chassis1

Maximum extension of the bottom-support rails6Vertical rack-mount posts and rails2

Depth of the front clearance area (this equals the

7Chassis width3

depth of the chassis)

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

Installing the Chassis

Preparing to Install the Switch, page 11

•

Installing a Rack or Cabinet, page 13

•

Unpacking and Inspecting the Switch, page 14

•

Installing the Chassis in a Four-Post Rack, page 15

•

Grounding the Chassis, page 21

•

Powering Up the Switch, page 23

•

Preparing to Install the Switch

Before you install, operate, or service the switch, see the Regulatory, Compliance, and Safety Information for the Cisco Nexus 9000 Series for important safety information.

CHAPTER 3

Warning

Statement 1071—Warning Definition

IMPORTANT SAFETY INSTRUCTIONS

This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device.

SAVE THESE INSTRUCTIONS

The switch can be installed in the following types of racks using a rack-mount kit shipped with the switch:

Open EIA rack

•

Perforated EIA cabinet

•

The rack or cabinet that you use must meet the requirements listed in Rack Specifications, on page 43.

You can order the switch chassis with port-side-intake or port-side exhaust airflow. Port-side intake airflow enters the chassis through the port side (I/O module side) of the chassis and exhausts through the fan and

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

+ 49 hidden pages

Cisco Nexus 9336PQ, N9K-C9336PQ Hardware Installation Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide

CONTENTS

Preface

Audience

Documentation Conventions

Related Documentation

Documentation Feedback

Obtaining Documentation and Submitting a Service Request

Overview

Overview

Ports

Supported Transceivers and Cables

Fan Modules

Power Supply Modules

Preparing the Site

Temperature Requirements

Humidity Requirements

Altitude Requirements

Dust and Particulate Requirements

Minimizing Electromagnetic and Radio Frequency Interference

Shock and Vibration Requirements

Grounding Requirements

Planning for Power Requirements

Airflow Requirements

Rack and Cabinet Requirements

Clearance Requirements

Installing the Chassis

Preparing to Install the Switch