Cisco APIC M3, APIC L3 Installation And Service Manual

Cisco APIC M3/L3 Server Installation and Service Guide

First Published: 2018-10-25

Americas Headquarters

Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000

800 553-NETS (6387)

Fax: 408 527-0883

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.

The following information is for FCC compliance of Class B devices: This equipment has been tested and found to comply with the limits for a Class B digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiate radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If the equipment causes interference to radio or television reception, which can be determined by turning the equipment off and on, users are encouraged to try to correct the interference by using one or more of the following measures:

• Reorient or relocate the receiving antenna.

• Increase the separation between the equipment and receiver.

• Connect the equipment into an outlet on a circuit different from that to which the receiver is connected.

• Consult the dealer or an experienced radio/TV technician for help.

Modifications to this product not authorized by Cisco could void the FCC approval and negate your authority to operate the product

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright©1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED "AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com

go trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any

other company. (1721R)

CHAPTER 1

CHAPTER 2

Overview 1

External Features 1

Serviceable Component Locations 3

Summary of Server Features 5

Installing the Server 7

Preparing for Installation 7

Installation Warnings and Guidelines 7

Grounding Requirements 9

Rack Requirements 9

Installing the Server in a Rack 10

Installing the Cable Management Arm (Optional) 12

Reversing the Cable Management Arm (Optional) 13

Initial Server Setup 14

Connecting to the Server Locally For Setup 15

CHAPTER 3

Connecting to the Server Remotely For Setup 16

Setting Up the System With the Cisco IMC Configuration Utility 17

NIC Mode and NIC Redundancy Settings 19

Updating the BIOS and Cisco IMC Firmware 20

Accessing the System BIOS 21

Smart Access Serial 21

Smart Access USB 22

Maintaining the Server 23

Status LEDs and Buttons 23

Cisco APIC M3/L3 Server Installation and Service Guide

iii

Contents

Front-Panel LEDs 23

Rear-Panel LEDs 26

Internal Diagnostic LEDs 27

Preparing For Component Installation 28

Required Equipment For Service Procedures 29

Shutting Down and Removing Power From the Server 29

Shutting Down Using the Power Button 29

Shutting Down Using The Cisco IMC GUI 30

Shutting Down Using The Cisco IMC CLI 30

Removing the Server Top Cover 30

Serial Number Location 31

Hot Swap vs Hot Plug 31

Removing and Replacing Components 32

Serviceable Component Locations 32

Replacing SAS/SATA Hard Drives or Solid State Drives 34

SAS/SATA Drive Population Guidelines 34

4K Sector Format SAS/SATA Drives Considerations 34

Replacing a SAS/SATA Drive 35

Replacing a Front-Loading NVMe SSD 36

Front-Loading NVMe SSD Population Guidelines 36

Front-Loading NVME SSD Requirements and Restrictions 37

Enabling Hot-Plug Support in the System BIOS 37

Replacing a Front-Loading NVMe SSD 38

Installing a PCIe Cable For Front-Loading NVMe SSDs 39

Replacing HHHL Form-Factor NVMe Solid State Drives 40

HHHL SSD Population Guidelines 40

HHHL Form-Factor NVME SSD Requirements and Restrictions 40

Replacing an HHHL Form-Factor NVMe SSD 41

Replacing Fan Modules 42

Replacing Memory DIMMs 43

DIMM Population Rules and Memory Performance Guidelines 43

Replacing DIMMs 45

Replacing CPUs and Heatsinks 46

CPU Configuration Rules 46

Cisco APIC M3/L3 Server Installation and Service Guide

Tools Required For CPU Replacement 46

Replacing a CPU and Heatsink 47

Additional CPU-Related Parts to Order with RMA Replacement CPUs 53

Additional CPU-Related Parts to Order with RMA Replacement System Chassis 53

Moving an M5 Generation CPU 54

Replacing a Mini-Storage Module 58

Replacing a Mini-Storage Module Carrier 59

Replacing an SD Card in a Mini-Storage Carrier For SD 60

Replacing an M.2 SSD in a Mini-Storage Carrier For M.2 61

Replacing a Micro SD Card 61

Replacing an Internal USB Drive 63

Replacing a USB Drive 63

Enabling or Disabling the Internal USB Port 64

Contents

Replacing the RTC Battery 65

Replacing Power Supplies 66

Replacing AC Power Supplies 66

Replacing a PCIe Card 67

PCIe Slot Specifications 67

Replacing a PCIe Card 68

Cisco Virtual Interface Card (VIC) Considerations 70

Replacing an mLOM Card 71

Replacing an mRAID Riser (Riser 3) 72

Replacing a SAS Storage Controller Card (RAID or HBA) 74

Storage Controller Card Firmware Compatibility 74

Replacing a SAS Storage Controller Card (RAID or HBA) 75

Replacing the Supercap (RAID Backup) 76

Replacing a SATA Interposer Card 78

Replacing a Chassis Intrusion Switch 80

Installing a Trusted Platform Module (TPM) 81

TPM Considerations 81

Installing and Enabling a TPM 82

Service Headers and Jumpers 85

Using the Clear CMOS Header (J38, Pins 9 - 10) 86

Using the BIOS Recovery Header (J38, Pins 11 - 12) 86

Cisco APIC M3/L3 Server Installation and Service Guide

Contents

Procedure 1: Reboot With recovery.cap File 87

Procedure 2: Use BIOS Recovery Header and bios.cap Recovery File 87

Using the Clear Password Header (J38, Pins 13 - 14) 88

Using the Boot Alternate Cisco IMC Image Header (J39, Pins 1 - 2) 89

Using the Reset Cisco IMC Password to Default Header (J39, Pins 3 - 4) 90

Using the Reset Cisco IMC to Defaults Header (J39, Pins 5 - 6) 90

APPENDIX A

APPENDIX B

Server Specifications 93

Physical Specifications 93

Environmental Specifications 93

Power Specifications 94

770 W AC Power Supply 94

1050 W DC Power Supply 95

Power Cord Specifications 96

Storage Controller Considerations 99

Supported Storage Controllers and Cables 99

Storage Controller Card Firmware Compatibility 100

Write-Cache Policy for Cisco 12G SAS Modular RAID Controller 101

Mixing Drive Types in RAID Groups 101

RAID Controller Migration 102

Storage Controller and Backplane Connectors 102

Embedded SATA RAID Controller 104

Embbeded SATA RAID Requirements 104

Embedded SATA RAID Controller Considerations 105

Embedded SATA RAID: Two SATA Controllers 105

Enabling SATA Mode For the Embedded Controllers 106

Accessing the Software RAID Configuration Utility 107

Installing LSI MegaSR Drivers For Windows and Linux 107

Downloading the MegaSR Drivers 107

Microsoft Windows Server Drivers 108

Linux Drivers 109

For More RAID Utility Information 113

Cisco APIC M3/L3 Server Installation and Service Guide

Contents

APPENDIX C

APPENDIX D

GPU Card Installation 115

Server Firmware Requirements 115

GPU Card Configuration Rules 115

Requirement For All GPUs: Memory-Mapped I/O Greater Than 4 GB 115

Replacing a Single-Wide GPU Card 116

Installing Drivers to Support the GPU Cards 118

1. Updating the Server BIOS 118

2. Updating the GPU Card Drivers 119

Installation For Cisco UCS Manager Integration 121

Cisco APIC M3/L3 Server Installation and Service Guide

vii

Contents

viii

Cisco APIC M3/L3 Server Installation and Service Guide

Overview

• Overview, on page 1

• External Features, on page 1

• Serviceable Component Locations, on page 3

• Summary of Server Features, on page 5

Overview

Cisco APIC Server M3 and L3 (APIC-SERVER-M3 and APIC-SERVER-L3)—Small form-factor (SFF) drives, with 10-drive backplane. Supports up to 10 2.5-inch SAS/SATA drives. Drive bays 1 and 2 support NVMe SSDs.

External Features

This topic shows the external features of the server versions.

CHAPTER 1

Cisco APIC M3 and L3 Server (SFF Drives) Front Panel Features

The following figure shows the front panel features of the small form-factor drive versions of the server.

Cisco APIC M3/L3 Server Installation and Service Guide

External Features

Figure 1: Cisco APIC M3 and L3 Server (SFF Drives) Front Panel

Overview

Fan status LED7Drive bays 1 – 10 support SAS/SATA hard disk

drives (HDDs) and solid state drives (SSDs)

Network link activity LED8• APIC-Server-M3 and L3: Drive bays 1 and

2 support NVMe PCIe SSDs.

Temperature status LED9Power button/power status LED3

Pull-out asset tag10Unit identification button/LED4

11System status LED5

KVM connector

(used with KVM cable that provides one DB-15 VGA, one DB-9 serial, and two USB connectors)

-Power supply status LED6

Cisco APIC M3 and L3 Server Rear Panel Features

The rear panel features are the same for all versions of the server.

Cisco APIC M3/L3 Server Installation and Service Guide

Overview

Figure 2: Cisco APIC M3 and L3 Server Rear Panel

Serviceable Component Locations

Rear unit identification button/LED6USB 3.0 ports (two)1

LAN2)

The dual LAN ports can support 1 Gbps and 10 Gbps, depending on the link partner capability.

91-Gb Ethernet dedicated management port4

Note

The VIC 1455 has 4 ports, port-0, port-1, port-3, and port-4 from left to right.

• All ports must have the same speed, either 10-Gigabit or 25-Gigabit.

• Port-0 and port-1 is one pair, corresponding to eth2-1 on APIC and port-2 and port-3 is another pair, corresponding to eth2-2 on APIC. Only one connection is allowed for each pair. For example, you can connect one cable to either port-0 or port-1, and connect another cable to either port-2 or port-3 (please do not connect two cables on any pair).

Power supplies (two, redundant as 1+1)7Dual 1-Gb/10-Gb Ethernet ports (LAN1 and

PCIe riser 1/slot 1 (x16 lane)8VGA video port (DB-15 connector)3

VIC 1455 with external 10/25-Gigabit Ethernet ports (4)

Threaded holes for dual-hole grounding lug10Serial port (RJ-45 connector)5

Serviceable Component Locations

This topic shows the locations of the field-replaceable components and service-related items. The view in the following figure shows the server with the top cover removed.

Cisco APIC M3/L3 Server Installation and Service Guide

Serviceable Component Locations

Figure 3: Cisco APIC M3 and L3 Server, Serviceable Component Locations

Overview

RTC battery, vertical socket9Front-loading drive bays 1–10 support SAS/SATA

drives.

10Cooling fan modules (seven, hot-swappable)2

Power supplies (hot-swappable when redundant as 1+1)

11Supercap unit mounting bracket (RAID backup)3

Trusted platform module (TPM) socket on motherboard (not visible in this view)

12DIMM sockets on motherboard (12 per CPU)4

PCIe riser 1/slot 1 (half-height, x16 lane)

Includes PCIe cable connectors for front-loading NVMe SSDs (x8 lane)

13CPUs and heatsinks (up to two)5

VIC 1455 with external 10/25-Gigabit Ethernet ports (4)

Available (empty) PCIe slot14Mini storage module socket

Supports either an SD card module with two SD card slots; or an M.2 module with two NVMe or SATA M.2 SSD slots.

15Chassis intrusion switch (optional)7

PCIe cable connectors for front-loading NVMe SSDs on PCIe riser 2

Micro-SD card socket on PCIe riser 116Internal USB 3.0 port on motherboard8

Cisco APIC M3/L3 Server Installation and Service Guide

Overview

Summary of Server Features

The following table lists a summary of server features.

Summary of Server Features

DescriptionFeature

One rack-unit (1RU) chassisChassis

Central Processor

Baseboard management

Network and management I/O

Up to two CPUs from the Intel Xeon Processor Scalable Family. This includes CPUs from the following series:

• Intel Xeon Bronze 3XXX Processors

• Intel Xeon Silver 4XXX Processors

• Intel Xeon Gold 5XXX Processors

• Intel Xeon Gold 6XXX Processors

• Intel Xeon Platinum 8XXX Processors

24 DDR4 DIMM sockets on the motherboard (12 each CPU)Memory

Multi-bit error protection is supportedMulti-bit error protection

BMC, running Cisco Integrated Management Controller (Cisco IMC) firmware.

Depending on your Cisco IMC settings, Cisco IMC can be accessed through the 1-Gb dedicated management port, the 1-Gb/10-Gb Ethernet LAN ports, or a Cisco virtual interface card.

Rear panel:

• One 1-Gb Ethernet dedicated management port (RJ-45 connector)

• Two 1-Gb/10-Gb BASE-T Ethernet LAN ports (RJ-45 connectors)

The dual LAN ports can support 1 Gbps and 10 Gbps, depending on the link partner capability.

• One RS-232 serial port (RJ-45 connector)

• One VGA video connector port (DB-15 connector)

• Two USB 3.0 ports

Front panel:

• One front-panel keyboard/video/mouse (KVM) connector that is used with the KVM cable, which provides two USB

2.0, one VGA, and one DB-9 serial connector.

Cisco APIC M3/L3 Server Installation and Service Guide

Summary of Server Features

Overview

DescriptionFeature

Modular LOM

WoL

Power

ACPI

InfiniBand

Storage, front-panel

Storage, internal

One dedicated socket (x16 PCIe lane) that can be used to add an mLOM card for additional rear-panel connectivity.

The two 1-Gb/10-Gb BASE-T Ethernet LAN ports support the wake-on-LAN (WoL) standard.

Two power supplies, redundant as 1+1:

• AC power supplies 770 W AC each

The advanced configuration and power interface (ACPI) 4.0 standard is supported.

Seven hot-swappable fan modules for front-to-rear cooling.Cooling

Two horizontal PCIe expansion slots on a PCIe riser assembly.PCIe I/O

The PCIe bus slots in this server support the InfiniBand architecture.

Cisco APIC M3 and L3 (APIC-SERVER-M3 and APIC-SERVER-L3)—Small form-factor (SFF) drives, with 10-drive backplane. Supports up to 10 2.5-inch SAS/SATA drives. Drive bays 1 and 2 support NVMe SSDs.

The server has these internal storage options:

Storage management

RAID backup

• One USB port on the motherboard.

• Mini-storage module socket, optionally with either:

• SD card module. Supports up to two SD cards.

• M.2 SSD module. Supports either two SATA M.2 SSDs or two NVMe M.2 SSDs.

• One micro-SD card socket on PCIe riser 1.

The server has a dedicated internal mRAID riser that supports one of the following storage-controller options:

• A PCIe-style Cisco modular RAID controller card (SAS/SATA).

• A PCIe-style interposer card for the server’s embedded SATA RAID controller.

The server has a mounting bracket near the cooling fans for the supercap unit that is used with the Cisco modular RAID controller card.

Integrated VGA video.Integrated video

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

• Preparing for Installation, on page 7

• Installing the Server in a Rack, on page 10

• Initial Server Setup, on page 14

• NIC Mode and NIC Redundancy Settings, on page 19

• Updating the BIOS and Cisco IMC Firmware, on page 20

• Accessing the System BIOS, on page 21

• Smart Access Serial, on page 21

• Smart Access USB, on page 22

Preparing for Installation

This section contains the following topics:

Installation Warnings and Guidelines

CHAPTER 2

Note

Warning

Before you install, operate, or service a server, review the Regulatory Compliance and Safety Information for important safety information.

IMPORTANT SAFETY INSTRUCTIONS

This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device.

Statement 1071

Cisco APIC M3/L3 Server Installation and Service Guide

Installation Warnings and Guidelines

Installing the Server

Warning

To prevent the system from overheating, do not operate it in an area that exceeds the maximum recommended ambient temperature of: 35° C (95° F).

Statement 1047

The plug-socket combination must be accessible at all times, because it serves as the main disconnecting device.

Statement 1019

Installation of the equipment must comply with local and national electrical codes.

Statement 1074

This unit is intended for installation in restricted access areas. A restricted access area can be accessed only through the use of a special tool, lock, and key, or other means of security.

Statement 1017

Warning

Caution

This product relies on the building's installation for short-circuit (over current) protection. Ensure that the protective devices is rated not greater than 20A (North America), 16A (Europe),and 13A (UK).

Statement 1005

To ensure proper airflow it is necessary to rack the servers using rail kits. Physically placing the units on top of one another or “stacking” without the use of the rail kits blocks the air vents on top of the servers, which could result in overheating, higher fan speeds, and higher power consumption. We recommend that you mount your servers on rail kits when you are installing them into the rack because these rails provide the minimal spacing required between the servers. No additional spacing between the servers is required when you mount the units using rail kits.

Avoid uninterruptible power supply (UPS) types that use ferroresonant technology. These UPS types can become unstable with systems such as the Cisco UCS, which can have substantial current draw fluctuations from fluctuating data traffic patterns.

To prevent loss of input power, ensure the total maximum loads on the circuits supplying power to the switch are within the current ratings for the wiring and breakers.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

When you are installing a server, use the following guidelines:

• Plan your site configuration and prepare the site before installing the server.

• Ensure that there is adequate space around the server to allow for accessing the server and for adequate airflow. The airflow in this server is from front to back.

• Ensure that the air-conditioning meets the thermal requirements listed in the Environmental Specifications,

on page 93.

• Ensure that the cabinet or rack meets the requirements listed in the Rack Requirements, on page 9.

• Ensure that the site power meets the power requirements listed in the Power Specifications, on page 94. If available, you can use an uninterruptible power supply (UPS) to protect against power failures.

Grounding Requirements

The switch is sensitive to variations in voltage supplied by the power sources. Overvoltage, undervoltage, and transients (or spikes) can erase data from memory or cause components to fail. To protect against these types of problems, ensure that there is an earth-ground connection for the switch. You can connect the grounding pad on the switch either directly to the earth-ground connection or to a fully bonded and grounded rack.

Grounding Requirements

When you properly install the chassis in a grounded rack, the switch is grounded because it has a metal-to-metal connection to the rack. Alternatively, you can ground the chassis by using a customer-supplied grounding cable that meets your local and national installation requirements (we recommend 6-AWG wire for U.S. installations) connected to the chassis with a grounding lug (provided in the switch accessory kit) and to the facility ground.

Note

You automatically ground AC power supplies when you connect them to AC power sources. For DC power supplies, you must connect a grounding wire when wiring the power supply to the DC power source.

Rack Requirements

The rack must be of the following type:

• A standard 19-in. (48.3-cm) wide, four-post EIA rack, with mounting posts that conform to English universal hole spacing, per section 1 of ANSI/EIA-310-D-1992.

• The rack-post holes can be square 0.38-inch (9.6 mm), round 0.28-inch (7.1 mm), #12-24 UNC, or #10-32 UNC when you use the Cisco-supplied slide rails.

• The minimum vertical rack space per server must be one rack unit (RU), equal to 1.75 in. (44.45 mm).

Supported Cisco Slide Rail Kits

The server supports the following rail kit options:

Rack Installation Tools Required

The slide rails sold by Cisco Systems for this server do not require tools for installation.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server in a Rack

Slide Rail and Cable Management Arm Dimensions

The slide rails for this server have an adjustment range of 24 to 36 inches (610 to 914 mm).

The optional cable management arm (CMA) adds additional length requirements:

• The additional distance from the rear of the server to the rear of the CMA is 5.4 inches (137.4 mm).

• The total length of the server including the CMA is 35.2 inches (894 mm).

Installing the Server in a Rack

Installing the Server

Warning

Step 1 Attach the inner rails to the sides of the server:

a) Align an inner rail with one side of the server so that the three keyed slots in the rail align with the three pegs on the

b) Set the keyed slots over the pegs, and then slide the rail toward the front to lock it in place on the pegs. The front slot

c) Install the second inner rail to the opposite side of the server.

Figure 4: Attaching the Inner Rail to the Side of the Server

To prevent bodily injury when mounting or servicing this unit in a rack, you must take special precautions to ensure that the system remains stable. The following guidelines are provided to ensure your safety:

This unit should be mounted at the bottom of the rack if it is the only unit in the rack.

When mounting this unit in a partially filled rack, load the rack from the bottom to the top with the heaviest component at the bottom of the rack.

If the rack is provided with stabilizing devices, install the stabilizers before mounting or servicing the unit in the rack.

Statement 1006

side of the server.

has a metal clip that locks over the front peg.

Inner rail2Front of server1

Step 2 Open the front securing plate on both slide-rail assemblies. The front end of the slide-rail assembly has a spring-loaded

securing plate that must be open before you can insert the mounting pegs into the rack-post holes.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

On the outside of the assembly, push the green-arrow button toward the rear to open the securing plate.

Figure 5: Front Securing Mechanism, Inside of Front End

Installing the Server in a Rack

securing plate

Step 3 Install the outer slide rails into the rack:

a) Align one slide-rail assembly front end with the front rack-post holes that you want to use.

The slide rail front-end wraps around the outside of the rack post and the mounting pegs enter the rack-post holes from the outside-front.

Note

The rack post must be between the mounting pegs and the open securing plate.

b) Push the mounting pegs into the rack-post holes from the outside-front. c) Press the securing plate release button, marked PUSH. The spring-loaded securing plate closes to lock the pegs in

place.

d) Adjust the slide-rail length, and then push the rear mounting pegs into the corresponding rear rack-post holes. The

slide rail must be level front-to-rear.

The rear mounting pegs enter the rear rack-post holes from the inside of the rack post.

e) Attach the second slide-rail assembly to the opposite side of the rack. Ensure that the two slide-rail assemblies are at

the same height and are level front-to-back.

f) Pull the inner slide rails on each assembly out toward the rack front until they hit the internal stops and lock in place.

3Front mounting pegs1

Securing plate shown pulled back to the open position

-Rack post between mounting pegs and opened

Step 4 Insert the server into the slide rails:

Caution

This server can weigh up to 64 pounds (29 kilograms) when fully loaded with components. We recommend that you use a minimum of two people or a mechanical lift when lifting the server. Attempting this procedure alone could result in personal injury or equipment damage.

a) Align the rear ends of the inner rails that are attached to the server sides with the front ends of the empty slide rails

on the rack. b) Push the inner rails into the slide rails on the rack until they stop at the internal stops. c) Slide the inner-rail release clip toward the rear on both inner rails, and then continue pushing the server into the rack

until its front slam-latches engage with the rack posts.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Cable Management Arm (Optional)

Figure 6: Inner-Rail Release Clip

Installing the Server

Outer slide rail attached to rack post3Inner-rail release clip1

-Inner rail attached to server and inserted into

outer slide rail

Step 5 (Optional) Secure the server in the rack more permanently by using the two screws that are provided with the slide rails.

Perform this step if you plan to move the rack with servers installed.

With the server fully pushed into the slide rails, open a hinged slam latch lever on the front of the server and insert a screw through the hole that is under the lever. The screw threads into the static part of the rail on the rack post and prevents the server from being pulled out. Repeat for the opposite slam latch.

Installing the Cable Management Arm (Optional)

Note

The cable management arm (CMA) is reversible left-to-right. To reverse the CMA, see Reversing the Cable

Management Arm (Optional), on page 13 before installation.

Step 1 With the server pushed fully into the rack, slide the CMA tab of the CMA arm that is farthest from the server onto the

end of the stationary slide rail that is attached to the rack post. Slide the tab over the end of the rail until it clicks and locks.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

Figure 7: Attaching the CMA to the Rear Ends of the Slide Rails

Reversing the Cable Management Arm (Optional)

3CMA tab on arm farthest from server attaches

to end of stationary outer slide rail.

CMA tab on width-adjustment slider attaches to end of stationary outer slide rail.

Rear of server4CMA tab on arm closest to the server attaches

to end of inner slide rail attached to server.

Step 2 Slide the CMA tab that is closest to the server over the end of the inner rail that is attached to the server. Slide the tab

over the end of the rail until it clicks and locks

Step 3 Pull out the width-adjustment slider that is at the opposite end of the CMA assembly until it matches the width of your

rack.

Step 4 Slide the CMA tab that is at the end of the width-adjustment slider onto the end of the stationary slide rail that is attached

to the rack post. Slide the tab over the end of the rail until it clicks and locks.

Step 5 Open the hinged flap at the top of each plastic cable guide and route your cables through the cable guides as desired.

Reversing the Cable Management Arm (Optional)

Step 1 Rotate the entire CMA assembly 180 degrees, left-to-right. The plastic cable guides must remain pointing upward.

Step 2 Flip the tabs at the ends of the CMA arms so that they point toward the rear of the server.

Step 3 Pivot the tab that is at the end of the width-adjustment slider. Depress and hold the metal button on the outside of the tab

and pivot the tab 180 degrees so that it points toward the rear of the server.

Cisco APIC M3/L3 Server Installation and Service Guide

Initial Server Setup

Figure 8: Reversing the CMA

Installing the Server

Initial Server Setup

Note

This section describes how to power on the server, assign an IP address, and connect to server management when using the server in standalone mode.

Server Default Settings

The server is shipped with these default settings:

• The NIC mode is Shared LOM EXT.

Shared LOM EXT mode enables the 1-Gb/10-Gb Ethernet ports and the ports on any installed Cisco virtual interface card (VIC) to access the Cisco Integrated Management Interface (Cisco IMC). If you want to use the 10/100/1000 dedicated management ports to access Cisco IMC, you can connect to the server and change the NIC mode as described in Setting Up the System With the Cisco IMC Configuration

Utility, on page 17.

Metal button on outside of tab2CMA tab on end of width-adjustment slider1

• The NIC redundancy is Active-Active. All Ethernet ports are utilized simultaneously.

• DHCP is enabled.

• IPv4 is enabled.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

Connecting to the Server Locally For Setup

Connection Methods

There are two methods for connecting to the system for initial setup:

• Local setup—Use this procedure if you want to connect a keyboard and monitor directly to the system for setup. This procedure can use a KVM cable (Cisco PID N20-BKVM) or the ports on the rear of the server.

• Remote setup—Use this procedure if you want to perform setup through your dedicated management LAN.

Note

To configure the system remotely, you must have a DHCP server on the same network as the system. Your DHCP server must be preconfigured with the range of MAC addresses for this server node. The MAC address is printed on a label that is on the pull-out asset tag on the front panel. This server node has a range of six MAC addresses assigned to the Cisco IMC. The MAC address printed on the label is the beginning of the range of six contiguous MAC addresses.

This section contains the following topics:

Connecting to the Server Locally For Setup

This procedure requires the following equipment:

• VGA monitor

• USB keyboard

• Either the supported Cisco KVM cable (Cisco PID N20-BKVM); or a USB cable and VGA DB-15 cable

Step 1 Attach a power cord to each power supply in your server, and then attach each power cord to a grounded power outlet.

Wait for approximately two minutes to let the server boot to standby power during the first bootup. You can verify system power status by looking at the system Power Status LED on the front panel. The system is in standby power mode when the LED is amber.

Step 2 Connect a USB keyboard and VGA monitor to the server using one of the following methods:

• Connect an optional KVM cable (Cisco PID N20-BKVM) to the KVM connector on the front panel. Connect your USB keyboard and VGA monitor to the KVM cable.

• Connect a USB keyboard and VGA monitor to the corresponding connectors on the rear panel.

Step 3 Open the Cisco IMC Configuration Utility:

a) Press and hold the front panel power button for four seconds to boot the server. b) During bootup, press F8 when prompted to open the Cisco IMC Configuration Utility.

Note

The first time that you enter the Cisco IMC Configuration Utility, you are prompted to change the default password. The default password is password. The Strong Password feature is enabled.

The following are the requirements for Strong Password:

Cisco APIC M3/L3 Server Installation and Service Guide

Connecting to the Server Remotely For Setup

• The password can have minimum 8 characters; maximum 14 characters.

• The password must not contain the user’s name.

• The password must contain characters from three of the following four categories:

• English uppercase letters (A through Z)

• English lowercase letters (a through z)

• Base 10 digits (0 through 9)

• Non-alphabetic characters !, @, #, $, %, ^, &, *, -, _, =, “

Step 4 Continue with Setting Up the System With the Cisco IMC Configuration Utility, on page 17.

Connecting to the Server Remotely For Setup

Installing the Server

This procedure requires the following equipment:

• One RJ-45 Ethernet cable that is connected to your management LAN.

Before you begin

Note

Step 1 Attach a power cord to each power supply in your server, and then attach each power cord to a grounded power outlet.

Step 2 Plug your management Ethernet cable into the dedicated management port on the rear panel.

Step 3 Allow your preconfigured DHCP server to assign an IP address to the server node.

Step 4 Use the assigned IP address to access and log in to the Cisco IMC for the server node. Consult with your DHCP server

administrator to determine the IP address.

Note

The default user name for the server is admin. The default password is password.

Step 5 From the Cisco IMC Server Summary page, click Launch KVM Console. A separate KVM console window opens.

Step 6 From the Cisco IMC Summary page, click Power Cycle Server. The system reboots. Step 7 Select the KVM console window.

Note

The KVM console window must be the active window for the following keyboard actions to work.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

Setting Up the System With the Cisco IMC Configuration Utility

Step 8 When prompted, press F8 to enter the Cisco IMC Configuration Utility. This utility opens in the KVM console window.

Note

The first time that you enter the Cisco IMC Configuration Utility, you are prompted to change the default password. The default password is password. The Strong Password feature is enabled.

The following are the requirements for Strong Password:

• The password can have minimum 8 characters; maximum 14 characters.

• The password must not contain the user’s name.

• The password must contain characters from three of the following four categories:

• English uppercase letters (A through Z)

• English lowercase letters (a through z)

• Base 10 digits (0 through 9)

• Non-alphabetic characters !, @, #, $, %, ^, &, *, -, _, =, “

Step 9 Continue with Setting Up the System With the Cisco IMC Configuration Utility, on page 17.

Setting Up the System With the Cisco IMC Configuration Utility

Before you begin

The following procedure is performed after you connect to the system and open the Cisco IMC Configuration Utility.

Step 1 Set the NIC mode to choose which ports to use to access Cisco IMC for server management:

• Shared LOM EXT (default)—This is the shared LOM extended mode, the factory-default setting. With this mode, the Shared LOM and Cisco Card interfaces are both enabled. You must select the default Active-Active NIC redundancy setting in the following step.

In this NIC mode, DHCP replies are returned to both the shared LOM ports and the Cisco card ports. If the system determines that the Cisco card connection is not getting its IP address from a Cisco UCS Manager system because the server is in standalone mode, further DHCP requests from the Cisco card are disabled. Use the Cisco Card NIC mode if you want to connect to Cisco IMC through a Cisco card in standalone mode.

• Shared LOM—The 1-Gb/10-Gb Ethernet ports are used to access Cisco IMC. You must select either the Active-Active or Active-standby NIC redundancy setting in the following step.

• Dedicated—The dedicated management port is used to access Cisco IMC. You must select the None NIC redundancy setting in the following step.

• Cisco Card—The ports on an installed Cisco UCS Virtual Interface Card (VIC) are used to access the Cisco IMC. You must select either the Active-Active or Active-standby NIC redundancy setting in the following step.

See also the required VIC Slot setting below.

• VIC Slot—Only if you use the Cisco Card NIC mode, you must select this setting to match where your VIC is installed. The choices are Riser1, Riser2, or Flex-LOM (the mLOM slot).

Cisco APIC M3/L3 Server Installation and Service Guide

Setting Up the System With the Cisco IMC Configuration Utility

• If you select Riser1, you must install the VIC in slot 1.

• If you select Riser2, you must install the VIC in slot 2.

• If you select Flex-LOM, you must install an mLOM-style VIC in the mLOM slot.

Step 2 Set the NIC redundancy to your preference. This server has three possible NIC redundancy settings:

• None—The Ethernet ports operate independently and do not fail over if there is a problem. This setting can be used only with the Dedicated NIC mode.

• Active-standby—If an active Ethernet port fails, traffic fails over to a standby port. Shared LOM and Cisco Card modes can each use either Active-standby or Active-active settings.

• Active-active (default)—All Ethernet ports are utilized simultaneously. The Shared LOM EXT mode must use only this NIC redundancy setting. Shared LOM and Cisco Card modes can each use either Active-standby or Active-active settings.

Step 3 Choose whether to enable DHCP for dynamic network settings, or to enter static network settings.

Installing the Server

Note

Before you enable DHCP, you must preconfigure your DHCP server with the range of MAC addresses for this server. The MAC address is printed on a label on the rear of the server. This server has a range of six MAC addresses assigned to Cisco IMC. The MAC address printed on the label is the beginning of the range of six contiguous MAC addresses.

The static IPv4 and IPv6 settings include the following:

• The Cisco IMC IP address.

For IPv6, valid values are 1 - 127.

• The gateway.

For IPv6, if you do not know the gateway, you can set it as none by entering :: (two colons).

• The preferred DNS server address.

For IPv6, you can set this as none by entering :: (two colons).

Step 4 (Optional) Make VLAN settings.

Step 5 Press F1 to go to the second settings window, then continue with the next step.

From the second window, you can press F2 to switch back to the first window.

Step 6 (Optional) Set a hostname for the server.

Step 7 (Optional) Enable dynamic DNS and set a dynamic DNS (DDNS) domain.

Step 8 (Optional) If you check the Factory Default check box, the server reverts to the factory defaults.

Step 9 (Optional) Set a default user password.

Note

The factory default username for the server is admin. The default password is password.

Step 10 (Optional) Enable auto-negotiation of port settings or set the port speed and duplex mode manually.

Note

Auto-negotiation is applicable only when you use the Dedicated NIC mode. Auto-negotiation sets the port speed and duplex mode automatically based on the switch port to which the server is connected. If you disable auto-negotiation, you must set the port speed and duplex mode manually.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

NIC Mode and NIC Redundancy Settings

Step 11 (Optional) Reset port profiles and the port name.

Step 12 Press F5 to refresh the settings that you made. You might have to wait about 45 seconds until the new settings appear

and the message, “Network settings configured” is displayed before you reboot the server in the next step.

Step 13 Press F10 to save your settings and reboot the server.

Note

If you chose to enable DHCP, the dynamically assigned IP and MAC addresses are displayed on the console screen during bootup.

What to do next

Use a browser and the IP address of the Cisco IMC to connect to the Cisco IMC management interface. The IP address is based upon the settings that you made (either a static address or the address assigned by your DHCP server).

Note

The factory default username for the server is admin. The default password is password.

NIC Mode and NIC Redundancy Settings

Table 1: Valid NIC Redundancy Settings For Each NIC Mode

Valid NIC Redundancy SettingsNIC Mode

Active-activeShared LOM

EXT

NoneDedicated

Shared LOM

Active-active

Active-standby

Cisco Card

Active-active

Active-standby

This server has the following NIC mode settings that you can choose from:

In this NIC mode, DHCP replies are returned to both the shared LOM ports and the Cisco card ports. Use the Cisco Card NIC mode if you want to connect to Cisco IMC through a Cisco card in standalone mode.

• Shared LOM—The 1-Gb/10-Gb Ethernet ports are used to access Cisco IMC. You must select either the Active-Active or Active-standby NIC redundancy setting in the following step.

Cisco APIC M3/L3 Server Installation and Service Guide

Updating the BIOS and Cisco IMC Firmware

• Dedicated—The dedicated management port is used to access Cisco IMC. You must select the None NIC redundancy setting in the following step.

See also the required VIC Slot setting below.

• VIC Slot—Only if you use the Cisco Card NIC mode, you must select this setting to match where your VIC is installed. The choices are Riser1, Riser2, or Flex-LOM (the mLOM slot).

• If you select Riser1, you must install the VIC in slot 1.

• If you select Riser2, you must install the VIC in slot 2.

• If you select Flex-LOM, you must install an mLOM-style VIC in the mLOM slot.

This server has the following NIC redundancy settings that you can choose from:

• None—The Ethernet ports operate independently and do not fail over if there is a problem. This setting can be used only with the Dedicated NIC mode.

Installing the Server

• Active-standby—If an active Ethernet port fails, traffic fails over to a standby port. Shared LOM and Cisco Card modes can each use either Active-standby or Active-active settings.

Updating the BIOS and Cisco IMC Firmware

Caution

When you upgrade the BIOS firmware, you must also upgrade the Cisco IMC firmware to the same version or the server does not boot. Do not power off the server until the BIOS and Cisco IMC firmware are matching or the server does not boot.

Cisco provides the Cisco Host Upgrade Utility to assist with simultaneously upgrading the BIOS, Cisco IMC, and other firmware to compatible levels.

The server uses firmware obtained from and certified by Cisco. Cisco provides release notes with each firmware image. There are several possible methods for updating the firmware:

• Recommended method for firmware update: Use the Cisco Host Upgrade Utility to simultaneously upgrade the Cisco IMC, BIOS, and component firmware to compatible levels.

For the latest firmware release, see the Cisco Host Upgrade Utility Quick Reference Guide.

• You can upgrade the Cisco IMC and BIOS firmware by using the Cisco IMC GUI interface.

• You can upgrade the Cisco IMC and BIOS firmware by using the Cisco IMC CLI interface.

Cisco APIC M3/L3 Server Installation and Service Guide

Installing the Server

Accessing the System BIOS

Step 1 Enter the BIOS Setup Utility by pressing the F2 key when prompted during bootup.

Accessing the System BIOS

Note

Step 2 Use the arrow keys to select the BIOS menu page.

Step 3 Highlight the field to be modified by using the arrow keys.

Step 4 Press Enter to select the field that you want to change, and then modify the value in the field.

Step 5 Press the right arrow key until the Exit menu screen is displayed.

Step 6 Follow the instructions on the Exit menu screen to save your changes and exit the setup utility (or press F10). You can

exit without saving changes by pressing Esc.

The version and build of the current BIOS are displayed on the Main page of the utility.

Smart Access Serial

This server supports the Smart Access Serial feature. This feature allows you to switch between host serial and Cisco IMC CLI.

• This feature has the following requirements:

• A serial cable connection, which can use either the RJ-45 serial connector on the server rear panel, or a DB-9 connection when using the KVM cable (Cisco PID N20-BKVM) on the front-panel KVM console connector.

• Console redirection must be enabled in the server BIOS.

• Terminal type must be set to VT100+ or VTUFT8.

• Serial-over-LAN (SOL) must be disabled (SOL is disabled by default).

• To switch from host serial to Cisco IMC CLI, press Esc+9.

You must enter your Cisco IMC credentials to authenticate the connection.

• To switch from Cisco IMC CLI to host serial, press Esc+8.

Note

You cannot switch to Cisco IMC CLI if the serial-over-LAN (SOL) feature is enabled.

• After a session is created, it is shown in the CLI or web GUI by the name serial.

Cisco APIC M3/L3 Server Installation and Service Guide

Smart Access USB

This server supports the Smart Access USB feature. The board management controller (BMC) in this server can accept a USB mass storage device and access the data on it. This feature allows you to use the front-panel USB device as a medium to transfer data between the BMC and the user without need for network connectivity. This can be useful, for example, when remote BMC interfaces are not yet available, or are not accessible due to network misconfiguration.

• This feature has the following requirements:

• The KVM cable (Cisco PID N20-BKVM) connected to the front panel KVM console connector.

• A USB storage device connected to one of the USB 2.0 connectors on the KVM cable. The USB device must draw less than 500 mA to avoid disconnect by the current-protection circuit.

Note

Installing the Server

Any mouse or keyboard that is connected to the KVM cable is disconnected when you enable Smart Access USB.

• You can use USB 3.0-based devices, but they will operate at USB 2.0 speed.

• We recommend that the USB device have only one partition.

• The file system formats supported are: FAT16, FAT32, MSDOS, EXT2, EXT3, and EXT4. NTFS is not supported.

• The front-panel KVM connector has been designed to switch the USB port between Host OS and BMC.

• Smart Access USB can be enabled or disabled using any of the BMC user interfaces. For example, you can use the Cisco IMC Configuration Utility that is accessed by pressing F8 when prompted during bootup.

• Enabled: the front-panel USB device is connected to the BMC.

• Disabled: the front-panel USB device is connected to the host.

• In a case where no management network is available to connect remotely to Cisco IMC, a Device Firmware Update (DFU) shell over serial cable can be used to generate and download technical support files to the USB device that is attached to front panel USB port.

Cisco APIC M3/L3 Server Installation and Service Guide

+ 100 hidden pages

Cisco APIC M3, APIC L3 Installation And Service Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Cisco APIC M3/L3 Server Installation and Service Guide

CONTENTS

Overview

Overview

External Features

Serviceable Component Locations

Summary of Server Features

Installing the Server

Preparing for Installation

Installation Warnings and Guidelines

Grounding Requirements

Rack Requirements

Installing the Server in a Rack

Installing the Cable Management Arm (Optional)

Reversing the Cable Management Arm (Optional)

Initial Server Setup

Connecting to the Server Locally For Setup

Connecting to the Server Remotely For Setup

Setting Up the System With the Cisco IMC Configuration Utility

NIC Mode and NIC Redundancy Settings

Updating the BIOS and Cisco IMC Firmware

Accessing the System BIOS

Smart Access Serial

Smart Access USB