Cisco SFS InfiniBand Server Switch SFS 7000P, SFS 7000D, SFS 7000P Hardware Installation Manual

Download

Corporate Headquarters

Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000

800 553-NETS (6387)

Fax: 408 526-4100

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

November 2006

Text Part Number: OL-10853-01

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.

The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy. If it is not installed in accordance with Cisco’s installation instructions, it may cause interference with radio and television reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation. However, there is no guarantee that interference will not occur in a particular installation.

Modifying the equipment without Cisco’s written authorization may result in the equipment no longer complying with FCC requirements for Class A or Class B digital devices. In that event, your right to use the equipment may be limited by FCC regulations, and you may be required to correct any interference to radio or television communications at your own expense.

You can determine whether your equipment is causing interference by turning it off. If the interference stops, it was probably caused by the Cisco equipment or one of its peripheral devices. If the equipment causes interference to radio or television reception, try to correct the interference by using one or more of the following measures:

• Turn the television or radio antenna until the interference stops.

• Move the equipment to one side or the other of the television or radio.

• Move the equipment farther away from the television or radio.

• Plug the equipment into an outlet that is on a different circuit from the television or radio. (That is, make certain the equipment and the television or radio are on circuits controlled by different circuit breakers or fuses.)

Modifications to this product not authorized by Cisco Systems, Inc. could void the FCC approval and negate your authority to operate the product.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

CCVP, the Cisco Logo, and the Cisco Square Bridge logo are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems, Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing, FormShare, GigaDrive, GigaStack, HomeLink, Internet Quotient, IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys, MeetingPlace, MGX, Networking Academy, Network Registrar, Pack et , PIX, ProConnect, RateMUX, ScriptShare, SlideCast, SMARTnet, StackWise, The Fastest Way to Increase Your Internet Quotient, and TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0612R)

iii

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

78-10853-01

CONTENTS

Audience vii

Organization vii

Conventions viii

Related Documentation

For more information about the Cisco SFS 7000 switch product family, refer to these publications:

• Cisco SFS 7000D InfiniBand Server Switch Installation and Configuration Note

• Cisco SFS 7000P InfiniBand Server Switch Installation and Configuration Note

• Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note

• Cisco SFS 7008P InfiniBand Server Switch Hardware Installation Guide

• Cisco SFS 7000 Series Product Family Chassis Manager User Guide

• Cisco SFS 7000 Series Product Family Element Manager User Guide

• Cisco SFS 7000 Series Product Family Command Reference

• Regulatory Compliance and Safety Information for the Cisco SFS 7000P, SFS 7000D, SFS 7008P,

and SFS 3012R Server Switches

• InfiniBand SDR, DDR, and QDR Technical Overview

Obtaining Documentation

Cisco documentation and additional literature are available on Cisco.com. This section explains the product documentation resources that Cisco offers.

Cisco.com

You can access the most current Cisco documentation at this URL:

http://www.cisco.com/techsupport

You can access the Cisco website at this URL:

http://www.cisco.com

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preface

Documentation Feedback

You can access international Cisco websites at this URL:

http://www.cisco.com/public/countries_languages.shtml

Product Documentation DVD

The Product Documentation DVD is a library of technical product documentation on a portable medium. The DVD enables you to access installation, configuration, and command guides for Cisco hardware and software products. With the DVD, you have access to the HTML documentation and some of the PDF files found on the Cisco website at this URL:

http://www.cisco.com/univercd/home/home.htm

The Product Documentation DVD is created and released regularly. DVDs are available singly or by subscription. Registered Cisco.com users can order a Product Documentation DVD (product number DOC-DOCDVD= or DOC-DOCDVD=SUB) from Cisco Marketplace at the Product Documentation Store at this URL:

http://www.cisco.com/go/marketplace/docstore

Ordering Documentation

You must be a registered Cisco.com user to access Cisco Marketplace. Registered users may order Cisco documentation at the Product Documentation Store at this URL:

http://www.cisco.com/go/marketplace/docstore

If you do not have a user ID or password, you can register at this URL:

http://tools.cisco.com/RPF/register/register.do

Documentation Feedback

You can provide feedback about Cisco technical documentation on the Cisco Technical Support & Documentation site area by entering your comments in the feedback form available in every online document.

Cisco Product Security Overview

Cisco provides a free online Security Vulnerability Policy portal at this URL:

http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html

From this site, you will find information about how to do the following:

• Report security vulnerabilities in Cisco products

• Obtain assistance with security incidents that involve Cisco products

• Register to receive security information from Cisco

A current list of security advisories, security notices, and security responses for Cisco products is available at this URL:

http://www.cisco.com/go/psirt

xvi

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preface

Product Alerts and Field Notices

To see security advisories, security notices, and security responses as they are updated in real time, you can subscribe to the Product Security Incident Response Team Really Simple Syndication (PSIRT RSS) feed. Information about how to subscribe to the PSIRT RSS feed is found at this URL:

http://www.cisco.com/en/US/products/products_psirt_rss_feed.html

Reporting Security Problems in Cisco Products

Cisco is committed to delivering secure products. We test our products internally before we release them, and we strive to correct all vulnerabilities quickly. If you think that you have identified a vulnerability in a Cisco product, contact PSIRT:

• For emergencies only—security-alert@cisco.com

An emergency is either a condition in which a system is under active attack or a condition for which a severe and urgent security vulnerability should be reported. All other conditions are considered nonemergencies.

• For nonemergencies —psirt@cisco.com

In an emergency, you can also reach PSIRT by telephone:

• 1 877 228-7302

• 1 408 525-6532

Tip We encourage you to use Pretty Good Privacy (PGP) or a compatible product (for example, GnuPG) to

encrypt any sensitive information that you send to Cisco. PSIRT can work with information that has been encrypted with PGP versions 2.x through 9.x.

Never use a revoked encryption key or an expired encryption key. The correct public key to use in your correspondence with PSIRT is the one linked in the Contact Summary section of the Security Vulnerability Policy page at this URL:

http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html

The link on this page has the current PGP key ID in use.

If you do not have or use PGP, contact PSIRT to find other means of encrypting the data before sending any sensitive material.

Product Alerts and Field Notices

Modifications to or updates about Cisco products are announced in Cisco Product Alerts and Cisco Field Notices. You can receive Cisco Product Alerts and Cisco Field Notices by using the Product Alert Tool on Cisco.com. This tool enables you to create a profile and choose those products for which you want to receive information.

To access the Product Alert Tool, you must be a registered Cisco.com user. (To register as a Cisco.com user, go to this URL: http://tools.cisco.com/RPF/register/register.do) Registered users can access the tool at this URL: http://tools.cisco.com/Support/PAT/do/ViewMyProfiles.do?local=en

xvii

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preface

Obtaining Technical Assistance

Cisco Technical Support provides 24-hour-a-day award-winning technical assistance. The Cisco Technical Support & Documentation website on Cisco.com features extensive online support resources. In addition, if you have a valid Cisco service contract, Cisco Technical Assistance Center (TAC) engineers provide telephone support. If you do not have a valid Cisco service contract, contact your reseller.

Cisco Technical Support & Documentation Website

The Cisco Technical Support & Documentation website provides online documents and tools for troubleshooting and resolving technical issues with Cisco products and technologies. The website is available 24 hours a day at this URL:

http://www.cisco.com/techsupport

Access to all tools on the Cisco Technical Support & Documentation website requires a Cisco.com user ID and password. If you have a valid service contract but do not have a user ID or password, you can register at this URL:

http://tools.cisco.com/RPF/register/register.do

Note Use the Cisco Product Identification Tool to locate your product serial number before submitting a

request for service online or by phone. You can access this tool from the Cisco Technical Support & Documentation website by clicking the Tools & Resources link, clicking the All Tools (A-Z) tab, and then choosing Cisco Product Identification Tool from the alphabetical list. This tool offers three search options: by product ID or model name; by tree view; or, for certain products, by copying and pasting show command output. Search results show an illustration of your product with the serial number label location highlighted. Locate the serial number label on your product and record the information before placing a service call.

Tip Displaying and Searching on Cisco.com

If you suspect that the browser is not refreshing a web page, force the browser to update the web page by holding down the Ctrl key while pressing F5.

To find technical information, narrow your search to look in technical documentation, not the entire Cisco.com website. On the Cisco.com home page, click the Advanced Search link under the Search box and then click the Technical Support & Documentation radio button.

To provide feedback about the Cisco.com website or a particular technical document, click Contacts & Feedback at the top of any Cisco.com web page.

xviii

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preface

Obtaining Additional Publications and Information

Submitting a Service Request

Using the online TAC Service Request Tool is the fastest way to open S3 and S4 service requests. (S3 and S4 service requests are those in which your network is minimally impaired or for which you require product information.) After you describe your situation, the TAC Service Request Tool provides recommended solutions. If your issue is not resolved using the recommended resources, your service request is assigned to a Cisco engineer. The TAC Service Request Tool is located at this URL:

http://www.cisco.com/techsupport/servicerequest

For S1 or S2 service requests, or if you do not have Internet access, contact the Cisco TAC by telephone. (S1 or S2 service requests are those in which your production network is down or severely degraded.) Cisco engineers are assigned immediately to S1 and S2 service requests to help keep your business operations running smoothly.

To open a service request by telephone, use one of the following numbers:

Asia-Pacific: +61 2 8446 7411 Australia: 1 800 805 227 EMEA: +32 2 704 55 55 USA: 1 800 553 2447

For a complete list of Cisco TAC contacts, go to this URL:

http://www.cisco.com/techsupport/contacts

Definitions of Service Request Severity

To ensure that all service requests are reported in a standard format, Cisco has established severity definitions.

Severity 1 (S1)—An existing network is “down” or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation.

Severity 2 (S2)—Operation of an existing network is severely degraded, or significant aspects of your business operations are negatively affected by inadequate performance of Cisco products. You and Cisco will commit full-time resources during normal business hours to resolve the situation.

Severity 3 (S3)—Operational performance of the network is impaired while most business operations remain functional. You and Cisco will commit resources during normal business hours to restore service to satisfactory levels.

Severity 4 (S4)—You require information or assistance with Cisco product capabilities, installation, or configuration. There is little or no effect on your business operations.

Obtaining Additional Publications and Information

Information about Cisco products, technologies, and network solutions is available from various online and printed sources.

• The Cisco Online Subscription Center is the website where you can sign up for a variety of

Cisco e-mail newsletters and other communications. Create a profile and then select the subscriptions that you would like to receive. To visit the Cisco Online Subscription Center, go to this URL:

http://www.cisco.com/offer/subscribe

xix

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preface

Obtaining Additional Publications and Information

• The Cisco Product Quick Reference Guide is a handy, compact reference tool that includes brief

product overviews, key features, sample part numbers, and abbreviated technical specifications for many Cisco products that are sold through channel partners. It is updated twice a year and includes the latest Cisco channel product offerings. To order and find out more about the Cisco Product Quick Reference Guide, go to this URL:

http://www.cisco.com/go/guide

• Cisco Marketplace provides a variety of Cisco books, reference guides, documentation, and logo

merchandise. Visit Cisco Marketplace, the company store, at this URL:

http://www.cisco.com/go/marketplace/

• Cisco Press publishes a wide range of general networking, training, and certification titles. Both new

and experienced users will benefit from these publications. For current Cisco Press titles and other information, go to Cisco Press at this URL:

http://www.ciscopress.com

• Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering

professionals involved in designing, developing, and operating public and private internets and intranets. You can access the Internet Protocol Journal at this URL:

http://www.cisco.com/ipj

• Networking products offered by Cisco Systems, as well as customer support services, can be

obtained at this URL:

http://www.cisco.com/en/US/products/index.html

• Networking Professionals Connection is an interactive website where networking professionals

share questions, suggestions, and information about networking products and technologies with Cisco experts and other networking professionals. Join a discussion at this URL:

http://www.cisco.com/discuss/networking

• “What’s New in Cisco Documentation” is an online publication that provides information about the

latest documentation releases for Cisco products. Updated monthly, this online publication is organized by product category to direct you quickly to the documentation for your products. You can view the latest release of “What’s New in Cisco Documentation” at this URL:

http://www.cisco.com/univercd/cc/td/doc/abtunicd/136957.htm

• World-class networking training is available from Cisco. You can view current offerings at

this URL:

http://www.cisco.com/en/US/learning/index.html

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preface

Obtaining Additional Publications and Information

CHA PT ER

1-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Product Overview

The Cisco SFS 7000P and SFS 7000D Server Switches provide high-class data center infrastructure that interconnect discreet server resources together into high-performance, low-latency fabric switches.

The Cisco SFS 7000P and SFS 7000D Server Switches are described in the following sections:

• Introduction to Switches, page 1-1

• Switch Connectors, page 1-2

• Switch Components, page 1-2

• System Features, page 1-3

• Administrative Features, page 1-5

Introduction to Switches

The Cisco SFS 7000P and SFS 7000D Server Switches offer features that are described in the following sections. Figure 1-1 shows an illustration of the Cisco SFS 7000P Server Switch, and Figure 1-2 shows an illustration of the Cisco SFS 7000D Server Switch.

Figure 1-1 Cisco SFS 7000P Server Switch

154903

w e

r S

Power Supply

1-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 1 Product Overview

Switch Connectors

Figure 1-2 Cisco SFS 7000D Server Switch

Switch Connectors

The Cisco SFS 7000P Server Switch uses the following connectors:

• Twenty-four 10-Gbps 4x copper InfiniBand ports for server and interswitch connectivity

• One 10/100 Ethernet RJ-45 management-Ethernet port for out-of-band management

• One RJ-45 console port used to configure and monitor the Cisco SFS 7000P Server Switch

The Cisco SFS 7000D Server Switch uses the following connectors:

• Twenty-four dual-speed 20-Gbps 4x copper InfiniBand ports that are capable of operating in either

20-Gbps DDR or 10-Gbps SDR mode for server and interswitch connectivity

• One 10/100 Ethernet RJ-45 management-Ethernet port for out-of-band management

• One RJ-45 console port used to configure and monitor the Cisco SFS 7000D Server Switch

Note There is a performance degradation if you mix 10-Gbps and 20-Gbps InfiniBand ports.

Switch Components

This section describes the following components of the Cisco SFS 7000P and SFS 7000D Server Switches and contains these sections:

• LEDs, page 1-3

• Power Supply Bay, page 1-3

• Power Supply Module, page 1-3

180600

Power Supply

1-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 1 Product Overview

System Features

LEDs

The Cisco SFS 7000P and SFS 7000D Server Switches feature the following LEDs:

• Chassis LEDs—Show overall system status, power status, and fan status. LEDs and their use are

described in the “Chassis Status LEDs” section on page 5-1.

• InfiniBand Port LEDs—Show link status, diagnostics, and network traffic. LEDs and their uses are

described in the “InfiniBand Port LEDs” section on page 5-3.

Power Supply Bay

The chassis provides dual independent IEC connectors that are left- and right-justified.

Power Supply Module

The power supply module, which contains the power supply and fan unit, has the following features:

• Redundant and hot-swappable— The replacement of any one power supply module does not disrupt

the operation of the device and can be successfully completed without removing the device from a rack or disconnecting any cables.

• Auto-ranging 90 to 264 VAC, 47 to 63 Hz.

• Redundant, hot-swappable cooling.

See the “Installing a Power Supply Module” section on page 4-2 for more information.

System Features

This section describes the system features of the Cisco SFS 7000P and SFS 7000D Server Switches and includes the following topics:

• InfiniBand Connectivity, page 1-3

• Scalability, page 1-4

• High Availability, page 1-4

• Nonblocking Architecture, page 1-5

InfiniBand Connectivity

The Cisco SFS 7000P and SFS 7000D Server Switches can be used in a variety of networking environments, including database tiers, application tiers, and World Wide Web tiers.

The Cisco SFS 7000P Server Switch provides 10 Gbps connectivity to servers, while the Cisco SFS 7000D Server Switch provides 20 Gbps DDR or 10 Gbps SDR modes for connectivity to servers.

Note Copper passive DDR cables are limited to 8 meters in length. If you want to use a longer SDR cable in

place of a DDR cable, the port must be comfigured manually to run an SDR rate.

1-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 1 Product Overview

System Features

Scalability

The embedded subnet manager running on the Cisco SFS 7000P and SFS 7000D Server Switches can manage up to 1,152 hosts.

High Availability

High availability operates at the hardware, port, and fabric level. These features are described below.

Hardware

High availability at the hardware level means that the Cisco SFS 7000P and SFS 7000D Server Switches feature hot-swappable redundant power and cooling.

Ports

High availability is supported at the ports level such that if any single InfiniBand port fails, none of the other ports have interrupted service.

Fabric

High availability is supported at the fabric level as well. For redundancy, InfiniBand HCAs can be dual-connected to a redundant pair of switches.

When an InfiniBand fabric includes more than one Cisco SFS 7000P Server Switch or Cisco SFS 7000D Server Switch, and the subnet manager fails on the Cisco SFS 7000P Server Switch or the Cisco SFS 7000D Server Switch that is acting as the master, another subnet manager takes over within seconds. (See Figure 1-3.)

Figure 1-3 Example of Redundant InfiniBand Fabric

Note A performance degradation occurs if you mix 10-Gbps and 20-Gbps InfiniBand ports.

HCA HCA

EN mgmt

ports

EN mgmt

ports

Switch 1 Switch 1

InfiniBand

Chassis A

InfiniBand Chassis B

IB Host 1 IB Host 2

InfiniBand connections

154001

1-5

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 1 Product Overview

Administrative Features

Nonblocking Architecture

The Cisco SFS 7000P and SFS 7000D Server Switches provide nonblocking switch element architecture with full bisectional bandwidth for the switch chassis.

Administrative Features

This section describes the administrative features of the Cisco SFS 7000P and SFS 7000D Server Switches and includes these topics:

• Real-Time Clock, page 1-5

• Latency, page 1-5

• Nonvolatile Memory, page 1-5

• Diagnostics, page 1-5

Real-Time Clock

A real-time clock maintains correct time regardless of power conditions or connectivity.

Latency

The Cisco SFS 7000P Server Switch has port to port latency of less than 200ns.

The Cisco SFS 7000D Server Switch has port to port latency of less than 140ns

Nonvolatile Memory

The nonvolatile memory supports the following items:

• Three stored system images (not including recovery image)

• One week of log files at normal verbosity and one day of log files at maximum verbosity

Diagnostics

The following tests are used to determine operational status:

• POST is performed on all system components and is required during power-on to determine

operational readiness.

• Tests are available to check the operational status of the redundant component periodically during

normal operation, including the logic required to perform the transition from faulted or primary to redundant component. The test reports the detection of an abnormal status.

• Some other diagnostics that you can perform are card self-test, chassis self-test, and fan self-test.

See Chapter 6, “Hardware Diagnostic Tests” for more information.

1-6

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 1 Product Overview

Administrative Features

CHA PT ER

2-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Preparation for Installation

Planning a proper location for the Cisco SFS 7000P and SFS 7000D Server Switches and the proper layout of your equipment rack is essential for successful system operation. This chapter describes how to prepare your site for switch installation and contains these sections:

• Safety, page 2-1

• Site Requirements, page 2-1

• Power Requirements, page 2-5

• Site Preparation Checklist, page 2-6

Safety

Safety warnings appear throughout this document in procedures that may harm you if performed incorrectly. A warning symbol precedes each warning statement. The warnings below are general warnings that are applicable to the entire publication.

Warning

Only trained and qualified personnel should be allowed to install, replace, or service this equipment.

Statement 1030

Warning

This unit is intended for installation in restricted access areas. A restricted access area can be accessed only through the use of a special tool, lock and key, or other means of security.

Statement 1017

Site Requirements

These sections describe some of the basic site requirements that you should be aware of as you prepare to install your Cisco SFS 7000P and SFS 7000D Server Switches. Environmental factors can adversely affect the performance and longevity of your system. Planning a proper location for the switch and layout of your equipment rack or wiring closet is essential for successful system operation. You should install the switch in an enclosed, secure area, ensuring that only qualified personnel have access to the switch and control of the environment. Equipment that is placed too closely together or placed in inadequately ventilated areas can cause system overtemperature conditions leading to premature component failures. In addition, poor equipment placement can make chassis panels inaccessable and difficult to maintain.

2-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 2 Preparation for Installation

Site Requirements

The switch requires a dry, clean, well-ventilated, and air-conditioned environment. To ensure normal operation, maintain ambient airflow. If the airflow is blocked or restricted, or if the intake air is too warm, an overtemperature condition can occur. The switch environmental monitor can then shut down the system to protect the system components.

Multiple switches can be rack-mounted with little or no clearance above and below the chassis. However, when mounting a switch in a rack with other equipment, or when placing it on the floor near other equipment, ensure that the exhaust from other equipment does not blow into the air intake vent of the switch chassis.

Temperature

Temperature extremes can cause a system to operate at reduced efficiency and cause a variety of problems, including premature aging and failure of chips and failure of mechanical devices. Extreme temperature fluctuations can cause chips to become loose in their sockets. Ensure that the system is operating in an environment that does not exceed the specifications in Table A-1 on page A-1. Ensure that the chassis has adequate ventilation. Do not place the chassis within a closed-in wall unit or on top of cloth, which can act as insulation. Do not place it where it will receive direct sunlight, particularly in the afternoon. Do not place it next to a heat source of any kind, including heating vents during winter. Adequate ventilation is particularly important at high altitudes. Make sure that all slots and openings on the system remain unobstructed, especially the fan vent on the chassis. Clean the installation site at regular intervals to avoid buildup of dust and debris, which can cause a system to overheat. If the system has been exposed to abnormally cold temperatures, allow a two-hour warm-up period to bring it up to normal operating temperature before turning it on. Failure to do so can damage internal components.

Humidity

High-humidity conditions can cause moisture migration and penetration into the system. This moisture can cause corrosion of internal components and degradation of properties, such as electrical resistance, thermal conductivity, physical strength, and size. Extreme moisture buildup inside the system can result in electrical shorts, which can cause serious damage to the system. Ensure that the system is operating in an environment that does not exceed the specifications in Table A-1 on page A-1. Buildings in which climate is controlled by air-conditioning in the warmer months and by heat during the colder months usually maintain an acceptable level of humidity for system equipment. However, if a system is located in an unusually humid location, a dehumidifier can be used to maintain the humidity within an acceptable range.

Altitude

Operating a system at high altitude (low pressure) reduces the efficiency of forced and convection cooling and can result in electrical problems related to arcing and corona effects. This condition can also cause sealed components with internal pressure, such as electrolytic capacitors, to fail or perform at reduced efficiency. Ensure that the system is operating in an environment that does not exceed the specifications inTable A-1 on page A-1.

2-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 2 Preparation for Installation

Site Requirements

Dust and Particles

Exhaust fans cool power supplies and system fan trays cool systems by drawing in air and exhausting air out through various openings in the chassis. However, fans also ingest dust and other particles, causing contaminant buildup in the system and increased internal chassis temperature. A clean operating environment can greatly reduce the negative effects of dust and other particles, which act as insulators and interfere with the mechanical components in the system. In addition to regular cleaning, follow these precautions to avoid contamination of your equipment:

• Do not permit smoking anywhere near the system.

• Do not permit food or drink near the system.

Corrosion

Corrosion of system connectors is a gradual process that can eventually lead to intermittent failures of electrical circuits. The oil from a person’s fingers or prolonged exposure to high temperature or humidity can corrode the gold-plated edge connectors and pin connectors on various components in the system. To prevent corrosion, avoid touching contacts on boards and cards, and protect the system from extreme temperatures and moist, salty environments.

Electromagnetic and Radio Frequency Interference

EMI and RFI from a system can adversely affect devices such as radio and TV receivers operating near the system. Radio frequencies emanating from a system can also interfere with cordless and low-power telephones. Conversely, RFI from high-power telephones can cause spurious characters to appear on the system monitor. RFI is defined as any EMI with a frequency above 10 kHz. This type of interference can travel from the system to other devices through the power cable and power source or through the air like transmitted radio waves. The FCC publishes specific regulations to limit the amount of EMI and RFI emitted by computing equipment. Each system meets these FCC regulations. To reduce the possibility of EMI and RFI, follow these guidelines:

• Operate the system only when the chassis cover is installed.

• Ensure that all expansion slots are covered either by a card-mounting bracket or by a metal filler

bracket and that all drive bays have either a drive or a metal insert installed.

• Ensure that the screws on all peripheral cable connectors are securely fastened to their

corresponding connectors on the back of the chassis.

• Always use shielded cables with metal connector shells for attaching peripherals to the system.

When wires are run for any significant distance in an electromagnetic field, interference can occur between the field and the signals on the wires. This fact has two implications for the construction of plant wiring:

• Bad wiring practice can result in radio interference emanating from the plant wiring.

• Strong EMI, especially when it is caused by lightning or radio transmitters, can destroy the signal

drivers and receivers in the chassis and even create an electrical hazard by conducting power surges through lines into equipment.

Note To predict and remedy strong EMI, you might need to consult experts in RFI.

2-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 2 Preparation for Installation

Site Requirements

If you use twisted-pair cable in your plant wiring with sufficient distribution of grounding conductors, the plant wiring is unlikely to emit radio interference. If you exceed the recommended distances, use a high-quality twisted-pair cable with one ground conductor for each data signal when applicable.

If the wires exceed the recommended distances or if wires pass between buildings, carefully consider the effect of a lightning strike in your vicinity. The electromagnetic pulse caused by lightning or other high-energy phenomena can easily couple enough energy into unshielded conductors to destroy electronic devices. If you have had problems of this sort in the past, you may want to consult experts in electrical surge suppression and shielding.

Shock and Vibration

Cisco SFS 7000P and SFS 7000D Server Switches have been shock- and vibration-tested for operating ranges, handling, and office vibration and criteria, transportation vibration and criteria, and packaged equipment shock. (See Table A-1 on page A-1.)

Power Source Interruptions

Systems are especially sensitive to variations in voltage supplied by the AC power source. Overvoltage, undervoltage, and transients (or spikes) can erase data from memory or even cause components to fail. To protect against these types of problems, power cables should be grounded properly. Also, place the system on a dedicated power circuit (rather than sharing a circuit with other heavy electrical equipment). In general, do not allow the system to share a circuit with any of the following:

• Copy machines

• Air conditioners

• Vacuum cleaners

• Space heaters

• Power tools

• Teletype machines

• Laser printers

• Facsimile machines

• Any other motorized equipment

Besides these appliances, the greatest threats to a system power supply are surges or blackouts that are caused by electrical storms. Whenever possible, turn off the system and any peripherals, and unplug them from their power sources during thunderstorms. If a blackout occurs—even a temporary one—while the system is turned on, turn off the system immediately and disconnect it from the electrical outlet. Leaving the system on may cause problems when the power is restored; all other appliances left on in the area can create large voltage spikes that can damage the system.

Preventing Electrostatic Discharge Damage

ESD damage, which can occur when modules or other FRUs are improperly handled, results in intermittent or complete failures. Modules consist of printed circuit boards that are fixed in metal carriers. EMI shielding and connectors are integral components of the carrier. Although the metal carrier helps to protect the circuit board from ESD, always use an ESD grounding strap when handling modules.

2-5

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 2 Preparation for Installation

Power Requirements

Air Flow

Note The air flows from front to back through the Cisco SFS 7000P and SFS 7000D Server Switches. (See

Figure 2-1.)

Figure 2-1 Air Flow Direction in the Cisco SFS 7000P and SFS 7000D Server Switches

Follow these guidelines to ensure proper airflow:

• Maintain ambient airflow throughout the data center to ensure normal operation.

• If installing in an enclosed cabinet, ensure that the cabinet has adequate ventilation, and allow at

least 2.5 inches (6.4 cm) of clearance between the chassis vents and the cabinet wall.

• If placing the switch adjacent to a device that exhausts air towards the switch, the horizontal distance

between the devices should be a minimum of 6 inches (15.2 cm).

• Ensure that cables do not obstruct the airflow through the chassis.

• Consider heat dissipation of all equipment when determining the air-conditioning requirements, as

specified in the hardware installation guide. When evaluating airflow requirements, consider that hot air generated by equipment at the bottom of the rack can be drawn into the intake ports of the equipment above.

Power Requirements

Follow these requirements when preparing your site for the switch installation:

• In systems configured with two power supplies, connect each of the two power supplies to a separate

input power source. If you fail to do this, your system might be susceptible to total power failure due to a fault in the external wiring or a tripped circuit breaker.

• To prevent a loss of input power, be sure that the total maximum load on each source circuit is within

the current ratings of the wiring and breakers.

• The AC power receptacles used to plug in the chassis must be the grounding type. The grounding

conductors that connect to the receptacles should connect to protective earth ground at the service equipment.

181224

Front

Back

2-6

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 2 Preparation for Installation

Site Preparation Checklist

Table 2 -1 lists the site planning activities that you should perform prior to installing the

Cisco SFS 7000P and SFS 7000D Server Switches. Completing each activity helps ensure a successful switch installation.

Table 2-1 Site Planning Checklist

Task No. Planning Activity Verified By Time Date

1 Space evaluation:

• Space and layout

• Floor covering

• Impact and vibration

• Lighting

• Maintenance access

2 Environmental evaluation:

• Ambient temperature

• Humidity

• Altitude

• Atmospheric contamination

• Airflow

3 Power evaluation:

• Receptacle proximity to the equipment

• Dedicated (separate) circuits for redundant

power supplies

• UPS for power failures

4 Grounding evaluation:

• Circuit breaker size

5 EMI evaluation:

• Distance limitations for signaling

• Site wiring

• RFI levels

CHA PT ER

3-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Installing the Switches

This chapter describes how to install and manage the Cisco SFS 7000P and SFS 7000D Server Switches hardware and contains these sections:

• Safety, page 3-2

• Pre-installation Instructions, page 3-2

• Rack-Mounting the Switches, page 3-3

• Configuring Basic Connectivity, page 3-7

• Connecting Network Devices, page 3-11

• Managing the System, page 3-14

Warning

During this procedure, wear grounding wrist straps to avoid ESD damage to the card. Do not directly touch the backplane with your hand or any metal tool, or you could shock yourself.

Statement 94

Warning

Read the installation instructions before connecting the system to the power source.

Statement 1004

Warning

This unit is intended for installation in restricted access areas. A restricted access area can be accessed only through the use of a special tool, lock and key, or other means of security.

Statement 1017

Warning

Only trained and qualified personnel should be allowed to install, replace, or service this equipment.

Statement 1030

Warning

Ultimate disposal of this product should be handled according to all national laws and regulations.

Statement 1040

3-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Safety

Warning

Voltage is present on the backplane when the system is operating. To reduce risk of an electric shock, keep hands and fingers out of the power supply bays and backplane areas.

Statement 166

Warning

This equipment must be grounded. Never defeat the ground conductor or operate the equipment in the absence of a suitably installed ground conductor. Contact the appropriate electrical inspection authority or an electrician if you are uncertain that suitable grounding is available.

Statement 1024

Warning

This equipment must be installed and maintained by service personnel as defined by AS/NZS 3260. Incorrectly connecting this equipment to a general-purpose outlet could be hazardous. The telecommunications lines must be disconnected 1) before unplugging the main power connector or 2) while the housing is open, or both.

Statement 1043

Warning

This product requires short-circuit (overcurrent) protection, to be provided as part of the building installation. Install only in accordance with national and local wiring regulations.

Statement 1045

Warning

Class 1M laser radiation when open. Do not view directly with optical instruments.

Statement 1053

Warning

Use of controls, adjustments, or performing procedures other than those specified may result in hazardous radiation exposure.

Statement 1057

Pre-installation Instructions

Caution Before attempting any installation, read Chapter 2, “Preparation for Installation.”

To safely and successfully prepare the environment for your Cisco SFS 7000P and SFS 7000D Server Switches, you need to do the following:

• Check the contents of the switch package—For more information, see the parts list in the installation

and configuration note that shipped with your switch.

• Always use an approved ESD-preventative ankle or wrist strap.

• Make sure you have the correct cables and sufficient ventilation.

• Prepare a management workstation, such as a PC running a terminal program, with the cable and

adapter required for your specific configuration.

3-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Rack-Mounting the Switches

Warning

Two people are required to lift the chassis. Grasp the chassis underneath the lower edge and lift with both hands. To prevent injury, keep your back straight and lift with your legs, not your back. To prevent damage to the chassis and components, never attempt to lift the chassis with the handles on the power supplies or on the interface processors, or by the plastic panels on the front of the chassis. These handles were not designed to support the weight of the chassis.

Statement 5

Warning

To prevent bodily injury when mounting or servicing this unit in a rack, you must take special precautions to ensure that the system remains stable. The following guidelines are provided to ensure your safety:

• This unit should be mounted at the bottom of the rack if it is the only unit in the rack.

• When mounting this unit in a partially filled rack, load the rack from the bottom to the top with the heaviest component at the bottom of the rack.

•If the rack is provided with stabilizing devices, install the stabilizers before mounting or servicing the unit in the rack.

Statement 1006

This section describes how to install the Cisco SFS 7000P and SFS 7000D Server Switches in an equipment rack.

Requirements

In addition to the accessories provided with the switch, you need the following:

• A number 2 Phillips screwdriver

• 8 rack screws specific to your rack

• Any associated mounting clips to secure the brackets to your rack (two for each rail of the rack)

Caution Installing the rack to the system with screws other than the ones recommended could be hazardous.

• A rack with enough clearance for the server switch cables

• Two people to perform the installation (highly recommended)

Note The Cisco SFS 7000P and SFS 7000D Server Switches weigh more than 22 pounds (10 kilograms) each.

Rack-Mounting the Switch

To mount the Cisco SFS 7000P and SFS 7000D Server Switches in a rack, follow these steps:

Step 1 Remove the switch, rack-mount brackets, contents of accessory kit, and documentation from the box. Step 2 Place the switch on a secure, clean surface. Step 3 Record the MAC address and the Globally Unique Identifier (GUID) number for future reference.

3-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Rack-Mounting the Switches

The MAC address and GUID number are available on sticker labels placed on top of the switch.

Step 4 Check the slot in the rack for sufficient clearance. Step 5 Determine the desired method of installation, and follow the appropriate steps:

• Installing the Switch with One Person, page 3-4

• Installing the Switch with Two People, page 3-6

Installing the Switch with One Person

Because it can be difficult for one person to align the switch correctly along both rack rails, the following installation method is easier with two people but can be accomplished with one person.

To install the switch with one person, follow these steps:

Step 1 Separate one set of the rack brackets. (See Figure 3-1.)

Each side is assembled to its counterpart but should be separated before you attach them to the switch when you use this installation method.

Figure 3-1 Assembled Rack Brackets

Step 2

Attach the bracket that has the screw holes to the side of the switch, with the flange facing away from the switch, as shown in Figure 3-2.

Figure 3-2 Attaching One Rail to Switch Chassis

The standard method is to face the flange toward the front of the switch. However, you can also mount the flange toward the back if you want to mount the switch backward in the switch (service-side forward), as shown in Figure 3-2.

Step 3 Repeat Step 1 and Step 2 on the opposite side of the switch to attach the second bracket.

The two counterparts to these sliding rack brackets (that do not have screw holes) should still be unattached.

144949

144952

3-5

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Rack-Mounting the Switches

Step 4 Check the rack for clearance for the switch. The switch can be installed either directly on top of another

device or be suspended from the rack posts.

Step 5 Attach the remaining two rack brackets to your rack.

a. Position a bracket toward the back of the rack with the flange facing away from the rack. The flange

should go around the outside of the rack post, as shown in Figure 3-3.

Figure 3-3 Holding Bracket Inside of Rack

Note If you are rack-mounting the switch backward, the bracket should be installed to the front of the

rack.

b. Secure the bracket with the screws through the back of the rack, as shown in Figure 3-4. c. Repeat on the other side of the rack.

Figure 3-4 Secure Rail Bracket to Rack with Screws

Step 6

Lift the switch unit, and align the sliding brackets to the switch with the brackets attached to the posts before sliding them together.

144954

144953

3-6

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Rack-Mounting the Switches

Step 7 Carefully push the switch unit into the rack, as shown in Figure 3-5.

If the brackets do not slide easily, the alignment might be off. Pull the switch back toward you, and realign the brackets.

Figure 3-5 Sliding the Switch Into the Rack

Step 8

Maintain at least six inches between the cooling vents and any obstructions.

Step 9 Secure the switch with your screws through the front of the rack, as shown in Figure 3-6.

Figure 3-6 Secure Switch with Screws through the Front of the Rack

Installing the Switch with Two People

The following installation method requires two people to mount the switch into the rack. One person holds the switch while another person secures it to the rack.

To install the switch with two people, follow these steps:

Step 1 Separate the assembled rack brackets. (See Figure 3-1.)

144954

144955

3-7

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Configuring Basic Connectivity

Step 2 Attach the rack bracket that has screw holes to the sides of the switch by using the screws provided.

When attaching the rack bracket, the flanges of the rack bracket should be facing away from the switch. (See Figure 3-2.)

Step 3 Attach the rack brackets to their counterparts before inserting the switch into the rack. Step 4 Insert the switch into the rack with the rack bracket attached. You must tilt the switch unit to one side to

avoid hitting the sides of the rack bracket as they pass around the back rails.

Return the switch unit to a horizontal position after the switch is inside the rack. The rear bracket flanges should wrap around the outside of the back rack post. (See Figure 3-4.)

Step 5 Have one person hold the switch while another person secures the switch to the rack.

If required, adjust the length of the rack brackets to fit the dimensions of the rack posts.

Step 6 Maintain at least six inches between the cooling vents and any obstructions. Step 7 Attach the rack bracket to the back of the rack posts with screws that fit your rack. Step 8 Attach the front rails to the front of the rack with screws that fit your rack. (See Figure 3-6.)

Configuring Basic Connectivity

The following sections appear in this section:

• Obtaining an IP Address, page 3-7

• Connecting the 10/100 Ethernet Management Port, page 3-10

• Powering Up the Chassis, page 3-10

Obtaining an IP Address

Obtaining an IP Address from the DHCP Server for Ethernet Management

This section describes how to obtain an IP address from the Dynamic Host Configuration Protocol (DHCP) server by default.

Note DHCP is enabled by default in the Cisco SFS 7000D Server Switch. To obtain an IP address for the Cisco

SFS 7000P Server Switch, see “Obtaining a Static IP Address for Ethernet Management” section on

page 3-8.

To obtain an IP address using the default enabled DHCP, you must obtain the MAC address. The MAC address is printed on a label on your Cisco SFS 7000D Server Switch in the following format: AA:BB:CC:DD:EE.FF. (See the instructions in the “Rack-Mounting the Switch” section on page 3-3 for when to make a note of the MAC address.) You must configure your DHCP server to the MAC address before you attempt to obtain an IP address using DHCP.

Use the MAC address to configure your DHCP server to allocate an IP address of your choice to the Cisco SFS 7000D Server Switch management interface. Otherwise, the DHCP server (if present) allocates the first available IP address from the designated pool of addresses. In the latter case you can

3-8

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Configuring Basic Connectivity

find out which address was allocated by logging to the CLI on the serial console and using the show interface mgmt-eth command. See the Cisco SFS 7000 Series Product Family Command Reference for

further details on using CLI commands.

Use the IP address, to Telnet to the Cisco SFS 7000D Server Switch.

Obtaining a Static IP Address for Ethernet Management

A Cisco SFS 7000P Server Switch user must configure a static IP Address for Ethernet Management. A Cisco SFS 7000D Server Switch user can configure a static IP address and connect to a serial console port only if a DHCP server is not available.

To configure a static IP address, you must connect the serial console port to a PC serial for access to the server switch.

Before you configure the switch, ensure that the unit is connected properly. This section describes how to attach a serial console cable to a PC. The serial console port, located on the rear panel of the Cisco SFS 7000P and SFS 7000D Server Switches, is shown in Figure 3-7 and Figure 3-8.

Figure 3-7 Connecting the Console Cable to a Serial Console Port on the Cisco SFS 7000P Server

Switch

Figure 3-8 Connecting the Console Cable to a Serial Console Port on the Cisco SFS 7000D Server

Switch

181002

Ethernet management port

Console

cable

Serial

console

port

180944

Ethernet

management

port

Serial

console

port

Console

cable

3-9

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Configuring Basic Connectivity

You can connect the serial console port to a PC serial port for local administrative access to the switch. A terminal emulation software product, such as HyperTerminal, makes communication between your switch and your PC possible during setup and configuration.

To connect the console port to a PC, follow these steps:

Step 1 Configure the baud rate and character format of the PC terminal emulation program to match the

following management port default characteristics:

• Baud: 9600 bps

• Data Bits: 8

• Parity: None

• Stop Bits: 1

Step 2 Connect the supplied DB-9/RJ-45 female adapter or DB-25/RJ-45 female adapter (depending on your

PC connection) to the PC serial port.

Step 3 Connect one end of the supplied console cable (a flat rollover RJ-45 cable) to the serial console port.

(See Figure 3-7 and Figure 3-8.) Connect the other end to the DB-9/RJ-45 female adapter or DB-25/RJ-45 adapter at the PC serial port.

After you connect the serial console port, you may configure a static IP address for the Ethernet management interface by following these steps:

Step 1 Wait for the login prompt.

Step 2 Enter the username and password. The default username is super, and the default password is super.

Step 3 Enter the enable command.

SFS-7000D> enable SFS-7000D #

Step 4 Enter the configure command to enter the configuration mode.

SFS-7000D # configure SFS-7000D(config)#

Step 5 Enter the interface mgmt-ethernet command to enter the config-if-mgmt-ethernet mode.

SFS-7000D(config)# interface mgmt-ethernet SFS-7000D(config-if-mgmt-ethernet)#

Step 6 Enable the static address option, which turns off the default-enabled DHCP.

SFS-7000D(config)# address-option static

Step 7 Enter the IP address of the management interface, followed by the netmask, which enables the optional

static address.

SFS-7000D(config-if-mgmt-ethernet)# ip address ip address netmask

3-10

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Configuring Basic Connectivity

Step 8 Set the default gateway for the management interface. See your system administrator for a gateway

address.

SFS-7000D(config-if-mgmt-ethernet)# gateway ip address

Step 9 Enable the management interface by entering the no shutdown command.

SFS-7000D(config-if-mgmt-ethernet)# no shutdown

Step 10 Save the configuration to preserve it between reboots.

SFS-7000D(config-if-mgmt-ethernet)# exit SFS-7000D(config)# exit SFS-7000D# copy running-config startup-config

Connecting the 10/100 Ethernet Management Port

The autosensing 10/100 Ethernet management port is located on the rear panel/port connectors side of the Cisco SFS 7000P and SFS 7000D Server Switches. (See Figure 3-7 and Figure 3-8.) The Ethernet management port is used for out-of-band management of the Cisco SFS 7000P and SFS 7000D Server Switches.

Use the CAT-5 twisted pair RJ-45 straight-through cable to connect the 10/100 Ethernet management port to external hubs, switches, and routers.

Powering Up the Chassis

Warning

Hazardous voltage or energy is present on the backplane when the system is operating. Use caution when servicing.

Statement 1034

When you power up the chassis, use only the power cable provided with your InfiniBand system.

To power up the chassis, follow these steps:

Step 1 Remove the power cords from the shipping package.

Two power cords are available: UL-rated, 10 amps and 125 VAC or greater.

Step 2 Inspect the power cord to determine if it provides the proper plug and is appropriately certified for use

with your electrical system. Discard the cord if it is inappropriate for your national electrical system, and obtain the proper cord, as required by your national electrical codes or ordinances.

Grounding is supplied by the ground prong on the three-prong power plug. Do not attach a separate ground cable, and do not use adapter plugs. Do not remove the ground prong from the cable. Be sure that the ground connection on the power supply is correct and functioning before applying power to the switch.

Step 3 Connect the power cords to the power connectors on the rear of the switch.

The system automatically boots up. You can watch the operation status by using the serial console.

For power connector locations for the Cisco SFS 7000P Server Switch, see Figure 3-9. For power connector locations for the Cisco SFS 7000D Server Switch, see Figure 3-10.

3-11

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Connecting Network Devices

Figure 3-9 Power Connector Locations for the Cisco SFS 7000P Server Switch

Figure 3-10 Power Connector Locations for the Cisco SFS 7000D Server Switch

Step 4

Connect the other end of each AC power cable into a 90-to 264-VAC power outlet operating between 47 Hz and 63 Hz.

The switch automatically starts and boots up. Use the correct external power source. Attach the switch only to approved power sources, as indicated by the electrical ratings label. If you are unsure of the correct power-source to use, contact your support personnel or your local power company.

Step 5 Check the LEDs on the front of the Server Switch.

When the device first powers up, it performs a POST. (See the “LEDs” section on page 5-1.)

Step 6 Log in and assign a network address after you see the login prompt.

The default login is: super, and the default password is: super.

For additional management information, see the Cisco SFS 7000 Series Product Family Command Reference.

Connecting Network Devices

This section describes how to connect the InfiniBand switches to other network devices. These InfiniBand switches can then be interconnected to InfiniBand devices such as InfiniBand-enabled servers.

The Cisco SFS 7000P and SFS 7000D Server Switches support the following types of connectors:

• Serial console port

• Ethernet Management port—RJ-45 jack for unshielded twisted-pair connections

• Cisco SFS 7000P Server Switch—4x 10-Gbps InfiniBand connectors

and Cisco SFS 7000D Server Switch—4x 10/20-Gbps InfiniBand connectors

Note There is a performance degradation if you mix 10-Gbps and 20-Gbps InfiniBand ports.

144945

Power connector Power connector

180601

Power connector

3-12

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Connecting Network Devices

Connecting InfiniBand Devices

Warning

Invisible laser radiation may be emitted from the end of the unterminated fiber cable or connector. Do not view directly with optical instruments. Viewing the laser output with certain optical instruments (for example, eye loupes, magnifiers, and microscopes) within a distance of 100 mm may pose an eye hazard.

Statement 1056

To use InfiniBand to connect to other workstations or switches, you need standard 4x InfiniBand cables. InfiniBand cables can be used to connect any two InfiniBand devices, whether a switch or a host. (Figure 3-11 and Figure 3-12 show InfiniBand ports on the Cisco SFS 7000P and SFS 7000D Server Switches.) For more information about InfiniBand cables, see the InfiniBand SDR, DDR, and QDR Technology Overview.

Note Copper passive DDR cables are limited to 8 meters in length. If you want to use a longer SDR cable in

place of a DDR cable, the port must be comfigured manually to run an SDR rate.

Figure 3-11 Port Connections for the Cisco SFS 7000P Server Switch

Figure 3-12 Port Connections for the Cisco SFS 7000D Server Switch

To connect InfiniBand devices, follow these steps:

133860

InfiniBand port

connectors

Serial

console

port

Ethernet

management

port

180602

InfiniBand port

connectors

Serial

console

port

Ethernet

management

port

3-13

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Connecting Network Devices

Step 1 Connect InfiniBand cables from the host to the InfiniBand switch.

a. To connect an InfiniBand cable, push the connector into the interface until you hear or feel a click.

Figure 3-13 shows an InfiniBand cable with a pinch connector, and Figure 3-14 shows an InfiniBand

cable with a pull connector.

Figure 3-13 InfiniBand Cable with Pinch Connector

Figure 3-14 InfiniBand Cable with Pull Connector

Note If your host does not provide an ample amount of space around an InfiniBand port, double-check that

your InfiniBand cable connector engages fully. Move your connector back and forth to be sure that both sides of the connector have locked firmly into place.

b. To remove a cable with a pinch connector, pinch both sides of the back of the connector and pull the

connector away from the port. (See Figure 3-15.)

144960

144961

3-14

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Managing the System

Figure 3-15 Removing a Pinch Connector

To remove a cable with a pull connector, grasp the connector with one hand and push it toward the port, then pull the latch away from the port with your other hand and gently move the connector back and forth and away from the port. (See Figure 3-16.)

Figure 3-16 Removing a Pull Connector

Connecting Ethernet Management Devices

To connect the Ethernet management port to external devices such as switches, routers, and hubs, use an Ethernet cable. For the location of the Ethernet management port, see Figure 3-11 and Figure 3-12.

Managing the System

You can manage the InfiniBand system using the following methods:

144962

Press here

144963

Hold here

Pull here

3-15

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Managing the System

• Command-Line Interface (CLI)—A text-based interface accessible through a direct serial

connection, Telnet over IP, or SSH over IP

• Chassis Manager (GUI)—A web-based graphical user interface

• Element Manager (GUI)—A graphical interface installed on a workstation, accessible over IP

See the Cisco SFS 7000 Series Product Family Command Reference, Cisco SFS 7000 Series Product Family Chassis Manager User Guide, and the Cisco SFS 7000 Series Product Family Element Manager User Guide for more information about managing InfiniBand systems.

3-16

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 3 Installing the Switches

Managing the System

CHA PT ER

4-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Installing Field Replaceable Units

This chapter describes how to install the following FRUs in the Cisco SFS 7000P and SFS 7000D Server Switches:

• Power Supply Modules, page 4-1

• Installing a Power Supply Module, page 4-2

• Removing Power Supply Modules, page 4-4

Power Supply Modules

The Cisco SFS 7000P and SFS 7000D Server Switches power supply modules are hot-swappable. You can add a second module without powering off a switch. If you have two power supply modules installed, you can remove one of them without removing power from the switch.

Locating the Power Supply Unit

The Cisco SFS 7000P and SFS 7000D Server Switches can have up to two power supply modules. Each power supply module contains a power supply and a fan unit. Both power modules are located at the front of the switch. When facing the front of the switch, the power supply modules are located in the left and right slots of the switch.

Failed Power Supply Modules

The status of a power supply module can be checked using the CLI, the Chassis Manager, or the Element Manager. The available information includes the status, the VPD, the description of the error, or an error code.

If you suspect that a power supply module has failed, check the LEDs and view the status through the Element Manager. For more information about LEDs, see the “Power Supply LEDs” section on page 5-3.

In most cases, vital information can be retrieved from the console port of the management Ethernet port. (See the “Diagnostics” section on page 1-5.)

See the Cisco SFS 7000 Series Product Family Chassis Manager User Guide or Cisco SFS 7000 Series Product Family Element Manager User Guide for more information.

4-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 4 Installing Field Replaceable Units

Installing a Power Supply Module

Warning

Hazardous voltage or energy is present on the backplane when the system is operating. Use caution when servicing.

Statement 1034

Caution Never place your hand inside an empty module bay or anywhere inside the

Cisco SFS 7000P and SFS 7000D Server Switches.

Caution Unused module bays always should have a switch filler panel over the bay to ensure proper safety,

ventilation, and cooling.

To insert a power supply module, follow these steps:

Step 1 Ground yourself appropriately. Step 2 Remove the filler panel from the power supply bay with a number 1 Phillips or a slotted screwdriver, if

it is still in place. (See Figure 4-1.)

Figure 4-1 Removing a Filler Panel from the Switch

180916

4-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 4 Installing Field Replaceable Units

Installing a Power Supply Module

Step 3 Remove the power supply module if one is in place by pulling on the black handle. (See Figure 4-2.)

Figure 4-2 Removing the Power Supply from the Switch

Step 4 Insert the new power supply module into the open slot until it is fully seated. You might need to push the

unit with your thumbs to ensure that it is seated completely in the bay.

Step 5 Secure fasteners with a number 1 Phillips or a slotted screwdriver. Step 6 Check the LEDs to verify the status of the module. (See Figure 4-3 for the location of the LEDs.)

For more information, see the “Power Supply LEDs” section on page 5-3.

Figure 4-3 Power Supply LEDs

154011

154012

Power Supply

LEDs

4-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 4 Installing Field Replaceable Units

Removing Power Supply Modules

Caution Do not remove the power supply module without first removing the screws. Use a number 1 Phillips or

a slotted screwdriver.

To remove a power supply module, follow these steps:

Step 1 Make sure you have a number 1 Phillips or a slotted screwdriver to disengage the fasteners. Step 2 Ground yourself appropriately.

Caution Never place your hand inside an empty card or module bay or anywhere inside the

Cisco SFS 7000P and SFS 7000D Server Switches.

Step 3 Locate the power supply module that you want to remove.

If you have two power supply modules installed, you can remove one of them without removing power from the switch.

Step 4 Unscrew the fasteners that hold the power supply module in place. (See Figure 4-4.)

Figure 4-4 Removing the Screws from the Power Supply of the Switch

Step 5

Pull the unit from the bay.

Step 6 Install the filler panel in place of the power supply module, or install a new power supply module. If you

are installing a new power supply module, see the “Installing a Power Supply Module” section on

page 4-2.

Caution Never operate the device without a filler panel or power supply module in place because the device might

overheat.

154010

CHA PT ER

5-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Managing the Switch

This chapter describes how to manage the Cisco SFS 7000P and SFS 7000D Server Switches hardware. The following sections appear in this chapter:

• LEDs, page 5-1

• Managing the System with Element Manager, page 5-4

• Displaying System Information, page 5-6

LEDs

The Cisco SFS 7000P and SFS 7000D Server Switches have the following types of LED indicators:

• Chassis Status LEDs, page 5-1

• InfiniBand Port LEDs, page 5-3

• Power Supply LEDs, page 5-3

Chassis Status LEDs

The front of the chassis has a single bi-color chassis status LED. The rear of the chassis has one green and one yellow chassis status LED that convey the identical information as the single chassis status LED in the front of the chassis. See Tab le 5-1 for information about interpreting the chassis status LED and see Figure 5-1 through Figure 5-4 for the locations of the chassis status LEDs.

Table 5-1 Interpreting the Chassis Status LED

Color Indication

Off No system power or LED failure.

Yellow (solid) Operator intervention required. A system error was detected,

such as a fan error, a POST failure, or a power supply failure.

The ! label (available on the back of the chassis) indicates a failure.

Yellow (blinking)

Initiated automatically during the LED test that follows the application of power (16 seconds).

Solid green Indicates proper operation and no critical errors.

5-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

LEDs

Figure 5-1 Front View of Cisco SFS 7000P Server Switch Showing Chassis Status LED

Figure 5-2 Front View of Cisco SFS 7000D Server Switch Showing Chassis Status LED

Figure 5-3 Rear View of Cisco SFS 7000P Server Switch Showing Chassis Status LEDs

Figure 5-4 Rear View of Cisco SFS 7000D Server Switch Showing Chassis Status LEDs

180716

Bi-color chassis status LED

Power Supply

Cisco Systems

180718

Bi-color chassis status LED

Power Supply

Status

Cisco Systems

SFS-7000D

180717

Yellow chassis status LED

Green chassis status LED

180719

Yellow chassis status LED

Green chassis status LED

5-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

LEDs

InfiniBand Port LEDs

The InfiniBand port LED is located next to each InfiniBand port. The InfiniBand LED represents the logical link and the logical link activity. For InfiniBand port LED locations. See Figure 5-5 for the Cisco SFS 7000P Server Switch and Figure 5-6 for the Cisco SFS 7000D Server Switch.

Figure 5-5 InfiniBand Port LEDs Location for the Cisco SFS 7000P Server Switch

Figure 5-6 InfiniBand Port LEDs Location for the Cisco SFS 7000D Server Switch

Table 5 -2 describes the InfiniBand Port LED operation.

Power Supply LEDs

Each power supply module contains a power supply and a fan. The power supply module LEDs are located on the bottom left corner of each power supply module. For the location of the LEDs, see

Figure 5-7. For LED descriptions, see Tab le 5-3.

• The green LED is labeled with a DC symbol.

• The yellow LED is labeled with an exclamation point (!).

154013

Port LEDs

180704

Port LEDs

Table 5-2 InfiniBand Port LED

Color Indication

Off Logical link is not established.

Solid green Logical link is established.

Blinking green Logical link is established with activity.

5-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

Managing the System with Element Manager

Figure 5-7 Power Supply and Fan LEDs Location

Managing the System with Element Manager

For information about installing the Element Manager, see the Cisco SFS 7000 Series Product Family Element Manager User Guide.

To view the Health Status window, perform the following steps:

Step 1 Launch the Element Manager. Step 2 Select Health > Status. The Health Status window opens. (See Figure 5-8.)

Figure 5-8 Health Status Window

Table 5-3 Power Supply Module LEDs

Color Indication

Green (off) No DC input.

Green (solid) AC connected, DC output OK.

Yellow (off) No failure on the power supply.

Yellow (solid) Operator intervention required. Failure detected

within the power supply.

154012

Power Supply

LEDs

5-5

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

Managing the System with Element Manager

Using the Summary Tab

Use the Summary tab to view the status of the power, fans, and temperature sensors simultaneously. (See

Figure 5-8.)

The Summary tab has the following fields:

• Power

–

A green check indicates that at least one power source is connected and functioning properly.

–

A red check indicates that the power supply AC is disconnected.

• Fans

–

A green check indicates that at least one fan is present and functions properly.

–

A red check indicates a fan failure.

• Sensors

–

A green check indicates that the system temperature is at an acceptable level.

–

A red check indicates a high-temperature warning.

Using the Power Supplies Tab

Use the Power Supplies tab to view the operating status of the power supplies. (See Figure 5-9.)

Figure 5-9 Health Status Power Supplies Window

Using the Fans Tab

Use the Fans tab to view the operating status of the fans. (See Figure 5-10.)

Figure 5-10 Health Status Fans Window

5-6

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

Displaying System Information

In the OperStatus field, a status of up indicates that the fan is operating correctly within the power supply module.

In the Speed field, the integer represents a percentage. The percentage changes based on the ambient temperature of the unit and increases as the temperature rises.

Using the Sensors Tab

Use the Sensors tab to view the operating status of the temperature sensor of the system. (See Figure 5-11.)

Figure 5-11 Health Status Sensors Window

In the OperStatus field, a status of up indicates that the sensor functions properly.

The temperature fields show temperatures in Celsius. The Current Temp field shows the current internal temperature. The Alarm Temp field shows the temperature at which the system goes above its internal temperature limit. The Shutdown Temp field shows the temperature at which the system powercycles. For operating and non-operating temperatures, see Table A-1 on page A-1.

Displaying System Information

You can use CLI commands to monitor the power supplies, fans, and sensors.

Displaying Power Supply Information

To display information about the power supply, enter the show power-supply command.

SFS-7000P# show power-supply =============================================================== Power-supply Information =============================================================== ps type oper-status utilization voltage

---------------------------------------------------------------------------1 AC up 50 12 2 AC up 50 12 ===============================================================

Power-supply Seeprom =============================================================== product pca pca fru ps serial-number serial-number number number

--------------------------------------------------------------1 200000 820000 820000 1 2 200000 820000 820000 1

5-7

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

Displaying System Information

Displaying Fan Information

To display information about the fans, enter the show fan command.

An oper-status of up means that the fan is operating correctly within the power supply module.

In the Speed field, the integer represents a percentage. The percentage changes based on the ambient temperature of the unit and increases as the temperature rises.

SFS-7000P# show fan ================================================================================ Fan Information ================================================================================ fan oper-status speed(%)

-------------------------------------------------------------------------------1 up 73 2 up 73 3 up 73 4 up 73 ================================================================================ Fan Seeprom ================================================================================ product pca pca fru

fan serial-number serial-number number number

-------------------------------------------------------------------------------1

Displaying Sensor Information

To display information about the sensors, enter the show sensor command.

SFS-7000P# show sensor ================================================================================ Sensor Information ================================================================================ sensor oper-status oper-code temperature(c) alarm-temp(c) shutdown-temp(c)

-------------------------------------------------------------------------------1/1 up normal 29 65 73

The temperature fields show temperatures in Celsius. The Current Temp field shows the current internal temperature. The Alarm Temp field shows the temperature at which the system goes above its internal temperature limit. The Shutdown Temp field shows the temperature at which the system power cycles. For operating and non-operating temperatures, see Table A-1 on page A-1.

5-8

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 5 Managing the Switch

Displaying System Information

CHA PT ER

6-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Hardware Diagnostic Tests

This chapter describes how to run diagnostic tests on the hardware for the Cisco SFS 7000P and SFS 7000D Server Switches, and it contains the following sections:

• About Diagnostic Tests, page 6-1

• Displaying Hardware Errors, page 6-8

About Diagnostic Tests

Hardware diagnostic tests can be performed through the CLI or the Element Manager GUI. For instructions about how to run a test with the Element Manager GUI, see the Cisco SFS 7000 Series Product Family Element Manager User Guide. A POST is run automatically on various components at power-on.

The following sections refer to running and displaying tests with the CLI. For complete diagnostic command information, see the Cisco SFS 7000 Series Product Family Command Reference.

LED Tests

LED tests allow you to set a particular LED to blink in a specific way so that you can more easily identify a specific component in a rack. See Chapter 5, “Managing the Switch” for more information on LEDs.

Self-Tests

The Cisco SFS 7000P and SFS 7000D Server Switches provide the following diagnostic tests:

• Card Self-Test

• Chassis Self-Test

• Fan Self-Test

• Power Supply Self-Tests

6-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

About Diagnostic Tests

Running Card Self-Test

The term cards in this section refers to fabric controllers, management I/O modules, node cards, and the chassis ID module. Fans and power supplies are not included in the card tests; they can be tested through the standard chassis test or as individual components.

The following tests are available to locate and diagnose one or more cards in the chassis:

• LED

• Self-test

The example below shows how to run a card self-test.

Note The sample text in the steps below shows command output for the Cisco SFS 7000P Server Switch. For

the Cisco SFS 7000D Server Switch, you would see a similar display.

To perform a diagnostic self-test on a card, follow these steps:

Step 1 Enter the diag card number command.

SFS-7000P> enable SFS-7000P# config SFS-7000P(config)# diag card 11 SFS-7000P(config-diag-card-11)# test self-test SFS-7000P(config-diag-card-11)# start

Step 2 Exit the test to view the progress of the test.

The test takes approximately 5 to 8 minutes.

SFS-7000P(config-diag-card-11)# exit SFS-7000P(config)# exit

Note You can stop the diagnostic self-test at any time by entering the stop command.

Step 3 Enter the show diagnostic card # command.

SFS-7000P# show diagnostic card 1

====================================================================== Diagnostic Tests For Cards ====================================================================== test : self-test slot-id : 1 iterations : 1 action : start result : success percentage-completed : 100 result-string : Card Test Completed, Final report : Passed=1, Failed=0, Total=1

Step 4 If the test fails, enter the more syslog:hwif_log command to see the log information.

6-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

About Diagnostic Tests

Running Chassis Self-Test

The following tests are available to locate and diagnose the chassis:

• LED

• Standard

• Extended

• Extended Memory

The example below shows how to run a chassis standard test.

Note The sample text in the steps below shows command output for the Cisco SFS 7000P Server Switch. For

the Cisco SFS 7000D Server Switch, you would see a similar display.

The chassis standard test runs a diagnostic test on the following units in a chassis:

• The power supply and fan unit contained in the power supply module

• The base card

To perform a chassis standard test, follow these steps:

Step 1 Start the test by entering the diag chassis test standard command and the start command.

SFS-7000P> enable SFS-7000P# config SFS-7000P(config)# diag chassis test standard SFS-7000P(config-diag-chassis)# start SFS-7000P(config-diag-chassis)#

Step 2 Display the status of the test by entering the show diag chassis command.

SFS-7000P(config-diag-chassis)# exit SFS-7000P(config)# exit SFS-7000P# show diag chassis ====================================================================== Diagnostic Tests For Chassis ====================================================================== module-type : chassis module-number : 1 test : standard iterations : 1 option : none action : start result : success percentage-completed : 100

result-string : Standard System Test Completed, Final report : Passed=1, Failed=0, Total=1 SFS-7000P#

6-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

About Diagnostic Tests

Step 3 Display any errors on individual components by entering the show diag post command.

SFS-7000P-1# show diag post

====================================================================== Post Status ====================================================================== fru-slot post-status post-error

---------------------------------------------------------------------card(1) failed _FRU_LIM_CARDTYPE_ERR card(3) failed _FRU_SEEPROM_PROGRAM_ERR card(5) passed none card(6) passed none card(7) passed none card(8) passed none card(9) passed none card(11) failed _FRU_POWER_CONTROLLER_CONFIG_ERR card(12) passed none card(13) passed none card(15) passed none card(16) passed none card(17) passed none fan(1) passed none

Running Fan Self-Test

The procedures in this section describe the fan module tests and provide sample text from performing a fan test.

Note The sample text in the steps below shows command output for the Cisco SFS 7000P Server Switch. For

the Cisco SFS 7000D Server Switch, you would see a similar display.

To perform a diagnostic self-test on a fan, follow these steps:

Step 1 Identify the number of the fan you want to diagnose, if you do not already know the number. Power

supply 1 contains fan number 1, and power supply 2 contains fan number 2. (See Figure 6-1.)

6-5

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

About Diagnostic Tests

Figure 6-1 Power Supply Numbers on the Cisco SFS 7000P and SFS 7000D Server Switches

Enter the show diagnostic fan command.

SFS-7000P# show diagnostic fan

====================================================================== Diagnostic Tests For Fan ====================================================================== module-type : fan module-number : 2 test : self-test iterations : 1 action : start result : success percentage-completed : 100 result-string : Fan Self Test Completed, Final report : Passed=1, Failed=0, Total=1

Step 2 Enter the diag fan id command.

SFS-7000P# config SFS-7000P(config)# diag fan 1 SFS-7000P(config-diag-fan-1)# test self-test SFS-7000P(config-diag-fan-1)# start SFS-7000P(config-diag-fan-1)# exit SFS-7000P(config)# exit SFS-7000P# show diag fan ====================================================================== Diagnostic Tests For Fan ====================================================================== module-type : fan module-number : 1 test : self-test iterations : 1 action : start result : in-progress percentage-completed : 70 result-string : Self Test, Current report : In progress... SFS-7000P#

Step 3 Display the progress of the test by entering the show diag fan command.

SFS-7000P(config-diag-fan-1)# exit SFS-7000P(config)# exit

181046

Power Supply

6-6

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

Running Power Supply Tests

SFS-7000P# show diag fan ================================================================================ Diagnostic Tests For Fan ================================================================================ module-type : fan module-number : 1 test : self-test iterations : 1 action : start result : success percentage-completed : 100 result-string : Fan Self Test Completed, Final report : Passed=1, Failed=0, Total=1 SFS-7000P#

Step 4 Display any errors that appear in the result string by entering the show diag fru-error command.

SFS-7000P# show diag fru-error ============================================================ Fru-Error ============================================================ fru-slot fru-error

-----------------------------------------------------------card(1) none card(2) none card(7) none card(8) none card(9) none card(10) none card(11) none card(12) none card(13) none card(14) none card(15) none card(17) none fan(1) none fan(2) none

Step 5 Reboot the chassis when the self-test is complete.

SFS-7000P# reload System configuration is modified. Save? [yes(default) | no | filename] y Proceed with reload? [yes(default) | no] y

Running Power Supply Tests

The following tests are available to locate and diagnose fan modules:

• LED Test

• Self-Test

Note The sample text in this section shows command output for the Cisco SFS 7000P Server Switch. For the

Cisco SFS 7000D Server Switch, you would see a similar display.

6-7

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

Running Power Supply Tests

Running a LED Test on the Power Supply

You can use the LED test to identify a specific power supply by entering these commands:

SFS-7000P> enable SFS-7000P# config SFS-7000P(config)# diag power-supply 1 SFS-7000P(config-diag-power-supply-1)# test led SFS-7000P(config-diag-power-supply-1)# start SFS-7000P(config-diag-power-supply-1)# stop SFS-7000P(config-diag-power-supply-1)#

Running a Self-Test on a Power Supply

To perform a diagnostic self-test on a power supply, follow these steps:

Step 1 Determine the ID of the power supply that you want to diagnose, if you do not already know the number.

Enter the show power-supply command.

SFS-7000P# show power-supply ===================================================================== Power-supply Information ===================================================================== ps type admin-status oper-status utilization voltage

--------------------------------------------------------------------1 AC up up n/a 48 2 AC up up 15 47

Step 2 Specify the type of test (self-test is chosen by default) by entering the diag power-supply id command.

SFS-7000P> enable SFS-7000P# config SFS-7000P(config)# diag power-supply 2 SFS-7000P(config-diag-power-supply-2)# test self-test

Step 3 Start the test.

SFS-7000P(config-diag-power-supply-2)# start

Step 4 Display the progress of the test by entering the show diag fan command.

SFS-7000P(config-diag-power-supply-2)# exit SFS-7000P(config)# exit SFS-7000P# show diagnostic power-supply

====================================================================== Diagnostic Tests For Power Supply ====================================================================== module-type : power-supply module-number : 2 test : self-test iterations : 1 action : start result : success percentage-completed : 100 result-string : Power Supply Self Test Completed, Final report : Passed=1, Failed=0, Total=1

Step 5 Display any errors that appear in the result string by entering the show diag fru-error command.

SFS-7000P# show diag fru-error

6-8

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

Displaying Hardware Errors

================================================================ Fru-Error ================================================================ fru-slot fru-error

---------------------------------------------------------------card(9) none card(10) none card(11) none card(12) none card(13) none card(14) none card(15) none card(17) none fan(1) none fan(2) none fan(3) none fan(4) none power-supply(1) none power-supply(2) none SFS-7000P#

Step 6 Reboot the chassis when the self-test is complete.

SFS-7000P# reload System configuration is modified. Save? [yes(default) | no | filename] y Proceed with reload? [yes(default) | no] y

Displaying Hardware Errors

Note The sample text in the steps below shows command output for the Cisco SFS 7000P Server Switch. For

the Cisco SFS 7000D Server Switch, you would see a similar display.

To display POST results, non-fatal errors that can are recovered, informational logging (such as firmware updates), and hardware errors follow these steps:

Step 1 Enter the more syslog:hwif_log command to see the contents of the hardware log file.

SFS-7000P> enable SFS-7000P# more syslog:hwif_log Thu Mar 4 10:57:49 2004: POST: SEEPROM: PASSED Thu Mar 4 10:57:49 2004: POST: FPGA: PASSED Thu Mar 4 10:57:49 2004: POST: SUMMIT: PASSED Thu Mar 4 10:57:50 2004: POST: RTC: PASSED Thu Mar 4 10:57:54 2004: POST: FAN: PASSED Thu Mar 4 10:57:54 2004: card_startup.x : card is starting up Thu Mar 4 10:57:54 2004: Anafa2Init: a2update set to IGNORE Thu Mar 4 10:58:16 2004: Anafa2 POST: firmware check PASSED Thu Mar 4 19:01:55 2004: POST: SEEPROM: PASSED Thu Mar 4 19:01:55 2004: POST: FPGA: PASSED Thu Mar 4 19:01:55 2004: POST: SUMMIT: PASSED Thu Mar 4 19:01:56 2004: POST: RTC: PASSED Thu Mar 4 19:02:00 2004: POST: FAN: PASSED Thu Mar 4 19:02:00 2004: card_startup.x : card is starting up Thu Mar 4 19:02:00 2004: Anafa2Init: a2update set to IGNORE <output truncated>

6-9

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

Displaying Hardware Errors

You can also view POST error codes with the show diagnostic post command. This command displays the results of each POST that executes at power-up.

SFS-7000P# show diagnostic post

====================================================================== Post Status ====================================================================== fru-slot post-status error-codes

-------------------------------------------------------------------------------card(1) failed 1:0:0:0 fan(1) passed 0:0:0:0 fan(2) passed 0:0:0:0 power-supply(1) passed 0:0:0:0

Step 2 Display FRU errors on your switch by entering the show diagnostic fru-error command.

SFS-7000P# show diagnostic fru-error

====================================================================== Fru-Error ====================================================================== fru-slot fru-error

-------------------------------------------------------------------------------card(1) _FRU_POWER_CONTROLLER_CONFIG_ERR fan(1) none fan(2) none power-supply(1) none

Table 6 -1 lists the hardware errors that are logged for the power supply.

Table 6-1 Cisco SFS 7000P and SFS 7000D Server Switches Power Supply Errors

Error Number Description

1 Power supply DC fail error

2 Power supply fan fail error

3 Power supply I2C communication error

6-10

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Chapter 6 Hardware Diagnostic Tests

Displaying Hardware Errors

A-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

APPENDIX

Specifications and Compliance Certifications

This appendix describes the Cisco SFS 7000P and SFS 7000D Server Switches specifications and compliance certifications.

Chassis and Management Interface

Table A -1 lists the general specifications for the Cisco SFS 7000P and SFS 7000D Server Switches.

Table A-1 Cisco SFS 7000P and SFS 7000D Server Switches and Management General Specifications

Item Specification

Operating temperature 0 to 45°C (32°F to 113°F)—Cisco SFS 7000P Server Switch

0 to 40°C (32°F to 104°F)—Cisco SFS 7000D Server Switch

Nonoperating temperature -40 to 70°C (-40 to 158°F)

Operating temperature gradient 20°C (68°F) max. per 60 min

Operating altitude 0 to 3,048 m (0 to 10,000 ft)

Nonoperating altitude 0 to 12,192 m (0 to 40,000 ft)

Operating humidity: 8 to 80% non-condensing

Nonoperating humidity: 5 to 90% RH at 65°C (149°F) for 24 hours, non-condensing

Operating humidity gradient 10% maximum per 60 min

Operating shock 5 G max., 11 ms half-sine wave, 10 G max. 5 ms half-sine wave

Nonoperating shock 10 G max., 11 ms half-sine wave

Operating vibration, sinusoidal 0.25 G max., 3-200 Hz 15 min

Nonoperating vibration, sinusoidal 0.50 G max., 3-200 Hz 15 min

Nonoperating vibration, random 2.09 Grms, 3-axis, bottom/top, left/right, front/back

Max. operating inclination 15 degree

Physical Characteristics

Dimensions 1 RU, fits in standard 19-in. (48.26 cm) rack

Length 21 in. (53.34 cm)

Weight Maximum weight is 22 lb (10 kg)

A-2

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Appendix A Specifications and Compliance Certifications

Electrical Specifications

Table A -2 lists the electrical specifications for the Cisco SFS 7000P and SFS 7000D Server Switches.

Table A-2 Cisco SFS 7000P and SFS 7000D Server Switches Electrical Specifications

Category Specification

AC input Auto-ranging 90 to 264 VAC, 47 to 63 Hz

Power consumption Power with external powered

devices is 125 Watts

Power without external powered devices is 65 Watts—Cisco SFS 7000P Server Switch

Power without external powered devices is 80 Watts—Cisco SFS 7000D Server Switch

1. External powered device current is measured at 3.3 volts 0.4 amperes per port.

REVIEW DRAFT—CISCO CONFIDENTIAL

B-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

APPENDIX

Acronyms and Abbreviations

Table B -1 defines the acronyms and abbreviations that are used in this publication.

Table B-1 List of Acronyms and Abbreviations

Acronym Expansion

API Application Program Interface

CCelsius

CLI command-line interface

DDR double data rate

EMI electromagnetic interference

ESD electrostatic discharge

FCC Federal Communications Commission

FRU field-replaceable unit

GUI graphical user interface

GUID globally unique identifier

HCA Host Channel Adapter

kHz kilohertz

OEM original equipment manufacturer

POST power-on self test

RFI radio frequency interference

SDR single data rate

SNMP Simple Network Management Protocol

SSH secure shell

TV television

VPD vital product data

REVIEW DRAFT—CISCO CONFIDENTIAL

B-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

Appendix B Acronyms and Abbreviations

INDEX

adapter plug 3-10 administrative features 1-5 air flow 2-5 altitude 2-2 audience, document vii

bad wiring practice 2-3 baud rate 3-9 bi-color LED 5-1 blackout 2-4

configure connectivity 3-7

connect

InfiniBand devices management devices 3-14

network devices 3-11 connectivity, configuring 3-7 connector, types 3-11 console port 1-2 conventions, document viii corrosion 2-3

3-12

data bits 3-9 DDR 1-2, 1-3

Card Self-Test 6-2

chassis

interface power up 3-10

Chassis Self-Test 6-3

chassis status LED

bi-color single 5-1

OL-10853-01

A-1

5-1

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

degradation, performance 1-2 dehumidifier 2-2 diagnostics 1-3, 1-5 diagnostic tests 6-1

display

fan information

power supply information 5-6

sensor information 5-7

system information 5-6 display, hardware errors 6-8

5-7

IN-5

Index

document

audience

vii

conventions viii obtaining xiv organization vii

related xiv dual-connected, HCAs 1-4 dust 2-3

electrical storms 2-4 electrolytic capacitors 2-2 electrostatic discharge, damage 2-4 Element Manager 5-4

fabric switches

high-performance

1-1

low-latency 1-1 fan information, display 5-7 Fan Self-Test 6-4 Fans tab 5-5 FCC regulations 2-3 filler panel 4-4 flange 3-4, 3-7 FRU, power supply module 4-1

Element Manager GUI 6-1 EMI 2-3 empty, module bay 4-2 emulation software 3-9 environmental factors 2-1 ESD 2-4

grounding strap 2-4 preventative, ankle strap 3-2

preventative,wrist strap 3-2 Ethernet cable 3-14 exhaust fans 2-3

general specifications A-1 grounding 3-10 ground prong 3-10

hardware diagnostic tests 6-1 hardware errors, display 6-8 HCA 1-4 Health Status window 5-4

high availability

fabric features 1-4 hardware 1-4

1-4

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

IN-6

OL-10853-01

Index

ports 1-4 hot-swappable, power supply module 1-3, 4-1 humidity 2-2, 2-3 HyperTerminal 3-9

IEC connectors 1-3

InfiniBand

cables

connectivity 1-3

devices, connect 3-12

port 1-2

port LED 5-3

install

3-12

list

1-3

test 6-1

test, power supply 6-7 lightning 2-3 logical link 5-3 logical link activity 5-3 login, default 3-11 low pressure 2-2

management

interface

port 3-14

specifications A-1

A-1

power supply module

4-2

preparation 3-2 with one person 3-4 with two people 3-6

interface

chassis

A-1

management A-1

interswitch connectivity 1-2

latency 1-5 latency, port to port 1-5

LED

management devices, connect 3-14 management-Ethernet port 1-2 metal connector shells 2-3

module bay

empty

4-2

unused 4-2 moisture 2-2

network devices, connect 3-11 network traffic 1-3 nonblocking architecture 1-5 nonvolatile memory 1-5

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

IN-7

Index

organization, document vii overvoltage 2-4

parity 3-9 particles 2-3 performance, degradation 1-2 pinch connector 3-13

port

console Ethernet management 1-2 InfiniBand 1-2

1-2

information display 5-6

install 4-2

LED 5-3

LED test 6-7

module 1-3

module removal 4-4

redundant 1-3

Self-Test 6-7

test 6-6

unit 4-1

without 4-4 power supply module 1-3 power up, chassis 3-10 pull connector 3-13, 3-14

LED, InfiniBand 5-3

management 3-14 POST 1-5, 3-11, 6-1 power cord 3-10 power requirements 2-5 power source, interruptions 2-4 Power Supplies tab 5-5

power supply

1-3

bay

errors 6-9

failed 4-1

features 1-3

FRU 4-1

hot-swappable 1-3, 4-1

rack-mount, switch 3-3 radio transmitters 2-3 real-time clock 1-5 redundant, power supply 1-3 remove power supply module 4-4 RFI 2-3

safety warnings 2-1, 3-2 scalability 1-4 SDR 1-2, 1-3

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

IN-8

OL-10853-01

Index

Self-Test, power supply 6-7 Self tests 6-1 sensor information, display 5-7 Sensors tab 5-6 serial console, attach 3-8 shock 2-4 site preparation, checklist 2-6 site requirements 2-1

specifications

electrical

A-2

general A-1

management A-1 spikes 2-4

status

fan

1-3

overall system 1-3

conditions

2-2

extreme 2-2 fluctuations 2-2

transients 2-4

undervoltage 2-4 unused module bay 4-2

ventilation 2-2, 2-5 vibration 2-4 voltage, variations 2-4

power 1-3

show link 1-3 stop bits 3-9 subnet manager 1-4 Summary tab 5-5

switch

components

1-2

LED 1-3 system information, display 5-6

temperature

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

OL-10853-01

warning

definition safety 2-1 statement ix, 3-2 statement,translations ix

IN-9

Index

Cisco SFS 7000P and SFS 7000D InfiniBand Server Switches Hardware Installation Guide

IN-10

OL-10853-01

Cisco SFS InfiniBand Server Switch SFS 7000P, SFS 7000D, SFS 7000P Hardware Installation Manual

Specifications and Main Features

Frequently Asked Questions

User Manual