Cisco Systems SFS 3012 User Manual

0 (0)

Cisco SFS 3012 Multifabric Server Switch

Hardware Guide (Topspin 360 Hardware Guide)

Web Release v2.1.0

Part Number: OL-7722-01

Product ID: 140262

The Topspin Server Switch, Topspin Host Channel Adapter, Topspin Element Manager, and collateral software programs and documentation are subject to and made available only pursuant to the license agreement signed by you and Topspin, Communications, Inc. (or if no signed license agreement exists, the license agreement included with your original media) and may only be used in accordance with the terms of that agreement. Making copies, modifications, or compilation works of the software except as specifically allowed in that agreement is prohibited by law and constitutes a punishable violation of the law. This documentation may not be copied, modified or reduced to any electronic or machine-readable form without Topspin Communication, Inc.'s prior written consent.

As defined in FAR section 2.101, DFAR section 252.227-7014(a)(1) and DFAR section 252.227-7014(a)(5) or otherwise, the collateral software programs provided to you are “commercial items,” “commercial computer software” and/or “commercial computer software documentation.” Consistent with DFAR section 227.7202, FAR section 12.212 and other sections, any use, modification, reproduction, release, performance, display, disclosure or distribution thereof by or for the U.S. Government shall be governed solely by the terms of the license agreement and shall be prohibited except to the extent expressly permitted by the terms of that agreement. Any technical data provided that is not covered by the above provisions shall be deemed “technical data-commercial items” pursuant to DFAR section 227.7015(a) or equivalent. Any use, modification, reproduction, release, performance, display or disclosure of such technical data shall be governed by the terms of DFAR section 227.7015(b) or equivalent.

This documentation may include technical inaccuracies or typographical errors and is subject to correction and other revision without notice. TOPSPIN PROVIDES THIS DOCUMENTATION “AS IS” AND WITHOUT ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states or jurisdictions do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you.

©Copyright 2004, Topspin Communications, Inc. All rights reserved. Topspin is a registered trademark and the Topspin logo, TopspinOS, and Topspin Server Switch are trademarks of Topspin Communications, Inc. Other company, product, or service names are the property of their respective owners.

Document Version: Web Release

Part Number: OL-7722-01

Product ID: 140262

April, 2005

Printed in the United States of America.

iii

Contents

Regulatory Notices........................................	vii
FCC Statement..........................................................................................................................................	vii
Safety Information ....................................................................................................................................	vii
Electrical Cautions.......................................................................................................................	vii
General Cautions...........................................................................................................................	ix
Laser Cautions ..............................................................................................................................	ix
Contact Information.....................................................................................................................................	x
1: Product Registration...................................	xi
THANK YOU FOR PURCHASING YOUR TOPSPIN® PRODUCT!...................................................	xi
2: Standard Product Agreement...................	xiii
3: Topspin 360 Overview ...............................	1
Introduction..................................................................................................................................................	1
Features........................................................................................................................................................	2
Performance ....................................................................................................................................	2
Unified Fabric .................................................................................................................................	2
Transparent Topology.....................................................................................................................	2
Redundancy and High-Availability ................................................................................................	3
Scalability .......................................................................................................................................	3
4: Topspin 360 Components..........................	5

iv
Introduction..................................................................................................................................................	5
Controller Cards...........................................................................................................................................	6
Install a Controller Card .................................................................................................................	6
Ethernet and Fibre Channel Gateways.........................................................................................................	9
Replacing and Updating Your Gateways........................................................................................	9
Install an Ethernet or Fibre Channel Gateway..............................................................................	10
InfiniBand Switch Cards............................................................................................................................	10
Install InfiniBand Switch Cards....................................................................................................	11
Remove InfiniBand Switch Cards ................................................................................................	13
Power Supply Modules..............................................................................................................................	13
Install Power Supply Modules......................................................................................................	14
Removing a Power Supply ...........................................................................................................	15
Blower Trays..............................................................................................................................................	15
Install Blower Trays......................................................................................................................	15
Removing Blower Trays...............................................................................................................	16
5: Install and Boot the Topspin 360..............	19
Introduction................................................................................................................................................	19
Prepare the Site for the Topspin 360..........................................................................................................	19
Read the Cautionary Statements ...................................................................................................	20
Prepare the Physical Environment for the Topspin 360 ...............................................................	20
Install Topspin 360 Components ...............................................................................................................	20
Rack the Topspin 360 ................................................................................................................................	20
Attach a Serial Console Cable to a PC or Terminal...................................................................................	26
Boot your Topspin 360 and Configure Basic Connectivity.......................................................................	27
Assign a Static IP Address............................................................................................................	27
Obtain an IP Address with DHCP ................................................................................................	28
Connect InfiniBand Hosts..........................................................................................................................	29
Manage the Topspin 360............................................................................................................................	32
Viewing and Updating the Software Image on the Chassis.......................................................................	33
View the Software Image Version with the CLI ..........................................................................	33
View the Software Image Version with the GUI..........................................................................	34
Upgrade the Software Image with the CLI...................................................................................	35
Upgrade the Software Image with the GUI ..................................................................................	36
Delete an Image with the CLI.......................................................................................................	38
Delete an Image with the GUI ......................................................................................................	39
6: Topspin 360 Field Replaceable Units ......	41
Controller Cards.........................................................................................................................................	41
Expansion Modules....................................................................................................................................	41
InfiniBand Switch Cards............................................................................................................................	42
Installing an InfiniBand Switch Card ...........................................................................................	42
Configuring a New InfiniBand Switch Card ................................................................................	42
Removing an InfiniBand Switch Card..........................................................................................	44
Power Modules ..........................................................................................................................................	44

Installing a Power Supply .............................................................................................................	44
Removing a Power Supply ...........................................................................................................	45
Replacing Blower Trays ............................................................................................................................	46
Installing a Blower Tray ...............................................................................................................	46
Removing a Blower Tray..............................................................................................................	47
7: Topspin 360 LEDs....................................	49
Introduction................................................................................................................................................	49
Topspin 360 System LED..........................................................................................................................	50
Power Supply LEDs...................................................................................................................................	51
Blower Tray LEDs.....................................................................................................................................	52
Controller Card LEDs................................................................................................................................	53
IB Switch Card LEDs ................................................................................................................................	54
IB Port LEDs ................................................................................................................................	54
Fibre Channel Gateway LEDs ...................................................................................................................	55
FC Port LEDs................................................................................................................................	55
Ethernet Gateway LEDs ............................................................................................................................	56
Ethernet Port LEDs.......................................................................................................................	56
8: Specifications and Compliance................	57
General Specifications ...............................................................................................................................	57
Electrical Specifications.............................................................................................................................	58
Certifications/Regulatory...........................................................................................................................	58
Laser Specs ................................................................................................................................................	58

vii

Regulatory Notices

FCC Statement

This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense.

Safety Information

•“Electrical Cautions” on page vii

•“General Cautions” on page ix

•“Laser Cautions” on page ix

Electrical Cautions

CAUTION: Use only the power cable provided with your Topspin 360. Inspect the power cord and determine if it provides the proper plug and is appropriately certified for use with your electrical system. Discard the cord if it is inappropriate for your country’s electrical system and obtain the proper cord, as required by your national electrical codes or ordinances.

viii

CAUTION: Grounding is supplied by the ground-prong on the 3-prong power cable. Do not attach a separate ground cable. Do not use adapter plugs. Do not remove the ground prong from the cable. Ensure the ground connection on the power supply is correct and functioning before applying power to the Topspin 360.

CAUTION: Always ground yourself before touching any internal Topspin 360 component to avoid damage from electrostatic discharge (ESD).

CAUTION: The Topspin 360 has two power cables. The operator must disconnect all power cables before attempting to remove the Topspin 360 from the rack.

CAUTION: Remove power cables by grasping the cable connector and pulling straight out.If the chassis comes equipped with a power-plug retainer, move the bail wire used to retain the plug to the side before attempting to remove the power plug. To replace power-supply cables, grasp the cable connector, align the connector with the power jack, and insert the connector straight into the jack. If the chassis comes equipped with a power-plug retainer, move the bail wire back into place to keep the power plug firmly in place.

CAUTION: The Topspin 360 contains lithium batteries. Do not attempt to replace or discard these batteries. The batteries may only be serviced by Topspin service personnel.

CAUTION: Observe and follow service markings. Do not service the Topspin 360, except as explained in the Topspin documentation supplied with your Topspin 360. Opening the chassis or removing the enclosure cover exposes you to electrical shock and may damage Topspin 360 components.

If one or more of the following situations occurs, remove power to the Topspin 360 and contact your

Topspin support representative:

•The power cable or power plug is damaged.

•The Topspin 360 has been exposed to water.

•The Topspin 360 has been dropped or damaged in any way.

•The Topspin 360 does not operate correctly after following the installation instructions.

CAUTION: You must ensure the operating environment has adequate air circulation. Keep the Topspin 360 in a cool, well-ventilated room. Do not block cooling vents. Install filler panels into all unused slots.

CAUTION: Use only Topspin-approved gateway expansion modules, controller cards, fans, and power supply modules.

CAUTION: Never place your hand inside an empty card or module bay. You should never have cause to place a hand anywhere inside the Topspin chassis. Unused card and module bays should always have a Topspin cover over the bay to ensure proper safety, ventilation, and cooling.

CAUTION: The Topspin 360 contains hot-swappable controller, fan/blower tray, switch controller, power supply, and gateway expansion modules. The operator may swap these components while power is still applied to the Topspin 360.

General Cautions

CAUTION: No user is authorized to remove the Topspin 360 enclosure cover. The internal chassis contains no user-serviceable components. Removing the enclosure cover voids your warranty. See the warranty card for further details regarding the servicing of your chassis.

CAUTION: Opening and/or modifying expansion and system modules may void your warranty. Refer to your warranty to determine access restrictions.

In general

•Do not spill food or liquids on your Topspin 360 components.

•Protect your Topspin 360 from sudden power-surges and interruptions by using a surge suppressor, line conditioner, or uninterruptable power supply (UPS).

•Place cables appropriately so that they do not obstruct any egress within the data center and they do not block any ventilation inlets or outlets.

•Rack mount the Topspin chassis according to the rack manufacturer recommendations.

CAUTION: Installing the Topspin 360 into a rack is a 3-person process due to the weight and size of the 4U unit.

Laser Cautions

CAUTION: Only qualified service technicians should repair laser devices.

CAUTION: Only open, modify, or service laser devices as specified in the manufacturer documentation.

CAUTION: Use of controls or adjustments or performance of procedures other than those specified herein may result in hazardous laser radiation exposure.

Laser Specs

Table iii-1:

Topspin P/N	Application	Optical Parameters
48-00001-01	Fibre Channel	*CDRH/IEC Laser Class: 1/1
		Maximum Data Rate: 2.125 GB/s
		Wavelength: 850nm
		I/O connector:

* Standards used: CDRH - 21CFR Chapter I §1040.10; IEC - IEC60825-1:1994

Contact Information

Table iii-2: Customer Contact Information

Address	Topspin Communications, Inc.
	515 Ellis St.
	Mountain View, CA 94043

Telephone, Corporate Headquarters	(866)	TOPSPIN or (650) 316-3300

Fax	(650)	316-3269

Telephone, Technical Support	(800)	499-1473

Email, Technical Support	support@topspin.com

Web site, Support	http://support.topspin.com

Product Registration

THANK YOU FOR PURCHASING YOUR TOPSPIN® PRODUCT!

Topspin Communications is pleased that you have selected Topspin for your computing infrastructure. Topspin provides comprehensive customer support resources via telephone and Web access.

Topspin’s Customer Support Web site has the latest TopspinOS (IB Switch and I/O Chassis Operating System), Linux and Windows Host Side Drivers, Boot over InfiniBand Firmware and technical documentation. Topspin recommends that you download the latest Topspin software and documentation from the Topspin support web site so that you can take full advantage of the latest capabilities.

Registration Benefits

Customers must register on-line to receive the following benefits:

•Software updates and downloads

•Product warranty support

•Customer service and support

•Product documentation

Registration Instructions

Please immediately follow the instructions below to register yourself as a Topspin user:

1.Go to http://support.topspin.com

2.Click the “Register Now” link below the customer login prompt

3.Enter at least one of your product serial numbers in the spaces provided. Each serial number must be entered exactly as it appears on the product label, including dashes.

4.Enter your contact information in the spaces provided.

User Name and Password

Once you register, Topspin will provide you with a user name and password to access the secure customer support Web site. You may also contact Topspin customer support at:

•Email: support@topspin.com

xii

•Phone: (800) 499-1473

•Online Help Desk: http://support.topspin.com

xiii

Standard Product Agreement

PLEASE READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THE PRODUCTS THAT ARE PROVIDED WITH THIS AGREEMENT. BY USING ANY PRODUCTS, YOU ACKNOWLEDGE THAT YOU HAVE READ AND UNDERSTOOD ALL THESE TERMS AND CONDITIONS AND YOU WILL BE CONSENTING TO BE BOUND BY THEM. IF YOU DO NOT ACCEPT THESE TERMS AND CONDITIONS, DO NOT USE THE PRODUCTS AND RETURN PRODUCTS WITHIN 30 DAYS UNUSED IN THE ORIGINAL SHIPPING CONTAINER TO THE PLACE OF PURCHASE FOR A FULL REFUND.

1.General. These terms and conditions (“Agreement”) shall apply to all quotations made and products shipped by Topspin Communications, Inc. (“Products”) to Customer. Customer's acceptance of such quotations or Product shipments is expressly limited to the terms and conditions of this Agreement. Any additional or different terms or conditions in any communication by Customer (whether in a purchase order or otherwise) are hereby rejected and shall be null and void, irrespective of the means of Customer's acceptance. Any failure by Topspin Communications, Inc. (“Topspin”) to object to any additional or different provisions proposed by Customer shall not constitute a waiver of these terms and conditions, nor constitute acceptance of any such Customer's terms and conditions. All orders or contracts must be approved and accepted by Topspin at its home office at 515 Ellis Street, Mountain View, CA 94043. This Agreement shall be applicable whether or not they are attached to or enclosed with the Products. Topspin may deliver Products in partial shipments and such partial shipments may be invoiced individually. Customer may cancel submitted purchase orders only with the prior written consent of Topspin.

2.Prices/Taxes. All prices are exclusive of shipping, insurance and installation charges, all of which are Customer's sole responsibility. All prices are exclusive of all sales, use, excise, value added, withholding and other taxes, and all customs duties and tariffs now or hereafter claimed or imposed by any governmental authority upon the sale of Products. Any such charges will be added to the price or subsequently invoiced to the Customer. In the event Topspin is required to pay any such tax, duty or charge, Customer will promptly reimburse Topspin.

xiv

3.Payment/Financing. Topspin will determine Customer's credit terms on a per-order basis and such terms are subject to a credit review and approval by Topspin in its sole discretion. The amount of credit may be changed or credit withdrawn by Topspin at anytime in its sole discretion. All payments shall be made in U.S. Dollars net thirty (30) days from the date of invoice unless otherwise stated by Topspin in writing. Any amounts not paid when due will accrue interest at the rate of 1 1/2% per month, or the maximum amount allowed by law, if lower. In the event that any payment is more than thirty (30) days late, Topspin shall have the right to suspend performance until all payments are made current. Customer shall pay for all costs (including reasonable attorneys' fees) incurred by Topspin in connection with the collection of late payments. Each accepted purchase order is a separate, independent transaction, and Customer has no right of set-off against other purchase orders or other transactions with Topspin. Customer hereby grants Topspin a security interest in the Products in the amount of the unpaid balance of the purchase price until paid in full. Topspin may file a financing statement for such security interest and Customer shall sign any such statements or other documentation necessary to perfect Topspin's security interest.

4.Software License. (a) “Software” means Topspin's computer programs Products, in object code format, including without limitation firmware, delivered hereunder, whether incorporated in the hardware Products or delivered separately, and whether or not there is a separate charge therefor. The term “Software” shall also include any other new features, functionality or enhancements to the Software provided to Customer. All references to the “purchase,” “sale” of or “selling” Software shall mean the granting of a license to use such Software. (b) Subject to the terms and conditions of this Agreement, Topspin hereby grants Customer a non-exclusive, non-transferable, non-sublicensable license to use the Software (in object code form only) solely in accordance with the accompanying documentation for the internal business purposes of Customer. (c) Except as expressly provided in this Agreement, The license set forth above does not include any rights to and Customer shall not (i) reproduce, modify, translate or create any derivative work of all or any portion of the Products, (ii) sell, rent, lease, loan, provide, distribute or otherwise transfer all or any portion of the Software, (iii) reverse engineer, reverse assemble or otherwise attempt to gain access to the source code of all or any portion of the Products, (iv) use the Software for third-party training, commercial time-sharing or service bureau use, (v) remove, alter, cover or obfuscate any copyright notices, trademark notices or other proprietary rights notices placed or embedded on or in the Products, (vi) unbundle any component of the Products, or (vii) cause or permit any third party to do any of the foregoing. (d) If Customer is a European Union resident, Customer acknowledges that information necessary to achieve interoperability of the Software with other programs is available upon request. (e) Customer may make a single copy of standalone Software solely for its back-up purposes; provided that any such copy is the exclusive property of Topspin and its suppliers and includes all copyright and other intellectual property right notices that appear on the original. (f) Customer shall have the right to transfer the Software that is embedded in hardware Products in connection with a transfer of all of Customer's right, title and interest in such hardware Products to a third party; provided, that, Customer transfers the such Software and any copies thereof subject to, and such third party agrees in writing to be bound by, all the terms and conditions of this Agreement. (g) Customer acknowledges that the Software contains or is provided with copyrighted software of Topspin's suppliers as identified in associated documentation or other printed or electronic materials (“Third Party Software”) which are obtained under a license from such suppliers. Customer's use of any Third Party Software shall be subject to and Customer shall comply with the applicable restrictions and other terms and conditions set forth in the such documentation or materials.

5.Title/Risk of Loss/Acceptance. All sales are made F.O.B. point of shipment at Topspin's designated manufacturing facility, and Topspin's title to the Products (except for Software) and the risk of loss of or damage to the Products ordered by the Customer will pass to Customer at time of Topspin's delivery of Products to the carrier. The carrier shall be deemed Customer's agent, and any claims for damages in shipment must be filed with the carrier. Topspin is authorized to designate a carrier pursuant to Topspin's standard shipping practices unless otherwise specified in writing by Customer. All Products shall automatically be deemed accepted upon receipt by Customer. Customer must provide written notice to Topspin within thirty (30) days of delivery in the event that Products do not conform to the quotation made by Topspin.

6.Intellectual Property Rights. Title to and ownership of the Software and any associated documentation, and any improved, updated, modified or additional parts thereof, and all copyright, patent, trade secret, trademark and other intellectual property rights embodied in the Products, shall at all times remain the property of Topspin or Topspin's licensors. All suggestions or feedback provided by Customer or its employees or agents to Topspin with respect to the Products shall be Topspin's property and deemed confidential information of Topspin.

7.Force Majeure. Shipping dates are approximate and may be delayed absent prompt receipt from Customer of all necessary information. Topspin shall not be responsible for any failure to perform or delay attributable in whole or in part to any cause beyond its reasonable control, including but not limited to Acts of God, government actions, war, civil disturbance, insurrection, sabotage, labor shortages or disputes, failure or delay in delivery by Topspin's suppliers or subcontractors, transportation difficulties, shortage of energy, raw materials or equipment, or Customer's fault or negligence. In the event of any such delay the date of delivery shall, at the request of Topspin, be deferred for a period equal to the time lost by reason of the delay.

8.Indemnity. (a) Topspin shall defend Customer against any third party action alleging that the Products infringes any valid U.S. patent or copyright, and Topspin shall pay all settlements entered into, and all final judgments and costs (including reasonable attorneys' fees) awarded against Customer in connection with such action. If any Product, or parts thereof, becomes, or in Topspin's opinion may become, the subject of an infringement claim, Topspin may, at its option, (i) procure for Customer the right to continue using such Products, (ii) modify or replace such Products with substantially equivalent non-infringing products, or (iii) require the return of such Products and refund to Customer a pro-rata portion of the purchase price of such Products based on a three-year straight line amortization of the purchase price. (b) Topspin shall have no indemnification obligations with respect to any third party action alleging that (i) the use of any Products, or any part thereof, in combination with products not supplied by Topspin, or (ii) any service or other process utilizing any Products, or any part thereof, infringes any third party intellectual property right, and in such event Customer shall defend Topspin, and its officers, directors and employees, against any such action, and Customer shall pay all settlements entered into, and all final judgments and costs (including reasonable attorneys' fees) awarded against such party in connection with such action. (c) Each party's indemnification obligations shall be subject to the indemnified party (i) notifying the indemnifying party promptly in writing of such action, (ii) giving the indemnifying party exclusive control and authority over the defense or settlement of such action, (iii) not entering into any settlement or compromise of any such action without the indemnifying party's prior written consent and (iv) providing all reasonable assistance to the indemnifying party (provided that the indemnifying party reimburses the indemnified party for its out-of-pocket expenses incurred in providing such assistance). (d) THIS SECTION 8 STATES THE ENTIRE LIABILITY OF TOPSPIN FOR ANY INFRINGEMENT OF INTELLECTUAL PROPERTY RIGHTS.

9.Limited Warranty and Disclaimer. Any limited warranty for the Products and Topspin's sole and exclusivity liability thereunder is as set forth in Topspin's standard warranty documentation. Notwithstanding anything herein or otherwise to the contrary, Topspin makes no warranty with respect to any third party hardware or software products provided under this Agreement. Customer's sole remedy with respect to such third party products shall be pursuant to the original licensor's warranty, if any, to Customer, to the extent permitted by the original licensor. EXCEPT FOR ANY EXPRESS LIMITED WARRANTIES FROM TOPSPIN IN SUCH DOCUMENTATION, THE PRODUCTS ARE PROVIDED “AS IS”, AND TOPSPIN AND ITS SUPPLIERS MAKE NO WARRANTY, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, WITH RESPECT TO PRODUCTS OR ANY PART THEREOF, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THOSE ARISING FROM COURSE OF PERFORMANCE, DEALING, USAGE OR TRADE. NEITHER TOPSPIN NOR ANY OF ITS SUPPLIERS WARRANT THAT THE PRODUCTS OR ANY PART THEREOF WILL MEET CUSTOMER'S REQUIREMENTS OR BE UNINTERRUPTED, OR ERROR-FREE, OR THAT ANY ERRORS IN THE PRODUCTS WILL BE CORRECTED. SOME STATES/JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES SO THE ABOVE EXCLUSIONS MAY NOT APPLY TO CUSTOMER. TOPSPIN'S LIMITED WARRANTY GIVES CUSTOMER SPECIFIC LEGAL RIGHTS. CUSTOMER MAY ALSO

xvi

HAVE OTHER RIGHTS, WHICH VARY FROM STATE/JURISDICTION TO

STATE/JURISDICTION.

10.Confidentiality. “Confidential Information” means any oral, written, graphic or machine-readable information disclosed by Topspin that is designated in writing to be confidential or proprietary or should be reasonably understood to be confidential. Customer agrees to: (a) refrain from using Confidential Information except as necessary to exercise its rights herein and (b) use best efforts preserve and protect the confidentiality of the Confidential Information. Notwithstanding the foregoing, Confidential Information shall not include information: (i) already publicly known; (ii) otherwise known to Customer through no wrongful conduct, or (iii) to the extent required to be disclosed by law or court order.

11.Limitation of Liability. IN NO EVENT SHALL TOPSPIN OR ITS SUPPLIERS BE LIABLE TO CUSTOMER OR ANY THIRD PARTY FOR COSTS OF PROCUREMENT OF SUBSTITUTE PRODUCTS OR SERVICES, LOST PROFITS, DATA OR BUSINESS, OR FOR ANY INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND ARISING OUT OF OR IN CONNECTION WITH THIS AGREEMENT, HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY (WHETHER IN CONTRACT, TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY OR OTHERWISE). TOPSPIN'S TOTAL AND CUMULATIVE LIABILITY ARISING OUT OF OR IN CONNECTION WITH ANY PRODUCTS PURCHASED BY CUSTOMER HEREUNDER SHALL IN NO EVENT EXCEED THE PURCHASE PRICE PAID BY CUSTOMER FOR SUCH PRODUCTS. THE TOPSPIN OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES, AND NOTWITHSTANDING ANY FAILURE OF ESSENTIAL PURPOSE OF ANY LIMITED REMEDY.

12.Termination. If either party breaches a material provision of this Agreement and fails to cure such breach within thirty (30) days after receiving written notice of the breach, the non-breaching party shall have the right to terminate this Agreement at any time. Customer's breach of a payment obligation constitutes a default the date the payment is due and Topspin shall have the right to terminate this Agreement immediately. Either party may terminate this Agreement, effective immediately upon written notice, if the other party becomes the subject of a voluntary or involuntary petition in bankruptcy or any proceeding relating to insolvency, receivership, liquidation or composition for the benefit of creditors. In the event that a party terminates this Agreement pursuant to this Section, (i) the terms and conditions of this Agreement shall apply to all Products shipped prior to the date of termination of this Agreement, and (ii) orders for Products which have not been shipped as of the date of termination may be terminated at the option of the party terminating this Agreement.

13.Export Control. Customer acknowledges and agrees that the Products provided under this Agreement may be subject to restrictions and controls imposed by the United States Export Administration Act and the regulations thereunder. Customer warrants that it will not export or re-export any Products provided under this Agreement into any country in violation of such controls or any other laws, rules or regulations of any country, state or jurisdiction.

14.Government Restricted Rights. As defined in FAR section 2.101, DFAR section 252.227-7014(a)(1) and DFAR section 252.227-7014(a)(5) or otherwise, the Software and associated documentation are “commercial items,” “commercial computer software” and/or “commercial computer software documentation.” Consistent with DFAR section 227.7202, FAR section 12.212 and other sections, any use, modification, reproduction, release, performance, display, disclosure or distribution thereof by or for the U.S. Government shall be governed solely by the terms of this Agreement and shall be prohibited except to the extent expressly permitted by the terms of this Agreement.

15.Miscellaneous. This Agreement is governed and interpreted in accordance with the laws of the State of California, U.S.A. without reference to conflicts of laws principles and excluding the United Nations Convention on Contracts for the Sale of Goods. The parties consent to the exclusive jurisdiction of, and venue in, Santa Clara County, California, U.S.A. Customer shall not transfer, assign or delegate this Agreement or any rights or obligations hereunder, whether voluntarily, by operation of law or otherwise, without the prior written consent of Topspin (except as expressly set forth in Section 4(f)). Topspin may assign this Agreement to any successor by way of merger,

xvii

acquisition or sale of all or substantially all of assets relating to this Agreement. Topspin or any successor may assign all of part of the right to payments under this Agreement. Any assignment or transfer of this Agreement made in contravention of the terms hereof shall be null and void. Subject to the foregoing, the terms and conditions of this Agreement shall be binding upon and inure to the benefit of the parties to it and their respective heirs, successors, assigns and legal representatives. This Agreement constitutes the entire agreement between the parties with respect to the subject matter hereof, and merges all prior negotiations and drafts of the parties with regard thereto. No modification, addition or deletion, or waiver of any of the terms and conditions of this Agreement shall be binding on Topspin unless made in a non-preprinted agreement clearly understood by both parties to be a modification or waiver, and signed by a duly authorized representative of Topspin. If any of the provisions of this Agreement is held by a court of competent jurisdiction to be invalid or unenforceable under any applicable statute or rule of law, such provision shall, to that extent, be deemed omitted. All notices permitted or required under this Agreement shall be in writing and shall be delivered in person or mailed by first class, registered or certified mail, postage prepaid.

xviii

Topspin 360 Overview

The following sections appear in this chapter:

•“Introduction” on page 1

•“Features” on page 2

Introduction

The Topspin 360 Server Switch provides data center managers with a high-performance, low-latency interconnect that runs Fibre Channel and Ethernet traffic transparently over an InfiniBand (IB) backplane. Figure 1-1 displays the front of the Topspin 360 Server Switch (with the bezel on).

Figure 1-1: Topspin 360 Chassis - Front View (Bezel End)

Figure 1-2 displays the back of the Topspin 360 Server Switch and numbers the slots on the chassis.

3	4	5	6	7	8	9	10	11	12	13
2										14
2

Figure 1-2: Topspin 360 Chassis - Back View (Service End) with Slot Numbers

The vertical slots at the top of the service end of the Topspin 360 are numbered 1 to 14 from left to right. The horizontal slots at the bottom of the service end of the chassis are numbered 15 to 16 from top to bottom. Field replaceable units (FRUs) with interfaces populate the service side of the chassis. FRUs such as power supplies and blower modules populate the front of the chassis, behind the bezel.

Features

The Topspin 360 provides unprecedented levels of availability, scalability, and manageability for deploying server clusters and on-demand computing. Switch modules provide up to a terabit of internal bandwidth, and Ethernet or Fibre Channel field-replaceable interface cards enable up to 72 ports of Fibre Channel, Ethernet, and InfiniBand interfaces in a single chassis.

Performance

Expressly designed to apply networking principles to computing problems, the Topspin 360 uses a switched InfiniBand fabric to interconnect server resources at 10 Gbps speeds. The combination of high bandwidth and low latency allows resources to be disaggregated and dynamically reassembled without the limits of the shared PCI bus.

Unified Fabric

The Topspin 360 integrates storage, server, and networking I/O into one unified fabric. With only one interface card in each host, you can manage all resources on one fabric and eliminate the need to install and manage multiple Ethernet, Fibre Channel, and inter-process communications (IPC) cards. The Topspin 360 aggregates and load balances all types of I/O, significantly reducing the number of managed ports in your environment and increasing availability.

Transparent Topology

Based on the Topspin Transparent Topology architecture, the Topspin 360 creates virtual IP and Fibre Channel interfaces on every server, allowing existing IP and storage area network (SAN) resources to seamlessly extend to server clusters. Using Fibre Channel gateways, SAN management technologies can “see” each individual node, enabling critical storage management and security models, such as Fibre Channel fabric zoning and storage-based LUN zoning. By adding Ethernet gateways, IT managers can also enable full IP connectivity, extending the value of server clusters to the rest of the enterprise.

Redundancy and High-Availability

The Topspin 360 is architected for full redundancy within a single box. Every component in the data path can be dual-connected from end to end. For example, a server with a 2-port HCA can be dual-attached to the two switch blades on the Topspin 360. Each gateway is also dual attached to the dual switch blades. In turn, the gateway modules are dual attached to Ethernet or Fibre Channel networks in pairs. With the addition of centralized load balancing and port aggregation, Ethernet and Fibre Channel uplink connections can be trunked and load balanced, allowing the Topspin 360 to reroute around failures. Each independently removable component is hot-swappable. Figure 1-3 depicts the internal Topspin 360 architecture.

Controllers

IB switches

hosts

InfiniBand

backplane

Ethernet gateway

Fibre Channel Gateways

Fibre Channel switches

Ethernet switches

Figure 1-3: Fully Redundant System Architecture

The Topspin 360 can be used in a variety of networking environments, including database tiers, application tiers, and Web tiers. The Topspin 360 provides 10 Gbps connectivity to servers, 1 Gbps connectivity to Ethernet networks, and 2 Gbps connectivity to Fibre Channel networks. You can also aggregate uplinks for greater bandwidth and redundancy.

Scalability

You can populate the service end of the Topspin 360 with additional Fibre Channel gateways, Ethernet gateways, and InfiniBand switch cards to increase the port count of your Server Switch. A second IB switch card adds 12 InfiniBand ports. Each additional FC gateway adds 2 Fibre Channel ports, and each additional Ethernet gateway adds 4 or 6 Ethernet ports.

Topspin 360 Components

The following sections appear in this chapter:

•“Introduction” on page 5

•“Controller Cards” on page 6

•“Ethernet and Fibre Channel Gateways” on page 9

•“InfiniBand Switch Cards” on page 10

•“Power Supply Modules” on page 13

•“Blower Trays” on page 15

Introduction

The components of the Topspin 360 support the scalability and high availability of the Server Switch. The sections in this chapter discuss how to use, install, remove, and swap field-replaceable components to maximize the potential of your Topspin 360.

You can install the following components in your chassis:

•controller cards

•I/O gateways (Fibre Channel and Ethernet)

•InfiniBand switch cards

•power supplies

•blower modules

CAUTION: Never place your hand inside an empty card or module bay. You should never have cause to place a hand anywhere inside the Topspin 360 chassis. Unused card and module bays should always have a Topspin 360 blanking panel over the bay to ensure proper safety, ventilation, and cooling.

Controller Cards

Controller cards manage the Topspin 360 and provide Ethernet and Serial Console port access to the Server Switch. You can install controller cards in slots 1 and 14 of the Topspin 360. Figure 2-1 displays the Topspin 360 controller card.

Figure 2-1: Topspin 360 Controller Card

The following points apply to the controller cards:

•Each Topspin 360 comes with one active controller and one warm standby controller.

•Upon power-up of the Topspin 360, slot 1 is the master (active). Slot 14 is the standby.

•The startup configuration file can be synchronized between the active and the standby controllers. The CLI prompts users to synchronize the controllers.

•When the master controller fails or reboots, the standby controller automatically takes over as the new master. All I/O cards (Switch cards, Fibre Channel gateways and Ethernet gateways) reboot when a failover occurs.

•Users can force a fail-over with the Element Manager GUI (Maintenance > Reboot) or command-line interface (reload command).

•The software upgrade/install process upgrades both the master controller and the standby controller at the same time. You do not need to upgrade them separately.

•A subnet manager runs on each controller card. The SM can manage a single Topspin 360 or redundant Topspin 360s.

Install a Controller Card

To install a controller card, perform the following steps:

1.Ground yourself appropriately.

2.Remove the blanking panel if one resides in the slot.

+ 55 hidden pages