Oracle IS2-46 Installation Manual

Download

Page 1

Oracle® InfiniBand Switch IS2-46 Installation Guide

Part No: E65868-01

September 2016

Page 2

Page 3

Oracle InfiniBand Switch IS2-46 Installation Guide

Part No: E65868-01

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?

ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Page 4

Référence: E65868-01

Ce logiciel et la documentation qui l’accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d’utilisation et de divulgation. Sauf stipulation expresse de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, accorder de licence, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d’interopérabilité avec des logiciels tiers ou tel que prescrit par la loi.

Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu’elles soient exemptes d’erreurs et vous invite, le cas échéant, à lui en faire part par écrit.

Si ce logiciel, ou la documentation qui l’accompagne, est livré sous licence au Gouvernement des Etats-Unis, ou à quiconque qui aurait souscrit la licence de ce logiciel pour le compte du Gouvernement des Etats-Unis, la notice suivante s’applique:

Ce logiciel ou matériel a été développé pour un usage général dans le cadre d’applications de gestion des informations. Ce logiciel ou matériel n’est pas conçu ni n’est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer des dommages corporels. Si vous utilisez ce logiciel ou matériel dans le cadre d’ applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l’utilisation de ce logiciel ou matériel pour ce type d’applications.

Oracle et Java sont des marques déposées d’Oracle Corporation et/ou de ses affiliés. Tout autre nom mentionné peut correspondre à des marques appartenant à d’autres propriétaires qu’Oracle.

Intel et Intel Xeon sont des marques ou des marques déposées d’Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d’Advanced Micro Devices. UNIX est une marque déposée d’The Open Group.

Ce logiciel ou matériel et la documentation qui l’accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l’accès à des contenus, produits ou services tiers, ou à leur utilisation, sauf mention contraire stipulée dans un contrat entre vous et Oracle.

Accessibilité de la documentation

Pour plus d’informations sur l’engagement d’Oracle pour l’accessibilité à la documentation, visitez le site Web Oracle Accessibility Program, à l’adresse http://www.oracle.com/

pls/topic/lookup?ctx=acc&id=docacc.

Accès aux services de support Oracle

Les clients Oracle qui ont souscrit un contrat de support ont accès au support électronique via My Oracle Support. Pour plus d’informations, visitez le site http://www.oracle.com/

pls/topic/lookup?ctx=acc&id=info ou le site http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs si vous êtes malentendant.

Page 5

Using This Documentation ................................................................................. 9

Product Documentation Library ......................................................................... 9

Feedback ....................................................................................................... 9

Understanding the Switch ................................................................................. 11

Installation Task Overview .............................................................................. 11

Switch Overview ........................................................................................... 13

Front Panel Components ................................................................................. 14

Rear Panel Components .................................................................................. 15

Confirming Specifications ................................................................................. 17

Physical Specifications ................................................................................... 17

Electrical Specifications .................................................................................. 18

Environmental Specifications ........................................................................... 18

Airflow Precautions ....................................................................................... 19

Rack Compatibility ........................................................................................ 20

Rack Precautions ........................................................................................... 20

Preparing for Installation .................................................................................. 23

Shipping Kit ................................................................................................. 24

▼ Record the MAC IDs ................................................................................ 25

▼ Assign IP Addresses to the SP and SCP Through DHCP ................................. 26

Handling Precautions ..................................................................................... 26

ESD Precautions ........................................................................................... 27

Installation Tools ........................................................................................... 27

Installing the Switch ......................................................................................... 29

▼ Stabilize the Rack .................................................................................... 29

▼ Install the Switch Brackets ......................................................................... 30

▼ Assemble the Cable Management Extenders .................................................. 32

Page 6

Contents

▼ Prepare the Air Duct Assembly .................................................................. 33

▼ Install the Cage Nuts ................................................................................ 35

▼ Install the Rackmount Hardware ................................................................. 37

▼ Route the Power Cords ............................................................................. 38

▼ Route the SER MGT Cable ....................................................................... 39

▼ Install the Switch ..................................................................................... 40

▼ Install the CMB ....................................................................................... 45

Connecting Cables ............................................................................................ 49

Cable Requirements ....................................................................................... 49

Standard MT and PrizmMT Cables .................................................................. 50

Cable Precautions .......................................................................................... 52

▼ Assemble the SFP+ Cables ........................................................................ 54

▼ Route the Cables ...................................................................................... 56

▼ Connect the RJ-45 Ethernet Cables ............................................................. 57

▼ Connect the PrizmMT Cables ..................................................................... 58

▼ Connect the SFP+ Cables .......................................................................... 60

▼ Secure Cables to the CMB ......................................................................... 62

Powering On the Switch ................................................................................... 67

▼ Retrieve the Power Cords and SER MGT Cable ............................................ 67

▼ Connect the SER MGT Cable .................................................................... 71

▼ Connect a Serial Management Device to the SER MGT Port ............................ 74

▼ Connect the Power Cords .......................................................................... 74

▼ Install the Air Duct Assembly .................................................................... 76

▼ Power On the Switch ................................................................................ 78

▼ Log In to the SP (SER MGT) .................................................................... 79

▼ Assign a Static IP Address to the SP ........................................................... 80

▼ Assign a Static IP Address to the SCP ......................................................... 82

▼ Log In to the SP (Network Management) ..................................................... 84

Verifying Functionality ...................................................................................... 87

▼ Verify LED Status .................................................................................... 87

LED Status ................................................................................................... 88

▼ Verify That the Switch Is Operational .......................................................... 89

▼ Verify Connectivity .................................................................................. 91

▼ Log In to the Oracle Fabric Manager Interface .............................................. 94

▼ Log In to the Oracle Fabric OS Interface ...................................................... 96

▼ Run the Oracle Fabric OS Configuration Wizard ............................................ 97

6 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 7

Contents

Oracle Fabric OS Configuration Wizard Parameters ............................................. 98

Glossary ............................................................................................................ 99

Index ................................................................................................................ 107

Page 8

8 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 9

Using This Documentation

■

Overview – Describes how to prepare for and install the Oracle InfiniBand Switch IS2-46.

■

Audience – Installers, technicians, system administrators, and authorized service providers.

■

Required knowledge – Advanced experience installing network hardware.

Product Documentation Library

Documentation and resources for this product and related products are available at http://www.

oracle.com/goto/is2-46/docs.

Feedback

Provide feedback about this documentation at http://www.oracle.com/goto/docfeedback.

Using This Documentation 9

Page 10

10 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 11

Understanding the Switch

These topics describe the switch and the installation process.

■

“Installation Task Overview” on page 11

■

“Switch Overview” on page 13

■

“Front Panel Components” on page 14

■

“Rear Panel Components” on page 15

Related Information

■

“Confirming Specifications” on page 17

■

“Preparing for Installation” on page 23

■

“Installing the Switch” on page 29

■

“Connecting Cables” on page 49

■

“Powering On the Switch” on page 67

■

“Verifying Functionality” on page 87

Installation Task Overview

A full-time SER MGT connection is where there is a persistent serial connection to the SER MGT port, and it is used for switch administration. A temporary SER MGT connection is where the SER MGT port is used only for switch installation and configuration of network management. Thereafter, network management is used for switch administration. The installation task overview accommodates both connection types and identifies any deviations from the installation sequence.

Step Description Links

1. Familiarize yourself with the front and rear of the switch.

2. Confirm supplied power and environmental factors.

■ “Front Panel Components” on page 14

■ “Rear Panel Components” on page 15

■ “Physical Specifications” on page 17

■ “Electrical Specifications” on page 18

■ “Environmental Specifications” on page 18

Understanding the Switch 11

Page 12

Installation Task Overview

Step Description Links

3. Understand air flow characteristics and rack cautions.

■ “Airflow Precautions” on page 19

■ “Rack Compatibility” on page 20

■ “Rack Precautions” on page 20

4. Verify shipped components and accessories. ■ “Shipping Kit” on page 24

5. Record the MAC IDs. ■ “Record the MAC IDs” on page 25

6. If DHCP is to assign specific IP addresses to

■ “Assign IP Addresses to the SP and SCP Through DHCP” on page 26

the SP and SCP, configure the DHCP server.

7. Heed handling and ESD precautions. ■ “Handling Precautions” on page 26

■ “ESD Precautions” on page 27

Gather appropriate tools. ■ “Installation Tools” on page 27

9. Install the switch into the rack. ■ “Installing the Switch” on page 29

10. Understand cable requirements and precautions.

■ “Cable Requirements” on page 49

■ “Standard MT and PrizmMT Cables” on page 50

■ “Cable Precautions” on page 52

11. If necessary, assemble the optical data cables. ■ “Assemble the SFP+ Cables” on page 54

12. Route the data cables. ■ “Route the Cables” on page 56

13. Connect data cables and secure them to the cable management bracket (CMB).

■ “Connect the RJ-45 Ethernet Cables” on page 57

■ “Connect the PrizmMT Cables” on page 58

■ “Connect the SFP+ Cables” on page 60

■ “Secure Cables to the CMB” on page 62

14. Retrieve the power cords and SER MGT

■ “Retrieve the Power Cords and SER MGT Cable” on page 67

cable.

15. Connect the serial management device. ■ “Connect the SER MGT Cable” on page 71

■ “Connect a Serial Management Device to the SER MGT

Port” on page 74

16. Connect power cords. ■ “Connect the Power Cords” on page 74

17. If you have a full-time SER MGT connection,

■ “Install the Air Duct Assembly” on page 76

install the air duct. If you have a temporary SER MGT connection, instead go to Step 15.

18. Power on the switch. ■ “Power On the Switch” on page 78

19. Log in to the service processor (SP). ■ “Log In to the SP (SER MGT)” on page 79

■ “Assign a Static IP Address to the SP” on page 80

■ “Assign a Static IP Address to the SCP” on page 82

■ “Log In to the SP (Network Management)” on page 84

20. If you have a temporary SER MGT

■ “Install the Air Duct Assembly” on page 76

connection, install the airduct.

21. Verify switch functionality. ■ “Verify LED Status” on page 87

■ “Verify That the Switch Is Operational” on page 89

■ “Verify Connectivity” on page 91

22. Administer the switch. ■ “Log In to the Oracle Fabric Manager Interface” on page 94

■ “Log In to the Oracle Fabric OS Interface” on page 96

■ “Run the Oracle Fabric OS Configuration Wizard” on page 97

12 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 13

Related Information

■

“Switch Overview” on page 13

■

“Front Panel Components” on page 14

■

“Rear Panel Components” on page 15

Switch Overview

The switch is a complete access switching and I/O platform that consolidates and virtualizes the switching infrastructure for server and storage systems deployed in high-performance enterprise clouds.

The switch features up to 38 4x IB ports supporting EDR speeds. Additionally, 10 10GbE ports and 24 1GBASE-T ports provide both network and gateway capabilities.

Separate, dedicated, and optimized on-board SP, ABD, and SCP manage chassis functionality, Ethernet traffic, and IB fabric respectively, through user-friendly CLIs and GUIs.

The switch ships preconfigured for network management through DHCP. However, the procedures in this document are serial management focused, with instructions to configure static IP addresses.

Understanding the Switch 13

Page 14

Front Panel Components

Related Information

■

“Installation Task Overview” on page 11

■

“Front Panel Components” on page 14

■

“Rear Panel Components” on page 15

Front Panel Components

The front panel is the fan and power supply end of the switch. This end of the switch is installed to the cold aisle.

No. Description

1 Power supplies

2 Power cord receptacles

3 Fan modules

4 RFID tag

5 SER MGT port

6 SER MGT module (SMM), with battery inside

Related Information

■

“Installation Task Overview” on page 11

■

“Switch Overview” on page 13

■

“Rear Panel Components” on page 15

14 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 15

Rear Panel Components

The rear panel is the data cable connection end of the switch. This end of the switch is installed to the hot aisle.

No. Description

1 PrizmMT connectors:

■ Ports 1 to 24 – IB 4x (magenta housing)

■ Ports 25 and 26 – Either IB 4x or 10GbE (magenta housing)

■ Ports 27 to 30 – IB 12x (black housing)

2 SFP+ connectors

Rear Panel Components

■ Ports 1 to 8 (left) – 10GbE Ethernet overlay network (IB switch chip)

■ Ports 1 and 2 (right) – 10GbE network management uplinks (Ethernet switch chip)

Note - The two right-most connectors, labeled 1 and 2, can also be used for network

management.

3 RJ-45 connectors – Any one of these connectors can be used for network management.

4 Chassis status LEDs

Related Information

■

“Installation Task Overview” on page 11

■

“Switch Overview” on page 13

■

“Front Panel Components” on page 14

Understanding the Switch 15

Page 16

16 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 17

Confirming Specifications

These topics describe the specifications of the switch and related precautions for installation.

■

“Physical Specifications” on page 17

■

“Electrical Specifications” on page 18

■

“Environmental Specifications” on page 18

■

“Airflow Precautions” on page 19

■

“Rack Compatibility” on page 20

■

“Rack Precautions” on page 20

Related Information

■

“Understanding the Switch” on page 11

■

“Preparing for Installation” on page 23

■

“Installing the Switch” on page 29

■

“Connecting Cables” on page 49

■

“Powering On the Switch” on page 67

■

“Verifying Functionality” on page 87

Physical Specifications

Dimension Metric U.S.

RU 1 1

Width 445.0 mm 17.52 in.

Depth 609.6 mm 24 in.

Height 44.5 mm 1.75 in.

Weight 11.4 kg 23.0 lbs

Related Information

■

“Electrical Specifications” on page 18

Confirming Specifications 17

Page 18

Electrical Specifications

■

“Environmental Specifications” on page 18

■

“Airflow Precautions” on page 19

■

“Rack Compatibility” on page 20

■

“Rack Precautions” on page 20

Electrical Specifications

Parameter AC Requirement

Voltage 100 VAC to 240 VAC single phase, 47 to 63 Hz

Current (per input) 5.4A maximum per input at 100 VAC

Current (total) 5.6A maximum total for all inputs at 100 VAC

Power 550W (Total input power is approximately equally divided among the

operating power supplies.)

Related Information

■

“Physical Specifications” on page 17

■

“Environmental Specifications” on page 18

■

“Airflow Precautions” on page 19

■

“Rack Compatibility” on page 20

■

“Rack Precautions” on page 20

Environmental Specifications

Parameter Metric Operating U.S. Operating

Temperature 5˚C to 35˚C 41˚F to 95˚F

Humidity 5% to 85% noncondensing, 27˚C maximum wet

Elevation (Oracle requirement) Maximum 3000 meters at 40˚C Maximum 9840 feet at 104˚F

bulb

Related Information

■

“Physical Specifications” on page 17

■

“Electrical Specifications” on page 18

5% to 85% noncondensing, 80˚F maximum wet bulb

18 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 19

■

“Airflow Precautions” on page 19

■

“Rack Compatibility” on page 20

■

“Rack Precautions” on page 20

Airflow Precautions

Caution - Proper airflow is essential for keeping the switch's internal temperatures within a safe

operating range.

Air flows from the front (fan end) of the switch to the rear (connector end) of the switch. Air flows through the power supplies from the sides of the switch to out the front.

Airflow Precautions

Follow these guidelines to ensure unrestricted airflow in the switch:

■

Install the switch so that the front faces the cool aisle and the rear faces the warm aisle.

■

Do not direct warm air into the switch.

■

Prevent recirculation of air within a rack or cabinet.

■

After servicing the switch, ensure that the air duct assembly is properly installed.

■

Route cables so that they do not interfere with airflow.

Confirming Specifications 19

Page 20

Rack Compatibility

■

Install the air duct assembly at the front of the switch.

Related Information

■

“Physical Specifications” on page 17

■

“Electrical Specifications” on page 18

■

“Environmental Specifications” on page 18

■

“Rack Compatibility” on page 20

■

“Rack Precautions” on page 20

Rack Compatibility

This switch and rackmounting hardware is designed for installation into a Sun Rack II or compatible rack or cabinet. The switch has a shorter form factor than Oracle rackmounted computing products. There is a gap between the rack front door and the switch front panel. To aid in proper airflow management and minimize air recirculation within the rack, install the air duct assembly at the front of the switch.

Related Information

■

“Physical Specifications” on page 17

■

“Electrical Specifications” on page 18

■

“Environmental Specifications” on page 18

■

“Airflow Precautions” on page 19

■

“Rack Precautions” on page 20

Rack Precautions

Caution - Equipment Loading. Always load equipment into a rack from the bottom to the

top so that the rack does not become top heavy and tip over. Deploy your rack's antitilt legs to prevent the rack from tipping during equipment installation.

Caution - Elevated Operating Ambient Temperature. If the switch is installed in a closed

or multi-unit rack assembly, the operating ambient temperature of the rack environment might be greater than the room ambient temperature. Therefore, install the equipment only in an environment compatible with the Tma specified for the switch.

20 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 21

Rack Precautions

Caution - Reduced Air Flow. Install the equipment in a rack so that the amount of air flow is

adequate for the safe operation of the equipment.

Caution - Mechanical Loading. Mount the equipment in the rack so that the weight is

distributed evenly. A hazardous condition can exist with uneven mechanical loading.

Caution - Circuit Overloading. Do not overload the power supply circuits. Before connecting

the switch to the supply circuit, review the equipment nameplate power ratings and consider the effect that circuit overloading might have on overcurrent protection and supply wiring.

Caution - Reliable Grounding. Maintain reliable grounding of rackmounted equipment. Give

particular attention to supply connections other than direct connections to the branch circuit (for example, use of power strips).

Caution - Do not use rack mounted equipment as a shelf or a work space.

Related Information

■

“Physical Specifications” on page 17

■

“Electrical Specifications” on page 18

■

“Environmental Specifications” on page 18

■

“Airflow Precautions” on page 19

■

“Rack Compatibility” on page 20

Confirming Specifications 21

Page 22

22 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 23

Preparing for Installation

These topics provide information to prepare for the switch installation process.

■

“Shipping Kit” on page 24

■

“Record the MAC IDs” on page 25

■

“Assign IP Addresses to the SP and SCP Through DHCP” on page 26

■

“Handling Precautions” on page 26

■

“ESD Precautions” on page 27

■

“Installation Tools” on page 27

Related Information

■

“Understanding the Switch” on page 11

■

“Confirming Specifications” on page 17

■

“Installing the Switch” on page 29

■

“Connecting Cables” on page 49

■

“Powering On the Switch” on page 67

■

“Verifying Functionality” on page 87

Preparing for Installation 23

Page 24

Shipping Kit

No. Description

1 Switch

2 Front mounting brackets, long (shown mounted)

3 Front mounting brackets, short

4 C-channel brackets (shown mounted)

5 Fasteners

6 Power cords

7 Documentation

8 Cable management extenders

9 Long rails

10 Cable management cover

11 Cable management bracket (CMB)

12 Attachment brackets

13 Attachment plates

14 Air duct

15 Cover

16 Snap brackets

24 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 25

Related Information

■

“Record the MAC IDs” on page 25

■

“Assign IP Addresses to the SP and SCP Through DHCP” on page 26

■

“Handling Precautions” on page 26

■

“ESD Precautions” on page 27

■

“Installation Tools” on page 27

Record the MAC IDs

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Remove the Customer Information sheet from the outside of the shipping carton.

Record the MAC IDs

On the Customer Information sheet, find the switch serial number.

Record the serial number to a document intended for this purpose.

In the Network Interfaces table of the sheet, find the primary MAC ID.

This is the MAC ID of the SP within the switch.

Record the SP MAC ID to the document.

In the Network Interfaces table of the sheet, find the secondary MAC ID.

This is the MAC ID of the SCP within the switch.

Record the SCP MAC ID to the document.

Secure the Customer Information sheet in a safe place.

10.

Consider your next steps.

■ If you will assign IP addresses to the SP and SCP through DHCP, configure your DHCP server now.

See “Assign IP Addresses to the SP and SCP Through DHCP” on page 26.

■ If you will assign static IP addresses to the SP and SCP through the SER

MGT port, go to “Handling Precautions” on page 26.

Preparing for Installation 25

Page 26

Assign IP Addresses to the SP and SCP Through DHCP

Related Information

■

“Shipping Kit” on page 24

■

“Assign IP Addresses to the SP and SCP Through DHCP” on page 26

■

“Handling Precautions” on page 26

■

“ESD Precautions” on page 27

■

“Installation Tools” on page 27

Assign IP Addresses to the SP and SCP Through DHCP

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Log in to the DHCP server and configure it to assign a predetermined IP address and host name for the SP MAC ID you recorded.

Record this IP address and host name for future reference.

Configure the DHCP server to assign a predetermined IP address and host name for the SCP MAC ID you recorded.

Record this IP address and host name for future reference.

When the switch is connected to the management network and powered on, the SP and SCP will receive the predetermined IP addresses.

Continue the installation.

See “Handling Precautions” on page 26.

Related Information

■

“Shipping Kit” on page 24

■

“Record the MAC IDs” on page 25

■

“Handling Precautions” on page 26

■

“ESD Precautions” on page 27

■

“Installation Tools” on page 27

Handling Precautions

Caution - Deploy the antitilt legs on the equipment rack before beginning an installation.

26 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 27

Caution - Two people are required to lift and mount this switch into a rack when performing the

tasks in this document.

Caution - When performing a two-person task, always communicate your intentions clearly

before, during, and after each step to minimize confusion.

Related Information

■

“Shipping Kit” on page 24

■

“Record the MAC IDs” on page 25

■

“Assign IP Addresses to the SP and SCP Through DHCP” on page 26

■

“ESD Precautions” on page 27

■

“Installation Tools” on page 27

ESD Precautions

When installing the switch chassis, follow antistatic precautions:

■

Use an antistatic mat as a work surface.

■

Wear an antistatic wrist strap that is attached to either the mat or a metal portion of the switch chassis.

Related Information

■

“Shipping Kit” on page 24

■

“Record the MAC IDs” on page 25

■

“Assign IP Addresses to the SP and SCP Through DHCP” on page 26

■

“Handling Precautions” on page 26

■

“Installation Tools” on page 27

Installation Tools

These tools are necessary or beneficial for installing the switch:

■

Antistatic wrist strap

■

No. 2 Philips screwdriver

Preparing for Installation 27

Page 28

Installation Tools

■

No. 1 Philips screwdriver

■

Cage nut installation tool or flat-blade screwdriver

■

Serial terminal device and necessary cabling

■

Masking tape

■

Paper clip

■

PrizmMT ferrule cleaner - USConec part number 16899

■

Antistatic mat

■

Flashlight

■

Gloves

Related Information

■

“Shipping Kit” on page 24

■

“Record the MAC IDs” on page 25

■

“Assign IP Addresses to the SP and SCP Through DHCP” on page 26

■

“Handling Precautions” on page 26

■

“ESD Precautions” on page 27

28 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 29

Installing the Switch

Perform these tasks in the order presented to install the switch.

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Related Information

■

“Understanding the Switch” on page 11

■

“Confirming Specifications” on page 17

■

“Preparing for Installation” on page 23

■

“Connecting Cables” on page 49

■

“Powering On the Switch” on page 67

■

“Verifying Functionality” on page 87

Stabilize the Rack

Caution - To reduce the risk of personal injury, stabilize the rack and extend all antitilt devices

before installing the switch.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

Installing the Switch 29

Page 30

Install the Switch Brackets

See “Installation Task Overview” on page 11.

Open and remove the front and rear doors from the rack.

If there are leveling feet beneath the rack to prevent it from rolling, extend these leveling feet fully downward to the floor.

Fully extend the rack's antitilt legs or antitilt bar, which are located at the bottom front of the rack.

Install the mounting brackets on the switch.

See “Install the Switch Brackets” on page 30.

Related Information

■

“Install the Switch Brackets” on page 30

“Assemble the Cable Management Extenders” on page 32

“Prepare the Air Duct Assembly” on page 33

“Install the Cage Nuts” on page 35

“Install the Rackmount Hardware” on page 37

“Route the Power Cords” on page 38

“Route the SER MGT Cable” on page 39

“Install the Switch” on page 40

“Install the CMB” on page 45

Install the Switch Brackets

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Using a No. 1 Philips screwdriver, attach the long front brackets (with cutouts) to the front of the switch with four M3x6 flathead screws on each side.

30 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 31

Install the Switch Brackets

The flange of the long front mounting brackets points away from the switch.

Attach the C-channel brackets to the switch with four M3x6 flathead screws on each side.

The edge of each C-channel bracket is flush to the rear of the chassis.

Assemble the cable management extenders.

See “Assemble the Cable Management Extenders” on page 32.

Related Information

■

“Stabilize the Rack” on page 29

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Installing the Switch 31

Page 32

Assemble the Cable Management Extenders

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Slide the attachment bracket over the extender, so that the tab on the bracket is opposite the flange on the extender.

The open end of the tab is toward the flange. The flat end of the tab is toward the rear of the extender.

Place the attachment plate on the flange side of the extender, opposite the attachment bracket.

Use two M4x10 flathead screws to sandwich the attachment bracket and plate to the extender, in the position farthest from the flange.

Using a No. 1 Phillips screwdriver, tighten the two screws.

32 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 33

Prepare the Air Duct Assembly

Repeat Step 2 through Step 5 for the other cable management extender.

Prepare the air duct assembly.

See “Prepare the Air Duct Assembly” on page 33.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Prepare the Air Duct Assembly

Note - Both the cover and air duct are symmetrical, and have no defined top or bottom. If the

cover has a logo, that logo goes to the right.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Installing the Switch 33

Page 34

Prepare the Air Duct Assembly

Align the large opening of the air duct with the mesh on the back side of the cover.

Press the air duct into the cover, to make a firm friction fit.

Install the cage nuts into the rack.

See “Install the Cage Nuts” on page 35.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

34 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 35

Install the Cage Nuts

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Determine where you are installing the switch.

Mark this location on each rack post with masking tape.

Install the Cage Nuts

Counting the number of rack units from the bottom of the rack to the mounting location assists you when marking the other rack posts.

Install eight cage nuts into the rack, two cage nuts for each rack post.

Installing the Switch 35

Page 36

Install the Cage Nuts

Note - Each RU mounting location on the rack post has three square holes. For installing the

switch, there is no need for a cage nut in the center hole.

Orient the cage nut to the square hole with the flanges to the right and left.

Insert the outer flange into the square hole.

Using the cage nut installation tool or flat-blade screwdriver as a guide, catch the inner flange and pry it into the square hole.

Pull the cage nut installation tool or flat blade screwdriver from the flange.

Repeat Step 4a to Step 4d for each cage nut.

Install the rackmount hardware.

See “Install the Rackmount Hardware” on page 37.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Rackmount Hardware” on page 37

36 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Install the Rackmount Hardware

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Place the long rail at the mounting location on the post at the rear of the rack.

Butt the flange of the cable management extender against the flange of the long rail.

Install the Rackmount Hardware

Using a No. 2 Philips screwdriver, secure the assembly to the post with two M6x12 screws.

Repeat Step 2 through Step 4 for the other cable management extender and long rail.

Installing the Switch 37

Page 38

Route the Power Cords

Route the power cords.

See “Route the Power Cords” on page 38.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Route the Power Cords

The facility power receptacles for the power cords should be located such that the power cords are routed out of the way.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Route the female end of the power cords up from the power distribution panel or receptacles at the rear of the rack.

Route the cords across the rack ceiling to the front of the rack.

Route the cords down through the vertical support, so that the female end rests at the location where the switch will install.

Add about 3 inches of slack to enable connection and disconnection with the switch power supply receptacles.

Secure the power cord to this vertical length, so that it does not drop any farther.

Use hook and loop fastener straps to bundle and secure the cords.

Consider your next task.

■ If you will have a full-time SER MGT connection, route the SER MGT cable.

See “Route the SER MGT Cable” on page 39.

38 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 39

■ If the SER MGT connection is only temporary for installation, install the switch now.

See “Install the Switch” on page 40.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the SER MGT Cable” on page 39

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Route the SER MGT Cable

If there is to be a full-time connection of the SER MGT cable, such as to a terminal server, follow this task to route the cable.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Route the SER MGT cable from its entry point at the rear of the rack up to the ceiling of the rack.

Route the SER MGT cable across the rack ceiling to the front of the rack.

The SER MGT cable can run parallel to one of the power cords.

Route the SER MGT cable down through the vertical support, so that the end rests at the location where you are installing the switch.

Add slack to the SER MGT cable to enable connection and disconnection with the SER MGT port.

■ If the SER MGT cable is on the left side of the rack, add 12 inches of slack.

■ If the SER MGT cable is on the right side of the rack, add 24 inches of slack.

Installing the Switch 39

Page 40

Install the Switch

Secure the SER MGT cable to this vertical length, so that it does not drop any farther.

Use hook and loop fastener straps to bundle and secure the SER MGT cable.

Install the switch.

See “Install the Switch” on page 40.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Install the Switch” on page 40

■

“Install the CMB” on page 45

Install the Switch

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Carefully lift the switch and slide it into the rack, from the front rearward.

40 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 41

Install the Switch

Ensure that the ends of the long rails slide into the C-channel brackets at the rear of the switch chassis.

Using a No. 2 Philips screwdriver, hold the front of the switch chassis to the front rack posts with two M6x12 screws at each side.

Installing the Switch 41

Page 42

Install the Switch

Note - Do not fully tighten the screws at this time.

42 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 43

Slide the snap bracket under the loose screws with the tabs pointing into the rack.

Install the Switch

Installing the Switch 43

Page 44

Install the Switch

While pressing on the word PUSH embossed in the snap bracket, fully tighten the screws.

Repeat for the snap bracket on the other side.

Install the CMB.

See “Install the CMB” on page 45.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

■

“Install the CMB” on page 45

44 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 45

Install the CMB

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

If installed, loosen the two captive thumbscrews that secure the cover to the CMB and lift the cover off.

Install the CMB

Installing the Switch 45

Page 46

Install the CMB

Install the CMB to the tabs of the attachment brackets at the rear of the switch, tightening the thumbscrews on each side of the CMB.

Consider your next task.

■ If your SFP+ optical cables must be assembled, do that now.

See “Assemble the SFP+ Cables” on page 54.

■ If no cable assembly is required, route the cables to the switch.

See “Route the Cables” on page 56.

Related Information

■

“Stabilize the Rack” on page 29

■

“Install the Switch Brackets” on page 30

■

“Assemble the Cable Management Extenders” on page 32

■

“Prepare the Air Duct Assembly” on page 33

■

“Install the Cage Nuts” on page 35

■

“Install the Rackmount Hardware” on page 37

■

“Route the Power Cords” on page 38

■

“Route the SER MGT Cable” on page 39

46 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 47

■

“Install the Switch” on page 40

Install the CMB

Installing the Switch 47

Page 48

48 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 49

Connecting Cables

These topics provide cabling information and describe how to cable the switch.

Description Links

Understand cable requirements. “Cable Requirements” on page 49

Identify cable differences. “Standard MT and PrizmMT Cables” on page 50

Heed cable precautions. “Cable Precautions” on page 52

If necessary, assemble the optical data cables. “Assemble the SFP+ Cables” on page 54

Route the data cables. “Route the Cables” on page 56

Connect the data cables. ■ “Connect the RJ-45 Ethernet Cables” on page 57

■ “Connect the PrizmMT Cables” on page 58

■ “Connect the SFP+ Cables” on page 60

Secure the cables to the CMB. “Secure Cables to the CMB” on page 62

Related Information

■

“Understanding the Switch” on page 11

■

“Confirming Specifications” on page 17

■

“Preparing for Installation” on page 23

■

“Installing the Switch” on page 29

■

“Powering On the Switch” on page 67

■

“Verifying Functionality” on page 87

Cable Requirements

Minimal functionality of the switch requires this cabling:

■

Management console – The management console is the physical medium that the administrator uses to interact with the SP. There are two ways of making this connection:

■

Serial management – A serial device, such as a TIP connection, a serial terminal, or a terminal server, is cabled to the SER MGT port on the front of the switch. An advantage of serial management with a dedicated serial terminal is that the connection is more

Connecting Cables 49

Page 50

Standard MT and PrizmMT Cables

secure than through network management. Because of the nature of the serial signal, a serial management cable cannot be used reliably if it is more than 30 feet (9 meters) long.

■

Network Management – Network management permits you to use your existing network to connect an SFP+ or RJ-45 cable to any connector on the rear of the switch. This is the preferred means of connecting to the SP, because it gives you the flexibility to use any host on the network to interact with the SP and the Oracle ILOM web interface. To improve security, a VLAN ID can be configured for the SP.

■

Power supplies – Shipped with the switch are country-specific power cords to supply facility power to the switch from power distribution units inside of the rack.

■

RJ-45 Ethernet ports – These 24 ports provide switched 1GBASE-T functionality through CAT 6 rated cabling.

■

SFP+ Ethernet ports – These 10 ports provide switched 10GbE functionality with either copper cables or optical fiber.

■

PrizmMT ports - These 30 ports provide switched 100 Gbps EDR IB functionality with special optical fiber cables.

Related Information

■

“Standard MT and PrizmMT Cables” on page 50

■

“Cable Precautions” on page 52

■

“Assemble the SFP+ Cables” on page 54

■

“Route the Cables” on page 56

■

“Connect the RJ-45 Ethernet Cables” on page 57

■

“Connect the PrizmMT Cables” on page 58

■

“Connect the SFP+ Cables” on page 60

■

“Secure Cables to the CMB” on page 62

Standard MT and PrizmMT Cables

While PrizmMT and standard MT optical fiber cables look similar, they are not. PrizmMT cables use clear plastic ferrules that have lenses to focus the laser light across the connection air gap. Standard MT cables rely upon physical contact of fiber tips at each connection.

Caution - Connecting a PrizmMT cable to a standard MT receptacle, such as on a QSFP

transceiver, will damage the PrizmMT ferrule and render the cable unusable.

To help discriminate between PrizmMT and standard MT cables, PrizmMT cables are identified with certain features. The features are ferrule, housing color, alignment method, and fiber spacing.

50 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 51

Standard MT and PrizmMT Cables

Technology Ferrule Housing

Color

4x PrizmMT optical cable PrizmMT (clear with lenses) Magenta Post and Hole Wide

12x PrizmMT optical cable PrizmMT (clear with lenses) Black Post and Hole Wide

4x standard MT optical cable Standard MT (black) Aqua Pin or Hole Narrow

4x standard MT optical cable Standard MT (black) Black Pin or Hole Narrow

Alignment Method Fiber

Spacing

Note - Some supported cables have a PrizmMT connector on one end, and a standard MT

connector on the other. Pay close attention when connecting this cable, to identify which connector is which, and not confuse the two.

Visually comparing the design of the two cables, note the differences between the standard MT cable on the left and the PrizmMT cable on the right.

No. Description

1 Aqua colored housing (standard MT)

2 Standard ferrule (standard MT)

3 Alignment pin holes (standard MT)

4 Contact type optical fibers (standard MT)

Connecting Cables 51

Page 52

Cable Precautions

No. Description

5 Magenta colored housing (PrizmMT)

6 Prizm ferrule (PrizmMT)

7 Alignment post (PrizmMT)

8 Focused optical fiber array (PrizmMT)

9 Alignment hole (PrizmMT)

Related Information

■

“Cable Requirements” on page 49

■

“Cable Precautions” on page 52

■

“Assemble the SFP+ Cables” on page 54

■

“Route the Cables” on page 56

■

“Connect the RJ-45 Ethernet Cables” on page 57

■

“Connect the PrizmMT Cables” on page 58

■

“Connect the SFP+ Cables” on page 60

■

“Secure Cables to the CMB” on page 62

Cable Precautions

To prevent data cable damage, you must follow these cautions.

Do not uncoil the cable, as a kink might occur. Hold the coil closed as you unroll the cable, pausing to allow the cable to relax as it is unrolled.

Do not pull the cable out of the shipping box, through any opening, or around any corners. Unroll the cable as you lay it down and move it through turns.

Do not step on the cable or connectors. Plan cable paths away from foot traffic or rolling loads.

Do not bend the cables to a radius tighter than 85 mm (3.4 inches). Ensure that cable turns are as wide as possible.

52 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 53

Cable Precautions

Do not twist the cable to open a kink. If it is not severe, open the kink by unlooping the cable.

Do not straighten the cable to correct a bend that is too tight. Leave the cable bend as is.

Do not drop the cable or connectors from any height. Gently set the cable down, resting the cable connectors on a stable surface.

Do not drag the cable or its connectors over any surface. Carry the entire cable to and from the points of connection.

Do not pack the cable to fit a tight space. Use an alternative cable route.

Do not hang the cable for a length more than 2 meters (7 feet). Minimize the hanging weight with intermediate retention points.

Do not cinch the cable with hard fasteners or cable ties. Use soft hook-and-loop fastener for bundling and securing cables.

Do not force the cable connector into the receptacle by pushing on the cable. Apply connection or disconnection forces at the connector only.

Related Information

■

“Cable Requirements” on page 49

■

“Standard MT and PrizmMT Cables” on page 50

■

“Assemble the SFP+ Cables” on page 54

■

“Route the Cables” on page 56

■

“Connect the RJ-45 Ethernet Cables” on page 57

■

“Connect the PrizmMT Cables” on page 58

■

“Connect the SFP+ Cables” on page 60

■

“Secure Cables to the CMB” on page 62

Connecting Cables 53

Page 54

Assemble the SFP+ Cables

If the optical SFP+ cables are unassembled, you must assemble the cables before attaching them to the switch.

Note - The ends of the optical fiber cable and the receptacles of the transceivers must be clean

and optically clear before assembly. Do not remove the protective caps from the optical fiber cable or the plugs from the transceivers until instructed to do so.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Remove the SFP+ optical transceivers and fiber optic cables from their packaging.

Remove the plug from an SFP+ optical transceiver.

Remove the caps from an LC connector of the fiber optic cable.

Holding the shafts of the LC connector, insert the LC connector into the receptacles of the SFP+ optical transceiver.

54 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 55

Assemble the SFP+ Cables

Note - The LC connector and SFP+ receptacles are keyed for proper fitting.

Push the connector into the transceiver until it clicks.

Repeat from Step 3 for the other end of the cable.

Repeat from Step 2 for all SFP+ cables to be assembled.

Route the data cables.

See “Route the Cables” on page 56.

Related Information

■

“Cable Requirements” on page 49

■

“Standard MT and PrizmMT Cables” on page 50

■

“Cable Precautions” on page 52

■

“Route the Cables” on page 56

■

“Connect the RJ-45 Ethernet Cables” on page 57

Connecting Cables 55

Page 56

Route the Cables

■

“Connect the PrizmMT Cables” on page 58

■

“Connect the SFP+ Cables” on page 60

■

“Secure Cables to the CMB” on page 62

Route the Cables

When cabling a topology, start at the nodes of the fabric or hosts of the network, and work toward the switch.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

If necessary, prepare the cables to connect the nodes or hosts to the switch.

See “Assemble the SFP+ Cables” on page 54.

Prepare and verify that the nodes or hosts are operational, properly configured, and ready to accept fabric or network cabling.

For each node or host, begin data cabling.

Attach the cable to the appropriate connector of the node or host.

Route and bundle the data cable through the topology following the cautions provided in this document.

See “Cable Precautions” on page 52.

Bring the cables to the location in the rack where the switch is installed.

Repeat Step 4a through Step 4c for each node or host.

Connect the cables to the switch.

See “Connect the RJ-45 Ethernet Cables” on page 57.

Related Information

■

“Cable Requirements” on page 49

■

“Standard MT and PrizmMT Cables” on page 50

■

“Cable Precautions” on page 52

■

“Assemble the SFP+ Cables” on page 54

■

“Connect the RJ-45 Ethernet Cables” on page 57

■

“Connect the PrizmMT Cables” on page 58

56 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 57

■

“Connect the SFP+ Cables” on page 60

■

“Secure Cables to the CMB” on page 62

Connect the RJ-45 Ethernet Cables

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Orient the RJ-45 connector to where it will connect to the switch.

The tab is up for the upper row of receptacles, and the tab is down for the lower row of receptacles.

Connect the RJ-45 Ethernet Cables

Insert the connector into the receptacle until it clicks, securing it in place.

Lay the cable into a slot on the CMB.

Repeat Step 2 through Step 4 for each cable to be connected.

Consider your next task.

■ Connect the PrizmMT cables if necessary.

See “Connect the PrizmMT Cables” on page 58.

■ Connect the SFP+ cables if necessary.

See “Connect the SFP+ Cables” on page 60.

Connecting Cables 57

Page 58

Connect the PrizmMT Cables

■ Secure the cables to the CMB.

See “Secure Cables to the CMB” on page 62.

Related Information

■

“Cable Requirements” on page 49

■

“Standard MT and PrizmMT Cables” on page 50

■

“Cable Precautions” on page 52

■

“Assemble the SFP+ Cables” on page 54

■

“Route the Cables” on page 56

■

“Connect the PrizmMT Cables” on page 58

■

“Connect the SFP+ Cables” on page 60

■

“Secure Cables to the CMB” on page 62

Connect the PrizmMT Cables

Note - PrizmMT receptacles labeled 1 through 26 are for 4x magenta-colored housing PrizmMT

cables. PrizmMT receptacles labeled 27 through 30 are for 12x black-colored housing PrizmMT cables.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Remove any filler plug (if installed) from the PrizmMT receptacle where you are installing the cable and visually inspect the receptacle.

The receptacle should be clean and free of dirt or debris.

If the receptacle is dirty, clean it.

Remove the adapter from the end of the PrizmMT cleaner.

Insert the cleaner tip into the receptacle.

Pump the cleaner into the receptacle, advancing the cleaning cloth inside.

Remove the cleaner from the receptacle and verify cleanliness.

Repeat from Step 3b if the fibers are still dirty.

58 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 59

Connect the PrizmMT Cables

Replace the adapter onto the end of the PrizmMT cleaner.

Remove the protective cap from the cable connector and visually inspect the connector.

The connector should be clean and free of dirt or debris.

If the connector is dirty, clean it.

Open the lid of the adapter on the end of the PrizmMT cleaner.

Insert the connector into the adapter.

Press the connector against the adapter, thereby pumping the cleaner, and advancing the cleaning cloth inside.

Remove the connector from the adapter and verify cleanliness.

Repeat from Step 5b if the fibers are still dirty.

Close the lid of the adapter on the end of the PrizmMT cleaner.

Align the PrizmMT connector to where it will connect.

Firmly press the PrizmMT connector into the receptacle until you feel a detent.

Connecting Cables 59

Page 60

Connect the SFP+ Cables

Lay the cable into a slot on the CMB.

Repeat Step 2 through Step 8 for each cable to be connected.

10.

Consider your next task.

■ Connect the SFP+ cables if necessary.

■ Secure the cables to the CMB.

Related Information

■

See “Connect the SFP+ Cables” on page 60.

See “Secure Cables to the CMB” on page 62.

“Cable Requirements” on page 49

“Standard MT and PrizmMT Cables” on page 50

“Cable Precautions” on page 52

“Assemble the SFP+ Cables” on page 54

“Route the Cables” on page 56

“Connect the RJ-45 Ethernet Cables” on page 57

“Connect the SFP+ Cables” on page 60

“Secure Cables to the CMB” on page 62

Connect the SFP+ Cables

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Remove the protective cap from the connector or transceiver and visually inspect the cable connector.

The shell should not be bent and should be parallel to the inner boards. If the connector is bent or damaged, use a different cable.

Ensure that the retraction strap is folded back against the cable.

Orient the cable connector to the SFP+ receptacle squarely and horizontally.

60 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 61

Connect the SFP+ Cables

Ensure that the L groove is up for the top row of receptacles, or that the L groove is down for the bottom row of receptacles.

Slowly move the connector in.

As you slide the connector in, the shell should be in the center of the SFP+ receptacle.

■ If the connector stops or binds after about 1/4 in. (5 mm) travel, back out and

repeat from Step 4.

■ If the connector stops or binds with about 1/8 in. (2 mm) still to go, back out and repeat Step 5.

Continue to push the connector in until you feel a detent.

Lay the cable into a slot on the CMB.

Repeat Step 2 through Step 7 for each cable to be connected.

Secure the cables to the CMB.

Connecting Cables 61

Page 62

Secure Cables to the CMB

See “Secure Cables to the CMB” on page 62.

Related Information

■

“Cable Requirements” on page 49

■

“Standard MT and PrizmMT Cables” on page 50

■

“Cable Precautions” on page 52

■

“Assemble the SFP+ Cables” on page 54

■

“Route the Cables” on page 56

■

“Connect the RJ-45 Ethernet Cables” on page 57

■

“Connect the PrizmMT Cables” on page 58

■

“Secure Cables to the CMB” on page 62

Secure Cables to the CMB

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

62 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 63

Secure Cables to the CMB

While connecting the RJ-45, SFP+, and PrizmMT cables to the switch, lay the cables into an available slot on the CMB.

Connecting Cables 63

Page 64

Secure Cables to the CMB

After all cables have been laid, replace the cover for the CMB and tighten the thumbscrews.

Route the data cables so that they do not interfere with other cables, or with servicing the switch or other systems.

Use hook and loop fastener straps to bundle and secure the cables.

Note - Do not use cable zip ties to bundle or secure the cables, because the ties damage the

fibers inside the cables.

Retrieve the power cords and SER MGT cable (if previously routed) for connection to the switch.

See “Retrieve the Power Cords and SER MGT Cable” on page 67.

Related Information

■

“Cable Requirements” on page 49

■

“Standard MT and PrizmMT Cables” on page 50

■

“Cable Precautions” on page 52

■

“Assemble the SFP+ Cables” on page 54

■

“Route the Cables” on page 56

64 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 65

■

“Connect the RJ-45 Ethernet Cables” on page 57

■

“Connect the PrizmMT Cables” on page 58

■

“Connect the SFP+ Cables” on page 60

Secure Cables to the CMB

Connecting Cables 65

Page 66

66 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 67

Powering On the Switch

These topics describe how to power on the switch for the first time.

Description Links

Retrieve the power cords and SER MGT cable.

Connect the serial management device.

Connect the power cords. “Connect the Power Cords” on page 74

If you have a full-time SER MGT connection, install the air duct.

Power on the switch. “Power On the Switch” on page 78

If you are not using DHCP-assigned IP addresses, log in to the service processor (SP) through the SER MGT connection.

Assign a static IP address to the SP “Assign a Static IP Address to the SP” on page 80

Assign a static IP address to the SCP “Assign a Static IP Address to the SCP” on page 82

“Retrieve the Power Cords and SER MGT Cable” on page 67

■ “Connect the SER MGT Cable” on page 71

■ “Connect a Serial Management Device to the SER MGT

Port” on page 74

“Install the Air Duct Assembly” on page 76

“Log In to the SP (SER MGT)” on page 79

Related Information

■

“Understanding the Switch” on page 11

■

“Confirming Specifications” on page 17

■

“Preparing for Installation” on page 23

■

“Installing the Switch” on page 29

■

“Connecting Cables” on page 49

■

“Verifying Functionality” on page 87

Retrieve the Power Cords and SER MGT Cable

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

Powering On the Switch 67

Page 68

Retrieve the Power Cords and SER MGT Cable

See “Installation Task Overview” on page 11.

Take the paper clip and lift the smaller loop away from the larger loop.

Continue to open the paper clip and fold the smaller loop over, so that the paper clip now resembles the letter S.

Open the larger loop approximately 45 degrees to form a hook.

Holding the smaller loop, insert the larger loop into the opening of the long front mounting bracket, to hook the power cord hanging inside.

68 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 69

Retrieve the Power Cords and SER MGT Cable

Pull the power cord through the opening, until you can grasp it with your fingers.

Pull the cord through so that the end of the cord is free.

Repeat Step 5 through Step 7 for the other power cord.

If you will have a full-time SER MGT connection, repeat Step 5 through Step 7 for the SER MGT cable.

Powering On the Switch 69

Page 70

Retrieve the Power Cords and SER MGT Cable

Note - When pulling the SER MGT cable, be careful not to break the tab of the RJ-45

connector.

10.

Consider your next steps.

■ If you have DHCP-assigned IP addresses, connect the power cords.

See “Connect the Power Cords” on page 74.

■ Otherwise, connect the SER MGT cable.

See “Connect the SER MGT Cable” on page 71.

Related Information

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

70 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 71

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Connect the SER MGT Cable

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Consider your next step.

■ If the SER MGT cable will be a full-time connection, go to Step 3.

Connect the SER MGT Cable

■ If the SER MGT connection will only be temporary for installation, go to Step

Pass the end of the SER MGT cable through the opening on the side of the air duct assembly.

■ If the SER MGT cable is on the left, pass the cable from the outside, through the opening on the left side (no Oracle logo) in to the center.

Powering On the Switch 71

Page 72

Connect the SER MGT Cable

■ If the SER MGT cable is on the right, pass the cable from the outside, through the opening on the right side (Oracle logo) in to the center.

Continue to pull the cable through the air duct assembly to take up the slack.

Orient the RJ-45 connector to the SER MGT receptacle at the front of the switch, adjacent to the left power supply.

72 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 73

The tab is up for the receptacle.

Connect the SER MGT Cable

Insert the connector into the receptacle until it clicks, securing it in place.

Connect the other end of the SER MGT cable to a serial device.

See “Connect a Serial Management Device to the SER MGT Port” on page 74.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Powering On the Switch 73

Page 74

Connect a Serial Management Device to the SER MGT Port

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Configure the serial management device to these parameters:

■

9600 baud

■

8 data bits

■

No parity

■

1 stop bit

■

No handshake

Connect the SER MGT cable from the switch to the serial management device.

If necessary, route the SER MGT cable so that it does not interfere with other cables, with servicing the switch, or with other systems.

Connect the power cords.

See “Connect the Power Cords” on page 74.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Connect the Power Cords

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

74 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 75

Connect the Power Cords

See “Installation Task Overview” on page 11.

Ensure that the circuit breakers for the facility power receptacle or power distribution device are switched off.

Plug the power cords into the receptacles at the front of the switch chassis.

Plug each power cord into the facility power receptacle or power distribution device.

Note - To provide redundancy, connect each power cord to a separate power source. The switch

can operate with only one power source, but there is no redundancy in that case.

Caution - Do not apply power at this time.

Consider your next steps.

■ If you have DHCP-assigned IP addresses, install the air duct assembly.

Powering On the Switch 75

Page 76

Install the Air Duct Assembly

See “Install the Air Duct Assembly” on page 76.

■ If you have a full-time SER MGT connection, install the air duct assembly.

See “Install the Air Duct Assembly” on page 76.

■ If your SER MGT connection is only temporary, power on the switch.

See “Power On the Switch” on page 78.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Install the Air Duct Assembly

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Ensure that the power cords and SER MGT cable (if attached) are out of the way and will not interfere with installing the air duct assembly.

Tuck the power cord slack into the opening of the long front mounting bracket. Tuck the SER MGT cable slack into the cavity of the air duct assembly.

Lift the air duct assembly to where it installs on the switch.

76 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 77

Install the Air Duct Assembly

Though the air duct assembly is symmetrical, the logo, if present, goes to the right.

Push the air duct assembly in to the front of the switch, until it snaps into place.

Consider your next steps.

■ If you have DHCP-assigned IP addresses, power on the switch.

See “Power On the Switch” on page 78.

■ If you have a full-time SER MGT connection, power on the switch.

See “Power On the Switch” on page 78.

■ If your SER MGT connection was only temporary, verify the switch

installation.

See “Verify LED Status” on page 87.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

Powering On the Switch 77

Page 78

Power On the Switch

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Power On the Switch

When live power is delivered to the receptacles at the rear of the chassis, standby and main power is made available by the power supplies. The switch immediately comes to full power. Both the SP and SCP begin their boot processes.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Ensure that:

■

The nodes of the IB fabric and hosts of the Ethernet network are operational and securely cabled to the switch.

■

The serial management console is operational, configured with correct parameters, and cabled to the switch. See “Connect a Serial Management Device to the SER MGT

Port” on page 74.

■

The management network is operational and securely cabled to the switch.

■

The facility power source or power distribution device is properly cabled to the switch, available, and standing by.

■

All data, management, and power cabling to the switch is secure at the switch.

Energize the circuit breakers so that facility power is delivered to the switch.

These actions occur in order:

■

All chassis status LEDs illuminate, then go dark.

■

The power supply LEDs indicate input power, then output power.

■

The fans spin up to maximum speed.

Note - Over the next few minutes, the fans gradually decrease in speed.

■

Full power is applied to all circuits and systems inside the switch.

78 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 79

Log In to the SP (SER MGT)

■

Both the SP and SCP are powered on and booting up.

Note - You might see the boot sequence on the serial device.

■

The SP LED slow blinks.

When the SP LED stops blinking and stays lit, and the OK LED lights, it is possible to log in to the SP.

Consider your next steps.

■ If you have DHCP-assigned IP addresses, log in to the SP through the network.

See “Log In to the SP (Network Management)” on page 84.

■ Otherwise, log in to the SP through the SER MGT port.

See “Log In to the SP (SER MGT)” on page 79.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

On the serial management device, press the Return or Enter key several times to synchronize the connection.

The login prompt is displayed.

hostname login:

Powering On the Switch 79

Page 80

Assign a Static IP Address to the SP

where hostname is the host name of the SP. If no host name is assigned, ORACLESP-unknown might be displayed.

hostname login: root Password: changeme

The -> prompt is displayed.

Note - You can change the password at a later time. Refer to the Oracle EDR InfiniBand Switch

and Virtualized I/O Systems Administration Guide, changing the administrator password.

Consider your next task.

■ If you have a full-time SER MGT connection, go to “Verify LED

Status” on page 87 or “Assign a Static IP Address to the SP” on page 80.

■ If your SER MGT connection is only temporary, go to “Assign a Static IP

Address to the SP” on page 80.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Assign a Static IP Address to the SP

The switch ships with DHCP IP address assignment for the SP.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

80 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 81

Access the Oracle ILOM CLI.

See “Log In to the SP (SER MGT)” on page 79.

Configure the network management parameters.

-> set /SP/network property=value property=value ...

where:

■

property is the parameter of the network to configure.

■

value is the value of the property to configure.

These properties are supported:

■

pendingipaddress – value is the IP address of the SP to be configured.

■

pendingipdiscovery – value is the method of IP discovery to be configured, either static or dhcp.

■

pendingipgateway – value is the IP address of the routing gateway to be configured.

■

pendingipnetmask – value is the netmask to be configured.

Assign a Static IP Address to the SP

Note - You can configure one, several, or all properties in one command line.

For example:

-> set /SP/network pendingipdiscovery=static pendingipaddress=123.45.67.89 pendingipgateway=123.45.67.1 pendingipnetmask=255.255.255.0

Set 'pendingipdiscovery' to 'static' Set 'pendingipaddress' to '123.45.67.89' Set 'pendingipgateway' to '123.45.67.1' Set 'pendingipnetmask' to '255.255.255.0'

Commit the changes.

-> set /SP/network commitpending=true Set 'commitpending' to 'true'

The static IP address has been configured to 123.45.67.89. You can now manage the SP from any one of the RJ-45 connectors on the rear panel.

Consider your next steps.

■ Configure a static IP address for the SCP.

See “Assign a Static IP Address to the SCP” on page 82.

■ Log in to the SP through the management network.

Powering On the Switch 81

Page 82

Assign a Static IP Address to the SCP

See “Log In to the SP (Network Management)” on page 84.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SCP” on page 82

■

“Log In to the SP (Network Management)” on page 84

Assign a Static IP Address to the SCP

The switch ships with DHCP IP address assignment for the SCP.

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Access the Oracle ILOM CLI.

See “Log In to the SP (SER MGT)” on page 79 or “Log In to the SP (Network

Management)” on page 84.

Start the host console.

-> start /HOST/console

Are you sure you want to start /HOST/console (y/n)? y Serial console started. To stop, type ESC (

Press the Enter key.

Oracle Linux Server release 6.6 Kernel 4.0.4-15.el6uek.x86_64 on an x86_64 hostname login:

hostname login: root Password:password Last login: Fri May 25 09:51:36 on ttyS0

82 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 83

Assign a Static IP Address to the SCP

[root@hostname ~]#

The default password is changeme.

Set the SCP IP address.

[root@hostname ~]# nm3tool ipaddress SCP_IP_address|dhcp|ilom

where SCP_IP_address is the IP address to be assigned to the SCP. Alternatively, you can configure IP address assignment by DHCP or Oracle ILOM. If you provide a static IP address, you will also need to provide the netmask and the IP address of the management network gateway. For example:

[root@hostname ~]# nm3tool ipaddress 123.45.67.90 Enter netmask: 255.255.255.0 Enter default gateway: 123.45.67.1 Restart network now? (Y/n)

Restart the network to use the new values.

Restart network now? (Y/n)Y Shutting down interface eth0: [ OK ] Shutting down interface eth1: [ OK ] Shutting down loopback interface: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth0: Determining IP information for eth0... done. [ OK ] Bringing up interface eth1: Determining if ip address 169.254.2.1 is already in use for device eth1... [ OK ]

[root@hostname ~]#

(Optional) Set the hostname.

Provide the new host name. For example:

[root@hostname ~]# nm3tool hostname hostname

where hostname is the host name of the SCP. For example:

[root@hostname ~]# nm3tool hostname hostwiththemost [root@hostname ~]#

Apply the changes.

[root@hostname ~]# nm3tool apply Setting network configuration from XML [root@hostname ~]#

10.

Powering On the Switch 83

Page 84

Log In to the SP (Network Management)

See “Log In to the SP (Network Management)” on page 84.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Log In to the SP (Network Management)” on page 84

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Open an SSH session and connect to the SP by specifying the IP address you previously configured.

For example, type:

% ssh -l root IP_address password: password

where IP_address is the IP address of the SP. Initially, the password is changeme.

Note - You can change the password at a later time. Refer to the Oracle EDR InfiniBand

Switch and Virtualized I/O Systems Administration Guide, changing the root user password for

instructions.

The Oracle ILOM shell prompt (->) is displayed.

Consider your next steps.

■ If you have DHCP-assigned addresses, verify the switch installation.

See “Verify LED Status” on page 87.

■ If you have a full-time SER MGT connection, verify the switch installation.

84 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 85

See “Verify LED Status” on page 87.

■ If your SER MGT connection is only temporary, disconnect the SER MGT

cable and install the air duct assembly.

See “Install the Air Duct Assembly” on page 76.

Related Information

■

“Retrieve the Power Cords and SER MGT Cable” on page 67

■

“Connect the SER MGT Cable” on page 71

■

“Connect a Serial Management Device to the SER MGT Port” on page 74

■

“Connect the Power Cords” on page 74

■

“Install the Air Duct Assembly” on page 76

■

“Power On the Switch” on page 78

■

“Log In to the SP (SER MGT)” on page 79

■

“Assign a Static IP Address to the SP” on page 80

■

“Assign a Static IP Address to the SCP” on page 82

Powering On the Switch 85

Page 86

86 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 87

Verifying Functionality

These topics describe how to verify the functionality of the switch.

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

Related Information

■

“Understanding the Switch” on page 11

■

“Confirming Specifications” on page 17

■

“Preparing for Installation” on page 23

■

“Installing the Switch” on page 29

■

“Connecting Cables” on page 49

■

“Powering On the Switch” on page 67

Verify LED Status

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

At the switch, observe the state of the various chassis and component LEDs.

■

Chassis status LEDs

■

Power supply LEDs

■

Fan module LEDs

■

SFP+ receptacle LEDs

Verifying Functionality 87

Page 88

LED Status

■

RJ-45 receptacle LEDs

Note - If the power supply or fan module LEDs are lit, they can be seen through the air duct

assembly.

Compare the state of the LEDs to the status indicated.

See “LED Status” on page 88.

Verify that the switch is operating.

See “Verify That the Switch Is Operational” on page 89.

Related Information

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

LED Status

Glyph Name Color State and Meaning

Chassis Status

Locate White Indicates these conditions:

■ On – No function.

■ Fast blinking – Switch is identifying itself.

■ Off – Disabled.

Attention Amber Indicates these conditions:

■ On – Fault detected.

■ Off – No faults detected.

OK Green Indicates these conditions:

■ On – Switch is functional without fault.

■ Off – Switch is off or initializing.

SP Green Indicates these conditions:

■ On – SP is ready.

88 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 89

Glyph Name Color State and Meaning

■ Slow blinking – SP is booting up.

■ Off – SP is powered off, resetting, or at fault.

Cable Status

Link Green Indicates these conditions:

■ On – Link is up.

■ Intermittent flash – There is activity on the link.

■ Off – Link is down.

Power Supply Status

OK Green Indicates these conditions:

■ On – 12 VDC is supplied.

■ Off – No DC voltage is present.

Attention Amber Indicates these conditions:

■ On – Fault detected, 12 VDC shut down.

■ Off – No faults detected.

AC Green Indicates these conditions:

■ On – AC power present and good.

■ Off – AC power not present.

Fan Module Status

Attention Amber Indicates these conditions:

Verify That the Switch Is Operational

■ On – Fault detected,

■ Off – No faults detected.

Related Information

■

“Verify LED Status” on page 87

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

Verify That the Switch Is Operational

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

Verifying Functionality 89

Page 90

Verify That the Switch Is Operational

Access the Oracle ILOM CLI.

See “Log In to the SP (SER MGT)” on page 79 or “Log In to the SP (Network

Management)” on page 84.

Check the health of the switch.

-> show /System -t health

Target | Property | Value

---------------------------------------------+-----------+-------

If any target has a health value other than OK, investigate why.

Refer to Switch Service, detecting and managing faults.

Identify any open problems.

-> show /System/Open_Problems

Open Problems (0) Date/Time Subsystems Component

------------------------ ------------------ ------------

If any open problems are discovered, investigate the problem.

Refer to Switch Service, detecting and managing faults.

Verify the switch's connectivity to the IB fabric and Ethernet network.

See “Verify Connectivity” on page 91.

Related Information

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

90 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 91

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

Verify Connectivity

Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task.

See “Installation Task Overview” on page 11.

If you are at the switch, check the Link LEDs of the SFP+ and RJ-45 connectors.

See “LED Status” on page 88 for a description of Link LED states.

Access the Oracle ILOM CLI.

See “Log In to the SP (SER MGT)” on page 79 or “Log In to the SP (Network

Management)” on page 84.

Verify Connectivity

Verify Ethernet connectivity.

Start the EMS console.

-> start /System/Networking/Switches/Switch_0/fs_cli Are you sure you want to start /System/Networking/Switches/Switch_0/fs_cli (y/n)? y Management Switch for Oracle InfiniBand Switch IS2-46 SEFOS login:

SEFOS login: root Password: password SEFOS#

The default password is admin123.

Display the state of the Ethernet ports.

SEFOS# show interfaces status

Port Status Duplex Speed Negotiation Capability

---- ------ ------ ----- ----------- ---------Gi0/1 not connected Half - Auto Auto-MDIX on Gi0/2 connected Full 1 Gbps Auto Auto-MDIX on

Verifying Functionality 91

Page 92

Verify Connectivity

Gi0/3 connected Full 1 Gbps Auto Auto-MDIX on Gi0/4 not connected Half - Auto Auto-MDIX on Gi0/5 not connected Half - Auto Auto-MDIX on Gi0/6 connected Full 1 Gbps Auto Auto-MDIX on Gi0/7 not connected Half - Auto Auto-MDIX on ... Gi0/13 connected Full 1 Gbps Auto Auto-MDIX on ... Gi0/23 not connected Half - Auto Auto-MDIX on Gi0/23 connected Full 1 Gbps Auto Auto-MDIX on Gi0/25 connected Full 1 Gbps Auto Auto-MDIX on Gi0/26 connected Full 1 Gbps Auto Auto-MDIX on Ex0/1 not connected Full 10 Gbps No-Negotiation Auto-MDIX on Ex0/2 not connected Full 10 Gbps No-Negotiation Auto-MDIX on SEFOS#

In the output, Port is the port as seen on the rear panel. Ports Gi0/25 and Gi0/26 are internal to the switch architecture. Ports Ex0/1 and Ex0/2 are the two SFP+ uplink ports. The Status is the state of the port. The Duplex, Speed, and Negotiation are the parameters of the link. The Capability indicates if auto-crossover capability is on.

If the status of a known-connected port is anything other than connected, investigate why.

Refer to the EMS module documentation.

Exit the EMS console.

SEFOS# exit Connection closed by foreign host

Entering character mode Escape character is '^]'. start: The session with /System/Networking/Switches/Switch_0/fs_cli has ended.

Verify IB and gateway connectivity.

Start the host console.

-> start /HOST/console Are you sure you want to start /HOST/console (y/n)? y  Serial console started. To stop, type ESC (

Press Enter.

[user@hostname ~]#

where:

92 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 93

Verify Connectivity

■

user is the current username.

■

hostname is the host name of the SCP.

Switch to user admin with password admin.

[user@hostname ~]# su admin Welcome to IBOS Controller Copyright (c) 2012-2015 Oracle Corp. All rights reserved. Enter "help" for information on available commands. Enter the command "show system copyright" for licensing information admin@hostname[OFOS]

Note - From here forward in this document, the admin@hostname[OFOS] prompt is replaced

with simply [OFOS].

If this is the first time Oracle Fabric OS is accessed, the Oracle Fabric OS configuration wizard might start. If this happens, see “Run the Oracle Fabric OS Configuration

Wizard” on page 97.

Display the state of the gateway ports.

[OFOS] show ioports name type state descr v-resources

-----------------------------------------------------------------------------0/1 gwEthernet10GbPort up/indeterminate 0 0/2 gwEthernet10GbPort up/indeterminate 0 0/3 gwEthernet10GbPort up/indeterminate 0 0/4 gwEthernet10GbPort up/indeterminate 0 0/5 gwEthernet10GbPort up/indeterminate 0 0/6 gwEthernet10GbPort up/indeterminate 0 0/7 gwEthernet10GbPort up/indeterminate 0 0/8 gwEthernet10GbPort up/indeterminate 0 8 records displayed [OFOS]

In the output, the name is the port number as seen on the rear panel. The state is of the local port/peer port. The descr is the node descriptor of the peer port. The v-resources are the number of VLANs and VNICs supported on that port.

Display the state of the IB ports.

[OFOS] show infiniband-port name state mode_state guid

-----------------------------------------------------------------------------0/1 up/down switching 0 0/2 up/down switching 0 0/3 up/down switching 0

Verifying Functionality 93

Page 94

Log In to the Oracle Fabric Manager Interface

0/4 up/down switching 0 . . . 0/27 up/down switching 0 0/28 up/down switching 0 0/29 up/down switching 0 0/30 up/down switching 0 30 records displayed [OFOS]

In the output, the name is the port number as seen on the rear panel. The state is of the local port/peer port. The mode_state is the condition of the link. The guid is the GUID of the peer port.

If in the output of these commands a port is identified as being in a down state, or if a link mode_state is other than switching, investigate why.

Refer to the Oracle Fabric OS documentation for assistance.

Exit the host console.

Type exit and press the Enter key, then press the Esc key and type (.

[OFOS] exit [root@nsn156-168 ~]# Serial console stopped.

Administer the switch from Oracle Fabric Manager or Oracle Fabric OS.

See “Log In to the Oracle Fabric Manager Interface” on page 94 or “Log In to the Oracle

Fabric OS Interface” on page 96.

Related Information

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

From a management network attached host, start a web browser.

94 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 95

https://SCP_IP_address:8443/xms

where SCP_IP_address is the IP address of the SCP you configured in “Assign a Static IP

Address to the SCP” on page 82.

The Oracle Fabric Manager login page is displayed.

Type the username and password of the Oracle Fabric Manager administrator.

The default username is root, and the password is changeme.

Select a language from the drop-down menu and click Log In.

The Fabric Manager Dashboard is displayed.

From the Navigation pane, open the Managed Devices folder, and click Discovery Devices.

Find the switch in the list of devices, and highlight it.

Click the check mark icon.

Type the user name and password, and click Submit.

The default username is admin, and the password is admin.

The Discovery Devices pane updates, and the green circle checkmark in the Manage Status column indicates that the switch is enabled for management.

From the Navigation pane, open the General folder, and click Dashboard.

The Navigation pane and Fabric Manager Dashboard reload with additional functionality.

10.

You can now administer the switch through Oracle Fabric Manager.

Refer to the Oracle Fabric Manager documentation for further instructions.

Related Information

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

Verifying Functionality 95

Page 96

Log In to the Oracle Fabric OS Interface

When you log in to the Oracle Fabric OS interface for the very first time, the Oracle Fabric OS configuration wizard starts.

From a management network attached host, start an SSH session.

$ ssh user@SCP_IP_address user@SCP_IP_address's password: password

Last login: Wed Jun 13 22:50:04 2012 from sca-ss-01.us.oracle.com  Welcome to OFOS Controller Copyright (c) 2012-2015 Oracle Corp. All rights reserved. Enter "help" for information on available commands. Enter the command "show system copyright" for licensing information user@hostname[OFOS]

where:

■

user is the username of the Oracle Fabric OS administrator. The default is admin.

■

password is the password of the Oracle Fabric OS administrator. The default is admin.

■

SCP_IP_address is the IP address you configured in “Assign a Static IP Address to the

SCP” on page 82.

■

hostname is the host name you configured in “Assign a Static IP Address to the

SCP” on page 82.

Consider your next steps.

■ If this is the first login to Oracle Fabric OS, the Oracle Fabric OS configuration wizard runs automatically.

See “Run the Oracle Fabric OS Configuration Wizard” on page 97.

■ Otherwise, you can now administer the switch through Oracle Fabric OS.

Refer to the Oracle Fabric OS documentation for further instructions.

Related Information

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

96 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 97

Run the Oracle Fabric OS Configuration Wizard

Caution - Running the Oracle Fabric OS configuration wizard destroys data. If you are

manually running the wizard, consider backing up configuration data before proceeding.

Consider your first step.

■ If the Oracle Fabric OS configuration wizard has started automatically, go to

Step 2.

■ If you want to manually run the Oracle Fabric OS configuration wizard, at the Oracle Fabric OS prompt, type:

[OFOS} system clear config This is a destructive operation. Your configuration will be cleared and the system will be restarted. Please type 'confirm' to clear the configuration and restart the system. > confirm

Review the parameters that the Oracle Fabric OS configuration wizard requests.

See “Oracle Fabric OS Configuration Wizard Parameters” on page 98.

Permit the Oracle Fabric OS configuration wizard to run.

Type Y.

Answer the questions as the wizard requests.

When completed, the wizard reboots Oracle Fabric OS.

See “Log In to the Oracle Fabric OS Interface” on page 96.

Related Information

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Oracle Fabric OS Configuration Wizard Parameters” on page 98

Verifying Functionality 97

Page 98

Oracle Fabric OS Configuration Wizard Parameters

Parameter Value

What type of switch is this? Choose from displayed

Do you want to enable a strong password configuration? Yes/No

If yes, what is the minimum number of characters in the password? 1 - 8

If yes, what is the minimum number of lower case letters in the password? 1 - 8

If yes, what is the minimum number of digits in the password? 1 - 8

If yes, what is the minimum number of special characters in the password? 1 - 8

If yes, what is the minimum number of upper case letters in the password? 1 - 8

What is the password for the admin user?

What is the OSDN MAC address? Choose from displayed

Related Information

■

“Verify LED Status” on page 87

■

“LED Status” on page 88

■

“Verify That the Switch Is Operational” on page 89

■

“Verify Connectivity” on page 91

■

“Log In to the Oracle Fabric Manager Interface” on page 94

■

“Log In to the Oracle Fabric OS Interface” on page 96

■

“Run the Oracle Fabric OS Configuration Wizard” on page 97

Enter the admin password

98 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016

Page 99

Glossary

10GBASE-T –LREthernet technology that supports cable distances of over 300 meters.

10GBASE-T –SRStandard 10GbE Ethernet technology.

10GbE/40GbE module

16G Fibre Channel module

I/O module for the Oracle Fabric Interconnect F2-12 that supports 10GbE and 40GbE interfaces.

I/O module for the Oracle Fabric Interconnect F2-12 that supports Fibre Channel interfaces.

AD Abbreviation for Active Directory.

adapter Physical device, such as an HCA or NIC, that enables a host to communicate through a fabric

or network.

ASR Automatic Service Request. A feature of Oracle hardware that automatically opens service

requests when specific hardware faults occur. ASR is integrated with My Oracle Support (MOS) and requires a support agreement. See also MOS.

CFM Cubic feet per minute. A standard of measuring airflow.

CMB Abbreviation for cable management bracket. A component that supports and groups cables

extending from the chassis.

Glossary 99

Page 100

DN Abbreviation for distinguished name. An Active Directory term.

DSA key Digital Signature Algorithm key. A component of SSH authentication. See also RSA and SSH.

EDR Extended Data Rate. A throughput of InfiniBand (IB) technology, typically 100Gbps. See also

IB.

EMS module Embedded Management Switch module. A component within the switch chassis that provides

an Ethernet switch and management controller, using the SEFOS operating system. See also SEFOS.

Ethernet gateway

Ethernet management switch

A device that either enables data transference from one protocol to another, for example IB to Ethernet. Or a device that serves as a routing node for an Ethernet network.

An Ethernet switch that enables single point management of multiple service processors (SP) or hosts through their respective network management interfaces. See also SP.

fabric card A component of an IB switch chassis that provides a spine topology construct. See also IB

SFM module.

fabric interconnect

fan module A hot-swappable replaceable component that provides cooling air to the chassis interior.

Fibre Channel A data transference technology used primarily for storage systems.

FM Abbreviation of fan module. See also fan module.

Short name for an IB switch that provides gateway services and additional functionality.

GB Abbreviation of GigaByte. 1 GB is approximately 1000 MB.

GbE Abbreviation of Gigabit Ethernet. A unit of throughput for Ethernet technology.

100 Oracle InfiniBand Switch IS2-46 Installation Guide • September 2016