HP Fabric Clustering System Installation Manual

Download

Page 1

AT083A HP Fabric Clustering System PCIe 2-port 4X QDR Fabric Adapter Installation Guide

HP-UX Integrity Server Networking

HP Part Number: AT083-9000A_ed2 Published: February 2011 Edition: 2.0

Page 2

Legal Notices

The information in this document is subject to change without notice.

Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness

for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential

damages in connection with the furnishing, performance, or use of this material.

Warranty A copy of the specific warranty terms applicable to your Hewlett-Packard product and replacement parts can be obtained from your

local Sales and Service Office.

U.S. Government License Proprietary computer software. Valid license from HP required for possession, use or copying. Consistent with FAR

12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed

to the U.S. Government under vendor's standard commercial license.

Trademark Notices UNIX® is a registered trademark in the United States and other countries, licensed exclusively through The Open Group.

Feedback HP welcomes your input. Please email us at: netinfo_feedback@cup.hp.com with comments or suggestions on HP I/O Cards or

1 HP PCIe 2-port 4X QDR Fabric Adapter Installation

This document gives installation information for the HP Fabric Clustering System Host Channel Adapter (HCA) for InfiniBand™, AT083APCIe Adapter with 2 (two) 4X copper or fiber interfaces.

NOTE: The AT083A card is NOT supported on PA systems.

The AT083A product is for use in high-performance technical computing clusters and in database clusters.

Thank you for purchasing HP I/O Products If you are installing a Host Channel Adapter as an add-in device, please review this document before installation.

All product documentation, including an installation guide, release notes, as well as information on supported servers, HP-UX versions, drivers, and patches, is available online at:

http://www.hp.com/go/hpux-iocards-docs and select HP Fabric Clustering System for InfiniBand.

HP Fabric Clustering System Installation Prerequisites

NOTE: The latest copy of this document can be found at:

http://www.hp.com/go/hpux-iocards-docs within the HP Fabric Clustering System for InfiniBand

area.

Before installing the HP Fabric Clustering System, ensure that the following hardware and software prerequisites are met:

• HP requires all systems in the fabric to use both the same HP-UX OS software version, and the same HP-UX Fabric Clustering System for InfiniBand software version, such as version B.11.31.1103.

• HP requires all systems in the fabric to use the same bundle, IB4X-00.

• The AT083A card has been certified on 36 port QDR switch with firmware version 7.2.0.

• Check the IB4X-00 Driver for InfiniBand Release Notes for known problems, required patches, or other information needed for installation, available at:

http://www.hp.com/go/hpux-iocards-docs in the HP Fabric Clustering System for InfiniBand

section.

The latest switch firmware and is available online at:

http://support.hp.com

1. Select Software and Driver downloads.

2. Enter your product name.

3. Click the double arrow.

• Confirm that the HP Fabric adapter and software is supported on the HP server and IO slot chosen; see HP Fabric Clustering System Support Matrix, available at:

http://www.hp.com/go/hpux-iocards-docs in the HP Fabric Clustering System for InfiniBand

section.

• Each AT083A Host Channel Adapter card requires 512 MB host memory in supported servers.

• Determine where the switch is to be installed and which servers are to have an HCA installed.

• To use the Internet Protocol over InfiniBand (IPoIB) protocol, see “Internet Protocol over

InfiniBand (IPoIB)” (page 8).

• The installed version of IP Filter must be higher than “B9901AA A.03.05.06 HP IPFilter

3.5alpha5”.

HP Fabric Clustering System Installation Prerequisites 5

Page 6

• Check the cluster configuration plan for information on where to install adapters, switches, and cable guides, and where cables are to be routed.

• Check your server's documentation to see if additional tools may be required for component installation.

• For specific instructions on adapter installation, see system-specific documentation on "installing networking adapters" for each server that adapters will be installed into.

• Confirm that you have the required cables to make connections in the fabric (adapter to adapter, adapter to switch, or switch to switch).

• Confirm that HP Fabric Clustering System software suite software media is available. It is included on CD or available on the Web through:

http://www.software.hp.com

• Confirm that HP-UX super-user privileges are available; they are necessary to complete the installation.

• Confirm that the /usr/bin, /usr/sbin, /sbin, and /opt/rdma/bin directories are in your PATH by logging in as root and using the echo $PATH command.

• Confirm that the HP-UX operating system is the correct version, HP-UX 11i v3. Use the uname -a command to determine the HP-UX version of your system. For more information about the required operating system versions, see IB4X-00 Driver for InfiniBand Release Notes.

Information Available on the Host Channel Adapter If you are collecting information from the adapter before installation for inventory purposes, there is a GUID number on the card. This number can also be obtained through the HP-UX Fabric Clustering System utilities, but if recording information for inventory, include the GUID number as well, as it is unique for each adapter. The GUID will have a format such as: 00:16:35:FF:FF:BF:09:01 .

Install HP Fabric Clustering System Adapters

CAUTION: Adapters contain electronic components that can easily be damaged by small amounts of electricity. To avoid damage, follow these guidelines:

• Store adapters in their anti-static plastic bags until installation.

• Work in a static-free area, if possible.

• Handle adapters by the edges only. Do not touch electronic components or electrical traces.

• Use a grounding wrist strap.

• Use a suitable ground—any exposed metal surface on the computer chassis.

Installing the AT083A Host Channel Adapter

The adapter comes pre-configured. You do not have to set any jumpers or connectors.

The firmware for the HCA is compiled into the driver and the driver will download the current version onto the HCA if an older version is on the card.

1. Access the system card bay a. If the system is running, shut it down by executing: shutdown -h. b. Respond "y" to the continue to shutdown prompt. c. Wait for the system to shut down completely, and then power off the system by pressing

the system off button. Ensure that the system is grounded. d. Open the system to gain access to the PCIe backplane. e. Insert the card into the highest performance PCIe slot available.

6 HP PCIe 2-port 4X QDR Fabric Adapter Installation

Page 7

2. Install the card a. Slide the adapter edge-connector into the PCIe slot until it is fully seated. b. Secure the card and reassemble the system.

Figure 1-1 Inserting the Host Channel Adapter

PCIe slot4X IB Interfaces

Install HP Fabric Clustering System Software

This section describes the steps necessary to install HP Fabric Clustering System software. The same software version must be installed on each instance of the HP-UX operating system in the fabric.

1. Log on to the system as root.

2. Insert the software media into the appropriate drive. If the software is being loaded from a CD-ROM, go to Step 3; otherwise, go to Step 4. The software can also be located at:

http://www.software.hp.com

Locating the AT083A HCA Software on software.hp.com The drivers, utilities and manpages for the AT083A HCA are on the HP-UX application release CD. They are also available at the HP Software Depot. To locate the software at the HP Software Depot:

a. Go to http://www.software.hp.com b. Search for "AT083A" c. Click on "Receive for Free" d. Fill out the product registration form and click on "next" e. Look for "Download Software" and click on the depot that corresponds to your HP-UX

operating system version to download the drivers, utilities, and manpages for the AT083A adapter.

f. In the "Documents" column next to the "Download Software" column, click on

"Installation Instructions" to download instructions for using the Software Distributor tool to install the drivers, utilities, and manpages for the AT083A adapter.

3. Mount the CD-ROM drive by using the following command at the command prompt:

$ mount device_name

where device_name is the name assigned to the CD-ROM drive.

4. Run the swinstall program using the following command:

$ /usr/sbin/swinstall

This opens the Software Selection window.

Install HP Fabric Clustering System Software 7

Page 8

5. Change the Source Host Name, if necessary, and then enter the mount point of the drive in

the Source Depot Path field. Select the OK button to return to the Software Selection window.

6. The Software Selection window now contains a list of available software to install.

7. Highlight the HP Fabric for InfiniBand software and choose Mark for Install from the Actions

menu; this chooses the highlighted software.

You can also install the IPoIB software at this time as it is part of this same bundle to enable it to be functional. It can also be installed later if you choose to not install it now.

8. From the Actions menu, select the Install... menu, and then choose Install. This begins product installation and opens the Install Analysis window.

9. Select the OK button in the Install Analysis window when the Status field displays a Ready message.

10. Select the YES button in the Confirmation window to start software installation.

swinstall loads the fileset, runs the control script for the filesets, and builds the kernel. When the processing is finished, the Status field displays a Ready message. Select Done and then the Note window opens.

11. Select the OK button in the Note window to reboot. The user interface disappears and the system reboots.

When the system comes back up, log on to the system as root and view the /var/adm/sw/ swagent.log and /var/adm/sw/swinstall.log files to view error or warning messages that may have occurred during the installation.

12. Verify that all installed fabric adapters have a software state of CLAIMED, by running the ioscan -fk -C ib command.

Internet Protocol over InfiniBand (IPoIB)

In order to use the Internet Protocol over InfiniBand™ (IPoIB) the same patches need to be installed on each server used for IPoIB in the HP Fabric Clustering System. For current patch requirement , see the latest HP Fabric Clustering System for InfiniBand Support Matrix at:

http://www.hp.com/go/hpux-iocards-docs

The ifconfig command provides information about the IPoIB interface, enables or disables an IPoIB interface, configuring the link to the HP Fabric Clustering System.

Checking the IPoIB Interface Status To check the current status of the IPoIB interface.

Syntax and example: ifconfig lan9000

Output Example:

lan9000: flags=842<BROADCAST,RUNNING,MULTICAST> inet 192.32.32.255 netmask ffffff00 broadcast 192.32.32.255

Enable the IPoIB Interface To enable the IPoIB interface.

Syntax: ifconfig <ipoib_interface> [address] [netmask] up

Example: ifconfig lan9000 192.32.32.255 up

There is no direct output from this command.

To confirm that the interface is UP, run the ifconfig lan9000 command again.

Output Example:

lan9000: flags=843<UP,BROADCAST,RUNNING,MULTICAST> inet 192.32.32.255 netmask ffffff00 broadcast 192.32.32.255

Regulatory Information

Regulatory certifications for this product were filed using device part number MHQH29B-XTR.

8 HP PCIe 2-port 4X QDR Fabric Adapter Installation

Page 9

FCC RFI Statement

This device complies with part 15 of the FCC Rules for Class A Apparatus. Operation is subject to the following two conditions: (1) This device may not cause harmful interference, and (2) This device must accept any interference received, including interference that may cause undesired operation.

USA Radio Frequency Interference

The Federal Communications Commission (in 47 CFR Part 15 subpart B) has specified that the following notice be brought to the attention of the users of this product. NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense.

Canada RFI Statement

This Class A digital apparatus complies with Canadian ICES-003.

European Union RFI Statement

This is a Class A product. In a domestic environment, this product may cause radio interference in which case the user may be required to take adequate measures.

Korea RFI Statement

This equipment has been tested to comply with the limits for a Class A digital device. This equipment should be operated in a commercial environment. Please exchange if you purchased this equipment for noncommercial purposes.

Australia C-Tick Label

Regulatory Information 9