HP AWSXCIG-1A User Manual

HP Workgroup System and XC Software Installation Guide

Version 1.0

HP Part Number: A-WSXCIG-1A Published: August 2008 Edition: 1

Confidential computersoftware. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial

Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under

vendor's standardcommercial license.The informationcontained hereinis subject to change without notice. The only warranties forHP products

and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as

constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

The informationcontained hereinis subject to change without notice. Theonly warranties for HP productsand servicesare set forth in the express

warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP

shall not be liable for technical or editorial errors or omissions contained herein.

UNIX is a registered trademark of The Open Group.

AMD, the AMD Arrow logo and combinations thereof, and AMD Opteron are trademarks of Advanced Micro Devices, Inc.

Intel, Pentium, Xeon, Intel Inside, and the Intel Inside logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the

United States and other countries.

About This Document.........................................................................................................9

Intended Audience.................................................................................................................................9

Document Organization.........................................................................................................................9

Typographic Conventions......................................................................................................................9

Documentation Updates and Release Notes........................................................................................10

HP Encourages Your Comments..........................................................................................................10

1 HP Workgroup System Overview...............................................................................11

1.1 HP Workgroup System Views.........................................................................................................11

2 Hardware Prerequisites...............................................................................................13

2.1 Hardware Preinstallation Checklist................................................................................................13

2.2 Firmware Requirements..................................................................................................................13

3 Hardware Setup...........................................................................................................15

3.1 Unpack the Enclosure......................................................................................................................15

3.2 Installing and Starting Up the Hardware........................................................................................17

3.2.1 Setting IP Addresses................................................................................................................17

3.2.2 Setting Boot Order...................................................................................................................19

3.2.3 Setting Up iLO.........................................................................................................................19

3.2.4 Enabling Telnet Access............................................................................................................20

3.2.5 Setting the Power Regulator....................................................................................................20

3.2.6 Configuring Smart Array........................................................................................................20

4 Software Prerequisites..................................................................................................23

4.1 Software Preinstallation Checklist..................................................................................................23

4.2 Downloading XC Software Patches.................................................................................................24

4.3 Copying the XC.lic File to Your Laptop.......................................................................................24

4.4 Associating the Enclosure DVD to the Head Node........................................................................25

5 XC Software Installation..............................................................................................27

5.1 Booting the DVD.............................................................................................................................27

5.2 Running the cluster_prep Command........................................................................................28

5.3 Installing Patches from Your Laptop...............................................................................................29

5.4 Putting the License Key File in the Correct Location......................................................................30

5.5 Running the discover Command................................................................................................30

5.6 Running the cluster_config Command...................................................................................31

5.7 Running the startsys Command................................................................................................34

5.8 Configuring the SNMP Trap...........................................................................................................34

5.9 LSF Post-Configuration Tasks.........................................................................................................35

5.10 Verifying LSF-HPC with SLURM..................................................................................................35

5.11 Running the OVP to Verify Software and Hardware Components..............................................36

5.12 Nagios Web Interface.....................................................................................................................37

5.13 The nrg Command........................................................................................................................37

5.14 Creating a Baseline Copy of the Database.....................................................................................38

5.15 Creating a Baseline Report of the System Configuration..............................................................38

5.16 Setting Up VLAN..........................................................................................................................39

Table of Contents 3

6 Troubleshooting............................................................................................................41

6.1 Unable to Manually Set IP Addresses for the iLOs.........................................................................41

6.2 Changing External IP Addresses.....................................................................................................41

6.3 Lost Connection to the iLO.............................................................................................................42

6.4 Removing a Bad Golden Image.......................................................................................................42

6.5 Lost Terminal Window When in the IRC........................................................................................43

A Additional Software Setup Information.....................................................................45

B Additional Hardware Setup Information....................................................................47

B.1 HP Workgroup System Specifications............................................................................................47

B.1.1 Thermal Stabilization..............................................................................................................47

C IP Addresses on a Corporate Network.....................................................................49

C.1 Cabling............................................................................................................................................49

C.2 IP Addresses...................................................................................................................................49

D Configure sendmail..................................................................................................51

Glossary............................................................................................................................53

Index.................................................................................................................................57

4 Table of Contents

List of Figures

1-1 Example Front View......................................................................................................................11

1-2 Enclosure Bay Numbering............................................................................................................12

1-3 Example Rear View.......................................................................................................................12

3-1 Opening the Top of the Cardboard Box........................................................................................15

3-2 Removing the Box.........................................................................................................................15

3-3 Removing the Ramp and Front Cushion......................................................................................16

3-4 Attaching the Ramp.......................................................................................................................16

3-5 Rolling the Unit Down the Ramp..................................................................................................17

3-6 Rolling the Unit to the Installation Location.................................................................................17

3-7 Interconnect Switch.......................................................................................................................18

4-1 DVD Drive.....................................................................................................................................25

5-1 IRC Virtual Media Screen..............................................................................................................29

C-1 Interconnect Switch.......................................................................................................................49

C-2 Enclosure Settings Display............................................................................................................49

C-3 OA IP Address...............................................................................................................................50

List of Tables

2-1 HP BladeSystem c-Class Firmware and Management Tools Upgrades.......................................14

B-1 HP Workgroup System Specifications..........................................................................................47

B-2 Thermal Stabilization Specification...............................................................................................47

6 List of Tables

List of Examples

5-1 Sample XC.lic File......................................................................................................................30

5-2 cluster_config Command Output 1.......................................................................................32

5-3 cluster_config Command Output 2.......................................................................................33

6-1 Contents of network-scripts File............................................................................................41

6-2 Modify Database...........................................................................................................................42

6-3 Sample netinfo File....................................................................................................................42

About This Document

This document describes how to set up the hardware and install the XC software on your HP Workgroup System.

IMPORTANT: This document assumes that you are installing the XC Software Version 3.2.1

from a DVD with a valid license key on a new Cluster Platform Workgroup System which consists of a new HP BladeSystem c3000 enclosure that has no external network connection during initial setup and contains the following:

• Eight or less factory-installed server blades with current firmware and no operating system

installed

• A single GbE2c Ethernet switch to which you will connect a Windows laptop to access the

Onboard Administrator (OA)

• An internal DVD drive

Intended Audience

This document is intended for customers and system administrators experienced in the use of high-performance clusters.Certain operations described inthis document, if performed incorrectly, can cause system crashes and loss of data. If you are not familiar with installing and using high-performance clusters, HP recommendsthat youcontact HPGlobal Services.This document is also intended for HP service representatives and other persons trained to install High-Performance Computing Cluster Platform products. Such persons are expected to understand the hazards of working in this environment and to take suitable precautions to minimize danger to themselves and others.

Document Organization

This document is organized as follows:

Chapter 1 Provides an overview of the HP Workgroup System configuration described

in this document, and what you will have when you complete the procedures

described in this document. Chapter 2 Provides hardware preinstallation procedures. Chapter 3 Provides hardware setup proceduresfor unpacking, installing, and starting up. Chapter 4 Provides software preinstallation procedures. Chapter 5 Provides software installation procedures for XC System Software. Chapter 6 Provides troubleshooting information. Appendix A Provides additional software setup information. Appendix B Provides additional information about the hardware setup, such as system

specifications and thermal stablization. Appendix C Provides additional information about how to set up IP addresses on a corporate

network.. Appendix D

Provides information about how to configure the LSF sendmail program.

Typographic Conventions

This document uses the following typographical conventions:

%, $, or #

audit(5) A manpage. The manpage name is audit, and it is located in

A percent sign represents the C shell system prompt. A dollar sign represents the system prompt for the Bourne, Korn, and POSIX shells. A number sign represents the superuser prompt.

Section 5.

Intended Audience 9

Command Computer output

Ctrl+x A key sequence. A sequence such as Ctrl+x indicates that you

ENVIRONMENT VARIABLE The name of an environment variable; for example, PATH. [ERROR NAME] Key The name of a keyboard key. Enter and Return both refer to the

Term The defined use of an important word or phrase.

User input

Variable

[] The contents are optional in syntax. If the contents are a list

{} The contents are required in syntax. If the contents are a list

... The preceding element can be repeated an arbitrary number of

 Indicates the continuation of a code example. | Separates items in a list of choices. WARNING A warning calls attention to important information that if not

CAUTION A caution calls attention to important information that if not

IMPORTANT This alert provides essential information to explain a concept or

NOTE A note contains additional information to emphasize or

A command name or qualified command phrase. Text displayed by the computer.

must hold down the key labeled Ctrl while you press another key or mouse button.

The name of an error, usually returned in the errno variable.

same key.

Commands and other text that you type. The name of a placeholder in a command, function, or other

syntax display that you replace with an actual value.

separated by |, you must choose one of the items.

times.

understood or followed results in personal injury.

understood or followed results in data loss, data corruption, or damage to hardware or software.

to complete a task.

supplement important points of the main text.

Documentation Updates and Release Notes

Documentation updatesand release notes (if applicable)are provided on theHP High Performance Computing documentation website at: http://www.docs.hp.com/en/highperfcomp.html.

Use the release date of a document to determine that you have the latest version.

HP XCSystem Softwaredocumentation is available at: http://www.docs.hp.com/en/linuxhpc.html.

HP Encourages Your Comments

HP encourages your comments concerning this document. We are committed to providing documentation that meets your needs. Send any errors found, suggestions for improvement, or compliments to:

docsfeedback@hp.com

Include the document title, manufacturing part number, and any comment, error found, or suggestion for improvement you have concerning this document.

1 HP Workgroup System Overview

The HP Workgroup System is delivered to you factory assembled and ready for deployment.

HP Workgroup System solutions offer a choice of HP ProLiant c-Class BladeSystem half-height server blades with either Intel® Xeon™ or AMD Opteron™ processors. One server blade in the

configuration is designated as the head node, which you can configure separately from the remaining server blades (compute nodes) in the enclosure. The head node can also be used for preprocessing, postprocessing, and computationalworkload. Computenodes arenormally used for application computation rather than administrative tasks.

NOTE: A hardware configuration can contain a mixture of Opteron and Xeon nodes, but not

Itanium nodes.

The HP Workgroup System is comprised of a single-cabinet cluster containing a variety of components installed to meet the following specifications:

• A single Onboard Administrator (OA) module

• Up to eight half-height server blades

• A shared administration/Gigabit Ethernet network

• SB40 storage blade (optional)

The software option described in this document is HP XC System Software. It is not preinstalled. Installation procedures are included in this document in Chapter 5 (page 27). When you are done, the HP XC System Software will be fully installed, configured, and operational.

1.1 HP Workgroup System Views

The HP Workgroup System ships with the server blades and all of the factory integrated modules installed. Figure 1-1 shows an example front view of the HP Workgroup System.

Figure 1-1 Example Front View

The following list describes the callouts in Figure 1-1:

1. DVD optical drive

2. HP ProLiant BL260c, BL460c, or BL465c half-height server blades (eight)

3. Redundant OA module slot (reserved)

4. HP Insight Display

5. OA module

1.1 HP Workgroup System Views 11

IMPORTANT: Be sure there is a blade in Bay 1. See Figure 1-2 for enclosure bay numbering.

Figure 1-2 Enclosure Bay Numbering

Figure 1-3 shows an example rear view of the HP Workgroup System.

Figure 1-3 Example Rear View

The following list describes the callouts in Figure 1-3:

1. Reserved for future use

2. HP Gigabit Ethernet switch installed in interconnect module bay 1 (IMB1)

3. Active cool fans

4. IMB2 (reserved)

5. Enclosure downlink

6. Enclosure uplink and service port

7. iLO/OA port 1

8. Reserved for future use

9. Power supplies

10. Optional 4X DDR InfiniBand Interconnect installed in IMB3 and IMB4 (not described in this

document)

12 HP Workgroup System Overview

2 Hardware Prerequisites

Gather the following information before starting the installation process. HP recommend that you download all device drivers, firmware, and documentation to your laptop before you begin the software installation process.

2.1 Hardware Preinstallation Checklist

You need the following items to install your hardware:

• 2 RJ45 CAT5 patch cables

• A laptop connected to the switch

• 110 power cord with C13/C14 plug

• A box cutter or pocket knife

• Gloves

• Protective eye wear

• Hardware Documentation

— HP XC System Software Hardware Preparation Guide at:

http://docs.hp.com/en/A-XCHWP-321/A-XCHWP-321.pdf

— All product manuals for HP BladeSystem c3000 Enclosures — All product manuals for HP BladeSystem c-Class Onboard Administrator — HP Cluster Platform Workgroup System and Cluster Platform Express Overview and Hardware

Installation Guide at:

http://www.docs.hp.com/en/A-CPCPE-1D/A-CPCPE-1D-121707.pdf

— HP Cluster Platform Workgroup System Tower Hardware Installation Guide at:

http://www.docs.hp.com/en/A-CPWST-1B/A-CPWST-1B.pdf

— HP Cluster Platform Site Preparation Guide at:

http://www.docs.hp.com/en/A-CPSPG-1D/A-CPSPG-1D.pdf.

— HP BladeSystem c3000 QuickSpecs at:

http://h18004.www1.hp.com/products/quickspecs/12790_div/12790_div.html

— HP BladeSystem c3000 Enclosure Specifications at:

http://h18004.www1.hp.com/products/blades/components/enclosures/c-class/c3000/ specifications.html

— HP BladeSystem c3000 Enclosure and c3000 Tower Enclosure Maintenance and Service Guide

at:

All product manuals for HP BladeSystem c3000 Enclosures

— HP BladeSystem c3000 Enclosure Setup and Installation Guide available online from a link

at:

All product manuals for HP BladeSystem c3000 Enclosures

— HP XC Systems with HP Server Blades and Enclosures HowTo at:

http://www.docs.hp.com/en/10077/blade_howto.pdf

2.2 Firmware Requirements

HP installs the current version of firmware on Cluster Platform hardware components before shipping your order. To verify the firmware version for a component, see the following documents:

2.1 Hardware Preinstallation Checklist 13

• For information on displaying the installed version of firmware, see the service guide for

the hardware component.

• For the supported firmware versions, see the HP XC System Software: Master Firmware Tables

Version 3.2.1 available online at:

http://docs.hp.com/en/A-XCFM3-21A/index.html

To upgrade firmware, follow these steps:

1. Go to the web address specified in Table 2-1 to download the firmware.

2. For information on how to upgrade firmware, see the service guide for the hardware

component.

CAUTION: . Before installing or upgrading firmware on HP XC System components, see HP

XC System Software: Master Firmware Tables Version 3.2.1. Important guidelines and firmware

versions that are known to be incompatible are detailed in the document. Upgrading firmware to a version other than those specified in HP XC System Software: Master Firmware Tables Version

3.2.1 can cause problems.

Table 2-1 HP BladeSystem c-Class Firmware and Management Tools Upgrades

Web AddressComponent

HP ProLiant Server Blades

BL260c

BL460c

BL465c

HP c-Class BladeSystem Enclosure Components and Switches

Onboard Administrator (OA)

HP ProLiant Integrated Lights Out 2 (iLO

Server Blade mezzanine cards (if applicable)

HP BladeSystem Integrated Manager

HP Systems Insight Manager

HP Insight Control Environment for BladeSystem

HP Insight Control Linux Edition

GbE2c Ethernet Switch

c-Class Firmware and Upgrades. Click Server Blade Firmware and

Drivers.

c-Class Firmware and Upgrades. Click each of theapplicable downloads.

GbE2c Ethernet Blade Switch. Click Software & Drivers, and search for

your operating system.

14 Hardware Prerequisites

3 Hardware Setup

For more information about the HP Workgroup System specifications and thermal stabilization, see Appendix B (page 47).

3.1 Unpack the Enclosure

To unpack and remove the HP Workgroup System from a pallet, follow these steps:

1. Move the pallet to the installation location and leave several feet of space to move around the pallet.

NOTE: Carton graphics provide instructions on how to unpack the HP Workgroup System

and to open the top of the box first.

2. Open the top of the cardboard box. (Callout 1, Figure 3-1)

Figure 3-1 Opening the Top of the Cardboard Box

3. Remove the two boxes containing the security bezel, power cords, and other accessories. (Callout 2, Figure 3-1)

4. Remove the cushions from the top of the unit (Callout 3, Figure 3-1) and the cardboard box. (Callout 1, Figure 3-2)

Figure 3-2 Removing the Box

3.1 Unpack the Enclosure 15

5. Remove the front section of the bottom cushion (Callout 2) and the ramp. (callout 1,

Figure 3-3)

Figure 3-3 Removing the Ramp and Front Cushion

6. Attach the ramp to the plywood deck using the hook-and-loop. (Callout 1, Figure 3-4)

NOTE: Box clamps must be folded out of the way to lay the ramp down.

Figure 3-4 Attaching the Ramp

WARNING! The HP Workgroup System weighs 300+ pounds. To avoid injury, you might

require a second person to assist you with this step.

7. Gently roll the unit down the ramp. (Callout 1, Figure 3-5)

16 Hardware Setup

Figure 3-5 Rolling the Unit Down the Ramp

Figure 3-6 Rolling the Unit to the Installation Location

8. Plug the unit into a power source.

3.2 Installing and Starting Up the Hardware

Familiarize yourself with the back of the c3000 enclosure and identify the OA interface port and the HP GbE2c Ethernet switch. See Figure 1-3 (page 12). Verify that there is a blade in bay 1. See

Figure 1-2 (page 12) to identify bay numbers. For information about how to move blades in the

enclosure, see the HP BladeSystem c3000 Tower Enclosure Setup and Installation Guide .

http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01310294/c01310294.pdf

3.2.1 Setting IP Addresses

Before setting the IP addresses, verify that your c3000 is not connected to the corporate network. Make sure you have downloaded the information recommended in Chapter 2 (page 13).

1. Connect a cable from the OA to the GbE2c switch port 23. See Figure 3-7 callout 4. (For the exact location of the interconnect switch, see Figure 1-3 (page 12) Callout 2.)

3.2 Installing and Starting Up the Hardware 17

Figure 3-7 Interconnect Switch

The following list describes the callouts in Figure C-1:

1. Port 20 RJ-45 connector for 10/100/1000 Mb uplink

2. Port 21 RJ-45 connector for 10/100/1000 Mb uplink

3. Port 22 RJ-45 connector for 10/100/1000 Mb uplink

4. Port 23 RJ-45 connector for 10/100/1000 Mb uplink

5. Port 24 RJ-45 connector for 10/100/1000 Mb uplink

2. Connect a cable from your laptop to the switch port 21. Callout 2 Figure C-1

3. Verify that no external network is connected to the switch.

4. Use the menus on the Insight Display panel to manually set the static IP address and subnet mask for the OA.

• IP address of the OA: 172.31.32.1

• IP address of the installation PC: 172.31.32.20

• Netmask for the PC: 255.255.0.0

• Starting IP address for enclosure bay IP addressing: 172.31.15.1 (this uses the addresses

from 172.31.15.1 to 172.31.15.16)

5. On your laptop or PC, manually set a static IP address for the NIC in the same subnet as the IP address you set for the OA.

a. On your PC, go to Start → Control Panel → Network Connections. b. Right-click Local Area Connection and choose Internet Protocol (TCP/IP). c. Click Properties and choose Use the Following IP addresses and enter the IP

addresses listed below. Remember to reset your laptop after installation if you need to access the corporate network.

6. From your PC or laptop, use the browser to access the OA using the following address:

https://172.31.32.1

7. Log in to the OA. Use the default user name Administrator and the default password shown on the tag affixed to the OA.

8. Create a new user name and password. For example, you can make the User name: Root, and the Password: Test1234 as follows:

a. Click the plus sign (+) to open the User → Authentication menu in the left frame. b. Select the Local Users and click New. c. Add local user information including a password and confirmation. d. Change privilege level to Administratorand make sure thatthe User Enabled is checked. e. Check Onboard Administrator Bays under User Permissions. Then click the Update

f. Test by signing out of the OA and signing in with the new account information.

9. Assign IP addresses to all iLO2 devices in the enclosure:

a. Click the plus sign (+) to open the Enclosure Settings menu in the left frame. b. Select the Enclosure Bay IP Addressing menu item. c. Select the check box to Enable Enclosure Bay IP Addressing.

18 Hardware Setup

User button.

+ 42 hidden pages