Copyright Hewlett-Packard Company 1999. All Rights Reserved.
Reproduction, adaptation, or translation with out prior wr itten
permission is prohibited, except as allowed under the copyright laws.
The infor mat i on co n ta i ned i n th is do cument is s u bj ec t to c h ang e wit h out
notice.
Hewlett-Packard make s no warranty of any kind with regard to this
material, including, but not limited to, the implied warranties of
merchantability and fitness for a particular purpose. Hewlett-Packard
shall not be liable for errors contained herein or for incidental or
consequential damages in connection with the furnishing, performance
or use of this mater ial.
The Operator’s Guide HP 9000 V2500/V2600 Server documents the
information necessary to operate and monitor HP V-Class servers. This
book is intended to be a reference for system administrators, system
operators, and system managers.
Prefacexiii
Preface
Notational conventions
This section describes notational conventions used in this book.
bold monospace In command examples, bold monospace
identifies input that must be typed exactly as
shown.
monospaceIn paragraph text, monospace identifies
command names, system calls, and data
structures and types.
In command examples, monospace identifies
command output, including error messages.
italicIn paragraph text, italic identifies titles of
documents.
In command syntax diagrams, italic iden tifi es
variables that you must provide.
The following command example uses
brackets to indicate that the variable
output_file is optional:
commandinput_file [output_file]
In command syntax diagrams, text
surrounded by curly brackets indicates a
choice. The choices available are shown inside
the curly brackets and separated by the pipe
sign (|).
The following command example indicates
that you can enter either a or b:
command {a | b}
Preface
Horizontal ellipses
(...)
In command examples, horizontal ellipses
show repetition of the preceding items.
Vertical ellipsesVertical ellipses show that lines of code have
been left out of an example.
KeycapKeycap indicates the keyboard keys you must
press to execute the command example.
NOTEA note highlights important supplemental information.
CAUTIONCautions highlight procedures or information necessary to avoid injury
to personnel. The caution should tell the reader exactly what will result
from what actions and how to avoid them.
WARNINGA warning highlights procedures or information necessary to avoid
damage to equipment, damage to software, loss of data, or invalid test
results.
Prefacexv
Preface
Safety and regulatory information
Fo r your pro te ction, th is prod uct has b ee n te sted t o vari ous na tio nal an d
international regulations and standards. The scope of this regulatory
testing includes electrical/mechanical safety, radio frequency
interference, ergonomics, acoustics, and hazardous materials. Where
required, approvals obtained from third-party test agencies are shown on
the product label.
Safety in material handling
CAUTIONDo not lift the node manually. To avoid physical injury you must use a
mechanical lifting device.
USA radio frequency interference FCC Notice
The F ederal Communications Commission ( in CFR P art 15) has specif ied
that the following notice be brought to the attention of the users of this
product.
NOTEThis equipment has been tested and found to comply with the limits for a
Class A digital device, pursuant to Part 15 of the FCC Rules. These limits
are designed to provide reasonable protection against harmful interference
when the equipment is operated in a commercial environment. This
equipment generates, uses, and can radiate radio frequency energy and, if
not installed and used in accordance with the instruction manual, may cause
harmful interference to radio communications. Operation of this equipment
in a residential area is likely to cause harmful interference in which case the
user will be required to correct the interference at his own expense.
The user is cautioned that changes or modifications not expressly
approved by Hewlett-Packard could result in the equipment being
noncompliant with FCC Class A requirements and void the user’s
authority to operat e the equipment.
xviPreface
Japanese radio frequency interference VCCI
Figure 1Japanese radio frequency notice
This equipment is a Class A category (Information Technology
Equipment to be used in commercial and /or industrial areas) and
conforms to the standards set by the Voluntary Control Council for
Interference by Information Technology Equipment aimed at preventing
radio interference in commercial and/or industrial areas.
Preface
Consequently, when used in a residential area or in an adjacent area
thereto, radio interference may be caused to radios and TV receivers , etc .
Read the instructions for correct handling.
EMI statement (European Union only)
This is a Class A product. In a domestic environment this product may
cause radio interference in which case the user may be required to take
adequate measures.
Digital apparatus statement (Canada)
This Class A digital apparatus meets all requirements of the Canadian
Interference-Causing Equipment Regulations.
Cet appareil numérique de la classe A respecte toutes les exigences du
Règlement sur le matériel brouilleur du Canada.
BCIQ (Taiwan)
This product has been reviewed, evaluated by GesTek Taiwan and is
fully compliant to CNS 13438 (CISPR 22: 1993) Class A.
Prefacexvii
Preface
Figure 2BCIQ (Taiwan)
Acoustics (Germany)
Laermangabe (Schalldruckpregel LpA) gemessen am fiktiver
Arbeitsplatz bei normalem Betrieb nach DIN 45635, Teil 19: LpA =65.3
dB.
Acoustic Noise (A-weighted Sound Pressure Level LpA) measured at the
bystander position, normal operation, to ISO 7779: LpA = 65.3 dB.
3862H354
IT power system
This product has not been evaluated for connection to an IT power
system (an AC distribution system having no direct connection to earth
according to IEC 950).
High leakage current
CAUTIONHigh leakage current. Ground (earth) connection essential before
connecting the supply.
AttentionForts courants de peretes. Connection a une borne de terre est
essentielle avant tout raccord electrique.
AchtungHoher ableitstrom. Vor inbetreiebnahme schutzleiterverbindung
herstellen.
xviiiPreface
Installation conditions (U.S.)
See installation instructions before connecting to the supply.
Voir la notice d’installation avant de raccorder au réseau.
CAUTIONPlease note the following conditions of installation:
An insulated earthing conductor that is identical in size, insulation
material, and thickness to the earthed and unearthed branch-circuit
supply conductors except that it is green with or without one or more
yellow stripes is to be installed as part of the branch circuit that
supplies the unit or system. The earthing conductor described is to be
connected to earth that the service equipment or , if supplied by a
separately derived system, at the supply transformer or motorgenerator set.
The attachment-plug receptacles in the vicinity of the unit or system
are all to be of an earthing type, and the earthing conductors serving
these receptacles are to be connected to earth at the service
equipment.
Preface
CAUTIONFor supply connections, use wires suitable for at least 60 °C.
Utillser des fils convenant à une température de 60 °C pour les
connexions d’allmenation.
Fuse cautio ns
CAUTIONDisconnect power before changing fuse.
AttentionCoupier le courant avant de remplacer le fusible.
CAUTIONFor continued protection against risk of fire, replace fuses only with
same type and rating.
AttentionPour ne pas compromettre la protection contre les risques d’incendle,
remplacer par un fusible de même type et de mêmes caractéristiques
nominales.
If you have questions that are not answered in this book, contact the
Hewlett-Packard Response Center at the following locations:
•Within the continental U.S., call 1 (800) 633-3600.
•All others, contact your local Hewlett-Packard Response Center or
sales office for assistance.
Prefacexxi
Preface
Reader feedback
This document was produced by the System Supportability Lab Field
Engineering Support organization (SSL/FES). If you have editorial
suggestions or recommended improvements for this document, please
write to us.
Please report any technical inaccuracies immediately.
You can reach us through email at:
fes_feedback@rsn.hp.com
Please include the following information with your email:
•Title and part number of the document
•Edition number
xxiiPreface
1Overview
This chapter introduces Hewlett-Packard V-Class system components
and includes a brief overview of V2500/V2600 server hardware resources.
Some basic details about HP-UX use also are provided. For details on the
external cabinet controls and displays, see Chapter 2.
The V2500/V2600 model of V-Class server can have up to 128 processors,
128 Gbytes of memory, and 112 PCI I/O cards.
One new feature of the HP V2500/V2600 server is its Scalable
Computing Architecture (SCA) design, which allows multiple V2500/
V2600 cabinets to be connected to form a single HP-UX system. These
SCA features are made available through HP’s Coherent Toroidal
Interconnect (CTI) technology.
A V2500/V2600 server can include from one to four cabinets that contain
the server resources, with each V2500/V2600 cabinet containing from
two to 32 process ors , from 512 Mbyt es to 32 Gbyt es o f memory, an d up to
28 PCI I/O cards.
Each V-Class system also includes a dedicated workstation connected to
the server: the Servi ce Support Processor ( SSP workstati on). The Serv ice
Support Processor is used for server booting, monitoring, and other
operations. Details on using the Service Support Processor are provided
in Chapter 3.
This book covers both single-cabinet and multiple-cabinet server
configurations, support, and operations.
Chapter 11
Overview
V-Class System Components
V-Class System Components
Each V-Class system includes two main components: a V-Class server
and a Service Support Processor (SSP workstation) dedicated to
supporting the server, as shown below in Figure 3.
Figure 3V-Class Server Components: Cabinet
and Service Support Processor
C
O
N
D
S
C
E
O
N
O
L
A
F
E
B
F
L
E
C
O
N
S
S
L
E
O
C
D
L
U
E
C
R
O
E
N
T
O
C
V25U075
10/13/98
The V-Class cabinet contains all V-Class server resources, such as
processors, memory, disks, power, and so forth. The Service Support
Processor has software that allows you to monitor the resources in a VClass cabinet. The V-Class server and the Service Support Processor run
separate instances of the HP-UX operating system.
Multiple-cabinet servers may contain up to four V2500/V2600 cabinets,
which are booted as a single HP-UX system. Each cabinet has its own
cabinet ID (0, 2, 4, or 6) and contains processors, memory, and I/O
resources that are available to HP-UX and the applications that run on
the server. Cabinets are numbered based on their location in the server.
Cabinet ID 0 is the “monarch” or “root” cabinet, which contains the I/O
device used for booting and volume group 0. The other cabinets (IDs 2, 4,
and 6) are “serf” cabinets, located as shown in Figure 5 on page 5.
2Chapter 1
V-Class System Components
Figure 4 shows a four-cabinet V2500/V2600 server and the Service
Support Processor that is used for console, diagnostic, and other support
work. The V2500/V2600 cabinets are tightly interconnected by Coherent
Toroidal Interconnect (CTI) cables, as described in “Multiple-Cabinet
Server Connections” on p age 15. Connections among the Service Support
Processor and V2500/V2600 cabinets are covered in “Server Console and
Diagnostic Connections” on page 4.
Figure 4Four-Cabinet V2500/V2600 Server Components
CO
NS
D
C O
E
O
NABLE
LE
FF
C
O
NSLOLE
SEC
DC
UR
O
E
N
TOC
Overview
C
O
NSO
DC
EN
O
LE
AB
F
F
LE
C
O
NS
SE
LO
CU
D
LE
C
RE
O
N
TO
C
V25U074
10/6/98
The Service Support Processor
The Service Support Processor (SSP workstation) is an HP 712 or B180
workstation connected to the V-Class server. Key operations supported
by the Service Support Processor include booting, configuring, and
Chapter 13
Overview
V-Class System Components
monitoring the server hardware, as well as diagnostics operations. You
also must use the Service Support Processor when installing or
upgradin g V-Class firmware.
The Service Support Processor runs HP-UX V10.20. In addition to HPUX software, the Service Support Processor includes files and utility
software for managing and monitoring the V2500/V2600 server. These
items and all other V2500/V2600-related files, including log files, that
are stored on the S ervice Su pport P rocessor can be found in the dire ctory
/spp.
The default user account for Service Support Processor operations is
sppuser, with a home directory of /users/sppuser.
NOTEThe abbreviation “spp” stands for “scalable parallel processor” and is not
to be confused with “SSP”.
See Chapter 3 for more detailed information on the Service Support
Processor.
Server Console and Diagnostic Connections
The V2500/V2600 server’s utilities board provides connections from the
Service Support Processor to a V2500/V2600 server’s cabinet or cabinets.
Both the console port and diagnostic LAN on each cabinet are connected
to the Service Support Processor for system monitoring, booting, and
other operati on s.
The Service Support Processor connections to a V2500/V2600 server
provide only console, diagnostics, and preliminary booting support. For
multiple-cabinet servers, th e CTI cables between cabinets provide the
multiple-cabinet interconnections that create a single, unified V2500/
V2600 HP-UX system. Cross-cabinet connections are covered in the
section “Multiple-Cabinet Server Connections” on page 15.
A single-cabinet V2500/V2600 server is connected directly to the Service
Support Processor, as shown in Figure 6 on page 7.
As shown in Figure 5, multiple-cabinet V2500/V2600 servers have
connections from each cabinet’s utilities board to either the Service
Support Processor or a terminal server.
While a four-cabinet V2500/V2600 server configuration is shown in
Figure 5, a two-cabinet or three-cabinet configuration involves the same
type of set up among the Service Support Processor, V2500/V2600
cabinets, and the terminal server.
4Chapter 1
Figure 5Console and Diagnostic Connections
for a Four-Cabinet V2500/V2600 Server
26
Util.Util.
Overview
V-Class System Components
4
Util.
SSP Workstation
0
Util.
(diagnostic LAN)
012
Term. Server
(console)
The console port on cabinet ID 0’s utilities board connects to the Service
Support Processor, and console ports on cabinet IDs 2, 4, and 6 connect to
the terminal server (port numbers 2, 3, and 4, respectively).
The diagnostic LAN connects between, and is terminated at, the Service
Support Processor and the terminal server. Between these two points,
the diagnostic LAN runs in sequence to cabinet IDs 0, 2, 4, and 6.
Chapter 15
Overview
V-Class Server Architecture
V-Class Server Architecture
The V2500/V2600 server has a powerful set of interconnecting hardware
components that allow the server’s processors, memory, and I/O
components to operate with minimal interruptions or contentions for
resources.
The processor agents serve as a bus connection for a subset of the
system’s proces sors. Memo ry controllers prov ide cache- coherent access to
a large, shared memory. PCI controllers are the connections for PCI I/O
cards.
CTI controllers are an SCA feature used only in multiple-cabinet servers.
The CTI controllers are connected to memory controllers and provide
high-bandwidth connections to other cabinets that comprise the server.
See Figure 11 on page 16 for an overview of cross- cabinet CTI
connections.
V2500/V2600 Crossbar Interconnection
The primary interconnecting component of each V2500/V2600 server
cabinet is the HyperPlane Crossbar, which provides connections from
processors and I/O to memory.
The V2500/V2600 crossbar is a non-blocking 8x8 crossbar, which
supports eight send messages and eight receive messages
simultaneously. This crossbar provides a central connection among the
processor agents, memory controllers, and PCI controllers within a
V2500/V2600 cabinet. On multiple-cabinet V2500/V2600 servers highspeed CTI interconnections provide access to “remote” memory or other
resources on remote cabinets. See “Multiple-Cabinet Server Connections”
on page 15 for multiple-cabinet information.
As Figure 7 on page 8 shows, the crossbar has four Exemplar Routing
Access Controllers (ERACs), each of which connects to 4 processor agents
and four memory controllers. All memory controllers and processor
agents connect to two separate ERACs, thus making the entire system’s
memory addressable by all processors and I/O devices in the system.
6Chapter 1
Overview
V-Class Server Architecture
Figure 6Functional Diagram of a Single-Cabinet V2500/V2600 Server