HP 9000 Integrity rp8440, 9000 Integrity rx8640, 9000 rp8440 Installation Manual
Installation Guide, HP Integrity rx8640 and
HP 9000 rp8440 Servers
HP Part Number: AB297-9012A-en
Published: September 2007
Edition: Fourth Edition
© Copyright 2007 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to changewithoutnotice.The only warranties for HP products and services are set forth in the express
warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP
shall not be liable for technical or editorial errors or omissions contained herein. Intel, Pentium, Intel Inside, and the Intel Inside logo are trademarks
or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
Table of Contents
About This Document.........................................................................................................9
Book Layout............................................................................................................................................9
Intended Audience.................................................................................................................................9
Typographic Conventions......................................................................................................................9
Publishing History................................................................................................................................10
Related Information..............................................................................................................................11
Typographic Conventions.....................................................................................................................12
HP Encourages Your Comments..........................................................................................................13
1 Server Overview...........................................................................................................15
Detailed Server Description..................................................................................................................15
Dimensions and Components.........................................................................................................16
Front Panel.......................................................................................................................................19
Front Panel Indicators and Controls..........................................................................................19
Enclosure Status LEDs...............................................................................................................19
Cell Board........................................................................................................................................19
PDH Riser Board........................................................................................................................20
Central Processor Units..............................................................................................................21
Memory Subsystem....................................................................................................................21
DIMMs .......................................................................................................................................22
Valid Memory Configurations...................................................................................................23
Cells and nPartitions........................................................................................................................24
Internal Disk Devices ......................................................................................................................24
System Backplane............................................................................................................................25
System Backplane to Cell Board Connectivity...........................................................................26
System Backplane to Core I/O Card Connectivity.....................................................................26
System Backplane to PCI-X/PCIe Backplane Connectivity........................................................26
Clocks and Reset........................................................................................................................26
PCI-X I/O Subsystem.......................................................................................................................26
PCI-X/PCIe I/O Backplane..........................................................................................................29
PCIe Slot Boot Paths...................................................................................................................30
Core I/O Card.............................................................................................................................31
Core I/O Boot Paths...............................................................................................................32
Mass Storage (Disk) Backplane..................................................................................................32
2 Installing the System.....................................................................................................35
Receiving and Inspecting the Server Cabinet.......................................................................................35
Unpacking the Server Cabinet.........................................................................................................35
Securing the Cabinet........................................................................................................................38
Standalone and To-Be-Racked Systems................................................................................................39
Rack-Mount System Installation.....................................................................................................39
Lifting the Server Cabinet Manually....................................................................................................40
Using the RonI Model 17000 SP 400 Lifting Device.............................................................................40
Installing the Wheel Kit........................................................................................................................42
Installing the Top and Side Covers.......................................................................................................47
Removing the Top Cover.................................................................................................................48
Installing the Top Cover..................................................................................................................49
Removing the Side Cover................................................................................................................49
Installing the Side Cover.................................................................................................................50
Table of Contents 3
Installing the Power Distribution Unit.................................................................................................50
Installing Additional Cards and Storage..............................................................................................51
Installing an Additional Hard Disk Drive ......................................................................................51
Removable Media Drive Installation...............................................................................................52
PCI-X Card Cage Assembly I/O Cards............................................................................................52
Installing an Additional PCI-X I/O Card...................................................................................56
Installing a A6869B VGA/USB PCI Card in a Server......................................................................58
Troubleshooting the A6869B VGA/USB PCI Card..........................................................................59
No Console Display...................................................................................................................60
Reference URL............................................................................................................................60
System Console Selection......................................................................................................................60
VGA Consoles..................................................................................................................................61
Interface Differences Between Itanium-based Systems...................................................................61
Other Console Types.......................................................................................................................61
Additional Notes on Console Selection...........................................................................................61
Cabling and Powering On the Server...................................................................................................62
Checking the Voltage.......................................................................................................................62
Verifying the Voltage Range of the Recptacle............................................................................62
Verifying the Safety Ground (Single Power Source)..................................................................63
Verifying the Safety Ground (Dual Power Source)....................................................................64
Voltage Check (Additional Procedure)...........................................................................................65
Connecting AC Input Power...........................................................................................................66
Applying Power to the Server....................................................................................................69
Installing The Line Cord Anchor (rack mounted servers)..............................................................69
Four Cell Server Installation (rp8400, rp8420, rp8440, rx8620, rx8640)......................................69
MP Core I/O Connections................................................................................................................70
Setting Up the Customer Engineer Tool (PC) .................................................................................71
Setting CE Tool Parameters........................................................................................................71
Connecting the CE Tool to the Local RS-232 Port on the MP ....................................................72
Turning On Housekeeping Power and Logging In to the MP........................................................72
Configuring LAN Information for the MP......................................................................................74
Accessing the Management Processor via a Web Browser.............................................................76
Verifying the Presence of the Cell Boards.......................................................................................78
System Console Selection................................................................................................................78
VGA Consoles............................................................................................................................79
Interface Differences Between Itanium-based Systems.............................................................79
Other Console Types..................................................................................................................79
Additional Notes on Console Selection.....................................................................................80
Configuring AC Line Status............................................................................................................80
Booting the Server ...........................................................................................................................80
Selecting a Boot Partition Using the MP ...................................................................................81
Verifying the System Configuration Using the EFI Shell...........................................................82
Booting HP-UX Using the EFI Shell...........................................................................................82
Adding Processors with Instant Capacity ......................................................................................82
Installation Checklist.......................................................................................................................83
4 Table of Contents
List of Figures
1-1 16-Socket Server Block Diagram...................................................................................................16
1-2 Server (Front View With Bezel).....................................................................................................17
1-3 Server (Front View Without Bezel)................................................................................................17
1-4 Server (Rear View).........................................................................................................................18
1-5 Front Panel LEDs and Power Switch.............................................................................................19
1-6 Cell Board......................................................................................................................................20
1-7 Socket Locations on Cell Board.....................................................................................................21
1-8 Memory Subsystem.......................................................................................................................22
1-9 DIMM Slot Layout.........................................................................................................................23
1-10 Internal Disks Locations................................................................................................................24
1-11 System Backplane Block Diagram.................................................................................................25
1-12 PCI-X/PCIe Board to Cell Board Block Diagram...........................................................................27
1-13 Mass Storage Block Diagram.........................................................................................................33
2-1 Removing the Polystraps and Cardboard.....................................................................................36
2-2 Removing the Shipping Bolts and Plastic Cover...........................................................................37
2-3 Preparing to Roll Off the Pallet.....................................................................................................38
2-4 Securing the Cabinet......................................................................................................................39
2-5 Positioning the Lifter to the Pallet.................................................................................................41
2-6 Raising the Server Off the Pallet Cushions....................................................................................42
2-7 Server on Shipping Pallet..............................................................................................................43
2-8 Removing Cushion from Front Edge of Server.............................................................................44
2-9 Attaching a Caster Wheel to the Server........................................................................................44
2-10 Attaching the Ramp to the Pallet..................................................................................................45
2-11 Removing Side Cushion from Server............................................................................................46
2-12 Securing Each Caster Cover to the Server.....................................................................................46
2-13 Completed Wheel Kit Installation.................................................................................................47
2-14 Cover Locations ............................................................................................................................48
2-15 Top Cover Detail ...........................................................................................................................49
2-16 Side Cover Detail...........................................................................................................................50
2-17 Disk Drive Location.......................................................................................................................51
2-18 Removable Media Drive Location.................................................................................................52
2-19 PCI I/O Slot Details........................................................................................................................58
2-20 PCI/PCI-X Card Location..............................................................................................................59
2-21 Console Output Device menu.......................................................................................................61
2-22 Voltage Reference Points for IEC-320 C19 Plug............................................................................62
2-23 Safety Ground Reference Check — Single Power Source.............................................................63
2-24 Safety Ground Reference Check — Dual Power Source...............................................................64
2-25 Wall Receptacle Pinouts................................................................................................................66
2-26 AC Power Input Labeling..............................................................................................................67
2-27 Distribution of Input Power for Each Bulk Power Supply............................................................68
2-28 Four Cell Line Cord Anchor (rp8400, rp8420, rp8440, rx8620, rx8640).........................................70
2-29 Line Cord Anchor and Velcro straps.............................................................................................70
2-30 LAN and RS-232 Connectors on the Core I/O Board ...................................................................72
2-31 Front Panel Display ......................................................................................................................73
2-32 BPS LED Location..........................................................................................................................73
2-33 MP Main Menu..............................................................................................................................74
2-34 The lc Command Screen................................................................................................................75
2-35 The ls Command Screen................................................................................................................76
2-36 Example sa Command..................................................................................................................77
2-37 Browser Window...........................................................................................................................77
2-38 The du Command Screen..............................................................................................................78
5
2-39 Console Output Device menu.......................................................................................................79
2-40 The pwrgrd Command Screen.....................................................................................................80
6 List of Figures
List of Tables
1-1 Cell Board CPU Module Load Order............................................................................................21
1-2 DIMM Sizes Supported.................................................................................................................22
1-3 DIMM Load Order........................................................................................................................23
1-4 Removable Media Drive Path........................................................................................................25
1-5 Hard Disk Drive Path....................................................................................................................25
1-6 PCI-X Slot Boot Paths Cell 0..........................................................................................................27
1-7 PCI-X Slot Boot Paths Cell 1..........................................................................................................27
1-8 PCI-X Slot Types............................................................................................................................29
1-9 PCIe Slot Types..............................................................................................................................31
1-10 Core I/O Boot Paths.......................................................................................................................32
2-1 Wheel Kit Packing List..................................................................................................................42
2-2 HP Integrity rx8640 Server PCI-X/PCIe I/O Cards........................................................................53
2-3 HP 9000 rp8440 Server PCI-X I/O Cards.......................................................................................54
2-4 Single Phase Voltage Examples.....................................................................................................63
2-5 BPS-to-Cell Board Configuration to Achieve N+1........................................................................69
2-6 Factory-Integrated Installation Checklist......................................................................................83
7
8
About This Document
This document covers the HP Integrity rx8640 and the HP 9000 rp8440 server systems.
This document does not describe system software or partition configuration in any detail. For
detailed information concerning those topics, refer to the HP System Partitions Guide:
Administration for nPartitions.
Book Layout
This document contains the following chapters:
• Chapter 1 - Server Overview
• Chapter 2 - Installing the System
Intended Audience
This document is intended to be used by customer engineers assigned to support the HP Integrity
rx8640 and HP 9000 rp8440 servers.
Typographic Conventions
This document uses the following typographical conventions:
%, $, or #
A percent sign represents the C shell system prompt. A dollar
sign represents the system prompt for the Bourne, Korn, and
POSIX shells. A number sign represents the superuser prompt.
audit(5) A manpage. The manpage name is audit, and it is located in
Section 5.
Command
A command name or qualified command phrase.
Computer output
Text displayed by the computer.
Ctrl+x A key sequence. A sequence such as Ctrl+x indicates that you
must hold down the key labeled Ctrl while you press another
key or mouse button.
ENVIRONMENT VARIABLE The name of an environment variable, for example, PATH.
[ERROR NAME]
The name of an error, usually returned in the errno variable.
Key The name of a keyboard key. Return and Enter both refer to the
same key.
Term The defined use of an important word or phrase.
User input
Commands and other text that you type.
Variable
The name of a placeholder in a command, function, or other
syntax display that you replace with an actual value.
[] The contents are optional in syntax. If the contents are a list
separated by |, you must choose one of the items.
{} The contents are required in syntax. If the contents are a list
separated by |, you must choose one of the items.
... The preceding element can be repeated an arbitrary number of
times.
Indicates the continuation of a code example.
| Separates items in a list of choices.
WARNING A warning calls attention to important information that if not
understood or followed will result in personal injury or
nonrecoverable system problems.
Book Layout 9
CAUTION A caution calls attention to important information that if not
understood or followed will result in data loss, data corruption,
or damage to hardware or software.
IMPORTANT This alert provides essential information to explain a concept or
to complete a task
NOTE A note contains additional information to emphasize or
supplement important points of the main text.
Publishing History
The document printing date and part number indicate the document’s current edition. The
printing date will change when a new edition is printed. Minor changes may be made at reprint
without changing the printing date. The document part number will change when extensive
changes are made. Document updates may be issued between editions to correct errors or
document product changes. To ensure that you receive the updated or new editions, you should
subscribe to the appropriate product support service. See your HP sales representative for details.
You can find the latest version of this document on line at:
http://www.docs.hp.com.
March 2006. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .First Edition
September 2006. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Second Edition
January 2007. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Third Edition
September 2007. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Fourth Edition
10
Related Information
You can access other information on HP server hardware management, Microsoft® Windows®
administratuon, and diagnostic support tools at the following web sites:
http://docs.hp.com The main web site for HP technical documentation is http://docs.hp.com.
Server Hardware Information: http://docs.hp.com/hpux/hw/ The
http://docs.hp.com/hpux/hw/ web site is the systems hardware portion of docs.hp.com.
It provides HP nPartition server hardware management information, including site preparation,
installation, and more.
Related Information 11
Windows Operating System Information You can find information about administration of the
Microsoft® Windows® operating system at the following web sites, among others:
• http://docs.hp.com/windows_nt/
• http://www.microsoft.com/technet/ HP books are available worldwide through
bookstores, online booksellers, and office and computer stores.
Diagnostics and Event Monitoring: Hardware Support Tools Complete information about HP
hardware support tools, including online and offline diagnostics and event monitoring tools, is
at the http://docs.hp.com/hpux/diag/ web site. This site has manuals, tutorials, FAQs,
and other reference material.
Web Site for HP Technical Support: http://us-support2.external.hp.com HP IT resource center
web site at http://us-support2.external.hp.com/ provides comprehensive support
information for IT professionals on a wide variety of topics, including software, hardware, and
networking.
Books about HP-UX Published by Prentice Hall The http://www.hp.com/hpbooks/ web
site lists the HP books that Prentice Hall currently publishes, such as HP-UX books including:
• HP-UX 11i System Administration Handbook and Toolkit
http://www.hp.com/hpbooks/prentice/ptr_0130600814.html
• HP-UX Virtual Partitions
http://www.hp.com/hpbooks/prentice/ptr_0130352128.html
HP books are available worldwide through bookstores, online booksellers, and office and
computer stores.
Typographic Conventions
The following notational conventions are used in this publication.
WARNING! A warning lists requirements that you must meet to avoid personal injury.
CAUTION: A caution provides information required to avoid losing data or avoid losing system
functionality.
NOTE: A note highlights useful information such as restrictions, recommendations, or important
details about HP product features.
• Commands and options are represented using this font.
• Text that you type exactly as shown is represented using this font.
• Text to be replaced with text that you supply is represented using this font.
Example:
“Enter the ls -l filename command” means you must replace filename with your own text.
• Keyboard keys and graphical interface items (such as buttons, tabs, and menu items)
are represented using this font.
Examples:
The Control key, the OK button, the General tab, the Options menu.
• Menu —> Submenu represents a menu selection you can perform.
Example:
“Select the Partition —> Create Partition action” means you must select the Create Partition
menu item from the Partition menu.
• Example screen output is represented using this font.
12
HP Encourages Your Comments
HP encourages your comments concerning this document. We are committed to providing
documentation that meets your needs. Send any errors found, suggestions for improvement, or
compliments to:
feedback@fc.hp.com
Include the document title, manufacturing part number, and any comment, error found, or
suggestion for improvement you have concerning this document.
HP Encourages Your Comments 13
14
1 Server Overview
The HP Integrity rx8640 server and the HP 9000 rp8440 server are members of the HP
business-critical computing platform family of mid-range, mid-volume servers, positioned
between the HP Integrity rx7640, HP 9000 rp7440 and HP Integrity Superdome servers.
IMPORTANT: Ensure a valid UUID is either in place or available prior to maintenance of these
servers. This step is vital when performing upgrades and is recommended for existing hardware
service restoration. Specific information for upgrades is found in the Upgrade Guide, Mid-Range
Four-Cell HP Servers to HP Integrity rx8640, HP 9000 rp8440 Servers, located at the following URL:
http://docs.fc.hp.com.
The server is a 17U1high, 16-socket symmetric multiprocessor (SMP) rack-mount or standalone
server. Features of the server include:
• Up to 512 GB of physical memory provided by dual inline memory modules (DIMMs).
• Up to 32 processors with a maximum of 4 processor modules per cell board and a maximum
of 4 cell boards. Supports dual-core processors.
• One cell controller (CC) per cell board.
• Turbo fans to cool CPUs and CCs on the cell boards.
• Up to four embedded hard disk drives.
• Up to two internal DVD drives or one DVD drive and one DDS-4 DAT drive.
• Nine front chassis mounted N+1 fans.
• Twelve rear chassis mounted N+1 fans.
• Six N+1 PCI-X card cage fans.
• Up to six N+1 bulk power supplies.
• Two N+1 PCI-X power supplies.
• N+1 hot-swappable system clock oscillators.
• Sixteen PCI-X slots are divided into two I/O chassis. Each I/O chassis can accommodate up
to eight PCI/PCI-X/PCIe/PCI-X 2.0 cards.
• Up to two core I/O cards.
• One failover service processor per core I/O card.
• Four 220 V AC power plugs. Two are required and the other two provide power source
redundancy.
Detailed Server Description
The following section provides detailed intormation about the server components.
1. The U is a unit of measurement specifying product height. One U is equal to 1.75 inches.
Detailed Server Description 15
Figure 1-1 16-Socket Server Block Diagram
Dimensions and Components
The following section describes server dimensions and components.
16 Server Overview
Figure 1-2 Server (Front View With Bezel)
Figure 1-3 Server (Front View Without Bezel)
The server has the following dimensions:
• Depth: Defined by cable management constraints to fit into a standard 36-inch deep rack:
25.5 inches from front rack column to PCI connector surface
Detailed Server Description 17
26.7 inches from front rack column to core I/O card connector surface
30 inches overall package dimension, including 2.7 inches protruding in front of the front
rack columns
• Width: 17.5 inches, constrained by EIA standard 19-inch racks
• Height: 17 U (29.55 inches), constrained by package density
The mass storage section located in the front enables access to removable media drives without
removal of the bezel. The mass storage bay accommodates two 5.25-inch removable media drives
and up to four 3.5-inch hard disk drives. The front panel display, containing LEDs and the system
power switch, is located directly above the hard drive media bays.
Below the mass storage section and behind a removable bezel are two PCI-X power supplies.
Each PCI-X power supply powers both I/O partitions. Two PCI-X power supplies offer a N+1
configuration.
Enclosed with protective finger guards are nine front online replace (OLR) fan modules.
The bulk power supply is partitioned by a sealed metallic enclosure located in the bottom of the
server. This enclosure houses the N+1 fully redundant bulk power supplies. Install these power
supplies from the front of the server after removing the front bezel. The power supply is 2.45 X
5.625 X 20.0 inches.
Figure 1-4 Server (Rear View)
Access the PCI-X/PCIe I/O card section, located toward the rear by removing the top cover.
The PCI card bulkhead connectors are located at the rear top.
The PCI-X OLR fan modules are located in front of the PCI cards. They are housed in plastic
carriers.
The 12 rear OLR fans attached outside the chassis house 120-mm exhaust fans.
18 Server Overview
The cell boards are located on the right side of the server behind a removable side cover. For
rack mounted servers on slides, the rack front door requires removal if it is hinged on the right
side of the rack. Removal will allow unrestricted access to server sides after sliding server out
for service..
The two redundant core I/O cards are positioned vertically end-to-end at the rear of the chassis.
Redundant line cords attach to the AC power receptacles at the bottom rear. Two 20-amp cords
are required to power the server. Two additional line cords provide redundancy.
Access the system backplane by removing the left side cover. The system backplane hinges from
the lower edge and is anchored at the top with a single large jack screw assembly.
The SCSI ribbon cable assembly also routes across and fastens to the backside of the system
backplane near the connectors that attach the core I/O boards.
The blue deployment handles hinge outward for manual lift. When server is slide mounted, they
retract against chassis to enable slide action without obstruction.
Front Panel
Front Panel Indicators and Controls
The front panel, located on the front of the server, includes the power switch. Refer to Figure 1-5.
Enclosure Status LEDs
The following status LEDs are on the front panel:
• Locate LED (blue)
• Power LED (tricolor)
• Management processor (MP) status LED (tricolor)
• Cell 0, 1, 2, 3 status (tricolor) LEDs
Figure 1-5 Front Panel LEDs and Power Switch
Cell Board
The cell board, illustrated in Figure 1-6, contains the processors, main memory, and the CC
application specific integrated circuit (ASIC) which interfaces the processors and memory with
the I/O. The CC is the heart of the cell board, providing a crossbar connection that enables
communication with other cell boards in the system. It connects to the processor dependent
hardware (PDH) and microcontroller hardware. Each cell board holds up to four processor
modules and 16 memory DIMMs. One to four cell boards can be installed in the server. A cell
board can be selectively powered off for adding processors, memory or maintenance of the cell
board, without affecting cells in other configured partitions.
Detailed Server Description 19
Figure 1-6 Cell Board
The server has a 48 V distributed power system and receives the 48 V power from the system
backplane board. The cell board contains DC-to-DC converters to generate the required voltage
rails. The DC-to-DC converters on the cell board do not provide N+1 redundancy.
The cell board contains the following major buses:
• Front side buses (FSB) for each of the four processors
• Four memory buses (one going to each memory quad)
• Incoming and outgoing I/O bus that goes off board to an SBA chip
• Incoming and outgoing crossbar busses that communicate to the crossbar chips on the system
backplane
• PDH bus that goes to the PDH and microcontroller circuitry
All of these buses come together at the CC chip.
Because of space limitations on the cell board, the PDH and microcontroller circuitry reside on
a riser board that plugs at a right angle into the cell board. The cell board also includes clock
circuits, test circuits, and decoupling capacitors.
PDH Riser Board
The server PDH riser board is a small card that plugs into the cell board at a right angle. The
PDH riser interface contains the following components:
• Microprocessor memory interface microcircuit
• Hardware including the processor dependant code (PDH) flash memory
• Manageability microcontroller with associated circuitry
The PDH obtains cell board configuration information from cell board signals and from the cell
board local power module (LPM).
20 Server Overview
Central Processor Units
The cell board can hold up to four CPU modules. Each CPU module can contain up to two CPU
cores on a single die. Modules are populated in increments of one. On a cell board, the processor
modules must be the same family, type, and clock frequencies. Mixing of different processors
on a cell or a partition is not supported. See Table 1-1 for the load order that must be maintained
when adding processor modules to the cell board. See Figure 1-7 for the locations on the cell
board for installing processor modules.
NOTE: Unlike previous HP cell based systems, the server cell board does not require that a
termination module be installed at the end of an unused FSB. System firmware is allowed to
disable an unused FSB in the CC. This enables both sockets of the unused bus to remain
unpopulated.
Table 1-1 Cell Board CPU Module Load Order
Socket 0Socket 1Socket 3Socket 2Number of
CPU Modules
Installed
CPU installedEmpty slotEmpty slotEmpty slot1
CPU installedEmpty slotEmpty slotCPU installed2
CPU installedCPU installedEmpty slotCPU installed3
CPU installedCPU installedCPU installedCPU installed4
Figure 1-7 Socket Locations on Cell Board
Memory Subsystem
Figure 1-8 shows a simplified view of the memory subsystem. It consists of four independent
access paths, each path having its own address bus, control bus, data bus, and DIMMs . Address
and control signals are fanned out through register ports to the synchronous dynamic random
access memory (SDRAM) on the DIMMs.
Detailed Server Description 21
The memory subsystem comprises four independent quadrants. Each quadrant has its own
memory data bus connected from the cell controller to the two buffers for the memory quadrant.
Each quadrant also has two memory control buses: one for each buffer.
Figure 1-8 Memory Subsystem
DIMMs
The memory DIMMs used by the server are custom designed by HP. Each DIMM contains DDR-II
SDRAM memory that operates at 533 MT/s. Industry standard modules do not support the high
availability and shared memory features of the server. Therefore, industry standard DIMM
modules are not supported.
The server supports DIMMs with densities of 1, 2, 4, and 8 GB. Table 1-2 lists each supported
DIMM size, the resulting total server capacity, and the memory component density. Each DIMM
is connected to two buffer chips on the cell board.
Table 1-2 DIMM Sizes Supported
Memory Component DensityTotal CapacityDIMM Size
256 Mb64 GB1 GB
512 Mb128 GB2 GB
1024 Mb256 GB4 GB
2048 Mb512 GB8 GB
22 Server Overview
Valid Memory Configurations
The first cell must have one DIMM pair loaded in slots 0A/0B. The server can support as little
as 2 GB of main memory using two 1 GB DIMMs installed on one of the cell boards and as much
as 512 GB by filling all 16 DIMM slots on all four cell boards with 8 GB DIMMs.
The following rules explain the memory configuration:
1. DIMMs must be loaded in pairs (same size within a pair).
2. DIMM pairs must be loaded in slot order (0A/0B, 1A/1B, 2A/2B, ...)
3. Largest DIMMs must be loaded first followed by progressively smaller DIMM module sizes.
A paired set of DIMMs is called a rank. DIMMs in a rank must be of the same capacity. See
Table 1-3 and Figure 1-9 for DIMM load order and layout on the cell board.
A quad is a grouping of four DIMMs (Figure 1-9). Configurations with 8 or 16 DIMM slots loaded
are recommended. Adding a rank enables a dedicated DDR-II bus on a cell to increase the amount
of usable memory bandwidth available. Available memory is proportional to the amount of
memory installed.
Table 1-3 DIMM Load Order
Quad LocationDIMM Location on Cell
Board
Action TakenNumber of DIMMs Installed
Quad 20A and 0BInstall first2 DIMMs = 1 rank
Quad 11A and 1BAdd second4 DIMMs = 2 rank
Quad 32A and 2BAdd third6 DIMMs = 3 rank
Quad 03A and 3BAdd fourth8 DIMMs = 4 rank
Quad 24A and 4BAdd fifth10 DIMMs = 5 rank
Quad 15A and 5BAdd sixth12 DIMMs = 6 rank
Quad 36A and 6BAdd seventh14 DIMMs = 7 rank
Quad 07A and 7BAdd last16 DIMMs = 8 rank
Figure 1-9 DIMM Slot Layout
Detailed Server Description 23
Cells and nPartitions
An nPartition comprises one or more cells working as a single system. Any I/O chassis that is
attached to a cell belonging to an nPartition is also assigned to the nPartition. Each I/O chassis
has PCI card slots, I/O cards, attached devices, and a core I/O card assigned to the I/O chassis.
On the server, each nPartition has its own dedicated portion of the server hardware which can
run a single instance of the operating system. Each nPartition can boot, reboot, and operate
independently of any other nPartitions and hardware within the same server complex.
The server complex includes all hardware within an nPartition server: all cabinets, cells, I/O
chassis, I/O devices and racks, management and interconnecting hardware, power supplies, and
fans.
A server complex can contain one or more nPartitions, enabling the hardware to function as a
single system or as multiple systems.
NOTE: Partition configuration information is available on the Web at:
http://docs.hp.com.
Refer to HP System Partitions Guide: Administration for nPartitions for details.
Internal Disk Devices
Figure 1-10 (page 24) shows the top internal disk drives connect to cell 0 through the core I/O
for cell 0, in a server cabinet. The bottom internal disk drives connect to cell 1 through the core
I/O for cell 1.
The upper removable media drive connects to cell 0 through the core I/O card for cell 0 and the
lower removable media drive connects to cell 1 through the core I/O card for cell 1.
A list of replacement disk drives for the server is in Appendix A. The list contains both removable
media disk drives and hard disk drives.
Figure 1-10 Internal Disks Locations
24 Server Overview
Table 1-4 Removable Media Drive Path
PathRemovable Media
0/0/0/2/1.x1.0Slot 0 media
1/0/0/2/1.x1.0Slot 1 media
1 X equals 2 for a DVD drive while X equals 3 for a DDS-4 DAT drive.
Table 1-5 Hard Disk Drive Path
PathHard Drive
0/0/0/2/0.6.0Slot 0 drive
0/0/0/3/0.6.0Slot 1 drive
1/0/0/2/0.6.0Slot 2 drive
1/0/0/3/0.6.0Slot 3 drive
System Backplane
The system backplane board contains the following components:
• Two crossbar chips (XBC)
• Clock generation logic
• Preset generation logic
• Power regulators
• Two local bus adapter (LBA) chips that create internal PCI buses for communicating with
the core I/O card.
The backplane also contains connectors for attaching the cell boards, PCI-X backplane, MP core
I/O cards SCSI cables, bulk power, chassis fans, front panel display, intrusion switches, and
external system bus adapters (SBA) link connectors.
Figure 1-11 System Backplane Block Diagram
Detailed Server Description 25
The two LBA PCI bus controllers on the system backplane create the PCI bus for the core I/O
cards. You must shut down the partition for the core I/O card before removing the card.
Having the SCSI connectors on the system backplane allows replacement of the core I/O card
without having to remove cables in the process.
System Backplane to Cell Board Connectivity
The system backplane provides four sets of connectors, one set for each cell board.
The system backplane routes the signals from the cell boards to the communication crossbars.
Cell boards 0 and 1 are directly connected to the I/O backplane found in the server. Cell boards
2 and 3 can be connected to a separate I/O expansion chassis connected to the system backplane.
System Backplane to Core I/O Card Connectivity
The core I/O cards connect at the rear of the system backplane through two connectors. SCSI and
LAN on a core I/O are accessed via a PCI-X 66 MHz bus. Two LBA bus controllers located on
the system backplane allow communication to the I/O devices. The LBAs are connected to the
SBA on the PCI-X/PCIe backplane by single ropes.
The system backplane routes the signals to the various components in the system. The core I/O
signals include the SCSI bus for the system hard drives and the bus for the removable media
devices. Each core I/O card provides SCSI buses for the mass storage devices.
The management processor for the chassis resides on the core I/O card, so the system backplane
also provides interfaces required for management of the system. These interfaces and the
manageability circuitry run on standby power.
You can remove the core I/O cards from the system as long as you shut down the partition for
the core I/O card before removing the card. The hot-plug circuitry that enables this feature is
located on the system backplane near the core I/O sockets.
System Backplane to PCI-X/PCIe Backplane Connectivity
The PCI-X/PCIe backplane uses two connectors for the SBA link bus and two connectors for the
high-speed data signals and the manageability signals.
SBA link bus signals are routed through the system backplane to the cell controller on each
corresponding cell board.
The high-speed data signals are routed from the SBA chips on the PCI-X/PCIe backplane to the
two LBA PCI bus controllers on the system backplane.
Clocks and Reset
The system backplane contains reset and clock circuitry that propagates through the whole
system. The system backplane central clocks drive all major chip set clocks. The system central
clock circuitry features redundant, hot-swappable oscillators.
PCI-X I/O Subsystem
The cell board to the PCI-X board path runs from the CC to the SBA, from the SBA to the ropes,
from the ropes to the LBA, and from the LBA to the PCI slots as shown in Figure 1-12. The CC
on cell board 0 and cell board 1 communicates through an SBA over the SBA link. The SBA link
consists of both an inbound and an outbound link with a peak bandwidth of approximately 11.5
GB/s at 3.2 GT/s. The SBA converts the SBA link protocol into “ropes.” A rope is defined as a
high-speed, point-to-point data bus. The SBA can support up to 16 of these high-speed
bidirectional rope links for a total aggregate bandwidth of approximately 11.5 GB/s.
There are LBA chips on the PCI-X backplane that act as a bus bridge, supporting either one or
two ropes for PCI-X 133 MHz slots and the equivalent bandwidth of four ropes for PCI-X 266
slots. Each LBA acts as a bus bridge, supporting one or two ropes and capable of driving 33 MHz
or 66 MHz for PCI cards. The LBAs can also drive at 66 MHz or 133 MHz for PCI-X mode 1 cards,
26 Server Overview
Loading...