HP INTEGRITY RX6600 User Manual
HP Integrity rx6600 User Service Guide
HP Part Number: AB464-9003C-ed4
Published: August 2008
Edition: Fourth edition
© Copyright 2006–2008 Hewlett-Packard Development Company, L.P
Legal Notices
©Copyright 2006 Hewlett-Packard Development Company, L.P.
The informationcontained hereinis subjectto changewithout notice.The onlywarranties for HP products and services are set forth in the express
warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP
shall not be liable for technical or editorial errors or omissions contained herein.
Intel®, Pentium®, Intel Inside®,Itanium®, and the Intel Insidelogo aretrademarks or registered trademarks ofIntel Corporationor its subsidiaries
in the United States and other countries.
Linux® is a U.S. registered trademark of Linus Torvalds.
Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.
UNIX® is a registered trademark of The Open Group.
Table of Contents
About This Document.......................................................................................................19
Intended Audience................................................................................................................................19
Publishing History................................................................................................................................19
Document Organization.......................................................................................................................19
Typographic Conventions...............................................................................................................20
Related Documents...............................................................................................................................20
HP Encourages Your Comments..........................................................................................................20
1 Overview.......................................................................................................................21
Server Subsystems................................................................................................................................21
I/O....................................................................................................................................................21
PCI/PCI-X IOBP..........................................................................................................................23
PCI/PCI-X/PCIe IOBP.................................................................................................................24
PCIe MPS Optimization........................................................................................................24
Processor..........................................................................................................................................25
Memory...........................................................................................................................................25
Cooling.............................................................................................................................................26
Power...............................................................................................................................................27
Front Display Panel, DVD, and Diagnostic Panel...........................................................................29
Mass Storage....................................................................................................................................29
Firmware...............................................................................................................................................30
User Interface...................................................................................................................................31
Event IDs for Errors and Events......................................................................................................31
Controls, Ports, and LEDs.....................................................................................................................31
Front Panel.......................................................................................................................................31
Storage and Media Devices.............................................................................................................33
Hot-Pluggable Disk Drive LEDs................................................................................................33
Hot-Pluggable Disk Drive Slot Availability LEDs.....................................................................35
DVD Drive..................................................................................................................................35
Diagnostic Panel..............................................................................................................................35
Rear Panel........................................................................................................................................36
iLO 2 MP.....................................................................................................................................37
iLO 2 MP Reset Button..........................................................................................................38
Core I/O Board Ports.............................................................................................................38
iLO 2 MP Status and LAN LEDs..........................................................................................39
System LAN...............................................................................................................................39
Power Supply.............................................................................................................................40
Rear Panel UID LED and Button................................................................................................41
PCI/PCI-X/PCIe Card Slot..........................................................................................................41
2 System Specifications...................................................................................................43
Server Specifications.............................................................................................................................43
Dimensions and Weight........................................................................................................................43
Grounding.............................................................................................................................................44
Electrical Specifications.........................................................................................................................44
System Power Specifications...........................................................................................................44
Power Consumption and Cooling...................................................................................................45
Environmental Specifications...............................................................................................................45
Table of Contents 3
3 Installing the System.....................................................................................................47
Safety Information................................................................................................................................47
Installation Sequence and Checklist.....................................................................................................47
Unpacking and Inspecting the Server...................................................................................................48
Verifying Site Preparation...............................................................................................................48
Inspecting the Shipping Containers for Damage............................................................................48
Unpacking the Server......................................................................................................................49
Checking the Inventory...................................................................................................................49
Returning Damaged Equipment.....................................................................................................49
Unloading the Server with a Lifter..................................................................................................49
Installing Additional Components.......................................................................................................50
Removing and Replacing the Top Cover.........................................................................................50
Removing the Top Cover............................................................................................................50
Replacing the Top Cover............................................................................................................51
Removing and Replacing the Memory Carrier Assembly Cover...................................................51
Removing the Memory Carrier Assembly Cover......................................................................51
Replacing the Memory Carrier Assembly Cover.......................................................................52
Installing a Hot-Swappable Power Supply.....................................................................................52
Power Supply Loading Guidelines............................................................................................53
Installing a Power Supply..........................................................................................................53
Removing and Replacing Hot-Swappable Disk Drive Fillers.........................................................54
Removing a Hot-Swappable Disk Drive Filler...........................................................................54
Replacing a Hot-Swappable Disk Drive Filler...........................................................................55
Installing a Hot-Pluggable SAS Hard Drive....................................................................................56
Installing a Hot-Pluggable Disk Drive.......................................................................................56
Installing a PCI/PCI-X/PCIe Card...................................................................................................57
PCI/PCI-X/PCIe Configurations.................................................................................................58
Offline Installation of a PCI Card...............................................................................................60
Removing and Replacing the Memory Carrier Assembly..............................................................61
Removing the Memory Carrier Assembly.................................................................................61
Replacing the Memory Carrier Assembly..................................................................................62
Installing System Memory DIMMs.................................................................................................63
Memory Installation Conventions.............................................................................................65
Supported DIMM Sizes and Memory Configurations.........................................................65
Memory Load Order.............................................................................................................66
Memory Loading Rules and Guidelines...............................................................................67
Installing Memory......................................................................................................................68
Removing and Replacing the Processor Board Assembly..............................................................70
Removing the Processor Board Assembly.................................................................................71
Replacing the Processor Board Assembly..................................................................................72
Installing a Dual-Core Processor.....................................................................................................72
Processor Load Order.................................................................................................................73
Required Tools............................................................................................................................73
Installing a Dual-Core Processor................................................................................................73
Installing the Server into a Rack or Pedestal Mount.............................................................................78
Installing the Server into a Rack......................................................................................................78
HP Rack......................................................................................................................................79
Non-HP Rack..............................................................................................................................79
Installing the Server into a Pedestal Mount....................................................................................79
Connecting the Cables..........................................................................................................................79
AC Input Power...............................................................................................................................79
Power States...............................................................................................................................80
Applying Standby Power to the Server......................................................................................80
LAN.................................................................................................................................................80
4 Table of Contents
Console Setup.......................................................................................................................................81
Overview.........................................................................................................................................82
Setup Checklist................................................................................................................................82
Setup Flowchart...............................................................................................................................83
Preparation......................................................................................................................................84
Determining the Physical iLO 2 MP Access Method.................................................................84
Determining the iLO 2 MP LAN Configuration Method..........................................................85
Configuring the iLO 2 MP LAN Using DHCP and DNS................................................................86
Configuring the iLO 2 MP LAN Using ARP Ping...........................................................................87
Configuring the iLO 2 MP LAN Using the RS-232 Serial Port........................................................88
Logging In to the iLO 2 MP.............................................................................................................89
Additional Setup..............................................................................................................................90
Modifying User Accounts and Default Password.....................................................................90
Setting Up Security.....................................................................................................................91
Security Access Settings........................................................................................................91
Accessing the Host Console..................................................................................................................91
Accessing the iLO 2 MP With the Web Browser.............................................................................91
Help............................................................................................................................................93
Accessing the Host Console With the TUI - CO Command............................................................93
Accessing the Host Console With vKVM - Integrated Remote Console.........................................93
Accessing the Host Console with the SMASH SM CLP..................................................................94
Accessing the Graphic Console Using VGA ..................................................................................94
Powering Off and Powering On the Server..........................................................................................94
Server Power Button........................................................................................................................94
Power States.....................................................................................................................................95
Powering Off the Server..................................................................................................................95
Powering Off the Server Using the iLO 2 MP............................................................................96
Powering Off the Server Manually............................................................................................96
Powering On the Server...................................................................................................................96
Powering On the Server Using the iLO 2 MP............................................................................96
Powering On the Server Manually.............................................................................................96
Core I/O Card Configuration................................................................................................................97
Integrated RAID..............................................................................................................................97
Integrated Mirror.......................................................................................................................97
Global Hot Spare........................................................................................................................97
HP 8 Internal Port SAS HBA (SAS Controller)................................................................................97
MPTUTIL Utility........................................................................................................................98
Flashing Firmware on First Controller.................................................................................98
Flashing BIOS and EFI Driver on the First Controller..........................................................99
Common Questions About Flashing Firmware....................................................................99
Viewing the VPD Information for EFI Driver and RISC Firmware......................................99
EFI Commands...........................................................................................................................99
DRVCFG Utility..................................................................................................................100
CFGGEN Utility.......................................................................................................................108
Starting CFGGEN................................................................................................................108
CFGGEN Operation............................................................................................................108
Rules for creating IM volumes and hot spare disks...........................................................108
CFGGEN Commands..........................................................................................................109
Smart Array P400, P600 and P800 Controllers..............................................................................110
Quick Installation Procedure....................................................................................................110
Connecting External Storage....................................................................................................111
SAS Cable Part Numbers..........................................................................................................111
SAUPDATE Utility...................................................................................................................111
Syntax..................................................................................................................................112
Commands..........................................................................................................................112
Table of Contents 5
List.......................................................................................................................................112
UPDATE..............................................................................................................................112
UPDATE all.........................................................................................................................113
HELP or ?............................................................................................................................114
Error Messages....................................................................................................................114
EBSU Utility..............................................................................................................................114
Configuring the Array..............................................................................................................117
Comparing the Utilities............................................................................................................117
ORCA Utility............................................................................................................................118
Creating a Logical Drive Using ORCA...............................................................................118
ACU Utility...............................................................................................................................119
Installation Troubleshooting...............................................................................................................119
Troubleshooting Methodology......................................................................................................119
Troubleshooting Using the Server Power Button..........................................................................120
Server Does Not Power On............................................................................................................120
EFI Menu is Not Available.............................................................................................................121
Operating System Does Not Boot..................................................................................................121
Operating System Boots with Problems........................................................................................121
Intermittent Server Problems.........................................................................................................121
DVD Problems...............................................................................................................................122
Hard Drive Problems.....................................................................................................................122
Console Problems..........................................................................................................................122
Downloading and Installing the Latest Version of the Firmware.................................................122
Downloading the Latest Version of the Firmware...................................................................122
Installing the Latest Version of the Firmware on the Server....................................................123
Enabling the Trusted Platform Module..............................................................................................123
Introduction...................................................................................................................................123
Enabling the TPM..........................................................................................................................123
4 Booting and Shutting Down the Operating System...............................................125
Configuring System Boot Options......................................................................................................125
Booting and Shutting Down HP-UX...................................................................................................127
Adding HP-UX to the Boot Options List.......................................................................................127
Booting HP-UX in Standard Mode................................................................................................128
Booting HP-UX From the EFI Boot Manager...........................................................................128
Booting HP-UX From the EFI Shell..........................................................................................129
Booting HP-UX in Single-User Mode............................................................................................130
Booting HP-UX in LVM-Maintenance Mode.................................................................................131
Shutting Down HP-UX..................................................................................................................131
Booting and Shutting Down HP OpenVMS.......................................................................................132
Adding HP OpenVMS to the Boot Options List............................................................................132
Booting HP Open VMS..................................................................................................................133
Booting HP OpenVMS from the EFI Boot Manager................................................................133
Booting HP OpenVMS from the EFI Shell...............................................................................133
Shutting Down HP OpenVMS.......................................................................................................134
Booting and Shutting Down Microsoft Windows..............................................................................135
Adding Microsoft Windows to the Boot Options List...................................................................135
Booting the Microsoft Windows Operating System......................................................................136
Shutting Down Microsoft Windows..............................................................................................137
Shutting Down Windows from the Start Menu......................................................................137
Shutting Down Windows from the Command Line................................................................137
Booting and Shutting Down Linux.....................................................................................................138
Adding Linux to the Boot Options List.........................................................................................138
Booting the Red Hat Enterprise Linux Operating System............................................................139
6 Table of Contents
Booting Red Hat Enterprise Linux from the EFI Boot Manager Menu....................................140
Booting Red Hat Enterprise Linux from the EFI Shell.............................................................140
Booting the SuSE Linux Enterprise Server Operating System......................................................140
Selecting a SuSE Linux Enterprise Server entry from the EFI Boot Manager menu................140
Booting SuSE Linux Enterprise Server from the EFI Shell.......................................................140
Shutting Down Linux....................................................................................................................141
5 Troubleshooting..........................................................................................................143
Methodology.......................................................................................................................................143
General Troubleshooting Methodology........................................................................................143
Recommended Troubleshooting Methodology ............................................................................144
Basic and Advanced Troubleshooting Tables................................................................................145
Troubleshooting Tools.........................................................................................................................151
LEDs ..............................................................................................................................................151
Front Panel................................................................................................................................151
External Health LED (EHLED)...........................................................................................152
Internal Health LED (IHLED).............................................................................................152
System Health LED (SHLED).............................................................................................153
Locator Switch/LED (Unit Identifier or UID).....................................................................154
Diagnostics Panel LEDs............................................................................................................154
Customer Replaceable Unit Health LEDs................................................................................155
Diagnostics.....................................................................................................................................155
Online Diagnostics/Exercisers.......................................................................................................155
Online Support Tool Availability.............................................................................................156
Online Support Tools List.........................................................................................................156
Linux Online Support Tools..........................................................................................................156
Offline Support Tools List..............................................................................................................156
General Diagnostic Tools...............................................................................................................157
Fault Management Overview........................................................................................................157
HP-UX Fault Management............................................................................................................157
WBEM indication providers and EMS Hardware Monitors....................................................158
Errors and Reading Error Logs...........................................................................................................158
Event Log Definitions....................................................................................................................158
Using Event Logs...........................................................................................................................158
iLO 2 MP Event Logs.....................................................................................................................159
System Event Log (SEL) Review....................................................................................................159
Supported Configurations..................................................................................................................161
System Block Diagram...................................................................................................................161
System Build-Up Troubleshooting Procedure...............................................................................164
CPU/Memory/SBA..............................................................................................................................165
Troubleshooting the CPU..............................................................................................................166
IPF Processor Load Order........................................................................................................166
Processor Module Behaviors....................................................................................................166
Customer Messaging Policy.....................................................................................................166
Troubleshooting Memory..............................................................................................................168
Memory DIMM Load Order....................................................................................................169
Memory Subsystem Behaviors.................................................................................................169
Customer Messaging Policy.....................................................................................................169
Troubleshooting SBA.....................................................................................................................171
Power Subsystem (BPS and I/O VRM)................................................................................................171
Power Subsystem Behavior...........................................................................................................171
Power LED/Switch.........................................................................................................................172
Power Supply Power LED.............................................................................................................172
I/O VRM.........................................................................................................................................173
Table of Contents 7
Cooling Subsystem..............................................................................................................................173
Cooling Subsystem Behavior.........................................................................................................174
Common I/O Backplane (LBAs/Ropes/PDH/PCI-X Slots)..................................................................174
I/O Subsystem Behaviors...............................................................................................................174
Customer Messaging Policy..........................................................................................................174
Management Subsystem (iLO 2 MP/BMC).........................................................................................178
Manageability LAN LED on the Core I/O board CRU bulkhead.................................................178
Manageability Reset Button on Core I/O Board FRU’s Bulkhead.................................................178
Manageability Status LED.............................................................................................................179
I/O Subsystem (SAS/SATA/SCSI/DVD/HDD/Core I/O).....................................................................179
Verifying Hard Disk Drive Operation...........................................................................................179
LAN LEDs......................................................................................................................................180
HBA Bulkhead LAN LEDs.......................................................................................................180
LAN A Connector LEDs.....................................................................................................180
LAN B Connector LEDs......................................................................................................180
Booting ...............................................................................................................................................180
Firmware.............................................................................................................................................181
Identifying and Troubleshooting Firmware Problems..................................................................182
Updates..........................................................................................................................................182
Server Interface (System Console)......................................................................................................182
Troubleshooting Tips.....................................................................................................................183
Environment .......................................................................................................................................183
Reporting Your Problems to HP.........................................................................................................183
Online Support..............................................................................................................................183
Phone Support...............................................................................................................................184
Information to Collect Before you Contact Support......................................................................184
6 Removing and Replacing Server Components.......................................................185
Required Service Tools........................................................................................................................185
Safety Information...............................................................................................................................186
HP Integrity rx6600 Component Classification..................................................................................186
Hot-Swappable Components.........................................................................................................186
Hot-Pluggable Components..........................................................................................................187
Cold-Swappable Components.......................................................................................................187
Accessing a Rack-Installed Server.......................................................................................................187
Extending the Server from the Rack..............................................................................................188
Inserting the Server into the Rack..................................................................................................188
Accessing a Pedestal-Installed Server.................................................................................................188
Removing and Replacing the Top Cover............................................................................................189
Removing the Top Cover...............................................................................................................189
Replacing the Top Cover................................................................................................................190
Removing and Replacing the Memory Carrier Assembly Cover.......................................................191
Removing the Memory Carrier Assembly Cover..........................................................................191
Replacing the Memory Carrier Assembly Cover..........................................................................191
Removing and Replacing a Hot-Swappable Chassis Fan Unit...........................................................192
Removing an Internal Hot-Swappable Chassis Fan Unit..............................................................192
Replacing an Internal Hot-Swappable Chassis Fan Unit..............................................................193
Removing a Rear External Hot-Swappable Chassis Fan Unit.......................................................194
Replacing a Rear External Hot-Swappable Chassis Fan Unit.......................................................195
Removing and Replacing a Hot-Swappable Power Supply...............................................................195
Power Supply Loading Guidelines................................................................................................196
Removing a Hot-Swappable Power Supply..................................................................................196
Replacing a Hot-Swappable Power Supply...................................................................................197
Removing and Replacing a Hot-Swappable Disk Drive Filler...........................................................197
8 Table of Contents
Removing a Hot-Swappable Disk Drive Filler..............................................................................198
Replacing a Hot-Swappable Disk Drive Filler...............................................................................198
Removing and Replacing a Hot-Pluggable Disk Drive......................................................................199
Removing a Hot-Pluggable Disk Drive.........................................................................................199
Disk Drive Load Order..................................................................................................................201
Replacing a Hot-Pluggable Disk Drive..........................................................................................202
Removing and Replacing PCI/PCI-X/PCIe Card Dividers.................................................................202
Removing a PCI/PCI-X/PCIe Card Divider...................................................................................203
Replacing a PCI/PCI-X/PCIe Card Divider...................................................................................204
Removing and Replacing a Hot-Pluggable PCI/PCI-X/PCIe Card.....................................................204
PCI/PCI-X/PCIe Configurations....................................................................................................206
PCI/PCI-X/PCIe Card Path Logging.........................................................................................208
Online Addition (OLA)..................................................................................................................211
Online Replacement (OLR)............................................................................................................212
Removing a PCI/PCI-X/PCIe Card Offline....................................................................................213
Installing a PCI/PCI-X/PCIe Card Offline.....................................................................................214
Removing and Replacing the DVD Drive...........................................................................................215
Removing the DVD Drive..............................................................................................................215
Replacing the DVD Drive..............................................................................................................216
Removing and Replacing the Memory Carrier Assembly..................................................................216
Removing the Memory Carrier Assembly....................................................................................216
Replacing the Memory Carrier Assembly.....................................................................................218
Removing and Replacing System Memory........................................................................................218
Removing System Memory...........................................................................................................218
Memory Installation Conventions.................................................................................................221
Supported DIMM Sizes and Memory Configurations.............................................................221
Memory Load Order................................................................................................................222
Memory Loading Rules and Guidelines..................................................................................223
Installing Memory.........................................................................................................................224
Removing and Replacing the Front Bezel...........................................................................................226
Removing the Front Bezel..............................................................................................................227
Replacing the Front Bezel..............................................................................................................227
Removing and Replacing the Processor Board Assembly..................................................................228
Removing the Processor Board Assembly.....................................................................................228
Replacing the Processor Board Assembly.....................................................................................229
Removing and Replacing a Dual-Core Processor...............................................................................230
Processor Load Order....................................................................................................................230
Required Tools...............................................................................................................................231
Removing a Dual-Core Processor..................................................................................................231
Installing a Dual-Core Processor...................................................................................................235
Removing and Replacing the I/O Board Assembly............................................................................237
Removing the I/O Board Assembly...............................................................................................237
Replacing the I/O Board Assembly................................................................................................239
Removing and Replacing the System Battery.....................................................................................241
Removing the System Battery........................................................................................................241
Replacing the System Battery........................................................................................................243
Removing and Replacing the I/O Voltage Regulator Module............................................................243
Removing the I/O VRM.................................................................................................................243
Replacing the I/O VRM..................................................................................................................244
Removing and Replacing the Trusted Platform Module....................................................................244
Removing the TPM........................................................................................................................245
Replacing the TPM.........................................................................................................................246
Removing and Replacing the Core I/O Board....................................................................................247
Removing the Core I/O Board.......................................................................................................248
Replacing the Core I/O Board........................................................................................................248
Table of Contents 9
Removing and Replacing the Core I/O Board Battery........................................................................249
Removing the Core I/O Board Battery...........................................................................................249
Replacing the Core I/O Board Battery...........................................................................................250
Removing and Replacing the SAS Core I/O Card..............................................................................250
Removing the SAS Core I/O Card.................................................................................................251
Replacing the SAS Core I/O Card..................................................................................................252
Removing and Replacing the LAN Core I/O Card.............................................................................252
Removing the LAN Core I/O Card................................................................................................252
Replacing the LAN Core I/O Card................................................................................................253
Removing and Replacing the Display Board......................................................................................253
Removing the Display Board.........................................................................................................254
Replacing the Display Board.........................................................................................................256
Removing and Replacing the SAS Backplane Board..........................................................................257
Removing the SAS Backplane Board.............................................................................................258
Replacing the SAS Backplane Board.............................................................................................259
Removing and Replacing the Interconnect Board..............................................................................260
Removing the Interconnect Board.................................................................................................260
Replacing the Interconnect Board..................................................................................................261
Removing and Replacing the Midplane Board...................................................................................262
Removing the Midplane Board.....................................................................................................262
Replacing the Midplane Board......................................................................................................265
A Customer Replaceable Units Information................................................................267
Parts Only Warranty Service...............................................................................................................267
Customer Self Repair..........................................................................................................................267
Parts List..............................................................................................................................................268
B Upgrades....................................................................................................................273
I/O Backplane Upgrade.......................................................................................................................273
I/O Backplane Upgrade Overview................................................................................................273
Required Service Tools.............................................................................................................274
Safety Information....................................................................................................................274
I/O Backplane Upgrade Procedure..........................................................................................275
Installing Core I/O Cards....................................................................................................................279
Installing the HP Eight-Internal Port SAS Host Bus Adapter.......................................................280
Installing the HP Smart Array P600..............................................................................................281
Installing the HP Smart Array P400..............................................................................................282
Completing the Adapter Installation on HP-UX......................................................................283
Completing the Adapter Installation on Windows and Linux................................................285
Installing the HP Smart Array P800..............................................................................................286
Windows Installation...............................................................................................................286
Connecting the Controller to Other Devices.......................................................................287
Completing the Adapter Installation..................................................................................288
Linux Installation.....................................................................................................................288
Connecting the Controller to Other Devices.......................................................................289
Completing the Adapter Installation..................................................................................290
Processor Upgrades............................................................................................................................291
Upgrading Verses Adding On.......................................................................................................291
Firmware........................................................................................................................................292
Operating systems.........................................................................................................................292
C Core I/O Card Utilities.............................................................................................293
Integrated RAID..................................................................................................................................293
10 Table of Contents
Integrated Mirror...........................................................................................................................293
Global Hot Spare............................................................................................................................293
HP 8 Internal Port SAS HBA (SAS Controller)...................................................................................293
MPTUTIL Utility............................................................................................................................293
Flashing Firmware on First Controller.....................................................................................294
Flashing BIOS and EFI Driver on the First Controller.............................................................295
Common Questions About Flashing Firmware.......................................................................295
Viewing the VPD Information for EFI Driver and RISC Firmware.........................................295
EFI Commands..............................................................................................................................295
DRVCFG Utility........................................................................................................................295
Starting the DRVCFG Utility...............................................................................................295
Using the DRVCFG Utility..................................................................................................296
Configuration Utility Screens.............................................................................................296
DRVCFG Screens.................................................................................................................296
CFGGEN Utility.......................................................................................................................309
Starting CFGGEN................................................................................................................309
CFGGEN Operation............................................................................................................310
Rules for creating IM volumes and hot spare disks...........................................................310
CFGGEN Commands..........................................................................................................310
Smart Array P400, P600 and P800 Controllers....................................................................................316
Quick Installation Procedure.........................................................................................................316
Connecting External Storage.........................................................................................................317
SAS Cable Part Numbers...............................................................................................................317
SAUPDATE Utility........................................................................................................................317
Syntax.......................................................................................................................................318
Commands...............................................................................................................................318
List............................................................................................................................................318
UPDATE...................................................................................................................................318
UPDATE all..............................................................................................................................319
HELP or ?..................................................................................................................................319
Error Messages.........................................................................................................................320
EBSU Utility...................................................................................................................................320
Configuring the Array...................................................................................................................322
Comparing the Utilities.................................................................................................................323
ORCA Utility.................................................................................................................................323
Creating a Logical Drive Using ORCA....................................................................................324
ACU Utility....................................................................................................................................325
D Utilities........................................................................................................................327
Extensible Firmware Interface Boot Manager.....................................................................................327
EFI Commands..............................................................................................................................328
EFI/POSSE Commands.......................................................................................................................330
help................................................................................................................................................330
Syntax.......................................................................................................................................330
Parameters................................................................................................................................330
Operation..................................................................................................................................330
baud...............................................................................................................................................332
Syntax.......................................................................................................................................333
Parameters................................................................................................................................333
Operation..................................................................................................................................333
boottest...........................................................................................................................................333
Syntax.......................................................................................................................................333
Parameters................................................................................................................................333
cpuconfig.......................................................................................................................................334
Table of Contents 11
Syntax.......................................................................................................................................334
Parameters................................................................................................................................334
Operation..................................................................................................................................334
conconfig........................................................................................................................................335
Syntax.......................................................................................................................................335
Parameters................................................................................................................................335
Notes.........................................................................................................................................335
ioconfig..........................................................................................................................................336
Syntax.......................................................................................................................................336
Parameters................................................................................................................................336
Operation..................................................................................................................................336
default............................................................................................................................................337
Syntax.......................................................................................................................................337
Parameters................................................................................................................................337
Operation..................................................................................................................................337
errdump.........................................................................................................................................337
Syntax.......................................................................................................................................338
Parameters................................................................................................................................338
Operation..................................................................................................................................338
info.................................................................................................................................................338
Syntax.......................................................................................................................................338
Parameters................................................................................................................................338
lanaddress......................................................................................................................................344
Syntax:......................................................................................................................................344
Parameters................................................................................................................................344
monarch.........................................................................................................................................344
Syntax.......................................................................................................................................344
Parameters................................................................................................................................344
Operation..................................................................................................................................345
pdt..................................................................................................................................................345
Syntax.......................................................................................................................................345
Parameters................................................................................................................................345
Operation..................................................................................................................................345
sysmode.........................................................................................................................................346
Syntax.......................................................................................................................................346
Parameters................................................................................................................................346
Operation..................................................................................................................................346
Specifying SCSI Parameters................................................................................................................347
Using the SCSI Setup Utility..........................................................................................................347
Using the Boot Option Maintenance Menu........................................................................................353
Paths...............................................................................................................................................353
Boot From a File........................................................................................................................353
Add a Boot Option...................................................................................................................354
Delete Boot Option(s)...............................................................................................................355
Change Boot Order...................................................................................................................355
Manage BootNext Setting.........................................................................................................355
Set Auto Boot TimeOut............................................................................................................356
Select Active Console Output Devices.....................................................................................356
Select Active Console Input Devices........................................................................................357
Select Active Standard Error Devices.......................................................................................358
Using the System Configuration Menu.........................................................................................358
Security/Password Menu..........................................................................................................358
Resetting Passwords.................................................................................................................358
iLO 2 MP.............................................................................................................................................358
12 Table of Contents
Index...............................................................................................................................361
E Physical and Environmental Specifications..............................................................367
Table of Contents 13
List of Figures
1-1 I/O Subsystem Block Diagram......................................................................................................22
1-2 PCI/PCI-X/PCIe I/O Subsystem Block Diagram............................................................................23
1-3 Memory Carrier Assembly............................................................................................................26
1-4 Fan Units........................................................................................................................................27
1-5 Power Subsystem Block Diagram.................................................................................................28
1-6 Front Panel Display Card Location...............................................................................................29
1-7 Front Panel Control, Port, and LED Locations..............................................................................32
1-8 Front Control Panel LEDs.............................................................................................................33
1-9 Hot-Pluggable Disk Drive LEDs...................................................................................................34
1-10 Hot-Pluggable Disk Drive Slot Availability LEDs........................................................................35
1-11 DVD Drive.....................................................................................................................................35
1-12 Diagnostic Panel Label and LEDs.................................................................................................36
1-13 Rear Panel Control, Port, and LED Locations...............................................................................37
1-14 Core I/O Board Controls, Ports, and LEDs....................................................................................38
1-15 LAN Link and Status LEDs...........................................................................................................40
1-16 Power LEDs...................................................................................................................................41
3-1 Removing and Replacing the Top Cover......................................................................................51
3-2 Removing and Replacing the Memory Carrier Assembly Cover.................................................52
3-3 Removing and Replacing a Hot-Swappable Power Supply..........................................................54
3-4 Removing and Replacing a Hot-Swappable Disk Drive Filler......................................................55
3-5 Removing and Replacing a Hot-Pluggable Disk Drive................................................................57
3-6 Disk Drive Slot IDs........................................................................................................................57
3-7 PCI/PCI-X Slot Identification and Card Divider Locations..........................................................58
3-8 Removing and Replacing the Memory Carrier Assembly............................................................62
3-9 Memory Carrier Assembly............................................................................................................64
3-10 Memory Carrier Assembly with Side Cover Open.......................................................................65
3-11 48-DIMM Memory Carrier Board Slot IDs....................................................................................67
3-12 Inserting a DIMM into the Memory Board Connector.................................................................70
3-13 Removing the Processor Board Assembly....................................................................................72
3-14 Installing a Dual-Core Processor on the Processor Board.............................................................75
3-15 Processor Power Cable..................................................................................................................76
3-16 Processor Alignment Posts and Lock/Unlock Mechanism...........................................................77
3-17 Processor Alignment Holes and Lock/Unlock Mechanism..........................................................78
3-18 Rear Panel LAN Ports....................................................................................................................81
3-19 Setup Flowchart.............................................................................................................................84
3-20 Server Rear Ports...........................................................................................................................85
3-21 Web Login Page.............................................................................................................................92
3-22 Status Summary Page....................................................................................................................93
3-23 rx6600 Server.................................................................................................................................95
3-24 Accessed Screens in the drvcfg Utility........................................................................................101
3-25 Adapter Properties Screen...........................................................................................................102
3-26 Select New Array Type Screen....................................................................................................103
3-27 EBSU Welcome Screen.................................................................................................................115
3-28 EBSU Main Menu........................................................................................................................115
3-29 EBSU Maintain Firmware Screen................................................................................................116
3-30 EBSU Maintain Firmware Update Screen...................................................................................116
3-31 ORCA Main Menu.......................................................................................................................118
5-1 Front Panel LEDs.........................................................................................................................146
5-2 rx6600 Diagnostic Panel LEDs.....................................................................................................154
5-3 rx6600 Server System with PCI/PCI-X I/O Backplane Block Diagram........................................162
5-4 rx6600 Server System with PCI/PCI-X/PCIe I/O Backplane Block Diagram...............................163
6-1 Rack Screw Location....................................................................................................................188
14 List of Figures
6-2 Server in Pedestal Kit...................................................................................................................189
6-3 Removing and Replacing the Top Cover.....................................................................................190
6-4 Removing and Replacing the Memory Carrier Assembly Cover...............................................191
6-5 Removing and Replacing Hot-Swap Chassis Fan Units (Internal).............................................193
6-6 Fan Unit Release Button (External).............................................................................................194
6-7 Removing and Replacing a Rear External Fan Unit....................................................................195
6-8 Removing and Replacing a Hot-Swappable Power Supply........................................................197
6-9 Removing and Replacing a Hot-Swappable Disk Drive Filler....................................................198
6-10 Removing and Replacing a Hot-Pluggable Disk Drive...............................................................200
6-11 Disk Drive Slot IDs......................................................................................................................201
6-12 PCI/PCI-X/PCIe Card Divider.....................................................................................................204
6-13 PCI/PCI-X/PCIe Slot Identification and Card Divider Locations................................................206
6-14 DVD Drive Removal and Replacement.......................................................................................215
6-15 Removing and Replacing the Memory Carrier Assembly..........................................................217
6-16 Memory Carrier Assembly..........................................................................................................220
6-17 Memory Carrier Assembly with Side Cover Open.....................................................................221
6-18 48-DIMM Memory Carrier Board Slot IDs..................................................................................223
6-19 Inserting DIMM into Memory Board Connector........................................................................226
6-20 Removing the Processor Board Assembly..................................................................................229
6-21 Processor Board Assembly..........................................................................................................232
6-22 Processor Power Cable................................................................................................................233
6-23 Processor Alignment Posts and Lock/Unlock Mechanism..........................................................234
6-24 Processor Alignment Holes and Lock/Unlock Mechanism.........................................................235
6-25 I/O Board Assembly Removal and Replacement........................................................................239
6-26 Battery Location on I/O Board.....................................................................................................242
6-27 Removing and Replacing the I/O VRM.......................................................................................244
6-28 TPM Location on I/O Board.........................................................................................................246
6-29 Battery Location on UCIO Card..................................................................................................250
6-30 Display Board Location...............................................................................................................255
6-31 Display Board Removal and Replacement..................................................................................256
6-32 SAS Backplane Board Removal and Replacement......................................................................259
6-33 Interconnect Board Removal and Replacement..........................................................................261
6-34 Midplane Board Screw Location (Rear of Chassis).....................................................................264
6-35 Midplane Board Screw Location (Front of Chassis)....................................................................264
6-36 Midplane Board...........................................................................................................................265
B-1 OS Reinstallation Flowchart........................................................................................................274
C-1 Accessed Screens in the DRVCFG Utility....................................................................................296
C-2 Adapter Properties Screen...........................................................................................................297
C-3 Select New Array Type Screen....................................................................................................299
C-4 SAS Topology Screen - Expander Closed....................................................................................303
C-5 SAS Topology Screen - Expander Open......................................................................................304
C-6 Advanced Adaptor Properties Screen.........................................................................................306
C-7 EBSU Welcome Screen.................................................................................................................320
C-8 EBSU Main Menu........................................................................................................................321
C-9 EBSU Maintain Firmware Screen................................................................................................321
C-10 EBSU Maintain Firmware Update Screen...................................................................................322
C-11 ORCA Main Menu.......................................................................................................................324
D-1 EFI Boot Sequence.......................................................................................................................327
15
List of Tables
1 Publishing History Details............................................................................................................19
1-1 PCI/PCI-X I/O Rope Groups..........................................................................................................23
1-2 PCI/PCI-X/PCIe I/O Rope Groups.................................................................................................24
1-3 Memory Carrier Configurations...................................................................................................26
1-4 SAS Configurations.......................................................................................................................30
1-5 Front Panel Controls and LEDs.....................................................................................................32
1-6 Hot-Pluggable Disk Drive LEDs...................................................................................................34
1-7 Core I/O Board Ports.....................................................................................................................39
1-8 iLO 2 MP Status and LAN LEDs...................................................................................................39
1-9 System LAN Link Status and Speed LEDs....................................................................................39
1-10 Power Supply LED........................................................................................................................40
1-11 Rear Panel UID LED......................................................................................................................41
2-1 Server Specifications......................................................................................................................43
2-2 Rack- or Pedestal-Installed Server Dimensions and Values..........................................................43
2-3 System Power Specifications.........................................................................................................44
2-4 Additional Component Power Consumption...............................................................................45
2-5 Environmental Specifications (system processing unit with hard disk).......................................45
2-6 Physical and Environmental Specifications..................................................................................46
3-1 Installation Sequence Checklist.....................................................................................................48
3-2 PCI/PCI-X Card Slot Frequency and Bus Mode Compatibility for Shared Slots..........................60
3-3 Supported Memory Carrier Assembly Configurations................................................................61
3-4 Memory Configuration Requirements..........................................................................................65
3-5 48-DIMM Memory Carrier Loading Examples.............................................................................68
3-6 Processor Load Order....................................................................................................................73
3-7 Power States...................................................................................................................................80
3-8 Elements Required to Start a Console Session..............................................................................82
3-9 Setup Checklist..............................................................................................................................82
3-10 Physical Connection Matrix..........................................................................................................85
3-11 LAN Configuration Methods........................................................................................................86
3-12 ARP Ping Commands....................................................................................................................88
3-13 Power States...................................................................................................................................95
3-14 mptutil Commands and Functions...............................................................................................98
3-15 SAS Cable Part Numbers.............................................................................................................111
3-16 ACU and ORCA Supported Features and Procedures...............................................................117
3-17 Server Power Button Functions...................................................................................................120
5-1 Troubleshooting Entry Points .....................................................................................................145
5-2 Front Panel LED States................................................................................................................146
5-3 Basic Entry Class Troubleshooting..............................................................................................147
5-4 Advanced Entry Class Troubleshooting......................................................................................150
5-5 External Health LED States.........................................................................................................152
5-6 VFP External Health Description................................................................................................152
5-7 Internal Health LED States..........................................................................................................153
5-8 VFP Internal Health Description.................................................................................................153
5-9 System Health LED States...........................................................................................................153
5-10 VFP System Health Description..................................................................................................154
5-11 Diagnostics Panel LED States......................................................................................................155
5-12 Online Support Tools List............................................................................................................156
5-13 Offline Support Tools List...........................................................................................................157
5-14 General Diagnostic Tools List......................................................................................................157
5-15 Processor Events That Light Diagnostic Panel LEDs..................................................................167
5-16 Processor Events That May Light Diagnostic Panel LEDs..........................................................167
5-17 Memory Subsystem Events That Light Diagnostic Panel LEDs..................................................170
16 List of Tables
5-18 Memory Subsystem Events That May Light Diagnostic Panel LEDs.........................................170
5-19 Power LED States.........................................................................................................................172
5-20 Power Supply Conditions Mapped to Power LED States...........................................................173
5-21 I/O Power Events That Light Diagnostic Panel LEDs.................................................................173
5-22 Cooling Subsystem Events That Light Diagnostic Panel LEDs...................................................174
5-23 I/O Subsystem Events That Light Diagnostic Panel LEDs..........................................................175
5-24 I/O Card Events That May Light Diagnostic Panel LEDs...........................................................175
5-25 PCI/PCI-X Slot-Rope-ACPI Paths................................................................................................176
5-26 PCI/PCI-X/PCIe Slot-Rope-ACPI Paths.......................................................................................177
5-27 Manageability LAN LED States and Speeds...............................................................................178
5-28 Manageability Status LED...........................................................................................................179
5-29 LAN A Connector LEDs..............................................................................................................180
5-30 LAN B Connector LEDs...............................................................................................................180
5-31 Normal Boot Process LED States.................................................................................................181
5-32 Environmental Specifications......................................................................................................183
6-1 PCI/PCI-X Card Slot Frequency/Bus Mode Compatibility for Shared Slots...............................208
6-2 PCI/PCI-X I/O Paths.....................................................................................................................208
6-3 PCI/PCI-X/PCIe I/O Paths............................................................................................................209
6-4 Supported Memory Carrier Assembly Configurations..............................................................216
6-5 Memory Configuration Requirements........................................................................................221
6-6 48-DIMM Memory Carrier Loading Examples...........................................................................224
6-7 HP Integrity rx6600 Processor Load Order.................................................................................231
6-8 SAS Core I/O Card Locations and SAS Configurations..............................................................251
6-9 SAS Configurations.....................................................................................................................257
A-1 Customer Self Repair Information..............................................................................................268
A-2 Parts List .....................................................................................................................................268
B-1 Slot Speed Comparison and Planning Table...............................................................................273
B-2 I/O Backplane Upgrade Checklist...............................................................................................275
B-3 Cable kit part numbers................................................................................................................280
B-4 SAS cable part numbers...............................................................................................................287
B-5 SAS cable part numbers...............................................................................................................290
B-6 Processor Upgrades.....................................................................................................................291
C-1 mptutil Commands and Functions..............................................................................................294
C-2 SAS Cable Part Numbers.............................................................................................................317
C-3 ACU and ORCA Supported Features and Procedures...............................................................323
D-1 EFI Commands............................................................................................................................328
D-2 Communications Parameters......................................................................................................333
D-3 Server Sockets..............................................................................................................................353
D-4 Server Drives...............................................................................................................................353
D-5 Console Output Devices..............................................................................................................357
D-6 Console Input Devices.................................................................................................................358
E-1 Physical and Environmental Specifications................................................................................367
17
List of Examples
6-1 Enabling the TPM........................................................................................................................247
D-1 help command...........................................................................................................................331
D-2 help bch command...................................................................................................................331
D-3 help configuration command.............................................................................................331
D-4 help cpuconfig command.....................................................................................................332
D-5 help ioconfig command........................................................................................................332
D-6 boottest command...................................................................................................................334
D-7 boottest early_cpu off command....................................................................................334
D-8 cpuconfig command................................................................................................................335
D-9 cpuconfig 2 command............................................................................................................335
D-10 conconfig command................................................................................................................335
D-11 conconfig 2 primary command..........................................................................................336
D-12 conconfig 3 off command...................................................................................................336
D-13 conconfig 3 on command.....................................................................................................336
D-14 ioconfig command...................................................................................................................337
D-15 info all command...................................................................................................................339
D-16 info cpu command...................................................................................................................342
D-17 info mem command...................................................................................................................342
D-18 info io command.....................................................................................................................343
D-19 info boot command................................................................................................................344
D-20 lanaddress command..............................................................................................................344
D-21 monarch command.....................................................................................................................345
D-22 pdt command..............................................................................................................................346
D-23 pdt clear command................................................................................................................346
D-24 sysmode command.....................................................................................................................347
18 List of Examples
About This Document
This document describes how to troubleshoot and diagnose server problems, and remove and
replace server components for the HP Integrity rx6600, Regulatory Model Number: RSVLA-0405.
The document printing date and part number indicate the document’s current edition. The
printing date will change when a new edition is printed. Minor changes may be made at reprint
without changing the printing date. The document part number will change when extensive
changes are made.
Document updatesmay be issued between editions to correct errors or document product changes.
To ensure that you receive the updated or new editions, you should subscribe to the appropriate
product support service. See your HP sales representative for details.
The latest version of this document can be found online at http://www.docs.hp.com.
Intended Audience
This document is intended to provide technical product and support information for authorized
service providers, customer system administrators, and HP support personnel.
Publishing History
Table 1 lists the publishing history details for this document.
Table 1 Publishing History Details
Document Organization
The HP Integrity rx6600 User Service Guide is divided into several chapters; each chapter contains
information about servicing the HP Integrity rx6600. The appendixes contain supplemental
information.
Chapter 1 (page 21) Use this chapter to develop a high-level understanding of themajor
Chapter 2 (page 43) This chapter contains server specifications, including features and
Chapter 3 (page 47) Use this chapter to learn how to install the system.
Chapter 4 (page 125) This chapter provides information on how to boot and shut down
Chapter 5 (page 143) Use this chapter to help troubleshoot and diagnose server problems.
Chapter 6 (page 185) Use this chapter to learn how to remove and replace server
Appendix A (page 267) This appendix lists the field replaceable units (FRU).
Appendix B (page 273) Use this appendix to learn upgrade procedures.
Appendix C (page 293) Use this appendix for information on core I/O cards that need
Appendix D (page 327) This appendix provides information on Extensible Firmware
Publication DateDocument Manufacturing Part Number
September 2006AB464-9003A
February 2007AB464-9003B
November 2007AB464-9003C
August 2008AB464-9003C-ed4
server subsystems.
weights.
the operating systems supported on the server.
components.
additional configuration.
Interface (EFI) Boot Manager.
Intended Audience 19
Typographic Conventions
This document uses the following conventions:
audit(5) An HP-UX manpage. In this example, audit is the name and 5 is the section in
the HP-UX Reference. On the Web and on the Instant Information CD, it may
be a hot link to the manpage itself. From the HP-UX command line, you can
enter “man audit” or “man 5 audit” to view the manpage. See man(1).
Book Title The title of a book. On the Web and on the Instant Information CD, it may be
a hot link to the book itself.
KeyCap The name of a keyboard key. Note that Return and Enter both refer to the
same key.
Emphasis Text that is emphasized.
Bold Text that is strongly emphasized.
ComputerOut
UserInput
Command
Variable
[ ] The contents are optional in formats and command descriptions. If the contents
{ } The contentsare required in formats and command descriptions. If the contents
... The preceding element can be repeated an arbitrary number of times.
| Separates items in a list of choices.
Text displayed by the computer.
Commands and other text that you type.
A command name or qualified command phrase.
The name of a variable that you can replace in a command or function or
information in a display that represents several possible values.
are a list separated by a pipe (|), you must choose one of the items.
are a list separated by a pipe (|), you must choose one of the items.
Related Documents
The HP Server Documentation CD has been provided with the server. It contains a complete
documentation set for the server, including localized versions of key documents.
Other documents in this collection include:
HP Integrity rx6600 Installation Guide
HP Integrity rx6600 Site Preparation Guide
HP Integrity rx6600 Safety and Regulatory Information
HP Integrity rx3600 and rx6600 Console Quick Start
HP Integrity iLO 2 MP Operations Guide
HP Integrity rx3600 & rx6600 Upgrade Guide
The latest versions of these documents, and any updates to these documents are posted under
the Enterprise Servers, Workstations and Systems Hardware collection under HP Integrity rx6600 at
http://www.docs.hp.com.
HP Encourages Your Comments
HP encourages your comments concerning this document. We are truly committed to providing
documentation that meets your needs.
Send comments to: http://docs.hp.com/en/feedback.html.
Include document title, manufacturing part number, and any comments, errors found, or
suggestions for improvement you have concerning this document. Also, please include what we
did right so we can incorporate it into other documents.
20
1 Overview
The HP Integrity rx6600 server is a high performance, high availability server that provides the
following features:
• Form factor - 7U rack- or pedestal installed
• Internal peripherals - serial-attached SCSI (SAS) hard drives and a DVD or DVD+RW drive
• High availability components - N+1 hot-swappable fans, 1+1 hot-swappable power supplies,
and hot-pluggable PCI-X or PCIe cards and SAS drives
• Processors - up to four dual-core Intel® Itanium® processors
• Memory - up to 384 GB of memory
This chapter addresses the following topics:
• “Server Subsystems” (page 21)
— “I/O” (page 21)
— “Processor” (page 25)
— “Memory” (page 25)
— “Cooling” (page 26)
— “Power” (page 27)
— “Front Display Panel, DVD, and Diagnostic Panel” (page 29)
— “Mass Storage” (page 29)
• “Firmware” (page 30)
• “Controls, Ports, and LEDs” (page 31)
— “Front Panel” (page 31)
— “Storage and Media Devices” (page 33)
— “Diagnostic Panel” (page 35)
— “Rear Panel” (page 36)
Server Subsystems
I/O
The server may contain either a 10-slot PCI/PCI-X IOBP or a 10-slot PCI/PCI-X/PCIe IOBP. The
PCI/PCI-X/PCIe interfaces are classified under two major categories: public and private.
Public interfaces Public interfaces are PCI slot connectors that are left available to the customer
for the I/O-card adapters which they wish to install, based on their application needs, provided
the adapter is supported in the slot. All public slots support HP server traditional OL* hot-plug
operations.
Private interfaces Private interfaces are interfaces dedicated to meeting the core I/O system
requirements. There are two types of core-I/O: fast-core and slow-core, also known as the UCIO
(Unified Core-I/O). The Private slots and UCIO are not hot-pluggable.
Hot-plug PCI/PCI-X/PCIe hot-pluggable operations enable you to add or remove a
PCI/PCI-X/PCIe card while the server is powered on. Each slot is physically separated with a
divider that contains a card extraction mechanism and a mechanical retention latch (MRL). Slots
3 - 10 on the 10 slot PCI/PCI-X IOBP and slots 5 - 10 on the 10 slot PCI/PCI-X/PCIe IOBP support
HP-server traditional OL* "hot-plug" operations. PDHP slots rely on the facilities of DHPC (Dual
Hot-plug Controller) FPGAs to enable OL* hot-plug functionality. See Table 1-1 (page 23) and
Table 1-2 (page 24) for details.
LEDs Attention LEDs, doorbell buttons, and power LEDsfor each hot-pluggable PCI/PCI-X/PCIe
slot are located on the PCI bulkhead at the rear of the server chassis.
Server Subsystems 21
Wake on LAN Wake on LAN, a hardware feature that enables the remote power on of computers
PORT
Ports From
Chassis Rear
PORT
Ports From
Chassis Front
Single - Rope
Dual - Rope
Single - Rope
ZX1 PCI-X Bridge
PCIx-66
PCIx-66
PCIx-133
Public PHP I/O Slots
LANLAN
Single - Rope
PCIx-66
Dual - Rope
PCIx-133
Private Slots
Core LAN
Core Disk Storage
DISK
BK PLN
Quad - Rope
PCIx-267
DHPC
DHPC
Quad - Rope
PCIx-267
ZX1 PCI-X Bridge
ZX1 PCI-X Bridge
ZX1 PCI-X Bridge
ZX2 PCI-X Bridge
ZX2 PCI-X Bridge
ZX1 PCI-X Bridge
Common IO Board
Single - Rope
DMD
DMD
Bridge Video
USB
RMP3
BMC
PCI-33
LPC
PDH
ROM
UART
RTC
COM
MP LAN
VGA
USBUSB
Common Display Panel board
DVD
TPM
USB
USB
IPMB
32 bit
ZX1 PCI Bridge
Unified Core IO Board
I2C
COM
SRAM
PCIx-266
PCIx-266
through special network packets, is enabled in core PCI/PCI-X slots one and two.
Ropes Ropes is an HP-proprietary, custom bus interface. It clocks datapackets across long-length
nets using source synchronous clocking schemes. There are eighteen signals bundled per Ropes
group, of which there are a total of 16 in the rx6600. Each group is capable of 0.5GB/s peak
bandwidth. Table 1-1 (page 23) and Table 1-2 (page 24) display the association of the Ropes
group(s) to the PCI slot / HBA to which they connect.
Figure 1-1 (page 22) is a block diagram of the I/O subsystem.
Figure 1-1 I/O Subsystem Block Diagram
Figure 1-2 (page 23) is a block diagram of the PCI/PCI-X/PCIe I/O subsystem.
22 Overview
Figure 1-2 PCI/PCI-X/PCIe I/O Subsystem Block Diagram
PORT
Ports From
Chassis Rear
PORT
Ports From
Chassis Front
Single - Rope
Dual - Rope
Single - Rope
ZX1 PCI-X Bridge
PCIx-66
PCIx-133
PCIx-133
Public PHP I/O Slots
LANLAN
Single - Rope
PCIx-66
Quad - Rope
X8 PCI-E
Private Slots
Core LAN
Core Disk Storage
DISK
BK PLN
Quad - Rope
X8 PCI-E
DHPC
Express
Switch
Dual - Rope
X8 PCI-E
ZX1 PCI-X Bridge
ZX1 PCI-X Bridge
ZX2 PCI-E Bridge
ZX2 PCI-E Bridge
ZX2 PCI-E Bridge
ZX1 PCI-X Bridge
Common IO Board
Single - Rope
DMD
DMD
Bridge Video
USB
SP
BMC
PCI-33
LPC
PDH
ROM
RTC
SRAM
COM
MP LAN
VGA
USBUSB
Common Display Panel board
DVD
TPM
USB
USB
IPMB
32 bit
ZX1 PCI Bridge
Unified Core IO Board
I2C
COM
UART
PCI/PCI-X IOBP
On the 10 slot PCI/PCI-XIOBP there are a total 8 Public slots (PCI-X mode1/2), 2 Private Fast-core
slots (PCI-X mode1), and 1 Private Slow-core UCIO slot (PCI 32-bit/33-MHz). The eight Public
slots are further subdivided into three speed/bandwidth configurations:
• Four PDHP (Public Dual Hot-plug), which operate at 64-bit/66-MHz PCI-X
• Two PSHP-SDR (Public Single Hot-plug - Single Data Rate), operating at 64-bit/133-MHz
PCI-X
• Two PSHP-DDR (Public Single Hot-plug - Double Data Rate), at 64-bit/266-MHz (133-MHz
double clocked) PCI-X mode2
Four 66-MHz PCI/PCI-X slots are shared in groups of two. Shared slots have many speed and
mode change restrictions during hot-plug add or remove operations. For more information on
PCI/PCI-X card configuration and restrictions, see “PCI/PCI-X/PCIe Configurations” (page 206).
Four of the server PCI/PCI-X slots are not shared. Two of the nonshared slots are 133 MHz
PCI/PCI-X, and two of the slots are 266-MHz PCI/PCI-X. Shared slots have many speed and
mode change restrictions during hot-plug add or remove operations.
Table 1-1 PCI/PCI-X I/O Rope Groups
3
4
5
(Quad
Bandwidth)
Bandwidth)
Bandwidth)
FunctionSpeedBitsPCI BusRope NumbersSlot #
Hot Swap /
OL*
NCore I/O (Private)66 MHz64181, 2
YPCI-X 2.0, DDR (Public)266 MHz64212, 13, 14, 15
YPCI-X 2.0, DDR (Public)266 MHz6434, 5, 6, 7 (Quad
Server Subsystems 23
YHigh-Speed PCI-X (Public)133 MHz64410, 11 (Dual
Table 1-1 PCI/PCI-X I/O Rope Groups (continued)
6
Bandwidth)
PCI/PCI-X/PCIe IOBP
On the 10 slot PCI/PCI-X/PCIe IOBP there are a total of eight Public slots (four PCI-X mode 1
and four PCIe), two Private Fast-core slots (PCI/PCI-X mode 1 64-bit/66-MHz), and one Private
Slow-core UCIO slot (PCI 32-bit/33-MHz). The eight Public slots are further divided into three
speed/bandwidth configurations:
• Two PDHP, which operate at 64-bit/66-MHz PCI-X
• Two PSHP, operating at 64-bit/133-MHz PCI-X
• Four PCIe 8-lane (x8) 2.5 Gbps, two of which are switched
The two 66 MHz PCI/PCI-X slots are shared. Shared slots have many speed and mode change
restrictions during hot-plug add or remove operations.
Table 1-2 PCI/PCI-X/PCIe I/O Rope Groups
FunctionSpeedBitsPCI BusRope NumbersSlot #
Hot Swap /
OL*
YHigh-Speed PCI-X (Public)133 MHz6452, 3 (Dual
YGeneral PCI-X (Public)66 MHz64697, 8
YGeneral PCI-X (Public)66 MHz64719, 10
NUCIO (Private)33 MHz3200-
Hot Swap/OLRFunctionSpeedBitsRope NumbersSlot #
NCore I/O (Private)66 MHz6481, 2
2.5 Gbpsx810, 113, 4
Core I/O)
NPCIe x8(Public or Private depending upon
YPCIe x8 (Public)2.5 Gbpsx812, 13, 14, 155
YPCIe x8 (Public)2.5 Gbpsx84, 5, 6, 76
YHigh-Speed PCI-X (Public)133 MHz642, 37
YHigh-Speed PCI-X (Public)133 MHz6498
YGeneral PCI-X (Public)66 MHz6419, 10
NUCIO (Private)33 MHz320-
PCIe MPS Optimization
For PCIe-based systems, each PCIe device has a configurable MPS (maximum payload size)
parameter. Larger MPS values can enable the optimization to gain higher performance.MPS
Optimization is supported on PCIe systems running HP-UX, Open VMS, and Linux. System
firmware level greater than 02.03 performs an optimization during boot time to set the MPS
value to the largest size supported by both a PCIe root port and the devices below it.
The default server state is optimization disabled. When disabled system firmware sets MPS to
the minimum value on each PCIe device.
The info io command will display the current PCIe MPS optimization setting. See “info”
(page 338).
To enable PCIe MPS optimization use the ioconfig mps_optimize command. See “ioconfig”
(page 336).
For non-PCIe systems, ioconfig and info io will not display the MPS optimization policy
setting. TheSet PCIe MPS Optimization boot manager menu also will not be displayed. Running
24 Overview
the ioconfig mps_optimize [on|off] command from a non-PCIe system, the following
output will be displayed:
------------Shell> ioconfig mps_optimize
ioconfig: PCIe MPS optimization is not supported.
Shell> ioconfig mps_optimize on
ioconfig: PCIe MPS optimization is not supported.
Exit status code: Unsupported
Shell>
-----------------
To restore MPS to the default values use the default clear command. See “default” (page 337).
Processor
The server processor subsystem accommodates one, two, three, or four dual-core Itanium
processor modules. The processor subsystem consists of the following components:
• zx2 CEC front side bus, memory, and I/O controller
• System clock generation and distribution
• Circuitry for manageability and fault detection
The zx2 CEC and the processor modules are located on the processor board assembly. Each
processor connectsto the processor board through a zero insertion force(ZIF) socket. The processor
board is mounted on a removable carrier tray that is attached to the processor board access door.
Access this assembly from the front of the server after the memory carrier is removed.
Memory
The server DIMMS are seated on memory boards that are enclosed in an extractable memory
carrier assembly.
Server Subsystems 25
Figure 1-3 Memory Carrier Assembly
Cooling
The memory boards plug directly into sockets on the processor board when the memory carrier
assembly is fully seated.
The server supports the following DIMM sizes:
• 512 MB
• 1 GB
• 2 GB
• 4 GB
• 8 GB
Table 1-3 lists the memory carrier configurations, the minimum and maximum memory for each
configuration.
Table 1-3 Memory Carrier Configurations
Maximum
Configuration
192 GB (six quads:
24x8-GB DIMMs)
384 GB (12 quads:
48x8-GB DIMMs)
Carrier
24-DIMM
48-DIMM
Minimum ConfigurationMemory BoardsMemory
2 GB (one quad: four 512-MB DIMMS)One 24-DIMM memory multiplexer
(mux)-based board
2 GB (one quad: four 512-MB DIMMS)Two 24-DIMM memory multiplexer
(mux)-based boards
There are six system fans assemblies that cool the server. The fans are redundant, hot-swappable,
and interchangeable. You service the fans through the top and rear of the chassis. The fans are
N+1, meaning that the server has six fans, but can operate for a very limited time with five fans
running. If the time threshold is reached, the server automatically shuts down to prevent an
overtemperature condition.
26 Overview
Airflow enters through the front and exhausts out the rear. Airflow baffles might be required to
optimize air circulation within the enclosure. Fans are monitored by the system to indicate
performance.
The airflow strategy contains two thermal zones with three fan units in the bottom zone, and
three fan units in the upper zone along with two power supplies that provide their own individual
cooling. Depending on the configuration, certain component locations will need filler panels.
These installed airflow filler panels and blockers ensure airflow and EMI integrity. Internally,
the system may require a CPU airflow baffle and a memory airflow baffle. Rear I/O bulkhead
covers are required in unoccupied slots. Each hot-swappable fan unit consists of two rotors.
Figure 1-4 Fan Units
Power
The rx6600 has the following cooling components:
• Thermal sensor in the front panel [0xd8] that measures inlet ambient air temperature.
• Up to four dual processors each with its own thermal register [0xd9, 0xda], measureing
internal die temperature.
• Intrusion sensor, located on the common display pca, and held in contact mode by the top
cover.
• Power supplies, each with its own pair of fans running at full speed and its own (virtual)
health sensor [0x40, 0x41].
• Fan units 1 through 6 consisting of 2 individual fans rotors [0xe0 to 0xe9].
The server power subsystem is designed to provide high availability with 1+1 power supply
redundancy. Figure 1-5 (page 28) is a block diagram of the power subsystem, including voltage
labels for each main server subsystem that requires power.
Server Subsystems 27
Figure 1-5 Power Subsystem Block Diagram
Two hot-swappable ac/dc power supplies generate main system power and standby power
voltage. One active power supply is sufficient to operate the system at maximum load. Each
power supply receives ac power through the integrated ac inlet. The system can operate at 180-264
V ac and achieve 1+1 redundancy. The power supplies are power factor corrected and the
maximum dc power output of the power system is 1600 W. Service the hot-swappable power
supplies by sliding them out the rear of the chassis.
Power Button The power button on the rx6600 is a momentary contact push button. The BMC
polls the front panel power button at a rate of at least 2 Hz. The power button is an input to the
System Power State Management. If the system is off, a single button press will turn on the
system. If the system has booted to an OS, and a short button press is detected, a
graceful-shutdown request will be sent to the system by pulsing ACPI_PWR_BTN_L; when the
ACPI bits are set to note the OS has shut down, the BMC will perform a hard power down. If
the system has not booted to an OS, or if a long (5 second) button press is detected, the system
will do an immediate hard power off.
System Power State Management The system power may be controlled from the power button,
an IPMI Chassis command, Wake-On-LAN, loss or gain of ac.
Applying System Power in normal customer usage, the rx6600 runs on 180 to 264V. Standby
power will be supplied on either; hence the BMC will power up when the power supplies are
plugged in. The BPS0_AC_OK and BPS1_AC_OK signals indicate whether the ac voltage to the
power supplies is within the requiredrange. If neither BPS0_AC_OK nor BPS1_AC_OK is asserted,
then the BMC should log an event and prevent the system from turning on.
Procedure 1-1 Power On Sequence
1. Update the cache of DIMM SPD information.
2. Ensure that the memory board is detected and that the cpu board has a processor in socket
28 Overview
0. If these FRUs are not detected the BMC logs an event against the Missing Device sensor
(sensor 0x15).
3. Check fora BPS0_AC_OK or a BPS1_AC_OK signal. If neither is asserted, then the ac supply
has a problem.
4. If any FRUs are missing or both ac supplies are not valid, then return to power off state.
5. Initialize I/O Expander settings prior to turning on power.
6. Set Power Sequencer Order. Set system frequency in Power Sequencer.
7. Pulse BMC_PWR_CMD to tell the Power Sequencer to enable the voltages in the system.
8. Wait for SEQ_MPON to know the power sequencerhas finished and check if SEQ_STATUS=0
for a fault condition. If a fault has occurred, scan sensors for the cause and generate events.
9. Perform any pre-Reset Hardware Setup needed while power is on.
10. Release Reset by setting MPON=1.
Front Display Panel, DVD, and Diagnostic Panel
The front display panel, DVD, and diagnostic panel are supported on a single board, called the
display board, located in the front of the chassis. Service the display board from the top of the
chassis. The front display panel consists of the system status LEDs and a power switch. Use the
front display panel to determine the power status of the server and monitor the server as it
progresses through the boot cycle. Use the various LED states to assist with troubleshooting
system problems.
A slimline DVD drive, or optional DVD+RW drive, is located above the hard disk drives in the
horizontal orientation of the front panel. There is a USB 2.0 port positioned between the DVD
drive and the front display panel.
Each customer replaceable unit (CRU), with the exception of the power supplies, has a unique
set of status indicators located on a diagnostic panel that you view through the top cover. CRUs
include components such as individual memory DIMMs, processors, and fans. LEDs that
correspond to each CRU illuminate when there is a problem.
Figure 1-6 Front Panel Display Card Location
Mass Storage
The server mass storage subsystem (SAS) features the following SAS components:
• Hard drives
• Cables
Server Subsystems 29
• Backplane board
• Core I/O cards
The server supports up to 16 SAS hard drives, each with LEDs that indicate drive activity and
device status, and an LED used to locate each drive. Additionally, there is a set of slot availability
LEDs positioned in the middle of the disk drive bays on the front of the chassis. The illuminated
LEDs indicate connection and power from a SAS controller to the bank of SAS disk drive slots.
The disk drives plug directly into the SAS backplane board. The server ships standard with two
SAS backplane boards and one SAS core I/O card. Two cables connect from the SAS backplane
board to the SAS core I/O card located in PCI/PCI-X slot 1. An optional second SAS core I/O card
is available. Service the SAS backplane board and SAS core I/O card from the top of the chassis.
IMPORTANT: The number and type of SAS core I/O cards determines the SAS configuration.
The SAS configuration affects the location of the LAN core I/O card. In a single SAS core I/O card
configuration, the secondary set of SAS cables connect to the secondary SAS backplane, but are
routed through the server I/O backplane area.
Table 1-4 lists the available SAS configurations.
Table 1-4 SAS Configurations
LAN Core I/O Card LocationSAS Core I/O Card LocationSAS Core I/O Cards
Slot 2Slot 11
Firmware
Firmware consists of many individually linked binary imagesthat are bound together by a single
framework at run time. Internally, the firmware employs a software database called a device
tree to represent the structure of the hardware platform and to provide a means of associating
software elements with hardware functionality.
The firmware incorporates the following main interfaces:
Processor Abstraction Layer (PAL) PAL provides a seamless firmware abstraction between
System Abstraction Layer (SAL) SAL provides a uniform firmware interface, and initializes
Extensible Firmware Interface (EFI) EFI provides an interface between the operating system
Advanced Configuration andPower
Interface (ACPI)
Slot 10Slots 1 and 22
Slot 2Slots 1 and 32
Slot 2Slot 31
Slot 2Slots 3 and 42
the processor and system software and platform firmware.
and configures the platform.
and the platform firmware. EFI uses data tables that contain
platform-related information, and boot and run-time
service calls that are available to the operating system and
its loader to provide a standard environment for booting.
ACPI provides a standard environment for configuring
and managing server systems. ACPI moves system power
configuration and management from the system firmware
to the operating system, and abstracts the interface between
the platform hardware and the operating system software.
This enables each to evolve independently of the other.
The firmware supports HP-UX 11i version 2, June 2006 release, Linux®, Windows®, and
OpenVMS 8.3 operating systems through the Itanium processor family standards and extensions,
30 Overview
Loading...