Page 1
HP ProLiant DL360 Gen9 Server
User Guide
Abstract
This document is for the person who installs, administers, and troubleshoots servers and storage systems. HP assumes you are qualified in the
servicing of computer equipment and trained in recognizing hazards in products with hazardous energy levels.
Part Number: 767927-001
September 2014
Edition: 1
Page 2
© Copyright 2014 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express
warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall
not be liable for technical or editorial errors or omissions contained herein.
Microsoft® and Windows® are U.S. registered trademarks of the Microsoft group of companies.
Page 3
Contents
Component identification ............................................................................................................... 7
Front panel components ................................................................................................................................ 7
Front panel LEDs and buttons ......................................................................................................................... 8
Power fault LEDs ................................................................................................................................. 9
Systems Insight Display LEDs ....................................................................................................................... 10
Systems Insight Display LED combinations ..................................................................................................... 11
Rear panel components .............................................................................................................................. 12
Rear panel LEDs and buttons ....................................................................................................................... 13
Non-hot-plug PCI riser board slot definitions .................................................................................................. 14
System board components .......................................................................................................................... 15
System maintenance switch ............................................................................................................... 16
NMI jumper ..................................................................................................................................... 16
DIMM slots ...................................................................................................................................... 17
Device numbers ......................................................................................................................................... 17
Hot-plug drive LED definitions ...................................................................................................................... 18
Hot-plug fans ............................................................................................................................................. 18
Operations................................................................................................................................. 20
Power up the server .................................................................................................................................... 20
Power down the server ............................................................................................................................... 20
Extend the server from the rack .................................................................................................................... 20
Remove the server from the rack .................................................................................................................. 21
Remove the access panel ............................................................................................................................ 21
Install the access panel ............................................................................................................................... 21
Remove the hot-plug fan .............................................................................................................................. 21
Remove the PCI riser cage ........................................................................................................................... 22
Install the PCI riser cage .............................................................................................................................. 23
Setup ......................................................................................................................................... 25
Support and deployment services ................................................................................................................. 25
Optimum environment................................................................................................................................. 25
Space and airflow requirements ........................................................................................................ 25
Temperature requirements ................................................................................................................. 26
Power requirements .......................................................................................................................... 27
Electrical grounding requirements ...................................................................................................... 27
Rack warnings ........................................................................................................................................... 27
Identifying the contents of the server shipping carton ...................................................................................... 28
Installing hardware options ......................................................................................................................... 28
Installing the server into the rack .................................................................................................................. 28
Powering on and selecting boot options in UEFI Boot Mode ........................................................................... 30
Installing the operating system ..................................................................................................................... 30
Registering the server.................................................................................................................................. 31
Hardware options installation ....................................................................................................... 32
Introduction ............................................................................................................................................... 32
Processor and fan option ............................................................................................................................ 32
Memory options ......................................................................................................................................... 37
Contents 3
Page 4
HP SmartMemory ............................................................................................................................. 38
Memory subsystem architecture ......................................................................................................... 38
Single-, dual-, and quad-rank DIMMs ................................................................................................. 39
DIMM identification .......................................................................................................................... 39
Memory configurations ..................................................................................................................... 40
General DIMM slot population guidelines ........................................................................................... 41
Installing a DIMM ............................................................................................................................. 42
Hot-plug hard drive guidelines ..................................................................................................................... 43
Removing the hard drive blank .......................................................................................................... 44
Installing a hot-plug SAS or SATA drive .............................................................................................. 44
Removing a hot-plug SAS or SATA drive ............................................................................................. 45
2 SFF SAS/SATA drive cage assembly option ............................................................................................... 45
8 SFF Universal Media Bay option ............................................................................................................... 48
4 LFF Universal Media Bay option ................................................................................................................ 51
Full height PCIe x16 riser cage assembly option ............................................................................................ 54
Low profile PCIe slot riser cage option .......................................................................................................... 56
GPU riser and cable option ......................................................................................................................... 57
HP Smart Array P440ar Controller option ..................................................................................................... 59
HP Smart Array P840 Controller board option .............................................................................................. 60
Internal SAS cable option ............................................................................................................................ 63
Systems Insight Display module .................................................................................................................... 65
FlexibleLOM option .................................................................................................................................... 69
Serial cable option ..................................................................................................................................... 70
Expansion board options ............................................................................................................................ 72
Removing the expansion slot blanks ................................................................................................... 72
Installing an expansion board ........................................................................................................... 74
Location discovery services ear option .......................................................................................................... 76
HP Trusted Platform Module option .............................................................................................................. 81
Installing the Trusted Platform Module board ....................................................................................... 82
Retaining the recovery key/password ................................................................................................. 83
Enabling the Trusted Platform Module ................................................................................................. 84
Cabling ..................................................................................................................................... 85
Cabling overview ....................................................................................................................................... 85
2-SFF backplane cabling ............................................................................................................................. 85
8-SFF Universal Media Bay cabling .............................................................................................................. 86
4-LFF Universal Media Bay cabling .............................................................................................................. 87
Smart Array controller cabling ..................................................................................................................... 88
Embedded SATA cabling ............................................................................................................................ 89
Software and configuration utilities ............................................................................................... 91
Server mode .............................................................................................................................................. 91
Product QuickSpecs .................................................................................................................................... 91
HP iLO ...................................................................................................................................................... 91
Active Health System ........................................................................................................................ 92
HP RESTful API support for HP iLO ..................................................................................................... 93
Integrated Management Log .............................................................................................................. 93
HP Insight Remote Support ................................................................................................................ 94
Intelligent Provisioning ................................................................................................................................ 94
HP Insight Diagnostics ...................................................................................................................... 95
Erase Utility ..................................................................................................................................... 95
Scripting Toolkit for Windows and Linux ....................................................................................................... 96
HP Service Pack for ProLiant ........................................................................................................................ 96
HP Smart Update Manager ............................................................................................................... 96
Contents 4
Page 5
HP UEFI System Utilities............................................................................................................................... 97
Using HP UEFI System Utilities ........................................................................................................... 97
Flexible boot control ......................................................................................................................... 97
Restoring and customizing configuration settings ................................................................................. 98
Secure Boot configuration ................................................................................................................. 98
Embedded UEFI shell ........................................................................................................................ 99
Embedded Diagnostics option ........................................................................................................... 99
HP RESTful API support for UEFI ......................................................................................................... 99
Re-entering the server serial number and product ID ............................................................................. 99
Utilities and features ................................................................................................................................. 100
HP Smart Storage Administrator ....................................................................................................... 100
ROMPaq utility ............................................................................................................................... 100
Automatic Server Recovery .............................................................................................................. 100
USB support .................................................................................................................................. 100
Redundant ROM support ................................................................................................................. 101
Keeping the system current ........................................................................................................................ 101
Drivers .......................................................................................................................................... 101
Software and firmware ................................................................................................................... 102
Version control ............................................................................................................................... 102
HP operating systems and virtualization software support for ProLiant servers ....................................... 102
HP Technology Service Portfolio ....................................................................................................... 102
Change control and proactive notification ........................................................................................ 103
Troubleshooting ........................................................................................................................ 104
Troubleshooting resources ......................................................................................................................... 104
Battery replacement .................................................................................................................. 105
Regulatory information .............................................................................................................. 107
Safety and regulatory compliance .............................................................................................................. 107
Belarus Kazakhstan Russia marking ............................................................................................................ 107
Turkey RoHS material content declaration ................................................................................................... 108
Ukraine RoHS material content declaration ................................................................................................. 108
Warranty information ............................................................................................................................... 108
Electrostatic discharge ............................................................................................................... 109
Preventing electrostatic discharge .............................................................................................................. 109
Grounding methods to prevent electrostatic discharge .................................................................................. 109
Specifications ........................................................................................................................... 110
Environmental specifications ...................................................................................................................... 110
Server specifications ................................................................................................................................. 110
Power supply specifications ....................................................................................................................... 111
HP 500W Flex Slot Platinum Hot-plug Power Supply .......................................................................... 111
HP 800W Flex Slot Platinum Hot-plug Power Supply .......................................................................... 112
HP 1400W Flex Slot Platinum Plus Hot-plug Power Supply ................................................................. 112
Hot-plug power supply calculations ............................................................................................................ 113
Support and other resources ......................................................................................................
Before you contact HP .............................................................................................................................. 114
HP contact information .............................................................................................................................. 114
Customer Self Repair ................................................................................................................................ 114
114
Acronyms and abbreviations ...................................................................................................... 122
Documentation feedback ........................................................................................................... 125
Contents 5
Page 6
Index ....................................................................................................................................... 126
Contents 6
Page 7
Component identification
1
2
7
Front panel components
• 8 SFF
Item Description
Serial label pull tab
2
3
4
5
6
7
Front video connector (optional)
USB 2.0 connector (optional)
Optical drive (optional)
Systems Insight Display (optional)
USB 3.0 connector
SAS/SATA/SSD drive bays
• 4 LFF
Item Description
1
3
4
5
6
Optical drive (optional)
Serial label pull tab
Front video connector (optional)
USB 2.0 connector (optional)
Systems Insight Display (optional)
USB 3.0 connector
SAS/SATA/SSD drive bays
Component identification 7
Page 8
Front panel LEDs and buttons
•
•
•
• 8 SFF
Item Description Status
1
2
3
4
Power On/Standby button
and system power LED*
Health LED* Solid green = Normal
NIC status LED* Solid green = Link to network
UID button/LED* Solid blue = Activated
Solid green = System on
Flashing green (1 Hz/cycle per sec) = Performing power on sequence
Solid amber = System in standby
Off = No power present**
Flashing green (1 Hz/cycle per sec) = iLO is rebooting.
Flashing amber = System degraded
Flashing red (1 Hz/cycle per sec) = System critical†
Flashing green (1 Hz/cycle per sec) = Network active
Off = No network activity
Flashing blue:
1 Hz/cycle per sec = Remote management or firmware upgrade
in progress
4 Hz/cycle per sec = iLO manual reboot sequence initiated
8 Hz/cycle per sec = iLO manual reboot sequence in progress
Off = Deactivated
*When all four LEDs described in this table flash simultaneously, a power fault has occurred. For more information, see
"Power fault LEDs (on page 9)."
**Facility power is not present, power cord is not attached, no power supplies are installed, power supply failure has
occurred, or the power button cable is disconnected.
†If the health LED indicates a degraded or critical state, review the system IML or use iLO to review the system health
status.
Component identification 8
Page 9
• 4 LFF
•
•
•
Item Description Status
1
UID button/LED* Solid blue = Activated
Flashing blue:
1 Hz/cycle per sec = Remote management or firmware upgrade
in progress
4 Hz/cycle per sec = iLO manual reboot sequence initiated
8 Hz/cycle per sec = iLO manual reboot sequence in progress
Off = Deactivated
2
3
4
*When all four LEDs described in this table flash simultaneously, a power fault has occurred. For more information, see
"Power fault LEDs (on page 9)."
**Facility power is not present, power cord is not attached, no power supplies are installed, power supply failure has
occurred, or the power button cable is disconnected.
†To identify components in a degraded or critical state, see the Systems Insight Display LEDs, check iLO/BIOS logs, and
reference the server troubleshooting guide.
NIC status LED* Solid green = Link to network
Flashing green (1 Hz/cycle per sec) = Network active
Off = No network activity
Health LED* Solid green = Normal
Flashing green (1 Hz/cycle per sec) = iLO is rebooting.
Flashing amber = System degraded
Flashing red (1 Hz/cycle per sec) = System critical†
Power On/Standby button
and system power LED*
Solid green = System on
Flashing green (1 Hz/cycle per sec) = Performing power on sequence
Solid amber = System in standby
Off = No power present**
Power fault LEDs
The following table provides a list of power fault LEDs, and the subsystems that are affected. Not all power
faults are used by all servers.
Component identification 9
Page 10
Memory
If power is off, the front panel LED is not active. For status,
Subsystem LED behavior
System board
Processor
Riser board PCIe slots
FlexibleLOM
Removable HP Flexible Smart Array
controller/Smart SAS HBA controller
System board PCIe slots
Power backplane or storage backplane
Power supply
1 flash
2 flashes
3 flashes
4 flashes
5 flashes
6 flashes
7 flashes
8 flashes
9 flashes
Systems Insight Display LEDs
The HP Systems Insight Display LEDs represent the system board layout. The display provides status for all
internal LEDs and enables diagnosis with the access panel installed. To view the LEDs, access the HP Systems
Insight Display.
Description Status
Processor LEDs
DIMM LEDs
Fan LEDs
NIC LEDs
Off = Normal
Amber = Failed processor
Off = Normal
Amber = Failed DIMM or configuration issue
Off = Normal
Amber = Failed fan or missing fan
Off = No link to network
Solid green = Network link
Flashing green = Network link with activity
see "Rear panel LEDS and buttons (on page 13)."
Component identification 10
Page 11
Description Status
•
•
•
•
PCI riser (amber)
•
•
•
Power supply LEDs
PCI riser LED
Over temp LED
Amp Status LED
Power cap LED
Off = Normal
Amber = Failed power supply
Off = Normal
Amber = Incorrectly installed PCI riser cage
Off = Normal
Amber = High system temperature detected
Off = AMP modes disabled
Solid green = AMP mode enabled
Solid amber = Failover
Flashing amber = Invalid configuration
Off = System is in standby, or no cap is set.
Solid green = Power cap applied
For more information, see "Systems Insight Display LED combinations (on page 11)."
Systems Insight Display LED combinations
When the health LED on the front panel illuminates either amber or red, the server is experiencing a health
event. Combinations of illuminated Systems Insight Display LEDs, the system power LED, and the health LED
indicate system status.
Systems Insight Display
LED and color
Processor (amber)
Processor (amber)
DIMM (amber)
DIMM (amber)
Over temp (amber)
Over temp (amber)
Fan (amber)
Fan (amber)
Power supply (amber)
Power supply (amber)
Health LED
System power
LED
Red Amber One or more of the following conditions may
Amber Green Processor in socket X is in a pre-failure
Red Green One or more DIMMs have failed.
Amber Green DIMM in slot X is in a pre-failure condition.
Amber Green The Health Driver has detected a cautionary
Red Amber The server has detected a hardware critical
Red Green The PCI riser cage is not seated properly.
Amber Green One fan has failed or has been removed.
Red Green Two or more fans have failed or been
Red Amber One or more of the following conditions may
Amber Green One or more of the following conditions may
Status
exist:
Processor in socket X has failed.
Processor X is not installed in the socket.
Processor X is unsupported.
ROM detects a failed processor during
POST.
condition.
temperature level.
temperature level.
removed.
exist:
Only one power supply is installed and
that power supply is in standby.
Power supply fault
System board fault
Component identification 11
Page 12
•
•
•
•
Power cap (green)
If more than one DIMM slot LED is illuminated, further troubleshooting is required.
12
Systems Insight Display
LED and color
Health LED
System power
LED
Status
exist:
Redundant power supply is installed and
only one power supply is functional.
AC power cord is not plugged into
redundant power supply.
Redundant power supply fault
Power supply mismatch at POST or
power supply mismatch through hot-plug
addition
Power cap (off)
Power cap (green)
— Amber Standby
— Flashing green Waiting for power
— Green Power is available.
Power cap (flashing
amber)
IMPORTANT:
— Amber Power is not available.
Test each bank of DIMMs by removing all other DIMMs. Isolate the failed DIMM by replacing
each DIMM in a bank with a known working DIMM.
Rear panel components
Item Description
1
2
3
4
5
6
7
8
9
10
11
Slot 1 PCIe3 x16 (16, 8, 4, 1)
Slot 2 PCIe 3 x8 ( 8, 4, 1)
Slot 3 PCIe 3 x16 ( 16, 8, 4, 1)*
Power supply 2
Power supply 1
Video connector
NIC connector 4
NIC connector 3
NIC connector 2
NIC connector 1
iLO 4 connector
Serial connector (optional)
Component identification 12
Page 13
Item Description
•
•
•
•
•
13
14
*The slot 3 PCIe 3 riser is optional and requires a second processor before installation.
USB 3.0 connectors
FlexibleLOM bay
Rear panel LEDs and buttons
Item Description Status
1
2L
2R
3
4
UID LED Solid blue = Identification is activated.
Flashing blue = System is being managed
remotely.
Off = Identification is deactivated.
HP iLO/standard
NIC activity LED
HP iLO/standard
NIC link LED
Power supply 2 LED Solid green = Normal
Solid green = Activity exists.
Flashing green = Activity exists.
Off = No activity exists.
Solid green = Link exists.
Off = No link exists.
Off = One or more of the following conditions
exists:
AC power unavailable
Power supply failed
Power supply in standby mode
Power supply exceeded current limit
Power supply 1 LED Solid green = Normal
Off = One or more of the following conditions
exists:
AC power unavailable
Component identification 13
Page 14
Item Description Status
•
•
•
Power supply failed
Power supply in standby mode
Power supply exceeded current limit
Non-hot-plug PCI riser board slot definitions
• Primary riser cage connector, connected to processor 1 or the southbridge
PCIe 3-slot riser cage* PCIe 2-slot x16 riser cage
1 - 3/4L/FH
2 - LP
3 - LP/3/4L/FH
*The server ships with one PCIe3 riser cage installed in the primary riser cage connector.
**The x16 PCIe3 riser cage is optional and can be converted to a FH riser. This conversion requires a
second processor to be installed.
Notes:
PCIe3 x8 (8,4,2,1) PCIe3 x16 (16,8,4,2,1)
PCIe3 x8 (8,4,2,1) PCIe3* x8 (8,4,2,1)
PCIe3 x16 (16, 8,4,2,1)** —
• "Primary" denotes the riser cage is installed in the primary riser connector.
• "Secondary" denotes the riser cage is installed in the secondary riser connector.
• Installing the riser cages listed in the table above in either the primary or secondary riser connectors
determines the form factor of the PCI cards supported by those riser cages.
• FL/FH denotes full-length, full-height. HL/FH denotes half-length, full-height. LP denotes low profile.
Component identification 14
Page 15
System board components
10
15
Item Description
1
2
3
4
5
6
7
8
9
11
12
13
14
16
17
18
19
20
FlexibleLOM connector
Primary (processor 1) PCI riser connector
System maintenance switch
Front VGA/USB 2.0 connector
x4 SATA port 1
x4 SATA port 2
Backplane presence detect connector
Optical/SATA port 5
SATA port 4
Front power/USB 3.0 connector
HP Smart Storage Battery connector
Optional Location Discovery Services connector
Drive backplane power connector
MicroSD card slot
Dual internal USB 3.0 connector
Smart Array/HBA connector
Secondary (processor 2) PCI riser connector
System battery
TPM connector
Optional serial port connector
Component identification 15
Page 16
On = ROM reads system configuration
System maintenance switch
Position Default Function
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
Off Off = HP iLO security is enabled.
On = HP iLO security is disabled.
Off Off = System configuration can be
changed.
On = System configuration is locked.
Off Reserved
Off Reserved
Off Off = Power-on password is enabled.
On = Power-on password is disabled.
Off Off = No function
as invalid.
Off Off = Set default boot mode to UEFI.
On = Set default boot mode to legacy.
— Reserved
— Reserved
— Reserved
— Reserved
— Reserved
To access the redundant ROM, set S1, S5, and S6 to on.
When the system maintenance switch position 6 is set to the On position, the system is prepared to erase all
system configuration settings from both CMOS and NVRAM.
CAUTION: Clearing CMOS and/or NVRAM deletes configuration information. Be sure to
properly configure the server or data loss could occur.
IMPORTANT: Before using the S7 switch to change to Legacy BIOS Boot Mode, be sure the HP
Dynamic Smart Array B140i Controller is disabled. Do not use the B140i controller when the
server is in Legacy BIOS Boot Mode.
NMI jumper
The NMI jumper allows administrators to perform a memory dump before performing a hard reset. Crash
dump analysis is an essential part of eliminating reliability problems, such as hangs or crashes in OSs, device
drivers, and applications. Many crashes can freeze a system, requiring you to do a hard reset. Resetting the
system erases any information that would support root cause analysis.
Systems running Microsoft® Windows® experience a blue-screen trap when the OS crashes. When this
happens, Microsoft® recommends that system administrators perform an NMI event by temporarily shorting
the NMI header with a jumper. The NMI event enables a hung system to become responsive again.
Component identification 16
Page 17
DIMM slots
DIMM slots are numbered sequentially (1 through 12) for each processor. The supported AMP modes use the
letter assignments for population guidelines.
Device numbers
• 4 LFF configuration
• 8 SFF configuration
Component identification 17
Page 18
• 10 SFF configuration
The drive is rebuilding or performing a RAID migration, strip size
Hot-plug drive LED definitions
Item LED Status Definition
1
2
3
4
Locate Solid blue The drive is being identified by a host application.
Flashing blue The drive carrier firmware is being updated or requires an update.
Activity ring Rotating green Drive activity
Off No drive activity
Do not remove Solid white Do not remove the drive. Removing the drive causes one or more of
the logical drives to fail.
Off Removing the drive does not cause a logical drive to fail.
Drive status Solid green The drive is a member of one or more logical drives.
Flashing green
migration, capacity expansion, or logical drive extension, or is
erasing.
Flashing
amber/green
The drive is a member of one or more logical drives and predicts
the drive will fail.
Flashing amber The drive is not configured and predicts the drive will fail.
Solid amber The drive has failed.
Off The drive is not configured by a RAID controller.
Hot-plug fans
CAUTION: To avoid damage to server components, fan blanks must be installed in fan bays 1
and 2 in a single-processor configuration.
Component identification 18
Page 19
The only two valid fan configurations are listed in the following table.
Configuration Fan bay 1 Fan bay 2 Fan bay 3 Fan bay 4 Fan bay 5 Fan bay 6 Fan bay 7
1 processor
Fan blank Fan blank Fan Fan Fan Fan Fan
2 processors
Fan Fan Fan Fan Fan Fan Fan
For a single-processor configuration, five fans and two blanks are required in specific fan bays for
redundancy.
For a dual-processor configuration, seven fans are required for redundancy. A fan failure or missing fan
causes a loss of redundancy. A second fan failure or missing fan causes an orderly shutdown of the server.
The high performance fan option might be necessary for the following installations:
• LR DIMM configuration
• ASHRAE compliant configurations
For more information, see the HP website (http://www.hp.com/servers/ASHRAE).
The server supports variable fan speeds. The fans operate at minimum speed until a temperature change
requires a fan speed increase to cool the server. The server shuts down during the following
temperature-related scenarios:
• At POST and in the OS, HP iLO performs an orderly shutdown if a cautionary temperature level is
detected. If the server hardware detects a critical temperature level before an orderly shutdown occurs,
the server performs an immediate shutdown.
• When the Thermal Shutdown feature is disabled in RBSU, HP iLO does not perform an orderly shutdown
when a cautionary temperature level is detected. Disabling this feature does not disable the server
hardware from performing an immediate shutdown when a critical temperature level is detected.
CAUTION: A thermal event can damage server components when the Thermal Shutdown feature
is disabled in RBSU.
Component identification 19
Page 20
Operations
Power up the server
To power up the server, press the Power On/Standby button.
Power down the server
Before powering down the server for any upgrade or maintenance procedures, perform a backup of critical
server data and programs.
IMPORTANT: When the server is in standby mode, auxiliary power is still being provided to the
To power down the server, use one of the following methods:
• Press and release the Power On/Standby button.
• Press and hold the Power On/Standby button for more than 4 seconds to force the server to enter
• Use a virtual power button selection through HP iLO.
system.
This method initiates a controlled shutdown of applications and the OS before the server enters standby
mode.
standby mode.
This method forces the server to enter standby mode without properly exiting applications and the OS.
If an application stops responding, you can use this method to force a shutdown.
This method initiates a controlled remote shutdown of applications and the OS before the server enters
standby mode.
Before proceeding, verify the server is in standby mode by observing that the system power LED is amber.
Extend the server from the rack
NOTE: If the optional cable management arm option is installed, you can extend the server
without powering down the server or disconnecting peripheral cables and power cords. These
1. Power down the server (on page 20).
2. Disconnect all peripheral cables and power cords.
3. Loosen the front panel thumbscrews.
4. Extend the server on the rack rails until the server rail-release latches engage.
steps are only necessary with the standard cable management solution.
WARNING: To reduce the risk of personal injury or equipment damage, be sure that the rack is
adequately stabilized before extending a component from the rack.
Operations 20
Page 21
WARNING: To reduce the risk of personal injury, be careful when pressing the server rail-release
latches and sliding the server into the rack. The sliding rails could pinch your fingers.
5. After performing the installation or maintenance procedure, slide the server into the rack:
a. Slide the server fully into the rack.
b. Secure the server by tightening the thumbscrews.
6. Connect the peripheral cables and power cords.
Remove the server from the rack
To remove the server from an HP, Compaq branded, telco, or third-party rack:
1. Power down the server (on page 20).
2. Extend the server from the rack (on page 20).
3. Disconnect the cabling and remove the server from the rack. For more information, see the
documentation that ships with the rack mounting option.
4. Place the server on a sturdy, level surface.
Remove the access panel
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
To remove the component:
internal system components to cool before touching them.
CAUTION: Do not operate the server for long periods with the access panel open or removed.
Operating the server in this manner results in improper airflow and improper cooling that can
lead to thermal damage.
1. Power down the server (on page 20).
2. Extend the server from the rack (on page 20).
3. Open or unlock the locking latch, slide the access panel to the rear of the chassis, and remove the
access panel.
Install the access panel
1. Place the access panel on top of the server with the hood latch open. Allow the panel to extend past the
rear of the server approximately 1.25 cm (0.5 in).
2. Push down on the hood latch. The access panel slides to a closed position.
3. Tighten the security screw on the hood latch.
Remove the hot-plug fan
To remove the component:
1. Do one of the following:
o Extend the server from the rack (on page 20).
Operations 21
Page 22
o
Remove the server from the rack (on page 21).
2. Remove the access panel (on page 21).
3. Remove the fan.
CAUTION: Do not operate the server for long periods with the access panel open or removed.
Operating the server in this manner results in improper airflow and improper cooling that can
lead to thermal damage.
IMPORTANT: For optimum cooling, install fans in all primary fan locations. For more
information, refer to the fan locations table.
To replace the component, reverse the removal procedure.
Remove the PCI riser cage
Operations 22
Page 23
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Remove the PCI riser cage.
Install the PCI riser cage
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
Operations 23
Page 24
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Install the PCI riser cage.
6. Install the access panel (on page 21).
7. Install the server into the rack.
8. Connect each power cord to the server.
9. Connect each power cord to the power source.
10. Power up the server (on page 20).
Operations 24
Page 25
Setup
Support and deployment services
Delivered by experienced, certified engineers, HP Care Pack services help you keep your servers up and
running with support packages tailored specifically for HP ProLiant systems. HP Care Packs let you integrate
both hardware and software support into a single package. A number of service level options are available
to meet your needs.
HP Care Pack Services offer upgraded service levels to expand your standard product warranty with
easy-to-buy, easy-to-use support packages that help you make the most of your server investments. Some of
the Care Pack services are:
• Hardware support
o 6-Hour Call-to-Repair
o 4-Hour 24x7 Same Day
o 4-Hour Same Business Day
• Software support
o Microsoft
o Linux
o HP ProLiant Essentials (HP SIM and RDP)
o VMware
• Integrated hardware and software support
o Proactive Care
o Collaborative Support
o Support Plus
o Support Plus 24
• Startup and implementation services for both hardware and software
For more information on HP Care Pack Services, see the HP website
(http://www.hp.com/services/carepack).
Optimum environment
When installing the server in a rack, select a location that meets the environmental standards described in
this section.
Space and airflow requirements
To allow for servicing and adequate airflow, observe the following space and airflow requirements when
deciding where to install a rack:
Setup 25
Page 26
• Leave a minimum clearance of 63.5 cm (25 in) in front of the rack.
• Leave a minimum clearance of 76.2 cm (30 in) behind the rack.
• Leave a minimum clearance of 121.9 cm (48 in) from the back of the rack to the back of another rack
or row of racks.
HP servers draw in cool air through the front and expel warm air through the rear. Therefore, the front and
rear rack doors must be adequately ventilated to allow ambient room air to enter the cabinet, and the rear
door must be adequately ventilated to allow the warm air to escape from the cabinet.
CAUTION: To prevent improper cooling and damage to the equipment, do not block the
When vertical space in the rack is not filled by a server or rack component, the gaps between the
components cause changes in airflow through the rack and across the servers. Cover all gaps with blanking
panels to maintain proper airflow.
The 9000 and 10000 Series Racks provide proper server cooling from flow-through perforations in the front
and rear doors that provide 64 percent open area for ventilation.
ventilation openings.
CAUTION: Always use blanking panels to fill empty vertical spaces in the rack. This arrangement
ensures proper airflow. Using a rack without blanking panels results in improper cooling that can
lead to thermal damage.
CAUTION: When using a Compaq branded 7000 series rack, install the high airflow rack door
insert (PN 327281-B21 for 42U rack, PN 157847-B21 for 22U rack) to provide proper
front-to-back airflow and cooling.
CAUTION: If a third-party rack is used, observe the following additional requirements to ensure
adequate airflow and to prevent damage to the equipment:
• Front and rear doors—If the 42U rack includes closing front and rear doors, you must allow
5,350 sq cm (830 sq in) of holes evenly distributed from top to bottom to permit adequate
airflow (equivalent to the required 64 percent open area for ventilation).
• Side—The clearance between the installed rack component and the side panels of the rack
must be a minimum of 7 cm (2.75 in).
Temperature requirements
To ensure continued safe and reliable equipment operation, install or position the system in a well-ventilated,
climate-controlled environment.
The maximum recommended ambient operating temperature (TMRA) for most server products is 35°C
(95°F). The temperature in the room where the rack is located must not exceed 35°C (95°F).
For information on ambient operating temperature support above 35°C, see the HP website
(http://www.hp.com/servers/ASHRAE).
CAUTION: To reduce the risk of damage to the equipment when installing third-party options:
• Do not permit optional equipment to impede airflow around the server or to increase the
internal rack temperature beyond the maximum allowable limits.
• Do not exceed the manufacturer’s TMRA.
Setup 26
Page 27
Power requirements
Installation of this equipment must comply with local and regional electrical regulations governing the
installation of information technology equipment by licensed electricians. This equipment is designed to
operate in installations covered by NFPA 70, 1999 Edition (National Electric Code) and NFPA-75, 1992
(code for Protection of Electronic Computer/Data Processing Equipment). For electrical power ratings on
options, refer to the product rating label or the user documentation supplied with that option.
WARNING: To reduce the risk of personal injury, fire, or damage to the equipment, do not
overload the AC supply branch circuit that provides power to the rack. Consult the electrical
When installing more than one server, you might need to use additional power distribution devices to safely
provide power to all devices. Observe the following guidelines:
• Balance the server power load between available AC supply branch circuits.
• Do not allow the overall system AC current load to exceed 80% of the branch circuit AC current rating.
authority having jurisdiction over wiring and installation requirements of your facility.
CAUTION: Protect the server from power fluctuations and temporary interruptions with a
regulating uninterruptible power supply. This device protects the hardware from damage caused
by power surges and voltage spikes and keeps the system in operation during a power failure.
• Do not use common power outlet strips for this equipment.
• Provide a separate electrical circuit for the server.
For more information on the hot-plug power supply and calculators to determine server power consumption
in various system configurations, see the HP Power Advisor website
(http://www.hp.com/go/hppoweradvisor).
Electrical grounding requirements
The server must be grounded properly for proper operation and safety. In the United States, you must install
the equipment in accordance with NFPA 70, 1999 Edition (National Electric Code), Article 250, as well as
any local and regional building codes. In Canada, you must install the equipment in accordance with
Canadian Standards Association, CSA C22.1, Canadian Electrical Code. In all other countries, you must
install the equipment in accordance with any regional or national electrical wiring codes, such as the
International Electrotechnical Commission (IEC) Code 364, parts 1 through 7. Furthermore, you must be sure
that all power distribution devices used in the installation, such as branch wiring and receptacles, are listed
or certified grounding-type devices.
Because of the high ground-leakage currents associated with multiple servers connected to the same power
source, HP recommends the use of a PDU that is either permanently wired to the building’s branch circuit or
includes a nondetachable cord that is wired to an industrial-style plug. NEMA locking-style plugs or those
complying with IEC 60309 are considered suitable for this purpose. Using common power outlet strips for
the server is not recommended.
Rack warnings
Setup 27
Page 28
WARNING: To reduce the risk of personal injury or damage to the equipment, be sure that:
Never stand in front of the rack when it is rolling down the ramp from the pallet. Always handle
• The leveling jacks are extended to the floor.
• The full weight of the rack rests on the leveling jacks.
• The stabilizing feet are attached to the rack if it is a single-rack installation.
• The racks are coupled together in multiple-rack installations.
• Only one component is extended at a time. A rack may become unstable if more than one
component is extended for any reason.
WARNING: To reduce the risk of personal injury or equipment damage when unloading a rack:
• At least two people are needed to safely unload the rack from the pallet. An empty 42U rack
can weigh as much as 115 kg (253 lb), can stand more than 2.1 m (7 ft) tall, and might
become unstable when being moved on its casters.
•
the rack from both sides.
Identifying the contents of the server shipping carton
Unpack the server shipping carton and locate the materials and documentation necessary for installing the
server. All the rack mounting hardware necessary for installing the server into the rack is included with the
rack or the server.
The contents of the server shipping carton include:
• Server
• Power cord
• Hardware documentation and software products
• Rack-mounting hardware and documentation
In addition to the supplied items, you might need:
• Operating system or application software
• Hardware options
• Screwdriver
Installing hardware options
Install any hardware options before initializing the server. For options installation information, refer to the
option documentation. For server-specific information, refer to "Hardware options installation (on page 32)."
Installing the server into the rack
To install the server into a rack with square, round, or threaded holes, refer to the instructions that ship with
the rack hardware kit.
If you are installing the server into a telco rack, order the appropriate option kit at the RackSolutions.com
website (http://www.racksolutions.com/hp). Follow the server-specific instructions on the website to install
the rack brackets.
Use the following information when connecting peripheral cables and power cords to the server.
Setup 28
Page 29
ack server installations. A third person may be required to help align the server if the
14
WARNING: This server is very heavy. To reduce the risk of personal injury or damage to the
equipment:
• Observe local occupational health and safety requirements and guidelines for manual
material handling.
• Get help to lift and stabilize the product during installation or removal, especially when the
product is not fastened to the rails. HP recommends that a minimum of two people are required
for all r
server is installed higher than chest level.
• Use caution when installing the server in or removing the server from the rack; it is unstable
when not fastened to the rails.
CAUTION: Always plan the rack installation so that the heaviest item is on the bottom of the rack.
Install the heaviest item first, and continue to populate the rack from the bottom to the top.
1. Install the server and cable management arm into the rack. See the installation instructions that ship with
the selected rail system.
2. Connect the peripheral devices to the server.
Item Description
1
2
3
4
5
6
7
8
9
10
11
12
13
Slot 1 PCIe3 x16 (16, 8, 4, 1)
Slot 2 PCIe 3 x8 ( 8, 4, 1)
Slot 3 PCIe 3 x16 ( 16, 8, 4, 1)*
Power supply 2
Power supply 1
Video connector
NIC connector 4
NIC connector 3
NIC connector 2
NIC connector 1
iLO 4 connector
Serial connector (optional)
USB 3.0 connectors
FlexibleLOM bay
Setup 29
Page 30
*The slot 3 PCIe 3 riser is optional and requires a second processor before installation.
Connect the power cord to the server.
3.
4. Use the Velcro strip to secure the power cord.
5. Connect the power cord to the power source.
Powering on and selecting boot options in UEFI Boot
Mode
On servers operating in UEFI Boot Mode, the boot controller and boot order are set automatically.
1. Press the Power On/Standby button.
2. During the initial boot:
o To modify the server configuration ROM default settings, press the F9 key in the HP ProLiant POST
screen to enter the UEFI System Utilities screen. By default, the System Utilities menus are in the
English language.
o If you do not need to modify the server configuration and are ready to install the system software,
press the F10 key to access Intelligent Provisioning.
For more information on automatic configuration, see the UEFI documentation on the HP website
(http://www.hp.com/go/ProLiantUEFI/docs).
Installing the operating system
This HP ProLiant server does not ship with provisioning media. Everything needed to manage and install the
system software and firmware is preloaded on the server.
To operate properly, the server must have a supported operating system. For the latest information on
operating system support, see the HP website (http://www.hp.com/go/supportos).
To install an operating system on the server, use one of the following methods:
• Intelligent Provisioning—iLO contains Intelligent Provisioning for embedded deployment, updating, and
provisioning capabilities. Intelligent Provisioning can configure the server and install an operating
system.
To install an operating system on the server with Intelligent Provisioning (local or remote):
a. Connect the Ethernet cable between the network connector on the server and a network jack.
b. Press the Power On/Standby button.
c. During server POST, press the F10 key.
d. Complete the initial Preferences and Registration portion of Intelligent Provisioning (on page 94).
Setup 30
Page 31
e.
At the 1 Start screen, click the Configure and Install button.
f. To finish the installation, follow the onscreen prompts. An Internet connection is required to update
the firmware and systems software.
• Remote deployment installation—To deploy an operating system remotely, use Insight Control server
deployment for an automated solution.
For additional system software and firmware updates, download the HP Service Pack for ProLiant from the HP
website (http://www.hp.com/go/spp/download). Software and firmware must be updated before using
the server for the first time, unless any installed software or components require an older version. For more
information, see "Keeping the system current (on page 101)."
For more information on using these installation methods, see the HP website (http://www.hp.com/go/ilo).
Registering the server
To experience quicker service and more efficient support, register the product at the HP Product Registration
website (http://register.hp.com).
Setup 31
Page 32
Hardware options installation
Introduction
If more than one option is being installed, read the installation instructions for all the hardware options and
identify similar steps to streamline the installation process.
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
Processor and fan option
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause electrostatic discharge.
The server supports single-processor and dual-processor operations.
CAUTION: To avoid damage to the processor and system board, only authorized personnel
should attempt to replace or install the processor in this server.
CAUTION: To prevent possible server malfunction and damage to the equipment, multiprocessor
configurations must contain processors with the same part number.
IMPORTANT: If installing a processor with a faster speed, update the system ROM before
installing the processor.
To install a processor:
1. Power down the server (on page 20).
2. Remove all power:
a. Disconnect each power cord from the power source.
b. Disconnect each power cord from the server.
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
4. Remove the access panel (on page 21).
5. Remove the processor blank.
Hardware options installation 32
Page 33
6.
Open each of the processor locking levers in the order indicated in the following illustration, and then
open the processor retaining bracket.
7. Remove the clear processor socket cover. Retain the processor socket cover for future use.
CAUTION: THE PINS ON THE SYSTEM BOARD ARE VERY FRAGILE AND EASILY DAMAGED. To
avoid damage to the system board, do not touch the processor or the processor socket contacts.
Hardware options installation 33
Page 34
8.
Install the processor. Verify that the processor is fully seated in the processor retaining bracket by
visually inspecting the processor installation guides on either side of the processor. THE PINS ON THE
SYSTEM BOARD ARE VERY FRAGILE AND EASILY DAMAGED.
9. Close the processor retaining bracket. When the processor is installed properly inside the processor
retaining bracket, the processor retaining bracket clears the flange on the front of the socket.
CAUTION: Do not press down on the processor. Pressing down on the processor may cause
damage to the processor socket and the system board. Press only in the area indicated on the
processor retaining bracket.
CAUTION: Close and hold down the processor cover socket while closing the processor locking
levers. The levers should close without resistance. Forcing the levers closed can damage the
processor and socket, requiring system board replacement.
Hardware options installation 34
Page 35
10.
Close and hold down the processor cover socket while closing the processor locking
Press and hold the processor retaining bracket in place, and then close each processor locking lever.
Press only in the area indicated on the processor retaining bracket.
CAUTION:
levers. The levers should close without resistance. Forcing the levers closed can damage the
11. Remove the heatsink cover.
processor and socket, requiring system board replacement.
CAUTION: After the cover is removed, do not touch the thermal interface media.
12. Install the heatsink:
a. Position the heatsink on the processor backplate.
Hardware options installation 35
Page 36
b.
Tighten one pair of diagonally opposite screws halfway, and then tighten the other pair of screws.
c. Finish the installation by completely tightening the screws in the same sequence.
13. Remove the fan blanks from locations 1 and 2. For fan location and numbering information, see
"Hot-plug fans (on page 18)" or the label attached to the chassis next to the fans.
Hardware options installation 36
Page 37
14.
Install the fans into locations 1 and 2.
15. Install the access panel (on page 21).
16. Install the server into the rack.
17. Connect each power cord to the server.
18. Connect each power cord to the power source.
19. Press the Power On/Standby button.
20. The server exits standby mode and applies full power to the system. The system power LED changes
from amber to green.
Memory options
IMPORTANT: This server does not support mixing LRDIMMs or RDIMMs. Attempting to mix any
The memory subsystem in this server can support LRDIMMs and RDIMMs:
• RDIMMs offer address parity protection.
• LRDIMMs support higher densities than single- and dual-rank RDIMMs, and higher speeds than
All types are referred to as DIMMs when the information applies to all types. When specified as LRDIMM or
RDIMM, the information applies to that type only. All memory installed in the server must be the same type.
The server supports the following DIMM speeds:
combination of these DIMMs can cause the server to halt during BIOS initialization.
quad-rank RDIMMs. This support enables you to install more high capacity DIMMs, resulting in higher
system capacities and higher bandwidth.
• Single- and dual-rank PC4-2133 (DDR4-2133) RDIMMs operating at up to 2133 MT/s
• Quad-rank PC4L-2133 (DDR4-2133) LRDIMMs operating at up to 2133 MT/s
Speed and capacity
DIMM type DIMM rank DIMM capacity Native speed (MT/s)
RDIMM
Single-rank 8 GB 2133
Hardware options installation 37
Page 38
DIMM type DIMM rank DIMM capacity Native speed (MT/s)
RDIMM
LRDIMM
Depending on the processor model, the number of DIMMs installed, and whether LRDIMMs or RDIMMs are
installed, the memory clock speed can be reduced to 1600 MT/s.
Populated DIMM speed (MT/s)
DIMM type DIMM rank 1 DIMM per channel 2 DIMMs per channel 3 DIMMs per channel
RDIMM
RDIMM
LRDIMM
Single-rank (8 GB)
Dual-rank (16 GB)
Quad-rank (32
GB)
For more information about product features, specifications, options, configurations, and compatibility, see
the product QuickSpecs on the HP website (http://www.hp.com/go/qs).
HP SmartMemory
HP SmartMemory authenticates and unlocks certain features available only on HP Qualified memory and
verifies whether installed memory has passed HP qualification and test processes. Qualified memory is
performance-tuned for HP ProLiant and BladeSystem servers and provides future enhanced support through
HP Active Health and manageability software.
Dual-rank 16 GB 2133
Quad-rank 32 GB 2133
2133 2133 1600
2133 2133 1600
2133 2133 1600
Memory subsystem architecture
The memory subsystem in this server is divided into channels. Each processor supports four channels, and
each channel supports three DIMM slots, as shown in the following table.
Channel Population order Slot number
1
2
3
4
For the location of the slot numbers, see "DIMM slot locations ("DIMM slots" on page 17)."
This multi-channel architecture provides enhanced performance in Advanced ECC mode. This architecture
also enables Online Spare Memory mode.
DIMM slots in this server are identified by number and by letter. Letters identify the population order. Slot
numbers indicate the DIMM slot ID for spare replacement.
A
E
I
B
F
J
C
G
K
D
H
L
12
11
10
9
8
7
1
2
3
4
5
6
Hardware options installation 38
Page 39
Single-, dual-, and quad-rank DIMMs
To understand and configure memory protection modes properly, an understanding of single-, dual-, and
quad-rank DIMMs is helpful. Some DIMM configuration requirements are based on these classifications.
A single-rank DIMM has one set of memory chips that is accessed while writing to or reading from the
memory. A dual-rank DIMM is similar to having two single-rank DIMMs on the same module, with only one
rank accessible at a time. A quad-rank DIMM is, effectively, two dual-rank DIMMs on the same module. Only
one rank is accessible at a time. The server memory control subsystem selects the proper rank within the
DIMM when writing to or reading from the DIMM.
Dual- and quad-rank DIMMs provide the greatest capacity with the existing memory technology. For
example, if current DRAM technology supports 8-GB single-rank DIMMs, a dual-rank DIMM would be 16
GB, and a quad-rank DIMM would be 32 GB.
LRDIMMs are labeled as quad-rank DIMMs; however, they function more like dual-rank DIMMs. There are
four ranks of DRAM on the DIMM, but the LRDIMM buffer creates an abstraction that allows the DIMM to
appear as a dual-rank DIMM to the system. The LRDIMM buffer also isolates the electrical loading of the
DRAM from the system to allow for faster operation. These two changes allow the system to support up to
three LRDIMMs per memory channel, providing for up to 50% greater memory capacity and higher memory
operating speed compared to quad-rank RDIMMs.
DIMM identification
To determine DIMM characteristics, use the label attached to the DIMM and the following illustration and
table.
Item Description Definition
1
2
3
Capacity 8 GB
Rank 1R = Single-rank
Data width x4 = 4-bit
16 GB
32 GB
2R = Dual-rank
4R = Quad-rank
x8 = 8-bit
Hardware options installation 39
Page 40
Item Description Definition
4
Memory generation DDR4
5
6
7
Maximum memory
speed
CAS latency P=15
DIMM type R = RDIMM (registered)
2133 MT/s
L = LRDIMM (load reduced)
For more information about product features, specifications, options, configurations, and compatibility, see
the product QuickSpecs on the HP website (http://www.hp.com/go/qs).
Memory configurations
To optimize server availability, the server supports the following AMP modes:
• Advanced ECC—Provides up to 4-bit error correction and enhanced performance over Lockstep mode.
This mode is the default option for this server.
• Online spare memory—Provides protection against failing or degraded DIMMs. Certain memory is
reserved as spare, and automatic failover to spare memory occurs when the system detects a DIMM that
is degrading. This allows DIMMs that have a higher probability of receiving an uncorrectable memory
error (which would result in system downtime) to be removed from operation.
Advanced Memory Protection options are configured in the BIOS/Platform Configuration (RBSU). If the
requested AMP mode is not supported by the installed DIMM configuration, the server boots in Advanced
ECC mode. For more information, see the HP UEFI System Utilities User Guide for HP ProLiant Gen9 Servers
on the HP website (http://www.hp.com/go/uefi/docs).
Maximum capacity
DIMM type DIMM rank One processor Two processors
RDIMM
RDIMM
LRDIMM
Single-rank (8 GB) 96 GB 192 GB
Dual-rank (16 GB) 192 GB 384 GB
Quad-rank (32 GB) 384 GB 768 GB
For the latest memory configuration information, see the QuickSpecs on the HP website
(http://www.hp.com).
Advanced ECC memory configuration
Advanced ECC memory is the default memory protection mode for this server. Standard ECC can correct
single-bit memory errors and detect multi-bit memory errors. When multi-bit errors are detected using
Standard ECC, the error is signaled to the server and causes the server to halt.
Advanced ECC protects the server against some multi-bit memory errors. Advanced ECC can correct both
single-bit memory errors and 4-bit memory errors if all failed bits are on the same DRAM device on the DIMM.
Advanced ECC provides additional protection over Standard ECC because it is possible to correct certain
memory errors that would otherwise be uncorrected and result in a server failure. Using HP Advanced
Memory Error Detection technology, the server provides notification when a DIMM is degrading and has a
higher probability of uncorrectable memory error.
Hardware options installation 40
Page 41
Online Spare memory configuration
Online spare memory provides protection against degraded DIMMs by reducing the likelihood of
uncorrected memory errors. This protection is available without any operating system support.
Online spare memory protection dedicates one rank of each memory channel for use as spare memory. The
remaining ranks are available for OS and application use. If correctable memory errors occur at a rate
higher than a specific threshold on any of the non-spare ranks, the server automatically copies the memory
contents of the degraded rank to the online spare rank. The server then deactivates the failing rank and
automatically switches over to the online spare rank.
General DIMM slot population guidelines
Observe the following guidelines for all AMP modes:
• Install DIMMs only if the corresponding processor is installed.
• When two processors are installed, balance the DIMMs across the two processors.
• White DIMM slots denote the first slot of a channel (Ch 1-A, Ch 2-B, Ch 3-C, Ch 4-D)
• Do not mix RDIMMs and LRDIMMs.
• When one processor is installed, install DIMMs in sequential alphabetic order: A, B, C, D, E, F, and so
forth.
• When two processors are installed, install the DIMMs in sequential alphabetic order balanced between
the two processors: P1-A, P2-A, P1-B, P2-B, P1-C, P2-C, and so forth.
• When single-rank, dual-rank, and quad-rank DIMMs are populated for two DIMMs per channel or three
DIMMs per channel, always populate the higher number rank DIMM first (starting from the farthest slot).
For example, first quad-rank DIMM, then dual-rank DIMM, and then lastly single-rank DIMM.
• DIMMs should be populated starting farthest from the processor on each channel.
• For DIMM spare replacement, install the DIMMs per slot number as instructed by the system software.
For more information about server memory, see the HP website (http://www.hp.com/go/memory).
DIMM speeds are supported as indicated in the following table.
Populated slots
(per channel)
1
2
2
3
Depending on the processor model, the number of DIMMs installed, and whether LRDIMMs or RDIMMs are
installed, the memory clock speed can be reduced to 1600 MT/s.
Rank Speeds supported (MT/s)
Single-, dual-, or
quad-rank
Single- or dual-rank 2133
Quad-rank 2133
Single-, dual-, or
quad-rank
2133
1600
Advanced ECC population guidelines
For Advanced ECC mode configurations, observe the following guidelines:
• Observe the general DIMM slot population guidelines (on page 41).
Hardware options installation 41
Page 42
• DIMMs may be installed individually.
Online spare population guidelines
For Online Spare memory mode configurations, observe the following guidelines:
• Observe the general DIMM slot population guidelines (on page 41).
• Each channel must have a valid online spare configuration.
• Each channel can have a different valid online spare configuration.
• Each populated channel must have a spare rank. A single dual-rank DIMM is not a valid configuration.
Population order
For memory configurations with a single processor or multiple processors, DIMMs must be populated
sequentially in alphabetical order (A through L).
After installing the DIMMs, use the BIOS/Platform Configuration (RBSU) in the UEFI System Utilities to
configure supported AMP modes.
Installing a DIMM
The server supports up to 24 DIMMs. To install a DIMM:
1. Power down the server (on page 20).
2. Remove all power:
a. Disconnect each power cord from the power source.
b. Disconnect each power cord from the server.
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
4. Open the DIMM slot latches.
Hardware options installation 42
Page 43
5.
Install the DIMM.
6. Install the access panel (on page 21).
7. Install the server into the rack.
8. Connect each power cord to the server.
9. Connect each power cord to the power source.
10. Power up the server (on page 20).
Use the BIOS/Platform Configuration (RBSU) in the UEFI System Utilities to configure the memory mode.
For more information about LEDs and troubleshooting failed DIMMs, see "Systems Insight Display LED
combinations (on page 11)."
Hot-plug hard drive guidelines
When adding hard drives to the server, observe the following general guidelines:
• The system automatically sets all device numbers.
• If only one hard drive is used, install it in the bay with the lowest device number.
• Drives should be the same capacity to provide the greatest storage space efficiency when drives are
grouped together into the same drive array.
Hardware options installation 43
Page 44
Removing the hard drive blank
Remove the component as indicated.
Installing a hot-plug SAS or SATA drive
The server can support 8 SAS or SATA hard drives in a SFF configuration, or 10 drives in a SFF configuration
with the optional hard drive cage. In a LFF configuration, the server can support 4 SAS or SATA hard drives.
To install the component:
1. Remove the drive blank.
2. Prepare the drive.
Hardware options installation 44
Page 45
3.
Install the drive.
Removing a hot-plug SAS or SATA drive
CAUTION: For proper cooling, do not operate the server without the access panel, baffles,
expansion slot covers, or blanks installed. If the server supports hot-plug components, minimize
1. Determine the status of the drive from the hot-plug SAS drive LED combinations ("Hot-plug drive LED
2. Back up all server data on the drive.
3. Remove the drive.
the amount of time the access panel is open.
definitions" on page 18).
2 SFF SAS/SATA drive cage assembly option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
Hardware options installation 45
Page 46
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Remove the universal media bay blank.
6. Install the drive cage assembly, and then tighten the three screws.
Hardware options installation 46
Page 47
7.
Connect the data cables to the SATA storage connector, the SATA optical/storage drive connector, and
the backplane presence detect connector.
8. Connect the power cable to the left power connector on the backplane.
CAUTION: To prevent improper cooling and thermal damage, do not operate the server or the
enclosure unless all drive and device bays are populated with either a component or a blank.
Hardware options installation 47
Page 48
9.
Install drives or blanks in the empty drive bays.
8 SFF Universal Media Bay option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
4. Remove the access panel (on page 21).
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
Hardware options installation 48
Page 49
5.
Remove the universal media bay blank.
6. Install the universal media bay option.
o 8 SFF DVD-RW/VGA/USB
Hardware options installation 49
Page 50
o
8 SFF VGA/USB
7. Route the cables along the edge of the system board, and then connect the cables to the system board.
o 8 SFF DVD-RW/VGA/USB connections
Connect the VGA/USB cable to the Front VGA/USB connector toward the rear of the system board.
Connect the SATA optical cable to the SATA optical/storage drive connector.
o 8 SFF VGA/USB connections
Hardware options installation 50
Page 51
Connect the VGA/USB cable to the Front VGA/USB connector toward the rear of the system board.
8. Install the access panel (on page 21).
9. Slide the server into the rack.
10. Connect all power cords to the server, and then power up the server.
4 LFF Universal Media Bay option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
4. Remove the access panel (on page 21).
5. Remove the bay blanks.
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
Hardware options installation 51
Page 52
o
4 LFF optical drive blank
o 4 LFF VGA/USB blank
6. Install the universal media bay option.
o 4 LFF optical drive.
Hardware options installation 52
Page 53
After installing the 4 LFF optical drive, connect the ODD cable to the rear of the drive.
o 4 LFF VGA/USB
7. Route the cables along the edge of the system board, and then connect the cables to the system board.
o 4 LFF VGA/USB connections
Hardware options installation 53
Page 54
Connect the VGA/USB cable to the Front VGA/USB connector toward the rear of the system board.
o 4 LFF optical drive connection
Connect the SATA optical cable to the SATA optical/storage drive connector.
8. Install the access panel (on page 21).
9. Slide the server into the rack.
10. Connect all power cords to the server, and then power up the server.
Full height PCIe x16 riser cage assembly option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
Hardware options installation 54
Page 55
1.
Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Remove the primary PCI riser cage ("Remove the PCI riser cage" on page 22).
6. If installed, remove the low profile riser board from the secondary riser board slot. Remove the riser
cage bracket from the primary PCIe riser cage.
7. Lift and remove the riser cage latch.
Hardware options installation 55
Page 56
8.
Install the full height PCIe x16 riser cage latch.
9. Align the PCIe x16 riser cage with the corresponding connector on the system board, and then press
down on the riser cage.
If a GPU is installed in the full height PCIe riser cage option, connect the GPU cable to the riser cage
connector and the installed GPU.
CAUTION: To prevent improper cooling and thermal damage, do not operate the server unless
all PCI slots have either an expansion slot cover or an expansion board installed.
Low profile PCIe slot riser cage option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
Hardware options installation 56
Page 57
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Install the low profile riser cage.
GPU riser and cable option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
4. Remove the access panel (on page 21).
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
Hardware options installation 57
Page 58
5. Remove the PCI riser cage (on page 22).
6. Install the GPU in the x16 slot in the primary PCI riser cage position. The PCI riser cage assembly with
the full-length, full-height PCIe3 x16 riser slot is required for GPU installation. For more information, see
"Full height PCIe x16 riser cage assembly option (on page 54)."
Before installing a high-power GPU in the server, be sure that the power supplies support the installation
of the option. Because of the high power requirements for the GPU, a 750-W or higher power supply
might be required. For more information, see the HP Enterprise Configurator website
(http://h30099.www3.hp.com/configurator/).
7. Install the GPU support bracket.
Hardware options installation 58
Page 59
8.
Install the riser cage.
9. Connect the GPU riser cable to the GPU and the primary riser cage PCA.
HP Smart Array P440ar Controller option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
Hardware options installation 59
Page 60
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Install the P440ar controller.
The P440ar controller with cut heatsink allows for clearance when a GPU is installed in the secondary
full height riser cage.
HP Smart Array P840 Controller board option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
Hardware options installation 60
Page 61
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Remove the primary PCI riser cage ("Remove the PCI riser cage" on page 22).
6. Install the P840 controller board.
7. Install the riser board assembly.
Hardware options installation 61
Page 62
8.
Connect the mini-SAS cables from the standard backplane to port one on the optional P840 controller
board.
9. If the optional 2 SFF SAS/SATA backplane is installed, connect the mini-SAS cable from the optional
back plane to port two on the P840 controller board.
Hardware options installation 62
Page 63
Internal SAS cable option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
4. Remove the access panel (on page 21).
5. Remove the primary PCI riser cage ("Remove the PCI riser cage" on page 22).
6. Remove the slot two expansion board blank.
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
The controller board can be installed in slot 1 or slot 2.
Hardware options installation 63
Page 64
7.
Install the controller board into slot 2 of the riser cage.
8. Install the riser cage in the server ("Install the PCI riser cage" on page 23).
9. Connect the cables from port one and two on the backplane to port one on the P440 controller board.
If the H240 controller board is installed, the SAS cables will use both ports.
o 8 SFF
Hardware options installation 64
Page 65
o
4 LFF
Systems Insight Display module
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
4. Remove the access panel (on page 21).
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
Hardware options installation 65
Page 66
5.
Disconnect the Power/UID/USB cable from the front power button/USB 3.0 connector on the system
board.
6. Remove the Power/UID/USB assembly.
o 8 SFF
Hardware options installation 66
Page 67
o
4 LFF
CAUTION: When routing cables, always be sure that the cables are not in a position where they
can be pinched or crimped.
To install the SID module:
1. Guide the SID cable through the front of the server.
2. Install the SID module into the front panel and secure the module to the chassis with the screws from the
kit.
Hardware options installation 67
Page 68
o
8 SFF
o 4 LFF
Hardware options installation 68
Page 69
3.
Connect the SID cables to the front power button/USB 3.0 connector on the system board.
FlexibleLOM option
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
4. Remove the access panel (on page 21).
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
Hardware options installation 69
Page 70
5.
Remove the FlexibleLOM PCIe blank.
6. Install the component:
a. Firmly seat the FlexibleLOM in the slot.
b. Tighten the thumbscrew.
7. Install the access panel (on page 21).
8. Slide the server into the rack.
9. Connect the LAN segment cables.
10. Connect each power cord to the server.
11. Connect each power cord to the power source.
12. Power up the server (on page 20).
Serial cable option
Hardware options installation 70
Page 71
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Remove the serial cable blank.
Hardware options installation 71
Page 72
6.
Connect the serial cable option. Then, remove the backing from the double-sided tape and press down
where indicated.
Expansion board options
The server supports PCI Express expansion boards. The server ships with PCIe riser boards and expansion
Removing the expansion slot blanks
slots. PCIe expansion boards are supported with optional riser boards.
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Disconnect any external cables that are connected to the expansion board.
6. Disconnect any internal cables that are connected to the expansion board.
7. Remove the primary PCI riser cage ("Remove the PCI riser cage" on page 22).
8. Remove the expansion slot blank:
Hardware options installation 72
Page 73
o
Primary PCI riser cage
Hardware options installation 73
Page 74
o
Secondary PCI riser cage
Installing an expansion board
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
1. Back up all server data.
2. Power down the server (on page 20).
3. Do one of the following:
internal system components to cool before touching them.
CAUTION: To prevent damage to electrical components, properly ground the server before
beginning any installation procedure. Improper grounding can cause ESD.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
Hardware options installation 74
Page 75
For instructions on how to extend or remove the server from the rack, see the documentation that ships
with the rack rail system.
4. Remove the access panel (on page 21).
5. Remove the primary PCI riser cage ("Remove the PCI riser cage" on page 22).
6. Remove the expansion slot blank ("Removing the expansion slot blanks" on page 72).
7. Install the expansion board:
o Primary riser cage
o Secondary riser cage
8. Connect any required internal or external cables to the expansion board. See the documentation that
ships with the expansion board.
9. Install the PCI riser cage (on page 23).
10. Install the access panel (on page 21).
11. Install the server into the rack.
Hardware options installation 75
Page 76
12.
Connect each power cord to the server.
13. Connect each power cord to the power source.
14. Power up the server (on page 20).
Location discovery services ear option
1. Remove all power:
a. Disconnect each power cord from the power source.
b. Disconnect each power cord from the server.
2. Remove the access panel (on page 21).
Hardware options installation 76
Page 77
3.
Remove the hot-plug fan or fan blank from fan bay 1.
Hardware options installation 77
Page 78
4. Remove the location discovery cable protection panel from the left side of the server.
Hardware options installation 78
Page 79
5.
Remove the standard ear.
6. Install the location discovery services ear option and route the location discovery services cable through
side channel.
Hardware options installation 79
Page 80
7.
Install the discovery cable protection panel.
8. Connect the location discovery services cable to the location discovery services connector on the system
board.
Hardware options installation 80
Page 81
9.
Install hot plug fan or fan blank into fan bay 1.
HP Trusted Platform Module option
For more information about product features, specifications, options, configurations, and compatibility, see
the product QuickSpecs on the HP website (http://www.hp.com/go/qs).
Use these instructions to install and enable a TPM on a supported server. This procedure includes three
sections:
1. Installing the Trusted Platform Module board.
2. Retaining the recovery key/password (on page 83).
3. Enabling the Trusted Platform Module (on page 84).
Enabling the TPM requires accessing BIOS/Platform Configuration (RBSU) in HP UEFI System Utilities
(on page 97).
TPM installation requires the use of drive encryption technology, such as the Microsoft Windows BitLocker
Drive Encryption feature. For more information on BitLocker, see the Microsoft website
(http://www.microsoft.com).
CAUTION: Always observe the guidelines in this document. Failure to follow these guidelines
When installing or replacing a TPM, observe the following guidelines:
can cause hardware damage or halt data access.
• Do not remove an installed TPM. Once installed, the TPM becomes a permanent part of the system
board.
• When installing or replacing hardware, HP service providers cannot enable the TPM or the encryption
technology. For security reasons, only the customer can enable these features.
• When returning a system board for service replacement, do not remove the TPM from the system board.
When requested, HP Service provides a TPM with the spare system board.
• Any attempt to remove an installed TPM from the system board breaks or disfigures the TPM security
rivet. Upon locating a broken or disfigured rivet on an installed TPM, administrators should consider the
system compromised and take appropriate measures to ensure the integrity of the system data.
Hardware options installation 81
Page 82
• When using BitLocker, always retain the recovery key/password. The recovery key/password is
required to enter Recovery Mode after BitLocker detects a possible compromise of system integrity.
• HP is not liable for blocked data access caused by improper TPM use. For operating instructions, see the
encryption technology feature documentation provided by the operating system.
Installing the Trusted Platform Module board
WARNING: To reduce the risk of personal injury, electric shock, or damage to the equipment,
remove the power cord to remove power from the server. The front panel Power On/Standby
button does not completely shut off system power. Portions of the power supply and some internal
To install the component:
1. Power down the server (on page 20).
2. Remove all power:
3. Do one of the following:
4. Remove the access panel (on page 21).
5. Remove the PCI riser cage (on page 22).
circuitry remain active until AC power is removed.
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the
internal system components to cool before touching them.
a. Disconnect each power cord from the power source.
b. Disconnect each power cord from the server.
o Extend the server from the rack (on page 20).
o Remove the server from the rack (on page 21).
CAUTION: Any attempt to remove an installed TPM from the system board breaks or disfigures
the TPM security rivet. Upon locating a broken or disfigured rivet on an installed TPM,
administrators should consider the system compromised and take appropriate measures to ensure
the integrity of the system data.
Hardware options installation 82
Page 83
6.
Install the TPM board. Press down on the connector to seat the board ("System board components" on
page 15).
7. Install the TPM security rivet by pressing the rivet firmly into the system board.
8. Install the PCI riser cage (on page 23).
9. Install the access panel (on page 21).
10. Install the server into the rack.
11. Power up the server (on page 20).
Retaining the recovery key/password
The recovery key/password is generated during BitLocker setup, and can be saved and printed after
BitLocker is enabled. When using BitLocker, always retain the recovery key/password. The recovery
key/password is required to enter Recovery Mode after BitLocker detects a possible compromise of system
integrity.
Hardware options installation 83
Page 84
To help ensure maximum security, observe the following guidelines when retaining the recovery
key/password:
• Always store the recovery key/password in multiple locations.
• Always store copies of the recovery key/password away from the server.
• Do not save the recovery key/password on the encrypted hard drive.
Enabling the Trusted Platform Module
1. During the server startup sequence, press the F9 key to access System Utilities.
2. From the System Utilities screen, select System Configuration > BIOS/Platform Configuration (RBSU) >
Server Security.
3. Select Trusted Platform Module Options and press the Enter key.
4. Select Enabled to enable the TPM and BIOS secure startup. The TPM is fully functional in this mode.
5. Press the F10 key to save your selection.
6. When prompted to save the change in System Utilities, press the Y key.
7. Press the ESC key to exit System Utilities. Then, press the Enter key when prompted to reboot the server.
The server then reboots a second time without user input. During this reboot, the TPM setting becomes
effective.
You can now enable TPM functionality in the OS, such as Microsoft Windows BitLocker or measured boot.
CAUTION: When a TPM is installed and enabled on the server, data access is locked if you fail
to follow the proper procedures for updating the system or option firmware, replacing the system
For more information on firmware updates and hardware procedures, see the HP Trusted Platform Module
Best Practices White Paper on the HP website (http://www.hp.com/support).
board, replacing a hard drive, or modifying OS application TPM settings.
For more information on adjusting TPM usage in BitLocker, see the Microsoft website
(http://technet.microsoft.com/en-us/library/cc732774.aspx).
Hardware options installation 84
Page 85
Cabling
Cabling overview
This section provides guidelines that help you make informed decisions about cabling the server and
hardware options to optimize performance.
For information on cabling peripheral components, refer to the white paper on high-density deployment at
the HP website (http://www.hp.com/products/servers/platforms).
CAUTION: When routing cables, always be sure that the cables are not in a position where they
2-SFF backplane cabling
can be pinched or crimped.
1. Connect the data cables to the SATA storage connector, the SATA optical/storage drive connector, and
the backplane presence detect connector.
Cabling 85
Page 86
2.
Connect the power cable to the left power connector on the backplane.
NOTE: Cabling from the 2 SFF backplane will vary if a PCIe storage controller is installed.
8-SFF Universal Media Bay cabling
Route the cables along the edge of the system board, and then connect the cables to the system board:
• 8 SFF DVD-RW/VGA/USB connections
o Connect the VGA/USB cable to the Front VGA/USB connector toward the rear of the system board.
o Connect the SATA optical cable to the SATA optical/storage drive connector.
• 8 SFF VGA/USB connections
Cabling 86
Page 87
Connect the VGA/USB cable to the Front VGA/USB connector toward the rear of the system board.
4-LFF Universal Media Bay cabling
Route the cables along the edge of the system board, and then connect the cables to the system board:
• 4 LFF VGA/USB connections
Connect the VGA/USB cable to the Front VGA/USB connector toward the rear of the system board.
• 4 LFF optical drive connection
Cabling 87
Page 88
Connect the SATA optical cable to the SATA optical/storage drive connector.
Smart Array controller cabling
Connect the SAS/SATA cables to the backplane and Smart Array controller. Install the SAS/SATA cables
according to their labels as Port 1 or Port 2. These labels are on the cables and their connections.
• 8 SFF
Cabling 88
Page 89
• 4 LFF
Embedded SATA cabling
WARNING: Eliminate the risk of electric shock by removing all AC power from the system before
installing or replacing any non-hot-plug hardware option. Disconnect all power cords to
Connect the SATA cables from the backplane to the Mini SAS/SATA connectors on the system board
according to their labels as Port 1 or Port 2. These labels are on the cables and their connections.
• 8 SFF
completely remove power from the server.
• 4 LFF
Cabling 89
Page 90
Connect the SATA cable from port one on the backplane to port one on the system board as indicated
on the connectors.
Cabling 90
Page 91
Software and configuration utilities
HP Insight Remote Support (on page 94)
Server mode
The software and configuration utilities presented in this section operate in online mode, offline mode, or in
both modes.
Software or configuration utility Server mode
HP iLO (on page 91)
Active Health System (on page 92)
HP RESTful API ("HP RESTful API support for HP iLO" on page 93)
Integrated Management Log (on page 93)
Intelligent Provisioning (on page 94)
HP Insight Diagnostics (on page 95)
HP Insight Online (on page 94)
Erase Utility (on page 95)
Scripting Toolkit for Windows and Linux (on page 96)
HP Service Pack for ProLiant (on page 96)
HP Smart Update Manager (on page 96)
HP UEFI System Utilities (on page 97)
HP Smart Storage Administrator (on page 100)
ROMPaq utility (on page 100)
Product QuickSpecs
For more information about product features, specifications, options, configurations, and compatibility, see
the product QuickSpecs on the HP website (http://www.hp.com/go/qs).
Online and Offline
Online and Offline
Online and Offline
Online and Offline
Offline
Online and Offline
Online
Online
Offline
Online
Online and Offline
Online and Offline
Offline
Online and Offline
Offline
HP iLO
The HP iLO subsystem is a standard component of HP ProLiant servers that simplifies initial server setup,
server health monitoring, power and thermal optimization, and remote server administration. The HP iLO
subsystem includes an intelligent microprocessor, secure memory, and a dedicated network interface. This
design makes HP iLO independent of the host server and its operating system.
HP iLO enables and manages the Active Health System (on page 92) and also features Agentless
Management. All key internal subsystems are monitored by HP iLO. If enabled, SNMP alerts are sent directly
by HP iLO regardless of the host operating system or even if no host operating system is installed.
Embedded remote support software is available on HP ProLiant Gen8 and later servers with iLO 4, regardless
of the operating system software and without installing OS agents on the server.
Software and configuration utilities 91
Page 92
Using HP iLO, you can do the following:
• Access a high-performance and secure Integrated Remote Console to the server from anywhere in the
world if you have a network connection to the server.
• Use the shared .NET Integrated Remote Console to collaborate with up to four server administrators.
• Remotely mount high-performance Virtual Media devices to the server.
• Securely and remotely control the power state of the managed server.
• Implement true Agentless Management with SNMP alerts from HP iLO, regardless of the state of the host
server.
• Download the Active Health System log.
• Register for HP Insight Remote Support.
• Use iLO Federation to manage multiple servers from one system running the iLO web interface.
• Use Virtual Power and Virtual Media from the GUI, the CLI, or the iLO scripting toolkit for many tasks,
including the automation of deployment and provisioning.
• Control iLO by using a remote management tool.
For more information about HP iLO features, see the HP iLO documentation on the HP website
(http://www.hp.com/go/ilo/docs).
The HP iLO 4 hardware and firmware features and functionality, such as NAND size and embedded user
partition, vary depending on the server model. For a complete list of supported features and functionality, see
the HP iLO 4 QuickSpecs on the HP website
(http://h18000.www1.hp.com/products/quickspecs/14276_div/14276_div.pdf).
Active Health System
HP Active Health System provides the following features:
• Combined diagnostics tools/scanners
• Always on, continuous monitoring for increased stability and shorter downtimes
• Rich configuration history
• Health and service alerts
• Easy export and upload to Service and Support
The HP Active Health System monitors and records changes in the server hardware and system configuration.
The Active Health System assists in diagnosing problems and delivering rapid resolution if server failures
occur.
The Active Health System collects the following types of data:
• Server model
• Serial number
• Processor model and speed
• Storage capacity and speed
• Memory capacity and speed
• Firmware/BIOS
Software and configuration utilities 92
Page 93
HP Active Health System does not collect information about Active Health System users' operations, finances,
customers, employees, partners, or data center, such as IP addresses, host names, user names, and
passwords. HP Active Health System does not parse or change operating system data from third-party error
event log activities, such as content created or passed through by the operating system.
The data that is collected is managed according to the HP Data Privacy policy. For more information see the
HP website (http://www.hp.com/go/privacy).
The Active Health System, in conjunction with the system monitoring provided by Agentless Management or
SNMP Pass-thru, provides continuous monitoring of hardware and configuration changes, system status, and
service alerts for various server components.
The Agentless Management Service is available in the SPP, which can be downloaded from the HP website
(http://www.hp.com/go/spp/download). The Active Health System log can be downloaded manually from
HP iLO or HP Intelligent Provisioning and sent to HP.
For more information, see the following documents:
• HP iLO User Guide on the HP website (http://www.hp.com/go/ilo/docs)
• HP Intelligent Provisioning User Guide on the HP website
(http://www.hp.com/go/intelligentprovisioning/docs)
HP RESTful API support for HP iLO
HP iLO 4 firmware version 2.00 and later includes the HP RESTful API. The HP RESTful API is a management
interface that server management tools can use to perform configuration, inventory, and monitoring of an HP
ProLiant server via iLO. A REST client sends HTTPS operations to the iLO web server to GET and PATCH
JSON-formatted data, and to configure supported iLO and server settings, such as the UEFI BIOS settings.
HP iLO 4 supports the HP RESTful API with HP ProLiant Gen8 and later servers. For more information about
the HP RESTful API, see the HP website (http://www.hp.com/support/restfulinterface/docs).
Integrated Management Log
The IML records hundreds of events and stores them in an easy-to-view form. The IML timestamps each event
with 1-minute granularity.
You can view recorded events in the IML in several ways, including the following:
• From within HP SIM
• From within HP UEFI System Utilities (on page 97)
• From within the Embedded UEFI shell (on page 99)
• From within operating system-specific IML viewers:
o For Windows: IML Viewer
o For Linux: IML Viewer Application
• From within the HP iLO web interface
• From within HP Insight Diagnostics (on page 95)
Software and configuration utilities 93
Page 94
HP Insight Remote Support
HP strongly recommends that you register your device for remote support to enable enhanced delivery of
your HP Warranty, HP Care Pack Service, or HP contractual support agreement. HP Insight Remote Support
supplements your monitoring continuously to ensure maximum system availability by providing intelligent
event diagnosis, and automatic, secure submission of hardware event notifications to HP, which will initiate
a fast and accurate resolution, based on your product’s service level. Notifications can be sent to your
authorized HP Channel Partner for onsite service, if configured and available in your country.
For more information, see HP Insight Remote Support and Insight Online Setup Guide for ProLiant Servers
and BladeSystem c-Class Enclosures on the HP website
(http://www.hp.com/go/insightremotesupport/docs). HP Insight Remote Support is available as part of HP
HP Insight Remote Support central connect
Warranty, HP Care Pack Service, or HP contractual support agreement.
When you use the embedded Remote Support functionality with HP ProLiant Gen8 and later server models
and HP BladeSystem c-Class enclosures, you can register a server or enclosure to communicate to HP through
an HP Insight Remote Support centralized Hosting Device in your local environment. All configuration and
service event information is routed through the Hosting Device. This information can be viewed by using the
local HP Insight Remote Support user interface or the web-based view in HP Insight Online.
For more information, see HP Insight Remote Support Release Notes on the HP website
(http://www.hp.com/go/insightremotesupport/docs).
HP Insight Online direct connect
When you use the embedded Remote Support functionality with HP ProLiant Gen8 and later server models
and HP BladeSystem c-Class enclosures, you can register a server or enclosure to communicate directly to HP
Insight Online without the need to set up an HP Insight Remote Support centralized Hosting Device in your
local environment. HP Insight Online will be your primary interface for remote support information.
For more information, see the product documentation on the HP website
(http://www.hp.com/go/insightremotesupport/docs).
HP Insight Online
HP Insight Online is a capability of the HP Support Center portal. Combined with HP Insight Remote Support
central connect or HP Insight Online direct connect, it automatically aggregates device health, asset, and
support information with contract and warranty information, and then secures it in a single, personalized
dashboard that is viewable from anywhere at any time. The dashboard organizes your IT and service data
to help you understand and respond to that information more quickly. With specific authorization from you,
an authorized HP Channel Partner can also view your IT environment remotely using HP Insight Online.
For more information about using HP Insight Online, see the HP Insight Online User’s Guide on the HP
website (http://www.hp.com/go/insightremotesupport/docs).
Intelligent Provisioning
Intelligent Provisioning is a single-server deployment tool embedded in HP ProLiant Gen8 and later servers
that simplifies HP ProLiant server setup, providing a reliable and consistent way to deploy HP ProLiant server
configurations:
Software and configuration utilities 94
Page 95
• Intelligent Provisioning assists with the OS installation process by preparing the system for installing
"off-the-shelf" and HP branded versions of operating system software and integrating optimized HP
ProLiant server support software.
• Intelligent Provisioning provides maintenance-related tasks using the Perform Maintenance window.
• Intelligent Provisioning provides installation help for Microsoft Windows, Red Hat and SUSE Linux, and
VMware operating systems. For specific OS support, see the HP Intelligent Provisioning Release Notes
on the HP website (http://www.hp.com/go/intelligentprovisioning/docs).
For more information about Intelligent Provisioning software, see the HP website
(http://www.hp.com/go/intelligentprovisioning). For Intelligent Provisioning recovery media downloads,
see the Resources tab on the HP website (http://www.hp.com/go/ilo). For consolidated drive and firmware
update packages, see the HP Smart Update: Server Firmware and Driver Updates page on the HP website
(http://www.hp.com/go/SmartUpdate).
HP Insight Diagnostics
HP Insight Diagnostics is a proactive server management tool, available in both offline and online versions,
that provides diagnostics and troubleshooting capabilities to assist IT administrators who verify server
installations, troubleshoot problems, and perform repair validation.
HP Insight Diagnostics Offline Edition performs various in-depth system and component testing while the OS
is not running. To run this utility, boot the server using Intelligent Provisioning (on page 94).
HP Insight Diagnostics Online Edition is a web-based application that captures system configuration and
other related data needed for effective server management. Available in Microsoft Windows and Linux
versions, the utility helps to ensure proper system operation.
For more information or to download the utility, see the HP website (http://www.hp.com/servers/diags). HP
Insight Diagnostics Online Edition is also available in the SPP ("HP Service Pack for ProLiant" on page 96).
HP Insight Diagnostics survey functionality
HP Insight Diagnostics (on page 95) provides survey functionality that gathers critical hardware and software
information on ProLiant servers.
This functionality supports operating systems that are supported by the server. For operating systems
supported by the server, see the HP website (http://www.hp.com/go/supportos).
If a significant change occurs between data-gathering intervals, the survey function marks the previous
information and overwrites the survey data files to reflect the latest changes in the configuration.
Survey functionality is installed with every Intelligent Provisioning-assisted HP Insight Diagnostics installation,
or it can be installed through the SPP ("HP Service Pack for ProLiant" on page 96).
Erase Utility
CAUTION: Perform a backup before running the Erase Utility. The utility sets the system to its
original factory state, deletes the current hardware configuration information, including array
setup and disk partitioning, and erases all connected hard drives completely. Before using this
Use the Erase Utility to erase drives and Active Health System logs, and to reset UEFI System Utilities settings.
Run the Erase Utility if you must erase the system for the following reasons:
utility, see the instructions in the HP Intelligent Provisioning User Guide.
Software and configuration utilities 95
Page 96
• You want to install a new operating system on a server with an existing operating system.
• You encounter an error when completing the steps of a factory-installed operating system installation.
To access the Erase Utility, click the Perform Maintenance icon from the Intelligent Provisioning home screen,
and then select Erase.
For more information about the Erase Utility, see the HP Intelligent Provisioning User Guide on the HP website
(http://www.hp.com/go/intelligentprovisioning/docs).
Scripting Toolkit for Windows and Linux
The Scripting Toolkit for Windows and Linux is a server deployment product that delivers an unattended
automated installation for high-volume server deployments. The Scripting Toolkit is designed to support
ProLiant BL, ML, DL, and SL servers. The toolkit includes a modular set of utilities and important
documentation that describes how to apply these tools to build an automated server deployment process.
The Scripting Toolkit provides a flexible way to create standard server configuration scripts. These scripts are
used to automate many of the manual steps in the server configuration process. This automated server
configuration process cuts time from each deployment, making it possible to scale rapid, high-volume server
deployments.
For more information, and to download the Scripting Toolkit, see the HP website
(http://www.hp.com/go/ProLiant/STK).
HP Service Pack for ProLiant
SPP is a comprehensive systems software (drivers and firmware) solution delivered as a single package with
major server releases. This solution uses HP SUM as the deployment tool and is tested on all supported HP
ProLiant servers including HP ProLiant Gen8 and later servers.
SPP can be used in an online mode on a Windows or Linux hosted operating system, or in an offline mode
where the server is booted to an operating system included on the ISO file so that the server can be updated
automatically with no user interaction or updated in interactive mode.
For more information or to download SPP, see one of the following pages on the HP website:
• HP Service Pack for ProLiant download page (http://www.hp.com/go/spp)
HP Smart Update Manager
• HP Smart Update: Server Firmware and Driver Updates page (http://www.hp.com/go/SmartUpdate)
HP SUM is a product used to install and update firmware, drivers, and systems software on HP ProLiant
servers. HP SUM provides a GUI and a command-line scriptable interface for deployment of systems software
for single or one-to-many HP ProLiant servers and network-based targets, such as iLOs, OAs, and VC Ethernet
and Fibre Channel modules.
For more information about HP SUM, see the product page on the HP website
(http://www.hp.com/go/hpsum).
To download HP SUM, see the HP website (http://www.hp.com/go/hpsum/download).
To access the HP Smart Update Manager User Guide, see the HP SUM Information Library
(http://www.hp.com/go/hpsum/documentation).
Software and configuration utilities 96
Page 97
HP UEFI System Utilities
The HP UEFI System Utilities is embedded in the system ROM. The UEFI System Utilities enable you to perform
a wide range of configuration activities, including:
• Configuring system devices and installed options
• Enabling and disabling system features
• Displaying system information
• Selecting the primary boot controller
• Configuring memory options
• Selecting a language
• Launching other pre-boot environments such as the Embedded UEFI Shell and Intelligent Provisioning
For more information on the HP UEFI System Utilities, see the HP UEFI System Utilities User Guide for HP
ProLiant Gen9 Servers on the HP website (http://www.hp.com/go/ProLiantUEFI/docs).
Scan the QR code located at the bottom of the screen to access mobile-ready online help for the UEFI System
Utilities and UEFI Shell. For on-screen help, press F1.
Using HP UEFI System Utilities
To use the System Utilities, use the following keys.
Action Key
Access System Utilities
Navigate menus
Select items
Save selections
Access Help for a highlighted configuration
option*
*Scan the QR code on the screen to access online help for the UEFI System Utilities and UEFI Shell.
Default configuration settings are applied to the server at one of the following times:
• Upon the first system power-up
• After defaults have been restored
Default configuration settings are sufficient for typical server operations; however, you can modify
configuration settings as needed. The system prompts you for access to the System Utilities each time the
system is powered up.
Flexible boot control
F9 during server POST
Up and Down arrows
Enter
F10
F1
This feature enables you to do the following:
• Add Boot Options
o Browse all FAT16 and FAT32 file systems.
o Select an X64 UEFI application with an .EFI extension to add as a new UEFI boot option, such as an
OS boot loader or other UEFI application.
Software and configuration utilities 97
Page 98
The new boot option is appended to the boot order list. When you select a file, you are prompted
to enter the boot option description (which is then displayed in the Boot menu), as well as any
optional data to be passed to an .EFI application.
• Boot to System Utilities
Toward the end of the boot process, the boot options screen appears. This screen is visible for several
seconds before the system attempts to boot from a supported boot device. During this time, you can
access the System Utilities by pressing the F9 key.
• Choose between supported modes: Legacy BIOS Boot Mode or UEFI Boot Mode
IMPORTANT: If the default boot mode settings are different than the user defined settings, the
system may not boot the OS installation if the defaults are restored. To avoid this issue, use the
User Defined Defaults feature in UEFI System Utilities to override the factory default settings.
For more information, see the HP UEFI System Utilities User Guide for HP ProLiant Gen9 Servers on the HP
website (http://www.hp.com/go/ProLiantUEFI/docs).
Restoring and customizing configuration settings
You can reset all configuration settings to the factory default settings, or you can restore system default
configuration settings, which are used instead of the factory default settings.
You can also configure default settings as necessary, and then save the configuration as the custom default
configuration. When the system loads the default settings, it uses the custom default settings instead of the
factory defaults.
Secure Boot configuration
Secure Boot is integrated in the UEFI specification on which the HP implementation of UEFI is based. Secure
Boot is completely implemented in the BIOS and does not require special hardware. It ensures that each
component launched during the boot process is digitally signed and that the signature is validated against a
set of trusted certificates embedded in the UEFI BIOS. Secure Boot validates the software identity of the
following components in the boot process:
• UEFI drivers loaded from PCIe cards
• UEFI drivers loaded from mass storage devices
• Pre-boot UEFI shell applications
• OS UEFI boot loaders
Once enabled, only firmware components and operating systems with boot loaders that have an appropriate
digital signature can execute during the boot process. Only operating systems that support Secure Boot and
have an EFI boot loader signed with one of the authorized keys can boot when Secure Boot is enabled. For
more information about supported operating systems, see the HP UEFI System Utilities and Shell Release
Notes on the HP website (http://www.hp.com/go/ProLiantUEFI/docs).
A physically present user can customize the certificates embedded in the UEFI BIOS by adding/removing
their own certificates.
Software and configuration utilities 98
Page 99
Embedded UEFI shell
The system BIOS in all HP ProLiant Gen9 servers includes an Embedded UEFI Shell in the ROM. The UEFI
Shell environment provides an API, a command line prompt, and a set of CLIs that allow scripting, file
manipulation, and system information. These features enhance the capabilities of the UEFI System Utilities.
For more information, see the following documents:
• HP UEFI Shell User Guide for HP ProLiant Gen9 Servers on the HP website
(http://www.hp.com/go/ProLiantUEFI/docs)
• UEFI Shell Specification on the UEFI website (http://www.uefi.org/specifications)
Embedded Diagnostics option
The system BIOS in all HP ProLiant Gen9 servers includes an Embedded Diagnostics option in the ROM. The
Embedded Diagnostics option can run comprehensive diagnostics of the server hardware, including
processors, memory, drives, and other server components.
For more information on the Embedded Diagnostics option, see the HP UEFI System Utilities User Guide for
HP ProLiant Gen9 Servers on the HP website (http://www.hp.com/go/ProLiantUEFI/docs).
HP RESTful API support for UEFI
HP ProLiant Gen9 servers include support for a UEFI compliant System BIOS, along with UEFI System Utilities
and Embedded UEFI Shell pre-boot environments. HP ProLiant Gen9 servers also support configuring the
UEFI BIOS settings using the HP RESTful API, a management interface that server management tools can use
to perform configuration, inventory, and monitoring of an HP ProLiant server. A REST client uses HTTPS
operations to configure supported server settings, such as UEFI BIOS settings.
For more information about the HP RESTful API and the HP RESTful Interface Tool, see the HP website
(http://www.hp.com/support/restfulinterface/docs).
Re-entering the server serial number and product ID
After you replace the system board, you must re-enter the server serial number and the product ID.
1. During the server startup sequence, press the F9 key to access UEFI System Utilities.
2. Select the System Configuration > BIOS/Platform Configuration (RBSU) > Advanced Options >
Advanced System ROM Options > Serial Number, and then press the Enter key.
3. Enter the serial number and press the Enter key. The following message appears:
The serial number should only be modified by qualified service personnel.
This value should always match the serial number located on the chassis.
4. Press the Enter key to clear the warning.
5. Enter the serial number and press the Enter key.
6. Select Product ID. The following warning appears:
Warning: The Product ID should ONLY be modified by qualified service
personnel. This value should always match the Product ID located on the
chassis.
7. Enter the product ID and press the Enter key.
8. Press the F10 key to confirm exiting System Utilities. The server automatically reboots.
Software and configuration utilities 99
Page 100
Utilities and features
HP Smart Storage Administrator
HP SSA is a configuration and management tool for HP Smart Array controllers. Starting with HP ProLiant
Gen8 servers, HP SSA replaces ACU with an enhanced GUI and additional configuration features.
HP SSA exists in three interface formats: the HP SSA GUI, the HP SSA CLI, and HP SSA Scripting. Although
all formats provide support for configuration tasks, some of the advanced tasks are available in only one
format.
Some HP SSA features include the following:
• Supports online array capacity expansion, logical drive extension, assignment of online spares, and
RAID or stripe size migration
• Suggests the optimal configuration for an unconfigured system
• Provides diagnostic and SmartSSD Wear Gauge functionality on the Diagnostics tab
• For supported controllers, provides access to additional features.
For more information about HP SSA, see the HP website (http://www.hp.com/go/hpssa).
ROMPaq utility
The ROMPaq utility enables you to upgrade the system firmware (BIOS). To upgrade the firmware, insert a
ROMPaq USB Key into an available USB port and boot the system. In addition to ROMPaq, Online Flash
Components for Windows and Linux operating systems are available for updating the system firmware.
The ROMPaq utility checks the system and provides a choice (if more than one exists) of available firmware
revisions.
For more information, go to the HP website (http://www.hp.com/go/hpsc) and click on Drivers, Software
& Firmware. Then, enter your product name in the Find an HP product field and click Go.
Automatic Server Recovery
ASR is a feature that causes the system to restart when a catastrophic operating system error occurs, such as
a blue screen, ABEND, or panic. A system fail-safe timer, the ASR timer, starts when the System Management
driver, also known as the Health Driver, is loaded. When the operating system is functioning properly, the
system periodically resets the timer. However, when the operating system fails, the timer expires and restarts
the server.
ASR increases server availability by restarting the server within a specified time after a system hang. You can
disable ASR from the System Management Homepage or through UEFI System Utilities.
USB support
HP provides standard USB 2.0 support, standard USB 3.0 support, and legacy USB support. Standard
support is provided by the OS through the appropriate USB device drivers.
Software and configuration utilities 100
Loading...