No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of Huawei Technologies Co., Ltd.
Trademarks and Permissions
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and the
customer. All or part of the products, services and features described in this document may not be within the
purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,
and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Huawei Technologies Co., Ltd.
Address:Huawei Industrial Base
Bantian, Longgang
Shenzhen 518129
People's Republic of China
This document describes the CH121 V5 compute node (CH121 V5) in terms of its
components, common operations, basic configuration, hardware installation, troubleshooting,
software and configuration utilities, electrostatic discharge (ESD) prevention, and product
specifications.
About This Document
About This Document
Intended Audience
This document is intended for:
lEnterprise administrators
lEnterprise end users
Symbol Conventions
The symbols that may be found in this document are defined as follows:
Symbol
Description
Indicates an imminently hazardous situation which, if not
avoided, will result in death or serious injury.
Indicates a potentially hazardous situation which, if not
avoided, could result in death or serious injury.
Indicates a potentially hazardous situation which, if not
avoided, may result in minor or moderate injury.
Indicates a potentially hazardous situation which, if not
avoided, could result in equipment damage, data loss,
performance deterioration, or unanticipated results.
NOTICE is used to address practices not related to personal
injury.
2.1 Front Panel......................................................................................................................................................................5
2.1.2 Indicators and Buttons................................................................................................................................................. 6
3.3 Removing a CH121 V5................................................................................................................................................ 28
3.4 Installing a CH121 V5..................................................................................................................................................31
3.5 Removing the Cover.....................................................................................................................................................35
3.6 Installing the Cover...................................................................................................................................................... 35
3.7 Removing the Air Ducts............................................................................................................................................... 36
3.8 Installing the Air Duct.................................................................................................................................................. 37
3.9 Removing the PCIe Card Panel.................................................................................................................................... 39
3.10 Installing the PCIe Card Panel....................................................................................................................................40
3.11 Removing a PCIe Card............................................................................................................................................... 41
3.12 Installing a PCIe Card.................................................................................................................................................43
3.13 Removing a SAS/SATA Drive....................................................................................................................................46
3.14 Installing a SAS/SATA Drive.....................................................................................................................................47
3.15 Removing an NVMe Drive.........................................................................................................................................49
3.16 Installing an NVMe Drive ......................................................................................................................................... 62
4 Installation and Configuration................................................................................................. 71
4.1.1 Space and Airflow Requirements.............................................................................................................................. 71
4.1.2 Temperature and Humidity Requirements................................................................................................................. 71
4.2.4 Installing the Compute Node.....................................................................................................................................75
4.2.5 Connecting to Network..............................................................................................................................................75
4.3.3.1 Changing the Initial Password of the Default iBMC User..................................................................................... 78
4.3.4 Checking the Compute Node.....................................................................................................................................80
4.3.5 Setting the iBMC IP Address.................................................................................................................................... 82
4.3.7 Configuring the BIOS................................................................................................................................................83
4.3.7.1 Entering the BIOS.................................................................................................................................................. 84
4.3.7.1.1 Entering the BIOS (Skylake)...............................................................................................................................84
4.3.7.1.2 Entering the BIOS (Cascade Lake)..................................................................................................................... 86
4.3.7.2 Setting the System Boot Sequence......................................................................................................................... 88
4.3.7.3 Setting PXE for a NIC............................................................................................................................................ 89
4.3.7.4 Setting the BIOS Password.....................................................................................................................................90
FusionServer Pro CH121 V5 Compute Node
User GuideContents
4.3.7.5 Selecting a Language..............................................................................................................................................90
4.3.7.6 Restarting the Compute Node.................................................................................................................................91
4.3.8 Installing an OS......................................................................................................................................................... 91
5 Optional Part Installation.......................................................................................................... 93
5.2.1 Installing an M.2 FRU............................................................................................................................................... 93
5.2.2 Installing the Screw-in RAID Controller Card..........................................................................................................95
5.2.3 Installing the Avago SAS3004iMR PCIe RAID Control Card................................................................................. 97
5.2.4 Installing an M.2 FRU on the Avago SAS3004iMR PCIe RAID Controller Card................................................... 98
5.2.5 Installing the Supercapacitor..................................................................................................................................... 98
5.2.6 Installing a Mezzanine Card.................................................................................................................................... 101
5.2.7 Installing a Processor...............................................................................................................................................103
5.2.8 Installing a Memory Module................................................................................................................................... 110
5.2.9 Installing the TPM................................................................................................................................................... 113
6 Software and Hardware Compatibility.................................................................................115
8 Software and Configuration Utilities....................................................................................117
8.1 Upgrading the System.................................................................................................................................................117
9.2 Maintenance and Warranty......................................................................................................................................... 126
12 Common Operations...............................................................................................................134
12.1 Querying the iBMC IP Address................................................................................................................................134
12.2 Logging In to the iBMC WebUI............................................................................................................................... 135
12.3 Logging In to the MM910 WebUI............................................................................................................................ 139
12.4 Collecting Log Information on the MM910 WebUI.................................................................................................144
12.5 Logging In to a Compute Node Using MM910 SOL...............................................................................................145
FusionServer Pro CH121 V5 Compute Node
User GuideContents
12.6 Managing the E9000 Server Using the Local KVM................................................................................................ 146
12.6.1 Logging In to the MM910 CLI..............................................................................................................................146
12.6.2 Logging In to the Operating System of a Compute Node..................................................................................... 147
12.6.3 Mounting the DVD Drive to a Compute Node......................................................................................................148
12.6.4 Logging In to a Compute Node or a Switch Module over SOL............................................................................149
12.7 Logging In to the Desktop of a Server..................................................................................................................... 151
12.7.1 Using the Remote Virtual Console........................................................................................................................151
12.7.2 Using the Independent Remote Console............................................................................................................... 155
12.8 Logging In to the CLI...............................................................................................................................................161
12.8.1 Logging In to the CLI Using PuTTY over a Network Port...................................................................................161
12.10 Clearing Data from a Storage Device.....................................................................................................................164
A More Information..................................................................................................................... 167
The CH121 V5 is a half-width compute node powered by Intel® Xeon® Scalable processors.
It delivers supreme computing power, large memory capacity, and outstanding scalability.
The CH121 V5 provides dense computing capability and an ultra-large memory. It is
optimized for virtualization, cloud computing, high-performance computing, and computeintensive enterprise applications.
The CH121 V5 compute nodes are installed in an E9000 chassis and are centrally managed by
the management module.
l a: The drive slots support 2.5-inch SAS/SATA/NVMe drives and M.2 modules and
mixed configuration of them.
l b: An M.2 module is a 2.5-inch drive module that consists of one M.2 adapter and two
M.2 FRUs.
1.3 Logical Structure
Figure 1-3 CH121 V5 logical structure
1 Overview
lThe server supports one or two Intel® Xeon® Scalable processors.
lThe server supports up to 24 memory modules.
lThe CPUs (processors) interconnect with each other through two UPI links at a speed of
up to 10.4 GT/s.
lThe mezzanine cards connect to the processors through PCIe buses to provide service
ports.
lThe Platform Controller Hub (PCH) has a built-in MAC chip and provides two 10 Gbit/s
ports.
lThe storage module, consisting of a RAID controller card and a drive backplane,
connects to the CPUs through PCIe buses.
lThe BMC provides device management functions, such as compute node power control,
slot number acquisition, PSU detection, and KVM over IP.
Table 2-1 Indicators and buttons on the front panel
2 Hardware Description
SilkscreenIndicator/
Button
Power button/
indicator
UIDUID button/
indicator
Description
Power indicator:
l Off: The device is not powered on.
l Steady yellow: The device is powered on.
l Blinking yellow: The power button is locked. The
power button is locked when the iBMC is starting.
l Steady green: The device is ready to power on.
Power button:
l When the device is powered on, you can press this
button to gracefully shut down the OS.
l When the device is powered on, holding down this
button for 6 seconds will forcibly power off the
device.
l When the power indicator is steady green, you can
press this button to power on the device.
The UID button/indicator helps identify and locate a
device.
UID indicator:
l Off: The device is not being located.
l Blinking blue: The device has been located and is
differentiated from other devices that have also
been located.
l Steady blue: The device is being located.
UID button:
l You can turn on or off the UID indicator by
pressing the UID button or by using the iBMC or
MM910 CLI or WebUI.
l You can press this button to turn on or off the UID
indicator.
l You can press and hold down this button for 4 to 6
l Off: The device is powered off or is faulty.
l Blinking red at 1 Hz: A major alarm has been
generated for the device.
l Blinking red at 5 Hz: A critical alarm has been
generated for the device, or the device is not
securely installed.
l Steady green: The device is operating properly.
Page 16
FusionServer Pro CH121 V5 Compute Node
User Guide
2.1.3 Ports
Port Positions
Figure 2-3 Ports on the front panel
2 Hardware Description
1USB 3.0 ports2-
Port Description
Port
USB portUSB 3.02Used to connect to a USB device.
TypeQuantityDescription
2.1.4 Installation Positions
The CH121 V5 is installed in a half-width slot in the front of the E9000 chassis. An E9000
chassis can house a maximum of 16 CH121 V5 compute nodes.
NOTICE
Before connecting an external USB
device, check that the USB device
functions properly. The server may
operate abnormally if an abnormal
USB device is connected.
lThe server supports one or two processors.
lIf only one processor is required, install it in socket CPU1.
lThe same model of processors must be used in a server.
lContact your local Huawei sales representative or use the Intelligent Computing
Compatibility Checker to determine the components to be used.
l 1R: single-rank
l 2R: dual-rank
l 4R: quad-rank
l 8R: octal-rank
l X4: 4-bit
l X8: 8-bit
l PC3: DDR3
l PC4: DDR4
l 2133 MT/S
l 2400 MT/S
l 2666 MT/S
l 2933 MT/S
6Column Access Strobe (CAS)
latency.
7DIMM type.
2.3.2 Memory Subsystem Architecture
The CH121 V5 provides 24 memory slots. Each processor integrates six memory channels.
Install the memory modules in the primary memory channels first. If the primary memory
channel is not populated, the memory modules in secondary memory channels cannot be used.
lThe total memory capacity is the sum of the capacity of all DDR4 memory modules.
The total memory capacity cannot exceed the maximum memory capacity supported by
the CPUs.
lUse the Intelligent Computing Compatibility Checker to determine the capacity type
of a single memory module.
lThe maximum number of memory modules supported by a server varies depending on
the CPU type, memory type, rank quantity, and operating voltage.
NOTE
Each memory channel supports a maximum of 8 ranks. The number of memory modules
supported by each channel varies depending on the number of ranks supported by each channel:
Number of memory modules supported by each channel ≤ Number of ranks supported by each
memory channel/Number of ranks supported by each memory module
lA memory channel supports more than eight ranks for LRDIMMs.
NOTE
A quad-rank LRDIMM generates the same electrical load as a single-rank RDIMM on a memory
bus.
lDDR4 memory modules of different specifications (capacity, bit width, rank, and height)
can be used together for capacity expansion purposes. However, the memory Reliability,
Availability, and Serviceability (RAS) features may be affected.
l a: If the Cascade Lake processor is used, the rated speed and maximum operating speed
of the RDIMM can reach 2933 MT/s. If the Skylake processor is used, the rated speed
and maximum operating speed of a DDR4 memory module can reach 2666 MT/s only.
l b: The maximum number of DDR4 memory modules is based on dual-processor
configuration. The value is halved for a server with only one processor.
l c: The maximum DDR4 memory capacity varies depending on the processor type. The
value listed in this table is based on the assumption that twenty-four 128 GB DDR4
memory modules are configured.
2.3.4 Memory Installation Guidelines
lObserve the following when configuring DDR4 memory modules:
– Install memory modules only when corresponding processors are installed.
– Do not install LRDIMMs and RDIMMs in the same server.
– Install filler memory modules in vacant slots.
lObserve the following when configuring DDR4 memory modules in specific operating
mode:
– Memory sparing mode
nComply with the general installation guidelines.
nEach memory channel must have a valid online spare configuration.
nThe channels can have different online spare configurations.
nEach populated channel must have a spare rank.
– Memory mirroring mode
nComply with the general installation guidelines.
nInstall memory modules for channels 1 and 2 or channels 3 and 4. The
memory modules installed must be identical in size and organization.
nFor a multi-processor configuration, each processor must have a valid memory
mirroring configuration.
– Memory scrubbing mode
nComply with the general installation guidelines.
2 Hardware Description
2.3.5 Memory Installation Positions
A CH121 V5 supports a maximum of 24 DDR4 memory modules. Balanced memory
configuration is recommended for optimal memory performance.
The following memory protection technologies are supported:
lECC
lFull mirroring
lAddress range mirroring
lRank sparing mode
lFaulty DIMM isolation
lMemory thermal throttling
lMemory address parity protection
lAdaptive double device data correction (ADDDC)
lMemory demand/patrol scrubbing
lData scrambling
lADDDC+1
2 Hardware Description
2.4 Storage
2.4.1 Drive Configurations
Table 2-4 Drive configurations
Configuration
Front drives
l a: Only 2.5-inch drives fit into the front slots.
l b: Mixed configuration of M.2 modules and SAS/SATA/NVMe drives is supported.
l Contact your local Huawei sales representative or use the Intelligent Computing
Compatibility Checker to determine the components to be used.
Steady green/OffSteady yellowThe NVMe SSD is faulty.
lIf the VMD function is disabled, NVMe SSDs support only orderly hot swap.
Table 2-8 NVMe SSD indicators (VMD disabled)
Green Indicator
OffOffThe NVMe SSD cannot be detected.
Steady greenOffThe NVMe SSD is working properly.
Blinking green at 2HzOffData is being read from or written to the
OffBlinking yellow at
OffBlinking yellow at
Steady green/OffSteady yellowThe NVMe SSD is faulty.
Yellow IndicatorDescription
2 Hz
0.5 Hz
The data on the secondary NVMe drive is
being rebuilt.
NVMe SSD.
The NVMe SSD is being located or hotswapped.
The hot removal process is complete, and
the NVMe SSD is removable.
2.4.4 RAID Controller Card
The RAID controller card supports RAID configuration, RAID level migration, and drive
roaming.
lContact your local Huawei sales representative or use the Intelligent Computing
Compatibility Checker to determine the components to be used.
Table 2-9 RAID levels supported by the CH121 V5
RAID
Level
RAID 0LowHighHigh2100%
RAID 1HighLowLow250%
ReliabilityRead
2.5 Network
Performan
ce
Write
Performan
ce
Minimum
Number of
Drives
Drive
Utilization
2.5.1 LOMs
The LOM is a network interface module (X722) integrated in the PCH. It can be connected to
an I/O module (switch module). The LOM provides two 10GE network ports for connecting
to the Base network ports of the switch modules in slots 2X and 3X. The LOM supports Wake
on LAN (WOL) and PXE functions.
Figure 2-15 Connections between the LOM and I/O modules
2 Hardware Description
NOTE
l In addition to the LOM, the compute node connects to the Fabric ports of the switch modules
through the network ports on mezzanine cards.
l Powering off the compute node forcibly will make the WOL function of the LOM ports invalid.
l If flow control is enabled for a LOM port, the switch module connected to the LOM port must also
have flow control enabled.
Table 2-10 I/O modules supported by the LOM
I/O Module
CX9162X/3X√-
CX9202X/3X√-
I/O Module
Slot
1E/4E×The LOM cannot communicate
1E/4E×The LOM cannot communicate
LOMRemarks
with the I/O modules in slots 1E
and 4E.
with the I/O modules in slots 1E
and 4E.
2.6 I/O Expansion
2.6.1 PCIe Cards
lThe server supports a range of PCIe cards to provide diverse expandability and ease of
l The B/D/F (Bus/Device/Function Number) is the default value when the server is fully
configured with PCIe devices. The value may differ if the server is not fully configured
with PCIe devices or if a PCIe card with a PCI bridge is configured.
l Root Port (B/D/F) indicates the B/D/F of an internal PCIe root port of the processor.
l Device (B/D/F) indicates the B/D/F (displayed on the OS) of an onboard or extended
Figure 3-3 Positions and slot numbers of the CH121 V5 compute nodes
3 Basic Operations
Step 2 Power off the compute node.
For details, see 3.1 Power-Off Procedure.
Step 3 Remove the compute node.
l When removing a device, wear protective gloves to avoid burns.
l To prevent device damage or personal injury, hold the bottom of the device with both
hands when removing it.
1.Press the ejector release button on the compute node. See (1) in Figure 3-4.
2.Fully open the ejector lever. See (2) in Figure 3-4.
3.Holding the ejector lever, pull the compute node out of the chassis horizontally. See (3)
in Figure 3-4.
Step 4 Place the removed compute node in an ESD bag.
Place only one compute node in an ESD bag.
Step 5 Install filler panels.
Install filler panels in vacant slots. Otherwise, the ventilation, heat dissipation,
electromagnetic shielding, and dustproof effects of the chassis will be affected.
Insert the filler panel into the chassis horizontally until it clicks into place.
3.Place the removed filler panel in an accessory recycle bin.
Step 3 Install the compute node.
To prevent device damage or personal injury, hold the bottom of the device with both hands
when installing it.
1.Install a partition board.
NOTE
Install a partition board between two half-width compute nodes. Perform this operation only when
no partition board is installed between two half-width compute nodes.
a.Tilt the partition board to the chassis, insert it into the installation holes, and make it
upright. See (1) and (2) in Figure 3-8.
b.Pull the partition board outwards. See (3) and (4) in Figure 3-8.
The partition board is properly installed when you hear a click.
For details, see 3.9 Removing the PCIe Card Panel.
Step 7 Take the spare part out of its ESD bag.
Step 8 Determine the position of the PCIe card.
Figure 3-19 Position of the PCIe card
3 Basic Operations
Step 9 Install the PCIe card.
1.Slide the PCIe card into the PCIe card bracket as far as it will go and ensure that the edge
of the PCIe card is flush with the PCIe card bracket. See (1) in Figure 3-20.
2.Install the PCIe card bracket. Ensure that the PCIe card connectors align with the
connectors on the mainboard and the screw holes on the PCIe card filler panel align with
the screw holes on the front panel. See (2) in Figure 3-20.
3.Tighten the three screws to secure the PCIe card bracket. See (3) in Figure 3-20.
1.Press the ejector release button on the drive panel to release the ejector lever. See (1) in
Figure 3-22.
2.Open the drive ejector lever completely. See (2) in Figure 3-22.
3.Holding the ejector lever of the drive, pull out the drive for about 3 cm (1.18 in.). Wait
for at least 30 seconds and remove the drive from the server. See (3) in Figure 3-22.
Do not insert or remove drives frequently. If a drive is frequently removed and installed
in an interval shorter than 30 seconds, there are risks that the drive cannot be identified.
Figure 3-22 Removing a drive
3 Basic Operations
Step 5 Place the removed component in an ESD bag.
Step 6 Install a drive filler.
NOTE
Perform this operation only when a drive is not installed immediately.
----End
3.14 Installing a SAS/SATA Drive
l The SATA or SAS drives support hot swap. After a faulty drive is disconnected from its
backplane, wait 30 seconds before removing it. The alarm will not be cleared immediately
after a new drive is installed. It will be cleared after the RAID array rebuild is complete.
Do not frequently hot-swap a drive to prevent new alarms.
l To prevent damage caused by insufficient heat dissipation and high temperature, install
Step 2 Determine the slot and position of the drive.
For details, see 2.4.1 Drive Configurations and 2.4.2 Drive Numbering.
Figure 3-23 Positions of drives
3 Basic Operations
Step 3 Remove the drive filler.
NOTE
Perform this operation only when a drive filler is installed.
Step 4 Install a drive.
1.Fully open the ejector lever and push the drive into the slot as far as it will go. See (1) in
Figure 3-24.
2.Ensure that the ejector lever is fastened to the chassis beam, and then close it to
completely insert the drive into the slot. See (2) in Figure 3-24.
Step 5 Check the running status of the drive by observing its indicators.
For details, see SAS/SATA Drive Indicators.
Step 6 Restore the drive data.
3 Basic Operations
NOTE
Perform this operation only when you replace a drive.
For details, see Troubleshooting > Hard Drive Fault in Huawei V5 Server RAID
Controller Card User Guide.
----End
3.15 Removing an NVMe Drive
lIf the VMD function is enabled and the latest Intel RSTe driver is installed, the NVMe
drives support surprise hot swap.
– For details about how to enable the VMD function, see 12.9.1 Enabling VMD.
– Install the Intel RSTe driver. For details, see Huawei V5 Server RAID Controller
Card User Guide.
lIf the VMD function is disabled, the NVMe drives support only orderly hot swap.
For details about how to disable the VMD function, see 12.9.2 Disabling VMD.
Procedure (VMD Enabled)
NOTE
l Use the Intelligent Computing Compatibility Checker to obtain information about the operating
systems supported by the VMD function.
l Before removing an NVMe drive:
lStop accessing the NVMe drive.
lBack up date of the NVMe drive to prevent data loss.
l Remove the NVMe drives one by one. Remove the next NVMe drive only after the previous one is
removed completely.
l If the OS is RHEL 7.3 and the VMD function is enabled, the OS kernel 3.10.0-514.el7.x86_64 must
be used for surprise hot swap of NVMe drives. Otherwise, the operating system may restart.
Step 1 Determine the slot and position of the drive.
Figure 3-25 Positions of drives
3 Basic Operations
Step 2 Check the running status of the drive by observing its indicators.
For details, see NVMe SSD Indicators.
Step 3 Determine whether the drive needs to be removed.
lIf yes, go to Step 4.
lIf no, rectify the drive fault.
Step 4 Remove the drive.
1.Press the ejector release button on the drive panel to release the ejector lever. See (1) in
Figure 3-26.
2.Open the drive ejector lever completely. See (2) in Figure 3-26.
3.Holding the ejector lever of the drive, pull out the drive for about 3 cm (1.18 in.). Wait
for at least 30 seconds and remove the drive from the server. See (3) in Figure 3-26.
Do not insert or remove drives frequently. If a drive is frequently removed and installed
in an interval shorter than 30 seconds, there are risks that the drive cannot be identified.
Figure 3-28 Mapping between the slot ID and NVMe drive letter
3 Basic Operations
Table 3-1 Mapping between physical slot numbers, B/D/F, and slot IDs
Configuratio
n
24 x 2.5"
Physical Slot
No.
Root Port
(B/D/F)
Device
(B/D/F)
Slot ID
Slot085:02.086:00.080
NVMe drive
configuration
Slot285:03.087:00.081
The Device (B/D/F) values vary depending on the setting of the VMD function. The
values in this table are based on the assumption that the VMD function is disabled.
4.The following uses Huawei NVMe drive as an example to describe how to install the
NVMe driver hiodriver and NVMe tool NVMe Toolbox. The NVMe Toolbox can be
used to query NVMe drive information, upgrade the firmware, and perform hot swap.
NOTE
After installing ES3000 V5 NVMe drives, install the toolkit required by the OS to manage the
drives. The ES3000 V5 tool is incompatible with the built-in NVMe driver of the Windows OS.
Therefore, ensure that you have installed Huawei NVMe driver before using the tool. You can
obtain the tool package from the official Huawei website. For details, see Huawei ES3000 V5
SSD Upgrade Guide.
Step 5 In the program(x86)\hioadm directory of drive C, run NVMe Toolbox 4.0 to view the NVMe
Log in to the OS, and open /boot/efi/EFI/redhat/grub.cfg (if the OS is installed in UEFI
mode) or /boot/grub2/grub.cfg (if the OS is installed in legacy mode).
lIf RHEL 7.3 is installed in UEFI mode:
a.Run the cat /proc/cmdline command. If the command output does not contain
"pciehp.pciehp_force=1 pci=pcie_bus_perf", you need to configure kernel
parameters.
Figure 3-32 Opening the configuration file
b.Run the vi/boot/efi/EFI/redhat/grub.cfg command. In the command output, find
the sentence that is displayed after the cat/proc/cmdline command is executed in
Step 4.a, and add a space and then "pciehp.pciehp_force=1 pci=pcie_bus_perf" at
the end of the sentence. Do not add the content in a new line.
3 Basic Operations
Figure 3-33 Configuring kernel parameters (1)
Figure 3-34 Configuring kernel parameters (2)
c.Save the file and restart the compute node.
d.Run the cat /proc/cmdline command. If the command output contains
"pciehp.pciehp_force=1 pci=pcie_bus_perf", the kernel is configured successfully.
d.Run the cat /proc/cmdline command. If the command output contains
"pciehp.pciehp_force=1 pci=pcie_bus_perf", the kernel is configured successfully.
Figure 3-39 Checking the configuration
Step 5 Upgrade the OS kernel.
lExample 1: RHEL 7.3
Upgrade the kernel of RHEL 7.3 to kernel-3.10.0-514.26.2.e17.x86_64 or later to
support orderly hot swap of NVMe drives.
a.Run the uname -r command to check the current kernel version.
Figure 3-40 Kernel version
3 Basic Operations
b.Log in to the official Red Hat website to download the kernel upgrade package
(kernel-3.10.0-514.26.2.e17.x86_64.rpm for example).
c.Run the rpm -ivh kernel-3.10.0-514.26.2.e17.x86_64.rpm command to upload the
upgrade package to the OS for installation.
d.Restart the OS after the installation is complete.
e.Run the uname -r command. If 3.10.0-514.26.2.el7.x86_64 is displayed, the kernel
is upgraded successfully.
Figure 3-41 Checking the kernel version
lExample 2: SUSE 12.2
Upgrade the kernel of SUSE 12.2 to kernel-default-4.4.74-92.32.1.x86_64 or later to
support orderly hot swap of NVMe drives.
a.Run the uname -r command to check the current kernel version.
Figure 3-42 Kernel version
b.Log in to the official SUSE website to download the kernel upgrade package
c.Run the rpm -ivh kernel-default-4.4.74-92.32.1.x86_64.rpm command to upload
the upgrade package to the OS for installation.
Figure 3-43 Installing the upgrade package
d.Restart the OS after the installation is complete.
e.Run the uname -r command to check the kernel version.
If it is still the original kernel version, change the version in the grub.cfg file.
Step 6 Perform an orderly hot swap of an NVMe drive.
1.Obtain the slot number, B/D/F, and slot ID of the NVMe drive in the OS.
Generally, the slot number is the same as that marked on the compute node panel. If they
are inconsistent:
– Run find /sys -name nvme0n1 to query the bus number of the SSD device.
– Run lspci -vvv -xxx -s <B/D/F> |grep -i slot to query the slot No.
3 Basic Operations
Figure 3-44 Querying the slot No., B/D/F, and slot ID
Table 3-2 Mapping between physical slot numbers, B/D/F, and slot IDs
Configuratio
n
2 x 2.5"
Physical Slot
No.
Root Port
(B/D/F)
Device
(B/D/F)
Slot ID
Slot085:02.086:00.080
NVMe drive
configuration
Slot285:03.087:00.081
The Device (B/D/F) values vary depending on the setting of the VMD function. The
values in this table are based on the assumption that the VMD function is disabled.
2.Change the value of the a8 register. Otherwise, NVMe drives do not support orderly hot
swap.
3.Stop accessing the NVMe drive.
Stop I/O operations on the NVMe drive to be removed and unmount any file systems
mounted to the drive. If the drive is used by the mdadm utility, you also need to set the
drive to the fail state.
Run the following command:
umount /dev/NVMe drive letter
4.Run the following command to safely remove the drive:
echo n > /sys/bus/pci/slots/Slot ID/power
– In the command, the value of n can be 0 or 1. The value 0 indicates unloading a
drive, and the value 1 indicates loading a drive. You can load a drive only before the
unloaded drive is removed.
– Slot ID: see Table 3-2.
For example, to orderly hot remove the NVMe drive in slot 2, run the following
command:
echo 0 > /sys/bus/pci/slots/81/power
3 Basic Operations
5.Observe the indicator status of the NVMe drive. If the green indicator is off and the
yellow indicator blinks at 0.5 Hz, remove the NVMe drive slowly.
For details about the drive indicator status, see NVMe SSD Indicators.
For details about how to manually remove an NVMe drive, see 3.13 Removing a SAS/
SATA Drive.
6.Place the removed component in an ESD bag.
Step 7 Run the following command to restore the value of the a8 register to the original value:
setpci -s <B/D/F> a8.B=Original a8 register value
<B/D/F>: indicates the root port (B/D/F) of the NVMe drive. For details about the B/D/F
information, see Table 3-2.
Figure 3-47 Resetting the value of the a8 register of drive 2 (example)
If the value of the register is not restored to the original one, the orderly hot insertion of the NVMe
drives may be abnormal.
Step 8 Install a drive filler.
NOTE
Perform this operation only when a drive is not installed immediately.
----End
3.16 Installing an NVMe Drive
lIf the VMD function is enabled and the latest Intel RSTe driver is installed, the NVMe
drives support surprise hot swap.
– For details about how to enable the VMD function, see 12.9.1 Enabling VMD.
– Install the Intel RSTe driver. For details, see Huawei V5 Server RAID Controller
Card User Guide.
lIf the VMD function is disabled, the NVMe drives support only orderly hot swap.
For details about how to disable the VMD function, see 12.9.2 Disabling VMD.
3 Basic Operations
Procedure (VMD Enabled)
NOTE
l Use the Intelligent Computing Compatibility Checker to obtain information about the operating
systems supported by the VMD function.
l Install the NVMe drives one by one. Install the next NVMe drive only after the previous one is
installed completely.
l If the OS is RHEL 7.3 and the VMD function is enabled, the OS kernel 3.10.0-514.el7.x86_64 must
be used for surprise hot swap of NVMe drives. Otherwise, the operating system may restart.
l If the VMD function is enabled, the RHEL 7.3 graphical installation interface cannot be accessed.
You can use a USB flash drive to access the RHEL 7.3 graphical installation interface. For details,
see Installing RHEL 7.3 on an NVMe SSD.
Step 1 Take the spare part out of its ESD bag.
Step 2 Determine the slot and position of the drive.
Perform this operation only when a drive filler is installed.
Step 4 Install the driver for NVMe drives.
lInstall drivers for Intel series NVMe drives.
To ensure the stable running of NVMe drives, the following OSs are supported:
– The NVMe driver of the latest version must be installed for the Windows operating
system. For details about how to install the driver on the Windows OS, see IntelNVMe Install Guide_330547-003.pdf obtained after the driver package is
decompressed.
– For Linux, you are advised to use Linux versions listed on the compatibility list.
Certain Linux OSs do not support the NVMe feature. You need to download the
NVMe driver from the official Intel website for such Linux OSs. For details about
how to install the driver on the Linux OS, see
Intel_Linux_NVMe_Guide_330602-002.pdf obtained after the driver package is
decompressed.
– Download the NVMe driver of the latest version.
– If no such driver is available, use the latest drive driver provided by the OS.
lInstall drivers for Huawei series NVMe drives.
To ensure the stable running of NVMe drives, the following OSs are supported:
– The NVMe driver of the latest version must be installed for the Windows operating
system.
– Install the driver of the ES3000 V5 for a Linux OS. To enable hot swap, upgrade
Perform this operation only when a drive filler is installed.
Step 4 Install the driver for NVMe drives.
lInstall drivers for Intel series NVMe drives.
To ensure the stable running of NVMe drives, the following OSs are supported:
– The NVMe driver of the latest version must be installed for the Windows operating
system. For details about how to install the driver on the Windows OS, see IntelNVMe Install Guide_330547-003.pdf obtained after the driver package is
decompressed.
– For Linux, you are advised to use Linux versions listed on the compatibility list.
Certain Linux OSs do not support the NVMe feature. You need to download the
NVMe driver from the official Intel website for such Linux OSs. For details about
how to install the driver on the Linux OS, see
Intel_Linux_NVMe_Guide_330602-002.pdf obtained after the driver package is
decompressed.
– Download the NVMe driver of the latest version.
– If no such driver is available, use the latest drive driver provided by the OS.
lInstall drivers for Huawei series NVMe drives.
To ensure the stable running of NVMe drives, the following OSs are supported:
– The NVMe driver of the latest version must be installed for the Windows operating
system.
– Install the driver of the ES3000 V5 for a Linux OS. To enable hot swap, upgrade
For details about how to install the driver, see ES3000 V5 NVMe PCIe SSD User
Guide.
NOTE
– It is recommended that you install the latest driver.
– If Huawei NVMe driver is installed on a server using both Intel NVMe drives and Huawei
NVMe drives, the server cannot identify the Intel NVMe drives.
Step 5 Install NVMe drives.
NOTE
l When replacing an NVMe drive, install a new NVMe drive 30 seconds after the old one is removed.
l The NVMe drive power-on duration varies with the NVMe drives.
l Use the Intelligent Computing Compatibility Checker to obtain information about the OSs
supported by the NVMe drives with VMD disabled.
lWindows
a.Slowly insert the NVMe drive into the slot.
For details, see 3.14 Installing a SAS/SATA Drive.
b.Observe the drive indicators. If the green indicator is steady on and the yellow
indicator is off, the NVMe drive is detected and working properly.
For details, see NVMe SSD Indicators.
c.Check the status of the newly installed NVMe drive in the system.
nIf the drive status is normal, no further action is required.
nIf the newly installed NVMe drive cannot be identified, perform Step 5.d to
Step 5.e.
d.Go to Device Manager and scan for hardware changes.
e.Check the NVMe drive status on the OS.
lLinux
3 Basic Operations
a.Configure kernel parameters.
Log in to the OS, and open /boot/efi/EFI/redhat/grub.cfg (if the OS is installed in
UEFI mode) or /boot/grub2/grub.cfg (if the OS is installed in legacy mode).
nIf RHEL 7.3 is installed in UEFI mode:
1)Run the cat /proc/cmdline command. If the command output does not
contain "pciehp.pciehp_force=1 pci=pcie_bus_perf", you need to
configure kernel parameters.
Figure 3-50 Opening the configuration file
2)Run the vi/boot/efi/EFI/redhat/grub.cfg command. In the command
output, find the sentence that is displayed after the cat/proc/cmdline
command is executed in Step 5.a.1), and add a space and then
"pciehp.pciehp_force=1 pci=pcie_bus_perf" at the end of the sentence.
Do not add the content in a new line.
Figure 3-51 Configuring kernel parameters (1)
Figure 3-52 Configuring kernel parameters (2)
3 Basic Operations
3)Save the file and restart the compute node.
4)Run the cat /proc/cmdline command. If the command output contains
"pciehp.pciehp_force=1 pci=pcie_bus_perf", the kernel is configured
successfully.
Figure 3-53 Checking the configuration
nIf SUSE 12.2 is installed in UEFI:
1)Run the cat /proc/cmdline command. If the command output does not
contain "pciehp.pciehp_force=1 pci=pcie_bus_perf", you need to
configure kernel parameters.
2)Run the vi /boot/efi/EFI/sles/grub.cfg command. In the command
output, find the sentence that is displayed after the cat/proc/cmdline
command is executed in Step 5.a.1), and add a space and then
"pciehp.pciehp_force=1 pci=pcie_bus_perf" at the end of the sentence.
Do not add the content in a new line.
Figure 3-55 Configuring kernel parameters (1)
Figure 3-56 Configuring kernel parameters (2)
3 Basic Operations
3)Save the file and restart the compute node.
4)Run the cat /proc/cmdline command. If the command output contains
"pciehp.pciehp_force=1 pci=pcie_bus_perf", the kernel is configured
successfully.
Figure 3-57 Checking the configuration
b.Upgrade the OS kernel.
nExample 1: RHEL 7.3
Upgrade the kernel of RHEL 7.3 to kernel-3.10.0-514.26.2.e17.x86_64 or
later to support orderly hot swap of NVMe drives.
1)Run the uname -r command to check the current kernel version.
4)Restart the OS after the installation is complete.
5)Run the uname -r command to check the kernel version.
If it is still the original kernel version, change the version in the grub.cfg
file.
c.Slowly insert the NVMe drive into the slot.
For details, see 3.14 Installing a SAS/SATA Drive.
d.Observe the drive indicators. If the green indicator is steady on and the yellow
indicator is off, the NVMe drive is detected and working properly.
For details, see NVMe SSD Indicators.
e.Check the status of the newly installed NVMe drive in the system.
Step 6 Log in to the iBMC WebUI, and check whether the new component is normal.
For details, see FusionServer Pro E9000 Server iBMC User Guide.
Step 7 Determine whether to install the OS on the NVMe drive.
lIf yes, go to Step 8.
lIf no, no further action is required.
3 Basic Operations
Step 8 Install an OS on the NVMe drive.
NOTE
l When installing a supported OS on an NVMe drive, load the NVMe driver for the system to enable
the NVMe drives to be identified by the OS. Then, proceed with the OS installation.
l Only the UEFI mode supports OS installation on an NVMe drive.
1.Set the UEFI mode.
a.Press Del upon server startup to access the BIOS.
b.Open the boot configuration screen and set Boot Type to UEFI Boot Type.
c.Press F10 to save the setting and exit.
2.Install Windows Server 2012 (example) on an NVMe drive.
a.Install the OS.
If the NVMe drive cannot be detected when the system scans for the target
installation device, mount the NVMe driver file by using the virtual DVD drive or
floppy disk drive (FDD) on the server management WebUI.
b.Download the NVMe driver for Windows Server 2012, and create an ISO or IMG
file.
c.Load the NVMe driver on the installation UI. After the NVMe drive is detected,
To allow for servicing and adequate airflow, observe the following space and airflow
requirements:
lInstall the server in an access-restricted area.
lKeep the area in which the server is located clean and tidy.
lTo facilitate heat dissipation and maintenance, keep a clearance of 800 mm (31.50 in.)
between walls and the front and rear doors of the cabinet.
lDo not block the air intake vents. Otherwise, air intaking and heat dissipation will be
affected.
lThe air conditioning system in the equipment room provides enough wind to ensure
proper heat dissipation of all components.
4.1.2 Temperature and Humidity Requirements
To ensure continued safe and reliable equipment operation, install or position the system in a
well-ventilated, climate-controlled environment.
lUse temperature control devices all year long in any climates.
lIn dry and humid areas, maintain ambient humidity within range with humidifiers and
Table 4-1 Temperature and humidity requirements in the equipment room
ItemDescription
Temperature5ºC to 35ºC (41ºF to 95ºF)
Humidity8% RH to 90% RH (non-condensing)
4.1.3 Cabinet Requirements
lA general 19-inch cabinet with a depth of more than 1000 mm (39.37 in.) which
complies with the International Electrotechnical Commission 297 (IEC 297) standard.
lAir filters installed on cabinet doors.
4.2 Hardware Installation
4 Installation and Configuration
4.2.1 Installation Overview
Figure 4-1 Installation process
lProperly ground the server before installation to avoid damage to electronic components
from electrostatic discharge. Improper grounding may cause electrostatic discharge.
For details about how to prevent electrostatic discharge, see 10.1 Preventing
Electrostatic Discharge .
lBefore installing multiple components, read the installation instructions for all the
components and identify similar actions to simply the installation process.
Use the Intelligent Computing Compatibility Checker to obtain information about the
lConfigure the iBMC. The configuration method varies depending on the iBMC version.
For details, see FusionServer Pro E9000 Server iBMC User Guide.
lConfigure RAID settings. For details, see Huawei V5 Server RAID Controller Card
User Guide.
lConfigure BIOS settings. For details, see Huawei Server Purley Platform BIOS
Parameter Reference.
lInstall the operating system. For details, see Huawei Server OS Installation Guide.
lHandle alarms. For details, see FusionServer Pro E9000 Server V100R001 HMM
Alarm Handling.
lRectify faults. For details, see Huawei Servers Troubleshooting.
4.3.3 Changing Initial Passwords
4.3.3.1 Changing the Initial Password of the Default iBMC User
Scenarios
This section describes how to change the initial password of the default iBMC user on the
iBMC WebUI.
You can change the initial password of the default iBMC user on:
lMM910 WebUI
For details, see MM910 Management Module V100R001 User Guide.
lMM910 CLI
For details, see MM910 Management Module V100R001 Command Reference.
liBMC WebUI
For details, see FusionServer Pro E9000 Server iBMC User Guide.
liBMC CLI
For details, see FusionServer Pro E9000 Server iBMC User Guide.
NOTE
l The default user name of the iBMC is Administrator, and the default password is Admin@9000.
l For security purposes, change the initial password upon the first login and change the password
periodically.
l You are advised to use a password that meets complexity requirements or to enable the password
complexity check function.
l The password complexity check function is enabled by default.
Procedure
Step 1 Log in to the MM910 WebUI.
For details, see 12.3 Logging In to the MM910 WebUI.
Step 2 On the HMM Web page, choose System Management.
For details, see FusionServer Pro E9000 Server V100R001 HMM Alarm
Handling.
– If no, no further action is required.
----End
4.3.5 Setting the iBMC IP Address
Scenarios
This section describes how to set the iBMC IP address on the MM910 WebUI.
You can set the iBMC IP address on:
lBIOS
lMM910 WebUI
For details, see MM910 Management Module V100R001 User Guide.
lMM910 CLI
4 Installation and Configuration
For details, see MM910 Management Module V100R001 Command Reference.
liBMC WebUI
For details, see FusionServer Pro E9000 Server iBMC User Guide.
liBMC CLI
Run the following command: ipmcset -d ipaddr
For details, see FusionServer Pro E9000 Server iBMC User Guide.
Procedure (on the MM910 WebUI)
Step 1 Log in to the MM910 WebUI.
For details, see 12.3 Logging In to the MM910 WebUI.
Step 2 Set the iBMC IP address
To avoid conflicts with the network segment of the chassis management plane, the iBMC IP
address of the compute node cannot be configured in the 172.31.xx.xx network segment. If
you need to configure the iBMC IP address, change the IP addresses of the HMM and
compute node in the network segment of the chassis management plane. For details, see
section "Querying and Setting a Network Segment for the Chassis Management Plane
(subnet)" in MM910 Management Module V100R001 Command Reference.
1.On the HMM Web page, choose Chassis Settings.
The Chassis Settings page is displayed.
2.In the navigation tree, choose Network Settings > Node.
l To go to the BIOS front page, press F11 or F3.
l To boot from the network, press F12. Enter the password in the displayed dialog box.
l To access the Smart Provisioning GUI, press F6.
The screen for entering the BIOS password is displayed.
l The default BIOS password is Admin@9000.
l Press F2 to alternate between the English (US), French, and Japanese keyboards.
l For security purposes, change the administrator password periodically.
l The system will be locked if an incorrect password is entered three consecutive times. You can
restart the compute node to unlock it.
The Main screen of the Setup Utility program is displayed.
----End
4.3.7.1.2 Entering the BIOS (Cascade Lake)
Procedure
Step 1 Log in to the desktop of the compute node.
For details, see 12.7 Logging In to the Desktop of a Server.
Step 2 On the Remote Virtual Console, click
Step 3 Select Reset or Forced System Reset.
on the menu bar.
"Are you sure to perform this operation?" is displayed.
Step 3 Select Legacy Boot or UEFI Boot, and then press Enter.
NOTE
l The default boot mode is UEFI.
l For some OSs, if the capacity of the drive or RAID array for installing the OS is greater than 2 TB,
use the UEFI boot mode. For details, see the release notes of the OS.
l If the OS is installed on an NVMe drive, the boot mode must be the UEFI boot.
l The UEFI boot mode supports more boot devices than the legacy boot mode. The UEFI boot mode
is recommended if a compute node is configured with multiple boot devices. If the legacy mode is
set, some devices may fail to boot. If the legacy mode has to be set, disable serial port redirection or
NIC PXE based on service requirements so that the OS can start. For details, see sections "Setting
PXE for a NIC" and "Setting Serial Port Redirection" in Huawei Server Purley Platform BIOS
Parameter Reference.
Step 4 Select Boot Sequence and press Enter.
The Boot Sequence screen is displayed.
4 Installation and Configuration
NOTE
The default boot sequence is Hard Disk Drive > DVD-ROM Drive > PXE > Others.
Step 5 Select the target boot device and press F5 or F6 to change the boot order.
lPress F5 to move a boot option down.
lPress F6 to move a boot option up.
NOTE
The server boots in the order specified on this screen.
----End
4.3.7.3 Setting PXE for a NIC
If a compute node is configured with multiple NICs, you can set the PXE function for the
NICs on the BIOS.
NOTE
If multiple boot devices of the same type are configured for a compute node, you can set the system boot
sequence on the BIOS. For details about how to set the boot device, see Huawei Server Purley
MAC information of two network ports can be displayed on the PXE screen. PXE setting is not
supported. The PXE function is enabled for the two network ports by default.
----End
4.3.7.4 Setting the BIOS Password
For security purposes, change the administrator password upon the first login.
NOTE
l The password complexity check function is enabled by default.
l You are advised to use a password that meets complexity requirements or to enable the password
complexity check function.
l For security purposes, change the administrator password periodically.
Procedure
Step 1 On the BIOS main screen, choose Security.
4 Installation and Configuration
The Security screen is displayed.
Step 2 Select Manage Supervisor Password and press Enter.
The Manage Supervisor Password screen is displayed.
Step 3 Change the BIOS password.
NOTE
l The current password of the system administrator is required before you change the password. The
system will be locked if an incorrect password is entered three consecutive times. You can unlock
the system by restarting it.
l The default BIOS password is Admin@9000.
l The requirements for setting the administrator password are as follows:
–The password must be a string of 8 to 16 characters and contain special characters (including
spaces) and at least two types of uppercase letters, lowercase letters, and digits.
–The previous five passwords cannot be reused as a new password.
–After the administrator password is set, the Delete Supervisor Password parameter is
displayed, which can be used to clear the administrator password.
–If Simple Password is set to Enabled, the system does not verify the password complexity,