How it Works
Oracle
Loading...
X6-2
Table of contents
Loading...

Oracle X6-2 Service Manual

Download
Page 1

Oracle® Server X6-2 Service Manual

Part No: E62171-02
June 2016
Page 2
Page 3
Oracle Server X6-2 Service Manual
Part No: E62171-02
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:
U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.
This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.
Documentation Accessibility
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.
Access to Oracle Support
Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?
ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.
Page 4
Référence: E62171-02
Copyright © 2016, Oracle et/ou ses affiliés. Tous droits réservés.
Ce logiciel et la documentation qui l’accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d’utilisation et de divulgation. Sauf stipulation expresse de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, accorder de licence, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d’interopérabilité avec des logiciels tiers ou tel que prescrit par la loi.
Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu’elles soient exemptes d’erreurs et vous invite, le cas échéant, à lui en faire part par écrit.
Si ce logiciel, ou la documentation qui l’accompagne, est livré sous licence au Gouvernement des Etats-Unis, ou à quiconque qui aurait souscrit la licence de ce logiciel pour le compte du Gouvernement des Etats-Unis, la notice suivante s’applique:
U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.
Ce logiciel ou matériel a été développé pour un usage général dans le cadre d’applications de gestion des informations. Ce logiciel ou matériel n’est pas conçu ni n’est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer des dommages corporels. Si vous utilisez ce logiciel ou matériel dans le cadre d’ applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l’utilisation de ce logiciel ou matériel pour ce type d’applications.
Oracle et Java sont des marques déposées d’Oracle Corporation et/ou de ses affiliés. Tout autre nom mentionné peut correspondre à des marques appartenant à d’autres propriétaires qu’Oracle.
Intel et Intel Xeon sont des marques ou des marques déposées d’Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d’Advanced Micro Devices. UNIX est une marque déposée d’The Open Group.
Ce logiciel ou matériel et la documentation qui l’accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l’accès à des contenus, produits ou services tiers, ou à leur utilisation, sauf mention contraire stipulée dans un contrat entre vous et Oracle.
Accessibilité de la documentation
Pour plus d’informations sur l’engagement d’Oracle pour l’accessibilité à la documentation, visitez le site Web Oracle Accessibility Program, à l’adresse http://www.oracle.com/
pls/topic/lookup?ctx=acc&id=docacc.
Accès aux services de support Oracle
Les clients Oracle qui ont souscrit un contrat de support ont accès au support électronique via My Oracle Support. Pour plus d’informations, visitez le site http://www.oracle.com/
pls/topic/lookup?ctx=acc&id=info ou le site http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs si vous êtes malentendant.
Page 5

Contents

Using This Documentation ............. ................ ................ ................ ................ ... 11
Product Documentation Library .......................................................................  11
Feedback ............. ................ ................ ................ ................ ................ ......... 11
About the Oracle Server X6-2 ...........................................................................  13
Product Description .................................................................... ................ ...  13
About Controls and Connectors .......................................................................  14
Front Panel Controls and Indicators ............. ................ ................ .............  14
Server Back Panel View .........................................................................  16
About System Components ............... ................ ................ ................ ..............  17
Illustrated Parts Breakdown ..................................................................... 17
Customer-Replaceable Units ....................................................................  19
Field-Replaceable Units ..........................................................................  20
Server Internal Cables ............. ................ ................ ................ ...............  21
Troubleshooting and Diagnostics ........... ..........................................................  23
Troubleshooting Server Component Hardware Faults ... ................ ................ ........  23
Troubleshooting Server Hardware Faults ...................................................  24
Troubleshooting and Diagnostic Information ....... ................ ................ .......  27
Troubleshooting Using the Server Front and Rear Panel Status Indicators ... . 28
Troubleshooting System Cooling Issues ..................................................... 33
Troubleshooting Power Issues ................ ................ ................ ................ .. 35
Managing Server Hardware Faults Through the Oracle ILOM Fault
Management Shell .................... ................ ................ ................ .............  37
Troubleshooting With Diagnostic Tools .............................................................  38
Diagnostic Tools ...................................................................................  38
Diagnostic Tool Documentation ............... ................ ................ ................  39
Attaching Devices to the Server .............. ................ ................ ................ .........  40
▼ Attach Devices to the Rear of the Server ..............................................  40
Rear Panel Connector Locations .............. ................ ................ ................  40
5
Page 6
Contents
Configuring Serial Port Sharing ...............................................................  42
Server Operating System Names for the NVMe Storage Drives ...................... 43
Ethernet Port Boot Order and Device Naming ....... ................ ................ .....  43
Rear Panel Pinhole Switches ... ................ ................ ................ ................  44
Getting Help .. ................ ................ ................ ................ ................ ...............  45
Contacting Support .... ................ ................ ................ ................ ............  45
Locating the Chassis Serial Number .........................................................  46
Preparing for Service ........................................................................................  47
Safety Precautions .........................................................................................  47
Safety Symbols .............................................................................................  48
Electrostatic Discharge Safety ..........................................................................  48
FRU Key Identity Properties (KIP) Automated Update ......................................... 49
Related Information ...............................................................................  50
Required Tools ....... ................ ................ ................ ................ ................ .......  50
Preparing the Server for Component Replacement ...............................................  50
Powering Down the Server .....................................................................  51
▼  Disconnect Cables From the Server .....................................................  56
▼ Extend the Server to the Maintenance Position ......................................  56
▼  Remove the Server From the Rack ......................................................  58
▼ Take Antistatic Measures ...................................................................  59
▼ Open the Server Fan Door .................... ................ ................ .............  59
▼ Remove the Server Top Cover ............... ................ ................ .............  60
Servicing CRUs That Do Not Require Server Power-Off ...... ................ .............  63
Servicing Storage Drives (CRU) ............ ................ ................ ................ ..........  63
Storage Drives Hot-Plug Conditions .........................................................  64
Storage Drive Failure and RAID ..............................................................  64
Storage Drive Locations and Numbering ...................................................  64
Storage Drive Status Indicators ..... ................ ................ ................ ...........  65
Removing and Replacing a HDD or SSD Storage Drive .... ................ ...........  66
Removing and Replacing an NVMe Storage Drive Using Oracle Solaris ..........  69
Removing and Replacing an NVMe Storage Drive Using Oracle Linux ...........  72
Servicing Fan Modules (CRU) ......................................................................... 75
▼ Remove a Fan Module ............ ................ ................ ................ ..........  76
▼ Install a Fan Module . ................ ................ ................ ................ ........ 79
Servicing Power Supplies (CRU) .....................................................................  80
Power Supply Status Indicators ................................................................ 81
▼ Remove a Power Supply ...................................................................  81
6 Oracle Server X6-2 Service Manual • June 2016
Page 7
Contents
▼ Install a Power Supply ......................................................................  83
Servicing CRUs That Require Server Power-Off ............... ................ ................  85
Servicing the DIMMs (CRU) ..........................................................................  85
DIMM and Processor Physical Layout ......................................................  86
DIMM Population Scenarios ................................. ................ ................ ..  87
DIMM Population Rules .........................................................................  88
Populating DIMMs for Optimal System Performance ...................................  88
DIMM Operating Speeds ............................................................... .........  92
DIMM Rank Classification Labels ................ ................ ................ ............ 92
Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of
Failed DIMMs ......................................................................................  92
Using the Server Fault Remind Button ......................................................  92
▼ Identify and Remove the Failed DIMM ................................................  93
▼ Install a DIMM .. ................ ................ ................ ................ ..............  95
Servicing PCIe Risers (CRU) ..........................................................................  97
PCIe Riser Location and Differences ........................................................  98
▼ Remove a PCIe Riser From PCIe Slot 1 or 2 ... ................ ................ ......  99
▼ Install a PCIe Riser Into PCIe Slot 1 or 2 .... ................ ................ ........ 101
▼ Remove the PCIe Riser From PCIe Slots 3 and 4 .................................  103
▼ Install the PCIe Riser Into PCIe Slots 3 and 4 ......................................  105
Servicing PCIe Cards (CRU) .........................................................................  108
PCIe Slot Characteristics .......................................................................  108
▼ Remove a PCIe Card From PCIe Slot 1 or 2 ........................................  109
▼ Install a PCIe Card in PCIe Slot 1 or 2 ...............................................  110
▼ Remove a PCIe Card From PCIe Slot 3 ......... ................ ................ .....  111
▼  Install a PCIe Card in PCIe Slot 3 ........... ................ ................ ..........  112
Servicing the Internal USB Flash Drives (CRU) ................................................  113
▼ Remove an Internal USB Flash Drive ................................................. 114
▼  Install an Internal USB Flash Drive ...................................................  115
Servicing the Battery (CRU) .......... ................ ................ ................ ................  116
▼ Remove the Battery ........................................................................  116
▼ Install the Battery ...... ................ ................ ................ ................ .....  117
Servicing FRUs ................................................................................................  119
Servicing Processors (FRU) ........................................ ................ ................ ...  120
Selecting the Correct Processor Removal and Replacement Tool ................... 120
▼ Remove a Processor . ................ ................ ................ ................ .......  125
▼ Install a Processor ..........................................................................  131
7
Page 8
Contents
Servicing the Oracle PCIe NVMe Switch Card (FRU) ......... ................ ...............  137
▼ Remove the Oracle PCIe NVMe Switch Card From PCIe Slot 1 ..............  138
▼ Install the Oracle PCIe NVMe Switch Card in PCIe Slot 1 .....................  139
Servicing the Internal HBA Card (FRU) ..........................................................  140
▼ Remove the Internal HBA Card From PCIe Slot 4 ...... ................ ..........  141
▼ Install the Internal HBA Card in PCIe Slot 4 .......................................  145
Servicing the Disk Backplane (FRU) ............................................................... 150
▼ Remove the Disk Backplane ...... ................ ................ ................ .......  150
▼ Install the Disk Backplane ........... ................ ................ ................ ..... 153
Servicing the DVD Drive (FRU) ....................................................................  156
▼ Remove the DVD Drive ................ ................ ................ ................ ..  156
▼ Install the DVD Drive ..... ................ ................ ................ ................  158
Servicing the Front Indicator Module (FRU) ........... ................ ................ .........  158
▼  Remove the Front Indicator Module ................................................... 159
▼  Install the Front Indicator Module ........... ................ ..........................  161
Servicing the Motherboard (FRU) ................................................................... 162
▼ Remove the Motherboard ....... ................ ................ ................ ..........  162
▼ Install the Motherboard ...................................................................  169
Servicing the Internal HBA SAS Cable assembly ..............................................  173
▼ Remove the Internal HBA SAS Cable Assembly ....... ................ ...........  173
▼ Install the Internal HBA SAS Cable Assembly .....................................  176
Servicing the NVMe Cables ..........................................................................  177
▼ Removing the NVMe Cables ............................................................  178
▼ Install the NVMe Cables .................................................................. 180
Servicing the Power, FIM, Disk Backplane Data, and DVD Cables ........ ...............  181
▼ Removing the Power, FIM, Disk Backplane Data, and DVD Cables .......... 182
▼ Install the Power, FIM, Disk Backplane Data, and DVD Cables ...............  184
Returning the Server to Operation ..................................................................  187
Removing and Installing Server Filler Panels .................................................... 187
▼  Remove and Install Filler Panels .......................................................  188
▼ Install the Server Top Cover .....................................................................  188
▼ Remove Antistatic Measures ................ ................ ................ ................ ..... 190
▼  Reinstall the Server Into the Rack .............................................................  190
▼ Return the Server to the Normal Rack Position ............................................  191
▼ Reconnect Data Cables and Power Cords ................ ................ ................ ....  193
▼ Power On the Server ...............................................................................  194
Identifying the Server Ports ............... ................ ................ ................ .............  195
8 Oracle Server X6-2 Service Manual • June 2016
Page 9
Contents
Gigabit Ethernet Ports ..................................................................................  195
Network Management Port ............................................................................  196
Serial Management Port ...................................................... ................ ..........  197
Video Connector .............. ................ ................ ............................................  198
USB Ports ..................................................................................................  199
Setting Up BIOS Configuration Parameters ....................................................  201
Managing the BIOS Configuration ....... ................ ................ ................ ........... 201
Accessing the BIOS Setup Utility ...................................................................  202
BIOS Setup Utility Menus .......... ................ ................ ................ ..........  202
BIOS Key Mappings ............. ................ ................ ................ ...............  203
▼  Access BIOS Setup Utility Menus .....................................................  203
▼ Navigate BIOS Setup Utility Menus ............... ................ ................ ...  205
Using UEFI .. ................ ................ ................ ................ ................ ..............  206
Selecting Legacy BIOS or UEFI Boot Mode ....................... ................ .....  206
Switching Between Legacy BIOS and UEFI Boot Modes ............................  207
UEFI Boot Mode Advantages ..... ................ ................ ................ ...........  208
Configuration Utilities for Add-In Cards ............ ................ ................ ......  208
Using BIOS for Legacy Option ROM Resource Allocation .......... ................ ........ 209
Common BIOS Setup Utility Tasks ................................................................. 210
▼ Verify BIOS Factory Default Settings ..... ................ ................ ............  210
▼ Select Legacy BIOS or UEFI Boot Mode ............................................  211
▼ Select the Boot Device .............. ................ ................ ................ ......  213
▼ Configure TPM Support ..................................................................  215
▼  Configure SP Network Settings .........................................................  217
▼  Configure Option ROM Settings ............ ................ ................ ...........  220
▼  Configure I/O Resource Allocation ....................................................  223
▼ Exit BIOS Setup Utility ...................................................................  224
BIOS Setup Utility Menu Options ....................................................................  227
BIOS Main Menu Selections .........................................................................  227
BIOS Advanced Menu Selections .............. ................ ................ .....................  231
BIOS Advanced Menu Processor Configuration Options . ................ ............  232
BIOS Advanced Menu CPU Power Management Configuration Options ........  233
BIOS Advanced Menu Memory Configuration Option ...... ................ ..........  234
BIOS Advanced Menu USB Ports Options ...............................................  234
BIOS Advanced Menu Serial Port Console Redirection Options ...................  235
BIOS Advanced Menu Trusted Computing Options ........ ................ ...........  236
BIOS Advanced Menu Network Stack Options .........................................  237
9
Page 10
Contents
BIOS Advanced Menu BMC Network Configuration Options ......................  238
BIOS Advanced Menu iSCSI Configuration Options ..................................  241
BIOS Advanced Menu Ethernet Controller Options ...................................  243
BIOS IO Menu Selections ........ ................ ................ ................ ................ .....  244
BIOS IO Menu PCI Subsystem Settings Options ...... ................ ................ .  245
BIOS IO Menu IO Virtualization Options ................................................  246
BIOS IO Menu I/OAT Configuration Options ...........................................  246
BIOS IO Menu Internal Devices Options .................................................  247
BIOS IO Menu Add In Cards Options .... ................ ................ ................ .  247
BIOS Boot Menu Selections .......... ................ ................ ................ ................  248
BIOS Exit Menu Selections ...........................................................................  249
Monitoring Components and Identifying SNMP Messages .............................. 251
Monitoring Component Health and Faults Using Oracle ILOM ............................  251
Monitoring System Components ..................................................................... 252
System Chassis Components ..................................................................  253
Cooling Unit Components ........ ................ ................ ................ .............  255
Disk Backplane Components .................................................................  255
Memory Device Components .......... ................ ................ ................ .......  256
Power Unit Components ............... ................ ........................................  257
Processor Components ..........................................................................  258
System Board Components ....................................................................  259
System Firmware Components ...............................................................  261
Hard Disk Drive Components ..... ................ ................ ................ ...........  262
Identifying SNMP Trap Messages ..................................................................  263
Environmental Events ...................................................................................  263
Hard Disk Drive Events ................................................................................  265
Power Events ..............................................................................................  266
Fan Events .................................................................................................  270
Memory Events ........... ................ ................ ................ ................ ................  271
Entity Presence Events ............. ................ ....................................................  276
Physical Presence Events ..............................................................................  277
Index ............. ................ ................ ................ ................ ................ ................ ...  279
10 Oracle Server X6-2 Service Manual • June 2016
Page 11

Using This Documentation

■
Overview – This service manual explains how to remove and replace parts in the Oracle Server X6-2, and how to troubleshoot and maintain the system.
■
Audience – This guide is intended for trained technicians and authorized service personnel who have been instructed on the hazards within the equipment and qualified to remove and replace hardware.
■
Required knowledge – Advanced experience troubleshooting and replacing hardware.

Product Documentation Library

Documentation and resources for this product and related products are available at http://www.
oracle.com/goto/x6-2/docs.

Feedback

Provide feedback about this documentation at http://www.oracle.com/goto/docfeedback.
Using This Documentation 11
Page 12
12 Oracle Server X6-2 Service Manual • June 2016
Page 13

About the Oracle Server X6-2

These sections describe the controls, connectors, status indicators, system components, and replaceable components of the server.
Note - Always update the server with the latest firmware, drivers, and other hardware-related
software by downloading the latest software release package when you first receive the server, and for every new software release. For information about the software release packages and how to download the software, refer to “Getting Firmware and Software Updates” in Oracle
Server X6-2 Installation Guide.
Description Links
Review the product description. “Product Description” on page 13
Learn about server controls and connectors. “About Controls and Connectors” on page 14
Learn about system components. “About System Components” on page 17

Product Description

The Oracle Server X6-2 is an enterprise-class, one rack unit (1U) server. It supports the following components:
■
Up to two Intel processors. Processors with the following capabilities are supported:
■
22-core, 2.2 GHz, 145W processors
■
14-core, 2.6 GHz, 135W processors
■
10-core, 2.2 GHz, 85W processors
■
6-core, 3.4 GHz, 135W processors
■
Up to 12 DIMMs per processor for a maximum of 24 DDR4 DIMMs and a maximum of 1.5 TB of memory are supported in dual-processor systems. A maximum of 12 DDR4 DIMMs and a maximum of 768 GB of memory are supported in single-processor systems. RDIMMs (16 GB and 32 GB) and LRDIMMs (64 GB) are supported.
■
Four PCIe Gen3 slots in dual-processor systems: one slot, slot 4, is dedicated to the internal HBA. PCIe slot 1 is nonfunctional in single-processor systems.
■
Four Ethernet ports (NET0, NET1, NET2, and NET3 located on the rear of the server) are supported on dual-processor systems.
About the Oracle Server X6-2 13
Page 14

About Controls and Connectors

Note - Ethernet ports NET2 and NET3 are nonfunctional on single-processor systems.
■
Storage drive configurations include a DVD drive (optional) and up to eight hard disk drives (HDDs) or solid state drives (SSDs), with optional support for up to four PCIe-based NVMe (non-volatile memory express) SSD drives.
■
Two hot-pluggable, redundant power supplies.
■
An on-board Oracle Integrated Lights Out Manager (Oracle ILOM) service processor (SP) based on the Emulex Pilot3 chip.
About Controls and Connectors
The following sections describe the controls, indicators, connectors, and drives located on the front and rear panels.
■
“Front Panel Controls and Indicators” on page 14
■
“Server Back Panel View” on page 16

Related Information

■
“About System Components” on page 17
■
“Illustrated Parts Breakdown” on page 17

Front Panel Controls and Indicators

The following figure shows the controls, status indicators (LEDs), connectors, and drives on the front panel of the Oracle Server X6-2.
14 Oracle Server X6-2 Service Manual • June 2016
Page 15
About Controls and Connectors
FIGURE  1
Oracle Server X6-2 Front Panel
Figure Legend
1
Product Serial Number (PSN) label and Radio Frequency Identification (RFID) tag
2
Locator LED/Locator button: white
3
USB 2.0 connectors (2)
4
Service Required LED: amber
5
Power/OK LED: green
6
Power button
7
SP OK LED: green
8
Top Fan Fault LED: amber
9
Rear Power Supply (PS) Fault LED: amber
10
System Over Temperature Warning LED: amber
11
Storage drive 0 (HDD/SSD)
12
Storage drive 1(HDD/SSD)
13
Storage drive 2 (HDD/SSD/NVMe)
14
Storage drive 3 (HDD/SSD/NVMe)
15
Storage drive 4 (HDD/SSD/NVMe)
16
Storage drive 5 (HDD/SSD/NVMe)
17
Storage drive 6 (HDD/SSD)
About the Oracle Server X6-2 15
Page 16
About Controls and Connectors
18
Storage drive 7 (HDD/SSD)
19
(Optional) DVD drive
Related Information
■
“Server Back Panel View” on page 16
■
“About System Components” on page 17

Server Back Panel View

The following figure shows the Oracle Server X6-2 back panel and the location of power supplies, status indicators (LEDs), connectors, and PCIe slots.
FIGURE  2
Figure Legend
1
Power Supply (PS) 0
2
Power Supply (PS) 0 status indicators: Service Required LED: amber, AC OK LED: green
3
Power Supply (PS) 1
4
Power Supply (PS) 1 status indicators: Service Required LED: amber, AC OK LED: green
5
System status indicators: Locator LED: white, Service Required LED: amber, Power/OK LED: green
6
PCIe card slot 1 (This slot is for a standard PCIe card or the optional Oracle PCIe NVMe switch card. This slot is nonfunctional in single-processor systems.)
7
PCIe card slot 2
8
PCIe card slots 3 and 4 (Slot 4 is for the internal host bus adapter [HBA] card. Slot 4 is internal and is not visible from the rear of the server.)
Server Back PanelView
16 Oracle Server X6-2 Service Manual • June 2016
Page 17

About System Components

9
Oracle Integrated Lights Out Manager (ILOM) service processor (SP) network management 10/100/1000BASE­T port (NETMGT)
10
Serial management (SER MGT)/RJ-45 serial port
11
Network (NET) 100/1000/10000 port: NET 3 (Nonfunctional in single-processor systems.)
12
Network (NET) 100/1000/10000 port: NET 2 (Nonfunctional in single-processor systems.)
13
Network (NET) 100/1000/10000 port: NET 1
14
Network (NET) 100/1000/10000 port: NET 0
15
USB 2.0 connectors (2)
16
DB-15 video connector
Note - All of the PCIe slots comply with the PCI Express 3.0 specification and can
accommodate 25 watt PCIe3 cards.
Related Information
■
“Front Panel Controls and Indicators” on page 14
■
“About System Components” on page 17
About System Components
These sections describe the components of the server:
■
“Illustrated Parts Breakdown” on page 17
■
“Customer-Replaceable Units” on page 19
■
“Field-Replaceable Units” on page 20
■
“Server Internal Cables” on page 21

Related Information

■
“Servicing CRUs That Do Not Require Server Power-Off” on page 63
■
“Servicing CRUs That Require Server Power-Off” on page 85
■
“Servicing FRUs” on page 119

Illustrated Parts Breakdown

The following figure identifies the major components of the server.
About the Oracle Server X6-2 17
Page 18
About System Components
FIGURE  3
Server Illustrated Parts Breakdown
Callout Description
1 Storage drives
2 Front indicator module (FIM)
3 Disk backplane
18 Oracle Server X6-2 Service Manual • June 2016
Page 19
About System Components
Callout Description
4 Super capacitor (Energy storage module for the Oracle Storage 12 Gb/s SAS PCIe RAID HBA in PCIe slot
4.)
5 Fan modules
6 Motherboard
7 Processors
Note - In single-processor systems, neither a heatsink nor a processor socket filler are installed in the
processor socket 1 (P1). To protect the delicate processor socket pins, the cover that comes on the motherboard from manufacturing is left in place.
8 Heatsinks
9 Battery
10 USB flash drives
11 Air baffle
12 Top cover
13 (Optional) PCIe card in slot 3
14 Oracle Storage 12 Gb/s SAS PCIe RAID HBA card in PCIe slot 4
15 (Optional) PCIe card in slot 2
16 (Optional) Oracle PCIe NVMe switch card or standard PCIe card in slot 1
Note - PCIe slot 1 is nonfunctional in single-processor systems.
17 PCIe risers (3)
18 DIMMs
Note - A maximum of 12 DIMMs are supported in single-processor systems and the DIMMs must be
installed in sockets associated with processor 0 (P0). No DIMM filler panels are required in vacant DIMM sockets associated with processors 0 (P0) or 1 (P1).
19 Power supplies
20 Server chassis
21 (Optional) DVD drive

Customer-Replaceable Units

The following table lists the customer-replaceable units (CRUs) in the server and directs you to the replacement instructions.
CRU Description Replacement Instructions
Battery Lithium coin-cell battery that powers the CMOS BIOS
DIMMs Add or replace memory modules in the system. “Servicing the DIMMs (CRU)” on page 85
Storage drives Supports SAS hard-disk drives (HHDs), SATA solid-
Internal USB flash drives
and real-time clock.
state drives (SSDs), and NVMe SSDs.
Supports two internal USB flash drives. “Servicing the Internal USB Flash Drives
“Servicing the Battery (CRU)” on page 116
“Servicing Storage Drives (CRU)” on page 63
(CRU)” on page 113
About the Oracle Server X6-2 19
Page 20
About System Components
CRU Description Replacement Instructions
Fan modules Four fan modules for cooling the motherboard assembly
and installed components.
PCIe cards Optional add-on PCIe cards that can expand the
functionality of the server.
Note - The Oracle Storage 12 Gb/s SAS PCIe RAID
HBA card and the optional Oracle PCIe NVMe switch card are field replaceable units (FRUs) and should only be serviced by authorized Oracle Service personnel.
PCIe risers Houses and connects the PCIe cards. “Servicing PCIe Risers (CRU)” on page 97
Power supplies Two fully redundant AC power supplies. “Servicing Power Supplies (CRU)” on page 80
“Servicing Fan Modules (CRU)” on page 75
“Servicing PCIe Cards (CRU)” on page 108
Related Information
■
“Field-Replaceable Units” on page 20
■
“Illustrated Parts Breakdown” on page 17
■
“Servicing CRUs That Do Not Require Server Power-Off” on page 63
■
“Servicing CRUs That Require Server Power-Off” on page 85
■
“Server Internal Cables” on page 21

Field-Replaceable Units

The following table lists the field-replaceable units (FRUs) in the server and directs you to the replacement instructions.
FRU Description Replacement Instructions
Processor and heatsink
Oracle Storage 12 Gb/s SAS PCIe RAID HBA card
Oracle PCIe NVMe switch card
Disk backplane Serves as the interface between storage drives and the
DVD drive DVD drive “Servicing the DVD Drive (FRU)” on page 156
Front indicator module (FIM)
Motherboard assembly
SAS cables Provide signals and backup power between the disk
Carries out the instructions of the system. “Servicing Processors (FRU)” on page 120
HDD and SSD storage drives controller “Servicing the Internal HBA Card (FRU)” on page 140
NVMe storage drives controller “Servicing the Oracle PCIe NVMe Switch Card
internal host bus adapter (HBA) card and Oracle PCIe NVMe switch card.
Contains the front panel controls, indicators, and USB ports.
Provides connectors for the fans, DIMMs, processors, PCIe risers, internal USB ports, and power supplies.
backplane and the super capacitor and the Oracle Storage 12 Gb/s SAS PCIe RAID HBA.
(FRU)” on page 137
“Servicing the Disk Backplane (FRU)” on page 150
“Servicing the Front Indicator Module (FRU)” on page 158
“Servicing the Motherboard (FRU)” on page 162
“Servicing the Internal HBA SAS Cable assembly” on page 173
20 Oracle Server X6-2 Service Manual • June 2016
Page 21
FRU Description Replacement Instructions
NVMe cable Provide signals between the disk backplane and the
Oracle PCIe NVMe switch card.
“Servicing the NVMe Cables” on page 177
Related Information
■
“Customer-Replaceable Units” on page 19
■
“Illustrated Parts Breakdown” on page 17
■
“Servicing FRUs” on page 119
■
“Server Internal Cables” on page 21

Server Internal Cables

The following table lists the cables used in the server and directs you to the replacement instructions.
About System Components
Cable Name Part Number Description Removal/Replacement
Internal HBA SAS cable assembly
Oracle PCIe NVMe switch card cable assembly
Power cable 7064125 Connects the disk backplane
FIM cable 7064123 Ribbon cable that provides a
Disk backplane data cable 7064127 Ribbon cable that connects
DVD data/power cable 7064128 Combo cable that connects the
7076125 Eight-disk, SAS, combo cable
assembly that connects the internal host bus adapter (HBA) card to the disk backplane and the super capacitor.
7089783 Eight-disk backplane, combo
cable assembly that connects the Oracle PCIe NVMe switch card to the disk backplane.
to the motherboard. Provides power from the motherboard to the disk backplane.
signal and power connection between the motherboard to the Front Module Indicator (FIM)
the disk backplane to the motherboard. Provides a data connection between the two components.
DVD drive to the motherboard. Provides a data connection between the two components and provides power from the motherboard to the DVD drive.
Instructions
See “Servicing the
Internal HBA SAS Cable assembly” on page 173
See “Servicing the NVMe
Cables” on page 177
See “Servicing the Power, FIM,
Disk Backplane Data, and DVD Cables” on page 181
See “Servicing the Power, FIM,
Disk Backplane Data, and DVD Cables” on page 181
See “Servicing the Power, FIM,
Disk Backplane Data, and DVD Cables” on page 181
See “Servicing the Power, FIM,
Disk Backplane Data, and DVD Cables” on page 181
About the Oracle Server X6-2 21
Page 22
About System Components
Related Information
■
“Customer-Replaceable Units” on page 19
■
“Illustrated Parts Breakdown” on page 17
■
“Servicing FRUs” on page 119
22 Oracle Server X6-2 Service Manual • June 2016
Page 23

Troubleshooting and Diagnostics

This section includes information about troubleshooting hardware component faults for the Oracle Server X6-2. It contains the following topics.
Description Link
Maintenance-related information and procedures that you can use to troubleshoot and repair server hardware issues.
Information about software and firmware diagnostic tools that you can use to isolate problems, monitor the server, and exercise the server subsystems.
Information about attaching devices to the server to perform troubleshooting.
Information about contacting Oracle support. “Getting Help” on page 45

Troubleshooting Server Component Hardware Faults

“Troubleshooting Server Component Hardware Faults” on page 23
“Troubleshooting With Diagnostic Tools” on page 38
“Attaching Devices to the Server” on page 40
This section contains maintenance-related information and procedures that you can use to troubleshoot and repair server hardware issues. The following topics are covered.
Description Section Links
Troubleshooting overview information and procedure. “Troubleshooting Server Hardware
Resources for troubleshooting and diagnostic information.
Discerning the server state using the front panel indicators.
Causes, actions, and preventative measures for problems related to the cooling subsystem.
Causes, actions, and preventative measures for problems related to the power subsystem.
Faults” on page 24
“Troubleshooting and Diagnostic Information” on page 27
“Troubleshooting Using the Server Front and Rear Panel Status Indicators” on page 28
“Troubleshooting System Cooling Issues” on page 33
“Troubleshooting Power Issues” on page 35
Troubleshooting and Diagnostics 23
Page 24
Troubleshooting Server Component Hardware Faults

Troubleshooting Server Hardware Faults

When a server hardware fault event occurs the system lights the Service Required LED and captures the event in the system event log (SEL). If you have set up notifications through Oracle ILOM, you also receive an alert through the notification method you chose. When you become aware of a hardware fault, you should address it immediately.
To investigate a hardware fault, see the following:
■
“Basic Troubleshooting Process” on page 24
■
“Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface” on page 25
Basic Troubleshooting Process
Use the following process to address a hardware fault (for the step-by-step procedure, see
“Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface” on page 25).
1. Identify the server subsystem containing the fault.
You can use Oracle ILOM to identify the failed component.
2.
Review the Oracle Server X6-2 Product Notes.
Once you have identified the hardware issue, review the Oracle Server X6-2 Product Notes. This document contains up-to-date information about the server, including hardware-related issues.
3. Prepare the server for service using Oracle ILOM.
If you have determined that the hardware fault requires service (physical access to the server), use Oracle ILOM to take the server offline, activate the Locate LED, and power off the server.
4. Prepare the service work space.
Before servicing the server, prepare the work space, ensuring electrostatic discharge (ESD) protection for the server and components.
For handling instructions for devices sensitive to ESD, see “Electrostatic Discharge
Safety” on page 48.
5. Service components.
To service the components, see the removal, installation, and replacement procedures in this document.
Note - A component designated as FRU must only be replaced by authorized Oracle
Service personnel. Contact Oracle Services.
6. Clear the fault in Oracle ILOM.
24 Oracle Server X6-2 Service Manual • June 2016
Page 25
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Depending on the component, you might need to clear the fault in Oracle ILOM. Generally, components that have a FRU ID clear the fault automatically.
Related Information:
■
See “Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface” on page 25
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Note - The screens shown in this procedure might differ from those for your server.
This procedure uses the basic troubleshooting steps described in “Basic Troubleshooting
Process” on page 24.
Use this procedure to troubleshoot hardware faults with the Oracle ILOM web interface and, if necessary, prepare the server for service.
Before You Begin
1.
Note - This procedure provides one basic approach to troubleshooting hardware faults. It uses
a combination of the Oracle ILOM web interface and the command-line interface (CLI). This procedure can be performed using only the Oracle ILOM CLI interface. For more information
about the Oracle ILOM web interface and CLI, refer to http://www.oracle.com/goto/ilom/
docs.
■
Obtain the latest version of the Oracle Server X6-2 Product Notes at http://www.oracle.
com/goto/x6-2/docs.
Log in to the server SP Oracle ILOM web interface.
Open a browser and direct it using the IP address of the server SP. At the login screen, type a user name (with administrator privileges) and password at the log-in screen. The Summary Information screen appears.
The Status section of the Summary Information screen provides information about the server subsystems, including:
■
Processors
■
Memory
■
Power
■
Cooling
■
Storage
■
Networking
Troubleshooting and Diagnostics 25
Page 26
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
2.
In the Status section of the Oracle ILOM Summary Information screen, identify the server subsystem that requires service.
In the above example, the Status screen shows that the Processor subsystem requires service. This indicates that a hardware component within the subsystem is in a fault state.
3.
To identify the component, click on Processors in the Status section.
The Oracle ILOM Processors screen appears.
The above example shows the Processors subsystem screen and indicates that CPU 0 has a fault.
4.
To get more information, click one of the Open Problems links.
26 Oracle Server X6-2 Service Manual • June 2016
Page 27
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
The Open Problems screen provides detailed information, such as the time the event occurred, the component and subsystem name, and a description of the issue. It also includes a link to an Oracle KnowledgeBase article.
Tip - The System Log provides a chronological list of all the system events and faults that
have occurred since the log was last reset and includes additional information, such as severity levels and error counts. The System Log also includes information on device not reported in the Subsystem Summary screen. To access the System Log, in the left panel, click System Log.
In this example, the hardware fault with CPU 0 requires local/physical access to the server.
5.
Before accessing the physical server, review the Oracle X6-2 Server Product Notes for information related to the issue or the component.
The product notes contain up-to-date information about the server, including hardware-related issues.
6.
To prepare the server for service, see “Preparing for Service” on page 47.
7.
Service the component.
After servicing the component, you might need to clear the fault in Oracle ILOM. For more information, refer the service procedures for the component.

Troubleshooting and Diagnostic Information

The following table lists diagnostic- and troubleshooting-related procedures and references that can assist you with resolving server issues.
Description Link
Diagnostic information for the x86 servers, including procedures for performing runtime and firmware-based tests, using Oracle ILOM, and running U-Boot and UEFI Diagnostics tests to exercise the system and isolate subtle and intermittent hardware-related problems.
Administrative information for the Oracle X-6 series servers, including information about how to use the Oracle ILOM system event log (SEL) to identify a problem's possible source.
Information about troubleshooting front and rear panel status indicators.
Information about how to manage server hardware faults using the Oracle ILOM Fault Management Shell.
Oracle x86 Servers Diagnostics, Applications, and Utilities Guide for Servers with Oracle ILOM 3.1 and Oracle ILOM 3.2
Oracle X6 Series Servers Administration Guide
“Troubleshooting Using the Server Front and Rear Panel Status Indicators” on page 28
“Managing Server Hardware Faults Through the Oracle ILOM Fault Management Shell” on page 37
Troubleshooting and Diagnostics 27
Page 28
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface

Troubleshooting Using the Server Front and Rear Panel Status Indicators

The following sections describe the operation of the server status indicators.
■
“Server Boot Process and Normal Operating State Indicators” on page 28
■
“Server System-Level Status Indicators” on page 29
■
“Server Fan Status Indicators” on page 30
■
“Storage Drive Status Indicators” on page 30
■
“Power Supply Status Indicators” on page 31
■
“Network Management Port Status Indicators” on page 31
■
“Ethernet Ports Status Indicators” on page 32
■
“Motherboard Status Indicators” on page 32
Related Information
■
“Front Panel Controls and Indicators” on page 14
■
“Server Back Panel View” on page 16
Server Boot Process and Normal Operating State Indicators
A normal server boot process involves two indicators, the service processor (SP) OK/Fault LED indicator and the Power/OK LED indicator.
When AC power is connected to the server, the server boots into standby power mode:
1. The SP OK LED blinks slowly (0.5 seconds on, 0.5 seconds off) while the SP is starting, and the main Power/OK LED remains off until the SP is ready.
2. After a few minutes, the main Power/OK LED flashes the standby blink pattern (0.1 seconds on, 2.9 seconds off), indicating that the SP (and Oracle ILOM) is ready for use. In standby power mode, the server is not initialized or fully powered on at this point.
When powering on the server (either by the Power button or Oracle ILOM), the server boots to full power mode:
1. The Power/OK LED blinks slowly (0.5 seconds on, 0.5 seconds off), and the SP OK LED remains lit (no blinking).
2. When the server has successfully booted, the Power/OK LED remains lit. When the Power/ OK LED and the SP OK/Fault LED indicators remain lit, the server is in full power mode.
Note - The green Power/OK LED indicator and the green SP OK/Fault indicator remain lit (no
blinking) when the server is in a normal operating state.
28 Oracle Server X6-2 Service Manual • June 2016
Page 29
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Server System-Level Status Indicators
There are seven, system-level status indicators (LEDs), some of which are located on both the server front panel and the server back panel. The following table describes these indicators.
TABLE 1
Status Indicator Name
Locator LED and button
Service Required
Power/OK Green Indicates the operational state of the chassis.
SP OK Green Indicates the state of the service processor.
Server System-Level status Indicators
Icon Colors State and Meaning
White Indicates the location of the servers.
■ OFF – Server is operating normally.
■ FAST BLINK – Use Oracle ILOM to activate this LED to enable you to locate a particular system quickly and easily.
■ Pressing the Locate button will toggle the LED fast blink on or off.
Amber Indicates the fault state of the server.
■ OFF – The server is operating normally.
■ STEADY ON – A fault is present on the server. This LED indicator
■ Lights whenever a fault indicator lights for a replaceable component on the server.
Note - When this indicator is lit, a system console
message might appear that includes a recommended service action.
■ OFF – AC power is not present or the Oracle ILOM boot is not complete.
■ Standby BLINK – Standby power is on, but the chassis power is off and the Oracle ILOM SP is running.
■ SLOW BLINK – Startup sequence has been initiated on the host. This pattern should begin soon after you power on the server. This status indicates either: Power-on self-test (POST) code checkpoint tests are running on the server host system, or the host is transitioning from the powered-on state to the standby state on shutdown.
■ STEADY ON – The server is powered on, and all host POST code checkpoint tests are complete. The server is in one of the following states: the server host is booting the operating system (OS), or the server host is running the OS.
■ OFF – Service processor (SP) is not running.
■ SLOW BLINK – SP is booting.
■ STEADY ON – SP is fully operational.
Troubleshooting and Diagnostics 29
Page 30
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Status Indicator Name
Top Fan TOP Amber Indicates that one or more of the internal fan modules
Rear Power Supply Fault
System Over Temperature Warning
Icon Colors State and Meaning
have failed.
■ OFF – Indicates steady state; no service is required.
■ STEADY ON – Indicates service required.
REAR Amber Indicates that one of the server power supplies has
failed.
■ OFF – Indicates steady state; no service is required.
■ STEADY ON – Indicates service required; service the power supply.
Amber Indicates a warning for an over temperature
condition.
■ OFF – Normal operation; no service is required.
■ STEADY ON – The system is experiencing an overtemperature warning condition.
Note - This is a warning indication, not a fatal
overtemperature. Failure to correct this might result in the system overheating and shutting down unexpectedly.
Server Fan Status Indicators
Each fan module has one status indicator (LED). The LEDs are located on the chassis mid­wall adjacent to and aligned with the fan modules and are visible when the top cover fan door is open.
TABLE 2
Status Indicator Name
Fan Status Amber ■ Off – The fan module is correctly installed and operating within specification.
Server Fan Status Indicators
Icon Color State and Meaning
■ Amber – The fan module is faulty. The front TOP FAN LED and the front and rear panel Service Required LEDs are also lit if the system detects a fan module fault.
Storage Drive Status Indicators
There are three status indicators (LEDs) on each drive.
30 Oracle Server X6-2 Service Manual • June 2016
Page 31
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
TABLE 3
Status Indicator Name
OK/Activity Green ■ OFF – Power is off or the installed drive is not recognized by the system.
Service Required
OK to Remove Blue ■ STEADY ON – The storage drive can be removed safely during a hot-plug operation.
Server Front Storage Drive Indicators
Icon Color State and Meaning
■ STEADY ON – The drive is engaged and is receiving power.
■ Random BLINK – There is disk activity. The indicator blinks on and off to indicate activity.
Amber ■ OFF – The storage drive is operating normally.
■ STEADY ON – The system has detected a fault with the storage drive.
■ OFF – The storage drive has not been prepared for removal.
Power Supply Status Indicators
There are two status indicators (LEDs) on each power supply. These indicators are visible from the rear of the server.
TABLE 4
Status Indicator Name
AC OK/DC OK
Service Required
Server Power Supply Indicators
Icon Color State and Meaning
Green ■ OFF – No AC power is present.
■ SLOW BLINK – Normal operation. Input power is within specification. DC output voltage is not enabled.
■ STEADY ON – Normal operation. Input AC power and DC output voltage are within specification.
Amber ■ OFF – Normal operation. No service action is required.
■ STEADY ON – The power supply (PS) has detected a PS fan failure, PS overtemperature, PS over current, or PS over or under voltage.
Network Management Port Status Indicators
The server has one 10/100/1000 BASE-T Ethernet management domain interface, labeled NET MGT. There are two status indicators (LEDs) on this port. These indicators are visible from the rear of the server.
TABLE 5
Status Indicator Name
Activity Top right Green ■ ON – Link up.
Network Management Port Status Indicators
Location Color State and Meaning
■ OFF – No link or down link.
Troubleshooting and Diagnostics 31
Page 32
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Status Indicator Name
Link speed Top left Green ■ Green ON – 1000BASE-T link.
Location Color State and Meaning
■ FLASHING – Packet activity.
■ OFF – 10/100BASE-T link.
Ethernet Ports Status Indicators
The server has four Gigabit Ethernet ports (NET 3, NET 2, NET 1, and NET 0). There are two status indicators (LEDs) on each port. These indicators are visible from the rear of the server.
Note - Ethernet ports NET 2 and NET 3 are nonfunctional in single-processor systems.
TABLE 6
Status Indicator Name
Activity Top left Green ■ ON – No function.
Link speed Top right Bicolored: Amber/
Gigabit Ethernet Port Status Indicators
Location Color State and Meaning
■ OFF– No activity.
■ FLASHING – Packet activity.
Green
■ OFF – 100BASE-T link.
■ Amber ON – 1000BASE-T link.
■ Green ON – 10GBBASE-T link.
■ FLASHING – No function.
Motherboard Status Indicators
The motherboard contains the following status indicators (LEDs).
■
DIMM Fault Status Indicators - Each of the 24 DIMM sockets on the motherboard has an amber fault status indicator (LED) associated with it. If Oracle ILOM determines that a DIMM is faulty, pressing the Fault Remind button on the motherboard signals the service processor to light the fault LED associated with the failed DIMM. For more information on DIMM fault status indicators and the location of the Fault Remind Button, see “Servicing
the DIMMs (CRU)” on page 85.
■
Processor Fault Status Indicators: The motherboard includes a fault status indicator (LED) adjacent to each of the two processor sockets. These LEDs indicate when a processor fails. Pressing the Fault Remind button on the motherboard signals the service processor to light the fault status indicators associated with the failed processors. For more information on processor fault status indicators and the location of the Fault Remind Button, see
“Servicing Processors (FRU)” on page 120.
■
Fault Remind Status Indicator: This status indicator (LED) is located next to the Fault Remind button and is powered from the super capacitor that powers the fault LEDs on the
32 Oracle Server X6-2 Service Manual • June 2016
Page 33
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
motherboard. This LED lights to indicate that the fault remind circuitry is working properly in cases where no components have failed and, as a result, none of the component fault LEDs illuminate. For more information on the fault remind status indicator and the location of the Fault Remind Button, see “Using the Server Fault Remind Button” on page 92.
■
STBY PWRGD Status Indicator - This green status indicator (LED) is labeled STBY PWRGD and is located on the rear of the server near PCIe slot 2. This LED lights to inform the service technician that the motherboard is receiving standby power from at least one of the power supplies. This LED is provided to help prevent service actions on the server internal components while the AC power cords are installed and power is being supplied to the server.
Note - If a PCIe card is installed in PCIe slot 2, this LED might not be visible.
Note - The service instructions for all server internal components require that all AC power
be removed from the power supplies prior to removing the server top cover.

Troubleshooting System Cooling Issues

Maintaining the proper internal operating temperature of the server is crucial to the health of the server. To prevent server shutdown and damage to components, address over temperature- and hardware-related issues as soon as they occur. If your server has a temperature-related fault, the cause of the problem might be:
■
“External Ambient Temperature Too High” on page 33
■
“Airflow Blockage” on page 34
■
“Cooling Areas Compromised” on page 34
■
“Hardware Component Failure” on page 34
External Ambient Temperature Too High
Server component cooling relies on the movement of cool air through the server. The cool air is pulled into the server from its external environment. If the ambient temperature of the server's external environment is too high, cooling does not occur, and the internal temperature of the server and its components increases. This can cause poor server performance or a failure of one or more components.
Action: Check the ambient temperature of the server space against the environmental specifications for the server. If the temperature is not within the required operating range, remedy the situation immediately.
Troubleshooting and Diagnostics 33
Page 34
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Prevention: Periodically check the ambient temperature of the server space to ensure that it is within the required range, especially if you have made any changes to the server space (for example, added additional servers). The temperature must be consistent and stable.
Airflow Blockage
The server cooling system uses fans to pull cool air in from the server front intake vents and exhaust warm air out the server back panel vents. If the front or back vents are blocked, the airflow through the server is disrupted and the cooling system fails to function properly causing the server internal temperature to rise.
Action: Inspect the server front and back panel vents for blockage from dust or debris. Additionally, inspect the server interior for improperly installed components or cables that can block the flow of air through the server.
Prevention: Periodically inspect and clean the server vents using an ESD certified vacuum cleaner. Ensure that all components, such as cards, cable, fans, air baffles and dividers are properly installed. Never operate the server without the top cover installed.
Cooling Areas Compromised
To function properly, the server has cooling areas that are maintained by an air baffle, component filler panels, and the server top cover. These server components need to be in place for the server to function as a sealed system. If internal cooling areas are compromised, the server cooling system, which relies on the movement of cool air through the server, cannot function properly, and the airflow inside the server becomes chaotic and non-directional.
Action: Inspect the server interior to ensure that the air baffle is properly installed. Ensure that all external-facing slots (storage drive, DVD, PCIe) are occupied with either a component or a component filler panel. Ensure that the server top cover is in place and sits flat and snug on top of the server.
Prevention: When servicing the server, ensure that the air baffle is installed correctly and that the server has no unoccupied external-facing slots. Never operate the server without the top cover installed.
Hardware Component Failure
Components, such as power supplies and fan modules, are an integral part of the server cooling system. When one of these components fails, the server internal temperature can rise. This rise in temperature can cause other components to enter into an over-temperature state. Additionally, some components, such as processors, might overheat when they are failing, which can also generate an over-temperature event.
34 Oracle Server X6-2 Service Manual • June 2016
Page 35
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
To reduce the risk related to component failure, power supplies and fan modules are installed in pairs to provide redundancy. Redundancy ensures that if one component in the pair fails, the other functioning component can continue to maintain the subsystem. For example, power supplies serve a dual function; they provide both power and airflow. If one power supply fails, the other functioning power supply can maintain both the power and the cooling subsystems.
Action: Investigate the cause of the over-temperature event, and replace failed components immediately. For hardware troubleshooting information, see “Troubleshooting Server Hardware
Faults” on page 24.
Prevention: Component redundancy is provided to allow for component failure in critical subsystems, such as the cooling subsystem. However, once a component in a redundant system fails, the redundancy no longer exists, and the risk for server shutdown and component failures increases. Therefore, it is important to maintain redundant systems and replace failed components immediately.

Troubleshooting Power Issues

If your server does not power on, the cause of the problem might be:
■
“AC Power Connection” on page 35
■
“Power Supplies (PSUs)” on page 36
■
“Top Cover” on page 37
AC Power Connection
The AC power cords are the direct connection between the server power supplies and the power sources. The server power supplies need separate stable AC circuits. Insufficient voltage levels or fluctuations in power can cause server power problems. The power supplies are designed to operate at a particular voltage and within an acceptable range of voltage fluctuations (refer to
“Electrical Power Requirements” in Oracle Server X6-2 Installation Guide).
Action: Check that both AC power cords are connected to the server. Check that the correct power is present at the outlets and monitor the power to verify that it is within the acceptable range. You can verify proper connection and operation by checking the PSU indicator panels, which are located at the back of the server on the PSUs. Lit green AC OK and DC OK indicators show a properly functioning PSU. An amber AC OK indicator indicates that the AC power to the PSU is insufficient.
Prevention: Use the AC power cord reclosable retaining straps and position the cords to minimize the risk of accidental disconnection. Ensure that the AC circuits that supply power to the server are stable and not overburdened. For more information on the reclosable retaining straps, see “Power Supplies (PSUs)” on page 36.
Troubleshooting and Diagnostics 35
Page 36
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
Power Supplies (PSUs)
The server power supplies (PSUs) provide the necessary server voltages from the AC power outlets. If the PSUs are inoperable, unplugged, or disengaged from the internal connectors, the server cannot power on.
Note - The reclosable retaining straps shown in the following figure should be used to secure
the power cord connectors to the rear of the power supplies. The reclosable retaining straps minimize the risk of accidental disconnection.
Action: Check that the AC cables are connected to both PSUs. Check that the PSUs are operational (the PSU indicator panel should have a lit green AC OK indicator). Ensure that the PSU is properly installed. A PSU that is not fully engaged with its internal connector does not have power applied and does not have a lit green AC OK indicator.
Prevention: When a power supply fails, replace it immediately. To ensure redundancy, the server has two PSUs. This redundant configuration prevents server downtime, or an unexpected shutdown, due to a failed PSU. The redundancy allows the server to continue to operate if one of the PSUs fails. However, when a server is being powered by a single PSU, the redundancy no longer exists, and the risk for downtime or an unexpected shutdown increases. When
36 Oracle Server X6-2 Service Manual • June 2016
Page 37
Troubleshoot Hardware Faults Using the Oracle ILOM Web Interface
installing a power supply, ensure that it is fully seated and engaged with its connector inside the drive bay. A properly installed PSU has a lit green AC OK indicator.
Top Cover
The server top cover is an important component for a properly functioning server. The top cover is vital to the server cooling subsystem because it helps to maintain the pressure areas within the server, and the top cover also protects against accidental exposure to hazardous voltages and damage to internal components. For these reasons, the server top cover is interlocked to the server power. The interlock is created by a switch that is mounted inside the server on the metal bracket just behind the DIMMs. The switch engages when the cover is installed, allowing power to be applied to the server. When the cover is not installed, the server cannot power on. If the server is powered on when the cover is removed, power is immediately switched off.
Action: If the server does not power on, check that the switch is intact and properly aligned. Ensure that the server top cover is in place and sits flat and snug on top of the server. Ensure that the interlock switch components have not been damaged, removed, or misaligned. For instructions for properly installing the top cover, see “Install the Server Top
Cover” on page 188.
Prevention: After removing the top cover, take care that it does not get bent. When servicing the server, take care that the internally mounted interlock switch component does not get damaged or misaligned. Never operate the server without the top cover installed.

Managing Server Hardware Faults Through the Oracle ILOM Fault Management Shell

The Oracle ILOM Fault Management Shell enables Oracle Services personnel to view and manage fault activity on a managed servers and other types of devices.
For more information about how to use the Oracle ILOM Fault Management Shell, see the Oracle ILOM User's Guide for System Monitoring and Diagnostics Firmware Release 3.2.x in
the Oracle Integrated Lights Out Manager (ILOM) 3.2 Documentation Library at: http://www.
oracle.com/goto/ilom/docs
Caution - Caution - The purpose of the Oracle ILOM Fault Management Shell is to help Oracle
Services personnel diagnose system problems. Customers should not launch this shell or run fault management commands in the shell unless requested to do so by Oracle Services.
Troubleshooting and Diagnostics 37
Page 38

Troubleshooting With Diagnostic Tools

Troubleshooting With Diagnostic Tools
The server and its accompanying software and firmware contain diagnostic tools and features that can help you isolate component problems, monitor the status of a functioning system, and exercise one or more subsystem to disclose more subtle or intermittent hardware-related problems.
Each diagnostic tool has its own specific strength and application. Review the tools listed in this section and determine which tool might be best to use for your situation. Once you have determined the tool to use, you can access it locally, while at the server, or remotely.
■
“Diagnostic Tools” on page 38
■
“Diagnostic Tool Documentation” on page 39

Diagnostic Tools

The selection of diagnostic tools available for your server range in complexity from a comprehensive validation test suite (Oracle VTS) to a chronological event log (Oracle ILOM System Log). The selection of diagnostic tools also include standalone software packages, firmware-based tests, and hardware-based LED indicators.
The following table summarizes the diagnostic tools that you can use when troubleshooting or monitoring your server.
Diagnostic Tool
Oracle ILOM
Preboot Menu
Hardware­based LED indicators
Power-On Self-Test (POST)
Type What It Does Accessibility Remote Capability
SP firmware
SP firmware
Hardware and SP firmware
Host firmware
Monitors environmental condition and component functionality sensors, generates alerts, performs fault isolation, and provides remote access.
Enables you to restore some Oracle ILOM default settings when Oracle ILOM is not accessible.
Indicates status of overall system and particular components.
Tests core components of system: CPUs, memory, and motherboard I/O bridge integrated circuits.
Can function on either standby power mode or full power mode and is not OS dependent.
Can function on standby power and when operating system is not running.
Available when system power is available.
Runs on startup. Available when the operating system is not running.
Designed for remote and local access.
Local, but remote serial access is possible if the SP serial port is connected to a network­accessible terminal server.
Local, but sensor and indicators are accessible from Oracle ILOM web interface or command-line interface (CLI).
Local, but can be accessed through Oracle ILOM Remote System Console Plus.
38 Oracle Server X6-2 Service Manual • June 2016
Page 39
Troubleshooting With Diagnostic Tools
Diagnostic Tool
U-Boot SP
UEFI DiagnosticsSPfirmware
Oracle Solaris commands
Oracle Linux commands
Oracle VTS
Type What It Does Accessibility Remote Capability
firmware
Operating system software
Operating system software
Diagnostic tool standalone software
Initializes and test aspects of the service processor (SP) prior to booting the Oracle ILOM SP and operating system. Tests SP memory, SP, network devices and I/O devices.
The UEFI diagnostics can test and detect problems on all processor, memory, disk drives, and network ports. It is used on new Oracle systems, such as the Oracle Server X6.2.
Displays various kinds of system information.
Displays various kinds of system information.
Exercises and stresses the system, running tests in parallel.
Can function on standby power and when operating system is not running.
You can use either the Oracle ILOM web interface or the command-line interface (CLI) to run UEFI diagnostics.
Requires operating system.
Requires operating system.
Requires Oracle Solaris operating system. Install Oracle VTS software separately.
Local, but remote serial access is possible if the SP serial port is connected to a network­accessible terminal server.
Remote access through Oracle ILOM Remote System Console Plus.
Local, and over network.
Local, and over network.
View and control over network.

Diagnostic Tool Documentation

The following table identifies where you can find more information about diagnostic tools.
Diagnostic Tool
Oracle ILOM Oracle Integrated Lights Out Manager 3.2
Preboot Menu
System indicators and sensors
U-Boot or UEFI Diagnostics
Oracle VTS Oracle VTS software and documentation
Documentation Location
Documentation Library
Oracle x86 Servers Diagnostics Guide
This document “Troubleshooting Using the Server Front and Rear
Oracle x86 Servers Diagnostics Guide
http://www.oracle.com/goto/ilom/docs
http://www.oracle.com/goto/x86admindiag/ docs
Panel Status Indicators” on page 28
http://www.oracle.com/goto/x86admindiag/ docs
http://docs.oracle.com/cd/E19719-01/index. html
Troubleshooting and Diagnostics 39
Page 40

Attaching Devices to the Server

Attaching Devices to the Server
Th following sections contain procedural information for attaching devices to the server so you can access diagnostic tools when troubleshooting and servicing the server:
■
“Attach Devices to the Rear of the Server” on page 40
■
“Rear Panel Connector Locations” on page 40
■
“Configuring Serial Port Sharing” on page 42
■
“Server Operating System Names for the NVMe Storage Drives” on page 43
■
“Ethernet Port Boot Order and Device Naming” on page 43
■
“Rear Panel Pinhole Switches” on page 44

Attach Devices to the Rear of the Server

This procedure explains how to connect devices to the server (remotely and locally), so that you can interact with the service processor (SP) and the server console.
1.
Connect an Ethernet cable to the Gigabit Ethernet (NET) connectors as needed for OS support.
See “Rear Panel Connector Locations” on page 40.
2.
To connect to the service processor's Oracle ILOM over the network, connect an Ethernet cable to the Ethernet port labeled NET MGT.
See “Rear Panel Connector Locations” on page 40.
3.
To access the Oracle ILOM command-line interface (CLI) locally using the management port, connect a serial null modem cable to the RJ-45 serial port labeled SER MGT.
See “Rear Panel Connector Locations” on page 40.
4.
To interact with the system console locally, connect a mouse and keyboard to the server front panel USB connectors and a monitor to the server front panel DB-15 video connector.
See “Rear Panel Connector Locations” on page 40.

Rear Panel Connector Locations

The following illustration shows and describes the locations of the back panel connectors. Use this information to set up the server, so that you can access diagnostic tools and manage the server during service.
40 Oracle Server X6-2 Service Manual • June 2016
Page 41
No. Cable Port or Expansion Slot Description
1 Power supply 0 input power
The server has two power supply connectors, one for each power supply.
Attach Devices to the Rear of the Server
Power supply 1 input power
Do not attach power cables to the power supplies until you have finished connecting the data cables to the server. The server goes into standby power mode, and the Oracle ILOM service processor initializes when the AC power cables are connected to the power source. System messages might be lost after 60 seconds if the server is not connected to a terminal, PC, or workstation.
Note - Oracle ILOM will signal a fault on any installed power supply that
is not connected to an AC power source, since it might indicate a loss of redundancy.
2 Network management port (NET MGT) The service processor NET MGT port is the optional connection to the
Oracle ILOM service processor. The NET MGT port is configured by default to use Dynamic Host Configuration Protocol (DHCP). The service processor NET MGT port uses an RJ-45 cable for a 10/100/1000BASE-T connection.
3 Serial management port (SER MGT) The service processor SER MGT port uses an RJ-45 cable and is the default
connection to the Oracle ILOM service processor. This port supports local connections to the server and only recognizes Oracle ILOM command-line interface (CLI) commands. Typically you connect a terminal or terminal emulator to this port.
Note - This port does not support network connections.
4 Ethernet ports (NET 3, NET 2, NET 1, NET0) The four 10-Gigabit Ethernet ports enable you to connect the system to the
network.
Note - Ethernet ports NET2 and NET3 are nonfunctional in single-processor
systems.
5, 6 USB ports (USB 0, USB 1) The two USB ports support hot-plugging. You can connect and disconnect
USB cables and peripheral devices while the server is running without affecting system operations.
7 Video port (VGA) Use a 15-pin video cable to connect a VGA video device to the server.
Optionally, you can connect to the VGA port when installing the operating system.
Troubleshooting and Diagnostics 41
Page 42
Assign Serial Port Output Using the CLI

Configuring Serial Port Sharing

By default, the SP console (SER MGT) port sends serial port output from the server. Using Oracle ILOM, you can specify that the host console (COM1) be assigned as owner of the server serial port output. This feature is useful for Windows kernel debugging, as it enables you to view non-ASCII character traffic from the host console.
You should set up the network on the SP before attempting to change the serial port owner to the host server. If the network is not set up, and you switch the serial port owner to the host server, you will be unable to connect using the CLI or web interface to change the serial port owner back to the SP. To return the serial port owner setting to the SP, you need to restore access to the serial port on the server. For more details about restoring access to the server port on your server, see the Oracle Integrated Lights Out Manager (ILOM) 3.2 Documentation
Library at http://www.oracle.com/goto/ilom/docs.
You can assign serial port output using either the Oracle ILOM web interface or the command­line interface (CLI). For instructions, see the following procedures:
■
“Assign Serial Port Output Using the CLI” on page 42
■
“Assign Serial Port Output Using the Web Interface” on page 42
Assign Serial Port Output Using the CLI
1.
Open an SSH session and at the command line log in to the SP Oracle ILOM CLI.
Log in as a user with root or administrator privileges. For example:
ssh root@ipadress
Where ipadress is the IP address of the server module SP. For more information, refer to “Accessing Oracle ILOM” in the Oracle X5 Series Servers
Administration Guide.
The Oracle ILOM CLI prompt (->) appears.
2.
To set the serial port owner, type:
-> set /SP/serial/portsharing /owner=host
Note - The serial port sharing value by default is owner=SP.
3.
Connect a serial host to the server.
Assign Serial Port Output Using the Web Interface
1.
Log in to the SP Oracle ILOM web interface.
42 Oracle Server X6-2 Service Manual • June 2016
Page 43
Assign Serial Port Output Using the Web Interface
To log in, open a web browser and direct it using the IP address of the server SP. Log in as root or a user with administrator privileges. Refer to “Accessing Oracle ILOM” in the Oracle X6
Series Servers Administration Guide.
The Summary Information screen appears.
2.
In the Oracle ILOM web interface, select ILOM Administration --> Connectivity from the navigation menu on the left side of the screen.
3.
Select the Serial Port tab.
The Serial Port Settings page appears.
Note - The serial port sharing setting by default is Service Processor.
4.
In the Serial Port Settings page, select Host Server as the serial port owner.
5.
Click Save for the changes to take effect.
6.
Connect a serial host to the server.

Server Operating System Names for the NVMe Storage Drives

If the optional NVMe storage drives are installed in the server's front panel, they are labeled NVMe0, NVMe1, NVMe2, and NVMe3. However, the server operating systems assign these storage drives different names. For the corresponding names assigned by the operating systems, see the following table.
TABLE 7
Storage Drive Labels Names Assigned by the Server Operating Systems
NVMe0 PCI-Express Slot 10
NVMe1 PCI-Express Slot 11
NVMe2 PCI-Express Slot 12
NVMe3 PCI-Express Slot 13
Operating System NVMe Storage Drive Nomenclature

Ethernet Port Boot Order and Device Naming

This section contains information about the boot order and device naming for the four 10-Gigabit Ethernet ports on the rear panel of the server (see “Rear Panel Connector
Locations” on page 40). From right to left, the ports are numbered NET 0 to NET 3.
Troubleshooting and Diagnostics 43
Page 44
Assign Serial Port Output Using the Web Interface
Ethernet Port Boot Order
The order in which the BIOS detects the Ethernet ports during server boot is listed below.
Note - You can change the boot priority using the Boot Device Priority screen available in the
Boot menu of the BIOS Setup Utility.
1. NET 0
2. NET 1
3. NET 2
4. NET 3
Ethernet Port Device Naming
Note - Naming used by the interfaces might vary from that listed below depending on which
devices are installed in the system.
The device naming for the Ethernet interfaces is reported differently by different interfaces and operating systems. The following table shows BIOS (physical) and operating system (logical) naming conventions used for each interface. These naming conventions might vary depending on conventions of your operating system and which devices are installed in the server.
Port BIOS Solaris Linux Windows (example default
Net 0 3A00 ixgbe 0 eth0 Ethernet
Net 1 3A01 ixgbe 1 eth1 Ethernet 2
Net 2 8200 ixgbe 2 eth2 Ethernet 3
Net 3 8201 ixgbe 3 eth3 Ethernet 4
Note - For Windows, port names like Ethernet, Ethernet 2, Ethernet 3, and Ethernet 4 are used
names, see note below)
by default. However, actual port naming is based on the order of enumeration, typically during operating system installation. Additionally, Windows allows you to rename the ports to meet application-specific needs.

Rear Panel Pinhole Switches

This section shows the location of the rear panel pinhole switches and describes the function of each switch.
44 Oracle Server X6-2 Service Manual • June 2016
Page 45

Getting Help

TABLE 8
Figure Legend Description
1 Host Warm Reset
2 NMI (non-maskable interrupt) (For use by authorized
3 SP Reset
Getting Help
The following sections describe how to get additional help to resolve server-related problems.
■
“Contacting Support” on page 45
■
“Locating the Chassis Serial Number” on page 46

Contacting Support

If the troubleshooting procedures in this chapter fail to solve your problem, use the following table to collect information that you might need to communicate to support personnel.
Figure Legend
Oracle Service personnel only.)
System Configuration Information Needed Your Information
Service contract number
System model
Operating environment
System serial number
Peripherals attached to the system
Email address and phone number for you and a secondary contact
Street address where the system is located
Superuser password
Troubleshooting and Diagnostics 45
Page 46
Getting Help
System Configuration Information Needed Your Information
Summary of the problem and the work being done when the problem occurred
Other Useful Information
IP address
Server name (system host name)
Network or internet domain name
Proxy server configuration
Related Information
■
“Locating the Chassis Serial Number” on page 46

Locating the Chassis Serial Number

You might need to have your server serial number when you ask for service on your system. Record this number for future use. Use one of the following resources or methods to locate your server serial number.
■
The serial number is located on the Radio-frequency identification (RFID) label on the left side of the front panel bezel, next to the general status indicators. For illustrations of the server front panel, see “About Controls and Connectors” on page 14.
■
The serial number is recorded on the system information label that is attached to the top of the system in the front, left corner.
■
The serial number is recorded on the yellow Customer Information Sheet (CIS) attached to your server packaging. This sheet includes the serial number.
■
Using Oracle ILOM:
■
From the command-line interface (CLI), type the command: show/SYS.
■
From the web interface, view the serial number on the System Information screen.
46 Oracle Server X6-2 Service Manual • June 2016
Page 47

Preparing for Service

These sections describe safety considerations and provide prerequisite procedures and information to replace components within the server.
Description Links
Understand the safety precautions, understand the safety symbols, and take ESD precautions prior to removing or installing parts in the server.
Understand the key identity properties auto-update feature.
Assemble the required tools. “Required Tools” on page 50
Before working with components within the server, power down the server and prepare for servicing.

Related Information

■
“Returning the Server to Operation” on page 187
“Safety Precautions” on page 47
“Safety Symbols” on page 48
“Electrostatic Discharge Safety” on page 48
“FRU Key Identity Properties (KIP) Automated Update” on page 49
“Preparing the Server for Component Replacement” on page 50

Safety Precautions

For your protection, observe the following safety precautions when setting up your equipment:
■
Follow all standard cautions, warnings, and instructions marked on the equipment and described in the online Oracle Server X6-2 Safety and Compliance Guide and in the printed Important Safety Information for Oracle's Hardware Systems.
■
Ensure that the voltage and frequency of your power source match the voltage and frequency inscribed on the equipment's electrical rating label.
■
Follow the electrostatic discharge safety practices as described in “Electrostatic Discharge
Safety” on page 48.
■
Disconnect both power supply cords before servicing components.
Preparing for Service 47
Page 48

Safety Symbols

Related Information

■
“Safety Symbols” on page 48
■
“Electrostatic Discharge Safety” on page 48
Safety Symbols
The following symbols might appear in this book. Note their meanings.
Caution - Risk of personal injury or equipment damage. To avoid personal injury or
equipment damage, follow the instructions.
Caution - Hot surface. Avoid contact. Surfaces are hot and might cause personal injury if
touched.
Caution - Hazardous voltages are present. To reduce the risk of electric shock and danger to
personal health, follow the instructions.

Related Information

■
“Safety Precautions” on page 47
■
“Electrostatic Discharge Safety” on page 48

Electrostatic Discharge Safety

Devices that are sensitive to electrostatic discharge (ESD), such as the motherboard, PCIe cards, drives, processors, and memory cards require special handling.
Caution - The boards and drives contain electronic components that are extremely sensitive to
static electricity. Ordinary amounts of static electricity from clothing or the work environment can destroy components. Do not touch the components along their connector edges.
Do the following when handling ESD-sensitive components:
■
Use an antistatic wrist strap.
Wear an antistatic wrist strap and use an antistatic mat when handling components such as drive assemblies, boards, or cards. When servicing or removing server components, attach an antistatic strap to your wrist and then to a metal area on the chassis. Then disconnect
48 Oracle Server X6-2 Service Manual • June 2016
Page 49

FRU Key Identity Properties (KIP) Automated Update

the power cords from the server. Following this practice equalizes the electrical potentials between you and the server.
Note - An antistatic wrist strap is not included in the Ship Kit for the server. However,
antistatic wrist straps are included with options and components.
■
Use an antistatic mat.
Place ESD-sensitive components such as the motherboard, memory DIMMs, and other printed circuit boards on an antistatic mat. The following items can be used as an antistatic mat:
■
An antistatic bag used to wrap an Oracle replacement part
■
An Oracle ESD mat (orderable item)
■
A disposable ESD mat (shipped with some replacement parts or optional system components)

Related Information

■
“Safety Precautions” on page 47
■
“Safety Symbols” on page 48
■
“Preparing the Server for Component Replacement” on page 50
■
“Returning the Server to Operation” on page 187
FRU Key Identity Properties (KIP) Automated Update
Oracle ILOM includes a key identity properties (KIP) auto-update feature that ensures product information that is used for service entitlement and warranty coverage is accurately maintained by the server at all times, including during hardware replacement activities.
The KIP includes the server product name, product part number (PPN), and product serial number (PSN). The KIP are stored in the FRUID (field-replaceable unit identifiers) container of the three server FRUs that are designated quorum members.
The quorum members include:
■
Disk backplane (DBP), designated as a primary quorum member.
■
Motherboard (MB), designated as a backup quorum member.
■
Power supply (PS), designated as a backup quorum member.
When a server FRU that contains the KIP is removed and a replacement component is installed, the KIP of the replacement component is programmed by Oracle ILOM to contain the same KIP as the other two components.
Preparing for Service 49
Page 50

Required Tools

Note - Only one of the quorum members can be replaced at a time. Automated updates can only
be completed when two of the three quorum members contain matching key identity properties.

Related Information

■
“Servicing Power Supplies (CRU)” on page 80
■
“Servicing the Disk Backplane (FRU)” on page 150
■
“Servicing the Motherboard (FRU)” on page 162
Required Tools
The server can be serviced with the following tools:
■
Antistatic wrist strap
■
Antistatic mat
■
No. 2 Phillips screwdriver

Related Information

■
“Preparing for Service” on page 47
■
“Servicing CRUs That Do Not Require Server Power-Off” on page 63
■
“Servicing CRUs That Require Server Power-Off” on page 85
■
“Servicing FRUs” on page 119

Preparing the Server for Component Replacement

Before you can remove and install components that are inside the server, you must perform the procedures in the following tasks:
Note - When replacing the storage drives or power supplies, not all of these procedures are
necessary. See the replacement tasks for those components for more information.
■
“Powering Down the Server” on page 51
■
“Disconnect Cables From the Server” on page 56
■
“Extend the Server to the Maintenance Position” on page 56
■
“Remove the Server From the Rack” on page 58
50 Oracle Server X6-2 Service Manual • June 2016
Page 51
Power Down Server Gracefully Using the Oracle ILOM CLI
■
“Take Antistatic Measures” on page 59
■
“Remove the Server Top Cover” on page 60

Related Information

■
“Returning the Server to Operation” on page 187

Powering Down the Server

To determine how you want to power off the server, view the options in the following table:
Description Link
Power down the server gracefully to save all data and to prevent data from being corrupted.
Performing a graceful shutdown ensures that the system is ready for restart.
If the server is not responding, or you must shut down the server quickly, perform an immediate shutdown.
■ “Power Down Server Gracefully Using the
Oracle ILOM CLI” on page 51
■ “Power Down Server Gracefully Using the
Oracle ILOM Web Interface” on page 52
■ “Power Down Server Gracefully Using the
Power Button” on page 53
■ “Use the Power Button for Immediate
Shutdown” on page 54
■ “Use the Oracle ILOM CLI for Immediate
Shutdown” on page 54
■ “Use the Oracle ILOM Web Interface for
Immediate Shutdown” on page 55
Related Information
■
“Power On the Server” on page 194
Power Down Server Gracefully Using the Oracle ILOM CLI
Performing a graceful shut down ensures that all of your data is saved and the system is ready for restart.
1.
Log in to the server as superuser or equivalent.
Depending on the nature of the problem, you might want to view the system status or the log files or run diagnostics before you shut down the system. For more information, see the Oracle
ILOM 3.2 Documentation Library at http://www.oracle.com/goto/ilom/docs.
2.
Notify affected users that the server will be powered down.
Preparing for Service 51
Page 52
Power Down Server Gracefully Using the Oracle ILOM Web Interface
3.
Save any open files, and quit all running applications.
Refer to your application documentation for specific information on these processes.
4.
Log in to the Oracle ILOM command-line interface (CLI) using an Administrator account.
For instructions, refer to “Logging In to or Out of Oracle ILOM” in Oracle Server X6-2
Installation Guide.
5.
At the Oracle ILOM prompt, shut down the operating system:
-> stop /System
If the system is running the Oracle Solaris OS, refer to the Oracle Solaris system administration documentation for additional information.
6.
Disconnect the power cords and data cables from the server.
See “Disconnect Cables From the Server” on page 56.
Related Information
■
“Power Down Server Gracefully Using the Oracle ILOM Web Interface” on page 52
■
“Power Down Server Gracefully Using the Power Button” on page 53
■
“Use the Power Button for Immediate Shutdown” on page 54
■
“Power On the Server” on page 194
Power Down Server Gracefully Using the Oracle ILOM Web Interface
1.
Log in to the server as superuser or equivalent.
Depending on the nature of the problem, you might want to view the system status or the log files or run diagnostics before you shut down the system. For more information, see the Oracle
ILOM 3.2 Documentation Library at http://www.oracle.com/goto/ilom/docs.
2.
Notify affected users that the server will be powered down.
3.
Save any open files and quit all running applications.
Refer to your application documentation for specific information about these processes.
4.
Log in to the Oracle ILOM web interface using an Administrator account.
For instructions, see the “Logging In to or Out of Oracle ILOM” in Oracle Server X6-2
Installation Guide.
52 Oracle Server X6-2 Service Manual • June 2016
Page 53
Power Down Server Gracefully Using the Power Button
The Oracle ILOM web interface System Information → Summary Information page appears.
5.
In the left pane, click Host Management → Power Control, and select Graceful Shutdown and Power Off from the Action list.
6.
Click Save and then click OK.
The host server performs an orderly power shutdown.
7.
Disconnect the power cords an data cables from the server.
See “Disconnect Cables From the Server” on page 56.
Related Information
■
“Power Down Server Gracefully Using the Oracle ILOM CLI” on page 51
■
“Power Down Server Gracefully Using the Power Button” on page 53
■
“Use the Power Button for Immediate Shutdown” on page 54
■
“Power On the Server” on page 194
Power Down Server Gracefully Using the Power Button
1.
Press and quickly release the Power button on the front panel.
This action causes ACPI-enabled operating systems to perform an orderly shutdown of the operating system. Servers not running ACPI-enabled operating systems shut down to standby power mode immediately.
When main power is off, the Power/OK LED on the front panel will begin flashing, indicating that the server is in standby power mode. See “Server System-Level Status
Indicators” on page 29.
2.
Disconnect the power cords and data cables from the server.
See “Disconnect Cables From the Server” on page 56.
Caution - When you press the Power button to enter standby power mode, power is still
directed to the service processor remote management subsystem and power supply fans. To completely power off the server, you must disconnect the power cords from the power supplies.
Related Information
■
Table 1
■
“Power Down Server Gracefully Using the Oracle ILOM CLI” on page 51
■
“Power Down Server Gracefully Using the Oracle ILOM Web Interface” on page 52
■
“Use the Power Button for Immediate Shutdown” on page 54
■
“Power On the Server” on page 194
Preparing for Service 53
Page 54
Use the Power Button for Immediate Shutdown
Use the Power Button for Immediate Shutdown
Caution - An immediate power down might corrupt system data, therefore, only use this
procedure to power down the server after attempting the graceful power down procedure.
1.
Press and hold the Power button for four seconds to force the main power off and to enter standby power mode.
When main power is off, the Power/OK LED on the front panel will begin flashing, indicating that the server is in standby power mode. See “Server System-Level Status
Indicators” on page 29.
2.
Disconnect the power and data cables from the server.
See “Disconnect Cables From the Server” on page 56.
Caution - When you press the Power button to enter standby power mode, power is still
directed to the service processor remote management subsystem and power supply fans. To completely power off the server, you must disconnect the power cords from the power supplies.
Related Information
■
Table 1
■
“Power Down Server Gracefully Using the Oracle ILOM CLI” on page 51
■
“Power Down Server Gracefully Using the Oracle ILOM Web Interface” on page 52
■
“Power Down Server Gracefully Using the Power Button” on page 53
■
“Use the Oracle ILOM CLI for Immediate Shutdown” on page 54
■
“Use the Oracle ILOM Web Interface for Immediate Shutdown” on page 55
■
“Power On the Server” on page 194
Use the Oracle ILOM CLI for Immediate Shutdown
Caution - An immediate power down might corrupt system data, therefore, only use this
procedure to power down the server after attempting the graceful power down procedure.
1.
Log in to the Oracle ILOM command-line interface (CLI) using an Administrator account.
Oracle ILOM displays the default command prompt (->), indicating that you have successfully logged in to Oracle ILOM.
2.
From the CLI prompt, type the following command:
-> stop -f /System
54 Oracle Server X6-2 Service Manual • June 2016
Page 55
Use the Oracle ILOM Web Interface for Immediate Shutdown
The server powers down immediately.
3.
Disconnect the power and data cables from the server.
See “Disconnect Cables From the Server” on page 56.
Caution - When the server is powered down, it enters standby power mode In standby power
mode, power is still directed to the service processor remote management subsystem and power supply fans. To completely power off the server, you must disconnect the power cords from the power supplies.
Related Information
■
“Use the Power Button for Immediate Shutdown” on page 54
■
“Use the Oracle ILOM Web Interface for Immediate Shutdown” on page 55
Use the Oracle ILOM Web Interface for Immediate Shutdown
Caution - An immediate power down might corrupt system data, so only use this procedure to
power down the server after attempting the graceful power down procedure.
1.
Log in to the Oracle ILOM web interface using an Administrator account.
The Oracle ILOM web interface System Information page appears.
2.
In the left pane, click Host Management → Power Control, and select Immediate Power Off in the Select Action list.
3.
Click Save, then click OK.
The server powers down immediately.
4.
Disconnect the power and data cables from the server.
See “Disconnect Cables From the Server” on page 56.
Caution - When the server is powered down, it enters standby power mode In standby power
mode, power is still directed to the service processor remote management subsystem and power supply fans. To completely power off the server, you must disconnect the power cords from the power supplies.
Related Information
■
“Use the Power Button for Immediate Shutdown” on page 54
■
“Power Down Server Gracefully Using the Oracle ILOM CLI” on page 51
■
“Power On the Server” on page 194
Preparing for Service 55
Page 56

Disconnect Cables From the Server

Disconnect Cables From the Server
Caution - The system supplies standby power to the circuit boards when the power cords are
connected even when the system is powered off.
1.
Label all cables connected to the server.
2.
Power down the server.
See “Powering Down the Server” on page 51.
3.
Disconnect the power cords from the rear of the server.
4.
Disconnect all data cables from the rear of the server.
Related Information
■
“Server Back Panel View” on page 16
■
“Powering Down the Server” on page 51
■
“Remove the Server From the Rack” on page 58
■
“Reconnect Data Cables and Power Cords” on page 193

Extend the Server to the Maintenance Position

The following components can be serviced with the server in the maintenance position:
■
Storage drives
■
Fan modules
■
Power supplies
■
DVD drive module
■
PCIe risers
■
PCIe cards and cables
■
SAS cables
■
NVMe cables
■
DIMMs
■
Internal USB flash drives
■
Motherboard battery
■
Processors
■
Disk backplane
■
FIM (front indicator module)
■
Motherboard
56 Oracle Server X6-2 Service Manual • June 2016
Page 57
Extend the Server to the Maintenance Position
If the server is installed in a rack with extendable slide-rails, use this procedure to extend the server to the maintenance position.
1.
To prevent the rack from tipping forward when the server is extended, extend all rack anti-tilt devices.
Caution - To reduce the risk of personal injury, stabilize the expansion rack cabinet and extend
all anti-tilt devices before extending the server from the rack.
For instructions for stabilizing the rack, see the “Stabilize the Rack for Installation” in Oracle
Server X6-2 Installation Guide.
2.
Verify that no cables will be damaged or will interfere when the server is extended.
Although the cable management arm (CMA) that is supplied with the server is hinged to accommodate extending the server, you should ensure that all cables and cords are capable of extending.
3.
From the front of the server, open and hold the left and right release latch covers in the open position (see Figure 4 ).
When in an open position, the release latch covers engage the slide-rail release latches [frame 1].
FIGURE  4
Extending the Server to the Maintenance Position
Preparing for Service 57
Page 58

Remove the Server From the Rack

4.
While the release latch covers are in the open position, slowly pull the server forward until the slide-rails latch into a locked position [2].
The server is now in the extended maintenance position. .
Related Information
■
“Disconnect Cables From the Server” on page 56
■
“Remove the Server From the Rack” on page 58
■
“Reinstall the Server Into the Rack” on page 190
Remove the Server From the Rack
Caution - The server weighs approximately 18.1 kg (40.0 lbs). Two people are required to
dismount and carry the chassis.
1.
Disconnect all the cables and power cords from the server.
2.
Remove the cable management arm (CMA).
For instructions for removing the CMA, see the “Remove the Cable Management Arm” in
Oracle Server X6-2 Installation Guide.
3.
Extend the server to the maintenance position.
For instructions, see “Extend the Server to the Maintenance Position” on page 56.
4.
From the front of the server, pull the green slide-rail release tabs toward the front of the server, and pull the server out of the rack until it is free of the rack rails.
A slide-rail release tab is located on each slide-rail (see Figure 4 [frame 2]).
Note - To pull the green slide-rail release tab, place your finger in the center of the tab, not on
the end, and apply pressure as you pull the tab toward the front of the server.
5.
Set the server on a sturdy work surface.
Related Information
■
“Disconnect Cables From the Server” on page 56
■
“Take Antistatic Measures” on page 59
■
“Extend the Server to the Maintenance Position” on page 56
■
“Reinstall the Server Into the Rack” on page 190
58 Oracle Server X6-2 Service Manual • June 2016
Page 59

Take Antistatic Measures

Take Antistatic Measures
1.
Prepare an antistatic surface on which to set parts during removal and installation.
Place electrostatic discharge (ESD)-sensitive components such as the printed circuit boards on an antistatic mat. The following items can be used as an antistatic mat:
■
An antistatic bag used to wrap a replacement part
■
An Oracle ESD mat (orderable item)
■
A disposable ESD mat (shipped with some replacement parts or optional system components)
2.
Attach an antistatic wrist strap.
When servicing or removing server components, attach an antistatic strap to your wrist and then to a metal area on the chassis. Then disconnect the power cords from the server.
Note - An antistatic wrist strap is not included in the Ship Kit for the server. However, antistatic
wrist straps are still included with options and components.
Related Information
■
“Remove Antistatic Measures” on page 190

Open the Server Fan Door

Servicing the server fan modules and other components located in the front of the server, such as the front indication module, the DVD drive, and disk backplanes, requires that the fan door be opened. It is also easier to remove the server top cover if you open the fan door first.
To open the server fan door, slide the fan door latches to the front of the server and swing the door up to the open position.
Preparing for Service 59
Page 60

Remove the Server Top Cover

FIGURE  5
Opening the Server Fan Door
Related Information
■
“Take Antistatic Measures” on page 59
■
“Remove the Server Top Cover” on page 60
Remove the Server Top Cover
Servicing most of the server components requires that the top cover be removed.
Caution - If the top cover is removed without first removing the AC power cords, the server
host shuts down immediately and an event is logged to indicate that the chassis top cover has been removed.
1.
Ensure that the AC power cords are disconnected from the server power supplies.
2.
Open the server fan door.
See “Open the Server Fan Door” on page 59.
3.
To open the server top cover, press and hold down the top cover release button and use the recessed area to slide the top cover toward the rear of the server about 0.5 inches (12.7 mm) [1].
60 Oracle Server X6-2 Service Manual • June 2016
Page 61
Remove the Server Top Cover
FIGURE  6
4.
Lift the cover off the chassis and set it aside [2].
Removing the Server Top Cover
Related Information
■
“Take Antistatic Measures” on page 59
■
“Install the Server Top Cover” on page 188
Preparing for Service 61
Page 62
62 Oracle Server X6-2 Service Manual • June 2016
Page 63

Servicing CRUs That Do Not Require Server Power-Off

These sections describe how to service customer-replaceable units (CRUs) that do not require you to power off the server. For more information about CRUs, see “Illustrated Parts
Breakdown” on page 17 and “Customer-Replaceable Units” on page 19.
Description Links
Service the storage drives. “Servicing Storage Drives (CRU)” on page 63
Service the fan modules. “Servicing Fan Modules (CRU)” on page 75
Service the power supplies. “Servicing Power Supplies (CRU)” on page 80

Related Information

■
“Servicing CRUs That Require Server Power-Off” on page 85
■
“Servicing FRUs” on page 119

Servicing Storage Drives (CRU)

These sections describe how to remove and install storage drives:
■
“Storage Drives Hot-Plug Conditions” on page 64
■
“Storage Drive Failure and RAID” on page 64
■
“Storage Drive Locations and Numbering” on page 64
■
“Storage Drive Status Indicators” on page 65
■
“Removing and Replacing a HDD or SSD Storage Drive” on page 66
■
“Removing and Replacing an NVMe Storage Drive Using Oracle Solaris” on page 69
■
“Removing and Replacing an NVMe Storage Drive Using Oracle Linux” on page 72

Related Information

■
“Removing and Installing Server Filler Panels ” on page 187
Servicing CRUs That Do Not Require Server Power-Off 63
Page 64
Servicing Storage Drives (CRU)

Storage Drives Hot-Plug Conditions

The hard-disk drives (HDDs), solid-state drives (SSDs), and NVMe SSD drives installed in the server are hot-pluggable, but this capability depends on how the drives are configured. To hot­plug a drive you must be able to take the drive offline before you can remove it. When you take the drive offline, you prevent any application from accessing the drive and remove the logical software links to the drive.
The following conditions inhibit the ability to perform hot-plugging of a drive:
■
The drive provides the operating system, and the operating system is not mirrored on another drive.
■
The drive cannot be logically isolated from the online operations of the server.
If either of the above drive restrictions is true, then you must shut down the system before you replace the drive. See “Powering Down the Server” on page 51.
Note - Replacing a storage drive does not require removing the server from a rack.

Storage Drive Failure and RAID

A single storage drive failure does not cause a data failure if the storage drives are configured as a mirrored RAID 1 volume (optional). The storage drive can be removed, and when a new storage drive is inserted, the contents are automatically rebuilt from the rest of the array with no need to reconfigure the RAID parameters. If the replaced storage drive was configured as a hot­spare, the new storage drive is automatically configured as a new hot-spare.
For instructions for configuring RAID on the server, refer to “Configuring Storage Drives for
Operating System Installation” in Oracle Server X6-2 Installation Guide.

Storage Drive Locations and Numbering

The following illustration and table show the locations of the HDD, SSD, and optional NVMe SSD drives.
FIGURE  7
Server Storage Drives
64 Oracle Server X6-2 Service Manual • June 2016
Page 65
Servicing Storage Drives (CRU)
TABLE 9
Drive 1 (HDD/SSD) Drive 3 (HDD/SSD/
Drive 0 (HDD/SSD) Drive 2 (HDD/SSD/
Server Storage Drive Locations and Numbering
NVMe1)
NVMe0)
Drive 5 (HDD/SSD/ NVMe3)
Drive 4 (HDD/SSD/ NVMe2)
DVD
Drive 6 (HDD/SSD) Drive 7 (HDD/SSD)
Note - If the optional Oracle PCIe NVMe switch card is installed in the server, the associated
NVMe storage drives are installed in drive locations 2, 3, 4, and 5, as shown in the above table.
Note - The NVMe storage drives are labeled NVMe0, NVMe1, NVMe2, and NVMe3;
however, operating systems installed on the server assign different names to the NVMe storage drives. For the corresponding names assigned to the NVMe storage drives by server operating systems, see “Server Operating System Names for the NVMe Storage Drives” on page 43.

Storage Drive Status Indicators

The following figure and table describe the storage drive status indicators (LEDs).
FIGURE  8
Storage Drive Status Indicators
TABLE 10
Legend LED Color State and Meaning
1 OK to Remove Blue ■ STEADY ON – The storage drive can be removed safely during a hot-plug
2 Service Required Amber ■ OFF – The storage drive is operating normally.
3 OK/Activity Green ■ OFF – Power is off or the installed drive is not recognized by the system.
Storage Drive Status Indicators
operation.
■ OFF – The storage drive has not been prepared for removal.
■ STEADY ON – The system has detected a fault with the storage drive.
■ STEADY ON – The drive is engaged and is receiving power.
■ STEADY BLINK – There is disk activity. The LED blinks on and off to show activity.
Servicing CRUs That Do Not Require Server Power-Off 65
Page 66
Remove a HDD or SSD Storage Drive

Removing and Replacing a HDD or SSD Storage Drive

This section describes how to remove and replace HDD and SSD drives.
Remove a HDD or SSD Storage Drive
1.
Prepare the system for the drive removal.
See “Take Antistatic Measures” on page 59.
2.
Identify the location of the drive that you want to remove.
For storage drive locations, see “Storage Drive Locations and Numbering” on page 64.
3.
If you are removing a HDD or SSD storage drive, type the operating system commands required to stop the server from using the drive; otherwise, proceed to the next step.
Note - The blue OK to Remove LED on the storage drive might not light, as support for this
varies depending on the operating system in use.
The exact commands required depend on the configuration of your drives. Unmount file systems or issue RAID commands as needed.
4.
On the drive you plan to remove, push the latch release button to open the drive latch.
66 Oracle Server X6-2 Service Manual • June 2016
Page 67
Remove a HDD or SSD Storage Drive
FIGURE  9
Caution - The latch is not an ejector. Do not open the latch too far to the right. Doing so can
Locating the Hard Disk Drive Release Button and Latch
damage the latch.
5.
Grasp the latch and pull the drive out of the drive slot.
6.
Consider your next steps:
■
If you are replacing the drive, continue to “Install a HDD or SSD Storage
Drive” on page 68.
■
If you are not replacing the drive, install a filler panel in the empty drive slot to maintain proper airflow and perform administrative tasks to configure the server to operate without the drive.
For information on how to install a storage drive filler panel, see “Removing and Installing
Server Filler Panels ” on page 187.
Related Information
■
“Storage Drive Status Indicators” on page 65
■
“Storage Drives Hot-Plug Conditions” on page 64
■
“Storage Drive Failure and RAID” on page 64
■
“Install a HDD or SSD Storage Drive” on page 68
Servicing CRUs That Do Not Require Server Power-Off 67
Page 68
Install a HDD or SSD Storage Drive
Install a HDD or SSD Storage Drive
1.
Remove the replacement drive from its packaging and place the drive on an antistatic mat.
2.
If necessary, remove the drive filler panel.
3.
Align the replacement drive with the drive slot.
The drive is physically addressed according to the slot in which it is installed. It is important to install a replacement drive in the same slot as the drive that was removed.
4.
Slide the drive into the slot until the drive is fully seated [1].
FIGURE  10
5.
Close the drive latch to lock the drive in place [2].
6.
Perform administrative tasks to reconfigure the drive.
Installing a Storage Drive
The procedures that you perform at this point depend on how your data is configured. You might need to partition the drive, create file systems, load data from backups, or have the drive updated from a RAID configuration.
Related Information
■
“Storage Drive Status Indicators” on page 65
■
“Storage Drives Hot-Plug Conditions” on page 64
■
“Storage Drive Failure and RAID” on page 64
■
“Remove a HDD or SSD Storage Drive” on page 66
68 Oracle Server X6-2 Service Manual • June 2016
Page 69
Unmount an NVMe Storage Drive

Removing and Replacing an NVMe Storage Drive Using Oracle Solaris

Note - NVMe storage drives are supported only on servers that are running the Oracle Solaris or
Oracle Linux operating system. Servers that are running Oracle VM, Windows Server, or Red Hat Enterprise Linux do not support NVMe drives.
The following sections describe how to remove and replace an NVMe storage drive on a server that is running the Oracle Solaris operating system.
■
“Unmount an NVMe Storage Drive” on page 69
■
“Remove an NVMe Storage Drive From the Server” on page 70
■
“Verify Removal of an NVMe Storage Drive” on page 70
■
“Install an NVMe Storage Drive in the Server” on page 70
■
“Power On an NVMe Storage Drive and Attach a Device Driver” on page 71
Unmount an NVMe Storage Drive
1.
Log in to Oracle Solaris that is running on the server.
2.
To find the NVMe drive slot number, type:
# hotplug list –lc
This command produces output similar to the following for each of the NVMe drives installed in the server:
# hotplug list –lc Connection State Description Path
-----------------------------------------------------------------­pcie13 ENABLED PCIe-Native /pci@7a,0/pci8086,2f08@3/pci111d,80b5@4
3.
To prepare the NVMe drive for removal, that is, detach the device driver and power off the drive slot, type the following commands:
# hotplug poweroff pcie13
# hotplug list –lc
The following output appears for the NVMe drive that has been unmounted:
# hotplug list –lc Connection State Description Path
-----------------------------------------------------------------­pcie13 PRESENT PCIe-Native /pci@7a,0/pci8086,2f08@3/pci111d,80b5@4
Servicing CRUs That Do Not Require Server Power-Off 69
Page 70
Remove an NVMe Storage Drive From the Server
4.
Verify that the OK to Remove indicator (LED) on the NVMe storage drive is lit.
Remove an NVMe Storage Drive From the Server
Perform this procedure to physically remove an NVMe storage drive from the server.
1.
Identify the physical location of the NVMe drive that you want to remove.
For storage drive locations, see “Storage Drive Locations and Numbering” on page 64.
2.
Verify that the OK to Remove indicator (LED) on the NVMe storage drive is lit.
3.
On the drive you plan to remove, push the latch release button to open the drive latch.
See Figure 9.
4.
Grasp the latch and pull the drive out of the drive slot.
5.
Consider your next steps:
■
If you are replacing the drive, continue to “Install an NVMe Storage Drive in the
Server” on page 70.
■
If you are not replacing the drive, install a filler panel in the empty drive slot to maintain proper airflow and perform administrative tasks to configure the server to operate without the drive.
For information on how to install a storage drive filler panel, see “Removing and Installing
Server Filler Panels ” on page 187.
Verify Removal of an NVMe Storage Drive
To verify that the NVMe drive has been removed, type:
# hotplug list –lc
The following output appears for the NVMe drive that you removed:
# hotplug list –lc Connection State Description Path
-----------------------------------------------------------------­pcie13 EMPTY PCIe-Native /pci@7a,0/pci8086,2f08@3/pci111d,80b5@4
Install an NVMe Storage Drive in the Server
Perform this procedure to physically install an NVMe storage drive into the server.
70 Oracle Server X6-2 Service Manual • June 2016
Page 71
Power On an NVMe Storage Drive and Attach a Device Driver
Note - After you physically remove an NVMe storage drive from the server, wait at least 10
seconds before installing a replacement drive.
1.
Remove the replacement drive from its packaging and place the drive on an antistatic mat.
2.
If necessary, remove the drive filler panel.
3.
Align the replacement drive with the drive slot.
The drive is physically addressed according to the slot in which it is installed. It is important to install a replacement drive in the same slot as the drive that was removed.
4.
Slide the drive into the slot until the drive is fully seated.
See frame [1] in Figure 10.
5.
Close the drive latch to lock the drive in place.
See frame [2] in Figure 10.
Power On an NVMe Storage Drive and Attach a Device Driver
1.
Type: # hotplug enable pcie13
2.
Type: # hotplug list –lc
The following status is displayed for the NVMe storage drive.
# hotplug list –lc Connection State Description Path
-----------------------------------------------------------------­pcie13 Enabled PCIe-Native /pci@7a,0/pci8086,2f08@3/pci111d,80b5@4
3.
To check the NVMe drive health, firmware level, temperature, get error log, SMART data, security erase, low level format, etc., type:
# nvmeadm list
Related Information
■
“Storage Drive Status Indicators” on page 65
■
“Storage Drives Hot-Plug Conditions” on page 64
■
“Storage Drive Failure and RAID” on page 64
■
“Storage Drive Locations and Numbering” on page 64
Servicing CRUs That Do Not Require Server Power-Off 71
Page 72
Unmount an NVMe Storage Drive

Removing and Replacing an NVMe Storage Drive Using Oracle Linux

Note - NVMe storage drives are supported only on servers that are running the Oracle Solaris or
Oracle Linux operating system. Servers that are running Oracle VM, Windows Server, or Red Hat Enterprise Linux do not support NVMe drives.
The following sections describe how to remove and replace an NVMe storage drive on a server that is running the Oracle Linux operating system.
■
“Unmount an NVMe Storage Drive ” on page 72
■
“Remove an NVMe Storage Drive” on page 73
■
“Verify Removal of an NVMe Storage Drive” on page 74
■
“Install an NVMe Storage Drive In the Server” on page 74
■
“Power On an NVMe Storage Drive and Attach a Device Driver” on page 75
■
“Verify Operation of an NVMe Storage Drive” on page 75
Unmount an NVMe Storage Drive
1.
Log in to Oracle Linux that is running on the server.
2.
Remove the NVMe storage device path.
a.
To find the PCIe addresses (Bus Device Function), type:
# find /sys/devices |egrep ‘nvme[0-9][0-9]?$’
This commands returns output similar to the following:
/sys/devices/pci0000:80/0000:80:03.0/0000:b0:00.0/0000:b1:04.0/0000:b2:00.0/misc/nvme0 /sys/devices/pci0000:80/0000:80:03.0/0000:b0:00.0/0000:b1:05.0/0000:b4:00.0/misc/nvme1
In the above listing the PCIe addresses are highlighted in bold text.
b.
To obtain the slot number (APIC ID) for the bus address, type the following command to list all of the PCIe slot numbers and the corresponding bus addresses:
# egrep –H ‘.*’ /sys/bus/pci/slots/*/address
This commands returns output similar to the following:
/sys/bus/pci/slots/10/address:0000:b8:00 /sys/bus/pci/slots/11/address:0000:b6:00 /sys/bus/pci/slots/12/address:0000:b2:00 (instance nvme0, pcie slot 12, drive label nvme2)
72 Oracle Server X6-2 Service Manual • June 2016
Page 73
Remove an NVMe Storage Drive
/sys/bus/pci/slots/13/address:0000:b4:00 (instance nvme1, pcie slot 13, drive label nvme3)
In the above output, the bus addresses for the corresponding NVMe instances are highlighted in bold text.
Note - In the above output, notice that the instance names for the NVMe drives does not
correspond to the NVMe drive labels on the front of the server, that is, pci/slots/12/address: 0000:b2:00 corresponds to instance nvme0; however, on the front of the server, this drive is labeled NVMe2. For a table that shows the relationship between the pci/slot# and the NVMe storage drive label on the front of the server, see “Server Operating System Names for the
NVMe Storage Drives” on page 43 .
c.
Disconnect all users of the NVMe drive and back up the NVMe drive data as needed.
i.
Use the umount command to unmount any file systems that are mounted on the device.
ii.
Remove the device from any multiple device (md) and Logical Volume Manager (LVM) volume using it.
If the device is a member of an LVM Volume group, then it may be necessary to move data off the device using the pvmove command, then use the vgreduce command to remove the physical volume, and (optionally) pvremove to remove the
LVM meta data from the disk.
iii.
If the device uses multipathing, run multipath -l and note all the paths to the device. Then, remove the multipathed device using the multipath
-f device command.
iv.
Run the blockdev --flushbufs device command to flush any outstanding I/O to all paths to the device.
3.
To prepare the NVMe drive for removal, that is, detach the NVMe device driver and power off the NVMe drive slot, type:
# echo 0 > /sys/bus/pci/slots/$slot/power
Where $slot is the slot number obtained in step Step 2b above.
4.
Verify that the OK to Remove indicator (LED) on the NVMe drive is lit.
Remove an NVMe Storage Drive
Perform this procedure to physically remove an NVMe storage drive from the server.
Servicing CRUs That Do Not Require Server Power-Off 73
Page 74
Verify Removal of an NVMe Storage Drive
1.
Identify the location of the NVMe drive that you want to remove.
For storage drive locations, see “Storage Drive Locations and Numbering” on page 64.
2.
Verify that the OK to Remove indicator (LED) on the NVMe drive is lit.
3.
On the NVMe drive you plan to remove, push the latch release button to open the drive latch.
See Figure 9.
4.
Grasp the latch and pull the drive out of the drive slot.
5.
Consider your next steps:
■
If you are replacing the drive, continue to “Verify Removal of an NVMe Storage
Drive” on page 74.
■
If you are not replacing the drive, install a filler panel in the empty drive slot to maintain proper airflow and perform administrative tasks to configure the server to operate without the drive.
For information on how to install a storage drive filler panel, see “Removing and Installing
Server Filler Panels ” on page 187.
Verify Removal of an NVMe Storage Drive
1.
To check the NVMe drive enumeration to verify that the NVMe drive has been removed, type:
# lspci -nnd :0953
2.
View the command output and verify that the entry for the slot number that was disabled no longer appears.
This commands returns output similar to the following:
b2:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:0953] (rev 01) b6:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:0953] (rev 01) b8:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:0953] (rev 01)
Install an NVMe Storage Drive In the Server
Perform this procedure to physically install an NVMe storage drive into the server.
After you physically remove an NVMe drive from the server, wait at least 10 seconds before installing a replacement drive.
1.
Remove the replacement drive from its packaging and place the drive on an antistatic mat.
74 Oracle Server X6-2 Service Manual • June 2016
Page 75
Power On an NVMe Storage Drive and Attach a Device Driver
2.
If necessary, remove the drive filler panel.
3.
Align the replacement drive with the drive slot.
The drive is physically addressed according to the slot in which it is installed. It is important to install a replacement drive in the same slot as the drive that was removed.
4.
Slide the drive into the slot until the drive is fully seated.
See frame [1] Figure 10.
5.
Close the drive latch to lock the drive in place [2].
See frame [2] Figure 10.
Power On an NVMe Storage Drive and Attach a Device Driver
To power on the slot and attach the device driver, type:
# echo 1 /sys/bus/pci/slots/$slot/power
Where $slot is the slot number for the NVMe storage drive.
Verify Operation of an NVMe Storage Drive
To verify that an NVMe drive is operating properly, do one of the following:
■
Check the /var/log/messages log file.
■
Type: # ls -l /dev/nvme*.
Related Information
■
“Storage Drive Status Indicators” on page 65
■
“Storage Drives Hot-Plug Conditions” on page 64
■
“Storage Drive Failure and RAID” on page 64
■
“Storage Drive Locations and Numbering” on page 64

Servicing Fan Modules (CRU)

The fan modules are located in the front of the server. See the following procedures:
■
“Remove a Fan Module” on page 76
■
“Install a Fan Module” on page 79
Servicing CRUs That Do Not Require Server Power-Off 75
Page 76

Remove a Fan Module

Related Information

■
Remove a Fan Module
You do not have to power off the server to service the fan modules.
You should not begin this procedure unless you have a replacement fan and you are ready to install it right away.
Caution - When removing and replacing a fan module in the Oracle Server X6-2, you must
complete the entire procedure within 20 seconds to maintain adequate cooling within the system. Remove and replace only one fan module at a time. Ensure that you have obtained the replacement fan module and that is it is ready for installation before starting the replacement procedure. Each fan module contains two fans, with two fan motors per fan. The four fan motors provide separate tachometer signals so that the fan module reports four tach signals to Oracle ILOM. Even if only one fan motor is faulted within the fan module, the Oracle ILOM service processor detects that four fan motors have failed to spin while the fan module is removed. If the fan module is not replaced within 20 seconds of removal, Oracle ILOM will take the protective action to shut down the system to prevent thermal damage to the system.
“Open the Server Fan Door” on page 59
1.
Extend the server to the maintenance position.
See “Extend the Server to the Maintenance Position” on page 56.
2.
To access the fan modules, open the server fan door.
See “Open the Server Fan Door” on page 59.
Caution - Close the server fan door within 60 seconds to maintain adequate airflow to properly
cool the server. Leaving the door open for more than 60 seconds, while the server is running, might cause the server to overheat and shut down.
3.
Identify the faulty fan module.
Each fan module has a status indicator (LED). If the LED is off, the fan is good. If the LED is lit amber, the fan has failed. The LEDs are located on the chassis mid-wall adjacent to and aligned with the fan modules, as shown in the following figure.
76 Oracle Server X6-2 Service Manual • June 2016
Page 77
Remove a Fan Module
FIGURE  11
Server Fan Module Status Indicator
Status Indicator Name
Fan Module Status
4.
Remove the faulty fan module.
a.
Using your thumb and forefinger, grasp the fan module in the finger
Color State Meaning
Off/Amber ■ Off – The fan module is correctly installed and operating within
recesses located in the plastic between the fans.
specification.
■ Amber – The fan module is faulty. The front TOP FAN LED and the front and rear panel Service Required LEDs are also lit if the system detects a fan module fault.
Servicing CRUs That Do Not Require Server Power-Off 77
Page 78
Remove a Fan Module
Caution - When removing a fan module, do not rock it back and forth. Rocking a fan module
can cause damage to the motherboard connectors.
b.
Lift the fan module straight up and out of the chassis.
FIGURE  12
Removing a Fan Module
5.
Set the fan module aside.
Caution - Do not service any other components in the fan compartment unless the system is
shut down and the power cords are removed.
6.
Consider your next step:
■
If you removed the fan assembly as part of another procedure, return to that procedure.
■
Otherwise, continue to “Install a Fan Module” on page 79.
Related Information
■
“Install a Fan Module” on page 79
78 Oracle Server X6-2 Service Manual • June 2016
Page 79

Install a Fan Module

1.
Remove the replacement fan module from its packaging and place it on an antistatic mat.
2.
With the fan door open, position the replacement fan module into the server.
The fan modules are keyed to ensure that they are installed in the correct orientation.
Install a Fan Module
FIGURE  13
Installing a Fan Module
3.
Press down on the fan module and apply firm pressure to fully seat the fan module.
4.
Verify that the fan module status indicator (LED) that aligns with the replaced fan module is extinguished.
5.
Close the fan door.
6.
Verify that the Top Fan Fault LED on the front of the server and the Service Required LEDs on the front and rear of the server are extinguished.
See “Troubleshooting Using the Server Front and Rear Panel Status Indicators” on page 28 for more information about identifying and interpreting system indicators.
Servicing CRUs That Do Not Require Server Power-Off 79
Page 80

Servicing Power Supplies (CRU)

7.
Consider your next step:
■
If you installed the fan module as part of another procedure, return to that procedure.
■
Otherwise, return the server to operation. See “Returning the Server to
Operation” on page 187.
Related Information
■
“Remove a Fan Module” on page 76
Servicing Power Supplies (CRU)
The redundant power supplies on the server support concurrent maintenance, which enables you to remove and replace a power supply without shutting down the server, provided that the other power supply is online and working.
The server supports model A256 (600 watt) power supplies. The A256 power supply unit (PSU) provides conversion from the AC lines to the system, accepting ranges from 100-240 volts AC (VAC). These PSUs are designed to be hot-swappable, and in most cases will provide fully redundant ”1+1” power, allowing the system to suffer the loss of a PSU or an AC feed with no loss to system availability.
In maximally configured systems, it is possible that the worst-case power consumption of the system could exceed the capacity of a single PSU. The PSUs provide an over-subscription mode, which allows the system to operate with fault-tolerance even with modest excursions beyond the rated capacity of a single PSU. This over-subscription support is accomplished using hardware signaling between the PSU and motherboard circuitry, which can force the system to throttle maximally CPU and memory power in the event that a PSU is lost. The resulting power savings will be enough to allow the system to continue to run (in a lower­performance state) until the power problem is resolved.
The following indicators (LEDs) are lit when a power supply fault is detected:
■
Front and rear Service Required LEDs
■
Amber Service Required LED on the faulty power supply
■
Rear Power Supply Fault LED located on the server front panel (see “Front Panel Controls
and Indicators” on page 14)
If a power supply fails and you do not have a replacement available, leave the failed power supply installed to ensure proper airflow in the server.
For more information, refer to the following topic and procedures:
■
“Power Supply Status Indicators” on page 81
■
“Remove a Power Supply” on page 81
■
“Install a Power Supply” on page 83
80 Oracle Server X6-2 Service Manual • June 2016
Page 81

Remove a Power Supply

Related Information

■
“Troubleshooting Using the Server Front and Rear Panel Status Indicators” on page 28
■
“Powering Down the Server” on page 51

Power Supply Status Indicators

Each power supply contains two status indicators (LEDs) on the back panel. The following figure and table provide a description of the power supply status indicators.
FIGURE  14
TABLE 11
Legend Status
1 AC OK/DC
2 Service
Server Power Supply Indicators
Indicator Name
OK
Required
Power Supply Status Indicators
Icon Color State Meaning
Green ■ OFF – No AC power is present.
■ SLOW BLINK – Normal operating. Input power is within specification. DC output voltage is not enabled.
■ STEADY ON – Normal operation. Input AC power and DC output voltage are within specification.
Amber ■ OFF – Normal operation; no service required.
■ STEADY ON – The power supply (PS) has detected a PS fan failure, PS overtemperature, PS over current, or PS over/under voltage.
Remove a Power Supply
1.
Identify which power supply requires replacement.
A lit amber Service Required LED on a power supply indicates that a failure was detected. You can also use the Oracle ILOM show faulty command at the Oracle ILOM command line
prompt (->) to identify a power supply failure.
Servicing CRUs That Do Not Require Server Power-Off 81
Page 82
Remove a Power Supply
Alternatively, to list all known faults in the server, log into the Oracle Solaris OS and issue the fmadm faulty command, or log into the Oracle ILOM service processor from the Oracle ILOM Fault Management Shell and issue the fmadm faulty command. For more information
about how to use the Oracle ILOM Fault Management Shell and supported commands, see the Oracle ILOM User's Guide for System Monitoring and Diagnostics Firmware Release 3.2.x in
the Oracle Integrated Lights Out Manager (ILOM) 3.2 Documentation Library at http://www.
oracle.com/goto/ilom/docs.
Note - The fans of a failed power supply might still be spinning when the system is powered on.
You can remove a power supply while the fans are still spinning.
2.
Gain access to the rear of the server where the faulty power supply is located.
3.
If the cable management arm (CMA) is installed, disconnect both CMA left-side connectors and move the CMA out of the way.
For instructions for disconnecting the CMA left-side connectors, refer to“Remove the Cable
Management Arm” in Oracle Server X6-2 Installation Guide.
Caution - When disconnecting the CMA left-side connectors, use your arm to support the
CMA so that it does not hang down under its own weight and stress the right-side connectors; otherwise, the CMA might be damaged. You must continue to support the CMA until you have reconnected both of the left-side connectors.
4.
Disconnect the power cord from the faulty power supply.
5.
Grasp the power supply handle and push the power supply latch to the left [1].
FIGURE  15
82 Oracle Server X6-2 Service Manual • June 2016
Removing a Power Supply
Page 83

Install a Power Supply

Figure Legend
1
Power Supply 0 (PS0)
2
Power Supply 1 (PS1)
6.
Pull the power supply out of the chassis [2].
Caution - Whenever you remove a power supply, you should replace it with another power
supply; otherwise, the server might overheat due to improper airflow.
7.
Continue to “Install a Power Supply” on page 83.
Related Information
■
“Server Back Panel View” on page 16
■
“Power Supply Status Indicators” on page 81
■
“Install a Power Supply” on page 83
Install a Power Supply
Caution - Always replace a faulty power supply with a power supply of the same type (model).
1.
Remove the replacement power supply from its packaging and place it on an antistatic mat.
2.
Align the replacement power supply with the empty power supply slot [1].
3.
Slide the power supply into the bay until it is fully seated [2].
Servicing CRUs That Do Not Require Server Power-Off 83
Page 84
Install a Power Supply
4.
5.
You will hear an audible click when the power supply fully seats.
Reconnect the power cord to the power supply.
Verify that the amber LED on the replaced power supply and the Service Required LEDs on the front and rear panels of the server are not lit.
Note - After you have replaced Power Supply 0, you must reset the Oracle ILOM service
processor (SP) to propagate the key identity properties (KIP) data to the new power supply. For instructions on resetting the SP, see the Oracle Integrated Lights Out Manager (ILOM)
3.2 Configuration and Maintenance Guide at http://www.oracle.com/goto/ilom/docs. Power Supply 1 does not contain KIP data, and therefore does not require an SP reset after replacement.
6.
If you disconnected the two CMA left-side connectors, reconnect the connectors.
For instructions for reconnecting the CMA left-side connectors, refer to “Install the Cable
Management Arm” in Oracle Server X6-2 Installation Guide.
Related Information
■
“Server Back Panel View” on page 16
■
“Power Supply Status Indicators” on page 81
■
“Remove a Power Supply” on page 81
84 Oracle Server X6-2 Service Manual • June 2016
Page 85

Servicing CRUs That Require Server Power-Off

These sections describe how to service customer-replaceable units (CRUs) that require you to power off the server. For more information about CRUs, see “Illustrated Parts
Breakdown” on page 17 and “Customer-Replaceable Units” on page 19.
Description Use
Service the DIMMs. “Servicing the DIMMs (CRU)” on page 85
Service the PCIe Risers. “Servicing PCIe Risers (CRU)” on page 97
Service the PCIe cards. “Servicing PCIe Cards (CRU)” on page 108
Service the internal USB flash drives. “Servicing the Internal USB Flash Drives
Service the battery. “Servicing the Battery (CRU)” on page 116

Related Information

■
“Servicing CRUs That Do Not Require Server Power-Off” on page 63
■
“Servicing FRUs” on page 119
(CRU)” on page 113

Servicing the DIMMs (CRU)

The Oracle Server X6-2 supports a variety of DDR4 DIMM configurations that can include quad-rank (QR) and dual-rank (DR) DDR4 DIMMs.
Caution - These procedures require that you handle components that are sensitive to
electrostatic discharge. This sensitivity can cause the components to fail. To avoid damage, ensure that you follow antistatic practices as described in “Take Antistatic
Measures” on page 59.
Caution - Ensure that all power is removed from the server before removing or installing DDR4
DIMMs, or damage to the DDR4 DIMMs might occur. You must disconnect all power cables from the system before performing these procedures.
Servicing CRUs That Require Server Power-Off 85
Page 86
Servicing the DIMMs (CRU)
The following topics and procedures provide information to assist you when replacing a DIMM or upgrading DIMMs:
■
“DIMM and Processor Physical Layout” on page 86
■
“DIMM Population Scenarios” on page 87
■
“DIMM Population Rules” on page 88
■
“Populating DIMMs for Optimal System Performance” on page 88
■
“DIMM Operating Speeds” on page 92
■
“DIMM Rank Classification Labels” on page 92
■
“Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of Failed DIMMs” on page 92
■
“Using the Server Fault Remind Button” on page 92
■
“Identify and Remove the Failed DIMM” on page 93
■
“Install a DIMM” on page 95

Related Information

■
“Servicing CRUs That Do Not Require Server Power-Off” on page 63
■
“Servicing FRUs” on page 119

DIMM and Processor Physical Layout

The physical layout of the DIMMs and processor(s) is shown in the following figure. When viewing the server from the front, processor 0 (P0) is on the left. Notice that each processor, P0 and P1, has four memory channels that are labeled, from left to right, Ch C, Ch D, Ch B, and Ch A.
86 Oracle Server X6-2 Service Manual • June 2016
Page 87
Servicing the DIMMs (CRU)
FIGURE  16
Note - In single-processor systems, the DIMM sockets associated with the processor 1 (P1)
DIMM and Processor Physical Layout
are nonfunctional and should not be populated with DIMMs. A maximum of 12 DIMMs are supported in single-processor systems and the DIMMs must be installed in DIMM sockets associated with the P0 processor socket.
Related Information
■
“DIMM Population Scenarios” on page 87
■
“DIMM Population Rules” on page 88
■
“Populating DIMMs for Optimal System Performance” on page 88
■
“DIMM Operating Speeds” on page 92
■
“DIMM Rank Classification Labels” on page 92
■
“Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of Failed DIMMs” on page 92

DIMM Population Scenarios

There are two scenarios in which you are required to populate DIMMs:
■
A DIMM fails and needs to be replaced.
In this scenario, you can use the Fault Remind button to determine the failed DIMM, then remove the failed DIMM and replace it. To ensure that system performance is maintained,
Servicing CRUs That Require Server Power-Off 87
Page 88
Servicing the DIMMs (CRU)
you must replace the failed DIMM with a DIMM of the same size (in gigabytes) and type (quad-rank or dual-rank). In this scenario, you should not change the DIMM configuration.
■
You have purchased new DIMMs and you want to use them to upgrade the server's memory.
In this scenario, you should adhere to DIMM population rules and follow the recommended DIMM population order for optimal system performance.
Related Information
■
“DIMM Population Rules” on page 88
■
“Populating DIMMs for Optimal System Performance” on page 88
■
“Using the Server Fault Remind Button” on page 92

DIMM Population Rules

The population rules for adding DIMMs to the server are as follows:
■
The server supports 64-GB quad-rank (QR) Load-Reduced DIMMs (LRDIMMs) and 32­GB dual-rank (DR) and 16-GB DR Registered DIMMs (RDIMMs).
■
Do not mix 64-GB LRDIMMs and 16-GB or 32-GB RDIMMs in the same server. If the server has 64-GB LRDIMMs installed, you cannot install 16-GB or 32-GB RDIMMs and vice versa.
■
Within a memory channel, DIMMs must be populated in the black sockets first, then in the black sockets with white tabs, and then in the white sockets.
■
If you are installing a mix of RDIMM sizes (for example, 16-GB and 32-GB), install the larger size RDIMMs first, then the smaller RDIMMs, until all of the RDIMMs are installed.
■
There are no restrictions on the minimum number of DIMM sockets that must be populated. The server will operate properly with one DIMM installed.

Populating DIMMs for Optimal System Performance

Optimal performance is achieved by populating the DIMMs so that the memory is symmetrical, or balanced. Symmetry is achieved by adhering to the following guidelines:
■
In single-processor systems, populate DIMMs of the same size in multiples of four.
■
In dual-processor systems, populate DIMMs of the same size in multiples of eight.
■
Populate the DIMM sockets in the order described in the following sections.
The following sections provide an example of how to populate the DIMM sockets to achieve optimal system performance.
88 Oracle Server X6-2 Service Manual • June 2016
Page 89
Servicing the DIMMs (CRU)
Note - Not all possible configurations are shown here.
■
“Populating DIMMs in Single-Processor Systems for Optimal System Performance” on page 89
■
“Populating DIMMs in Dual-Processor Systems for Optimal System Performance” on page 90
Populating DIMMs in Single-Processor Systems for Optimal System Performance
In single-processor systems, install DIMMs only into DIMM sockets associated with processor 0 (P0). Starting with socket P0 D0. Fill the black sockets first, then the black sockets with white tabs, and then the white sockets, as shown in the following figure. Notice that the order of populating the black sockets is Ch C first, Ch A second, Ch D third, and Ch B last. Repeat the same population sequence for the black sockets with white tabs, and finally for the white sockets.
FIGURE  17
DIMM Population Order for Single-Processor Systems
The following table describes the proper order in which to populate DIMMs in a single­processor system using the numbered callouts in the above figure, the memory channel labels (Ch A through Ch D), and the DIMM socket labels (D0 through D11).
Servicing CRUs That Require Server Power-Off 89
Page 90
Servicing the DIMMs (CRU)
TABLE 12
Population Order DIMMs Sockets for Processor 0 (P0) Memory Channels
Fill the black sockets first
First D0 Ch C
Second D11 Ch A
Third D3 Ch D
Fourth D8 Ch B
Then fill the black sockets with white tabs
Fifth D1 Ch C
Sixth D10 Ch A
Seventh D4 Ch D
Eighth D7 Ch B
Then fill the white sockets
Ninth D2 Ch C
Tenth D9 Ch A
Eleventh D5 Ch D
Twelfth D6 Ch B
DIMM Population Order for Single-Processor Systems
Populating DIMMs in Dual-Processor Systems for Optimal System Performance
In dual-processor systems, populate DIMMs into DIMM sockets starting with processor 0 (P0) D0, then, alternating between sockets associated with processor 0 (P0) and matching sockets for processor 1 (P1). Fill the black sockets first, then the black sockets with white tabs, and then the white sockets, as shown in the following figure. Notice that the order of populating the black DIMM sockets is Ch C first, Ch A second, Ch D third, and Ch B last. Repeat the same population sequence for the black sockets with white tabs, and finally, for the white sockets.
90 Oracle Server X6-2 Service Manual • June 2016
Page 91
Servicing the DIMMs (CRU)
FIGURE  18
DIMM Population Order for Dual-Processor Systems
The following table describes the proper order in which to install DIMMs in a dual-processor system using the numbered callouts in the above figure, the memory channels labels (Ch A through Ch D), and the DIMM socket labels (D0 through D11).
TABLE 13
Population Order DIMM Sockets for Processor
Fill the black sockets first (alternating between processor 0 and processor 1)
First Fill D0 first Then fill D0 Ch C
Second Fill D11 first Then fill D11 Ch A
Third Fill D3 first Then fill D3 Ch D
Fourth Fill D8 first Then fill D8 Ch B
Then fill the black sockets with white tabs (alternating between processor 0 and processor 1)
Fifth Fill D1 first Then fill D1 Ch C
Sixth Fill D10 first Then fill D10 Ch A
Seventh Fill D4 first Then fill D4 Ch D
Eighth Fill D7 first Then fill D7 Ch B
Then fill the white sockets (alternating between processor 0 and processor 1)
Ninth Fill D2 first Then fill D2 Ch C
Tenth Fill D9 first Then fill D9 Ch A
Eleventh Fill D5 first Then fill D5 Ch D
DIMM Population Order for Dual-Processor Systems
0 (P0)
DIMM Sockets for Processor 1 (P1)
Servicing CRUs That Require Server Power-Off 91
Memory Channels
Page 92
Servicing the DIMMs (CRU)
Population Order DIMM Sockets for Processor
0 (P0)
Twelfth Fill D6 first Then fill D6 Ch B
DIMM Sockets for Processor 1 (P1)
Memory Channels

DIMM Operating Speeds

The maximum supported memory speed is 2400 MT/s. However, not all system configurations will support operation at this speed. The maximum attainable memory speed will be limited by the maximum speed supported by the specific type of processor and the specific memory population. All memory installed in the system will operate at the same speed, or frequency.

DIMM Rank Classification Labels

DIMMs come in a variety of ranks: dual or quad. Each DIMM is shipped with a label identifying its rank classification. The following table identifies the label corresponding to each DIMM rank classification.
TABLE 14
Rank Classification Label
Quad-rank LRDIMM 4Rx4
Dual-rank RDIMM 2Rx4
DIMM Rank Classifications Labels

Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of Failed DIMMs

When a single DIMM is marked as failed by Oracle ILOM (for example, fault.memory. intel.dimm.training-failed is listed in the service processor Event Log), BIOS might
disable the entire memory channel that contains the failed DIMM, up to three DIMMs. As a result, none of the memory installed in the disabled channel will be available to the operating system. However, when the Fault Remind button is pressed, only the fault status indicator (LED) associated with the failed DIMM lights. The fault LEDs for the other DIMMs in the memory channel remain off. Therefore, you can correctly identify the failed DIMM using the lit LED.

Using the Server Fault Remind Button

When the Fault Remind button is pressed, an LED located next to the Fault Remind button lights green to indicate that there is sufficient voltage present in the fault remind circuit to light
92 Oracle Server X6-2 Service Manual • June 2016
Page 93

Identify and Remove the Failed DIMM

any fault LEDs that were lit due to a component failure. If this LED does not light when you press the Fault Remind button, it is likely that the capacitor powering the fault remind circuit has lost its charge. This can happen if the Fault Remind button is pressed for several minutes with fault LEDs lit or if power has been removed from the server for more than 15 minutes.
The following figure shows the location of the Fault Remind button.
FIGURE  19
Fault Remind Button Location
Identify and Remove the Failed DIMM
1.
Prepare the server for service.
a.
Power off the server, and disconnect the power cords from the power supplies.
See “Powering Down the Server” on page 51
b.
Extend the server to the maintenance position.
Servicing CRUs That Require Server Power-Off 93
Page 94
Identify and Remove the Failed DIMM
See “Extend the Server to the Maintenance Position” on page 56.
c.
Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis.
See “Take Antistatic Measures” on page 59.
d.
Remove the server top cover.
See “Remove the Server Top Cover” on page 60.
2.
Identify and note the location of the failed DDR4 DIMM by pressing the Fault Remind button on the motherboard (see “Using the Server Fault Remind
Button” on page 92).
Failed DDR4 DIMMs are identified with a corresponding amber LED on the motherboard.
■
If the DIMM fault LED is off, the DIMM is operating properly.
■
If the DIMM fault LED is on (amber), then the DIMM has failed and should be replaced.
3.
To remove the failed DIMM do the following:
a.
Rotate both DIMM socket ejectors outward as far as they will go (See Figure
20).
The DIMM is partially ejected from the socket.
94 Oracle Server X6-2 Service Manual • June 2016
Page 95

Install a DIMM

FIGURE  20
Figure Legend
1
DIMM connector socket
2
DIMM connector key
3
DIMM ejector lever
b.
Carefully lift the DIMM straight up to remove it from the socket.
DIMM Socket Release and Alignment
4.
Replace each failed DIMM with either another DIMM of the same rank size (quad rank or dual rank) or leave the socket empty.
For DIMM replacement instructions, see “Install a DIMM” on page 95.
Related Information
■
“DIMM and Processor Physical Layout” on page 86
■
“Install a DIMM” on page 95
Install a DIMM
1.
Unpack the replacement DDR4 DIMM and place it on an antistatic mat.
2.
Ensure that the replacement DDR4 DIMM matches the size of the DIMM it is replacing.
Servicing CRUs That Require Server Power-Off 95
Page 96
Install a DIMM
You must not replace a dual-rank DIMM with a quad-rank DIMM and vice versa. If you violate this rule, the performance of the server might be adversely affected. For DIMM socket population rules, see “DIMM Population Rules” on page 88.
3.
Install a DIMM.
a.
Ensure that the ejector tabs are in the open position.
b.
Align the notch in the replacement DIMM with the connector key in the connector socket.
The notch ensures that the DIMM is oriented correctly.
c.
Push the DDR4 DIMM into the connector until the ejector tabs lock the DIMM in place.
If the DIMM does not easily seat into the connector socket, verify that the notch in the DIMM is aligned with the connector key in the connector socket. If the notch is not aligned, damage to the DIMM might occur.
Note - If you want to move a faulty DDR4 DIMM to a new DIMM socket or re-seat a DIMM
for troubleshooting, you must first manually clear the associated DIMM fault. Otherwise, the DIMM fault might follow the DIMM to the new socket location or reoccur on the current socket location, possibly causing a false DIMM fault condition.
4.
Repeat Step 3 until all replacement DDR4 DIMMs are installed.
5.
Return the server to operation.
a.
Install the server top cover.
See “Install the Server Top Cover” on page 188.
b.
Close the fan door.
c.
Return the server to the normal rack position.
See “Return the Server to the Normal Rack Position” on page 191.
d.
Reconnect the power cords to the server power supplies and reconnect the data cables.
See “Reconnect Data Cables and Power Cords” on page 193.
e.
Power on the server.
See “Power On the Server” on page 194.
Verify that the Power/OK status indicator is steady on.
96 Oracle Server X6-2 Service Manual • June 2016
Page 97

Servicing PCIe Risers (CRU)

6.
(Optional) Use Oracle ILOM to clear server DDR4 DIMM faults.
DDR4 DIMM faults are automatically cleared after a new DIMM has been installed. If you need to manually clear DDR4 DIMM faults, refer to the Oracle Integrated Lights Out Manager
(ILOM) 3.2 Documentation Library at http://www.oracle.com/goto/ilom/docs.
Note - If you want to move a faulty DDR4 DIMM to a new DIMM socket or re-seat a DIMM
for troubleshooting, you must first manually clear the associated DIMM fault. Otherwise, the DIMM fault might follow the DIMM to the new socket location or reoccur on the current socket location, causing a false DIMM fault condition.
Related Information
■
“DIMM and Processor Physical Layout” on page 86
■
“DIMM Population Scenarios” on page 87
■
“DIMM Population Rules” on page 88
■
“Populating DIMMs for Optimal System Performance” on page 88
■
“DIMM Rank Classification Labels” on page 92
■
“Identify and Remove the Failed DIMM” on page 93
Servicing PCIe Risers (CRU)
PCIe cards in all slots are installed on vertical risers. You must remove the relevant riser to remove and replace a PCIe card. You must remove all three PCIe risers when replacing the motherboard.
Caution - These procedures require that you handle components that are sensitive to
electrostatic discharge. This sensitivity can cause the components to fail. To avoid damage, ensure that you follow antistatic practices as described in “Take Antistatic
Measures” on page 59.
Caution - Ensure that all power is removed from the server before removing or installing PCIe
risers, or damage to the PCIe cards installed in the risers might occur. You must disconnect all power cables from the system before performing these procedures.
The following sections provide information to assist you in servicing PCIe risers:
■
“PCIe Riser Location and Differences” on page 98
■
“Remove a PCIe Riser From PCIe Slot 1 or 2” on page 99
■
“Install a PCIe Riser Into PCIe Slot 1 or 2” on page 101
■
“Remove the PCIe Riser From PCIe Slots 3 and 4” on page 103
Servicing CRUs That Require Server Power-Off 97
Page 98
Servicing PCIe Risers (CRU)
■
“Install the PCIe Riser Into PCIe Slots 3 and 4” on page 105

Related Information

■
“Servicing PCIe Cards (CRU)” on page 108

PCIe Riser Location and Differences

The PCIe riser that installs in PCIe slots 3 and 4 is different than the risers in PCIe slots 1 and 2. The riser for slots 3 and 4 supports two cards: a standard PCIe card and the internal HBA card. Do not attempt to install the PCIe riser for PCIe slots 3 and 4 in slots 1 or 2, and vice versa.
FIGURE  21
PCIe Riser Locations
Figure Legend
1
PCIe riser and installed PCIe card or the optional Oracle PCIe NVMe switch card in slot 1 (This slot is nonfunctional in single-processor systems.)
2
PCIe riser and installed PCIe card in slot 2
98 Oracle Server X6-2 Service Manual • June 2016
Page 99

Remove a PCIe Riser From PCIe Slot 1 or 2

3
PCIe riser and installed cards (2) in slots 3 and 4 (the internal HBA card is installed slot 4 of this riser.) Note: This riser is different from the risers in slots 1 and 2.
Related Information
■
“PCIe Slot Characteristics” on page 108
Remove a PCIe Riser From PCIe Slot 1 or 2
Note - Either a standard PCIe card (CRU) or the optional PCIe NVMe switch card can be
installed in the PCIe riser for slot 1. The PCIe NVMe switch card is a field replaceable unit (FRU) and, therefore, must be removed and replaced only by authorized Oracle Services personnel. For instructions for servicing the PCIe NVMe switch card, see “Servicing the Oracle
PCIe NVMe Switch Card (FRU)” on page 137.
Note - PCIe slot 1 is nonfunctional in single-processor systems.
1.
Prepare the server for service.
a.
Power off the server and disconnect the power cords from the power supplies.
See “Powering Down the Server” on page 51.
b.
Extend the server to the maintenance position.
See “Extend the Server to the Maintenance Position” on page 56.
c.
Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis.
See “Take Antistatic Measures” on page 59.
d.
Remove the server top cover.
See “Remove the Server Top Cover” on page 60.
2.
Select the PCIe riser that you want to remove.
See “PCIe Slot Characteristics” on page 108.
3.
If a PCIe card is installed in the riser, disconnect any external cables from the card.
Servicing CRUs That Require Server Power-Off 99
Page 100
Remove a PCIe Riser From PCIe Slot 1 or 2
4.
Lift the green-tabbed latch on the rear of the server chassis next to the applicable PCIe slot to release the rear bracket on the PCIe card [1].
Note - If the riser does not have a PCIe card installed, then lift the latch to release the PCIe slot
filler panel.
5.
Lift the green-tabbed riser release lever on the PCIe riser with one hand and use your other hand to remove the riser from the motherboard connector [2, 3].
6.
If there is a PCIe card installed in the riser, place the riser on an antistatic mat and note the slot in which the PCIe riser was originally installed; otherwise, set the PCIe riser aside.
Related Information
■
“Install a PCIe Riser Into PCIe Slot 1 or 2” on page 101
100 Oracle Server X6-2 Service Manual • June 2016
Loading...
Buy Points How it Works FAQ Contact Us Questions and Suggestions Privacy Policy Cookie Policy Terms of Use