Oracle Sun Fire X4800 Product Notes

Download

Page 1

Sun Fire X4800 Server Product Notes

Part No: E69641-07

July 2017

Page 2

Page 3

Sun Fire X4800 Server Product Notes

Part No: E69641-07

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?

ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Page 4

Référence: E69641-07

Ce logiciel et la documentation qui l'accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d'utilisation et de divulgation. Sauf stipulation expresse de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, accorder de licence, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d'interopérabilité avec des logiciels tiers ou tel que prescrit par la loi.

Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu'elles soient exemptes d'erreurs et vous invite, le cas échéant, à lui en faire part par écrit.

Si ce logiciel, ou la documentation qui l'accompagne, est livré sous licence au Gouvernement des Etats-Unis, ou à quiconque qui aurait souscrit la licence de ce logiciel pour le compte du Gouvernement des Etats-Unis, la notice suivante s'applique :

Ce logiciel ou matériel a été développé pour un usage général dans le cadre d'applications de gestion des informations. Ce logiciel ou matériel n'est pas conçu ni n'est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer un risque de dommages corporels. Si vous utilisez ce logiciel ou ce matériel dans le cadre d'applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l'utilisation de ce logiciel ou matériel pour des applications dangereuses.

Oracle et Java sont des marques déposées d'Oracle Corporation et/ou de ses affiliés. Tout autre nom mentionné peut correspondre à des marques appartenant à d'autres propriétaires qu'Oracle.

Intel et Intel Xeon sont des marques ou des marques déposées d'Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d'Advanced Micro Devices. UNIX est une marque déposée de The Open Group.

Ce logiciel ou matériel et la documentation qui l'accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l'accès à des contenus, produits ou services tiers, ou à leur utilisation, sauf mention contraire stipulée dans un contrat entre vous et Oracle.

Accès aux services de support Oracle

Les clients Oracle qui ont souscrit un contrat de support ont accès au support électronique via My Oracle Support. Pour plus d'informations, visitez le site http://www.oracle.com/

pls/topic/lookup?ctx=acc&id=info ou le site http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs si vous êtes malentendant.

Page 5

Using This Documentation ............. ................ ................ ................ ................ ... 11

Product Documentation Library ....................................................................... 11

Feedback ............. ................ ................ ................ ................ ................ ......... 11

Important Operating Notes ............. ................ ................ ................ ................ ... 13

Server Security, Software Releases, and Critical Patch Updates ... ................ ........... 13

▼ IMPORTANT - Install Latest OS Updates, Patches, and Firmware . ............ 14

Diagnosing SAS Data Path Failures on Servers Using MegaRAID Disk

Controllers ............. ................ ................ ................ ................ ................ ...... 15

Oracle ILOM Deprecation Notice for IPMI 2.0 Management Service Present

Behavior ............. ................ ................ ................ ................ ................ ......... 16

Oracle ILOM Deprecation Notice for Default Self-Signed Certificate ...................... 17

Broken Links in Sun Server X4800 Documentation Library . ................ ................ . 17

Supported Software and Firmware ................................................................... 19

Supported Operating Systems ........ ................ .................................................. 19

Supported Firmware .... ................ ................ ................ ................ ................ ... 20

Additional Software .... ................ ................ ................ ................ ................ ... 20

Sun Java Enterprise System ............................................................................ 20

Oracle Enterprise Manager Ops Center .............................................................. 21

MegaRAID Storage Manager (MSM) ............................................................... 21

SunVTS Bootable Diagnostics CD-ROM ........... ................................................ 21

Oracle Integrated Lights Out Manager (Oracle ILOM) ......................................... 22

Oracle Hardware Management Pack ....... ................ ................ ................ .......... 22

Documentation Errata ..................................................................................... 22

Use Oracle ILOM 3.2 Documentation After Updating to System Software

Release 1.7.0 or Newer .......................................................................... 22

Antistatic Wrist Straps Are Not Included With All CRUs and FRUs ................ 23

Page 6

Contents

Previous Versions of the Service Manual Incorrectly State That a FEM Is Only

Supported on CMOD 0 (CR 7107085) ...................................................... 23

Links to Oracle Websites Have Been Updated ............................................ 23

Name Correction for Oracle Integrated Lights Out Manager .. ................ ........ 24

Front Mounting Bracket Illustration from Installation Guide is Incorrect . ......... 24

Oracle x86 Products Accessibility ............ ................ ................ ................ ........ 27

Hardware Accessibility ................................................................................... 27

Oracle ILOM Accessibility ............................................................................. 28

Oracle Hardware Management Pack Accessibility ............................................... 29

BIOS Accessibility ........................................................................................ 29

Documentation Accessibility ........................................................................... 30

Hardware Issues ................................................................................................ 31

System Posts CPU Fault and Fails to Boot On System With Six Sun InfiniBand Dual

Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards (22536804) .... ....... 32

Reset Takes a Long Time and Causes the Server to Power Cycle ............................ 32

Update SP Firmware After Replacing a CMOD or SP Module (15465497) ............... 33

MegaCLI CfgEachDskRaid0 Command Does Not Work Correctly (CR 7121867) ....... 33

DIMM Mismatch on Odd Numbered Pairs Results in DDR Training Failed on Entire

Branch (CR7101624 and CR7111545) .............. ................ ................ ................ . 34

If DIMM Failure Causes Branch Failure, Oracle ILOM Might Not Identify Them as

Faulty (CR7099038 and CR7111543) ................................................................ 35

When Inserting a CMOD, Simultaneously Rotate Handles Until They Touch the

Chassis ............. ................ ................ ................ ................ ................ ........... 35

Video Output of CMM Is Distorted or Missing ............ ....................................... 35

SATA Drives Not Accessible Through NEM1 (CR 7003993) ................................. 36

Supported Rack Mounting Configuration ........... ................ ................ ................ 36

DIMM Population Rules ................................................................................. 36

Matching Parameters .............................................................................. 37

DIMM Population Order ........................................................................ 37

Locking Screws Secure Shelf Rails to Rear Mounting Brackets .............................. 39

▼ How to Install the Locking Screws ...................................................... 39

Message About Missing Disks After Failed Boot (CR 6939507) ............................ 41

System Does Not Power Up After Power Cycling (CR 6950414) . ................ .......... 41

System Does Not Recognize DIMM Due to SPD Corruption (CR 6946921) ......... .... 41

▼ How to Identify a Corrupted DIMM .................................................... 42

6 Sun Fire X4800 Server Product Notes • July 2017

Page 7

Contents

Oracle Solaris 10 Operating System Issues ................ ................ ...................... 43

Oracle Solaris Installation Takes a Long Time On Systems With Four or More Sun

InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards ..... 43

Solaris 10 Operating System Overview ............................................................. 44

System Panics During Solaris 10 10/09 OS Installation When Solaris Attempts to

Boot (CR 6880963) ....................................................................................... 45

System Panics or Hangs During Solaris OS Installation (CR 6920274) .................... 46

Messages Complain of Failure to Allocate I/O Resources (CR 6984329) .................. 47

Patch Required for Solaris FMA ...................................................................... 47

Solaris MSI Interrupts Are Depleted (CR 6669984 ) ... ................ ................ ......... 48

System Might Panic With "unowned mutex" Message (CR 6893274) .............. ........ 49

Hotplugging PCIe Express Modules in Slots 2.0 or 2.1 Might Not Work (CR

6954869) ............. ................ ................ ................ ................ ................ ......... 49

A System With a Combo GbE/8Gb FC Express Module HBA Might Get a BAD

TRAP Panic (CR 6942158) ........ ................ ................ ................ ................ ..... 49

Linux Issues ...................................................................................................... 51

Oracle and Red Hat Linux 7.2 Installation Fails With AER Error on Systems With Four or More Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel

Adapter M2 Cards (22686146) ........................................................................ 52

False MCE Errors Appear in /var/log/mcelog (CR 7104293) .............................. 52

I/O Does Not Work On SLES11 SP1 with XEN (CR 6965290 and CR7110443) ........ 53

RHEL6 Kdump Runs Out of Memory (CR 7000993 and CR 7000942) .................... 53

SLES11 SP1 Does Not Produce A Kdump File After A Crash (CR7001706) ............. 54

SLES11 Fails to Install On Systems Equipped with 6–Core CMODs (CR

7024769) ............. ................ ................ ................ ................ ................ ......... 54

Messages Complain of Failure to Allocate I/O Resources (CR 6984329) .................. 55

Xen Profiler Is Not Supported in Oracle Linux 5.5 (CR 6839366) .......................... 55

Oracle Linux 5.5 Does Not Support CPU Throttling (CR 6847286) ........................ 55

InfiniBand PCIe Module Does Not Function Correctly With SLES 11 (CR 6909512,

CR 6921163) ................................................................................................ 56

Bug in Kernel Causes Incorrect MSI Processing (CR 6921163) ......... ................ ..... 56

LEDs on PCIe ExpressModule Work Incorrectly with Oracle Linux 5.5 (CR

6894954) ............. ................ ................ ................ ................ ................ ......... 56

Oracle Linux 5.5 Xen Kernel Hangs Due to Long Timeout (CR 6899040) ................ 57

Oracle Linux 5.5 Issues Xen Warning: Domain attempted WRMSR

0000000000000277 (CR 6902715) ... ................ ................ ................ ................ 58

Page 8

Contents

10GbE PCIe Card in 8-Socket System with SLES11 Can Cause Kernel to Crash (CR

6912821) ............. ................ ................ ................ ................ ................ ......... 58

SLES11 with Xen Hangs When Many PCIe ExpressModule Cards Are Installed (CR

6917397) ............. ................ ................ ................ ................ ................ ......... 59

Oracle VM Issues .............................................................................................. 61

Messages Complain of Failure to Allocate I/O Resources (CR 6984329) .................. 61

Earlier Versions of Oracle VM Only Support 64 Logical CPUs (CR 6915014) ........... 62

Oracle VM Server Installation Parameters Required on Sun Fire X4800 (CR

7094126) ............. ................ ................ ................ ................ ................ ......... 62

Windows Operating System Issues .................................................................. 63

Unspecified CPU Fault After Warm Reset (CR 7054657) ........... ................ .......... 63

Hot Plugging of Some PCIe ExpressModules Not Supported by Windows Server

2008 R2 (CR 6914898, CR 6793369, CR 6923132) ............................................. 64

PXE Installation of Windows Server 2008 R2 Fails at Authentication (CR

6899582) ............. ................ ................ ................ ................ ................ ......... 64

Messages Complain of Failure to Allocate I/O Resources (CR 6984329) .................. 65

Windows 2008 Cannot Be Installed to Combo GbE/8Gb FC ExpressModule HBA

Disks (CR 6984746) ...................................................................................... 66

Hot Plugging of PCIe ExpressModules Is Not Supported by Windows 2008 ............. 66

ESX Issues ........................................................................................................ 67

Messages Complain of Failure to Allocate I/O Resources (CR 6984329) .................. 67

Option Cards Can Fail to Load Device Drivers in Some Configurations (CR

6933436) ............. ................ ................ ................ ................ ................ ......... 68

Warning Messages Appear During Bootup and in Log Files (CRs 6862766, 6861903,

and 6890263) .. ................ ................ .............................................................. 69

VMware ESXi 5.5 Does Not Support MMIO Regions Above 4GB (16480679,

17013064) ............. ................ ................ ................ ................ ................ ....... 69

VMware ESXi 5.5 Runs Out of Interrupts With PCIe Cards (16494653) .................. 70

Oracle ILOM Enhancements and Issues ...... ................ ................ ................ ..... 71

File Transfer Using URI Fails if Target Password Contains Certain Special

Characters (25917655) ................................................................................... 72

Oracle ILOM SNMP v3 Traps Are Not Delivered After SNMP Engine ID Change

(23634048) ............. ................ ................ ................ ................ ................ ...... 72

8 Sun Fire X4800 Server Product Notes • July 2017

Page 9

Contents

Third-Party Web Scan and Test Tools Cause Sluggish Oracle ILOM Performance

(23564626) ............. ................ ................ ................ ................ ................ ...... 73

▼ Oracle ILOM Enhancement Allows IPv4 Only, IPv6 Only, or Dual Stack ............ 74

New Procedures for Updating CPLD (CR 7043418) ........... ................ ................ . 75

Update Oracle ILOM and BIOS Firmware Before Updating Other Device Firmware

(CR 6537282) ............................................................................................... 75

Use the Locate Button to Prove Physical Presence (CR 6881237) .......... ................ . 76

NEM Expander Firmware Update Procedure (CR 6979140) .................................. 76

▼ Workaround 1 Using Oracle ILOM Web GUI ........................................ 76

▼ Workaround 2 Using Oracle ILOM CLI ................ ................ ............... 79

Network Management Port 1 Does Not Work (CR 6935471 and 6854694) ... ............ 81

start /SYS and stop /SYS Commands Cause Power Button Pressed Event In Log

(CR 6906176) ............................................................................................... 81

Event Log Messages Might Be Out of Sequence and Duplicated (CR 6915976) . ........ 81

Incorrect Error Message in Event Log After Restore When Serial Console or

JavaRconsole Session Are Open (CR 6917474) .................................................. 82

Allocated Power Figures Are Incomplete (CR 6931837) ....................................... 82

(Fixed in SW 1.1) LDAP Account Cannot Be Used to Start Console (CR

6969473) ............. ................ ................ ................ ................ ................ ......... 83

(Fixed in SW 1.1) Power Cycling the Host Using the Web Interface Generates an

Error (CR 6909374) ........... ................ ................ ................ ................ ............ 83

(Fixed in SW 1.1) DIMM Mismatch Fault Not Cleared After DIMM Replacement

and Host Power Cycle (CR 6972285) ........ ................ ................ ................ ....... 83

(Fixed in SW 1.0.1) Console Redirect from CLI Fails (CR 6939479) ...................... 84

(Fixed in SW 1.0.1) Oracle ILOM Only Allows One Console Session (CR

6895423) ............. ................ ................ ................ ................ ................ ......... 84

(Fixed in SW 1.0.1) Faults Disappear After SP Reset (CR 6928998) ..... ................ .. 85

(Fixed in SW 1.0.1) Clearing a Power Supply Unit (PSU) Fault Erases Other Fault

Information (CR 6933348 and CR 6939306) ...................................................... 85

(Fixed in SW 1.0.1) pciflash Is Not Available (CR 6867060 and CR 6904922) .. ........ 85

Oracle Hardware Installation Assistant Issues ................................................. 87

ipmitool Chassis Policy and Persistent Boot Option Are Not Supported (22695087,

22686277) ............. ................ ................ ................ ................ ................ ....... 87

Oracle Hardware Installation Assistant Image Does Not Boot On a System With

More Than 512 GB of Memory (CR 6972927) ......... ................ ................ .......... 87

BIOS Issues ................ ................ ................ ................ ................ ................ ....... 89

Page 10

Contents

Auto Boot Host On Power Loss Control Is Deactivated ........................................ 89

Update Oracle ILOM and BIOS Firmware Before Updating Other Devices' Firmware

(CR 6537282) ............................................................................................... 89

Hotplugging PCIe EM Card Fails When Replacing the Original Card With One of a

Different Type (CR 7003634) .......................................................................... 90

Messages Complain of Failure to Allocate I/O Resources (CR 6984329) .................. 90

DIMM Failure Causes Other DIMMs To Be Disabled (CR 6929978) ....... ............... 91

10 Sun Fire X4800 Server Product Notes • July 2017

Page 11

Using This Documentation

■

Overview – Provides late-breaking information, issue status, and product announcements for Oracle's Sun Fire X4800 Server.

■

Audience – System administrators, network administrators, and service technicians.

■

Required knowledge – Advanced understanding of server systems.

Product Documentation Library

Documentation and resources for this product and related products are available at http://

docs.oracle.com/cd/E19140-01/index.html.

Feedback

Provide feedback about this documentation at http://www.oracle.com/goto/docfeedback.

Using This Documentation 11

Page 12

12 Sun Fire X4800 Server Product Notes • July 2017

Page 13

Important Operating Notes

This section provides information about critical issues that affect your server. These include:

■

“Server Security, Software Releases, and Critical Patch Updates” on page 13

■

“IMPORTANT - Install Latest OS Updates, Patches, and Firmware” on page 14

■

“Diagnosing SAS Data Path Failures on Servers Using MegaRAID Disk Controllers” on page 15

■

“Oracle ILOM Deprecation Notice for IPMI 2.0 Management Service Present Behavior” on page 16

■

“Oracle ILOM Deprecation Notice for Default Self-Signed Certificate” on page 17

■

“Broken Links in Sun Server X4800 Documentation Library” on page 17

For known issues affecting system components, see:

■

“Hardware Issues” on page 31

■

“Oracle Solaris 10 Operating System Issues” on page 43

■

“Linux Issues” on page 51

■

“Oracle VM Issues” on page 61

■

“Windows Operating System Issues” on page 63

■

“ESX Issues” on page 67

■

“Oracle ILOM Enhancements and Issues” on page 71

■

“Oracle Hardware Installation Assistant Issues” on page 87

■

“BIOS Issues” on page 89

Server Security, Software Releases, and Critical Patch Updates

To ensure continued security of your system, Oracle strongly recommends that you apply the latest Software Releases. Server Software Releases include Oracle ILOM, BIOS, and other firmware updates, often referred to as “patches.” Oracle publishes these patches regularly on the My Oracle Support site. Applying these patches helps ensure optimal system performance, security, and stability. You can identify the latest Software Release for your system at: http://

Important Operating Notes 13

Page 14

IMPORTANT - Install Latest OS Updates, Patches, and Firmware

www.oracle.com/technetwork/systems/patches/firmware/release-history-jsp-138416. html

To download a Software Release, go to My Oracle Support at: https://support.oracle.com

Oracle notifies customers about security vulnerability fixes for all its products four times a year through the Critical Patch Update (CPU) program. Customers should review the CPU advisories to ensure that the latest software release updates are applied to their Oracle products. Note that updates for Engineered Systems are specifically published for a specific Engineered Systems product (that is, you need not look at specific updates for individual software components included in your Engineered System). For more information about the Oracle CPU program, go to: http://www.oracle.com/technetwork/topics/security/

alerts-086861.html

Oracle also recommends that you update to the latest operating system release when it becomes available. Although a minimum operating system release is supported, updating to the latest OS release ensures that you have the most up-to-date software and security patches. To confirm that you have the latest OS release, refer to the Oracle Hardware Compatibility Lists. See

“Supported Operating Systems” on page 19.

For details about the current system software update, see: “IMPORTANT - Install Latest OS

Updates, Patches, and Firmware” on page 14

IMPORTANT - Install Latest OS Updates, Patches, and Firmware

Some product features are enabled only when the latest versions of operating systems, patches, and firmware are installed. To retain optimal performance, security, and stability, you must install the latest available operating systems, patches, and firmware.

System Software Release 1.11.0 is associated with system firmware version 3.2.10.21. Newer versions have either a higher number, or a letter added. For example, a later system Software Release might be associated with system firmware 3.2.11.0.

To verify that the server firmware version is a minimum of 3.2.10.21 or higher:

Use Oracle ILOM to check your system firmware version.

■ From the web interface, click System Information > Summary, then view the System Firmware Version in the General Information table.

■ From the CLI, type: show /System/Firmware or version.

14 Sun Fire X4800 Server Product Notes • July 2017

Page 15

Diagnosing SAS Data Path Failures on Servers Using MegaRAID Disk Controllers

For more details, refer to information about viewing system information and inventory in the Oracle ILOM Administrators Guide for Configuration and Maintenance, which is available at http://www.oracle.com/goto/ilom/docs.

Ensure that the server firmware version is at the minimum required version, shown above, or a subsequent release, if available.

If the required firmware (or newer) is not installed:

Download the firmware from My Oracle Support at: https://support.oracle.

com

Install the downloaded firmware.

Refer to the information about performing firmware updates in the Oracle ILOM Administrators Guide for Configuration and Maintenance, which is available at http:

//www.oracle.com/goto/ilom/docs. Ensure that you perform the preparatory steps

described in that document before updating the firmware.

Note - Occasionally after installing the firmware, the Oracle ILOM web interface cannot

display the power state correctly on the power control page. To correct this problem, clear your browser cache before logging in to the Oracle ILOM web interface.

Diagnosing SAS Data Path Failures on Servers Using MegaRAID Disk Controllers

On Oracle x86 servers using MegaRAID disk controllers, Serial Attached SCSi (SAS) data path errors can occur.

To triage and isolate a data path problem on the SAS disk controller, disk backplane (DBP), SAS cable, SAS expander, or hard disk drive (HDD), gather and review the events in the disk controller event log. Classify and analyze all failure events reported by the disk controller based on the server SAS topology.

To classify a MegaRAID disk controller event, gather and parse the MegaRAID disk controller event logs using the MegaCLI command:

For example, at the root prompt, type:

root# ./MegaCli64 adpeventlog getevents –f event.log aall Success in AdpEventLog

Important Operating Notes 15

Page 16

Oracle ILOM Deprecation Notice for IPMI 2.0 Management Service Present Behavior

Exit Code: 0x00

Note - Use the existing name of the event log as the name for the disk controller event log. This

produces a MegaRAID controller event log with the given file name event.log.

The following SCSI sense key errors found in the event log in SAS data path failures indicate a SAS data path fault:

B/4B/05 :SERIOUS: DATA OFFSET ERROR

B/4B/03 :SERIOUS: ACK/NAK TIMEOUT

B/47/01 :SERIOUS: DATA PHASE CRC ERROR DETECTED

B/4B/00 :SERIOUS: DATA PHASE ERROR

A communication fault between the disk and the host bus adapter causes these errors. The presence of these errors, even on a single disk, means there is a data path issue. The RAID controller, SAS cables, SAS expander, or disk backplanes might be causing the interruption to the communication in the path between the RAID controller and the disks.

Oracle Service personnel can find more information about the diagnosis and triage of hard disk and SAS data path failures on x86 servers at the My Oracle Support web site: https://

support.oracle.com. Refer to the Knowledge Article Doc ID 2161195.1 If there are multiple,

simultaneous disk problems on an Exadata server, Oracle Service personnel can refer to Knowledge Article Doc ID 1370640.1.

Oracle ILOM Deprecation Notice for IPMI 2.0 Management Service Present Behavior

Present Behavior: IPMI 2.0 Sessions - Enabled (Default). Support for IPMI 2.0 client interfaces.

Future Behavior: The following IPMI Management Service changes will occur in future Oracle ILOM firmware releases after firmware version 3.2.7.

■

First feature change: Oracle ILOM will add a new client interface as an alternative to the IPMI 2.0 Client interface.

■

Second feature change: The default configuration property for IPMI 2.0 Sessions will change from Enabled to Disabled in a future release. Clients relying on IPMI 2.0 will be unable to communicate with Oracle ILOM unless the configuration property for IPMI 2.0 Sessions is manually enabled.

16 Sun Fire X4800 Server Product Notes • July 2017

Page 17

Oracle ILOM Deprecation Notice for Default Self-Signed Certificate

■

Third feature change: Removal of IPMI 2.0 client support. IPMI 2.0 clients will no longer be able to communicate with Oracle ILOM.

For future updates about IPMI Management Service support in Oracle ILOM, refer to the latest firmware release information in the Oracle ILOM Feature Updates and Release Notes for Firmware 3.2.x.

Oracle ILOM Deprecation Notice for Default Self-Signed Certificate

Present Behavior: An earlier version of the default SSL self-signed certificate is provided by Oracle ILOM.

Future Behavior: A newer version of the default SSL self-signed certificate will be provided in a future Oracle ILOM firmware release.

Impact to Customer Configuration: After updating to a future firmware release, users connecting to Oracle ILOM through the web interface will need to accept a newer version of the default SSL self-signed certificate that is provided by Oracle ILOM. Customer provided SSL certificates will not be impacted by this change.

For future updates about the default SSL self-signed certificate that is provided by Oracle ILOM, refer to the latest firmware release information in the Oracle ILOM Feature Updates and Release Notes for Firmware 3.2.x.

Broken Links in Sun Server X4800 Documentation Library

The following table lists the broken links in the Sun Server X4800 documentation library.

Note - The Sun Server X4800 Documentation Library includes a Sun Server X4800 HTML

Documentation Collection. This collection has the same chapters and content as the individual documents in the following table. Many of the broken links listed below are also broken in the corresponding sections of the Sun Server X4800 HTML Documentation Collection.

Location Link Destination Broken Link Correct Link

Sun Fire X4800 Server Service Manual

Sun Fire X4800 Server Product Notes

http://www.oracle.com/ pls/topic/lookup?ctx=8210281&id=g5prodnotes

http://docs.oracle.com/ cd/E19140-01/html/E69641/ index.html

Important Operating Notes 17

Page 18

Broken Links in Sun Server X4800 Documentation Library

Location Link Destination Broken Link Correct Link

Sun Fire X4800 Server product page

http://www.oracle.com/goto/ x4800

http://www.oracle.com/us/ products/servers-storage/ servers/x86/sun-fire-x4800ds-079895.pdf

Oracle Integrated Lights Out Manager (ILOM) 3.0 Supplement for the Sun Fire X4800 Server

Sun Fire X4800 Server Diagnostics Guide

Troubleshooting DIMM Problems in Sun Fire X4800 Server Diagnostics Guide

Sun Disk Management Overview Information about disk

management for numerous servers

http://www.oracle.com/ pls/topic/lookup?ctx=8210284&id=g5lomsupp

http://www.oracle.com/ pls/topic/lookup?ctx=8210796&id=g5diag

http://www.oracle.com/pls/topic/ lookup?ctx=821-0796&id=gjfvc

Multiple links to servers' documentation

http://docs.oracle.com/cd/ E19140-01/html/821-0284/ index.html

http://docs.oracle.com/cd/ E19140-01/html/821-0796/ index.html

Find information about your server in one of the following libraries:

■

http://www.oracle. com/technetwork/ documentation/oraclex86-servers-190077.html

■

http://docs.oracle.com/ cd/E19121-01/index.html

Sun Fire X4800 Server Rack Mounting and Shipping Bracket User's Guide

Sun Fire X4800 Server product page

http://www.oracle.com/goto/ x4800

http://www.oracle.com/us/ products/servers-storage/ servers/x86/sun-fire-x4800ds-079895.pdf

Sun Fire X4800 Server Installation Guide for Linux Operating Systems

Source for syslinux http://www.kernel.org/pub/linux/

utils/boot/syslinux/

Oracle YUM repository setup information

http://www.oracle.com/ technology/tech/linux/htdocs/ yum-repository-setup.htm

http://www.syslinux. org/wiki/index.php? title=Download

https://oracle-base.com/ articles/linux/create-alocal-yum-repository-fororacle-linux-6

Sun Fire X4800 Server Installation Guide for ESX Software

Sun Fire X4800 Server product page

http://www.oracle.com/goto/ x4800

http://www.oracle.com/us/ products/servers-storage/ servers/x86/sun-fire-x4800ds-079895.pdf

Sun Fire X4800 Server Installation Guide

Oracle Integrated Lights Out Manager (ILOM) 3.0 Supplement for the Sun Fire X4800 Server

Oracle Enterprise Opcenter http://www.oracle.com/us/

products/enterprise-manager/ opscenter/index.html

Sun Fire X4800 Server product page

http://www.oracle.com/goto/ x4800

http://www.oracle.com/ technetwork/oem/ops-center/ index.html

http://www.oracle.com/us/ products/servers-storage/ servers/x86/sun-fire-x4800ds-079895.pdf

18 Sun Fire X4800 Server Product Notes • July 2017

Page 19

Supported Software and Firmware

The following topics contain information about software and firmware supported for the Sun Fire X4800 server:

■

“Supported Operating Systems” on page 19

■

“Supported Firmware” on page 20

■

“Additional Software” on page 20

■

“Sun Java Enterprise System” on page 20

■

“Oracle Enterprise Manager Ops Center” on page 21

■

“MegaRAID Storage Manager (MSM)” on page 21

■

“SunVTS Bootable Diagnostics CD-ROM” on page 21

■

“Oracle Integrated Lights Out Manager (Oracle ILOM)” on page 22

■

“Oracle Hardware Management Pack” on page 22

■

“Documentation Errata” on page 22

Supported Operating Systems

To find the latest operating system version supported for the Sun Fire X4800, go to the following sites and search using your server model number:

■

Oracle Solaris -- http://www.oracle.com/webfolder/technetwork/hcl/index.html

■

Oracle Linux -- http://linux.oracle.com/pls/apex/f?p=117:1:3991604960223967

■

Oracle VM -- http://linux.oracle.com/pls/apex/f?p=117:1:3991604960223967

■

Windows -- http://www.windowsservercatalog.com/

■

VMware ESXi -- http://www.vmware.com/resources/compatibility/search.php/

■

Red Hat Enterprise Linux -- https://access.redhat.com/certifications

■

SUSE Linux Enterprise Server -- https://www.suse.com/yessearch/Search.jsp

The following list shows the minimum supported operating systems.

■

Oracle Solaris 10 8/11, 10 9/10, 10 10/09 (64-bit)

Supported Software and Firmware 19

Page 20

Supported Firmware

■

Oracle Solaris 11, 11.1, and 11.2 (64-bit)

■

Oracle Linux 5.5, 5.6, 5.7, 5.9, 5.10, 6.1, 6.4, 6.5 6.6, 7.0, and 7.1 (64-bit)

■

Oracle VM 2.2.1, 2.2.2, 3.0, 3.2.1, 3.3.1, 3.3.2

■

Red Hat Enterprise Linux 5.5, 5.6, 5.7, 6.0, 6.1, 6.0, 7.1

■

SUSE Enterprise Linux (SLES) 11 and SLES11 SP1 (64-bit)

■

Windows 2008 and R2 SP1 (64-bit)

■

VMware ESX/ESXi 4.0 U2, ESX/ESXi 4.1 U1, ESXi 5.0

Supported Firmware

SW 1.11.0 includes the following firmware:

■

Oracle ILOM -- 3.2.10.21 (r118444)

■

BIOS -- 11080300

SW 1.10.0 includes the following firmware:

■

Oracle ILOM -- 3.2.9.25 (r116305)

■

BIOS -- 11080300

Additional Software

Note - To obtain optimal performance, security, and stability, install system software release

1.11.0 or newer. Oracle recommends that you always install the latest available firmware.

The following additional software is available for download:

■

Oracle Hardware Installation Assistant (OHIA) 2.5

■

SunVTS 7.0 Patch Set 12 or later

Sun Java Enterprise System

Sun Java Enterprise System (Java ES) is a set of software components that provide services needed to support enterprise-strength applications that are distributed across a network or Internet environment. The Sun Java ES is preloaded on your server.

20 Sun Fire X4800 Server Product Notes • July 2017

Page 21

Oracle Enterprise Manager Ops Center

Oracle Enterprise Manager Ops Center is a highly scalable, unified management platform for physical and virtual environments. Use Oracle Enterprise Manager Ops Center to manage multiplatform x86 and SPARC systems that are distributed throughout a global data center and to integrate these systems with existing toolsets. Ready to facilitate many aspects of compliance reporting (ITIL) and data center automation, Oracle Enterprise Manager Ops Center enables management of thousands of systems simultaneously.

MegaRAID Storage Manager (MSM)

MegaRAID Storage Manager (MSM) is a configuration application that enables you to configure, monitor, and maintain storage configurations on Integrated RAID controllers. This graphical user interface (GUI) makes it easy for you to create and manage storage configurations. There is also a MegaCLI command-line configuration utility, to manage your RAID controllers. These applications are available on the Tools and Drivers image on the product download site.

Oracle Enterprise Manager Ops Center

MSM enables you to easily configure the controllers, disk drives, and virtual disks on your system. The Configuration wizard greatly simplifies the process of creating disk groups and virtual disks. The Configuration wizard guides you through several simple steps to create your storage configurations.

For more information on software for your integrated RAID controller, see: http://docs.

oracle.com/cd/E19418-01/index.html

SunVTS Bootable Diagnostics CD-ROM

The server is shipped with a bootable diagnostics CD-ROM. This CD-ROM is designed so that the server boots using the Solaris OS on the CD-ROM and starts SunVTS software. Diagnostic tests run and write output to log files that the service technician can use to determine the problem with the server.

For information on SunVTS, see Using SunVTS Diagnostics Software in Sun Fire X4800 Server Diagnostics Guide.

Note - SunVTS is now called Oracle VTS.

Supported Software and Firmware 21

Page 22

Oracle Integrated Lights Out Manager (Oracle ILOM)

Oracle ILOM is system management firmware that allows you to manage your server when the host system is powered down. This is possible because Oracle ILOM runs on a separate Service Processor (SP) that is powered by the host system’s standby power.

The following interfaces provide network access to the Oracle ILOM: command-line interface (CLI), web interface, SNMP, and IPMI.

Oracle ILOM also supports remote access to the host’s system console through a network remote keyboard video and mouse (RKVM). The host’s I/O to optical and floppy drives can be redirected to real and virtual drives on the network. This allows a remote user to perform most maintenance operations, including installing an operating system.

For more information on Oracle ILOM, refer to the Oracle ILOM 3.2 collection at: http://

www.oracle.com/goto/ilom/docs

Oracle Hardware Management Pack

Oracle Hardware Management Pack allows you to monitor hardware through the host operating system either remotely using SNMP, or locally using command-line interface tools.

For more information, refer to the Oracle Hardware Management Pack Documentation Library at http://www.oracle.com/goto/ohmp/docs.

Documentation Errata

This topic provides corrections to the Sun Fire X4800 documentation.

Use Oracle ILOM 3.2 Documentation After Updating to System Software Release 1.7.0 or Newer

When you update to system software release 1.7.0 or newer it updates Oracle ILOM as well. Note the following documentation changes:

22 Sun Fire X4800 Server Product Notes • July 2017

Page 23

Documentation Errata

Note - Oracle recommends you update to system software release 1.11.0 or newer as descibed

in “IMPORTANT - Install Latest OS Updates, Patches, and Firmware” on page 14.

■

The Oracle ILOM 3.0 collection no longer applies to your server. Instead refer to the Oracle ILOM 3.2 collection at http://www.oracle.com/goto/ilom/docs.

■

The information in the Oracle Integrated Lights Out Manager (ILOM) 3.0 Supplement for the Sun Fire X4800 is out of date. Instead, refer to the Oracle ILOM 3.2 collection.

Antistatic Wrist Straps Are Not Included With All CRUs and FRUs

The service and installation documentation might state that antistatic wrist straps are included with Customer Replaceable Units (CRUs) and Field Replaceable Units (FRUs). This is not always true. Some CRUs and FRUs are shipped without antistatic wrist straps.

Previous Versions of the Service Manual Incorrectly State That a FEM Is Only Supported on CMOD 0 (CR 7107085)

In previous versions of the service manual, in the section Removing and Installing a Fabric Expansion Module (CRU), the notes incorrectly state that the Fabric Expansion module (FEM) is supported only in CMOD0 (BL 0).

These notes are incorrect. FEMs are supported on all CMODs.

The incorrect notes appear on pages 95 and 96. They have been fixed in the latest version of the service manual.

Links to Oracle Websites Have Been Updated

Some printed documents have been reissued to update links to Oracle websites. The links in the printed documents might be outdated, but they should be corrected in online documents. These include:

■

Sun Fire X4800 Server Installation Guide

Supported Software and Firmware 23

Page 24

Documentation Errata

■

Sun Fire X4800 Server Rack Mounting and Shipping Bracket User's Guide

Links to downloads have been moved to:

http://www.oracle.com/technetwork/server-storage/sun-x86/downloads/index.html

Links to documentation have been moved to: http://docs.oracle.com/cd/E19140-01/index.

html

Name Correction for Oracle Integrated Lights Out Manager

The Oracle Integrated Lights out Manager (Oracle ILOM) might be referred to as the Sun Integrated Lights Out Manager in some documents and screen displays.

Front Mounting Bracket Illustration from Installation Guide is Incorrect

A figure in How to Install the Rack Mounting Hardware in a Round-Hole Rack in Sun Fire X4800 Server Installation Guide is incorrect. It shows the screws for installing the front

mounting bracket in a threaded rack incorrectly being inserted from the front of the rack.

Instead, the screws should be inserted from the back of the rack.

The following figure shows the front mounting brackets for both square-hole racks (1) and for threaded racks (2). In the Installation Guide and in this figure, (1) is correct and unchanged. However, in the Installation Guide, (2) is incorrect. The following figure shows (2) correctly.

24 Sun Fire X4800 Server Product Notes • July 2017

Page 25

■

This figure is incorrect in 821–0285–10, the printed version of this document.

■

They are shown correctly in 821–0285–11, the online version of this document.

Documentation Errata

Supported Software and Firmware 25

Page 26

26 Sun Fire X4800 Server Product Notes • July 2017

Page 27

Oracle x86 Products Accessibility

This section describes the accessibility features that are part of Oracle x86 hardware, firmware, and related documentation.

Oracle strives to make its products, services, and supporting documentation usable and accessible to the disabled community. To that end, products, services, and documentation include features that make the product accessible to users of assistive technology.

For more information about Oracle's commitment to accessibility, go to:

■

http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc

Hardware Accessibility

Oracle x86 hardware has color-coded labels, component touch points, and status indicators (LEDs) that provide information about the system. These labels, touch points, and indicators can be inaccessible features for sight-impaired users. The product HTML documentation provides context and descriptive text available to assistive technologies to aid in interpreting status and understanding the system. System-level descriptions and status indicator interpretation can be found in the product Service Manual. The documentation also provides diagrams and screenshots that do not rely on color. Within the diagrams, callouts indicate the referenced component information. The callout descriptions are mapped within a table. All images and tables in the documentation include descriptive alternative text.

Another method to obtain information about the system is to use the built-in Oracle Integrated Lights Out Manager (ILOM). Oracle ILOM provides a browser-based interface and a command-line interface that support assistive technologies for real-time viewing of system status, indicator interpretation, and system configuration. For details, see "Oracle ILOM Accessibility."

You can access the accessible HTML documentation for Oracle x86 hardware products at:

■

http://docs.oracle.com/en/servers/

Oracle x86 Products Accessibility 27

Page 28

Oracle ILOM Accessibility

You can use the Oracle Integrated Lights Out Manager (ILOM) browser user interface (BUI) to monitor and manage the server hardware. The Oracle ILOM BUI does not require a special accessibility mode; rather, its accessibility features are always available. The BUI was developed using standard HTML and JavaScript and its features conform to accessibility guidelines.

To navigate a BUI page and select items or enter commands, you can use standard keyboard inputs, such as using the Tab key to go to a selection, or the up and down arrow keys to scroll through the page. You can also make menu selections by using standard keyboard combinations.

For example, using the Oracle ILOM Open Problems BUI page, you can identify faulted memory modules (DIMMs) or processors (CPUs) that would otherwise be identified by a lit LED indicator on the motherboard. Likewise, you can use the Oracle ILOM BUI to monitor the hardware power states that are also indicated by flashing LED indicators on the hardware.

The Oracle ILOM command-line interface (CLI) is an alternative and equivalent way to access the Oracle ILOM BUI features and functionality. Because the operating systems that run on the Oracle server hardware support assistive technologies to read the content of the screen, you can use the CLI as an equivalent means to access the color-based, mouse-based, and other visual-based utilities that are part of the BUI. For example, you can use a keyboard to enter CLI commands to identify faulted hardware components, check system status, and monitor system health.

You can use the Oracle ILOM Remote Console Plus to access both a text-based serial console and a graphics-based video console that enable you to remotely redirect host server system keyboard, video, mouse, and storage devices. Note, however, that the Oracle ILOM Java Remote Console does not support scaling of the video frame within the Java application. You need to use assistive technology to enlarge or reduce the content in the Java Remote Console Plus display.

As an alternative method to using the BIOS Setup Utility to configure BIOS settings, Oracle ILOM provides a set of configurable properties that can help you manage the BIOS configuration parameters on an Oracle x86 server. Using Oracle ILOM, you can:

■

Back up a copy of the BIOS configuration parameters to an XML file using the Oracle ILOM BUI.

■

Edit the XML file using a standard XML editor. The BIOS XML tags correlate directly to the BIOS screen labels.

■

Restore the XML file of the backed up or edited configuration parameters to BIOS.

The BUI and CLI methods for using Oracle ILOM are described in the accessible HTML documentation for Oracel ILOM at:

28 Sun Fire X4800 Server Product Notes • July 2017

Page 29

Oracle Hardware Management Pack Accessibility

■

http://www.oracle.com/goto/ilom/docs

Oracle Hardware Management Pack Accessibility

Oracle Hardware Management Pack software is a set of command-line interface (CLI) tools. Oracle Hardware Management Pack software does not include product-specific accessibility features. Using a keyboard, you can run the CLI tools as text commands from the operating system of a supported Oracle server. All output is text-based.

Additionally, most Oracle Hardware Management Pack tools support command output to a text log file or XML file, which can be used for text-to-speech conversion. Accessible manual pages (man pages) are available that describe the Hardware Management Pack tools on the system on which those tools are installed.

Installation and uninstallation of Oracle Hardware Management Pack can be performed manually, using text commands entered from the CLI. Assistive technology products such as screen readers, digital speech synthesizers, or magnifiers can be used to read the content of the screen.

Refer to the assistive technology product documentation for information about operating system and command-line interface support.

The CLI tools for using the software are described in the accessible HTML documentation for Hardware Management Pack at:

■

http://www.oracle.com/goto/ohmp/docs

BIOS Accessibility

When viewing BIOS output from a terminal using the serial console redirection feature, some terminals do not support function key input. However, BIOS supports the mapping of function keys to Control key sequences when serial redirection is enabled. Descriptions of the function key to Control key sequence mappings are provided in the product documentation, typically within the server Service Manual. You can navigate the BIOS Setup Utility by using either a mouse or keyboard commands.

As an alternative method of configuring BIOS settings using the BIOS Setup Utility screens, Oracle ILOM provides a set of configurable properties that can help you manage the BIOS configuration parameters on an Oracle x86 server. For more information, see "Oracle ILOM Accessibility."

Oracle x86 Products Accessibility 29

Page 30

Documentation Accessibility

BIOS information and its functions are typically documented in the product Service Manual or Installation Guide.

Documentation Accessibility

Documentation for Oracle hardware is provided in HTML and PDF formats. The HTML documents are accessible using standard operating system controls and assistive technology. PDF documents are also provided; however, PDF is not an accessible format. PDF documents are considered support documents because the PDF content is available in accessible HTML format.

Product documentation provides figures, other types of images, and screenshots that do not rely on color for interpretation. Within the figures, callouts indicate the referenced component information. The callouts are mapped within a table to provide text descriptions of the referenced parts of the figures. In addition, alternative text is provided for all tables and images that provides the context of the information and images.

Note that screen readers might not always correctly read the code examples in the documentation. The conventions for writing code require that closing braces should appear on an otherwise empty line. However, some screen readers might not always read a line of text that consists solely of a bracket or brace.

The documentation might contain links to web sites of other companies and organizations that Oracle does not own or control. Oracle neither evaluates nor makes any representations regarding the accessibility of these web sites.

You can access the accessible HTML documentation for Oracle x86 products at:

■

http://docs.oracle.com/en/servers/

30 Sun Fire X4800 Server Product Notes • July 2017

Page 31

Hardware Issues

The following table lists the current hardware issues.

Hardware Issues Workaround

“System Posts CPU Fault and Fails to Boot On System With Six Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards (22536804)” on page 32

“Reset Takes a Long Time and Causes the Server to Power Cycle” on page 32 N/A

“Update SP Firmware After Replacing a CMOD or SP Module (15465497)” on page 33 Yes

“MegaCLI CfgEachDskRaid0 Command Does Not Work Correctly (CR

7121867)” on page 33

“DIMM Mismatch on Odd Numbered Pairs Results in DDR Training Failed on Entire Branch (CR7101624 and CR7111545)” on page 34

“If DIMM Failure Causes Branch Failure, Oracle ILOM Might Not Identify Them as Faulty (CR7099038 and CR7111543)” on page 35

“When Inserting a CMOD, Simultaneously Rotate Handles Until They Touch the Chassis” on page 35

“Video Output of CMM Is Distorted or Missing” on page 35 Yes

“SATA Drives Not Accessible Through NEM1 (CR 7003993)” on page 36 Yes

“Supported Rack Mounting Configuration” on page 36 N/A

“DIMM Population Rules” on page 36 N/A

“Supported Rack Mounting Configuration” on page 36 N/A

“Locking Screws Secure Shelf Rails to Rear Mounting Brackets” on page 39 Yes

“Message About Missing Disks After Failed Boot (CR 6939507)” on page 41 Yes

“System Does Not Power Up After Power Cycling (CR 6950414)” on page 41 Yes

“System Does Not Recognize DIMM Due to SPD Corruption (CR

6946921)” on page 41

Yes

Hardware Issues 31

Page 32

System Posts CPU Fault and Fails to Boot On System With Six Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards (22536804)

A system equipped with six Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards might post CPU faults and fail to boot.

Note - The Sun Fire X4800 supports a maximum of six Sun InfiniBand Dual Port 4x QDR PCIe

Low Profile Host Channel Adapter M2 Cards.

Workaround

1. Access the BIOS Setup Utility.

2. Navigate to RC Settings > QPI and change:

MMIOH Size per IOH from 2Gb to 4Gb (the default is 2Gb).

3. Navigate to RC Settings > Chipset > NorthBridge Configuration and change:

PCIE-MMIO-64 Bits Support to Enabled (the default is Disabled).

4. Save your changes and exit the BIOS Setup Utility.

5. Reboot the server.

Reset Takes a Long Time and Causes the Server to Power Cycle

If you have a pending BIOS upgrade, a routine reset might take longer than expected and might cause your server to power cycle and reboot several times. This is expected behavior as it is necessary to power cycle the server to upgrade the BIOS firmware. If the upgrade includes an FPGA update, it can take more than 30 minutes to complete.

A pending BIOS upgrade exists when both conditions are true:

■

You update the BIOS and service processor firmware using Oracle ILOM.

■

You select the option to Delay BIOS Upgrade.

■

The host is powered on.

If you then reboot the server expecting a routine server reset and instead initiate a (delayed) BIOS upgrade, wait until the upgrade is finished. Do not interrupt the process, as this can result in corrupted firmware and server down time.

32 Sun Fire X4800 Server Product Notes • July 2017

Page 33

Update SP Firmware After Replacing a CMOD or SP Module (15465497)

On older versions of Oracle ILOM, a pending BIOS upgrade does not take place until you power cycle your server.

On newer versions of Oracle ILOM, a pending BIOS upgrade takes place when you reset your system.

Caution - If you reboot the server expecting a routine server reset and instead initiate a

(delayed) BIOS upgrade, wait until the upgrade is finished. Do not interrupt the process as this can result in corrupted firmware and server down time.

Note - Oracle ILOM and BIOS updates are designed to work together. When you have a

pending BIOS upgrade, it is recommended that you install the upgrade by resetting or power cycling your server as soon as possible.

Update SP Firmware After Replacing a CMOD or SP Module (15465497)

When the replacement of either a CMOD or the service processor results in an incompatibilty between the hardware revision of the component and the firmware version of either the SP or the BIOS, the recommendation is to maintain compatibility with the SP firmware. Therefore, update or downgrade the system firmware package to the version compatible with the SP.

MegaCLI CfgEachDskRaid0 Command Does Not Work Correctly (CR 7121867)

The CfgEachDskRaid0 command has two problems when run in the preboot CLI:

■

It assigns newly created logical volumes in apparently random order.

■

It sometimes fails to create all volumes.

These problems only occur when the command is run in the preboot environment. Normally, you only configure a single boot volume in the preboot CLI anyway.

Workaround

This workaround allows you to configure the drives individually using the CfgLdAdd command in the preboot CLI.

Hardware Issues 33

Page 34

DIMM Mismatch on Odd Numbered Pairs Results in DDR Training Failed on Entire Branch (CR7101624 and CR7111545)

1. Run the following command to determine your enclosure number:

-pdlist -a0 | more

Adapter #0

Enclosure Device ID: 252

Slot Number: 0

Note - The typical enclosure number is 252, unless there are more than eight drive slots.

2. Use the CfgLdAdd command to build RAID volumes for each disk. For example:

-CfgLdAdd -r0[252:0] -a0

-CfgLdAdd -r0[252:1] -a0

-CfgLdAdd -r0[252:2] -a0

-CfgLdAdd -r0[252:3] -a0

Note - If your enclosure number is not 252, substitute your enclosure number in these

commands.

DIMM Mismatch on Odd Numbered Pairs Results in DDR Training Failed on Entire Branch (CR7101624 and CR7111545)

If a DIMM mismatch error (size, rank, or speed) occurs on odd-numbered DIMM pairs, it might cause a ddr-training-failed error on the entire branch, thus disabling all DIMMs on that branch.

Furthermore, Oracle ILOM might not mark them as faulty or send a diagnostic message, but only send one ereport on boot:

ereport.memory.mrc.init-no-memory@/sys/blX/pX/mrX

Workaround

Use the Oracle ILOM Fault Manager to view information about active faults (fmadm faulty) to identify the mismatched DIMM pair.

34 Sun Fire X4800 Server Product Notes • July 2017

Page 35

If DIMM Failure Causes Branch Failure, Oracle ILOM Might Not Identify Them as Faulty (CR7099038 and CR7111543)

Then replace the pair with a matched pair. If the fault isn't cleared automatically, you must manually clear it using the appropriate Oracle ILOM fmadm repair command as described in the Oracle ILOM documentation.

If DIMM Failure Causes Branch Failure, Oracle ILOM Might Not Identify Them as Faulty (CR7099038 and CR7111543)

When fault occurs on DIMMs that result in all DIMMs on the same branch being disabled, Oracle ILOM might not mark them as faulty or send diagnostic message, but only send one ereport:

ereport.memory.mrc.init-no-memory@/sys/blX/pY/mrZ

Where X is the blade slot number, Y is the processor, and Z is the DIMM memory branch.

Workaround

Use the Oracle ILOM Fault Manager to view information about active faults (fmadm faulty) to identify the mismatched DIMM pair.

When Inserting a CMOD, Simultaneously Rotate Handles Until They Touch the Chassis

When you insert a CMOD into the chassis, simultaneously rotate both handles in past the locks until they touch the chassis before releasing them. It is important that the handles actually touch the chassis to ensure that the pawl completes the action of seating the CMOD connectors.

Video Output of CMM Is Distorted or Missing

Under certain rare conditions, the video output from the CMM might be distorted, colored incorrectly, or not visible. This occurs when a video monitor is connected to the video port on the multiport cable.

Hardware Issues 35

Page 36

SATA Drives Not Accessible Through NEM1 (CR 7003993)

Workaround

This condition is benign, and if the video display is acceptable, you can ignore it.

1. Connect to the video display using Oracle ILOM.

For instructions for using Oracle ILOM to view the video display, refer to Communicating With the ILOM and the System Console in Sun Fire X4800 Server Installation Guide.

SATA Drives Not Accessible Through NEM1 (CR 7003993)

SATA drives only work with NEM0. If NEM0 is not present, the system is unable to see or access SATA drives.

This is expected behavior. Multipathing is not supported for SATA drives.

Workaround

If your system has SATA drives, make sure that NEM0 is present.

Supported Rack Mounting Configuration

You can install up to eight Sun Fire X4800 servers in a Sun Rack II 1042/1214 as long as your datacenter meets the cooling requirements to support these systems.

DIMM Population Rules

This section provides the rules for adding and replacing DIMMs on a CMOD. It updates and enhances the information in DIMM Population Rules in Sun Fire X4800 Server Service Manual.

For additional information, including how to prepare the system for servicing, how to remove a CMOD, and how to remove or replace DIMMs, see Removing and Installing Components in Sun Fire X4800 Server Service Manual.

Each CMOD has 32 DIMM slots, with 16 allocated to each CPU. CMODs are shipped with configurations of 16 or 32 DIMMs, meaning that each CPU has either 8 or 16 DIMMs.

■

On a CMOD with 16 DIMMs, you can replace DIMMs, and you can add additional DIMMs.

■

On a CMOD with 32 DIMMs, you can replace DIMMs, but because they are fully populated, you cannot add DIMMs.

36 Sun Fire X4800 Server Product Notes • July 2017

Page 37

DIMM Population Rules

The rules for adding and replacing DIMMs are:

■

When you replace DIMMs, you must match parameters.

■

When you add DIMMs, you must match parameters and populate them in the specified order.

Matching Parameters

There are four parameters to consider when replacing DIMMs: size, speed, density, and manufacturer's model number.

■

All of the DIMMs in the CMOD must be the same size, speed, and density.

■

Both of the DIMMs in a pair must be the same size, speed, and density, and have the same manufacturer's model number. The pairs are 0/4, 1/5, 2/6, 3/7, 8/12, 9/13, 10/14, and 11/15.

The following table shows the requirements.

Parameter and Value Match Within CMOD Match Within Pairs

Size — 2 GB, 4 GB, or 8 GB x x

Speed — 1066 MHz JEDEC or DDR3 ECC RDIMMs x x

Density — Single or dual rank x x

Manufacturer's model number x

DIMM Population Order

You can add DIMMs according to the following rules:

■

You cannot add DIMMs to a 32–DIMM system because it is already fully populated.

■

All DIMMs must have their parameters matched, as described above.

■

Only four or eight DIMM upgrades are supported. As a result, all color blocks must be either full or empty.

■

DIMMs must be populated in the correct order.

On a 16–DIMM CMOD, all the blue and white slots should already be populated.

When you add DIMMs, fill the black slots (D1/D5 and D10/14) first, and fill the green slots (D3/D7 and D11/D15) last.

Order Numbers Slot Color Minimum Configuration

1 D0/D4 Blue

These should be filled by default

Hardware Issues 37

Page 38

DIMM Population Rules

The following figure shows the location of the DIMMs on the CMOD.

Order Numbers Slot Color Minimum Configuration

2 D8/D12 Blue

3 D2/D6 White

4 D10/D14 White

5 D1/D5 Black

6 D9/D13 Black

7 D3/D7 Green

8 D11/D15 Green

Fill these first

Fill these last

38 Sun Fire X4800 Server Product Notes • July 2017

Page 39

Locking Screws Secure Shelf Rails to Rear Mounting Brackets

Legend Description

1 CPU 0

2 CPU 1

3 Towards the front of the CMOD

Locking Screws Secure Shelf Rails to Rear Mounting Brackets

A pair of locking screws (provided) secure the shelf rails to the rear mounting brackets. These screws prevent the mounting brackets from being accidentally disengaged, which could cause the server to fall out of the rack.

How to Install the Locking Screws

Before You Begin

Install the server in the rack, as described in Installing the Server in a Rack Using the Universal Rack Mounting Kit in Sun Fire X4800 Server Installation Guide.

From the rear of the server, insert the screw (1) through the rack (2) and the rear mounting bracket (3) so that it protrudes from the mounting bracket just above the flange on the shelf rail (5).

■

For a round hole (tapped) rack, use either an M6 x 12 or a 10–32 x 7/16 screw.

■

For a square hole (untapped) rack (shown below), use an M6 cage nut (4) and an M6 x 16 screw.

The following figure shows the details for a square-hole (untapped) rack.

Hardware Issues 39

Page 40

How to Install the Locking Screws

Note - The shelf rails must be inserted into the mounting bracket before performing this

procedure. They are shown separated in this picture to highlight the alignment.

Figure Legend

1 Screw

2 Cage nut (used for square hole racks only)

3 Rack

4 Rear mounting bracket

5 Shelf rail

Repeat Step 1 for the other side.

40 Sun Fire X4800 Server Product Notes • July 2017

Page 41

Message About Missing Disks After Failed Boot (CR 6939507)

Message About Missing Disks After Failed Boot (CR

6939507)

On rare occasions, after initial power up, the system fails to boot and displays a message:

All of the disks from your previous configuration are gone. If this is an unexpected message, then please power off your system and check your cables to ensure all disks are present. Press any key to continue, or "C" to load the configuration utility.

Workaround

1. Power cycle the system.

The system should boot normally.

2. If the system still does not boot, and the same message appears, contact Oracle customer service.

System Does Not Power Up After Power Cycling (CR

6950414)

Under rare conditions, when the system is power-cycled, it might not power up.

Workaround

Remove and reapply AC power to and from the system.

Either:

■

Switch the AC power Off then switch it back On.

■

Remove all the AC power cords from the power supplies and then plug them back in.

■

Remove the power supplies and then replace them.

System Does Not Recognize DIMM Due to SPD Corruption (CR 6946921)

Under rare conditions, a DIMM can get corrupted. When this happens, the system no longer recognizes that DIMM, and as a result, the system has less memory available to it than what was configured.

Hardware Issues 41

Page 42

How to Identify a Corrupted DIMM

Log on to Oracle ILOM.

Enter the command to display faulted DIMMs.

■ From the web interface, select the Fault Management tab.

■

From the CLI, enter show/SP/faultmgt.

Oracle ILOM displays the faulted DIMMs with the fault class. It might show one of the following:

fault.memory.intel.nex.dimm_ce fault.memory.intel.nex.dimm_ue fault.memory.intel.dimm.mismatched fault.memory.intel.dimm.something else

If the fault class is anything else besides dimm_ce or dimm_ue, contact your Oracle service representative.

42 Sun Fire X4800 Server Product Notes • July 2017

Page 43

Oracle Solaris 10 Operating System Issues

The following table lists the Oracle Solaris OS issues.

Current Solaris OS Issues Workaround

“Oracle Solaris Installation Takes a Long Time On Systems With Four or More Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards” on page 43

“System Panics During Solaris 10 10/09 OS Installation When Solaris Attempts to Boot (CR

6880963)” on page 45

“System Panics or Hangs During Solaris OS Installation (CR 6920274)” on page 46 Yes

“Messages Complain of Failure to Allocate I/O Resources (CR 6984329)” on page 47 Yes

“Patch Required for Solaris FMA” on page 47 Yes

“Solaris MSI Interrupts Are Depleted (CR 6669984 )” on page 48 Yes

“System Might Panic With "unowned mutex" Message (CR 6893274)” on page 49 No

“Hotplugging PCIe Express Modules in Slots 2.0 or 2.1 Might Not Work (CR

6954869)” on page 49

“A System With a Combo GbE/8Gb FC Express Module HBA Might Get a BAD TRAP Panic (CR 6942158)” on page 49

Yes

Oracle Solaris Installation Takes a Long Time On Systems With Four or More Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards

If you install Oracle Solaris on a system with four or more Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards, the installation can take much longer than normal.

Note - The Sun Fire X4800 server supports a maximum of six Sun InfiniBand Dual Port 4x

QDR PCIe Low Profile Host Channel Adapter M2 Cards.

Workaround

Oracle Solaris 10 Operating System Issues 43

Page 44

Solaris 10 Operating System Overview

1. Remove the Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards.

2. Complete the Oracle Solaris installation.

3. Reinstall the Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards

Solaris 10 Operating System Overview

The Solaris 10 Operating System (OS) delivers the security, manageability, and performance that IT professionals need to help increase service levels and decrease costs and risk. It also serves as the foundation for the Sun Java Enterprise System, an integrated, open, standardsbased software system delivered using a new predictable approach for development, testing, and servicing.

If the Solaris OS is preinstalled on your server, it already includes required patches. If you use your own custom OS install environment, you must update your install environment with the following patches, which can be downloaded from https://support.oracle.com.

■

Solaris 10 9/10 requires patch 146025–01 or later. It is recommended that you apply the latest patch cluster.

■

Solaris 10 10/09 is supported for existing installations but deprecated for new installations.

Note - It is highly recommended that you upgrade to Solaris 10 9/10 or install the latest

Solaris 10 patch cluster.

Solaris 10 10/09 requires the following patches:

■

142901–09 or higher. This addresses the following issues:

■

“System Panics or Hangs During Solaris OS Installation (CR

6920274)” on page 46

■

“Patch Required for Solaris FMA” on page 47

■

SunVTS 7.0ps08 or later

■

143958–03 or later if you have a ComboGbE/8Gb FC Express Module

■

144374–01 or later

■

Other patches might be available as well.

44 Sun Fire X4800 Server Product Notes • July 2017

Page 45

System Panics During Solaris 10 10/09 OS Installation When Solaris Attempts to Boot (CR 6880963)

When installing Solaris 10 10/09, the system panics if it has a large memory size (approximately 768 GB or larger)

For example:

panic[cpu0]/thread=fffffffffbc288a0: mutex_enter: bad mutex, lp=18 owner=f00006f7f0000748 thread=fffffffffbc288a0

>> warning! 8-byte aligned %fp = fffffffffbc315b8 fffffffffbc315b8 unix:mutex_panic+6f () >> warning! 8-byte aligned %fp = fffffffffbc31618 fffffffffbc31618 unix:mutex_vector_enter+1f6 () ...

Workaround 1

Upgrade to Solaris 10 9/10 or newer.

Workaround 2

1. Download patch 142901–09 or higher.

Note - If you are doing a PXE install, you might choose to add patch 142091–09 or higher

to your install image. This allows you to complete the installation without reducing the memory size. However, you must still reduce the memory size (step 2) and add the patch to the installed system (step 4) before you can boot it.

2. Reduce the system memory to 512 GB.

We suggest you remove CMODs 3 and 4. You can use other techniques if desired.

For details, refer to Adding, Removing, and Installing a CMOD (CRU) in Sun Fire X4800 Server Service Manual.

3. Install the operating system.

This is not necessary if your system already has an operating system installed.

4. Install patch 142901–09 or higher.

5. Replace the memory.

If you removed the CMODs, for details, refer to Adding, Removing, and Installing a CMOD (CRU) in Sun Fire X4800 Server Service Manual.

Oracle Solaris 10 Operating System Issues 45

Page 46

System Panics or Hangs During Solaris OS Installation (CR 6920274)

System Panics or Hangs During Solaris OS Installation (CR

6920274)

If you try to do a network install using a 32–bit version of the Solaris OS on the Sun Fire X4800 server, it might fail. It might panic or hang during installation.

32–bit versions of the Solaris OS are not supported. When you install the Solaris OS from a PXE server, the netboot environment must be configured to use the 64–bit kernel.

When a 32–bit install environment is used, it displays a message similar to the following:

Workaround

You can fix this by changing the PXE boot menu to include amd64 on the kernel and module lines.

The following displays show examples of the lines before and after this change. The lines have been wrapped to make the display fit the page.

■

Before (incorrect):

kernel /I86PC.Solaris_10-16/multiboot kernel/unix install -B

install media=10.6.78.11:/images/s10u8_08a,console=ttya,install_

config=10.6.78.11:/images/s10u8_08a/jumpstart,sysid_

config=10.6.78.11:/images/s10u8_08a

module /I86PC.Solaris_10-16/x86.miniroot

■

After (correct):

kernel /I86PC.Solaris_10-16/multiboot

kernel/amd64/unix install -B install_

media=10.6.78.11:/images/s10u8_08a,console=ttya,install_

config=10.6.78.11:/images/s10u8_08a/jumpstart,sysid_

config=10.6.78.11:/images/s10u8_08a

module /I86PC.Solaris_10-16/amd64/x86.miniroot

46 Sun Fire X4800 Server Product Notes • July 2017

Page 47

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Messages that complain about failures to allocate I/O resources might appear in POST and in log files.

For example, you might see:

Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.0 Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.1

Workaround

BIOS tries to allocate I/O resources. If it is unable to allocate I/O resources successfully, it displays error messages.

The OS might try as well. However if it tries, it fails and generates additional failure messages because it uses the same method as BIOS. However, most drivers can manage this condition.

1. Usually, you can ignore these messages.

2. If you continue to experience I/O resource issues, see I/O and Interrupt Resource Allocation in the Sun Fire X4800 Server Installation Guide.

Patch Required for Solaris FMA

For Solaris 10 10/09, in order to use Solaris Fault Management Architecture (FMA) with your server, you must install patch 142901–09 after installing the OS.

This patch is also recommended to utilize the maximum number of interrupts on the server.

Workaround 1

Upgrade to Solaris 10 9/10 or newer.

Workaround 2

1. Check to see if you have installed patch 142901–09 or later. If not:

a. Download patch 142901–09 or later from:

https://support.oracle.com

b. Follow the patch instructions to install the patch.

2. Issue the following commands:

Oracle Solaris 10 Operating System Issues 47

Page 48

Solaris MSI Interrupts Are Depleted (CR 6669984 )

rem_drv intel_nhmex

add_drv -i '"pci8086,3438"' intel_nhmex

3. Reboot your server.

Solaris MSI Interrupts Are Depleted (CR 6669984 )

The Solaris OS might run out of MSI interrupts on 8-socket systems.

The Solaris OS is designed to allocate 32 interrupt vectors at priority level six. Also, each network device is assigned two interrupts. If the system contains more than 16 network devices, it will exhaust the supply of interrupts on level six, leaving some devices non-functional.

When this occurs, error messages similar to the following appears:

■

On the console:

Feb 25 15:45:04 mpk12-3214-189-156 pcplusmp: WARNING: No interrupt vector:

pciex8086,10f7 instance 1

Feb 25 15:45:04 mpk12-3214-189-156 pcplusmp: WARNING: Sharing vectors:

pciex8086,10f7 instance 1 and SCI

Feb 25 15:45:06 mpk12-3214-189-156 pcplusmp: WARNING: No interrupt vector:

pciex8086,10f7 instance 5

Feb 25 15:45:06 mpk12-3214-189-156 pcplusmp: WARNING: Sharing vectors:

pciex8086,10f7 instance 1 and pciex8086,10f7 instance 5

■

In /var/adm/messages:

Feb 25 15:44:53 mpk12-3214-189-156 ixgbe: [ID 611667 kern.info]

NOTICE: ixgbe7: Insufficient interrupt handles available: 1

Feb 25 15:44:53 mpk12-3214-189-156 ixgbe: [ID 611667 kern.info]

NOTICE: ixgbe7: Allocate MSI-X failed, trying MSI interrupts...

Feb 25 15:44:53 mpk12-3214-189-156 ixgbe: [ID 611667 kern.info]

NOTICE: ixgbe7: MSI-X not used, force rx and tx queue number to 1

Note - If you have not installed patch 142901–09 or later, the Solaris OS is currently limited to

allocating 31 vectors at priority level six, meaning that it can only support 15 network devices on level six.

Workaround 1

For Solaris 10 9/10, install patch 146025–03 or higher.

This fixes the problem in most cases.

48 Sun Fire X4800 Server Product Notes • July 2017

Page 49

System Might Panic With "unowned mutex" Message (CR 6893274)

Note - This patch requires Solaris 10 9/10 and cannot be used with Solaris 10 10/09.

Workaround 2

When you encounter a shortage of I/O interrupts on level 6, Oracle recommends that you assign one of the drivers to interrupt level 5 or 4.

■

Interrupt level 5 is the first priority

■

Interrupt level 4 is next

See Allocating MSI Interrupt Space (Oracle Solaris OS Only) in the Sun Fire X4800 Server Installation Guide for corrective actions.

System Might Panic With "unowned mutex" Message (CR

6893274)

Under rare conditions, the system might panic and display the message:

turnstile_block: unowned mutex

This is a known Solaris OS issue.

Hotplugging PCIe Express Modules in Slots 2.0 or 2.1 Might Not Work (CR 6954869)

On an 8–socket system, hotplugging PCIe express modules in slots 2.0 or 2.1 might not work.

This is due to a possible shortage of hotplug interrupts on the system.

Workaround

Insert these modules before booting the system.

A System With a Combo GbE/8Gb FC Express Module HBA Might Get a BAD TRAP Panic (CR 6942158)

Due to a known issue in the Solaris OS, a system with a Combo GbE/8Gb FC Express Module HBA might panic during boot with a BAD TRAP message.

Oracle Solaris 10 Operating System Issues 49

Page 50

A System With a Combo GbE/8Gb FC Express Module HBA Might Get a BAD TRAP Panic (CR 6942158)

This issue applies to Solaris 10 10/09. It is fixed in Solaris 10 9/10.

Workaround

Install patch number 143858–03 or newer, or install Solaris 10 9/10.

50 Sun Fire X4800 Server Product Notes • July 2017

Page 51

Linux Issues

The following table lists the issues with related to the Linux operating system on the Sun Fire X4800 Server server.

Linux Issues Workaround

“Oracle and Red Hat Linux 7.2 Installation Fails With AER Error on Systems With Four or More Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards (22686146)” on page 52

“False MCE Errors Appear in /var/log/mcelog (CR 7104293)” on page 52

“I/O Does Not Work On SLES11 SP1 with XEN (CR 6965290 and CR7110443)” on page 53

“RHEL6 Kdump Runs Out of Memory (CR 7000993 and CR 7000942)” on page 53 Yes

“SLES11 SP1 Does Not Produce A Kdump File After A Crash (CR7001706)” on page 54 Yes

“SLES11 Fails to Install On Systems Equipped with 6–Core CMODs (CR

7024769)” on page 54

“Messages Complain of Failure to Allocate I/O Resources (CR 6984329)” on page 55 Yes

“Xen Profiler Is Not Supported in Oracle Linux 5.5 (CR 6839366)” on page 55 No

“Oracle Linux 5.5 Does Not Support CPU Throttling (CR 6847286)” on page 55 No

“InfiniBand PCIe Module Does Not Function Correctly With SLES 11 (CR 6909512, CR

6921163)” on page 56

“LEDs on PCIe ExpressModule Work Incorrectly with Oracle Linux 5.5 (CR

6894954)” on page 56

“Oracle Linux 5.5 Xen Kernel Hangs Due to Long Timeout (CR 6899040)” on page 57 Yes

“10GbE PCIe Card in 8-Socket System with SLES11 Can Cause Kernel to Crash (CR

6912821)” on page 58

Yes

N/A

Yes

Yes — Fixed in SP1

Yes

Linux Issues 51

Page 52

Oracle and Red Hat Linux 7.2 Installation Fails With AER Error on Systems With Four or More Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards (22686146)

Oracle Linux 7.2 or Red Hat Linux 7.2 installation might fail with an AER error on a system with four or more Sun InfiniBand Dual Port 4x QDR PCIe Low Profile Host Channel Adapter M2 Cards.

Note - The Sun Fire X4800 server supports a maximum of six Sun InfiniBand Dual Port 4x

QDR PCIe Low Profile Host Channel Adapter M2 Cards.

Workaround

1. Access the BIOS Setup Utility.

2. Navigate to RC Settings > QPI and change:

MMIOH Size per IOH from 2Gb to 4Gb (the default is 2Gb).

3. Navigate to RC Settings > Chipset > NorthBridge Configuration and change:

PCIE-MMIO-64 Bits Support to Enabled (the default is Disabled).

4. Save your changes and exit the BIOS Setup Utility.

5. Finish the operating system installation.

False MCE Errors Appear in /var/log/mcelog (CR 7104293)

Occasionally, many errors appear in the MCE log, /var/log/mcelog. For example:

CPU 0 BANK 9 MCG status: MCi status: Error enabled MCA: MEMORY CONTROLLER GEN_CHANNELunspecified_ERR Transaction: Generic undefined request STATUS 900000400009008f MCGSTATUS 0

■

If this is a one-time or an occasional event, you can ignore it. It should not impact performance.

■

If the errors persist, contact Oracle Services for further review. Open a support case with Oracle Services and reference (Doc ID #1459158.1) for further analysis and evaluation.

52 Sun Fire X4800 Server Product Notes • July 2017

Page 53

I/O Does Not Work On SLES11 SP1 with XEN (CR 6965290 and CR7110443)

If the number of I/O devices exceeds XEN's number of interrupts, the I/O devices might not work, and it might display corresponding error messages. For example:

Unable to allocate IRQ or map irq failed

Workaround

Add the following to the /boot/grub/menu.lst file:

extra_guest_irqs=64,2048 nr_irqs=2048

RHEL6 Kdump Runs Out of Memory (CR 7000993 and CR

7000942)

If Red Hat Linux 6 (RHEL6) calculates that the kdump size is too small, it displays a message:

"Your running kernel is using more than 70% of the amount of space you reserved for kdump, you should consider increasing your crashkernel reservation"

One of two conditions might be present:

■

It might have enough memory, but post the messages anyway.

■

It might really not have enough kdump memory space. In this case, if the system crashes, it might not produce a kdump file, and it might post messages similar to the following on the console:

Out of memory: kill process 322 (insmod) score 21 or a child Killed process 322 (insmod) vsr:1352kB, anon-res: 244kB, file res: 122kB

Workaround

1. To make the messages stop and to ensure your system has enough kdump memory space, change the following line in boot/grub/menu.lst:

Change:

crashkernel=auto

To:

Linux Issues 53

Page 54

SLES11 SP1 Does Not Produce A Kdump File After A Crash (CR7001706)

crashkernel=512M

Note - This step stops your server from displaying warning messages. If your server does

not have enough kdump memory space, complete step 2 as well.

Add pci=nomsi to the KDUMP_COMMANDLINE_APPEND line in /etc/sysconfig/kdump. For example, change:

KDUMP_COMMANDLINE_APPEND="irqpoll maxcpus=1 reset_devices cgroup_disable=memory"

To:

KDUMP_COMMANDLINE_APPEND="irqpoll maxcpus=1 pci=nomsi reset_devices cgroup_disable=memory"

SLES11 SP1 Does Not Produce A Kdump File After A Crash (CR7001706)

After a crash, SUSE Linux Enterprise Server 11 (SLES11) SP1 might not produce a kdump file. In this case, it might also display a message similar to the following on the console:

INFO: Cannot find debug information: Unable to find debuginfo file. [ 41.368017] Restarting system.

Workaround

Perform a regular update from Novell after installation.

SLES11 Fails to Install On Systems Equipped with 6–Core CMODs (CR 7024769)

Servers equipped with 6-socket CMODs (Model E7540) might fail to install or boot with SUSE Linux Enterprise Server 11 (SLES11).

Workaround

Install SLES11 SP1 or newer.

54 Sun Fire X4800 Server Product Notes • July 2017

Page 55

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Messages that complain about failures to allocate I/O resources might appear in POST and in log files.

For example, you might see:

Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.0 Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.1

Workaround

BIOS tries to allocate I/O resources. If it is unable to allocate I/O resources successfully, it displays error messages.

The OS might try as well. However if it tries, it fails and generates additional failure messages because it uses the same method as BIOS. However, most drivers can manage this condition.

1. Usually, you can ignore these messages.

2. If you continue to experience I/O resource issues, see I/O and Interrupt Resource Allocation in the Sun Fire X4800 Server Installation Guide.

Xen Profiler Is Not Supported in Oracle Linux 5.5 (CR

6839366)

The use of the Xen Profiler is not supported in Oracle Linux 5.5.

Oracle Linux 5.5 Does Not Support CPU Throttling (CR

6847286)

CPU throttling (T-state) is not supported in Oracle Linux 5.5.

Linux Issues 55

Page 56

InfiniBand PCIe Module Does Not Function Correctly With SLES 11 (CR 6909512, CR 6921163)

With SUSE Linux Enterprise Server 11 (SLES 11), if you use nox2apic, the InfiniBand PCIe ExpressModule does not work with SLES 11.

Workaround

Install SLES11 SP1.

Bug in Kernel Causes Incorrect MSI Processing (CR

6921163)

If x2apic is enabled (if VT–d is enabled in BIOS), a bug in the kernel allows only a limited number of interrupts to be used.

Workaround

Perform one or the other:

■

Update to the errata kernel from Novell.

■

Put nox2apic in the boot command line in /boot/grub/menu.1st append line.

LEDs on PCIe ExpressModule Work Incorrectly with Oracle Linux 5.5 (CR 6894954)

The lights on the PCIe ExpressModule do not work normally on systems with Oracle Linux 5.5.

When you insert the PCIe ExpressModule and press the attention button:

■

The LED should blink five times for one second each time and then to stay on.

■

Instead, it stays dark for about five seconds and then stays on.

When you press the attention button to remove the PCIe ExpressModule:

■

The LED should blink for five seconds and then turn off.

56 Sun Fire X4800 Server Product Notes • July 2017

Page 57

Oracle Linux 5.5 Xen Kernel Hangs Due to Long Timeout (CR 6899040)

■

Instead, it stays off for two seconds, blinks for three seconds, and then turns off.

Oracle Linux 5.5 Xen Kernel Hangs Due to Long Timeout (CR 6899040)

When booting from a PCIe ExpressModule or a Fabric Expansion Module (FEM), the system might hang because it runs out of MSI interrupt vectors.

Workaround

In the GRUB configuration, set pci=nomsi.

2. Boot a non-Xen kernel.

Edit the file /boot/grub/menu.lst.

4. Add the following text to the Xen kernel init line stanza:

pci=nomsi

For example:

# grub.conf generated by anaconda

# Note that you do not have to rerun grub after making changes to this file

# NOTICE: You have a /boot partition. This means that

# all kernel and initrd paths are relative to /boot/, eg.

# root (hd0,0)

# kernel /vmlinuz-version ro root=/dev/sda3

# initrd /initrd-version.img

#boot=/dev/sda

default=1

timeout=5

serial --unit=0 --speed=115200

terminal --timeout=5 serial console

title Enterprise Linux Server (2.6.18-194.el5xen)

root (hd0,0)

kernel /xen.gz-2.6.18-194.el5 com1=115200 console=com1

module /vmlinuz-2.6.18-194.el5xen ro root=LABEL=/

console=ttyS0,115200 debug pci=nomsi

module /initrd-2.6.18-194.el5xen.img

title Enterprise Linux Server-base (2.6.18-194.el5)

root (hd0,0)

kernel /vmlinuz-2.6.18-194.el5 ro root=LABEL=/

Linux Issues 57

Page 58

Oracle Linux 5.5 Issues Xen Warning: Domain attempted WRMSR 0000000000000277 (CR 6902715)

earlylprintk=ttyS0,115200 console=ttyS0,115200 debug

initrd /initrd-2.6.18-194.el5.img

5. Reboot the Xen kernel.

Oracle Linux 5.5 Issues Xen Warning: Domain attempted WRMSR 0000000000000277 (CR 6902715)

This warning can be ignored.

10GbE PCIe Card in 8-Socket System with SLES11 Can Cause Kernel to Crash (CR 6912821)

The kernel might crash in an 8-socket system with SUSE Linux Enterprise Server 11 (SLES11) and a Sun Dual 10GbE I2 SFP+ PCIe 2.0. PCIe ExpressModule.

Workaround 1

Add the following entry to the /etc/modprobes.conf file:

options ixgbe InterruptType=1,1,1,1,1,1,1,1

It might be necessary to temporarily remove the Sun Dual 10GbE I2 SFP+ PCIe 2.0. PCIe ExpressModule to prevent the kernel from crashing long enough for you to modify this file.

Workaround 2

Install drivers from Novell.

1. Navigate to the following page:

http://drivers.suse.com/driver-process/staging/pub/update/intel/sle11/ common/x86_64/

2. Download and install the following drivers:

intel-ixgbe-2.0.75.7-1.x86_64.rpm

intel-ixgbe-kmp-default-2.0.75.7_2.6.27.19_5-1.x86_64.rpm

intel-ixgbe-kmp-xen-2.0.75.7_2.6.27.19_5-1.x86_64.rpm

58 Sun Fire X4800 Server Product Notes • July 2017

Page 59

SLES11 with Xen Hangs When Many PCIe ExpressModule Cards Are Installed (CR 6917397)

SUSE Linux Enterprise Server 11 (SLES11) hangs with udev in dom0 when the number of interrupt resources required by I/O devices exceeds a threshold. This is most likely in systems where many PCIe ExpressModule cards are installed.

Workaround

In the GRUB configuration, set pci=nomsi.

2. Boot a non-Xen kernel.

Edit /boot/grub/menu.lst.

4. Add the following text to the Xen kernel init line stanza:

pci=nomsi

For example:

title Xen -- SUSE Linux Enterprise Server 11 - 2.6.27.19-5

root (hd0,1)

kernel /boot/xen.gz module /boot/vmlinuz-2.6.27.19-5-xen nn.nn.n ... other text, parameters, etc pci=nomsi

module /boot/initrd-2.6.27.19-5-xen

Use YAST to add pci=nomsi as a boot parameter.

5. Reboot the Xen kernel.

Linux Issues 59

Page 60

60 Sun Fire X4800 Server Product Notes • July 2017

Page 61

Oracle VM Issues

The following table lists the Oracle VM issues.

Oracle VM Issues Workaround

“Messages Complain of Failure to Allocate I/O Resources (CR

6984329)” on page 61

“Earlier Versions of Oracle VM Only Support 64 Logical CPUs (CR

6915014)” on page 62

“Oracle VM Server Installation Parameters Required on Sun Fire X4800 (CR

7094126)” on page 62

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Messages that complain about failures to allocate I/O resources might appear in POST and in log files.

Yes

For example, you might see:

Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.0 Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.1

Workaround

BIOS tries to allocate I/O resources. If it is unable to allocate I/O resources successfully, it displays error messages.

The OS might try as well. However if it tries, it fails and generates additional failure messages because it uses the same method as BIOS. However, most drivers can manage this condition.

1. Usually, you can ignore these messages.

Oracle VM Issues 61

Page 62

Earlier Versions of Oracle VM Only Support 64 Logical CPUs (CR 6915014)

2. If you continue to experience I/O resource issues, see I/O and Interrupt Resource Allocation in Sun Fire X4800 Server Installation Guide.

Earlier Versions of Oracle VM Only Support 64 Logical CPUs (CR 6915014)

Earlier versions of Oracle VM (2.2.x) support 64 logical CPUs. An 8–socket system with hyperthreading enabled has 128 logical CPUs.

Workaround

Disable hyperthreading in the BIOS.

Oracle VM Server Installation Parameters Required on Sun Fire X4800 (CR 7094126)

To allow the megaraid_sas driver to load correctly you must add parameters to Oracle VM:

■

When installing Oracle VM 2.x, add the following kernel parameters:

mboot.c32 xen.gz extra_guest_irqs=64,2048 nr_irqs=2048 --- vmlinuz --- initrd.img

■

If booting from the installation media, press F2 when the initial boot screen is displayed and add the above parameters to the boot command.

■

If using a kickstart installation, add the above parameters to the PXE configuration file.

■

For all Oracle VM installations, edit the /boot/grub/grub.conf file in your Oracle VM Server and add: extra_guest_irqs=64,2048 nr_irqs=2048

62 Sun Fire X4800 Server Product Notes • July 2017

Page 63

Windows Operating System Issues

The following table lists the Windows OS issues.

Windows Issues Workaround

“Unspecified CPU Fault After Warm Reset (CR 7054657)” on page 63 Yes

“Hot Plugging of Some PCIe ExpressModules Not Supported by Windows Server 2008 R2 (CR 6914898, CR 6793369, CR 6923132)” on page 64

“PXE Installation of Windows Server 2008 R2 Fails at Authentication (CR

6899582)” on page 64

“Messages Complain of Failure to Allocate I/O Resources (CR

6984329)” on page 65

“Windows 2008 Cannot Be Installed to Combo GbE/8Gb FC ExpressModule HBA Disks (CR 6984746)” on page 66

“Hot Plugging of PCIe ExpressModules Is Not Supported by Windows 2008” on page 66

Unspecified CPU Fault After Warm Reset (CR 7054657)

Yes

When the server undergoes a reset because of a CPU accessing memory from an uncorrectable DIMM, a reboot or warm reset from the server might result in an unspecified CPU fault. This has been observed with Windows 2008 R2 SP1 and Windows 2008 SP2.

The sequence of events is:

■

CPU accesses memory from an uncorrectable ECC DIMM.

■

As a result, the CPU initiates a platform reset.

■

The next reset results in a valid DIMM fault.

■

When the DIMMs are not replaced and the system continues to operate with the fault, a subsequent reboot or warm reset results in processors faults. However, these are false processor faults

Workaround

Windows Operating System Issues 63

Page 64

Hot Plugging of Some PCIe ExpressModules Not Supported by Windows Server 2008 R2 (CR 6914898, CR 6793369, CR

6923132)

1. Ignore these false processor faults when they're associated with a uncorrectable DIMM reset.

2. Handle the DIMM errors like you would any other DIMM error, and clear the DIMM faults.

3. Clear the processor faults.

4. Power-cycle the server.

For example, in the Oracle ILOM Remote Power Control tab, select Power Cycle or Immediate Power Off.

Note - Resetting the server without toggling the power (for example, by issuing a reset

command) does not clear the CPU error.

Hot Plugging of Some PCIe ExpressModules Not Supported by Windows Server 2008 R2 (CR 6914898, CR 6793369, CR

6923132)

Windows Server 2008 R2 does not support hot plugging on certain PCIe ExpressModules. These include:

■

SG-XPCIE2FCGBE-Q-Z - Sun StorageTek Dual FC Dual GbE HBA Express Module

■

SG-XPCIE2FCGBE-E-Z - Sun StorageTek Dual FC Dual GbE HBA Express Module

■

SG-XPCIEFCGBE-Q8-Z - Sun StorageTek Dual 8GB FC Dual GbE HBA Express Module

■

SG-XPCIEFCGBE-E8-Z - Sun StorageTek Dual 8GB FC Dual GbE HBA Express Module

■

X7284A-Z - Sun Quad GbE ExpressModule

Workaround

Reboot the system after changing one of these PCIe ExpressModules.

PXE Installation of Windows Server 2008 R2 Fails at Authentication (CR 6899582)

During a PXE installation of Windows Server 2008 R2, if PXE user authentication fails, check that the system date and time are set correctly. If the systems date and time is incorrect, PXE authentication fails, and the network installation of Windows Server 2008 R2 does not happen.

64 Sun Fire X4800 Server Product Notes • July 2017

Page 65

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Note - This failure occurs when the date is approximately seven days different from the actual

date.

Workaround

Use the system BIOS to set the system date and time.

1. Reboot or power on the server.

POST messages appear, and prompt you to press the F2 key.

2. Press the F2 key.

The BIOS Setup Utility main menu appears.

■

Use the left and right arrow keys or the Tab key to navigate through the top level of the utility.

■

Use the up and down arrows to navigate through the submenus.

■

Use the Help information for instructions to change options, save settings, and exit the utility.

3. Find the system date and time and set it.

4. Save your work and exit.

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Messages that complain about failures to allocate I/O resources might appear in POST, and in log files.

For example, you might see:

Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.0 Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.1

Workaround

BIOS tries to allocate I/O resources. If it is unable to allocate I/O resources successfully, it displays error messages.

The OS might try as well. However if it tries, it fails and generates additional failure messages because it uses the same method as BIOS. However, most drivers can manage this condition.

1. Usually, you can ignore these messages.

Windows Operating System Issues 65

Page 66

Windows 2008 Cannot Be Installed to Combo GbE/8Gb FC ExpressModule HBA Disks (CR 6984746)

2. If you continue to experience I/O resource issues, see I/O and Interrupt Resource Allocation in Sun Fire X4800 Server Installation Guide.

Windows 2008 Cannot Be Installed to Combo GbE/8Gb FC ExpressModule HBA Disks (CR 6984746)

If you attempt to install Windows 2008 on a Combo GbE/8Gb FC ExpressModule HBA Disk, the driver loads but the OS cannot see the disks.

Workaround

During the installation, load the drivers from the USB drive twice.

Loading the drivers twice is necessary because there are two devices on the Combo GbE/8Gb FC ExpressModule HBA.

Hot Plugging of PCIe ExpressModules Is Not Supported by Windows 2008

With Windows 2008 SP2 and 2008 R2 the following PCIe ExpressModules cannot be hotplugged:

■

Fibre Channel - 4 Gigabit/Sec PCI-X Dual FC Host Adapter (Emulex)

■

Sun Quad Gigabit Ethernet Module (X7284A-Z)

The affected Windows platforms include:

■

Windows Server 2008 Service Pack 2

■

Windows Server 2008 R2

66 Sun Fire X4800 Server Product Notes • July 2017

Page 67

ESX Issues

The following table lists the ESX VMware issues.

ESX Issues Workaround

“Messages Complain of Failure to Allocate I/O Resources (CR

6984329)” on page 67

“Option Cards Can Fail to Load Device Drivers in Some Configurations (CR

6933436)” on page 68

“Warning Messages Appear During Bootup and in Log Files (CRs 6862766, 6861903, and 6890263)” on page 69

“VMware ESXi 5.5 Does Not Support MMIO Regions Above 4GB (16480679,

17013064)” on page 69

“VMware ESXi 5.5 Runs Out of Interrupts With PCIe Cards (16494653)” on page 70

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Yes

N/A

Yes

Messages that complain about failures to allocate I/O resources might appear in POST, and in log files.

For example, you might see:

Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.0 Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.1

Workaround

BIOS tries to allocate I/O resources. If it is unable to allocate I/O resources successfully, it displays error messages.

ESX Issues 67

Page 68

Option Cards Can Fail to Load Device Drivers in Some Configurations (CR 6933436)

The OS might try as well. However if it tries, it fails and generates additional failure messages because it uses the same method as BIOS. However, most drivers can manage this condition.

1. Usually, you can ignore these messages.

2. If you continue to experience I/O resource issues, see I/O and Interrupt Resource Allocation in Sun Fire X4800 Server Installation Guide.

Option Cards Can Fail to Load Device Drivers in Some Configurations (CR 6933436)

Some combinations of option cards can result in failure to load device drivers and subsequent failure to mount file systems or configure network interfaces, and possibly failure to boot the system. This occurs in ESX 4x only.

The ESX host requires "Low Memory," or the memory addressed below 4 GB for the Service Console, device drivers and other data structures. The BIOS utilizes 2 GB of this memory area for its own uses. With the large number of supported I/O devices, this memory can become exhausted in some situations.

VMware describes this problem in their Knowledge Base at https://vmware.com.

Workaround

Use the following procedure to reduce the amount of memory reserved by the Service Console.

1. Restart the ESX host.

2. When the boot menu appears, ensure that the default boot option is highlighted.

3. Press e to edit the default boot option.

A new list appears.

Use the arrow keys to highlight the line that starts with kernel /vmlinuz.

5. Press the e key to edit the line in the boot configuration.

6. Use the arrow key to move over the portion of the line which has a parameter similar to the following:

mem=800M

7. Change the value to 600 megabytes.

For example:

mem=600M

8. Press Enter to accept the changes.

9. Press the b key to boot the host using the new configuration.

68 Sun Fire X4800 Server Product Notes • July 2017

Page 69

Warning Messages Appear During Bootup and in Log Files (CRs 6862766, 6861903, and 6890263)

The configuration is complete but it is not yet permanent.

10. After the host has booted, open the vSphere Client and connect to the host.

11. Select the Configuration tab.

12. Change the memory properties of the ESX host to permanently reduce the RAM assignment.

Warning Messages Appear During Bootup and in Log Files (CRs 6862766, 6861903, and 6890263)

Various warning messages appear during bootup, and in log files such as /var/log/ vmkwarning. These can be ignored, as they have no impact on performance.

Sample messages include:

Sep 24 02:05:07 mpk12-3214-189-114 vmkernel: 0:00:00:24.984 cpu0:4096: WARNING PCI: 1861: No such device Sep 24 02:05:07 mpk12-3214-189-114 last message repeated 9 times Sep 24 02:05:07 mpk12-3214-189-114 vmkernel: 0:00:01:03.280 cpu49:4166)WARNING: vmklinux26: dma_alloc_coherent: Out of memory Sep 24 04:13:21 mpk12-3214-189-114 vmkernel: 0:00:00:38.761 cpu0:4096)WARNING: PCI: 1861: No such device Sep 24 04:13:21 mpk12-3214-189-114 last message repeated 9 times Sep 24 04:13:22 mpk12-3214-189-114 vmkernel: 0:00:01:19.828 cpu16:4166)WARNING: vmklinux26: dma_alloc_coherent: Out of memory

VMware ESXi 5.5 Does Not Support MMIO Regions Above 4GB (16480679, 17013064)

The Sun Server X4800 defaults in BIOS to 64-bit MMIO (Memory Mapped I/O). This allows additional PCIe memory address space to be mapped above the standard 32-bit 4GB of space for PCIe cards that include option ROMs. However, VMware ESXi is incompatible with MMIO space above the standard 4GB. This issue can cause some PCIe cards not to function properly with ESXi.

Workaround

As a possible workaround, disable 64-bit MMIO through the server's BIOS Setup utility (under the IO > PCIe Subsystem Settings > PCI 64-bit Resources Allocation menu). This workaround has limitations. With some combinations of option cards, the system will require more MMIO

ESX Issues 69

Page 70

VMware ESXi 5.5 Runs Out of Interrupts With PCIe Cards (16494653)

space than what the system can allocate within 32 bits of address space. When that occurs, those option cards that could not be assigned MMIO address space (because there was not enough left) are unavailable for use.

VMware ESXi 5.5 Runs Out of Interrupts With PCIe Cards (16494653)

In certain configurations VMware ESXi can run out of interrupts for devices (this can include storage and networking).

For more information, refer to VMware's Configuration Maximums document for ESXi 5.5, under host maximums:

http://www.vmware.com/pdf/vsphere5/r55/vsphere-55-configuration-maximums.pdf

70 Sun Fire X4800 Server Product Notes • July 2017

Page 71

Oracle ILOM Enhancements and Issues

The following table lists the Oracle ILOM issues.

Note - Because of Java security changes, with the previous Oracle ILOM firmware if you used

Java 7 update 51 or newer, you could not launch Oracle ILOM Remote Console. With the new Oracle ILOM firmware if use Java 7 update 51 or newer, you can launch Oracle ILOM Remote Console.

Oracle ILOM Issues Workaround

“File Transfer Using URI Fails if Target Password Contains Certain Special Characters (25917655)” on page 72

“Oracle ILOM SNMP v3 Traps Are Not Delivered After SNMP Engine ID Change (23634048)” on page 72

“Third-Party Web Scan and Test Tools Cause Sluggish Oracle ILOM Performance (23564626)” on page 73

“Oracle ILOM Enhancement Allows IPv4 Only, IPv6 Only, or Dual Stack” on page 74

“New Procedures for Updating CPLD (CR 7043418)” on page 75 N/A

“Update Oracle ILOM and BIOS Firmware Before Updating Other Device Firmware (CR 6537282)” on page 75

“Use the Locate Button to Prove Physical Presence (CR 6881237)” on page 76 N/A

“NEM Expander Firmware Update Procedure (CR 6979140)” on page 76 Yes

“Network Management Port 1 Does Not Work (CR 6935471 and

6854694)” on page 81

“start /SYS and stop /SYS Commands Cause Power Button Pressed Event In Log (CR 6906176)” on page 81

“Event Log Messages Might Be Out of Sequence and Duplicated (CR

6915976)” on page 81

“Incorrect Error Message in Event Log After Restore When Serial Console or JavaRconsole Session Are Open (CR 6917474)” on page 82

“Allocated Power Figures Are Incomplete (CR 6931837)” on page 82 No

Fixed in SW 1.1

“(Fixed in SW 1.1) LDAP Account Cannot Be Used to Start Console (CR

6969473)” on page 83

Fixed in System Software Release 1.11.0

Yes

N/A

Yes

Oracle ILOM Enhancements and Issues 71

Page 72

File Transfer Using URI Fails if Target Password Contains Certain Special Characters (25917655)

Oracle ILOM Issues Workaround

“(Fixed in SW 1.1) Power Cycling the Host Using the Web Interface Generates an Error (CR 6909374)” on page 83

“(Fixed in SW 1.1) DIMM Mismatch Fault Not Cleared After DIMM Replacement and Host Power Cycle (CR 6972285)” on page 83

Fixed in SW 1.0.1

“(Fixed in SW 1.0.1) Console Redirect from CLI Fails (CR 6939479)” on page 84 Yes

“(Fixed in SW 1.0.1) Oracle ILOM Only Allows One Console Session (CR

6895423)” on page 84

“(Fixed in SW 1.0.1) Faults Disappear After SP Reset (CR 6928998)” on page 85 Yes

“(Fixed in SW 1.0.1) Clearing a Power Supply Unit (PSU) Fault Erases Other Fault Information (CR 6933348 and CR 6939306)” on page 85

“(Fixed in SW 1.0.1) pciflash Is Not Available (CR 6867060 and CR

6904922)” on page 85

Yes

File Transfer Using URI Fails if Target Password Contains Certain Special Characters (25917655)

This problem is fixed in System Software Release 1.11.0.

When using Oracle ILOM to transfer files using a Uniform Resource Identifier (URI), the transfer fails if the target host's password contains any of the following special characters:

# ; ?

Examples of these transfers include using host storage redirection, and backing up and restoring BIOS and SP configurations.

Workaround

Use a target host password that does not include any of the indicated special characters.

Oracle ILOM SNMP v3 Traps Are Not Delivered After SNMP Engine ID Change (23634048)

If you change the engine ID, create an SNMP v3 user, and configure an alert using that user without waiting approximately 10 seconds between each action, the internal user configuration might be incorrect and traps are missed.

72 Sun Fire X4800 Server Product Notes • July 2017

Page 73

Third-Party Web Scan and Test Tools Cause Sluggish Oracle ILOM Performance (23564626)

Workaround

Do not create multiple configuration changes without verifying the effect of each configuration change. To prevent misconfigured users and missed traps, insert sleep statements in the script. For example:

# change engineID set /SP/services/snmp engineid=NEWENGINEID # sleep 10 seconds to give snmp enough time to make the change sleep 10 # verify engineID show /SP/services/snmp engineid # verify SNMPv3 users have been deleted show /SP/services/snmp/users

# create snmpv3 user create /SP/services/snmp/users newuser authenticationpassword=... # sleep 10 seconds to give snmp enough time to make the change sleep 10 # verify user show /SP/services/snmp/users newuser # do a snmpget with that user to verify it

# configure alert set /SP/alertmgmt/rules/1 type=snmptrap ... # sleep 10 seconds to give snmp enough time to make the change sleep 10 # verify alert show /SP/alertmgmt/rules/1 set /SP/alertmgmt/rules/1 testrule=true

Third-Party Web Scan and Test Tools Cause Sluggish Oracle ILOM Performance (23564626)

Under certain conditions, third-party web scanning and test tools can cause Oracle ILOM to run extremely slowly.

Workaround

Install system software release 1.7.1 or newer.

Oracle ILOM Enhancements and Issues 73

Page 74

Oracle ILOM Enhancement Allows IPv4 Only, IPv6 Only, or Dual Stack

On systems with system software release 1.6 on newer, Oracle ILOM supports the ability to independently enable or disable IPv4 and IPv6. Also you can now configure a static IPv6 gateway.

For instructions on how launch Oracle ILOM from the CLI or web interface, see: Connecting to the ILOM in Sun Fire X4800 Server Installation Guide.

To modify the SP network settings, perform one of the following:

■ From the web interface, perform these steps:

Click ILOM Administration > Connectivity > Network.

Modify the settings on the Network Settings page as required.

For further details about how to configure the properties on the Network Setting page, click the More Details link.

Click Save to save your network property changes in Oracle ILOM.

Note - When you save your network settings, it might end your Oracle ILOM session. If this

happens, use the new settings to log in again.

■ From the CLI interface, perform these steps:

To view the assigned IPv4 and IPv6 network settings on the SP, perform the following:

For IPv4, type: show /SP/network

For IPv6, type: show /SP/network/ipv6

To view the descriptions about each IPv4 and IPv6 network property, perform the following:

For IPv4, type: help /SP/network

For IPv6, type: help /SP/network/ipv6

74 Sun Fire X4800 Server Product Notes • July 2017

Page 75

New Procedures for Updating CPLD (CR 7043418)

To modify the IPv4 and IPv6 network properties on the SP, issue the set command.

IPv4 Example:

set /SP/network state=enabled|disabled pendingipdiscovery=static|dhcp pendingipaddress=value pendingipgateway=value pendingipnetmask=value

IPv6 Example:

set /SP/network/ipv6 state=enabled|disabled pending_static_ipaddress=value/ subnet_mask_value pending_static_ipgatewayaddress=value

Note - The set /SP/network/state = enabled command does not enable IPv6. To enable

IPv6, use either set /SP/network/ipv6/state = enabled or (with SW 1.6 or newer), use set /SP/network/state = ipv6-only.

■ To commit the IPv4 and IPv6 pending network changes, type: set /SP/network

commitpending=true

Note - If you are logged in to Oracle ILOM using an Ethernet connection, your connection is

terminated when you set commitpending to true. When this happens, log back in using the new settings.

New Procedures for Updating CPLD (CR 7043418)

The procedures for updating CPLD have changed. For details, refer to Updating the CPLD in Sun Fire X4800 Server Service Manual.

Update Oracle ILOM and BIOS Firmware Before Updating Other Device Firmware (CR 6537282)

When updating the firmware for multiple devices, if your list of firmware updates includes Oracle ILOM and BIOS firmware, update it first.

Oracle ILOM Enhancements and Issues 75

Page 76

Use the Locate Button to Prove Physical Presence (CR 6881237)

Use the Locate Button to Prove Physical Presence (CR

6881237)

Certain Oracle ILOM procedures require you to “prove physical presence”. To do this, press the Locate button located on the system front panel.

For details, see Front and Back Features and Components in Sun Fire X4800 Server Installation Guide.

NEM Expander Firmware Update Procedure (CR 6979140)

Problems with the NEM firmware can cause various symptoms, including but not limited to:

■

System won't boot and displays an fma fault:

sunw-msg-id #SPX86–8002–QQID

■

Partial or no information in sensor data lists, especially in NEM sensors.

To fix these problems, you must update the NEM expander firmware.

Before You Begin

■

“Workaround 1 Using Oracle ILOM Web GUI” on page 76

■

“Workaround 2 Using Oracle ILOM CLI” on page 79

Workaround 1 Using Oracle ILOM Web GUI

Perform the following actions before upgrading your NEM firmware:

■

Download the firmware image for your server or CMM from the platform's product web site. For details, see https://support.oracle.com.

■

Copy the firmware image to the system on which the web browser is running, using a supported protocol (TFTP, FTP, HTTP, HTTPS).

■

Obtain an Oracle ILOM user name and password that has Administrator role account privileges. You must have Administrator privileges to update the firmware on the system.

Restart the server and enter the BIOS screen.

When POST messages appear, press F2 to enter the BIOS Setup Utility.

Note - You do not need to configure anything in the BIOS Setup Utility. It is used in this

procedure to ensure that the NEMs are powered-on but the OS does not boot.

76 Sun Fire X4800 Server Product Notes • July 2017

Page 77

Workaround 1 Using Oracle ILOM Web GUI

For more details, see How to Connect to the ILOM Web Interface in Sun Fire X4800 Server Installation Guide.

Select System Information –> Components.

The Component Management page appears.

Highlight NEM0.

From the Actions drop-down menu, select Update Firmware.

A screen asks for download details.

Oracle ILOM Enhancements and Issues 77

Page 78

Workaround 1 Using Oracle ILOM Web GUI

Note - If the Current Version is FW 5.3.3.4 or greater, you do not need to update your firmware.

Perform the following actions:

Specify the image location by performing one of the following:

■ Click Browse to select the location of the firmware image you want to install.

■ If supported on your system, click Specify URL. Then type the URL for the firmware image into the text box.

Select a transfer method from the drop-down list.

Click the Update button to upload the file and update the firmware.

The Update Status display appears, providing details about the update progress. When the update indicates 100%, the firmware upload is complete.

When the update is finished, it displays the message Firmware Update Successful. Auto-closing windows in 2 seconds, then it closes the display.

Repeat steps 1 through 6 for NEM1.

78 Sun Fire X4800 Server Product Notes • July 2017

Page 79

Press ESC in the BIOS Setup Utility to boot the system.

Workaround 2 Using Oracle ILOM CLI

Before You Begin

Perform the following actions before upgrading your NEM firmware:

■

Download the firmware image for your server or CMM from the platform's product web site. For details, see https://support.oracle.com.

■

Copy the firmware image to the system on which the web browser is running, using a supported protocol (TFTP, FTP, HTTP, HTTPS).

■

Obtain an Oracle ILOM user name and password that has Administrator role account privileges. You must have Administrator privileges to update the firmware on the system.

Restart the server and enter the BIOS screen.

When POST messages appear, press F2 to enter the BIOS the BIOS Setup Utility.

Note - You do not need to configure anything in the BIOS the BIOS Setup Utility. It is used in

this procedure to ensure that the NEMs are powered-on but the OS does not boot.

Use the cd command to navigate to NEM0.

Type the show command to see the NEM properties, including the firmware version.

The fru_extra_1 property field identifies the firmware version presently installed on the NEM. If it is 5.3.3.4 or greater, you do not need to update your NEM firmware.

-> cd /SYS/NEM0 /SYS/NEM0

-> show

/SYS/NEM0 Targets: PRSNT STATE ERR OK SERVICE OK2RM LOCATE

Oracle ILOM Enhancements and Issues 79

Page 80

Workaround 2 Using Oracle ILOM CLI

Properties: type = Network Express Module ipmi_name = NEM0 fru_manufacturer = VENDOR_Name fru_part_number = 511-1056-04 fru_extra_1 = FW 5.3.1.0 fault_state = OK load_uri = (none) clear_fault_action = (none) power_state = On

Commands: cd load reset set show start stop

Use the load_uri command to upload and install the firmware update package.

The following table shows the required parameters.

Transfer Method Examples

TFTP

FTP

SCP

HTTP

HTTPS

SFTP

load_uri=tftp://ip_address/rom_nem.pkg

load_uri=ftp://username:password@ip_address/rom_nem.pkg

load_uri=scp://username:password@ip_address/rom_nem.pkg

load_uri=http://username:password@ip_address/rom_nem.pkg

load_uri=https://username:password@ip_address/rom_nem.pkg

load_uri=sftp://username:password@ip_address/rom_nem.pkg

Where:

■

ip_address is the IP address of the system where the file is stored.

■

username is the login user name to the system where the file is stored.

■

password is the login password to the system where the file is stored.

■

rom_nem.pkg is the name of the firmware update package.

Wait a few moments for Oracle ILOM to confirm the completion of the firmware update process.

A success or failure status appears.

80 Sun Fire X4800 Server Product Notes • July 2017

Page 81

Network Management Port 1 Does Not Work (CR 6935471 and 6854694)

Use the show command to view and confirm the firmware version that is installed on the NEM.

Repeat steps 1 through 7 for NEM1.

Press ESC in the BIOS Setup Utility to restart the boot process.

Network Management Port 1 Does Not Work (CR 6935471 and 6854694)

The CMM has two network management ports. The connectors are labelled NET MGT 0 and NET MGT 1.

NET MGT 1 does not work. Use NET MGT 0 to connect the CMM to the Ethernet.

If you need to connect to more than one network, use an external Ethernet switch.

Note - Oracle ILOM allows you to select which management port to use. Even if you select port

1, it does not switch ports.

start /SYS and stop /SYS Commands Cause Power Button Pressed Event In Log (CR 6906176)

When you enter the start /SYS and stop /SYS commands, it causes a Power Button Pressed event to be logged. This log entry is incorrect.

You can ignore these log entries.

Event Log Messages Might Be Out of Sequence and Duplicated (CR 6915976)

Messages in the event log might be out of sequence and duplicated. For example:

14 Fri Apr 16 05:42:50 2010 IPMI Log critical ID = a : 04/16/2010 : 05:42:50 : System Firmware Progress : BIOS : Man agement controller initialization

Oracle ILOM Enhancements and Issues 81

Page 82

Incorrect Error Message in Event Log After Restore When Serial Console or JavaRconsole Session Are Open (CR 6917474)

13 Fri Apr 16 05:42:49 2010 IPMI Log critical ID = 9 : 04/16/2010 : 05:42:49 : System Boot Initiated : BIOS : Initia ted by hard reset 12 Fri Apr 16 05:42:48 2010 IPMI Log critical ID = a : 04/16/2010 : 05:43:54 : Button : sensor number = 0x00 : Power Button pressed 11 Fri Apr 16 05:42:47 2010 IPMI Log critical ID = 8 : 04/16/2010 : 05:42:47 : Module/Board : BL3/STATE : Transition to Running 10 Fri Apr 16 05:35:36 2010 IPMI Log critical ID = 7 : 04/16/2010 : 05:35:36 : Entity Presence : BL0/FMOD1/PRSNT : D evice Absent 9 Fri Apr 16 05:35:27 2010 IPMI Log critical ID = 6 : 04/16/2010 : 05:35:27 : Module/Board : BL0/STATE : Transition to Power Off

Note message number 12; its ID should be 8, and the IDs of subsequent messages should be decremented accordingly.

Incorrect Error Message in Event Log After Restore When Serial Console or JavaRconsole Session Are Open (CR

6917474)

If you do a restore while a serial console or a JavaRconsole session are open, you might see an error message in the event log. For example:

409 Restore Log major Fri Feb 26 19:42:40 2010 Config restore: Unable to restore property '/SP/serial/host/commitpending'.

This message is incorrect. You can ignore it.

Workaround

If a JavaRconsole session or a serial console is open, use the CLI to set console properties manually. Do not use a configuration restore.

Allocated Power Figures Are Incomplete (CR 6931837)

When you select Power Management – > Allocation in the web interface it shows the power consumption numbers for various server components. It also shows the total consumed by the server.

82 Sun Fire X4800 Server Product Notes • July 2017

Page 83

(Fixed in SW 1.1) LDAP Account Cannot Be Used to Start Console (CR 6969473)

There are two anomalies in the displayed results:

■

The display does not show all the components in the server. For example, it does not show processor modules.

■

As a result, the sum used by the (displayed) components is less than the total used by the system.

This is not really a problem. The Allocated Power figure shows how much power the server is using.

(Fixed in SW 1.1) LDAP Account Cannot Be Used to Start Console (CR 6969473)

If you use an LDAP account to log into Oracle ILOM, the start /SP/console command does not work.

Workaround

Log into Oracle ILOM using a non-LDAP account if you want to use the console.

(Fixed in SW 1.1) Power Cycling the Host Using the Web Interface Generates an Error (CR 6909374)

If you use the Oracle ILOM web interface to power cycle the server, it might display an error message stating that the operation failed, even though it succeeded.

You can ignore this error message.

(Fixed in SW 1.1) DIMM Mismatch Fault Not Cleared After DIMM Replacement and Host Power Cycle (CR 6972285)

If you have a DIMM mismatch fault and you replace the faulted DIMM and restart the host, the fault is not cleared.

Workaround

Oracle ILOM Enhancements and Issues 83

Page 84

(Fixed in SW 1.0.1) Console Redirect from CLI Fails (CR 6939479)

Add the following line to the file cpumem_g5.fdr:

event fault.memory.intel.dimm.mismatched@/sys/bl/p/d, update_fruid = 0;

(Fixed in SW 1.0.1) Console Redirect from CLI Fails (CR

6939479)

Under rare conditions, attempting to start the console using the CLI command start /SP/ console fails.

When this happens, Oracle ILOM returns to the CLI prompt without displaying any warning messages.

When the console redirection command is successful, it looks like this:

-> start /SP/console Are you sure you want to start /SP/console (y/n) y Disabling external host serial connection Serial console started. To stop, type ESC (

When the command fails, it looks like this:

-> start /SP/console Are you sure you want to start /SP/console (y/n) y

Workaround

1. Reboot the SP.

Use the command reset SP.

2. Wait for the SP to fully reboot.

Try the start /SP/console command again

The console redirect works after the SP boots.

(Fixed in SW 1.0.1) Oracle ILOM Only Allows One Console Session (CR 6895423)

Oracle ILOM only supports a single console session, which is started using the start /SP console command.

84 Sun Fire X4800 Server Product Notes • July 2017

Page 85

(Fixed in SW 1.0.1) Faults Disappear After SP Reset (CR 6928998)

(Fixed in SW 1.0.1) Faults Disappear After SP Reset (CR

6928998)

Resetting the SP clears information about faults, but does not remove the actual faults.

Workaround

Before you reset the SP, enter the show faulty command and save the output to retain a record about existing faults.

(Fixed in SW 1.0.1) Clearing a Power Supply Unit (PSU) Fault Erases Other Fault Information (CR 6933348 and CR

6939306)

When the service processor clears a power supply fault, it also erases other fault records. This might happen:

■

When you hotswap a power supply. However, you can preserve the information by creating copies of the other faults.

■

Occasionally a power supply fault clears itself. When this happens, the information about other faults might be lost.

Workaround

Perform this procedure before hotswapping a power supply.

1. Save your fault information immediately when you see a power fault, and before hot swapping a power supply.

■

In the web interface, this information is located in the Fault Management tab.

■

In the CLI, this information is in fmadm.

(Fixed in SW 1.0.1) pciflash Is Not Available (CR 6867060 and CR 6904922)

The command pciflash is not available. As a result, the command ipmiflash —l pci does not work either.

Oracle ILOM Enhancements and Issues 85

Page 86

(Fixed in SW 1.0.1) pciflash Is Not Available (CR 6867060 and CR 6904922)

Workaround

Use other methods to update Oracle ILOM, as described in the Oracle Integrated Lights Out Manager (ILOM) 3.0 Supplement for the Sun Fire X4800 Server.

86 Sun Fire X4800 Server Product Notes • July 2017

Page 87

Oracle Hardware Installation Assistant Issues

The following table lists the Oracle Hardware Installation Assistant issues.

Oracle Hardware Installation Assistant Issues Workaround

“ipmitool Chassis Policy and Persistent Boot Option Are Not Supported (22695087, 22686277)” on page 87

“Oracle Hardware Installation Assistant Image Does Not Boot On a System With More Than 512 GB of Memory (CR 6972927)” on page 87

ipmitool Chassis Policy and Persistent Boot Option Are Not Supported (22695087, 22686277)

The following ipmitool functions are not supported:

■

Set BIOS boot options to "persistent". For example, the following command does not work:

ipmitool -H sp_ip_address -U username -P password -I lanplus chassis bootdev bios options=persistent

■

Set chassis policy. For example, the following command does not work:

ipmitool -H sp_ip_address -U username -P password -I lanplus chassis policy

There is no workaround.

Oracle Hardware Installation Assistant Image Does Not Boot On a System With More Than 512 GB of Memory (CR

6972927)

When you boot Oracle Hardware Installation Assistant on a system with more than 512 GB of memory, the system repeatedly resets itself.

Oracle Hardware Installation Assistant Issues 87

Page 88

88 Sun Fire X4800 Server Product Notes • July 2017

Page 89

BIOS Issues

The following table lists the BIOS issues for Oracle's Sun Fire X4800 server.

BIOS Issues Workaround

“Auto Boot Host On Power Loss Control Is Deactivated” on page 89 N/A

“Update Oracle ILOM and BIOS Firmware Before Updating Other Devices' Firmware (CR

6537282)” on page 89

“Hotplugging PCIe EM Card Fails When Replacing the Original Card With One of a Different Type (CR 7003634)” on page 90

“Messages Complain of Failure to Allocate I/O Resources (CR 6984329)” on page 90 Yes

“DIMM Failure Causes Other DIMMs To Be Disabled (CR 6929978)” on page 91 N/A

Auto Boot Host On Power Loss Control Is Deactivated

The control to Auto Boot Host On Power Loss in the BIOS Setup Utility has been deactivated.

N/A

Yes

To set Auto Boot Host On Power Loss, use Oracle ILOM instead.

Update Oracle ILOM and BIOS Firmware Before Updating Other Devices' Firmware (CR 6537282)

When updating the firmware for multiple devices, if your list of devices includes Oracle ILOM and BIOS firmware, update it first.

BIOS Issues 89

Page 90

Hotplugging PCIe EM Card Fails When Replacing the Original Card With One of a Different Type (CR 7003634)

When you remove a PCIe ExpressModule (EM) card and replace it by hotplugging a card of a different type, the OS might not be able to configure the new card. This is because before booting to the OS, the BIOS allocates resources based on the card's requirements during POST. If the new card requires more resources than the original, the OS does not have the ability to allocate more resources to the new card at this time, and the card does not operate correctly.

Workaround 1

Hotplug PCIe EM cards in previously empty slots.

Workaround 2

If a hotplugged PCIe EM card fails to operate correctly, reboot the server.

Messages Complain of Failure to Allocate I/O Resources (CR 6984329)

Messages that complain about failures to allocate I/O resources might appear in POST, and in log files.

For example, you might see:

Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.0 Sep 8 15:50:49 nsg14-28 kernel: PCI: Failed to allocate I/O resource #2:20@0 for 0000:8d:00.1

Workaround

BIOS tries to allocate I/O resources. If it is unable to allocate I/O resources successfully, it displays error messages.

The OS might try as well. However if it tries, it fails and generates additional failure messages because it uses the same method as BIOS. However, most drivers can manage this condition.

Usually, you can ignore these messages.

If you continue to experience I/O resource issues, see I/O and Interrupt Resource Allocation in Sun Fire X4800 Server Installation Guide.

90 Sun Fire X4800 Server Product Notes • July 2017

Page 91

DIMM Failure Causes Other DIMMs To Be Disabled (CR 6929978)

DIMM Failure Causes Other DIMMs To Be Disabled (CR

6929978)

When a pair of DIMMs is disabled due to faults, BIOS disables other DIMMs to maintain a valid configuration. For example:

■

In an 8-socket system, if a single DIMM is disabled due to faults, BIOS disables either 4 or 8 DIMMs depending on the socket number:

Faulted DIMM Disabled DIMMs

D0 or D4 D0, D4, D1, D5, D8, D9, D12, D13

D1 or D5 D1, D5, D9, D13

D2 or D6 D2, D6, D3, D7, D10, D11, D14, D15

D3 or D7 D3, D7, D11, D15

D8 or D12 D0, D4, D1, D5, D8, D9, D12, D13

D9 or D13 D1, D5, D9, D13

D10 or D14 D2, D6, D3, D7, D10, D11, D14, D15

D11 or D15 D3, D7, D11, D15

■

In a 4–socket system, a faulted DIMM causes the BIOS to disable either two or four DIMMs, depending on the socket number.

Faulted DIMM Disabled DIMMs

D0 or D4 D0, D4, D1, D5

D1 or D5 D1, D5

D2 or D6 D2, D6, D3, D7

D3 or D7 D3, D7

D8 or D12 D8, D9, D12, D13

D9 or D13 D9, D13

D10 or D14 D10, D11, D14, D15

D11 or D15 D11, D15

Note - When faulty happens on odd-numbered DIMMS, see “DIMM Mismatch on Odd

Numbered Pairs Results in DDR Training Failed on Entire Branch (CR7101624 and CR7111545)” on page 34.

Workaround

1. Identify the faulted DIMMs.

BIOS Issues 91

Page 92

DIMM Failure Causes Other DIMMs To Be Disabled (CR 6929978)

Open the Oracle ILOM fault management tab.

Note - DIMMs are always faulted in pairs, and should always be replaced in pairs.

■

Faulty DIMMs are listed as:

■

fault.memory.intel.nex.dimm_ce, or

■

fault.memory.intel.nex.dimm_ue

■

In 4–socket systems, Oracle ILOM only lists the faulted DIMMs. In 8–socket systems, Oracle ILOM also lists disabled DIMMs. These are listed as fault.memory.intel. dimm.population.invalid.

■

If all the DIMM are listed as fault.memory.intel.dimm.population.invalid, then the configuration is invalid.

2. Replace the faulted DIMMs.

Refer to Removing and Installing DIMMs (CRU) in Sun Fire X4800 Server Service Manual.

3. Clear the DIMM faults in Oracle ILOM.

Refer to your Oracle ILOM documentation collection for details.

92 Sun Fire X4800 Server Product Notes • July 2017