Sun Oracle SPARC T3-1B Service Manual

SPARC T3-1B Server Module

Service Manual

Part No.: E29280-02 July 2012

Copyright ©2010, 2012Oracle and/orits afﬁliates.All rightsreserved. This softwareand related documentation areprovided under a license agreementcontaining restrictionson useand disclosure and are protected by

intellectual propertylaws. Exceptas expressly permitted inyour licenseagreement orallowed by law, you may not use, copy, reproduce,translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilationof thissoftware, unless requiredby lawfor interoperability, is prohibited.

The informationcontained hereinis subjectto changewithout noticeand isnot warrantedto beerror-free.If youﬁnd anyerrors, pleasereport them to us in writing.

If thisis softwareor related software documentationthat isdelivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following noticeis applicable:

U.S. GOVERNMENTRIGHTS Programs,software, databases, and related documentation and technical data deliveredto U.S.Government customersare "commercial computersoftware" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-speciﬁc supplemental regulations.As such,the use,duplication, disclosure, modiﬁcation, andadaptation shallbe subjectto therestrictions andlicense terms set forth inthe applicableGovernment contract,and, tothe extentapplicable bythe termsof theGovernment contract,the additionalrights setforth inFAR

52.227-19, CommercialComputer Software License (December2007). OracleAmerica, Inc.,500 OracleParkway, Redwood City,CA 94065. This softwareor hardware is developed forgeneral usein avariety ofinformation management applications. It is not developedor intendedfor usein any

inherently dangerous applications, includingapplications whichmay createa riskof personal injury. If you use this softwareor hardware in dangerous applications, thenyou shallbe responsibleto takeall appropriate fail-safe, backup,redundancy, and other measuresto ensureits safeuse. Oracle Corporation andits afﬁliatesdisclaim anyliability forany damagescaused byuse ofthis software or hardware in dangerous applications.

Oracle andJava areregistered trademarks of Oracle and/or its afﬁliates.Other namesmay betrademarks oftheir respectiveowners.

AMD, Opteron,the AMDlogo, andthe AMDOpteron logo are trademarks or registered trademarksof AdvancedMicro Devices. Intel and Intel Xeon are trademarks orregistered trademarks of Intel Corporation. All SPARC trademarks are used under license and aretrademarks orregistered trademarks of SPARCInternational, Inc.UNIX is a registered trademark licensed through X/Open Company, Ltd.

This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its afﬁliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its afﬁliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Copyright ©2010, 2012Oracle et/ouses afﬁliés.Tous droits réservés. Ce logicielet ladocumentation quil’accompagne sontprotégés parles loissur lapropriété intellectuelle. Ils sont concédés sous licence et soumis à des

restrictions d’utilisationet dedivulgation. Saufdisposition devotre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire,traduire, diffuser,modiﬁer, breveter, transmettre,distribuer,exposer,exécuter, publier ouafﬁcher lelogiciel, mêmepartiellement, sous quelque formeet par quelque procédéque cesoit. Parailleurs, ilest interdit de procéderà toute ingénierie inversedu logiciel, de ledésassembler ou de ledécompiler, excepté à des ﬁnsd’interopérabilité avecdes logicielstiers outel queprescrit par la loi.

Les informationsfournies dansce documentsont susceptiblesde modiﬁcationsans préavis.Par ailleurs,Oracle Corporationne garantitpas qu’elles soient exemptesd’erreurs etvous invite,le caséchéant, àlui enfaire part par écrit.

Si celogiciel, oula documentationqui l’accompagne,est concédésous licenceau Gouvernementdes Etats-Unis,ou àtoute entitéqui délivrela licencede ce logicielou l’utilisepour lecompte duGouvernement desEtats-Unis, lanotice suivantes’applique :

U.S. GOVERNMENTRIGHTS. Programs,software, databases, and related documentation and technical data deliveredto U.S.Government customers are "commercial computer software"or "commercial technical data"pursuant tothe applicableFederal AcquisitionRegulation andagency-speciﬁc supplemental regulations. As such, theuse, duplication,disclosure, modiﬁcation, and adaptation shall be subject to the restrictions and license terms set forth inthe applicableGovernment contract,and, tothe extentapplicable bythe termsof theGovernment contract,the additionalrights setforth inFAR

52.227-19, CommercialComputer Software License (December2007). Oracle America,Inc., 500Oracle Parkway, Redwood City, CA94065. Ce logicielou matériela étédéveloppé pourun usagegénéral dansle cadred’applications degestion desinformations. Celogiciel oumatériel n’estpas

conçu nin’est destinéà êtreutilisé dansdes applicationsà risque,notamment dansdes applicationspouvant causerdes dommagescorporels. Si vous utilisez celogiciel oumatériel dansle cadred’applications dangereuses, il estde votreresponsabilité de prendretoutes lesmesures de secours, de sauvegarde, deredondance et autres mesures nécessairesà sonutilisation dansdes conditionsoptimales desécurité. OracleCorporation etses afﬁliés déclinent touteresponsabilité quantaux dommagescausés parl’utilisation dece logicielou matérielpour cetype d’applications.

Oracle etJava sontdes marquesdéposées d’OracleCorporation et/oude sesafﬁliés.Tout autre nom mentionné peut correspondre à desmarques appartenant àd’autres propriétaires qu’Oracle.

AMD, Opteron,le logoAMD etle logoAMD Opteron sont desmarques oudes marques déposées d’Advanced MicroDevices. Intelet IntelXeon sontdes marques oudes marques déposées d’IntelCorporation. Toutes lesmarques SPARCsont utiliséessous licenceet sontdes marquesou desmarques déposées deSPARCInternational, Inc.UNIX est une marque déposée concédéesous licencepar X/OpenCompany, Ltd.

Ce logicielou matérielet ladocumentation quil’accompagne peuventfournir desinformations oudes liensdonnant accèsà descontenus, desproduits et des servicesémanant detiers. OracleCorporation etses afﬁliésdéclinent touteresponsabilité ou garantie expresse quant aux contenus, produits ou services émanantde tiers.En aucuncas, OracleCorporation etses afﬁliésne sauraientêtre tenus pour responsables des pertes subies, des coûts occasionnés oudes dommagescausés parl’accès àdes contenus,produits ouservices tiers,ou àleur utilisation.

Please

Recycle

Using This Documentation ix

Identifying Components 1

Front and Rear Panel Components 2

Illustrated Parts Breakdown 3

Detecting and Managing Faults 5

Diagnostics Overview 5

Diagnostics Process 7

Diagnostics LEDs 10

Managing Faults (Oracle ILOM) 12

Oracle ILOM Troubleshooting Overview 12

Fault Management 13

Fault Clearing 13

Oracle Solaris Fault Manager Commands in Oracle ILOM 14

HDD Faults 14

▼ Access the SP (Oracle ILOM) 15

▼ Display FRU Information (show Command) 17

▼ Check for Faults (show faulty Command) 18

▼ Check for Faults (fmadm faulty Command) 20

▼ Clear Faults (clear_fault_action Property) 21

Service-Related Oracle ILOM Command Summary 21

Interpreting Log Files and System Messages 23

iii

▼ Check the Message Buffer (dmesg Command) 24

▼ View the System Message Log Files 24

▼ List FRU Status (prtdiag Command) 25

Managing Faults (Oracle Solaris PSH) 25

Oracle Solaris PSH Technology Overview 26

PSH-Detected Fault Example 27

▼ Check for PSH-Detected Faults 28

▼ Clear PSH-Detected Faults 30

Managing Faults (POST) 31

POST Overview 32

Oracle ILOM Properties That Affect POST Behavior 33

▼ Configure How POST Runs 35

▼ Run POST With Maximum Testing 37

▼ Interpret POST Fault Messages 39

▼ Clear POST-Detected Faults 40

POST Error Message Syntax 42

Managing Components (ASR Commands) 44

ASR Overview 44

▼ Display System Components 45

▼ Disable System Components 47

▼ Enable System Components 48

Checking if Oracle VTS Software Is

Installed 48

Oracle VTS Overview 49

▼ Check if Oracle VTS Software Is Installed 50

Preparing for Service 51

General Safety Information 51

Safety Symbols 52

iv SPARC T3-1B Server Module Service Manual • July 2012

ESD Safety Measures 52

Antistatic Wrist Strap Use 52

Antistatic Mat 52

Tools Needed for Service 53

▼ Find the Modular System Serial Number 53

▼ Find the Server Module Serial Number 54

▼ Locate the Server Module 55

Removing the Server Module From the Modular System for Service 55

▼ Shut Down the Oracle Solaris OS 56

▼ Power Off the Server Module (Power Button - Standby Mode) 57

▼ Power Off the Server Module (Emergency Shutdown) 58

▼ Prepare the Server Module for Removal 58

▼ Remove the Server Module From the Modular System 59

▼ Remove the Cover 62

Servicing Hard Drives 63

Drive Hot-Plugging Rules 63

▼ Remove a Drive 64

▼ Replace or Add a Drive 65

▼ Remove a Drive Filler 67

▼ Install a Drive Filler 67

Servicing Memory 69

Memory Faults 69

▼ Locate a Faulty DIMM (LEDs) 70

▼ Remove a DIMM 73

▼ Install a Replacement DIMM 74

▼ Clear the Fault and Verify the Functionality of the Replacement DIMM 75

▼ Verify DIMM Functionality 79

Contents v

DIMM Configuration Reference 81

Servicing a REM 85

▼ Remove a REM 85

▼ Install a REM 86

Servicing a FEM 89

▼ Remove a FEM 89

▼ Install a FEM 90

Servicing a Service Processor Card 93

▼ Remove the Service Processor Card 93

▼ Install the Service Processor Card 94

Servicing the ID PROM 97

▼ Remove the ID PROM 97

▼ Install the ID PROM 98

▼ Verify the ID PROM 99

Servicing a USB Flash Drive 101

▼ Remove a USB Flash Drive 101

▼ Install a USB Flash Drive 102

Servicing the Battery 105

▼ Replace the Battery 105

Replacing the Server Module Enclosure Assembly 107

▼ Transfer Components to Another Enclosure Assembly 108

Returning the Server Module to Operation 111

▼ Replace the Cover 111

▼ Install the Server Module Into the Modular System 112

vi SPARC T3-1B Server Module Service Manual • July 2012

▼ Start the Server Module Host 114

Glossary 115

Index 121

Contents vii

viii SPARC T3-1B Server Module Service Manual • July 2012

Using This Documentation

This service manual explains how to identify faults, replace parts, and add additional options in the SPARC T3-1B server module from Oracle.

This document is written for technicians, system administrators, authorized service providers, and users who have advanced experience troubleshooting and replacing hardware.

■ “Product Notes” on page ix

■ “Feedback” on page x

■ “Support and Accessibility” on page xi

Product Notes

For late-breaking information and known issues about this product, refer to the product notes at:

http://www.oracle.com/pls/topic/lookup?ctx=SPARCT3-1B

Identifying Components

These topics explain the components of the server module, focusing on the components that can be removed and replaced for service.

■ “Front and Rear Panel Components” on page 2

■ “Illustrated Parts Breakdown” on page 3

Related Information

■ “Detecting and Managing Faults” on page 5

■ “Replacing the Server Module Enclosure Assembly” on page 107

Front and Rear Panel Components

FIGURE: Front and Rear Components

Figure Leg end

1 White LED: Locator (functions as the physical presence switch)

2 Blue LED: Ready to Remove

3 Amber LED: Service Action Required

4 Green LED: OK

5 Power button

6 Reset button: NMI (for service use only)

7 Green LED: Drive OK

8 Amber LED: Drive Service Action Required

9 Blue LED: Drive Ready to Remove

10 RFID (sticker indicates serial number of the server module)

11 Universal connector port (UCP)

12 Chassis power connector

13 Chassis data connector

2 SPARC T3-1B Server Module Service Manual • July 2012

Related Information

■ “Diagnostics LEDs” on page 10

■ “Illustrated Parts Breakdown” on page 3

Illustrated Parts Breakdown

This topic identifies components in the server module that you can install, or remove and replace.

The following table provides information about the replaceable components.

Identifying Components 3

TABLE: Replaceable Components

FRU Replacement Instructions Notes FRU Name (If Applicable)

1 Hard drives “Servicing Hard Drives”

on page 63

2 Replacement

enclosure

“Replacing the Server Module Enclosure

/SYS/HDDn where n = 0 - 3

/SYS/MB

Assembly” on page 107

3 Service processor

card

“Servicing a Service Processor Card” on

/SYS/MB/SP

page 93

4 DIMMs “Servicing Memory” on

/SYS/MP/CMP0/BOBn/CHn/Dn

page 69

5 FEM card “Servicing a FEM” on

/SYS/MB/FEMn

page 89

6 REM card “Servicing a REM” on

/SYS/MB/REM

page 85

7 Clock battery “Servicing the Battery”

/SYS/MB/BAT

on page 105

8 Connector cover Remove before inserting

the server module in a slot.

9 USB flash drive “Servicing a USB Flash

Drive” on page 101

Oracle does not offer supported USB flash drives for this server module.

10 ID PROM “Servicing the ID

/SYS/MB/SCC

PROM” on page 97

11 Drive filler “Servicing Hard Drives”

on page 63

Related Information

■ “Front and Rear Panel Components” on page 2

■ “Detecting and Managing Faults” on page 5

■ “Replacing the Server Module Enclosure Assembly” on page 107

4 SPARC T3-1B Server Module Service Manual • July 2012

Detecting and Managing Faults

These topics explain how to use various diagnostic tools to monitor server module status and troubleshoot faults in the server module.

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

■ “Diagnostics LEDs” on page 10

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

Related Information

■ “Preparing for Service” on page 51

Diagnostics Overview

You can use a variety of diagnostic tools, commands, and indicators to monitor and troubleshoot a server module:

■ LEDs – Provide a quick visual notification of the status of the server module and

of some of the FRUs.

■ Oracle ILOM – This firmware runs on the SP. In addition to providing the

interface between the hardware and OS, Oracle ILOM also tracks and reports the health of key server module components. Oracle ILOM works closely with POST and Oracle Solaris PSH technology to keep the system running even when there is a faulty component. You can log in to multiple SP accounts simultaneously and have separate Oracle ILOM shell commands executing concurrently under each account.

Note – Unless indicated otherwise, all examples of interaction with the SP are

depicted with Oracle ILOM shell commands.

■ POST – POST performs diagnostics on system components upon system reset to

ensure the integrity of those components. POST can be configured and works with Oracle ILOM to take faulty components offline if needed.

■ Oracle Solaris PSH - This technology continuously monitors the health of the

CPU, memory and other components, and works with Oracle ILOM to take a faulty component offline if needed. The PSH technology enables systems to accurately predict component failures and mitigate many serious problems before they occur.

■ Log files and command interface – Provide the standard Oracle Solaris OS log

files and investigative commands that can be accessed and displayed on the device of your choice.

■ Oracle VTS (formerly SunVTS) – An application that exercises the system,

provides hardware validation, and discloses possible faulty components with recommendations for repair.

The LEDs, Oracle ILOM, PSH, and many of the log files and console messages are integrated. For example, when the Oracle Solaris software detects a fault, it displays the fault, logs it, and passes information to Oracle ILOM where it is logged. Depending on the fault, one or more LEDs might also be illuminated.

The diagnostic flow chart in “Diagnostics Process” on page 7 describes an approach for using the server module diagnostics to identify a faulty field-replaceable unit (FRU). The diagnostics you use, and the order in which you use them, depend on the nature of the problem you are troubleshooting. Therefore, you might perform some actions and not others.

Related Information

■ SPARC T3 Series Servers Administration Guide

■ “Diagnostics Process” on page 7

■ “Diagnostics LEDs” on page 10

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

6 SPARC T3-1B Server Module Service Manual • July 2012

Diagnostics Process

The following flowchart illustrates the complementary relationship of the different diagnostic tools and indicates a default sequence of use.

Detecting and Managing Faults 7

The following table provides brief descriptions of the troubleshooting actions shown in the flowchart. The table also provides links to topics with additional information on each diagnostic action.

TABLE: Diagnostic Flowchart Reference Table

Diagnostic Action Possible Outcome Additional Information

Flowchart item 1.

Check the Power OK LED.

Flowchart item 2.

Run the Oracle ILOM show faulty command to check for faults.

Flowchart item 3.

Check the Oracle Solaris log files for fault information.

Flowchart item 4.

Run VTS software.

Flowchart item 5.

Run POST.

The Power OK LED is located on the front of the server module.

If this LED is not lit, check the power source and ensure that the server module is properly installed in the modular system chassis.

The show faulty command displays the following kinds of faults:

• Environmental and configuration faults

• PSH-detected faults

• POST-detected faults

Faulty FRUs are identified in fault messages using the FRU name.

All Oracle ILOM detected fault messages begin with the characters “SPT”.

For additional information on a reported fault, including possible corrective action, go to this web site:

http://www.sun.com/msg/message-ID

where message-ID is the message contained in the fault message.

The Oracle Solaris message buffer and log files record system events, and provide information about faults.

• If system messages indicate a faulty device, replace the FRU.

• For more diagnostic information, review the VTS report (flowchart item 4).

VTS is an application you can run to exercise and diagnose FRUs. To run VTS, the server module must be running the Oracle Solaris OS.

• If VTS reports a faulty device, replace the FRU.

• If VTS does not report a faulty device, run POST (flowchart item 5).

POST performs basic tests of the server module components and reports faulty FRUs.

• “Diagnostics LEDs” on

page 10

• “Service-Related Oracle

ILOM Command Summary” on page 21

• “Check for Faults (show

faulty Command)” on page 18

• “Interpreting Log Files and

System Messages” on page 23

• “Checking if Oracle VTS

Software Is Installed” on page 48

• “Managing Faults (POST)”

on page 31

• “Oracle ILOM Properties

That Affect POST Behavior” on page 33

8 SPARC T3-1B Server Module Service Manual • July 2012

TABLE: Diagnostic Flowchart Reference Table (Continued)

Diagnostic Action Possible Outcome Additional Information

Flowchart item 6.

Check if the fault is environmental.

Flowchart item 7.

Determine if the fault was detected by PSH.

Flowchart item 8.

Determine if the fault was detected by POST.

Flowchart item 9.

Contact technical support.

Determine if the fault is an environmental fault or a configuration fault.

If the fault listed by the show faulty command displays a temperature or voltage fault, then the fault is an environmental fault. Environmental faults can be caused by faulty FRUs, or by environmental conditions such as when computer room ambient temperature is too high, or airflow is blocked. When the environmental condition is corrected, the fault will automatically clear.

For additional information on a reported fault, including possible corrective action, go to this web site:

http://www.sun.com/msg/message-ID

where message-ID is the message contained in the fault message.

If the fault message does not begin with the characters “SPT”, the fault was detected by the PSH feature.

For additional information on a reported fault, including possible corrective action, go to this web site:

http://www.sun.com/msg/message-ID

where message-ID is the message contained in the fault message.

After the FRU is replaced, perform the procedure to clear PSH detected faults.

POST performs basic tests of the server module components and reports faulty FRUs. When POST detects a faulty FRU, POST logs the fault and if possible, takes the FRU offline. POST detected FRUs display the following text in the fault message:

Forced fail reason

where reason is the name of the power-on routine that detected the failure.

The majority of hardware faults are detected by the server module’s diagnostics. In rare cases a problem might require additional troubleshooting. If you are unable to determine the cause of the problem, contact your service representative for support.

• “Check for Faults (show

faulty Command)” on page 18

• “Managing Faults (Oracle

Solaris PSH)” on page 25

• “Clear PSH-Detected

Faults” on page 30

• “Managing Faults (POST)”

on page 31

• “Clear POST-Detected

Faults” on page 40

• “Support and Accessibility”

on page xi

Related Information

■ SPARC T3 Series Servers Administration Guide

■ “Diagnostics Overview” on page 5

■ “Diagnostics LEDs” on page 10

Detecting and Managing Faults 9

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

Diagnostics LEDs

The server module has LEDs on the front panel and on the hard drives. The LEDs conform to ANSI SIS. For the locations of these LEDs, see “Front and Rear Panel

Components” on page 2.

The table identifies the server module LEDs, and explains how to interpret their behavior.

LED or Button Icon or Label Color Description

Locator LED and button

Ready to Remove LED

White You can turn on the Locator LED to identify a particular server

module. When on, the LED blinks rapidly. There are two methods for turning a Locator LED on:

• Issuing the Oracle ILOM command set /SYS/LOCATE value=Fast_Blink.

• Pressing the Locator button.

The Locator LED functions as the physical presence switch.

Blue Steady state - If LED is off, it is not safe to remove the server

module from the modular system chassis. You must use Oracle ILOM to shut down the server module and put the blade into ready to remove state before this LED is on.

Service Action Required LED

Amber Indicates that service is required. POST and Oracle ILOM are two

diagnostics tools that can detect a fault or failure resulting in this indication. Also, faults detected by Solaris PSH can result in Oracle ILOM lighting this LED.

The Oracle ILOM show faulty command provides details about any faults that cause this indicator to light.

Under some fault conditions, individual component fault LEDs are turned on in addition to the Service Action Required LED.

10 SPARC T3-1B Server Module Service Manual • July 2012

LED or Button Icon or Label Color Description

Power OK LED Green Indicates the following conditions:

• Off – System is not running in its normal state. System power might be off. The SP might be running.

• Steady on – System is powered on and is running in its normal operating state. No service actions are required.

• Fast blink – System is running in standby mode and can be quickly returned to full function.

• Slow blink – A normal, but transitory activity is taking place. Slow blinking might indicate that system diagnostics are running, or the system is booting.

On/Standby button

n/a The recessed Power button toggles the system on or off.

• Press once to turn the system on.

• Press once to shut the system down to a standby state.

• Press and hold for 4 seconds to perform an emergency shutdown.

HDD Ready to Remove LED

Blue Indicates that a hard drive can be removed during a hot-plug

operation.

HDD Service Action Required LED

HDD OK/Activity LED

Amber Indicates that the hard drive has experienced a fault condition.

Green On HDDs, indicates the following drive status:

• On – Drive is idle and available for use.

• Off – Read or write activity is in progress.

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

Detecting and Managing Faults 11

Managing Faults (Oracle ILOM)

These topics explain how to use Oracle ILOM, the SP firmware, to diagnose faults and verify successful repairs.

■ “Oracle ILOM Troubleshooting Overview” on page 12

■ “Access the SP (Oracle ILOM)” on page 15

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (show faulty Command)” on page 18

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Service-Related Oracle ILOM Command Summary” on page 21

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

■ “POST Overview” on page 32

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

Oracle ILOM Troubleshooting Overview

The Oracle ILOM firmware enables you to remotely run diagnostics, such as POST, that would otherwise require physical proximity to the server module. You can also configure Oracle ILOM to send email alerts of hardware failures, hardware warnings, and other events related to the server module or Oracle ILOM.

The SP runs independently of the server module, using the server module’s standby power. Therefore, Oracle ILOM firmware and software continue to function when the server module OS goes offline or when the server module is powered off.

12 SPARC T3-1B Server Module Service Manual • July 2012

Fault Management

Error conditions detected by Oracle ILOM, POST, and the Oracle Solaris PSH technology are forwarded to Oracle ILOM for fault handling.

The Oracle ILOM fault manager evaluates error messages it receives to determine whether the condition being reported should be classified as an alert or a fault.

■ Alerts – When the fault manager determines that an error condition being

reported does not indicate a faulty FRU, the fault manager classifies the error as an alert.

Alert conditions are often caused by environmental conditions, such as computer room temperature, which might improve over time. Conditions might also be caused by a configuration error, such as the wrong DIMM type being installed.

If the conditions responsible for the alert go away, the fault manager will detect the change and will stop logging alerts for that condition.

■ Faults – When the fault manager determines that a particular FRU has an error

condition that is permanent, that error is classified as a fault. This condition causes the Service Action Required LEDs to be turned on, the FRUID PROMs updated, and a fault message logged. If the FRU has status LEDs, the Service Action Required LED for that FRU will also be turned on.

You must replace a FRU identified as having a fault condition.

In the event of a system fault, Oracle ILOM ensures that the Service Action Required LED is turned on, FRUID PROMs are updated, the fault is logged, and alerts are displayed. Faulty FRUs are identified in fault messages using the FRU name.

Fault Clearing

The SP can detect when a fault is no longer present. When this happens, it clears the fault state in the FRU PROM and extinguishes the Service Action Required LED.

A fault condition can be removed in two ways:

■ Unaided recovery – Faults caused by environmental conditions can clear

automatically if the condition responsible for the fault is no longer present.

Detecting and Managing Faults 13

■ Repaired fault – When a fault is repaired by human intervention, such as a FRU

replacement, the SP will usually detect the repair automatically and extinguish the Service Action Required LED. If the SP does not perform these actions, you must perform these tasks manually by setting the Oracle ILOM component_state or fault_state of the faulted component. The procedure for clearing faults manually is described in “Clear Faults (clear_fault_action Property)” on page 21.

Many environmental faults can automatically recover. For example, a temporary condition might cause the computer room temperature to rise above the maximum threshold, producing an overtemperature fault in the server module. If the computer room temperature then returns to the normal range and the server module’s internal temperature also drops back to an acceptable level, the SP will detect the new fault-free condition. The SP will extinguish the Service Action Required LED and clear the fault state from the FRU PROM.

The SP can automatically detect when a FRU is removed. In many cases, the SP does this even if you remove the FRU while the SP is not running (for example, if you unplug the system power cables during service procedures). This function enables Oracle ILOM to sense that a fault, diagnosed to a specific FRU, has been repaired.

Note – Oracle ILOM does not automatically detect hard drive replacement.

Oracle Solaris Fault Manager Commands in Oracle ILOM

The Oracle ILOM CLI includes a feature that enables you to access Oracle Solaris fault manager commands, such as fmadm, fmdump, and fmstat, from within the Oracle ILOM shell. This feature is referred to as the Oracle ILOM faultmgmt shell.

HDD Faults

The Oracle Solaris PSH technology does not monitor hard drives for faults. As a result, the SP does not recognize hard drive faults and will not light the fault LEDs on either the server module or the hard drive itself. Use the Oracle Solaris message files to view hard drive faults. See “View the System Message Log Files” on page 24.

Related Information

■ Oracle Integrated Lights Out Manager (ILOM) 3.0 Concepts Guide

■ SPARC T3 Series Servers Administration Guide

■ “Oracle ILOM Troubleshooting Overview” on page 12

■ “Access the SP (Oracle ILOM)” on page 15

14 SPARC T3-1B Server Module Service Manual • July 2012

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (show faulty Command)” on page 18

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Service-Related Oracle ILOM Command Summary” on page 21

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

▼ Access the SP (Oracle ILOM)

Note – Unless indicated otherwise, all examples of interaction with the SP are

depicted with Oracle ILOM shell commands, rather than the Oracle ILOM browser interface.

You can access the server module’s SP either directly or through the CMM of the modular system. The following steps are in terms of connecting directly to the server module.

1. Establish connectivity to the SP, using one of the following methods:

■ Serial management port – Connect a terminal device (such as an ASCII

terminal or laptop with terminal emulation) to the serial management port. On the CMM, this connector is labeled SER MGT.

Set up your terminal device for 9600 baud, 8 bit, no parity, 1 stop bit, and no handshaking, and use a null-modem configuration (transmit and receive signals crossed over to enable DTE-to-DTE communication). The crossover adapters supplied with the server module provide a null-modem configuration.

■ Network management port – Connect this port to an Ethernet network. On the

CMM, this connector is labeled NET MGT. This port requires an IP address. By default, this port is configured for DHCP, or you can assign an IP address.

2. Decide which interface to use:

■ Oracle ILOM CLI – The CLI is the default Oracle ILOM UI. Most of the

commands and examples in this service manual use this interface. The default login account is root with a password of changeme.

■ Oracle ILOM browser interface – Can be used when you access the SP through

the network management port and have a browser. Refer to the Oracle ILOM

3.0 documentation for details. This interface is not referenced in this service manual.

Detecting and Managing Faults 15

3. Log in to Oracle ILOM.

The default Oracle ILOM login account is root with a default password of changeme.

Example of logging in to the Oracle ILOM CLI:

ssh root@xxx.xxx.xxx.xxx Password: Waiting for daemons to initialize... Daemons ready Oracle (R) Integrated Lights Out Manager Version 3.0.12.1 r57146 Copyright (c) 2010, Oracle and/or its affiliates, Inc. All rights reserved. Warning: password is set to factory default.

The Oracle ILOM -> prompt indicates that you are accessing the SP with the Oracle ILOM CLI.

4. Perform Oracle ILOM commands that provide the diagnostic information you need.

The following Oracle ILOM commands are commonly used for fault management:

■ show command – Displays information about individual FRUs.

See “Display FRU Information (show Command)” on page 17.

■ show faulty command – Displays environmental, POST-detected, and

PSH-detected faults.

See “Check for Faults (show faulty Command)” on page 18.

Note – You can use fmadm faulty in the Oracle ILOM faultmgmt shell as an

alternative to show faulty.

■ clear_fault_action property of the set command – Manually clears

PSH-detected faults.

See “Clear Faults (clear_fault_action Property)” on page 21.

Related Information

■ Oracle Integrated Lights Out Manager (ILOM) 3.0 Concepts Guide

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (show faulty Command)” on page 18

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Service-Related Oracle ILOM Command Summary” on page 21

16 SPARC T3-1B Server Module Service Manual • July 2012

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

▼ Display FRU Information (show Command)

Use the Oracle ILOM show command to display information about individual FRUs.

● At the -> prompt, enter the show command.

In the following example, the show command displays information about a memory module.

-> show /SYS/MB/CMP0/BOB0/CH0/D0

/SYS/MB/CMP0/BOB0/CH0/D0

Targets:

T_AMB SERVICE

Properties:

Type = DIMM ipmi_name = B0/C0/D0 component_state = Enabled fru_name = 2048MB DDR3 SDRAM fru_description = DDR3 DIMM 2048 Mbytes fru_manufacturer = Samsung fru_version = 0 fru_part_number = **************** fru_serial_number = ****************** fault_state = OK clear_fault_action = (none)

Commands:

cd set show

Related Information

■ “Diagnostics Process” on page 7

■ “Access the SP (Oracle ILOM)” on page 15

■ “Check for Faults (show faulty Command)” on page 18

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Service-Related Oracle ILOM Command Summary” on page 21

Detecting and Managing Faults 17

▼ Check for Faults (show faulty Command)

Use the Oracle ILOM show faulty command to display the following kinds of faults and alerts:

■ Environmental or configuration faults – Faults caused by temperature or voltage

problems. Environmental faults can also be caused by room temperature or blocked air flow.

■ POST-detected faults – Faults on devices detected by the POST diagnostics.

■ PSH-detected faults – Faults detected by the Oracle Solaris PSH technology.

1. At the -> prompt, enter the show faulty command.

2. If a fault is displayed, check the output to determine the nature of the fault.

The following examples show the different kinds of output that might be displayed:

■ Example of the show faulty command when no faults are present:

-> show faulty Target | Property | Value

--------------------+------------------------+-------------------

-----------------------------------------------------------------

■ Example of the show faulty command displaying a fault when one of the AC

inputs for power supply PS0 is not plugged in:

-> show faulty Target | Property | Value

18 SPARC T3-1B Server Module Service Manual • July 2012

/SP/faultmgmt/0 | chassis_serial_number | 0000000-0000000000 faults/0 | |

■ Example of the show faulty command displaying a fault that was detected by

POST. These kinds of faults are identified by the message Forced fail reason, where reason is the name of the power-on routine that detected the fault.

-> show faulty Target | Property | Value

■ Example of the show faulty command displaying a fault that was detected by

the PSH technology. These kinds of faults are identified by the presence of a UUID value.

-> show faulty Target | Property | Value

Related Information

■ “Diagnostics Process” on page 7

■ “Access the SP (Oracle ILOM)” on page 15

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Service-Related Oracle ILOM Command Summary” on page 21

Detecting and Managing Faults 19

▼ Check for Faults (fmadm faulty Command)

The following is an example of the fmadm faulty command, which is an alternative to the show faulty command.

The Oracle Solaris fmadm faulty command is invoked from within the Oracle ILOM faultmgmt shell.

Note – The characters SPT at the beginning of a message ID indicate that Oracle

ILOM detected the fault.

1. At the -> prompt, access the Oracle ILOM faultmgmt shell.

-> start /SP/faultmgmt/shell Are you sure you want to start /SP/faultmgmt/shell (y/n)? y

2. At the faultmgmtsp> prompt, type the fmadm faulty command.

faultmgmtsp> fmadm faulty

------------------- ------------------------------------ ------------ ------Time UUID msgid Severity

------------------- ------------------------------------ ------------ ------2010-08-11/14:54:23 ********-****-****-****-******* SPT-8000-LC Critical ...

3. Type the exit command when you are finished using the Oracle ILOM faultmgt shell.

faultmgmtsp> exit

Related Information

■ “Diagnostics Process” on page 7

■ “Access the SP (Oracle ILOM)” on page 15

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (show faulty Command)” on page 18

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Service-Related Oracle ILOM Command Summary” on page 21

20 SPARC T3-1B Server Module Service Manual • July 2012

▼ Clear Faults (clear_fault_action Property)

Use the clear_fault_action property with the set command to manually clear PSH-detected faults for a FRU.

If Oracle ILOM detects a FRU replacement, it will automatically clear the fault so that you do not have to clear the fault manually. For PSH-diagnosed faults, if the replacement of the FRU is detected by the system or the fault is manually cleared on the host, the fault will also be cleared from Oracle ILOM. In such cases, you typically do not have to clear the fault manually.

Note – This procedure clears the fault from the SP but not from the host. If the fault

persists in the host, clear it manually as described in “Clear PSH-Detected Faults” on

page 30.

● At the -> prompt, use the set command with the clear_fault_action=True

property.

Example:

-> set /SYS/MB/CMP0/BOB0/CH0/D0 clear_fault_action=True Are you sure you want to clear /SYS/MB/CMP0/BOB0/CH0/D0 (y/n)? y Set ’clear_fault_action’ to ’true’

Related Information

■ “Diagnostics Process” on page 7

■ “Access the SP (Oracle ILOM)” on page 15

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (show faulty Command)” on page 18

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Service-Related Oracle ILOM Command Summary” on page 21

Service-Related Oracle ILOM Command Summary

The following table describes the Oracle ILOM shell commands most frequently used when performing service-related tasks.

Detecting and Managing Faults 21

Oracle ILOM Command Description

help [command] Displays a list of all available commands with syntax

and descriptions. Specifying a command name as an option displays help for that command.

set /HOST send_break_action=break Takes the host server module from the OS to either

kmdb or OBP (equivalent to a Stop-A), depending on

the mode Oracle Solaris software was booted.

set /SYS/component clear_fault_action=true Manually clears host-detected faults. The component is

the unique ID of the device with a fault to be cleared.

start /HOST/console Connects you to the host system.

show /HOST/console/history Displays the contents of the system’s console buffer.

set /HOST/bootmode property=value Controls the host server module OBP firmware

method of booting. property is state, config,or script

stop /SYS start /SYS

Powers off the host server module and then powers on the host server module.

stop /SYS Powers off the host server module.

start /SYS Powers on the host server module.

reset /SYS Generates a hardware reset on the host server module.

reset /SP Reboots the SP.

set /SYS keyswitch_state=value Sets the virtual keyswitch. value is normal, standby,

diag,or locked.

set /SYS/LOCATE value=value Turns the Locator LED on the server module on or off.

value is Fast_blink or Off.

show faulty Displays current system faults. See “Check for Faults

(show faulty Command)” on page 18.

show /SYS keyswitch_state Displays the status of the virtual keyswitch.

show /SYS/LOCATE Displays the current state of the Locator LED as either

on or off.

show /SP/logs/event/list Displays the history of all events logged in the SP

event buffers (in RAM or the persistent buffers).

show /HOST Displays information about the operating state of the

host system, whether the hardware is providing service, and system firmware version information.

show /SYS Displays information about the system serial number.

22 SPARC T3-1B Server Module Service Manual • July 2012

Related Information

■ “Oracle ILOM Troubleshooting Overview” on page 12

■ “Access the SP (Oracle ILOM)” on page 15

■ “Display FRU Information (show Command)” on page 17

■ “Check for Faults (show faulty Command)” on page 18

■ “Check for Faults (fmadm faulty Command)” on page 20

■ “Clear Faults (clear_fault_action Property)” on page 21

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

Interpreting Log Files and System Messages

With the Oracle Solaris OS running on the server module, you have the full complement of Oracle Solaris OS files and commands available for collecting information and for troubleshooting.

If POST or the Oracle Solaris PSH features do not indicate the source of a fault, check the message buffer and log files for notifications for faults. Hard disk drive faults are usually captured by the Oracle Solaris message files.

■ “Check the Message Buffer (dmesg Command)” on page 24

■ “View the System Message Log Files” on page 24

■ “List FRU Status (prtdiag Command)” on page 25

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

Detecting and Managing Faults 23

▼ Check the Message Buffer (dmesg Command)

The dmesg command checks the system buffer for recent diagnostic messages and displays them.

1. Log in as superuser.

2. Type:

# dmesg

Related Information

■ “View the System Message Log Files” on page 24

■ “List FRU Status (prtdiag Command)” on page 25

▼ View the System Message Log Files

The error logging daemon, syslogd, automatically records various system warnings, errors, and faults in message files. These messages can alert you to system problems such as a device that is about to fail.

The /var/adm directory contains several message files. The most recent messages are in the /var/adm/messages file. After a period of time (usually every week), a new message file is automatically created. The original contents of the messages file are rotated to a file named messages.0. Over a period of time, the messages are further rotated to messages.1 and messages.2, and then deleted.

1. Log in as superuser.

2. Type:

# more /var/adm/messages

Or, if you want to view all logged messages, type:

# more /var/adm/messages*

Related Information

■ “Check the Message Buffer (dmesg Command)” on page 24

24 SPARC T3-1B Server Module Service Manual • July 2012

▼ List FRU Status (prtdiag Command)

● From an Oracle Solaris OS command line, run the prtdiag command.

FRU status information is displayed.

Example:

# prtdiag System Configuration: Sun Microsystems sun4v SPARC T3-1B Memory size: 130560 Megabytes ================================ Virtual CPUs ================================

CPU ID Frequency Implementation Status

------ --------- ---------------------- ------0 1649 MHz SPARC-T3 on-line 1 1649 MHz SPARC-T3 on-line <...> <...>

============================ FRU Status ======================= All FRUs are enabled.

Related Information

■ “Check the Message Buffer (dmesg Command)” on page 24

■ “View the System Message Log Files” on page 24

■ “Display FRU Information (show Command)” on page 17

Managing Faults (Oracle Solaris PSH)

The following topics describe the Oracle Solaris PSH feature:

■ “Oracle Solaris PSH Technology Overview” on page 26

■ “PSH-Detected Fault Example” on page 27

■ “Check for PSH-Detected Faults” on page 28

■ “Clear PSH-Detected Faults” on page 30

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

Detecting and Managing Faults 25

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

■ “POST Overview” on page 32

Oracle Solaris PSH Technology Overview

The Oracle Solaris PSH technology enables the server module to diagnose problems while the Oracle Solaris OS is running and to mitigate many problems before they negatively affect operations.

The Oracle Solaris OS uses the fault manager daemon, fmd(1M), which starts at boot time and runs in the background to monitor the system. If a component generates an error, the daemon correlates the error with data from previous errors and other relevant information to diagnose the problem. Once diagnosed, the fault manager daemon assigns a UUID to the error. This value distinguishes this error across any set of systems.

When possible, the fault manager daemon initiates steps to self-heal the failed component and take the component offline. The daemon also logs the fault to the syslogd daemon and provides a fault notification with a message ID (sometimes labeled MSG-ID). You can use the message ID to get additional information about the problem from the knowledge article database.

The PSH technology covers the following server module components:

■ CPU

■ Memory

■ I/O subsystem

The PSH console message provides the following information about each detected fault:

■ Type

■ Severity

■ Description

■ Automated response

■ Impact

■ Suggested action for system administrator

26 SPARC T3-1B Server Module Service Manual • July 2012

If the PSH facility detects a faulty component, use the fmadm faulty command to display information about the fault. Alternatively, you can use the Oracle ILOM command show faulty for the same purpose.

Related Information

■ “Check for Faults (show faulty Command)” on page 18

■ “PSH-Detected Fault Example” on page 27

■ “Check for PSH-Detected Faults” on page 28

■ “Clear PSH-Detected Faults” on page 30

PSH-Detected Fault Example

When a PSH fault is detected, an Oracle Solaris console message similar to the following example is displayed.

SUNW-MSG-ID: SUN4V-8000-DX, TYPE: Fault, VER: 1, SEVERITY: Minor EVENT-TIME: Wed Jun 17 10:09:46 EDT 2009 PLATFORM: SUNW,system_name, CSN: -, HOSTNAME: server48-37 SOURCE: cpumem-diagnosis, REV: 1.5 EVENT-ID: f92e9fbe-735e-c218-cf87-9e1720a28004 DESC: The number of errors associated with this memory module has exceeded acceptable levels. Refer to http://sun.com/msg/SUN4V-8000-DX for more information. AUTO-RESPONSE: Pages of memory associated with this memory module are being removed from service as errors are reported. IMPACT: Total system memory capacity will be reduced as pages are retired. REC-ACTION: Schedule a repair procedure to replace the affected memory module. Use fmdump -v -u <EVENT_ID> to identify the module.

Note – The Service Action Required LED is also turned on for PSH-diagnosed faults.

Related Information

■ “Oracle Solaris PSH Technology Overview” on page 26

■ “Check for PSH-Detected Faults” on page 28

■ “Clear PSH-Detected Faults” on page 30

Detecting and Managing Faults 27

▼ Check for PSH-Detected Faults

Use the fmadm faulty command to display the list of faults detected by the Oracle Solaris PSH facility. You can run this command either from the host or through the Oracle ILOM fmadm shell.

As an alternative, you can display fault information by running the Oracle ILOM command show.

1. Check the event log:

# fmadm faulty TIME EVENT-ID MSG-ID SEVERITY Aug 13 11:48:33 21a8b59e-89ff-692a-c4bc-f4c5cccca8c8 SUN4V-8002-6E Major

Platform : sun4v Chassis_id : Product_sn :

Fault class : fault.cpu.generic-sparc.strand Affects : cpu:///cpuid=**/serial=*********************

faulted and taken out of service FRU : "/SYS/MB" (hc://:product-id=*****:product-sn=**********:server-id=***-******-*****: chassis-id=********:**************-**********:serial=******:revision=05/ chassis=0/motherboard=0)

faulty

Description : The number of correctable errors associated with this strand has

exceeded acceptable levels. Refer to http://sun.com/msg/SUN4V-8002-6E for more information.

Response : The fault manager will attempt to remove the affected strand

from service.

Impact : System performance may be affected.

Action : Schedule a repair procedure to replace the affected resource, the

identity of which can be determined using ’fmadm faulty’.

In this example, a fault is displayed, indicating the following details:

■ Date and time of the fault (Aug 13 11:48:33).

■ UUID, which is unique for every fault

(21a8b59e-89ff-692a-c4bc-f4c5cccca8c8).

■ Message identifier, which can be used to obtain additional fault information

(SUN4V-8002-6E).

28 SPARC T3-1B Server Module Service Manual • July 2012

■ Faulted FRU. The information provided in the example includes the part

number of the FRU (part=511127809) and the serial number of the FRU (serial=1005LCB-1019B100A2). The FRU field provides the name of the FRU (/SYS/MB for motherboard in this example).

2. Use the message ID to obtain more information about this type of fault:

a. Obtain the message ID from console output or from the Oracle ILOM show

faulty command.

b. Enter the message ID at the end of the PSH Knowledge Article web site,

http://www.sun.com/msg.

In the current example, enter this in the browser address window:

http://www.sun.com/msg/SUN4V-8002-6E

The following example shows the message ID SUN4V-8002-6E and provides information for corrective action.

Correctable strand errors exceeded acceptable levels

Type

Fault

Severity

Major

Description

The number of correctable errors associated with this strand has exceeded acceptable levels.

Automated Response

The fault manager will attempt to remove the affected strand from service.

Impact

System performance may be affected.

Suggested Action for System Administrator

Schedule a repair procedure to replace the affected resource, the identity of which can be determined using fmadm faulty.

Details

There is no more information available at this time.

3. Follow the suggested actions to repair the fault.

Related Information

■ “PSH-Detected Fault Example” on page 27

■ “Clear PSH-Detected Faults” on page 30

Detecting and Managing Faults 29

▼ Clear PSH-Detected Faults

When the Oracle Solaris PSH technology detects faults, the faults are logged and displayed on the console. In most cases, after the fault is repaired, the system detects the corrected state and repairs the fault condition automatically. However, you should verify this repair. In cases where the fault condition is not automatically cleared, you must clear the fault manually.

1. After replacing a faulty FRU, power on the server module.

2. At the host prompt, use the fmadm faulty command to determine whether the

replaced FRU still shows a faulty state.

# fmadm faulty TIME EVENT-ID MSG-ID SEVERITY Aug 13 11:48:33 21a8b59e-89ff-692a-c4bc-f4c5cccca8c8 SUN4V-8002-6E Major

Platform : sun4v Chassis_id : Product_sn :

Fault class : fault.cpu.generic-sparc.strand Affects : cpu:///cpuid=**/serial=*********************

faulty

Description : The number of correctable errors associated with this strand has

exceeded acceptable levels. Refer to http://sun.com/msg/SUN4V-8002-6E for more information.

Response : The fault manager will attempt to remove the affected strand

from service.

Impact : System performance may be affected.

Action : Schedule a repair procedure to replace the affected resource, the

identity of which can be determined using ’fmadm faulty’.

■ If no fault is reported, you do not need to do anything else. Do not perform the

subsequent steps.

■ If a fault is reported, continue to the next step.

30 SPARC T3-1B Server Module Service Manual • July 2012

3. Clear the fault from all persistent fault records.

In some cases, even though the fault is cleared, some persistent fault information remains and results in erroneous fault messages at boot time. To ensure that these messages are not displayed, type the following Oracle Solaris command:

# fmadm repair UUID

For the UUID in the example shown in Step 2, type:

# fmadm repair 21a8b59e-89ff-692a-c4bc-f4c5cccca8c8

4. Use the clear_fault_action property of the FRU to clear the fault.

-> set /SYS/MB clear_fault_action=True Are you sure you want to clear /SYS/MB (y/n)? y set ’clear_fault_action’ to ’true

Related Information

■ “Oracle Solaris PSH Technology Overview” on page 26

■ “PSH-Detected Fault Example” on page 27

■ “Clear PSH-Detected Faults” on page 30

Managing Faults (POST)

These topics explain how to use POST as a diagnostic tool.

■ “POST Overview” on page 32

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Configure How POST Runs” on page 35

■ “Run POST With Maximum Testing” on page 37

■ “Interpret POST Fault Messages” on page 39

■ “Clear POST-Detected Faults” on page 40

■ “POST Error Message Syntax” on page 42

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

Detecting and Managing Faults 31

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Components (ASR Commands)” on page 44

■ “Checking if Oracle VTS Software Is Installed” on page 48

POST Overview

POST is a group of PROM-based tests that run when the server module is powered on or when it is reset. POST checks the basic integrity of the critical hardware components in the server module (CMP, memory, and I/O subsystem).

You can also run POST as system-level hardware diagnostic tool. To do this, use the Oracle ILOM set command to set the parameter keyswitch_state to diag.

You can also set other Oracle ILOM properties to control various other aspects of POST operations. For example, you can specify the events that cause POST to run, the level of testing POST performs, and the amount of diagnostic information POST displays. These properties are listed and described in “Oracle ILOM Properties That

Affect POST Behavior” on page 33.

If POST detects a faulty component, the component is disabled automatically. If the system is able to run without the disabled component, it will boot when POST completes its tests. For example, if POST detects a faulty processor core, the core will be disabled. After POST completes its test sequence, the system will boot and run using the remaining cores.

Related Information

■ “Diagnostics Overview” on page 5

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Configure How POST Runs” on page 35

■ “Run POST With Maximum Testing” on page 37

■ “Interpret POST Fault Messages” on page 39

■ “Clear POST-Detected Faults” on page 40

■ “POST Error Message Syntax” on page 42

32 SPARC T3-1B Server Module Service Manual • July 2012

Oracle ILOM Properties That Affect POST Behavior

The following table describes the Oracle ILOM properties that determine how POST performs its operations.

Note – The value of keyswitch_state must be normal when individual POST

parameters are changed.

Parameter Values Description

/SYS keyswitch_state normal The system can power on and run POST (based on the

other parameter settings). This parameter overrides all other commands.

diag The system runs POST based on predetermined

settings.

standby The system cannot power on.

locked The system can power on and run POST, but no flash

updates can be made.

/HOST/diag mode off POST does not run.

normal Runs POST according to diag level value.

service Runs POST with preset values for diag level and

diag verbosity.

/HOST/diag level max If diag mode = normal, runs all the minimum tests

plus extensive processor and memory tests.

min If diag mode = normal, runs minimum set of tests.

/HOST/diag trigger none Does not run POST on reset.

hw-change (Default) Runs POST following an AC power cycle

and when the top cover is removed.

power-on-reset Only runs POST for the first power on.

error-reset (Default) Runs POST if fatal errors are detected.

all-resets Runs POST after any reset.

/HOST/diag verbosity normal POST output displays all test and informational

messages.

min POST output displays functional tests with a banner

and pinwheel.

Detecting and Managing Faults 33

Parameter Values Description

max POST displays all test, informational, and some

debugging messages.

debug

none No POST output is displayed.

The following flowchart illustrates the same set of Oracle ILOM set command variables.

The following table shows combinations of Oracle ILOM parameters and associated POST modes.

34 SPARC T3-1B Server Module Service Manual • July 2012

Oracle ILOM Parameter

keyswitch_state

Normal Diagnostic Mode (Default Settings) No POST Execution

normal normal diag

Service Mode Using the Keyswitch_state

/HOST/diag mode normal Off N/A

/HOST/diag level max N/A N/A

/HOST/diag trigger hw-change error-reset none N/A

/HOST/diag verbosity normal N/A N/A

Description of POST Execution

This is the default POST configuration. This configuration tests the system thoroughly and suppresses some of the

POST does not run, resulting in quick system initialization. This configuration is not suggested.

POST runs the full spectrum of tests with the maximum output displayed.

detailed POST output.

* The keyswitch_state parameter, when set to diag, overrides all the other POST variables.

Related Information

■ “POST Overview” on page 32

■ “Configure How POST Runs” on page 35

■ “Run POST With Maximum Testing” on page 37

■ “Interpret POST Fault Messages” on page 39

■ “Clear POST-Detected Faults” on page 40

■ “POST Error Message Syntax” on page 42

▼ Configure How POST Runs

1. Log in to Oracle ILOM -> prompt.

See “Access the SP (Oracle ILOM)” on page 15.

Detecting and Managing Faults 35

2. Set the virtual keyswitch to the value that corresponds to the POST configuration you want to run.

The following example sets the virtual keyswitch to normal, which will configure POST to run according to other parameter values.

-> set /SYS keyswitch_state=normal Set ‘keyswitch_state' to ‘Normal'

For possible values for the keyswitch_state parameter, see “Oracle ILOM

Properties That Affect POST Behavior” on page 33.

3. If the virtual keyswitch is set to normal, and you want to define the mode, level, verbosity,ortrigger, set the respective parameters.

Syntax:

set /HOST/diag property=value

See “Oracle ILOM Properties That Affect POST Behavior” on page 33 for a list of parameters and values.

Examples:

-> set /HOST/diag mode=normal

-> set /HOST/diag verbosity=max

4. To see the current values for settings, use the show command.

Example, showing default values:

-> show /HOST/diag

/HOST/diag Targets:

Properties:

error_reset_level = max

error_reset_verbosity = normal hw_change_level = max hw_change_verbosity = normal

level = max

mode = normal

power_on_level = max

power_on_verbosity = normal

trigger = hw-change error-reset

verbosity = normal

36 SPARC T3-1B Server Module Service Manual • July 2012

Commands: cd set show

Related Information

■ “POST Overview” on page 32

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Run POST With Maximum Testing” on page 37

■ “Interpret POST Fault Messages” on page 39

■ “Clear POST-Detected Faults” on page 40

▼ Run POST With Maximum Testing

This procedure describes how to configure the server module to run the maximum level of POST.

1. Access the Oracle ILOM -> prompt:

See “Access the SP (Oracle ILOM)” on page 15.

2. Set the virtual keyswitch to diag so that POST will run in service mode.

-> set /SYS/keyswitch_state=diag Set ‘keyswitch_state' to ‘Diag'

3. Reset the system so that POST runs.

There are several ways to initiate a reset. The following example shows a reset by issuing commands that will power cycle the host.

-> stop /SYS Are you sure you want to stop /SYS (y/n)? y Stopping /SYS

-> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS

Note – The server module takes about one minute to power off. Type the show

/HOST command to determine when the host has been powered off. The console will display status=Powered Off.

Detecting and Managing Faults 37

4. Switch to the system console to view the POST output.

-> start /HOST/console

The following example shows abridged POST output.

0:0:0>SPARC T3-1B POST 4.32.1.b 2010/11/15 21:42 0:0:0> 0:0:0>Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 0:0:0>POST enabling CMP 0 threads: ffffffff.ffffffff.ffffffff.ffffffff 0:0:0>Diag mode : 1 [Normal] 0:0:0>Diag level : 1 [Max] 0:0:0>Diag verbosity : 2 [Normal]

0:0:0>Test Memory....Done

0:0:0>Setup POST Mailbox ....Done

0:0:0>Master CPU Tests Basic....Done

0:0:0>Init MMU.....

0:0:0>Setup POST Mailbox ....Done

0:0:0>L2 Tests....Done

0:0:0>Extended CPU Tests....Done

0:0:0>Scrub Memory....Done

0:0:0>Functional CPU Tests....Done

0:0:0>Extended Memory Tests....Done

0:0:0>SPU CWQ Tests...Done 0:0:0>MAU Tests...Done

0:0:0>IOS register tests....Done

0:0:0>Network Interface Unit Port 0 Tests ..Done 0:0:0>Network Interface Unit Port 1 Tests ..Done 2010-11-18 22:24:47.330 0:0:0>INFO: 2010-11-18 22:24:47.338 0:0:0> POST Passed all devices. 2010-11-18 22:24:47.351 0:0:0>POST: Return to Host Config.

5. If you receive POST error messages, learn how to interpret them.

See “Interpret POST Fault Messages” on page 39.

Related Information

■ “POST Overview” on page 32

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Configure How POST Runs” on page 35

■ “Interpret POST Fault Messages” on page 39

■ “Clear POST-Detected Faults” on page 40

38 SPARC T3-1B Server Module Service Manual • July 2012

▼ Interpret POST Fault Messages

1. Run POST.

See “Run POST With Maximum Testing” on page 37.

2. View the output and watch for messages that look similar to the following syntax descriptions and example:

■ POST error messages use the following syntax, where c = the core number, s =

the strand number:

c:s> ERROR: TEST = failing-test

c:s > H/W under test = FRU

c:s > Repair Instructions: Replace items in order listed by

H/W under test above

c:s> MSG = test-error-message

c:s > END_ERROR

In this syntax, c = the core number, s = the strand number.

■ Warning and informational messages use the following syntax:

INFO: message

WARNING: message

Example:

3:2>ERROR: TEST = Data Bitwalk 3:2>H/W under test = /SYS/MB/BOB1/CH0/D0 3:2>Repair Instructions: Replace items in order listed by 'H/W under test' above. 3:2>MSG = Pin 149 failed on /SYS/MB/BOB1/CH0/D0 (J1101) 3:2>END_ERROR

3:2>Decode of Dram Error Log Reg Channel 2 bits

60000000.0000108c 3:2> 1 MEC 62 R/W1C Multiple corrected errors, one or more CE not logged 3:2> 1 DAC 61 R/W1C Set to 1 if the error was a DRAM access CE 3:2> 108c SYND 15:0 RW ECC syndrome. 3:2> 3:2> Dram Error AFAR channel 2 = 00000000.00000000 3:2> L2 AFAR channel 2 = 00000000.00000000

Detecting and Managing Faults 39

3. To obtain more information on faults, run the show faulty command.

See “Check for Faults (show faulty Command)” on page 18.

Related Information

■ “Clear POST-Detected Faults” on page 40

■ “POST Overview” on page 32

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Diagnostics Overview” on page 5

■ “Configure How POST Runs” on page 35

■ “Run POST With Maximum Testing” on page 37

▼ Clear POST-Detected Faults

Use this procedure if you suspect that a fault was not automatically cleared. This procedure describes how to identify a POST-detected fault and, if necessary, manually clear the fault.

In most cases, when POST detects a faulty component, POST logs the fault and automatically takes the failed component out of operation by placing the component in the ASR blacklist. (See “Managing Components (ASR Commands)” on page 44).

Usually, when a faulty component is replaced, the replacement is detected when the SP is reset or power cycled. Then the fault is automatically cleared from the system.

1. After replacing a faulty FRU, at the Oracle ILOM prompt use the show faulty

command to identify POST detected faults.

POST-detected faults are distinguished from other kinds of faults by the text:

Forced fail. No UUID number is reported. Example:

-> show faulty Target | Property | Value

40 SPARC T3-1B Server Module Service Manual • July 2012

2. Take one of the following actions based on the show faulty output:

■ No fault is reported – The system cleared the fault and you do not need to

manually clear the fault. Do not perform the subsequent steps.

■ Fault reported – Go to the next step in this procedure.

3. Use the component_state property of the component to clear the fault and remove the component from the ASR blacklist.

Use the FRU name that was reported in the fault in Step 1. Example:

-> set /SYS/MB/CMP0/BOB1/CH0/D0 component_state=Enabled

The fault is cleared and should not show up when you run the show faulty command. Additionally, the front panel Fault (Service Action Required) LED is no longer on.

4. Reset the server module.

You must reboot the server module for the component_state property to take effect.

5. At the Oracle ILOM prompt, use the show faulty command to verify that no faults are reported.

Example:

-> show faulty Target | Property | Value

--------------------+------------------------+------------------

Related Information

■ “POST Overview” on page 32

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Configure How POST Runs” on page 35

■ “Run POST With Maximum Testing” on page 37

■ “Clear POST-Detected Faults” on page 40

Detecting and Managing Faults 41

POST Error Message Syntax

POST error messages use the following syntax:

c:s > ERROR: TEST = failing-test c:s > H/W under test = FRU c:s > Repair Instructions: Replace items in order listed by H/W under test above c:s > MSG = test-error-message c:s > END_ERROR

In this syntax, c = the core number, s = the strand number.

Warning messages use the following syntax:

WARNING: message

Informational messages use the following syntax:

INFO: message

In the following example, POST reports an uncorrectable memory error affecting DIMM locations /SYS/MB/CMP0/BOB0/CH0/D0 and /SYS/MB/CMP0/BOB1/CH0/D0. The error was detected by POST running on node 0, core 7, strand 2.

2010-07-03 18:44:13.359 0:7:2>Decode of Disrupting Error Status Reg (DESR HW Corrected) bits 00300000.00000000 2010-07-03 18:44:13.517 0:7:2> 1 DESR_SOCSRE: SOC (non-local) sw_recoverable_error. 2010-07-03 18:44:13.638 0:7:2> 1 DESR_SOCHCCE: SOC (non-local) hw_corrected_and_cleared_error. 2010-07-03 18:44:13.773 0:7:2> 2010-07-03 18:44:13.836 0:7:2>Decode of NCU Error Status Reg bits

00000000.22000000 2010-07-03 18:44:13.958 0:7:2> 1 NESR_MCU1SRE: MCU1 issued a Software Recoverable Error Request 2010-07-03 18:44:14.095 0:7:2> 1 NESR_MCU1HCCE: MCU1 issued a Hardware Corrected-and-Cleared Error Request 2010-07-03 18:44:14.248 0:7:2> 2010-07-03 18:44:14.296 0:7:2>Decode of Mem Error Status Reg Branch 1 bits 33044000.00000000 2010-07-03 18:44:14.427 0:7:2> 1 MEU 61 R/W1C Set to 1 on an UE if VEU = 1, or VEF = 1, or higher priority error in same cycle. 2010-07-03 18:44:14.614 0:7:2> 1 MEC 60 R/W1C Set to 1 on a CE if VEC = 1, or VEU = 1, or VEF = 1, or another error in same cycle.

42 SPARC T3-1B Server Module Service Manual • July 2012

2010-07-03 18:44:14.804 0:7:2> 1 VEU 57 R/W1C Set to 1 on an UE, if VEF = 0 and no fatal error is detected in same cycle. 2010-07-03 18:44:14.983 0:7:2> 1 VEC 56 R/W1C Set to 1 on a CE, if VEF = VEU = 0 and no fatal or UE is detected in same cycle. 2010-07-03 18:44:15.169 0:7:2> 1 DAU 50 R/W1C Set to 1 if the error was a DRAM access UE. 2010-07-03 18:44:15.304 0:7:2> 1 DAC 46 R/W1C Set to 1 if the error was a DRAM access CE. 2010-07-03 18:44:15.440 0:7:2> 2010-07-03 18:44:15.486 0:7:2> DRAM Error Address Reg for Branch 1 = 00000034.8647d2e0 2010-07-03 18:44:15.614 0:7:2> Physical Address is

00000005.d21bc0c0 2010-07-03 18:44:15.715 0:7:2> DRAM Error Location Reg for Branch 1 = 00000000.00000800 2010-07-03 18:44:15.842 0:7:2> DRAM Error Syndrome Reg for Branch 1 = dd1676ac.8c18c045 2010-07-03 18:44:15.967 0:7:2> DRAM Error Retry Reg for Branch 1 = 00000000.00000004 2010-07-03 18:44:16.086 0:7:2> DRAM Error RetrySyndrome 1 Reg for Branch 1 = a8a5f81e.f6411b5a 2010-07-03 18:44:16.218 0:7:2> DRAM Error Retry Syndrome 2 Reg for Branch 1 = a8a5f81e.f6411b5a 2010-07-03 18:44:16.351 0:7:2> DRAM Failover Location 0 for Branch 1 = 00000000.00000000 2010-07-03 18:44:16.475 0:7:2> DRAM Failover Location 1 for Branch 1 = 00000000.00000000 2010-07-03 18:44:16.604 0:7:2> 2010-07-03 18:44:16.648 0:7:2>ERROR: POST terminated prematurely. Not all system components tested. 2010-07-03 18:44:16.786 0:7:2>POST: Return to VBSC 2010-07-03 18:44:16.795 0:7:2>ERROR: 2010-07-03 18:44:16.839 0:7:2> POST toplevel status has the following failures: 2010-07-03 18:44:16.952 0:7:2> Node 0 ------------------------------2010-07-03 18:44:17.051 0:7:2> /SYS/MB/CMP0/BOB0/CH1/D0 (J1001) 2010-07-03 18:44:17.145 0:7:2> /SYS/MB/CMP0/BOB1/CH1/D0 (J3001) 2010-07-03 18:44:17.241 0:7:2>END_ERROR

Related Information

■ “Oracle ILOM Properties That Affect POST Behavior” on page 33

■ “Run POST With Maximum Testing” on page 37

■ “Clear POST-Detected Faults” on page 40

Detecting and Managing Faults 43

Managing Components (ASR Commands)

The following topics explain the role played by the Automatic System Recovery (ASR) feature and how to manage the components it controls.

■ “ASR Overview” on page 44

■ “Display System Components” on page 45

■ “Disable System Components” on page 47

■ “Enable System Components” on page 48

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Checking if Oracle VTS Software Is Installed” on page 48

ASR Overview

The ASR feature enables the server module to automatically configure failed components out of operation until they can be replaced. In the server module, ASR manages the following components:

■ CPU strands

■ Memory DIMMs

■ I/O subsystem

The database that contains the list of disabled components is referred to as the ASR blacklist (asr-db).

In most cases, POST automatically disables a faulty component. After the cause of the fault is repaired (FRU replacement, loose connector reseated, and so on), you might need to remove the component from the ASR blacklist.

44 SPARC T3-1B Server Module Service Manual • July 2012

The following ASR commands enable you to view and add or remove components (asrkeys) from the ASR blacklist. You run these commands from the Oracle ILOM

-> prompt.

TABLE: ASR Commands

Command Description

show components Displays system components and their current state.

set asrkey component_state= Enabled

set asrkey component_state= Disabled

Removes a component from the asr-db blacklist, where asrkey is the component to enable.

Adds a component to the asr-db blacklist, where asrkey is the component to disable.

Note – The asrkey values vary from system to system, depending on how many

cores and memory are present. Use the show components command to see the asrkey values on a given system.

After you enable or disable a component, you must reset (or power cycle) the system for the component’s change of state to take effect. See the SPARC T3 Series Servers

Administration Guide

Related Information

■ “Display System Components” on page 45

■ “Disable System Components” on page 47

■ “Enable System Components” on page 48

▼ Display System Components

The show components command displays the system components (asrkeys) and reports their status.

● At the -> prompt, type the show components command.

In the following example, one of the DIMMs (BOB1/CH0/D0) is shown as disabled.

->show components Target | Property | Value

--------------------+------------------------+------------------------------/SYS/MB/REM | component_state | Enabled

Detecting and Managing Faults 45

46 SPARC T3-1B Server Module Service Manual • July 2012

SWITCH1 | |

Related Information

■ “View the System Message Log Files” on page 24

■ “Disable System Components” on page 47

■ “Enable System Components” on page 48

▼ Disable System Components

You disable a component by setting its component_state property to Disabled. This adds the component to the ASR blacklist.

1. At the -> prompt, set the component_state property to Disabled.

-> set /SYS/MB/CMP0/BOB1/CH0/D0 component_state=Disabled

2. Reset the server module so that the ASR command takes effect.

-> stop /SYS Are you sure you want to stop /SYS (y/n)? y Stopping /SYS

-> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS

Note – In the Oracle ILOM shell there is no notification when the system is actually

powered off. Powering off takes about a minute. Use the show /HOST command to determine if the host has powered off.

Related Information

■ “View the System Message Log Files” on page 24

■ “Display System Components” on page 45

■ “Enable System Components” on page 48

Detecting and Managing Faults 47

▼ Enable System Components

You enable a component by setting its component_state property to Enabled. This action removes the component from the ASR blacklist.

1. At the -> prompt, set the component_state property to Enabled.

-> set /SYS/MB/CMP0/BOB1/CH0/D0 component_state=Enabled

2. Reset the server module so that the ASR command takes effect.

-> stop /SYS Are you sure you want to stop /SYS (y/n)? y Stopping /SYS

-> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS

Note – In the Oracle ILOM shell there is no notification when the system is actually

powered off. Powering off takes about a minute. Use the show /HOST command to determine if the host has powered off.

Related Information

■ “View the System Message Log Files” on page 24

■ “Display System Components” on page 45

■ “Disable System Components” on page 47

Checking if Oracle VTS Software Is Installed

Oracle VTS (previously named SunVTS) is a validation test suite that you can use to test this server module. This section provides an overview and a way to check if VTS is installed. For comprehensive VTS information, refer to the Oracle VTS 7.0 documentation.

■ “Oracle VTS Overview” on page 49

■ “Check if Oracle VTS Software Is Installed” on page 50

48 SPARC T3-1B Server Module Service Manual • July 2012

Related Information

■ “Diagnostics Overview” on page 5

■ “Diagnostics Process” on page 7

■ “Managing Faults (Oracle ILOM)” on page 12

■ “Interpreting Log Files and System Messages” on page 23

■ “Managing Faults (Oracle Solaris PSH)” on page 25

■ “Managing Faults (POST)” on page 31

■ “Managing Components (ASR Commands)” on page 44

Oracle VTS Overview

VTS is a validation test suite that you can use to test this server module. VTS provides multiple diagnostic hardware tests that verify the connectivity and functionality of most hardware controllers and devices for this server module. VTS provides these kinds of test categories:

■ Audio

■ Communication (serial and parallel)

■ Graphic and video

■ Memory

■ Network

■ Peripherals (hard disk drives, CD-DVD devices, and printers)

■ Processor

■ Storage

Use VTS to validate a system during development, production, receiving inspection, troubleshooting, periodic maintenance, and system or subsystem stressing.

You can run VTS through a browser UI, terminal UI, or command UI.

You can run tests in a variety of modes for online and offline testing.

VTS also provides a choice of security mechanisms.

VTS software is provided in the preinstalled Oracle Solaris OS that shipped with the server module.

Related Information

■ Oracle VTS documentation

■ “Check if Oracle VTS Software Is Installed” on page 50

Detecting and Managing Faults 49

▼ Check if Oracle VTS Software Is Installed

1. Log in as superuser.

2. Check for the presence of VTS packages:.

# pkginfo -l SUNWvts SUNWvtsr SUNWvtsts SUNWvtsmn

■ If information about the packages is displayed, then VTS software is installed.

■ If you receive messages reporting ERROR: information for package was

not found, then VTS is not installed. You must take action to install the software before you can use it. You can obtain the VTS software from the following places:

■ Oracle Solaris OS media kit (DVDs)

■ As a download from the web

Related Information

■ Oracle VTS documentation

50 SPARC T3-1B Server Module Service Manual • July 2012

Preparing for Service

The following topics describe how to prepare the server module for servicing.

■ “General Safety Information” on page 51

■ “Tools Needed for Service” on page 53

■ “Find the Modular System Serial Number” on page 53

■ “Find the Server Module Serial Number” on page 54

■ “Locate the Server Module” on page 55

■ “Removing the Server Module From the Modular System for Service” on page 55

■ “Remove the Server Module From the Modular System” on page 59

■ “Remove the Cover” on page 62

Related Information

■ “Returning the Server Module to Operation” on page 111

General Safety Information

For your protection, observe the following safety precautions when setting up your equipment:

■ Follow all cautions and instructions marked on the equipment.

■ Follow all cautions and instructions described in the documentation that shipped

with your system and in the SPARC T3-1B Server Module Safety and Compliance Guide.

■ Ensure that the voltage and frequency of your power source match the voltage

and frequency inscribed on the equipment’s electrical rating label.

■ Follow the electrostatic discharge safety practices as described in this section.

Safety Symbols

You will see the following symbols in various places in the server module documentation. Note the explanations provided next to each symbol.

Caution – There is a risk of personal injury or equipment damage. To avoid

personal injury and equipment damage, follow the instructions.

Caution – Hot surface. Avoid contact. Surfaces are hot and might cause personal

injury if touched.

Caution – Hazardous voltages are present. To reduce the risk of electric shock and

danger to personal health, follow the instructions.

ESD Safety Measures

ESD sensitive devices, such as the motherboard, cards, hard drives, and DIMMs require special handling.

Caution – Circuit boards and hard drives contain electronic components that are

extremely sensitive to static electricity. Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards. Do not touch the components along their connector edges.

Antistatic Wrist Strap Use

Wear an antistatic wrist strap and use an antistatic mat when handling components such as hard drive assemblies, circuit boards, or PCI cards. When servicing or removing server module components, attach an antistatic strap to your wrist and then to a metal area on the chassis. Following this practice equalizes the electrical potentials between you and the server module.

Antistatic Mat

Place ESD-sensitive components such as cards and DIMMs on an antistatic mat.

52 SPARC T3-1B Server Module Service Manual • July 2012

Related Information

■ “Returning the Server Module to Operation” on page 111

Tools Needed for Service

The following tools are required for service procedures:

■ Antistatic wrist strap

■ Antistatic mat

■ Stylus or pencil (to operate the power button)

■ UCP-3 dongle (UCP-4 dongle can be used, but see instructions in the SPARC T3-1B

Server Module Installation Guide)

■ Blade filler panel

Related Information

■ “General Safety Information” on page 51

▼ Find the Modular System Serial

Number

To obtain support for your server module, you need the serial number of the Sun Blade 6000 modular system in which the server module is located, not the serial number of the server module. The serial number of the modular system is provided on a label on the upper left edge of the front bezel.

Use the following procedure to obtain the serial number remotely.

1. Log in to the CMM of the modular system.

See the documentation for the Sun Blade 6000 modular system.

2. Type

-> show /CH

Preparing for Service 53

3. In the output, locate the value for product_serial_number.

That number is the serial number of the modular system.

Related Information

■ “Find the Server Module Serial Number” on page 54

■ “Locate the Server Module” on page 55

▼ Find the Server Module Serial Number

Note – To obtain support for your server module, you need the serial number of the

Sun Blade 6000 modular system in which the server module is located, not the serial number of the server module. See “Find the Modular System Serial Number” on

page 53.

The serial number of the server module is located on a sticker on the RFID mounted in the center of the front panel. However, this label is not present on a system that has been moved into a new enclosure assembly. You also can type the Oracle ILOM show /SYS command to display the number.

● Access the Oracle ILOM CLI and type:

-> show /SYS

/SYS

Targets:

SERVICE LOCATE ACT PS_FAULT TEMP_FAULT FAN_FAULT

...

Properties:

type = Host System keyswitch_state = Normal product_name = SPARC T3-1B product_serial_number = 0723BBC006 <- fault_state = OK clear_fault_action = (none) power_state = On

54 SPARC T3-1B Server Module Service Manual • July 2012

Related Information

■ “Locate the Server Module” on page 55

■ “Find the Modular System Serial Number” on page 53

▼ Locate the Server Module

To identify a specific server module from others in the modular system, perform the following steps.

1. Log in to Oracle ILOM on the server module you plan to locate.

2. Type:

-> set /SYS/LOCATE value=fast_blink

The Locator LED on the server module blinks.

3. Identify the server module with a blinking white LED.

4. Once you locate the server module, press the Locator LED to turn it off.

Note – Alternatively, you can turn off the Locator LED by typing the Oracle ILOM

set /SYS/LOCATE value=off command.

Related Information

■ “Remove the Server Module From the Modular System” on page 59

Removing the Server Module From the Modular System for Service

Perform the following tasks:

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Prepare the Server Module for Removal” on page 58

■ “Remove the Server Module From the Modular System” on page 59

Preparing for Service 55

■ “Remove the Cover” on page 62

Related Information

■ “Install the Server Module Into the Modular System” on page 112

▼ Shut Down the Oracle Solaris OS

This topic describes one method for shutting down the Oracle Solaris OS. For information on other ways to shut down the Oracle Solaris OS, refer to the Oracle Solaris OS documentation.

1. Log in as superuser or equivalent.

Depending on the type of problem, you might want to view server module status or log files. You also might want to run diagnostics before you shut down the server module.

2. Notify affected users that the server module will be shut down.

Refer to the Oracle Solaris system administration documentation for additional information.

3. Save any open files and quit all running programs.

Refer to the application documentation for specific information on these processes.

4. (If applicable) Shut down all logical domains.

Refer to the Oracle Solaris system administration and Oracle VM Manager for SPARC documentation for additional information.

5. Shut down the Oracle Solaris OS and reach the ok prompt.

Refer to the Oracle Solaris system administration documentation for additional information.

The following example uses the Oracle Solaris shutdown command:

# shutdown -g0 -i0 -y

Shutdown started. Tue Jun 28 13:06:20 PDT 2010

Changing to init state 0 - please wait Broadcast Message from root (console) on server1 Tue Jun 28 13:06:20... THE SYSTEM server1 IS BEING SHUT DOWN NOW ! ! ! Log off now or risk your files being damaged # svc.startd: The system is coming down. Please wait. svc.startd: 100 system services are now being stopped.

56 SPARC T3-1B Server Module Service Manual • July 2012

Jun 28 13:06:34 dt90-366 syslogd: going down on signal 15 svc.startd: The system is down. syncing file systems... done Program terminated

SPARC T3-1B, No Keyboard OpenBoot 4.30, 16256 MB memory available, Serial # 87305111. Ethernet address 0:21:28:34:2b:90, Host ID: 85342b90. {0} ok

6. Switch from the system console to the -> prompt by typing the #. (Hash Period) key sequence.

7. At the -> prompt, type:

->stop /SYS

Note – You can also use the Power button on the front of the server module to

initiate a graceful shutdown. (See “Power Off the Server Module (Power Button -

Standby Mode)” on page 57. This button is recessed to prevent accidental server

module power-off. Use the tip of a pen or other stylus to operate this button.

Related Information

■ “Power Off the Server Module (Power Button - Standby Mode)” on page 57

■ “Power Off the Server Module (Emergency Shutdown)” on page 58

■ “Prepare the Server Module for Removal” on page 58

▼ Power Off the Server Module (Power Button -

Standby Mode)

This procedure places the server module in the power standby mode. In this mode, the Power OK LED blinks rapidly.

● Press and release the recessed Power button.

Use a stylus or the tip of a pen to operate this button. See “Front and Rear Panel

Components” on page 2.

Related Information

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Power Off the Server Module (Emergency Shutdown)” on page 58

Preparing for Service 57

■ “Prepare the Server Module for Removal” on page 58

▼ Power Off the Server Module (Emergency

Shutdown)

Caution – All applications and files will be closed abruptly without saving changes.

File system corruption might occur.

● Press and hold the Power button for four seconds.

Use a stylus or the tip of a pen to operate this button.

Related Information

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Power Off the Server Module (Power Button - Standby Mode)” on page 57

■ “Prepare the Server Module for Removal” on page 58

▼ Prepare the Server Module for Removal

1. Log in to Oracle ILOM on the server module you plan to remove.

2. Ensure the server module is in standby mode, with the host powered off. Type:

-> show /SYS/ power_state

/SYS

properties:

power_state = Off

If you do not see this message, check that you have performed all the steps in

“Shut Down the Oracle Solaris OS” on page 56

3. Type:

-> set /SYS/ prepare_to_remove_action=true Set ‘prepare_to_remove_action’ to ‘true’

The server module is in standby mode. Power is removed from the host while standby power is applied to the SP.

58 SPARC T3-1B Server Module Service Manual • July 2012

4. Confirm that the server module is in standby mode by viewing the blue Ready to Remove LED on the front of the server module.

See “Front and Rear Panel Components” on page 2 to locate this LED. If the Ready to Remove LED is on, the server module is ready for removal from the modular system chassis.

Related Information

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Power Off the Server Module (Power Button - Standby Mode)” on page 57

■ “Power Off the Server Module (Emergency Shutdown)” on page 58

▼ Remove the Server Module From the Modular

System

Before performing this task, review the following cautions.

Caution – A server module can weigh as much as 17 pounds (8.0 kg). During

removal, hold the server module firmly with both hands.

Caution – Do not stack server modules higher than five units tall.

Caution – Insert a filler panel into the empty server module slot within 60 seconds

after removing a server module ensure proper modular system chassis cooling.

1. If a cable is connected to the front of the server module, disconnect it.

Press the buttons on either side of the UCP to release the connector.

Preparing for Service 59

2. Open both ejector arms (panel 2).

Squeeze both latches on each of the two ejector arms.

60 SPARC T3-1B Server Module Service Manual • July 2012

3. Pull the server module halfway out (panel 3).

4. Close the ejector arms.

5. Remove the server module from the modular system.

Lift the server module with two hands.

6. Place the server module on an antistatic mat or surface.

7. Insert a filler panel into the empty chassis slot.

Note – When the modular system is operating, you must fill every slot with a filler

panel or a server module within 60 seconds.

Related Information

■ “Remove the Cover” on page 62

■ “Install the Server Module Into the Modular System” on page 112

Preparing for Service 61

▼ Remove the Cover

1. Attach an antistatic strap to your wrist and then to a metal area on the server module.

2. While pressing the cover release button, slide the cover toward the rear of the server module about half an inch (1 cm).

3. Lift the cover off the server module chassis.

Related Information

■ “Illustrated Parts Breakdown” on page 3

■ “Replace the Cover” on page 111

62 SPARC T3-1B Server Module Service Manual • July 2012

Servicing Hard Drives

The following topics apply to hard drives installed in the external slots of the server module.

Description Links

Determine if you can remove and replace a drive using hot-plugging capabilities.

Replace a drive. “Remove a Drive” on page 64

Add an additional drive to the server module.

Install a drive filler. “Install a Drive Filler” on page 67

Related Information

■ “Detecting and Managing Faults” on page 5

■ “Preparing for Service” on page 51

“Drive Hot-Plugging Rules” on page 63

“Replace or Add a Drive” on page 65

“Remove a Drive Filler” on page 67

“Replace or Add a Drive” on page 65

Drive Hot-Plugging Rules

To safely remove a hard drive, you must:

■ Prevent any applications from accessing the drive.

■ Remove the logical software links.

Drives cannot be hot-plugged if:

■ The drive provides the operating system, and the operating system is not mirrored

on another drive.

■ The drive cannot be logically isolated from the online operations of the server

module.

If your drive falls into these conditions, you must shut down the Oracle Solaris OS before you replace the drive.

Related Information

■ “Remove a Drive” on page 64

■ “Replace or Add a Drive” on page 65

■ “Shut Down the Oracle Solaris OS” on page 56

▼ Remove a Drive

1. Identify the drive you plan to remove.

Use the Ready to Remove indicator. See “Diagnostics LEDs” on page 10.

2. Prepare the drive for removal by performing one of the following:

■ Take the drive offline.

The exact commands required to take the drive offline depend on the configuration of your drives. For example, you might need to unmount file systems or perform certain RAID commands.

One command that is commonly used to take a drive offline is the cfgadm command. For more information, refer to the Solaris cfgadm man page.

■ Shut down the Solaris OS.

If the drive cannot be taken offline, shut down the Solaris OS on the server module. See “Shut Down the Oracle Solaris OS” on page 56

3. Verify whether the blue Drive Ready LED is illuminated on the front of the drive.

See “Diagnostics LEDs” on page 10. The blue LED will only be illuminated if the drive was taken offline using cfgadm or an equivalent command. It will not be illuminated if Oracle Solaris was shut down.

4. Remove the drive as described in the following steps:

a. Push the latch release button on the drive (panels 1 and 2).

64 SPARC T3-1B Server Module Service Manual • July 2012

b. Grasp the latch and pull the drive out of the drive slot (panel 3).

5. Insert a drive filler if you are not replacing the drive in this slot.

See “Install a Drive Filler” on page 67

Related Information

■ “Install a Drive Filler” on page 67

■ “Replace or Add a Drive” on page 65

▼ Replace or Add a Drive

The physical address of a hard drive is based on he hard drive is physically addressed based on the slot in which it is installed.

1. Identify the slot in which to install the drive.

■ If you are replacing a drive, ensure that you install the replacement drive in the

same slot as the drive you removed.

■ If you are adding an additional drive, install the drive in the next available

drive slot.

Servicing Hard Drives 65

2. If necessary, remove the drive filler from this slot.

See “Remove a Drive Filler” on page 67.

3. Slide the drive into the bay until it is fully seated (panel 1).

4. Close the latch to lock the drive in place (panels 2 and 3).

5. Perform administrative tasks to reconfigure the drive.

The procedures that you perform at this point depend on how your data is configured. You might need to partition the drive, create file systems, load data from backups, or have data updated from a RAID configuration.

The following commands might apply to your circumstances:

■ You can use the Solaris command cfgadm -al to list all disks in the device

tree, including unconfigured disks.

■ If the disk is not in the list, such as with a newly installed disk, you can use

devfsadm to configure it into the tree. See the devfsadm man page for details.

Related Information

■ “Remove a Drive” on page 64

66 SPARC T3-1B Server Module Service Manual • July 2012

▼ Remove a Drive Filler

All drive bays must be populated by either a drive or a filler.

1. Open the filler lever (panels 1 and 2).

2. Pull to remove the filler (panel 3).

Related Information

■ “Replace or Add a Drive” on page 65

■ “Install a Drive Filler” on page 67

▼ Install a Drive Filler

All drive bays must be populated by either a drive or a filler.

Servicing Hard Drives 67

1. Extend the filler handle, then align the filler to the empty drive bay (panel 1).

2. Push the filler into place.

3. Close the filler lever (panels 2 and 3).

Related Information

■ “Remove a Drive” on page 64

■ “Remove a Drive Filler” on page 67

68 SPARC T3-1B Server Module Service Manual • July 2012

Servicing Memory

The following topics describe how to determine which DIMMs are faulty, remove DIMMs, install DIMMs, and verify DIMM functionality after installation.

Description Links

Understand memory faults. “Memory Faults” on page 69

Replace a faulty DIMM. “Locate a Faulty DIMM (LEDs)” on page 70

“Remove a DIMM” on page 73

“Locate a Faulty DIMM (LEDs)” on page 70

“Install a Replacement DIMM” on page 74

“Clear the Fault and Verify the Functionality of the Replacement DIMM” on page 75

Add memory to the server module. “Install a Replacement DIMM” on page 74

“Verify DIMM Functionality” on page 79

“DIMM Configuration Reference” on page 81

Related Information

■ “Detecting and Managing Faults” on page 5

■ “Preparing for Service” on page 51

Memory Faults

A variety of features play a role in how the memory subsystem is configured and how memory faults are handled. Understanding the underlying features helps you identify and repair memory problems. This topic describes how the server module deals with memory faults.

The server module uses advanced ECC technology that corrects up to 4 bits in error on nibble boundaries, as long as the bits are all in the same DRAM. On some DIMMs, if a DRAM fails, the DIMM continues to function.

The following server module features independently manage memory faults:

■ POST – Based on Oracle ILOM configuration variables, POST runs when the

server module is powered on.

For correctable memory errors (sometimes called CEs), POST forwards the error to the Oracle Solaris PSH daemon for error handling.

If an uncorrectable memory fault is detected, POST displays the fault with the device name of the faulty DIMMs, and logs the fault. POST then disables the faulty DIMMs. Depending on the memory configuration and the location of the faulty DIMM, POST disables half of physical memory in the system, or half the physical memory and half the processor threads. When the offlining process occurs in normal operation, you must replace the faulty DIMMs based on the fault message and then enable the disabled DIMMs. See “Clear the Fault and Verify the

Functionality of the Replacement DIMM” on page 75.

■ Oracle Solaris PSH technology – A feature of the Solaris OS, PSH uses the fault

manager daemon (fmd) to watch for various kinds of faults. When a fault occurs, the fault is assigned a UUID and logged. PSH reports the fault and suggests a replacement for the DIMMs associated with the fault.

If you suspect that the server module has a memory problem, follow the

“Diagnostics Process” on page 7. The flowchart helps you determine if the memory

problem was detected by POST or by the PSH technology.

Once you identify which DIMMs you want to replace, see “Locate a Faulty DIMM

(LEDs)” on page 70. After replacing a faulty DIMM, You must perform the

instructions in “Clear the Fault and Verify the Functionality of the Replacement

DIMM” on page 75.

Related Information

■ “Locate a Faulty DIMM (LEDs)” on page 70

■ “Clear the Fault and Verify the Functionality of the Replacement DIMM” on

page 75

■ “Clear the Fault and Verify the Functionality of the Replacement DIMM” on

page 75

■ “Detecting and Managing Faults” on page 5

▼ Locate a Faulty DIMM (LEDs)

This procedure describes how to use the DIMM LEDs on the motherboard to pinpoint the physical location of a faulty DIMM.

70 SPARC T3-1B Server Module Service Manual • July 2012

Note – You can also obtain the location of the faulty DIMM using the Oracle ILOM

show faulty command. This command displays the FRU name (such as /SYS/MB/CMP0/BOB0/CH0). Use the FRU name and information to locate the faulty

DIMM. See “DIMM Configuration Reference” on page 81.

1. Check the front panel Fault LED.

See “Diagnostics LEDs” on page 10.

When a faulty DIMM is detected, the front panel Fault LED and the motherboard DIMM Fault LEDs are illuminated. Before opening the server module to check the DIMM Fault LEDs, verify that the System Fault LED is lit.

■ If the System Fault LED is not lit, and you suspect there is a problem, see

“Diagnostics Process” on page 7.

■ If the System Fault LED is lit, go to the next step.

2. (If needed) Prepare for service. See:

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Prepare the Server Module for Removal” on page 58

■ “Remove the Server Module From the Modular System” on page 59

■ “Remove the Cover” on page 62

■ “ESD Safety Measures” on page 52

3. Press the Remind button on the motherboard.

While the Remind button is pressed, an LED next to the faulty DIMM illuminates, enabling you to identify the faulty DIMM.

Tip – The DIMM Fault LEDs are small and difficult to identify when they are not

illuminated. If you do not see any illuminated LEDs in the area of the DIMM LEDs, assume that the DIMMs are not faulty.

Servicing Memory 71

FIGURE: Locating Faulty DIMMs

Figure Leg end

1 DIMM 1 (BOB0/CH1/D0)

2 Fault LED for DIMM 1

3 Locate button for LEDs of faulty DIMMs

4. Remove the faulty DIMM.

See “Remove a DIMM” on page 73.

Related Information

■ “DIMM Configuration Reference” on page 81

■ “Remove a DIMM” on page 73

72 SPARC T3-1B Server Module Service Manual • July 2012

▼ Remove a DIMM

Caution – This procedure involves handling circuit boards that are extremely

sensitive to static electricity. Ensure that you follow ESD preventative practices to avoid damaging the circuit boards.

Caution – Components inside the chassis might be hot. Use caution when servicing

components inside the chassis.

1. (If needed) Prepare for service. See:

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Prepare the Server Module for Removal” on page 58

■ “Remove the Server Module From the Modular System” on page 59

■ “Remove the Cover” on page 62

■ “ESD Safety Measures” on page 52

2. (If needed) Locate the faulty DIMM.

See “Locate a Faulty DIMM (LEDs)” on page 70.

3. Remove the DIMM from the motherboard as described in the following steps:

a. Push down on the ejector tabs on each side of the DIMM until the DIMM is

released (panel 1).

b. Grasp the top corners of the DIMM, and lift and remove it from the server

module (panel 2).

c. Place the DIMM on an antistatic mat.

4. Install a replacement DIMM.

See “Install a Replacement DIMM” on page 74.

Servicing Memory 73

Related Information

■ “Install a Replacement DIMM” on page 74

■ “DIMM Configuration Reference” on page 81

▼ Install a Replacement DIMM

Caution – This procedure involves handling circuit boards that are extremely

sensitive to static electricity. Ensure that you follow ESD preventative practices to avoid damaging the circuit boards.

Caution – Components inside the chassis might be hot. Use caution when servicing

components inside the chassis.

1. (If needed) Prepare the server module for service and remove the faulty DIMM.

See “Remove a DIMM” on page 73.

2. Unpackage the replacement DIMM and set it on an antistatic mat.

3. Ensure that the DIMM ejector tabs are in the open position (panel 1).

4. Line up the replacement DIMM with the connector.

Align the DIMM notch with the key in the connector, as in panel 3. This action ensures that the DIMM is oriented correctly. Panel 2 shows an incorrect alignment.

74 SPARC T3-1B Server Module Service Manual • July 2012

5. Push the DIMM into the connector until the ejector tabs lock the DIMM in place.

If the DIMM does not easily seat into the connector, verify that the orientation of the DIMM is correct. Never apply excessive force.

6. Return the server module to operation.

See “Returning the Server Module to Operation” on page 111

7. Perform one of the following tasks to verify the DIMM:

■ Verify a replacement DIMM. See “Clear the Fault and Verify the Functionality

of the Replacement DIMM” on page 75.

■ Verify additional memory. See “Verify DIMM Functionality” on page 79.

Related Information

■ “Remove a DIMM” on page 73

■ “DIMM Configuration Reference” on page 81

▼ Clear the Fault and Verify the

Functionality of the Replacement DIMM

This procedure describes how to clear a memory fault and how to verify the functionality of the replacement DIMM.

Ensure that the following conditions are met:

■ The server module is in Standby mode (installed in a powered modular system,

but the server module’s host is not started).

See “Prepare the Server Module for Removal” on page 58.

■ You have connectivity to the SP.

See “Access the SP (Oracle ILOM)” on page 15

1. Access the Oracle ILOM -> prompt.

See “Access the SP (Oracle ILOM)” on page 15.

Servicing Memory 75

2. Determine how to clear the fault.

The method you use to clear a fault depends on how the fault is identified by the show faulty command.

Examples:

■ If the fault is a host-detected fault (displays a UUID), continue to Step 3. For

example:

-> show faulty Target | Property | Value

■ If the fault was detected by POST and resulted in the DIMM being disabled,

you will see something similar to the following output:

-> show faulty Target | Property | Value

In most cases, the replacement of the faulty DIMM is detected when the SP is power cycled. In this case, the fault is automatically cleared from the system. If the fault is still displayed by the show faulty command, then use the set command to enable the DIMM and clear the fault.

Example:

-> set /SYS/MB/CMP0/BOB0/CH0/D0 component_state=Enabled

3. Perform the following steps to verify the repair:

76 SPARC T3-1B Server Module Service Manual • July 2012

a. Set the virtual keyswitch to diag so that POST will run in Service mode.

-> set /SYS/keyswitch_state=Diag Set ‘keyswitch_state’ to ‘Diag’

b. Power cycle the system.

-> stop /SYS Are you sure you want to stop /SYS (y/n)? y Stopping /SYS

-> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS

Note – The server module takes about one minute to power off. Use the show

/HOST command to determine when the host has been powered off. The console will display status=Powered Off.

c. Switch to the system console to view POST output.

-> start /HOST/console

Watch the POST output for possible fault messages. The following output is a sign that POST did not detect any faults:

. . . 0:0:0>INFO: 0:0:0> POST Passed all devices. 0:0:0>POST: Return to VBSC. 0:0:0>Master set ACK for vbsc runpost command and spin...

Note – Depending on the configuration of Oracle ILOM variables that affect POST

and whether POST detected faults or not, the system might boot, or the system might remain at the ok prompt. If the system is at the ok prompt, type boot.

d. Return the virtual keyswitch to Normal mode.

-> set /SYS keyswitch_state=Normal Set ‘ketswitch_state’ to ‘Normal’

Servicing Memory 77

e. Switch to the system console and type the Solaris OS fmadm faulty

command.

# fmadm faulty

No memory faults should be displayed.

If faults are reported, refer to the “Diagnostics Process” on page 7 for an approach to diagnose the fault.

4. Switch to the Oracle ILOM prompt (->).

5. Type the show faulty command.

■ If the fault was detected by the host and the fault information persists, the

output will be similar to the following example:

-> show faulty Target | Property | Value

■ If the show faulty command does not report a fault with a UUID, the fault is

cleared. You do not need to proceed with the following steps.

6. (Only if previous steps did not clear the fault) Type the set command.

-> set /SYS/MB/CMP0/BOB0/CH0/D0 clear_fault_action=true Are you sure you want to clear /SYS/MB/CMP0/BOB0/CH0/D0 (y/n)? y Set ’clear_fault_action’ to ’true’

7. (Only if previous steps did not clear the fault) Switch to the system console and

type the fmadm repair command with the UUID.

Use the same UUID that was displayed from the output of the Oracle ILOM show faulty command.

# fmadm repair 3aa7c854-9667-e176-efe5-e487e5207a8a

Related Information

■ “Install a Replacement DIMM” on page 74

78 SPARC T3-1B Server Module Service Manual • July 2012

■ “Verify DIMM Functionality” on page 79

▼ Verify DIMM Functionality

1. Access the Oracle ILOM -> prompt.

Refer to the SPARC T3 Series Servers Administration Guide for instructions.

2. Use the show faulty command to determine how to clear the fault.

■ If show faulty indicates a POST-detected fault, go to Step 3.

■ If show faulty output displays a UUID, which indicates a host-detected fault,

skip Step 3 and go to Step 4.

3. Use the set command to enable the DIMM that was disabled by POST.

In most cases, replacement of a faulty DIMM is detected when the SP is power cycled. In those cases, the fault is automatically cleared from the system. If show

faulty still displays the fault, the set command will clear it.

-> set /SYS/MB/CMP0/BOB0/CH0/D0 component_state=Enabled

4. For a host-detected fault, perform the following steps to verify the new DIMM:

a. Set the virtual keyswitch to diag so that POST will run in Service mode.

-> set /SYS/keyswitch_state=Diag Set ‘keyswitch_state’ to ‘Diag’

b. Power cycle the server module host.

-> stop /SYS Are you sure you want to stop /SYS (y/n)? y Stopping /SYS

-> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS

Note – Use the show /HOST command to determine when the host has been

powered off. The console will display status=Powered Off. Allow approximately one minute before running this command.

Servicing Memory 79

c. Switch to the system console to view POST output.

Watch the POST output for possible fault messages. The following output indicates that POST did not detect any faults:

-> start /HOST/console . . . 0:7:2>INFO: 0:7:2> POST Passed all devices. 0:7:2>POST: Return to VBSC. 0:7:2>Master set ACK for vbsc runpost command and spin...

Note – The system might boot automatically at this point. If so, go directly to Step e.

If it remains at the ok prompt go to Step d.

d. If the server module remains at the ok prompt, type boot.

e. Return the virtual keyswitch to Normal mode.

-> set /SYS keyswitch_state=Normal Set ‘ketswitch_state’ to ‘Normal’

f. Switch to the system console and type the Oracle Solaris OS fmadm faulty

command.

# fmadm faulty

If any faults are reported, see the diagnostics instructions in “Oracle ILOM

Troubleshooting Overview” on page 12.

5. Switch to the Oracle ILOM command shell.

80 SPARC T3-1B Server Module Service Manual • July 2012

6. Type the show faulty command.

-> show faulty Target | Property | Value

If the show faulty command reports a fault with a UUID go to Step 7.Ifshow faulty does not report a fault with a UUID, you have completed the verification

process.

7. Switch to the system console and type the fmadm repair command with the UUID.

Use the same UUID that was displayed from the output of the Oracle ILOM show faulty command.

# fmadm repair 3aa7c854-9667-e176-efe5-e487e520

Related Information

■ “Remove a DIMM” on page 73

■ “Install a Replacement DIMM” on page 74

■ “DIMM Configuration Reference” on page 81

DIMM Configuration Reference

This topic provides configuration guidelines, and the relationships between the DIMM physical locations and FRU names.

DIMM configuration guidelines:

■ There are 16 DIMM slots that support industry-standard DIMMs.

■ You can install quantities of 4, 8, or 16 DIMMs.

■ Supported DIMM capacities: 2 Gbyte, 4 Gbyte, and 8 Gbyte.

Refer to the SPARC T3-1B Server Module Product Notes for the latest information.

Servicing Memory 81

■ All DIMMs in the server module must be the same capacity.

FIGURE: DIMM Slot Locations

Figure Leg end

1 DIMM slots controlled by BOB0

2 DIMM slots controlled by BOB1

3 DIMM slots controlled by BOB3

4 DIMM slots controlled by BOB2

5 Fault remind button

6 Memory fault LED for the adjacent DIMM

82 SPARC T3-1B Server Module Service Manual • July 2012

The slots are color coded to indicate which slots to use to install different quantities of DIMMs.

■ 4 DIMMs: Blue slots

■ 8 DIMMs: White and blue slots

■ 16 DIMMs: Black, white, and blue slots

The following table summarizes details on using each of the 16 DIMM slots.

DIMM Location

Slot Color

Slot Is Used For This Quantity of DIMMs

FRU Name (all start with /SYS/MB/CMP0/)

1 Blue 4, 8, 16 BOB0/CH1/D0

2 Black 16 BOB0/CH1/D1

3 White 8, 16 BOB0/CH0/D0

4 Black 16 BOB0/CH0/D1

5 Black 16 BOB1/CH0/D1

6 White 8, 16 BOB1/CH0/D0

7 Black 16 BOB1/CH1/D1

8 Blue 4, 8, 16 BOB1/CH1/D0

9 Blue 4, 8, 16 BOB3/CH1/D0

10 Black 16 BOB3/CH1/D1

11 White 8, 16 BOB3/CH0/D0

12 Black 16 BOB3/CH0/D1

13 Black 16 BOB2/CH0/D1

14 White 8, 16 BOB2/CH0/D0

15 Black 16 BOB2/CH1/D1

16 Blue 4, 8, 16 BOB2/CH1/D0

* From top to bottomwhen viewed with the frontpanel of the server moduleon your left.

Related Information

■ “Memory Faults” on page 69

■ “Locate a Faulty DIMM (LEDs)” on page 70

■ “Remove a DIMM” on page 73

■ “Install a Replacement DIMM” on page 74

Servicing Memory 83

■ “Clear the Fault and Verify the Functionality of the Replacement DIMM” on

page 75

84 SPARC T3-1B Server Module Service Manual • July 2012

Servicing a REM

The server module support the installation of one REM. Only certain REMs are supported. For a list of supported REMs, refer to the SPARC T3-1B Server Module Product Notes.

Description Links

Replace a REM. “Remove a REM” on page 85

“Install a REM” on page 86

Install a REM. “Install a REM” on page 86

Related Information

■ “Detecting and Managing Faults” on page 5

■ “Preparing for Service” on page 51

▼ Remove a REM

1. Prepare for service. See:

■ “Shut Down the Oracle Solaris OS” on page 56

■ “Prepare the Server Module for Removal” on page 58

■ “Remove the Server Module From the Modular System” on page 59

■ “Remove the Cover” on page 62

■ “ESD Safety Measures” on page 52

2. Lift up on the REM lever (panel 1).

3. Rotate the card up and off the retainer (panels 2 and 3).

4. Set the card on an antistatic surface.

5. Install a REM.

See “Install a REM” on page 86.

Related Information

■ “Install a REM” on page 86

▼ Install a REM

This task describes how to install a REM onto the server module. For information about specific configuration tasks for your REM, refer to the REM documentation.

1. Prepare for service by performing the following tasks:

■ “Shut Down the Oracle Solaris OS” on page 56

86 SPARC T3-1B Server Module Service Manual • July 2012

■ “Prepare the Server Module for Removal” on page 58

■ “Remove the Server Module From the Modular System” on page 59

■ “Remove the Cover” on page 62

■ “ESD Safety Measures” on page 52

■ (If needed) “Remove a REM” on page 85

2. Align the REM for installation (panel 1)

3. Slide the end of the REM that is opposite the connector under the tabs of the plastic standoff (panel 2).

4. Press the REM until the connector is fully seated on the motherboard (panel 3).

If there is a rubber bumper on the REM, you can press down on it directly to seat the connector.

5. Return the server module to operation.

See “Returning the Server Module to Operation” on page 111.

6. Configure or verify the RAID after installing the REM.

Refer to the SPARC T3 Series Servers Administration Guide for information about RAID configuration on this server module.

Servicing a REM 87

Related Information

■ “Remove a REM” on page 85

88 SPARC T3-1B Server Module Service Manual • July 2012

Sun Oracle SPARC T3-1B Service Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Contents

Using This Documentation

Identifying Components

Front and Rear Panel Components

Illustrated Parts Breakdown

Detecting and Managing Faults

Diagnostics Overview

Diagnostics Process

Diagnostics LEDs

Managing Faults (Oracle ILOM)

Oracle ILOM Troubleshooting Overview

Fault Management

Fault Clearing

Oracle Solaris Fault Manager Commands in Oracle ILOM

HDD Faults

▼ Access the SP (Oracle ILOM)

Related Information

Related Information

Related Information

Related Information

Related Information

Service-Related Oracle ILOM Command Summary

Interpreting Log Files and System Messages

Related Information

▼ View the System Message Log Files

Related Information

Related Information

Managing Faults (Oracle Solaris PSH)

Oracle Solaris PSH Technology Overview

PSH-Detected Fault Example

▼ Check for PSH-Detected Faults

Related Information

▼ Clear PSH-Detected Faults

Related Information

Managing Faults (POST)

POST Overview

Oracle ILOM Properties That Affect POST Behavior

▼ Configure How POST Runs

Related Information

▼ Run POST With Maximum Testing

Related Information

▼ Interpret POST Fault Messages

Related Information

▼ Clear POST-Detected Faults

Related Information

POST Error Message Syntax

Managing Components (ASR Commands)

ASR Overview

▼ Display System Components

Related Information

▼ Disable System Components

Related Information

▼ Enable System Components

Related Information

Checking if Oracle VTS Software Is Installed

Oracle VTS Overview

▼ Check if Oracle VTS Software Is Installed

Related Information

Preparing for Service

General Safety Information

Safety Symbols

ESD Safety Measures

Antistatic Wrist Strap Use

Antistatic Mat

Tools Needed for Service

Related Information

▼ Find the Server Module Serial Number

Related Information

▼ Locate the Server Module

Related Information

Removing the Server Module From the Modular System for Service

▼ Shut Down the Oracle Solaris OS

Related Information

Related Information

Related Information

▼ Prepare the Server Module for Removal