IBM Power System 8335-GTW User Manual

Power Systems

Servicing the 8335-GTW

IBM

Note

Before using this information and the product it supports, read the information in “Safety notices” on page v, “Notices” on page 221, the IBM Systems Safety Notices manual, G229-9054, and the IBM Environmental Notices and User Guide, Z125–5823.

This edition applies to IBM® Power Systems servers that contain the POWER9™ processor and to all associated models.

US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Safety notices........................................................................................................v

Removing and replacing parts in the 8335-GTW or 8335-GTX................................. 1

BMC card...................................................................................................................................................... 3

Removing the BMC card......................................................................................................................... 3

Replacing the BMC card......................................................................................................................... 4

Cold plates....................................................................................................................................................7

Removing the cold plates from the system........................................................................................... 7

Replacing the cold plates in the cold plate installation carrier...........................................................32

Replacing the cold plates in the system..............................................................................................45

Disk drive....................................................................................................................................................64

Removing a disk drive.......................................................................................................................... 65

Replacing a disk drive...........................................................................................................................67

Disk drive and fan card.............................................................................................................................. 68

Removing the disk drive and fan card..................................................................................................68

Replacing the disk drive and fan card..................................................................................................74

Disk and fan signal cable........................................................................................................................... 80

Removing the disk and fan signal cable.............................................................................................. 81

Replacing the disk and fan signal cable...............................................................................................83

Fans............................................................................................................................................................84

Removing a fan.....................................................................................................................................84

Replacing a fan..................................................................................................................................... 86

Fan power cable.........................................................................................................................................87

Removing the fan power cable............................................................................................................ 87

Replacing the fan power cable.............................................................................................................90

Graphics processing unit........................................................................................................................... 91

Removing the graphics processing unit from a water-cooled system............................................... 91

Replacing the graphics processing unit in a water-cooled system.................................................... 96

Memory modules..................................................................................................................................... 101

Removing and replacing memory modules.......................................................................................102

Memory plugging rules.......................................................................................................................107

PCIe adapters.......................................................................................................................................... 108

Removing a PCIe adapter.................................................................................................................. 108

Replacing a PCIe adapter.................................................................................................................. 109

Power supplies........................................................................................................................................ 110

Removing a power supply..................................................................................................................111

Replacing a power supply..................................................................................................................114

Power switch and cable...........................................................................................................................114

Removing the power switch and cable..............................................................................................115

Replacing the power switch and cable..............................................................................................119

System backplane................................................................................................................................... 123

Preparing to remove the system backplane......................................................................................123

Removing the system backplane.......................................................................................................124

Replacing the system backplane.......................................................................................................135

Preparing the system for operation after replacing the system backplane..................................... 149

System processor module.......................................................................................................................150

Removing a system processor module............................................................................................. 150

Replacing the system processor module.......................................................................................... 160

Upgrading system processor modules..............................................................................................172

Time-of-day battery.................................................................................................................................173

iii

Water manifold........................................................................................................................................ 175

Preparing the system to remove and replace the water manifold................................................... 175

Removing the water manifold............................................................................................................179

Replacing the water manifold............................................................................................................183

Preparing the system for operation after replacing the water manifold.......................................... 187

Common procedures for servicing or installing features...................................... 191

Before you begin......................................................................................................................................191

Identifying the system that contains the part to replace.......................................................................195

System LEDs.......................................................................................................................................195

Identifying the system....................................................................................................................... 197

Preparing the system to remove and replace internal parts..................................................................198

Preparing the system for operation........................................................................................................ 202

Starting and stopping the system........................................................................................................... 204

Starting the system ........................................................................................................................... 204

Stopping the 8335-GTC, 8335-GTG, 8335-GTH, 8335-GTW, or 8335-GTX system...................... 205

Sensor status........................................................................................................................................... 205

Removing and replacing covers.............................................................................................................. 206

Removing the service access cover...................................................................................................206

Installing the service access cover....................................................................................................207

Removing the front cover...................................................................................................................208

Installing the front cover....................................................................................................................209

Service and operating positions..............................................................................................................209

Placing the system into the service position.....................................................................................209

Placing the system into the operating position.................................................................................211

Removing and replacing power cords.....................................................................................................214

Disconnecting power cords............................................................................................................... 214

Connecting power cords.................................................................................................................... 217

Notices..............................................................................................................221

Accessibility features for IBM Power Systems servers..........................................................................222

Privacy policy considerations .................................................................................................................223

Trademarks..............................................................................................................................................223

Electronic emission notices.....................................................................................................................224

Class A Notices...................................................................................................................................224

Class B Notices...................................................................................................................................227

Terms and conditions.............................................................................................................................. 230

Safety notices

Safety notices may be printed throughout this guide:

• DANGER notices call attention to a situation that is potentially lethal or extremely hazardous to people.

• CAUTION notices call attention to a situation that is potentially hazardous to people because of some existing condition.

• Attention notices call attention to the possibility of damage to a program, device, system, or data.

World Trade safety information

Several countries require the safety information contained in product publications to be presented in their national languages. If this requirement applies to your country, safety information documentation is included in the publications package (such as in printed documentation, on DVD, or as part of the product) shipped with the product. The documentation contains the safety information in your national language with references to the U.S. English source. Before using a U.S. English publication to install, operate, or service this product, you must rst become familiar with the related safety information documentation. You should also refer to the safety information documentation any time you do not clearly understand any safety information in the U.S. English publications.

Replacement or additional copies of safety information documentation can be obtained by calling the IBM Hotline at 1-800-300-8751.

German safety information

Das Produkt ist nicht für den Einsatz an Bildschirmarbeitsplätzen im Sinne § 2 der Bildschirmarbeitsverordnung geeignet.

Laser safety information

IBM servers can use I/O cards or features that are ber-optic based and that utilize lasers or LEDs.

Laser compliance

IBM servers may be installed inside or outside of an IT equipment rack.

DANGER:

Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard:

• If IBM supplied the power cord(s), connect power to this unit only with the IBM provided power cord. Do not use the IBM provided power cord for any other product.

• Do not open or service any power supply assembly.

• Do not connect or disconnect any cables or perform installation, maintenance, or reconguration of this product during an electrical storm.

• The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords.

– For AC power, disconnect all power cords from their AC power source. – For racks with a DC power distribution panel (PDP), disconnect the customer’s DC power

• When connecting power to the product ensure all power cables are properly connected.

When working on or around the system, observe the following precautions:

source to the PDP.

– For racks with AC power, connect all power cords to a properly wired and grounded electrical

outlet. Ensure that the outlet supplies proper voltage and phase rotation according to the system rating plate.

– For racks with a DC power distribution panel (PDP), connect the customer’s DC power source

to the PDP. Ensure that the proper polarity is used when attaching the DC power and DC power return wiring.

• Connect any equipment that will be attached to this product to properly wired outlets.

• When possible, use one hand only to connect or disconnect signal cables.

• Never turn on any equipment when there is evidence of re, water, or structural damage.

• Do not attempt to switch on power to the machine until all possible unsafe conditions are corrected.

• Assume that an electrical safety hazard is present. Perform all continuity, grounding, and power checks specied during the subsystem installation procedures to ensure that the machine meets safety requirements.

• Do not continue with the inspection if any unsafe conditions are present.

• Before you open the device covers, unless instructed otherwise in the installation and conguration procedures: Disconnect the attached AC power cords, turn off the applicable circuit breakers located in the rack power distribution panel (PDP), and disconnect any telecommunications systems, networks, and modems.

DANGER:

• Connect and disconnect cables as described in the following procedures when installing, moving, or opening covers on this product or attached devices.

To Disconnect:

1. Turn off everything (unless instructed otherwise).

2. For AC power, remove the power cords from the outlets.

3. For racks with a DC power distribution panel (PDP), turn off the circuit breakers located in the PDP and remove the power from the Customer's DC power source.

4. Remove the signal cables from the connectors.

5. Remove all cables from the devices.

To Connect:

1. Turn off everything (unless instructed otherwise).

2. Attach all cables to the devices.

3. Attach the signal cables to the connectors.

4. For AC power, attach the power cords to the outlets.

5. For racks with a DC power distribution panel (PDP), restore the power from the Customer's DC power source and turn on the circuit breakers located in the PDP.

6. Turn on the devices.

Sharp edges, corners and joints may be present in and around the system. Use care when handling equipment to avoid cuts, scrapes and pinching. (D005)

(R001 part 1 of 2):

DANGER:

• Heavy equipment–personal injury or equipment damage might result if mishandled.

• Always lower the leveling pads on the rack cabinet.

• Always install stabilizer brackets on the rack cabinet unless the earthquake option is to be installed.

• To avoid hazardous conditions due to uneven mechanical loading, always install the heaviest devices in the bottom of the rack cabinet. Always install servers and optional devices starting from the bottom of the rack cabinet.

Observe the following precautions when working on or around your IT rack system:

vi Power Systems: Servicing the 8335-GTW

• Rack-mounted devices are not to be used as shelves or work spaces. Do not place objects on top of rack-mounted devices. In addition, do not lean on rack mounted devices and do not use them to stabilize your body position (for example, when working from a ladder).

• Stability hazard:

– The rack may tip over causing serious personal injury. – Before extending the rack to the installation position, read the installation instructions. – Do not put any load on the slide-rail mounted equipment mounted in the installation position. – Do not leave the slide-rail mounted equipment in the installation position.

• Each rack cabinet might have more than one power cord.

– For AC powered racks, be sure to disconnect all power cords in the rack cabinet when directed

to disconnect power during servicing.

– For racks with a DC power distribution panel (PDP), turn off the circuit breaker that controls

the power to the system unit(s), or disconnect the customer’s DC power source, when directed to disconnect power during servicing.

• Connect all devices installed in a rack cabinet to power devices installed in the same rack cabinet. Do not plug a power cord from a device installed in one rack cabinet into a power device installed in a different rack cabinet.

• An electrical outlet that is not correctly wired could place hazardous voltage on the metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock. (R001 part 1 of 2)

(R001 part 2 of 2):

CAUTION:

• Do not install a unit in a rack where the internal rack ambient temperatures will exceed the manufacturer's recommended ambient temperature for all your rack-mounted devices.

• Do not install a unit in a rack where the air flow is compromised. Ensure that air flow is not blocked or reduced on any side, front, or back of a unit used for air flow through the unit.

• Consideration should be given to the connection of the equipment to the supply circuit so that overloading of the circuits does not compromise the supply wiring or overcurrent protection. To provide the correct power connection to a rack, refer to the rating labels located on the equipment in the rack to determine the total power requirement of the supply circuit.

• (For sliding drawers.) Do not pull out or install any drawer or feature if the rack stabilizer brackets are not attached to the rack or if the rack is not bolted to the floor. Do not pull out more than one drawer at a time. The rack might become unstable if you pull out more than one drawer at a time.

• (For xed drawers.) This drawer is a xed drawer and must not be moved for servicing unless specied by the manufacturer. Attempting to move the drawer partially or completely out of the rack might cause the rack to become unstable or cause the drawer to fall out of the rack. (R001 part 2 of 2)

Safety notices

vii

CAUTION: Removing components from the upper positions in the rack cabinet improves rack

stability during relocation. Follow these general guidelines whenever you relocate a populated rack cabinet within a room or building.

• Reduce the weight of the rack cabinet by removing equipment starting at the top of the rack cabinet. When possible, restore the rack cabinet to the conguration of the rack cabinet as you received it. If this conguration is not known, you must observe the following precautions:

– Remove all devices in the 32U position (compliance ID RACK-001 or 22U (compliance ID

RR001) and above. – Ensure that the heaviest devices are installed in the bottom of the rack cabinet. – Ensure that there are little-to-no empty U-levels between devices installed in the rack cabinet

below the 32U (compliance ID RACK-001 or 22U (compliance ID RR001) level, unless the

received conguration specically allowed it.

• If the rack cabinet you are relocating is part of a suite of rack cabinets, detach the rack cabinet from the suite.

• If the rack cabinet you are relocating was supplied with removable outriggers they must be reinstalled before the cabinet is relocated.

• Inspect the route that you plan to take to eliminate potential hazards.

• Verify that the route that you choose can support the weight of the loaded rack cabinet. Refer to the documentation that comes with your rack cabinet for the weight of a loaded rack cabinet.

• Verify that all door openings are at least 760 x 230 mm (30 x 80 in.).

• Ensure that all devices, shelves, drawers, doors, and cables are secure.

• Ensure that the four leveling pads are raised to their highest position.

• Ensure that there is no stabilizer bracket installed on the rack cabinet during movement.

• Do not use a ramp inclined at more than 10 degrees.

• When the rack cabinet is in the new location, complete the following steps:

(L001)

(L002)

– Lower the four leveling pads. – Install stabilizer brackets on the rack cabinet or in an earthquake environment bolt the rack to

the floor.

– If you removed any devices from the rack cabinet, repopulate the rack cabinet from the

lowest position to the highest position.

• If a long-distance relocation is required, restore the rack cabinet to the conguration of the rack cabinet as you received it. Pack the rack cabinet in the original packaging material, or equivalent. Also lower the leveling pads to raise the casters off of the pallet and bolt the rack cabinet to the pallet.

(R002)

DANGER:

this label attached. Do not open any cover or barrier that contains this label. (L001)

Hazardous voltage, current, or energy levels are present inside any component that has

viii

Power Systems: Servicing the 8335-GTW

(L003)

DANGER: Rack-mounted devices are not to be used as shelves or work spaces. Do not place objects on top of rack-mounted devices. In addition, do not lean on rack-mounted devices and do not use them to stabilize your body position (for example, when working from a ladder). Stability hazard:

• The rack may tip over causing serious personal injury.

• Before extending the rack to the installation position, read the installation instructions.

• Do not put any load on the slide-rail mounted equipment mounted in the installation position.

• Do not leave the slide-rail mounted equipment in the installation position.

(L002)

Safety notices

DANGER: Multiple power cords. The product might be equipped with multiple AC power cords or multiple DC power cables. To remove all hazardous voltages, disconnect all power cords and power cables. (L003)

(L007)

CAUTION:

x Power Systems: Servicing the 8335-GTW

A hot surface nearby. (L007)

(L008)

CAUTION: Hazardous moving parts nearby. (L008)

All lasers are certied in the U.S. to conform to the requirements of DHHS 21 CFR Subchapter J for class 1 laser products. Outside the U.S., they are certied to be in compliance with IEC 60825 as a class 1 laser product. Consult the label on each part for laser certication numbers and approval information.

CAUTION: This product might contain one or more of the following devices: CD-ROM drive, DVDROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information:

• Do not remove the covers. Removing the covers of the laser product could result in exposure to hazardous laser radiation. There are no serviceable parts inside the device.

• Use of the controls or adjustments or performance of procedures other than those specied herein might result in hazardous radiation exposure.

(C026)

CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical ber cable or open receptacle. Although shining light into one end and looking into the other end of a disconnected optical ber to verify the continuity of optic bers may not injure the eye, this procedure is potentially dangerous. Therefore, verifying the continuity of optical bers by shining light into one end and looking at the other end is not recommended. To verify continuity of a ber optic cable, use an optical light source and power meter. (C027)

CAUTION: This product contains a Class 1M laser. Do not view directly with optical instruments. (C028)

CAUTION: Some laser products contain an embedded Class 3A or Class 3B laser diode. Note the following information:

• Laser radiation when open.

• Do not stare into the beam, do not view directly with optical instruments, and avoid direct exposure to the beam. (C030)

(C030)

CAUTION: The battery contains lithium. To avoid possible explosion, do not burn or charge the battery.

Do Not:

• Throw or immerse into water

• Heat to more than 100 degrees C (212 degrees F)

• Repair or disassemble

Exchange only with the IBM-approved part. Recycle or discard the battery as instructed by local regulations. In the United States, IBM has a process for the collection of this battery. For information, call 1-800-426-4333. Have the IBM part number for the battery unit available when you call. (C003)

CAUTION: Regarding IBM provided VENDOR LIFT TOOL:

• Operation of LIFT TOOL by authorized personnel only.

Safety notices xi

• LIFT TOOL intended for use to assist, lift, install, remove units (load) up into rack elevations. It is not to be used loaded transporting over major ramps nor as a replacement for such designated tools like pallet jacks, walkies, fork trucks and such related relocation practices. When this is not practicable, specially trained persons or services must be used (for instance, riggers or movers).

• Read and completely understand the contents of LIFT TOOL operator's manual before using. Failure to read, understand, obey safety rules, and follow instructions may result in property damage and/or personal injury. If there are questions, contact the vendor's service and support. Local paper manual must remain with machine in provided storage sleeve area. Latest revision manual available on vendor's web site.

• Test verify stabilizer brake function before each use. Do not over-force moving or rolling the LIFT TOOL with stabilizer brake engaged.

• Do not raise, lower or slide platform load shelf unless stabilizer (brake pedal jack) is fully engaged. Keep stabilizer brake engaged when not in use or motion.

• Do not move LIFT TOOL while platform is raised, except for minor positioning.

• Do not exceed rated load capacity. See LOAD CAPACITY CHART regarding maximum loads at center versus edge of extended platform.

• Only raise load if properly centered on platform. Do not place more than 200 lb (91 kg) on edge of sliding platform shelf also considering the load's center of mass/gravity (CoG).

• Do not corner load the platforms, tilt riser, angled unit install wedge or other such accessory options. Secure such platforms -- riser tilt, wedge, etc options to main lift shelf or forks in all four (4x or all other provisioned mounting) locations with provided hardware only, prior to use. Load objects are designed to slide on/off smooth platforms without appreciable force, so take care not to push or lean. Keep riser tilt [adjustable angling platform] option flat at all times except for nal minor angle adjustment when needed.

• Do not stand under overhanging load.

• Do not use on uneven surface, incline or decline (major ramps).

• Do not stack loads.

• Do not operate while under the influence of drugs or alcohol.

• Do not support ladder against LIFT TOOL (unless the specic allowance is provided for one following qualied procedures for working at elevations with this TOOL).

• Tipping hazard. Do not push or lean against load with raised platform.

• Do not use as a personnel lifting platform or step. No riders.

• Do not stand on any part of lift. Not a step.

• Do not climb on mast.

• Do not operate a damaged or malfunctioning LIFT TOOL machine.

• Crush and pinch point hazard below platform. Only lower load in areas clear of personnel and obstructions. Keep hands and feet clear during operation.

• No Forks. Never lift or move bare LIFT TOOL MACHINE with pallet truck, jack or fork lift.

• Mast extends higher than platform. Be aware of ceiling height, cable trays, sprinklers, lights, and other overhead objects.

• Do not leave LIFT TOOL machine unattended with an elevated load.

• Watch and keep hands, ngers, and clothing clear when equipment is in motion.

• Turn Winch with hand power only. If winch handle cannot be cranked easily with one hand, it is probably over-loaded. Do not continue to turn winch past top or bottom of platform travel. Excessive unwinding will detach handle and damage cable. Always hold handle when lowering, unwinding. Always assure self that winch is holding load before releasing winch handle.

• A winch accident could cause serious injury. Not for moving humans. Make certain clicking sound is heard as the equipment is being raised. Be sure winch is locked in position before releasing handle. Read instruction page before operating this winch. Never allow winch to unwind freely.

xii

Power Systems: Servicing the 8335-GTW

Freewheeling will cause uneven cable wrapping around winch drum, damage cable, and may cause serious injury.

• This TOOL must be maintained correctly for IBM Service personnel to use it. IBM shall inspect condition and verify maintenance history before operation. Personnel reserve the right not to use TOOL if inadequate. (C048)

Power and cabling information for NEBS (Network Equipment-Building System) GR-1089-CORE

The following comments apply to the IBM servers that have been designated as conforming to NEBS (Network Equipment-Building System) GR-1089-CORE:

The equipment is suitable for installation in the following:

• Network telecommunications facilities

• Locations where the NEC (National Electrical Code) applies

The intrabuilding ports of this equipment are suitable for connection to intrabuilding or unexposed wiring or cabling only. The intrabuilding ports of this equipment must not be metallically connected to the interfaces that connect to the OSP (outside plant) or its wiring. These interfaces are designed for use as intrabuilding interfaces only (Type 2 or Type 4 ports as described in GR-1089-CORE) and require isolation from the exposed OSP cabling. The addition of primary protectors is not sufcient protection to connect these interfaces metallically to OSP wiring.

Note: All Ethernet cables must be shielded and grounded at both ends.

The ac-powered system does not require the use of an external surge protection device (SPD).

The dc-powered system employs an isolated DC return (DC-I) design. The DC battery return terminal shall not be connected to the chassis or frame ground.

The dc-powered system is intended to be installed in a common bonding network (CBN) as described in GR-1089-CORE.

Safety notices

xiii

xiv Power Systems: Servicing the 8335-GTW

Removing and replacing parts in the 8335-GTW or 8335-GTX

Use these procedures to remove and replace failing parts.

Note: See the International Information Bulletin for Customers - Installation of IBM Machines (http:// www.ibm.com/e-business/linkweb/publications/servlet/pbi.wss). This bulletin (Publication number SC27-6601-00) provides a list of the key IBM system installation activities and those activities that might be billable.

Before you begin a replacement, complete these tasks:

1. If you are completing a replacement procedure that might put your data at risk, ensure, if possible, that you have a current backup of your system or logical partition (including operating systems, licensed programs, and data).

2. Review the installation or replacement procedure for the feature or part.

3. Note the signicance of color on your system. The color blue on the part indicates a touch point.

4. Ensure that you have access to a medium, flat-blade screwdriver, and a Phillips screwdriver.

5. If parts are incorrect, missing, or visibly damaged, contact the provider of the part or your next level of support.