Oracle ZFS Storage Appliance, ZS4-4, ZS3-4, 7320, 7120 Service Manual

...

Download

Page 1

Oracle® ZFS Storage Appliance Customer Service Manual

For ZS4-4, ZS3-x, 7x20 Controllers, and DE2-24, Sun Disk Shelves, Release OS8.6.0

Part No: E71921-01

July 2016

Page 2

Page 3

Oracle ZFS Storage Appliance Customer Service Manual

Part No: E71921-01

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?

ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Page 4

Référence: E71921-01

Ce logiciel et la documentation qui l’accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d’utilisation et de divulgation. Sauf stipulation expresse de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, accorder de licence, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d’interopérabilité avec des logiciels tiers ou tel que prescrit par la loi.

Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu’elles soient exemptes d’erreurs et vous invite, le cas échéant, à lui en faire part par écrit.

Si ce logiciel, ou la documentation qui l’accompagne, est livré sous licence au Gouvernement des Etats-Unis, ou à quiconque qui aurait souscrit la licence de ce logiciel pour le compte du Gouvernement des Etats-Unis, la notice suivante s’applique:

Ce logiciel ou matériel a été développé pour un usage général dans le cadre d’applications de gestion des informations. Ce logiciel ou matériel n’est pas conçu ni n’est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer des dommages corporels. Si vous utilisez ce logiciel ou matériel dans le cadre d’ applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l’utilisation de ce logiciel ou matériel pour ce type d’applications.

Oracle et Java sont des marques déposées d’Oracle Corporation et/ou de ses affiliés. Tout autre nom mentionné peut correspondre à des marques appartenant à d’autres propriétaires qu’Oracle.

Intel et Intel Xeon sont des marques ou des marques déposées d’Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d’Advanced Micro Devices. UNIX est une marque déposée d’The Open Group.

Ce logiciel ou matériel et la documentation qui l’accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l’accès à des contenus, produits ou services tiers, ou à leur utilisation, sauf mention contraire stipulée dans un contrat entre vous et Oracle.

Accessibilité de la documentation

Pour plus d’informations sur l’engagement d’Oracle pour l’accessibilité à la documentation, visitez le site Web Oracle Accessibility Program, à l’adresse http://www.oracle.com/

pls/topic/lookup?ctx=acc&id=docacc.

Accès aux services de support Oracle

Les clients Oracle qui ont souscrit un contrat de support ont accès au support électronique via My Oracle Support. Pour plus d’informations, visitez le site http://www.oracle.com/

pls/topic/lookup?ctx=acc&id=info ou le site http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs si vous êtes malentendant.

Page 5

Servicing the Oracle ZFS Storage Appliance .......................... ......... ................. 13

Getting Started with Servicing the Hardware .......................... ......... ................. 15

Introduction to the BUI ...... ............................................................................ 15

Introduction to the CLI ............... ................................................................... 16

BUI Hardware View ....... ............................................................................... 16

▼ Viewing the Remaining SSD Endurance (BUI) ...................................... 17

▼ Setting a Threshold Alert for SSD Endurance (BUI) ... ............................ 18

▼ Viewing CPU Details (BUI) ............................................................... 19

System Overview .................................................. ......... ............................... 19

System Chassis .................................................................... ................. 20

Disk Shelves ....... .................................................................................. 21

Chassis Details View ..................................................................................... 22

Disk Shelf Details View .............................. ........................................... 23

SSD Endurance ........................................................... ......... ................. 25

InfiniBand Host Controller Adapters .............................. ........................... 25

Service Processor ......................................... ......................................... 26

CLI Hardware View ....... ......... ....................................................................... 26

Component Properties ........................................................................... . 27

▼ Viewing the Remaining SSD Endurance (CLI) ...................................... 28

▼ Setting a Threshold Alert for SSD Endurance (CLI) .............................. .. 29

▼ Viewing CPU Details (CLI) ... ............................................................ 29

▼ Performing a Diagnostic Reboot (CLI) ......................... ........................ 30

Configuring the Management Port ................ .................................................... 30

Using Oracle ILOM to Diagnose Hardware Faults .......... ..................................... 31

▼ Logging in to Oracle ILOM Using a Local Serial Connection ................... 31

▼ Logging in to Oracle ILOM Remotely Using a Web Interface ................... 32

▼ Logging in to Oracle ILOM Remotely Using a Command Line

Interface ...................................................................... ......................... 32

Page 6

Contents

▼ Viewing and Clearing CPU Faults from ILOM ............. ......... ................ 33

Understanding Hardware Status ...... ................................................................. 33

▼ Locating the Chassis Serial Number .................... ......... ....................... 34

▼ Locating a Failed Component (BUI) .. ......... ......................................... 34

▼ Locating a Failed Component (CLI) .................... ......... ....................... 34

Working with Problems ................................................................ .................. 36

Active Problems Display ................. ....................................................... 36

Repairing Active Problems ............................................................ .......... 37

Related Features .................................................. ......... ......................... 38

Using Logs .................................. ................................................................. 38

Alert Log Entries ............ ...................................................................... 38

Fault Log Entries ............ ...................................................................... 39

System Log Entries .............................................. ......... ........................ 40

Audit Log Entries ... ............................................................................... 40

Phone Home Log Entries ................................................................. ....... 41

▼ Viewing Logs (BUI) ........ ................................................................. 41

▼ Listing Logs (CLI) ............................................................... ............ 41

▼ Viewing All Log Entries (CLI) .................................... ....................... 42

▼ Viewing Groups of Log Entries (CLI) .................................................. 42

▼ Viewing Entry Details (CLI) ....................................................... ....... 44

▼ Exporting Logs (BUI) ....... ................................................................ 44

▼ Exporting Logs (CLI) ................ ....................................................... 46

Using Workflows .................................. ......................................................... 47

Prerequisites for Servicing Hardware ......................................... ...................... 49

Safety Information ......................................... ................................................ 49

Required Tools and Information ........................................................ ......... ...... 50

Powering Off the Controller ............................................................................ 50

▼ Extending the Storage Controller from the Rack ...................... ...................... 51

▼ Removing the Top Cover ................................................ ........................... 52

▼ Powering Off the Disk Shelf ................................ ......... ............................. 53

Servicing the ZS4-4 Controller ...................................... .................................... 55

▼ Replacing a ZS4-4 HDD or SSD ...................... ......... ................................. 55

▼ Replacing a ZS4-4 Power Supply ............................ ......... ........................... 57

▼ Replacing a ZS4-4 Fan Module .................................................................. 58

▼ Identifying a Faulted ZS4-4 Memory Module ................................................ 60

6 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 7

Contents

▼ Replacing a ZS4-4 DIMM ................. ........................................................ 62

▼ Replacing a ZS4-4 Memory Riser Card ........................................................ 64

▼ Replacing a ZS4-4 PCIe Card .................................................................... 66

▼ Replacing a ZS4-4 Battery ........ ................................................................. 71

ZS4-4 Controller Hardware Overview ............................................................... 74

ZS4-4 Base Configuration ....................................................................... 74

ZS4-4 Front and Rear Panel Components .................................................. 75

ZS4-4 PCIe I/O Cards ............................................................................ 79

ZS4-4 Internal Components .......................................... ........................... 83

ZS4-4 Attached Storage ................................... ......... .............................. 88

Servicing the ZS3-4 Controller ...................................... .................................... 89

▼ Replacing a ZS3-4 HDD or SSD ...................... ......... ................................. 89

▼ Replacing a ZS3-4 Fan Module .................................................................. 91

▼ Replacing a ZS3-4 Power Supply ............................ ......... ........................... 93

▼ Replacing a ZS3-4 DIMM ................. ........................................................ 95

▼ Replacing a ZS3-4 PCIe Card or Riser .. ......... ............................................ 104

▼ Replacing a ZS3-4 Battery ........................ ............................................... 108

ZS3-4 Controller Hardware Overview ....................................... ...................... 111

ZS3-4 Base Configuration .......................................... ......... .................. 111

ZS3-4 Specifications ........................................................................... . 111

ZS3-4 Front and Rear Panel Components ................................................. 112

ZS3-4 Internal Components ...... ......... .................................................... 116

ZS3-4 PCIe I/O Cards ............................... ......... .................................. 121

ZS3-4 Attached Storage ........ ................................................................ 125

Servicing the ZS3-2 Controller .. ...................................................................... 127

▼ Installing Optional PCIe Slot Cards in Order .......................... ......... ............ 128

▼ Disconnecting the RJ-45 Cable from the Cluster Serial Port ............................ 129

▼ Replacing a ZS3-2 HDD or SSD .................... ......... .................................. 130

▼ Replacing a ZS3-2 Power Supply ............................................ .................. 133

▼ Replacing a ZS3-2 Fan Module ................................................................. 136

▼ Replacing a ZS3-2 DIMM .......................................... ............................. 139

▼ Replacing a ZS3-2 PCIe Card or Riser .. ......... ............................................ 146

▼ Replacing a ZS3-2 Flash Drive ............ ..................................................... 152

▼ Replacing a ZS3-2 Battery ........................ ............................................... 154

ZS3-2 Controller Hardware Overview ....................................... ...................... 156

Page 8

Contents

ZS3-2 Base Configuration .......................................... ......... .................. 156

ZS3-2 Front and Rear Panel Components ................................................. 157

ZS3-2 Specifications ........................................................................... . 161

ZS3-2 Internal Components ...... ......... .................................................... 162

ZS3-2 PCIe I/O Cards ............................... ......... .................................. 169

ZS3-2 Optional Cable Management Arm ................................................. 170

ZS3-2 Attached Storage ........ ................................................................ 171

Servicing the 7x20 Controller ...................................... .................................... 173

▼ Replacing a 7x20 HDD or SSD ...................... ......... ................................. 173

▼ Replacing a 7x20 Fan Module .................................................................. 175

▼ Replacing a 7x20 Power Supply ............................ ......... ........................... 180

▼ Replacing a 7x20 DIMM ................. ........................................................ 184

▼ Replacing a 7x20 Controller PCIe Card or Riser ................................... ....... 197

▼ Replacing a 7x20 Battery ........ ................................................................. 208

7420 Controller Hardware Overview ............................................................... 214

7420 Configuration Options ...... ............................................................. 214

7420 Front and Back Panel Components .......................... ........................ 215

7420 Specifications ..... ......................................................................... 218

7420 Internal Components .......................................... ........................... 219

7420 PCIe I/O Cards ............................................................................ 224

7420 Attached Storage ................................... ......... .............................. 229

7320 Controller Hardware Overview ............................................................... 229

7320 Configuration Options ...... ............................................................. 229

7320 Front and Back Panel Components .......................... ........................ 232

7320 Electrical Specifications ................................................. ......... ...... 236

7320 Internal Components .......................................... ........................... 236

7320 Attached Storage ................................... ......... .............................. 241

7120 Controller Hardware Overview ............................................................... 241

7120 Base Configuration ....................................................................... 242

7120 Chassis Overview ................. ....................................................... 242

7120 Front Panel ................................................................................. 243

7120 Rear Panel ... ............................................................................... 243

7120 Physical Specifications .................................................................. 244

7120 Electrical Specifications ................................................. ......... ...... 244

7120 Internal Boards ............................................................................ 245

7120 CPU and Memory ................................................................. ....... 246

8 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 9

Contents

7120 Power Distribution, Fan Module and Disk Components ....................... 247

7120 I/O Components ..................................................... ..................... 248

7120 NIC/HBA Options ........................................................................ 249

7120 PCIe Options ..................................... .......................................... 250

7120 Connectors ................................ ......... ......................................... 251

7120 Cables ...................................................................................... .. 252

7120 Attached Storage ................................... ......... .............................. 253

Servicing the Disk Shelves ........................................ ..................................... 255

Electrostatic Discharge Precautions .................................... ......... .................... 255

▼ Replacing a Faulty Disk Shelf Drive .............. ............................................ 256

▼ Replacing a Disk Shelf Power Supply ... ..................................................... 259

▼ Replacing a Disk Shelf I/O Module ........................................................... 264

▼ Replacing a Disk Shelf SIM Board .................................. ......... ................. 266

▼ Adding a New Disk Shelf ........................................................................ 269

▼ Adding a Disk Shelf Cache Device ......................... ......... .......................... 270

Connecting to Attached Storage .................................................... ......... ........ 272

Disk Shelf Hardware Overview ...................................................................... 272

Disk Shelf Front and Back Panel Components ................................... ....... 272

Disk Shelf Indicators ............................................................................ 277

Disk Shelf Drive Locations .................................... ......... ...................... 284

Disk Shelf Configurations ................................................... .................. 286

Maximum Number of Disk Shelves per Controller Configuration .................. 289

SAS-2 ...................................................................... .......................... 289

Viewing System Disks Status .......................................................................... 291

Working with Support Bundles ......................... ......... ..................................... 293

Support Bundle Options ............................................ ......... ........................... 293

▼ Generating and Uploading a Support Bundle (BUI) ..... .................................. 294

▼ Generating and Uploading a Support Bundle (CLI) ................................ ....... 294

▼ Viewing Support Bundle Details (BUI) ....................................................... 295

▼ Viewing Support Bundle Details (CLI) ....................................................... 296

▼ Uploading a Locally Generated Bundle (CLI) .............................................. 296

▼ Retrying a Failed Support Bundle Upload (BUI) .......................................... 297

▼ Retrying a Failed Support Bundle Upload (CLI) .. ......... ................................ 297

Page 10

Contents

▼ Canceling a Pending Support Bundle Operation (BUI) ................. .................. 298

▼ Canceling a Pending Operation (CLI) ............................................... ......... . 298

▼ Deleting a Support Bundle (BUI) ..................................................... ......... 298

▼ Deleting a Support Bundle (CLI) ............................................................... 298

Performing the Initial Setup ............. ............................................................... 301

Restarting the Appliance ................... .............................................................. 303

Performing a Factory Reset ............. ............................................................... 305

Backing Up the Configuration ........... ......... ..................................................... 307

Configuration Backup Contents ................................................................... ... 308

Configuration Restore Impact ............... ......... ................................................ 308

Security Considerations for Configuration Backups ................... ......... ................ 310

▼ Creating a Configuration Backup (BUI) ...................................................... 310

▼ Creating a Configuration Backup (CLI) ......... ......... .................................... 311

▼ Restoring from a Saved Configuration (BUI) .................. ............................. 311

▼ Restoring from a Saved Configuration (CLI) .................................... ......... .. 312

▼ Deleting a Saved Configuration (BUI) .. ...................................................... 312

▼ Deleting a Saved Configuration (CLI) .................... .................................... 313

▼ Exporting a Saved Configuration (BUI) ......... ............................................. 313

▼ Exporting a Saved Configuration (CLI) .................. ......... ........................... 313

▼ Importing a Saved Configuration (BUI) ......... ............................................. 314

▼ Importing a Saved Configuration (CLI) .................. ......... ........................... 314

▼ Viewing Appliance Configurations (BUI) .................................................... 314

▼ Viewing Appliance Configurations (CLI) .................................................... 314

Working with Software Notifications and Updates ................................ .......... 317

▼ Setting Up Software Notification (BUI) ......... ............................................. 317

▼ Setting Up Software Notification (CLI) .................. ......... ........................... 318

▼ Checking for and Downloading Software Updates (BUI) .... ........................... 318

▼ Checking for and Downloading Software Updates (CLI) ...................... .......... 318

▼ Removing System Updates (BUI) ....... ....................................................... 319

▼ Removing System Updates (CLI) ......................... ..................................... 319

10 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 11

Contents

Upgrading the Software .............................................. ..................................... 321

Preparing to Upgrade the Software ........ ......................................................... 321

Upgrading Software on a Standalone Controller ................................................ 322

▼ Upgrading Software on a Standalone Controller (BUI) ........................... 323

▼ Upgrading Software on a Standalone Controller (CLI) ........................... 324

Upgrading Software on Clustered Controllers ....... ............................................ 327

▼ Preparing to Upgrade Clustered Controllers (BUI) .................... ......... ... 327

▼ Preparing to Upgrade Clustered Controllers (CLI) ................................. 328

▼ Upgrading Controller A (BUI) ......................... ......... ........................ 330

▼ Upgrading Controller A (CLI) ........................................... ............... 331

▼ Upgrading Controller B (BUI) .................................. ......... ............... 333

▼ Upgrading Controller B (CLI) .................................................... ....... 334

Working with Health Checks .................................... ....................................... 337

▼ Checking System Health (BUI) ............................................. .................... 337

▼ Checking System Health (CLI) ............................................................... .. 339

Troubleshooting Update Health Check Failures ................................... ......... ..... 340

Resolving Health Check Alerts ...................................................................... 340

▼ Resolving Health Check Alerts ......................................................... 341

Deferred Updates ....... ...................................................................................... 343

▼ Applying Deferred Updates (BUI) ............................................................. 344

▼ Applying Deferred Updates (CLI) ............................................................. 344

Passthrough x Deferred Update ...................................................................... 345

User Quotas Deferred Update .................................... .................................... 346

COMSTAR Deferred Update ........ ................................................................. 346

Triple Parity RAID Deferred Update .................................... ......... .................. 347

Dedup Deferred Update ..................................................... ......... .................. 347

Replication Deferred Update .......................................... ................................ 348

Received Properties Deferred Update ......................................................... ..... 348

Slim ZIL Deferred Update ............................................... ............................. 348

Snapshot Deletion Deferred Update ....................... ......................................... 349

Recursive Snapshots Deferred Update ....................................... ...................... 349

Multi Replace Deferred Update ...................................................................... 349

RAIDZ Mirror Deferred Update ........................................... .......................... 350

Optional Child Dir Deferred Update .......................................... ..................... 350

Multiple Initiator Groups per LUN Deferred Update .......................................... 350

Page 12

Contents

Large Block Sizes Deferred Update ...................... ......... ................................. 351

Sequential Resilvering Deferred Update ........................................................... 351

NDMP-zfs Replica Backup Deferred Update .................................. .................. 351

ACL Passthrough with Mode Preservation Deferred Update .............................. .. 352

Understanding Deferred Updates ........................................ ............................ 352

Working with Firmware Updates .................................................................... . 355

▼ Checking the Status of Firmware Updates (BUI) .......................................... 357

▼ Checking the Status of Firmware Updates (CLI) .......................................... 357

Rolling Back System Software ........................................................................ 359

▼ Fail-safe Rollback ............... ......... ........................................................... 359

▼ Rolling Back the System Software (BUI) ........................ ............................ 360

▼ Rolling Back the System Software (CLI) .......................................... ......... . 361

Removing an Update Package .. ...................................................................... 363

▼ Removing an Update Package (BUI) ............. ............................................. 363

▼ Removing an Update Package (CLI) ............................... ........................... 363

12 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 13

Servicing the Oracle ZFS Storage Appliance

This document encompasses the following sections that describe maintenance procedures for the Oracle ZFS Storage Appliance.

To service the appliance hardware components, use the following sections:

■

“Getting Started with Servicing the Hardware” on page 15

■

“Prerequisites for Servicing Hardware” on page 49

■

“Servicing the ZS4-4 Controller” on page 55

■

“Servicing the ZS3-4 Controller” on page 89

■

“Servicing the ZS3-2 Controller” on page 127

■

“Servicing the 7x20 Controller” on page 173

■

“Servicing the Disk Shelves” on page 255

To maintain the system, use the following sections.

■

“Viewing System Disks Status” on page 291

■

“Working with Support Bundles” on page 293

■

“Performing the Initial Setup” on page 301

■

“Restarting the Appliance” on page 303

■

“Backing Up the Configuration” on page 307

■

“Working with Software Notifications and Updates” on page 317

■

“Upgrading Software on a Standalone Controller (BUI)” on page 323

■

“Upgrading Software on a Standalone Controller (CLI) ” on page 324

■

“Upgrading Software on Clustered Controllers” on page 327

■

“Working with Health Checks” on page 337

■

“Deferred Updates” on page 343

■

“Working with Firmware Updates” on page 355

■

“Rolling Back System Software” on page 359

■

“Removing an Update Package (BUI)” on page 363

Servicing the Oracle ZFS Storage Appliance 13

Page 14

14 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 15

Getting Started with Servicing the Hardware

To get started with servicing the hardware, use the following sections:

■

“Introduction to the BUI” on page 15

■

“Introduction to the CLI” on page 16

■

“BUI Hardware View” on page 16

■

“System Overview” on page 19

■

“Chassis Details View” on page 22

■

“Disk Shelf Details View” on page 23

■

“CLI Hardware View” on page 26

■

“Configuring the Management Port” on page 30

■

“Using Oracle ILOM to Diagnose Hardware Faults” on page 31

■

“Understanding Hardware Status” on page 33

■

“Working with Problems” on page 36

■

“Using Logs” on page 38

■

“Using Workflows” on page 47

Introduction to the BUI

The Browser User Interface (BUI) is the graphical tool for administration of the appliance. The BUI provides an intuitive environment for administration tasks, visualizing concepts, and analyzing performance data.

The management software is designed to be fully featured and functional on a variety of web browsers, as described in “Browser User Interface (BUI)” in Oracle ZFS Storage Appliance

Administration Guide, Release OS8.6.0. Direct your browser to the system using either the IP

address or host name you assigned to the NET-0 port during initial configuration as follows:

https://ipaddress:215 or https://hostname:215. The login screen appears. The online help linked in the top right of the BUI is context-sensitive. For every top-level and second-level screen in the BUI, the associated help page appears when you click the Help button.

Getting Started with Servicing the Hardware 15

Page 16

Introduction to the CLI

The Command Line Interface (CLI) is designed to mirror the capabilities of the BUI, while also providing a powerful scripting environment for performing repetitive tasks. The following sections describe details of the CLI. When navigating through the CLI, there are two principles to be aware of:

■

Tab completion is used extensively: if you are not sure what to type in any given context, pressing the Tab key will provide you with possible options. Throughout the documentation, pressing Tab is presented as the word "tab" in bold italics.

■

Help is always available: the help command provides context-specific help. Help on a particular topic is available by specifying the topic as an argument to help, for example help commands. Available topics are displayed by tab-completing the help command, or by typing help topics.

You can combine these two principles, as follows:

hostname:> helptab builtins commands general help properties script

BUI Hardware View

The BUI hardware view provides interactive illustrations that enable you to browse through the appliance and attached disk shelf components.

The buttons in the hardware view are described in the following table:

TABLE 1

Icon Description Icon Description

Icons in the BUI Hardware View

Show a more detailed view of this component

Leave this detailed view Power off, reboot, or

Click for more details Offline disk

Hardware component is ok (green)

Hardware component is not present (grey)

Toggle blinking of the locator LED for this component

diagnostic reboot

Port active

Port inactive

16 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 17

Viewing the Remaining SSD Endurance (BUI)

Icon Description Icon Description

Hardware component is faulted (amber)

Viewing the Remaining SSD Endurance (BUI)

Use the following procedure to view the remaining SSD endurance.

Go to Maintenance > Hardware.

Click Show Details to view controller details.

Click Disk.

Hover over a disk row and click its information icon . The disk details are displayed, including the remaining SSD endurance.

Getting Started with Servicing the Hardware 17

Page 18

Setting a Threshold Alert for SSD Endurance (BUI)

Click OK.

Setting a Threshold Alert for SSD Endurance (BUI)

Use the following procedure to set a threshold alert for SSD endurance.

Go to Configuration > Alerts > Threshold Alerts.

Click the add icon .

In the Threshold field, select SSD: Endurance on any device and specify and its parameters.

Specify the parameters for Timing and Alert actions.

(Optional) Click TEST to verify the settings.

Click APPLY.

18 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 19

Viewing CPU Details (BUI)

Use the following procedure to view CPU details.

Go to Maintenance > Hardware.

Click Show Details to view controller details.

Click CPU.

Hover over a CPU row and click its information icon . The CPU details are displayed.

Click OK.

System Overview

The main hardware page lists the system chassis, a summary of its contents, and any attached disk shelves (on supported systems). This provides an overview of the hardware present on

the system. The power icon , located in the upper left of the view, presents a dialog box to

Getting Started with Servicing the Hardware 19

Page 20

Viewing CPU Details (BUI)

either power off, reboot (power cycle), or reboot the appliance with diagnostics. Only select the diagnostic reboot option when instructed by Oracle Service personnel because it could take a long time to complete and could have adverse results if not performed properly. Do not perform a diagnostic reboot when system-affecting operations are occurring, such as upgrading firmware, executing commands, and configuring or unconfiguring storage. The diagnostic reboot option is not available when using the system chassis power icon.

System Chassis

The primary system chassis is shown on the top half of the view. At the top left, click the right-

arrow icon to get more detail about the chassis. The indicator notes if there are any faulted components within the chassis, and the name of the chassis. The chassis name is initially set to the appliance name during installation. To change the chassis name, use the entry field on the Configuration > Services > System Identity screen. For more information, see “System Identity

Configuration” in Oracle ZFS Storage Appliance Administration Guide, Release OS8.6.0.

At the top right of the system chassis is the locator icon to light the locator LED, and the

power icon , which presents a dialog box to either power off or reboot (power cycle) the appliance.

A thumbnail of the controller is presented at left. Clicking on the thumbnail or the "Show Details" link takes you to a detailed view of the chassis, and is identical to clicking on the right-

arrow icon at the top left of the view.

The following information is presented in a summary view:

TABLE 2

Property Description

Manufacturer Manufacturer of the system

Model System model name

Serial System chassis hardware serial number

Processors Count and description of processors in the system

Memory Total memory in the system

System Size and number of system disks used for the system image

Data Size and number of data disks in the system chassis. This is only valid for

Cache Size and number of read-optimized cache devices in the system chassis. This

System Chassis Properties

standalone systems. If there are no data disks present, "-" will be displayed.

is only valid for expandable systems that support additional disk shelves.

20 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 21

Viewing CPU Details (BUI)

Property Description

If there are no cache devices installed in the system chassis, "-" will be displayed.

Log Size and number of log disks in the system chassis. This is only valid for

Total Total size and count of all disks in the system.

standalone systems. If there are no log devices installed in the system chassis, "-" will be displayed.

Disk Shelves

A list of disk shelves, if supported, is displayed at the bottom of the Maintenance > Hardware view. The thumbnail to the left represents the front of the currently selected disk shelf. Clicking on the right-pointing arrow or double-clicking on a row within the list will provide complete details about the disk shelf. The state indicator will be orange if the chassis contains any faulted components.

TABLE 3

Property Description

Name Name of the disk shelf, used in faults and alerts. This is initially set to the

MFR/MODEL The disk shelf manufacturer and the model number.

Type Device type HDD or SSD.

RPM Revolutions per minute; the speed of the disk drive.

Data Total size of all data disks within the disk shelf.

Cache Total size of all read-optimized cache devices within the drive shelf. If there

Log Total size of all write-optimized cache devices within the drive shelf. If there

Paths Total number of I/O paths to the disk shelf. The only supported

Locate

Disk Shelf Properties

serial number of the disk shelf, but can be changed by clicking on the name within the list.

are no cache devices within the shelf, then "-" is displayed.

Read cache devices are supported in DE2 disk shelves with a minimum software version. For software requirements and slot configuration rules, see

“Disk Shelf Configurations” on page 286.

are no log devices within the shelf, then "-" is displayed.

configurations are those with multiple paths to all disks, so this should read

"2" under normal operating circumstances. Clicking the information icon will bring up a dialog with information about each path. This includes which HBAs are connected to the disk shelf, and the state of any paths. If the disks within the disk shelf are not currently configured as part of a storage pool, complete path information will not be available, though it displays two paths to the chassis.

Toggle the locator LED for this disk shelf. If the LED is currently on, then this indicator will flash.

Getting Started with Servicing the Hardware 21

Page 22

Chassis Details View

Related Topics

■

“Disk Shelf Details View” on page 23

Chassis Details View

To view the chassis details, click on the right-arrow icon (or one of the alternative forms described above). This view includes some of the same controls in the upper left (state, name, locate, reset, power off), as well as listings of all the components in the chassis.

At the left is a set of images describing the chassis. If there are multiple views, then you can switch between them by clicking on the name of the view above the image.

For each view, faulted components will be highlighted in red. In addition, the currently selected component will be highlighted in the image. Clicking on a component within the image will select the corresponding component in the list to the right.

A tab is present for each component type in the following list. Each component type has a state icon which will be orange if there is a faulted component of the given type.

■

Disk

■

Slot

■

CPU (controller only)

■

DIMM (Memory) (controller only)

■

Fan

■

PSU (Power supply unit)

■

SP (Service processor) (controller only)

Clicking on a component type will display a list of all physical locations within the chassis where components may be present. Clicking on a component within the list will highlight it

within the appropriate chassis image. Clicking on the information icon while over a row or double-clicking a row will bring up a dialog with detailed information about the component. The information displayed in the list depends on the component type, but is a subset of the information available in the component detail. Disks and service processors support additional operations described below. Each component can report any or all of the following properties:

TABLE 4

Property Description

Label Human-readable identifier for this component within the chassis. This is

Chassis Component Properties

typically, but not necessarily, equivalent to the label printed on the physical chassis.

22 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 23

Chassis Details View

Property Description

FMRI Fault managed resource identifier (FMRI) for the component. This is

Active Problems For a faulted component, links to active problems affecting the component.

Manufacturer Component manufacturer.

Model Component model.

Build Manufacturing build identifier. This is used to identify a particular location or

Part Component part number, or core factory part number. The orderable part

Serial Component serial number.

Revision Firmware or hardware revision of the component.

Size Total memory or storage, in bytes.

Type Disk type. Can be one of System, Data, Log, Cache, Spare, or Peer. When a

Speed Processor speed, in gigahertz.

Cores Number of CPU cores.

GUID Hardware global unique identifier.

Endurance The lifetime percentage remaining on an SSD. Lifetime starts at 100% and

Last Update The date and time of the last endurance reading of the SSD.

an internal identifier used to identify the component within faults and is intended for service personnel.

batch where the component was manufactured.

number may differ, depending on whether a component is for replacement or expansion, and whether it is part of a larger assembly. Your service provider should be able to refer you to the appropriate orderable part. For components without part numbers, the model number should be used instead.

spare is active, it will be displayed as Spare [A]. When a disk type is shown as Peer, it indicates that the disk is configured as part of a pool that is owned by the cluster peer.

decreases with disk usage.

Disk Shelf Details View

The details view provides information for each device installed in the disk shelf.

Getting Started with Servicing the Hardware 23

Page 24

Chassis Details View

TABLE 5

Property Description

Disk Device Detail Properties

ID Unique identifier of the device.

MFR/MODEL The device manufacturer and the model number.

RPM Revolutions per minute; the speed of the disk drive.

Type Disk type. Can be one of System, Data, Log, Cache, Spare, or Peer.

Note - Read cache devices are supported in DE2 disk shelves with a

minimum software version. For software requirements and slot configuration rules, see “Disk Shelf Configurations” on page 286 .

Toggle the locator LED for this disk device. If the device LED is currently

Locate

Online

on, then this indicator will flash.

Offline the disk. This option is only available for disks that are part of a configured storage pool (including the system pool). Offlining a disk prevents the system from reading or writing to it. Faulted devices are already avoided, so this option should only be required if a disk is exhibiting performance problems that do not result in pathological failure. It is not

24 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 25

Chassis Details View

Property Description

possible to offline a disk that would prevent access to data (i.e. offlining both halves of a mirror). If the device is an active hot spare, this will also give the option of detaching the hot spare completely. Once a hot spare is detached, it cannot be activated except through another fault or hotplug event.

Note - Do not perform a pool configuration operation while a disk firmware

upgrade is occurring. To check if an upgrade is in progress, navigate to Maintenance > System in the BUI, or maintenance system updates in the CLI.

SSD Endurance

SSD endurance reports the remaining life expectancy of an SSD. Endurance properties are reported on the BUI Maintenance > Hardware chassis details page and in the CLI maintenance chassis disk context. You can set a threshold alert when an SSD exceeds a specified percentage. For example, set an alert to occur when one or more SSD devices exceeds a 95% threshold. For more information, see “Threshold Alerts” in Oracle ZFS Storage Appliance Administration

Guide, Release OS8.6.0.

You can use this feature to monitor the life expectancy of an SSD. For example, if SSD endurance reports 50% after two years, the SSD is estimated to last for another two years, assuming the workload is consistent.

Note - Do not use reported SSD endurance percentage as an indication to replace the SSD. SSD

warranty replacements are made only when a failure is reported.

InfiniBand Host Controller Adapters

InfiniBand Host Controller Adapters (HCA) report additional properties for the list of available ports:

TABLE 6

Action Description

State

GUID The hardware assigned port GUID.

Speed The current port speed enabled: Single Data Rate (SDR), Dual Data Rate

Additional Properties for InfiniBand Host Controller Adapters

When "active", the active-port icon is displayed. Other valid port states

("down", "init", and "arm") are denoted by the inactive-port icon . Mousing over the port icon will display the current port state in the tip popup.

(DDR) or Quad Data Rate (QDR)

Getting Started with Servicing the Hardware 25

Page 26

CLI Hardware View

Service Processor

The service processor behaves differently from other component nodes. Instead of providing a list of components, it presents a set of network properties that can be configured from the storage appliance. The following properties control the behavior of the service processor network management port.

TABLE 7

Property Description

MAC Address Hardware MAC address. This is read-only

IP Address Source Either 'DHCP' or 'Static'. Controls whether DHCP should be used on the

IP Address IPv4 Address, when using static IP configuration. IPv6 is not supported.

Subnet Dotted decimal subnet, when using static IP configuration.

Default Gateway IPv4 default gateway address.

Properties for the Service Processor Network Management Port

Changing multiple values in conflicting ways (such as changing static IP assignments while in DHCP mode) has undefined behavior.

CLI Hardware View

Hardware status details are available in the CLI under the maintenance hardware section. Use the show command to list the status of all components. The list command will list available chassis, which can be selected and then viewed using show.

hostname:> maintenance hardware show Chassis:

interface.

NAME STATE MANUFACTURER MODEL

chassis-000 0839QCJ01A ok Sun Microsystems, Inc. Sun Storage 7320

cpu-000 CPU 0 ok AMD Quad-Core AMD Op cpu-001 CPU 1 ok AMD Quad-Core AMD Op cpu-002 CPU 2 ok AMD Quad-Core AMD Op cpu-003 CPU 3 ok AMD Quad-Core AMD Op disk-000 HDD 0 ok STEC MACH8 IOPS disk-001 HDD 1 ok STEC MACH8 IOPS

26 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 27

CLI Hardware View

disk-002 HDD 2 absent - - disk-003 HDD 3 absent - - disk-004 HDD 4 absent - disk-005 HDD 5 absent - disk-006 HDD 6 ok HITACHI HTE5450SASUN500G disk-007 HDD 7 ok HITACHI HTE5450SASUN500G fan-000 FT 0 ok unknown ASY,FAN,BOARD,H2 fan-001 FT 0 FM 0 ok Sun Microsystems, Inc. 541-2068 fan-002 FT 0 FM 1 ok Sun Microsystems, Inc. 541-2068 fan-003 FT 0 FM 2 ok Sun Microsystems, Inc. 541-2068 fan-004 FT 1 ok unknown ASY,FAN,BOARD,H2 fan-005 FT 1 FM 0 ok Sun Microsystems, Inc. 541-2068 fan-006 FT 1 FM 1 ok Sun Microsystems, Inc. 541-2068 fan-007 FT 1 FM 2 ok Sun Microsystems, Inc. 541-2068 memory-000 DIMM 0/0 ok HYNIX 4096MB DDR-II 66 memory-001 DIMM 0/1 ok HYNIX 4096MB DDR-II 66 ...

A 5th and 6th column for serial number ("SERIAL") and revolutions per minute ("RPM") have been truncated in the above example, as has the length of this list.

Component Properties

If a particular component is selected, detailed information about its properties are reported. The following properties are supported, with the corresponding BUI property name. For a description of a particular property, see the earlier descriptions.

TABLE 8

CLI Property BUI Property

build Build

cores Cores

device N/A

faulted (status indicator)

label Label

locate (writable) (status indicator)

manufacturer Manufacturer

model Model

offline (writeable) (status indicator)

part Part

present (status indicator)

revision Revision

Component CLI Properties and BUI Equivalent Properties

Getting Started with Servicing the Hardware 27

Page 28

Viewing the Remaining SSD Endurance (CLI)

CLI Property BUI Property

serial Serial

size Size

speed Speed

type (combined with use)

use Type

When viewing a disk that is active as a hot spare, the detach command is also available.

Viewing the Remaining SSD Endurance (CLI)

Use the following procedure to view the remaining SSD endurance.

To list the properties of a disk, use the following CLI commands:

hostname maintenance hardware hostname:maintenance hardware> select chassis-001 hostname:maintenance hardware chassis-001> select disk hostname:maintenance hardware chassis-001> select disk-015 hostname:maintenance hardware chassis-001 disk-015> list Properties: label = HDD 15 present = true faulted = false manufacturer = SANDISK model = LB806M---SUN800G serial = 40042896 revision = S30E size = 745G type = data use = data rpm = - device = c0t5001E82002630190d0 pathcount = 2 interface = SAS endurance = 100% endurance_updated = 2014-3-3 22:04:14 locate = false offline = false

As shown above, disk-015 is at 100% which indicate a new SSD. The remaining endurance estimate decreases as the disk is used.

28 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 29

Setting a Threshold Alert for SSD Endurance (CLI)

Use the following procedure to set a threshold alert for SSD endurance.

To set a threshold alert for SSD endurance, use the following CLI commands:

hostname:configuration alerts> thresholds hostname:configuration alerts thresholds> create hostname:configuration alerts threshold (uncommitted)> set

statname=ssd.endurance[ssd]

statname = ssd.endurance[ssd] (uncommitted)

To list the properties of the threshold alert, use the following CLI command:

hostname:configuration alerts threshold (uncommitted)> list Properties: uuid = <generated on commit> statname = ssd.endurance[ssd] (uncommitted) type = normal limit = (unset) minpost = 5 minutes days = all window_start = none window_end = 00:00 frequency = 5 minutes minclear = 5 minutes

Note - The remaining fields are set the same way you would set them for any other threshold

alert.

Viewing CPU Details (CLI)

Use the following procedure to view CPU details.

To view CPU details, use the following CLI commands:

hostname:maintenance hardware> select chassis-000 hostname:maintenance chassis-000> select cpu hostname:maintenance chassis-000 cpu> select cpu-000 hostname:maintenance chassis-000 cpu-000> show

Getting Started with Servicing the Hardware 29

Page 30

Performing a Diagnostic Reboot (CLI)

Properties: label = CPU 0 present = true faulted = false manufacturer = AMD model = Quad-Core AMD Opteron(tm) Processor 8356 part = 1002 revision = 03 cores = 4 speed = 2.14G

Performing a Diagnostic Reboot (CLI)

Caution - Only issue the diagnostic reboot command as instructed by Oracle Service personnel.

A diagnostic reboot gathers diagnostic information before power cycling the appliance. This operation could take a long time to complete and could cause adverse results if not performed properly. Do not reboot when system-affecting operations are occurring, such as upgrading firmware, executing commands, and configuring or unconfiguring storage.

To perform a diagnostic reboot, use the following CLI command:

hostname:maintenance system> diagreboot

Configuring the Management Port

All standalone controllers should have at least one NIC port configured as a management interface. Select the Allow Admin option in the BUI to enable BUI connections on port 215 and CLI connections on ssh port 22.

All cluster installations should have at least one NIC port on each controller configured as a management interface as described above. In addition, the NIC instance number must be unique on each controller. For example, nodeA uses igb0 and nodeB uses igb1, so that neither may be used as a cluster data interface. In addition, these interfaces must be locked to the controller using the Configuration > Cluster option in the BUI. In some cases, this may require installation of an additional network interface card on each controller in a cluster configuration.

If access to the appliance data interfaces is impossible for any reason, the management network interface will maintain BUI and CLI access. During a cluster takeover, interfaces are taken

30 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 31

Using Oracle ILOM to Diagnose Hardware Faults

down on the failed controller. So, locked interface configuration is required to gather diagnostic information from a failed controller.

Note - Failure to configure locked management interfaces on a cluster may lead to longer than

necessary fault diagnosis and resolution times.

Using Oracle ILOM to Diagnose Hardware Faults

In rare cases, faults associated with uncorrectable CPU errors are not diagnosable or displayed in the controller. These faults are preserved by and observable on the Oracle Integrated Lights Out Manager (ILOM), which resides in the Service Processor (SP). Connect to the server ILOM to diagnose hardware faults that do not appear in the BUI.

The server ILOM provides options for (i) network and (ii) serial port connectivity. Network connection is the preferred choice because the ILOM serial port does not always allow adequate means of platform data collection. Network connections are on port 215, and serial connections are on SSH port 22.

Caution - Failure to configure Oracle ILOM connectivity may lead to longer than necessary

hardware fault diagnosis and resolution times.

For information about using Oracle ILOM, refer to the Oracle Integrated Lights Out Manager (ILOM) 3.1 Documentation Library at http://www.oracle.com/pls/topic/lookup?

ctx=ilom31.

Logging in to Oracle ILOM Using a Local Serial Connection

Use the following procedure to log in to Oracle ILOM using a local serial connection.

Prepare an administrative client (terminal or terminal emulator) with the serial connection parameters:

■

8N1: eight data bits, no parity, one stop bit

■

9600 baud

■

Disable hardware flow control (CTS/RTS)

■

Disable software flow control (XON/XOFF)

Getting Started with Servicing the Hardware 31

Page 32

Logging in to Oracle ILOM Remotely Using a Web Interface

Connect the administrative client to the SER MGT port on the controller using a serial null modem cable.

To establish a connection between your serial console and Oracle ILOM, press Enter on the administrative client.

A login prompt for Oracle ILOM is displayed.

Oracle ILOM displays a default command prompt (->), indicating that you have successfully logged in.

Logging in to Oracle ILOM Remotely Using a Web Interface

Use the following procedure to log in to Oracle ILOM remotely using a web interface.

Before You Begin

This procedure requires that you know the administrative account name and password (defaults are root and changeme), and that you know the IP address or host name of the controller Service Processor (SP). To improve response times, disable the web browser proxy server, if used.

Type the controller SP IP address into your web browser address field.

Example: https://172.16.82.26

Type the administrative user name and password.

Click Log In.

The System Information Summary page is displayed.

Logging in to Oracle ILOM Remotely Using a Command Line Interface

Use the following procedure to log in to Oracle ILOM remotely using a command line interface.

32 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 33

Viewing and Clearing CPU Faults from ILOM

Before You Begin

Using a Secure Shell (SSH) session, log in to Oracle ILOM by specifying your administrative account user name and the IP address or host name of the controller SP.

Examples:

ssh-l username host

ssh username@host

where host is either the IP address or host name of the controller SP when using DNS. For example: ssh root@172.16.82.26

Type the password for the administrative account.

Oracle ILOM displays a default command prompt (->), indicating that you have successfully logged in.

Viewing and Clearing CPU Faults from ILOM

Use the following procedure to view and clear CPU faults from ILOM.

Before You Begin

To view server faults, use the following CLI command:

-> show /SP/faultmgmt

To clear a CPU fault, use the following CLI command:

-> set /SYS/MB/Pn clear_fault_action=true Are you sure you want to clear /SYS/MB/Pn (y/n)? y

Understanding Hardware Status

To understand hardware status, use the following sections.

■

“Locating the Chassis Serial Number” on page 34

Getting Started with Servicing the Hardware 33

Page 34

Locating the Chassis Serial Number

■

“Locating a Failed Component (BUI)” on page 34

■

“Locating a Failed Component (CLI)” on page 34

Locating the Chassis Serial Number

To obtain support for your storage controller or to order new parts, you need your chassis serial number. There are three ways to locate the chassis serial number

You can find a chassis serial number label on the storage controller front panel on the left side. Another label is on the top of the storage controller.

You can click the Sun/Oracle logo in the BUI masthead to obtain the serial number.

You can use the following CLI command:

hostname: maintenance hardware show

Locating a Failed Component (BUI)

Use the following procedure to locate a failed component using the BUI.

Go to the Maintenance > Hardware screen.

Click the right-arrow icon on the Storage System or Disk Shelf which has the fault icon.

Locate the fault icon in the lists of hardware components, and click it. The image should be updated to show where that component is physically located.

(Optional) Click the locator icon for that component, if the component has it. The LED on the component will begin to flash.

Locating a Failed Component (CLI)

Use the following procedure to locate a failed component using the CLI.

34 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 35

Go to the maintenance hardware context:

hostname:> maintenance hardware

List the appliance components:

hostname:maintenance hardware> list NAME STATE MODEL SERIAL chassis-000 hostname ok Sun Storage 7320 unknown chassis-001 000000000C faulted J4410 000000000C

Select the chassis and list its components:

hostname:maintenance hardware> select chassis-001 hostname:maintenance chassis-001> list disk fan psu slot

Locating a Failed Component (CLI)

Select the component type and show all available disks:

hostname:maintenance chassis-001> select disk hostname:maintenance chassis-001 disk> show Disks: LABEL STATE MANUFACTURER MODEL SERIAL disk-000 HDD 0 ok ST3500630NS ST3500630NS 9QG1ACNJ disk-001 HDD 1 faulted ST3500630NS ST3500630NS 9QG1A77R disk-002 HDD 2 ok ST3500630NS ST3500630NS 9QG1AC3Z disk-003 HDD 3 ok ST3500630NS ST3500630NS 9QG1ACKW disk-004 HDD 4 ok ST3500630NS ST3500630NS 9QG1ACKF disk-005 HDD 5 ok ST3500630NS ST3500630NS 9QG1ACPM disk-006 HDD 6 ok ST3500630NS ST3500630NS 9QG1ACRR disk-007 HDD 7 ok ST3500630NS ST3500630NS 9QG1ACGD disk-008 HDD 8 ok ST3500630NS ST3500630NS 9QG1ACG4 disk-009 HDD 9 ok ST3500630NS ST3500630NS 9QG1ABDZ disk-010 HDD 10 ok ST3500630NS ST3500630NS 9QG1A769 disk-011 HDD 11 ok ST3500630NS ST3500630NS 9QG1AC27 disk-012 HDD 12 ok ST3500630NS ST3500630NS 9QG1AC41 disk-013 HDD 13 ok ST3500630NS ST3500630NS 9QG1ACQ5 disk-014 HDD 14 ok ST3500630NS ST3500630NS 9QG1ACKA disk-015 HDD 15 ok ST3500630NS ST3500630NS 9QG1AC5Y disk-016 HDD 16 ok ST3500630NS ST3500630NS 9QG1ACQ2 disk-017 HDD 17 ok ST3500630NS ST3500630NS 9QG1A76S

Getting Started with Servicing the Hardware 35

Page 36

Working with Problems

disk-018 HDD 18 ok ST3500630NS ST3500630NS 9QG1ACDY disk-019 HDD 19 ok ST3500630NS ST3500630NS 9QG1AC3Y disk-020 HDD 20 ok ST3500630NS ST3500630NS 9QG1ACG6 disk-021 HDD 21 ok ST3500630NS ST3500630NS 9QG1AC3X disk-022 HDD 22 ok ST3500630NS ST3500630NS 9QG1ACHL disk-023 HDD 23 ok ST3500630NS ST3500630NS 9QG1ABLW

Note - The RPM (revolutions per minute) of the disk drive is also shown in the output.

However, RPM is truncated in the above example.

Select the faulted disk and turn on the locator LED:

hostname:maintenance chassis-001 disk> select disk-001 hostname:maintenance chassis-001 disk-001> set locate=true locate = true (uncommitted) hostname:maintenance chassis-001 disk-001> commit

Working with Problems

To aid serviceability, the appliance detects persistent hardware failures (faults) and software failures (defects, often included under faults) and reports them as active problems on this screen. If the phone home service is enabled, active problems are automatically reported to Oracle Support, where a support case may be opened depending on the service contract and the nature of the fault.

Active Problems Display

For each problem, the appliance reports what happened, when the problem was detected, the severity and type of the problem, and whether it has been phoned home. Below are some example faults as they would be displayed in the BUI:

TABLE 9

Date Description Type Phoned Home

2009-09-16 13:56:36 SMART health-monitoring firmware

2009-09-05 17:42:55 A disk of a different type (cache, log, or

Example BUI Problem Displays

Major Fault Never

reported that a disk failure is imminent.

Minor Fault Never data) was inserted into a slot. The newly inserted device must be of the same type.

36 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 37

Working with Problems

Date Description Type Phoned Home

2009-08-21 16:40:37 The ZFS pool has experienced currently

unrecoverable I/O failures.

2009-07-16 22:03:22 A memory module is experiencing excessive

correctable errors affecting large numbers of pages.

Major Error Never

Major Fault Never

This information can also be viewed in the CLI:

hostname:> maintenance problems show Problems:

COMPONENT DIAGNOSED TYPE DESCRIPTION problem-000 2010-7-27 00:02:49 Major Fault SMART health-monitoring firmware reported that a failure is imminent on disk 'HDD 17'.

Selecting any fault shows more information about the fault including the impact to the system, affected components, the system's automated response (if any), and the recommended action for the administrator (if any). In the CLI, only the "uuid", "diagnosed", "severity", "type", and "status" fields are considered stable. Other property values may change from release to release.

For hardware faults, you may be able to select the affected hardware component to locate it on the Maintenance > Hardware screen.

Repairing Active Problems

Active problems can be a result of a hardware fault or software defect. To repair an active problem, perform the steps described in the suggested action section. A repair typically involves replacing a physical component (for hardware faults) or reconfiguring and restarting the affected service (for software defects). After repairing a problem, it no longer appears in the list of active problems.

While the system can detect repairs automatically, in some cases manual intervention is required. If a problem persists after completing the suggested action, contact Oracle support. You might be instructed to mark the problem as repaired. Manually marking a problem as repaired should only be done under the direction of Oracle service personnel or as part of a documented Oracle repair procedure.

Getting Started with Servicing the Hardware 37

Page 38

Using Logs

Related Features

■

Using Logs

This section describes different types of log entries and how to use logs:

■

A persistent log of all faults and defects is available under Logs as the Fault log. For more information, see “Fault Log Entries” on page 39.

Faults and defects are subcategories of Alerts. Filter rules can be configured to cause the appliance to email administrators or perform other actions when faults are detected. For more information about alerts, see “Configuring Alerts” in Oracle ZFS Storage Appliance

Administration Guide, Release OS8.6.0.

“Alert Log Entries” on page 38

“Fault Log Entries” on page 39

“System Log Entries” on page 40

“Audit Log Entries” on page 40

“Phone Home Log Entries” on page 41

“Viewing Logs (BUI)” on page 41

“Listing Logs (CLI)” on page 41

“Viewing All Log Entries (CLI)” on page 42

“Viewing Groups of Log Entries (CLI)” on page 42

“Viewing Entry Details (CLI)” on page 44

“Exporting Logs (BUI)” on page 44

“Exporting Logs (CLI)” on page 46

Alert Log Entries

This is the appliance alert log, recording key events of interest during appliance operation.

The following are example alert log entries as they would appear in the BUI:

TABLE 10

Time Event ID Description Type

2013-9-16 13:01:56

38 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Example BUI Alert Log Entry Displays

f18bbad18084-4cab-c95082ef5b8228ea

An I/O path from slot 'PCIe 0' to chassis 'JBOD #1' has been removed.

Major alert

Page 39

Using Logs

Time Event ID Description Type

2013-9-16 13:01:51

8fb8688c08f2-c994-a6a5ac6e755e53bb

446654fcb898-6da5-e87e8d23ff12d5d0

A disk has been inserted into slot 'HDD 4' of chassis 'JBOD #1'.

A disk has been inserted into slot 'HDD 15' of chassis 'JBOD #1'.

Minor alert

An information icon next to the Event ID means that extended information is available. Click the icon to display information below the list of alerts.

The appliance can also be configured to send email, raise an SNMP trap, or perform other actions when particular alerts occur. This is configured in the Alerts section. All alerts appear in this log, regardless of whether they have actions configured for them. For more information about alerts, see “Configuring Alerts” in Oracle ZFS Storage Appliance Administration Guide,

Release OS8.6.0.

Fault Log Entries

The fault log records hardware and software faults. This is a useful reference when troubleshooting hardware failure, as timestamps are available for these hardware fault events.

The following are example fault log entries as they would appear in the BUI:

TABLE 11

Date Event ID Description Type

2013-9-5 17:42:359e46fc0b-b1a4-4e69-f10f-

2013-9-3 19:20:15d37cb5cd-88a8-6408-e82d-

2013-8-21 16:40:48c91c7b32-83ce-6da8-e51e-

These faults generate alert log entries that use the alert reporting settings (such as sending email), if configured. Faults that require administrator attention appear in Problems. For more information, see “Working with Problems” on page 36.

Example BUI Fault Log Entry Displays

e7dbe80794fe

c05576c52279

a553964bbdbc

The device 'HDD 6' has failed or could not be opened.

External sensors indicate that a fan is no longer operating correctly.

The ZFS pool has experienced currently unrecoverable I/O failures.

Major Fault

Minor Fault

Major Error

Getting Started with Servicing the Hardware 39

Page 40

Using Logs

System Log Entries

This is the operating system log, available to read via the appliance interfaces. This may be useful when troubleshooting complex issues, but should only be checked after first examining the alert and fault logs.

The following are example system log entries as they would appear in the BUI:

TABLE 12

Time Module Priority Description

2013-10-11 14:13:38 ntpdate error no server suitable for synchronization found

2013-10-11 14:03:52 genunix notice SunOS Release 5.11 Version ak/

2013-10-11 14:02:04 genunix notice done

2013-10-11 14:02:01 genunix notice syncing file systems...

2013-10-11 13:52:16 nxge warning WARNING: nxge : ==>

Example BUI System Log Entry Displays

generic@2013.10.10,1-0 64-bit

nxge_rxdma_databuf_free: DDI

Audit Log Entries

The audit log records user activity events, including login and logout to the BUI and CLI, and administrative actions. If session annotations are used (see “Configuring Users” in Oracle ZFS

Storage Appliance Administration Guide, Release OS8.6.0), each audit entry should be noted

with a reason.

The following are example audit log entries as they would appear in the BUI:

TABLE 13

Example BUI Audit Log Entry Displays

Time User Host Summary Session Annotation

2013-10-12 05:20:24 root deimos Disabled ftp service

2013-10-12 03:17:05 root deimos User logged in

2013-10-11 22:38:56 root deimos Browser session

2013-10-11 21:13:35 root <console> Enabled ftp service

40 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

timed out

Page 41

Viewing Logs (BUI)

Phone Home Log Entries

If Phone Home is used, this log shows communication events with Oracle support. For information about Phone Home, see “Phone Home Configuration” in Oracle ZFS Storage

Appliance Administration Guide, Release OS8.6.0.

The following is an example phone home entry as it would appear in the BUI:

TABLE 14

Time Description Result

2013-10-12 05:24:09 Uploaded file 'cores/ak.45e5ddd1-ce92-c16e-b5eb-

Example BUI Phone Home Entry Display

9cb2a8091f1c.tar.gz' to Oracle support

Viewing Logs (BUI)

Use the following procedure to view logs using the BUI. The following log types are available: Alerts, Faults, System, Audit, and Phone Home.

To view logs, navigate to the Maintenance > Logs screen.

To navigate between the log types, click the Alerts, Faults, System, Audit, and Phone Home buttons.

To scroll through the logs, use the forward and back buttons.

Listing Logs (CLI)

Use the following procedure to list logs using the CLI. Up to 100 recent entries for each log are visible.

To list available logs and the time stamp of the last log entry for each log type, use the following CLI commands:

hostname:> maintenance logs hostname:maintenance logs> show Logs:

Getting Started with Servicing the Hardware 41

Page 42

Viewing All Log Entries (CLI)

LOG ENTRIES LAST alert 2 2013-10-16 02:44:04 audit 42 2013-10-16 18:19:53 fltlog 2 2013-10-16 02:44:04 scrk 0 system 100 2013-10-16 03:51:01

Viewing All Log Entries (CLI)

Use the following procedure to view all log entries.

To view all log entries, use the following CLI commands. The most recent entries are displayed at the bottom of the list.

hostname:maintenance logs> select audit hostname:maintenance logs audit> list -a

ENTRY TIME SUMMARY entry-000 2013-1-9 15:13:50 root, 10.154.161.197, User logged in entry-001 2013-1-9 15:17:42 root, 10.154.161.197, Released resources to cluster peer entry-002 2013-1-9 15:20:17 root, 10.154.161.197, Cluster takeover requested entry-003 2013-1-9 15:20:33 root, 10.154.161.197, Transferred ak:/net/igb1 to 2917b8aa-0b0a-4b74-f36b-ff0a8d150c3b ... entry-2077 2013-12-17 05:24:43 osc_agent, 10.80.218.16, User logged in entry-2078 2013-12-17 05:24:45 osc_agent, 10.80.218.16, User logged out entry-2079 2013-12-17 05:24:53 osc_agent, 10.80.218.16, User logged in

Viewing Groups of Log Entries (CLI)

Use the following procedure to view groups of log entries.

To view up to 100 log entries at a time, use the following CLI commands:

hostname:maintenance logs> select audit hostname:maintenance logs audit> show

ENTRY TIME SUMMARY entry-2874 2014-2-20 18:05:44 root, 10.159.134.211, Beginning system update

42 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 43

Viewing Groups of Log Entries (CLI)

to ak-nas@2014.02.11,1-0 entry-2875 2014-2-20 18:08:18 root, 10.159.134.211, Rebooting system as part of update to ak-nas@2014.02.11,1-0 entry-2876 2014-2-20 18:08:20 root, 10.159.134.211, User logged out entry-2877 2014-2-20 18:16:06 root, 10.159.134.211, User logged in entry-2878 2014-2-20 18:18:20 root, 10.159.134.211, Deleted update media ak-nas@2011.04.24.8.0,1-1.43 entry-2879 2014-2-20 18:34:19 root, 10.159.134.211, Cluster takeover requested ... entry-2970 2014-2-27 17:40:37 root, dhcp-amer-vpn-rmdc-anyconnect-10-159-100-157.vpn.oracle. com, User logged out of CLI entry-2971 2014-2-27 17:48:23 root, 10.159.100.157, User logged in entry-2972 2014-2-27 19:05:46 root, 10.159.100.157, Browser session timed out entry-2973 2014-2-27 23:53:31 root, dhcp-amer-vpn-rmdc-anyconnect-10-159-125-20.vpn.oracle. com, User logged in via CLI

To view the previous or next 100 log entries, use the following CLI commands:

hostname:maintenance logs alert> list

ENTRY TIME SUMMARY entry-023 2013-1-16 15:06:36 d1dd862d-93ff-6e3d-a51c-fe5f81159a3e, System software update cannot proceed: J4400 and J4500 disk shelves are not supported in this ... entry-122 2013-2-17 20:34:06 65f79e6b-1a77-6041-9a92-9440dbba4c40, Finished replicating project 'Test1' from appliance 'AdamZhang'., Minor Alert

hostname:maintenance logs alert> previous

ENTRY TIME SUMMARY entry-000 2013-11-21 15:45:23 67ccd46c-3d4d-eb86-8966-f9e0ec497293, System software update cannot proceed: J4400 and J4500 disk shelves are not supported in this release., Minor alert ... entry-022 2013-1-16 15:06:10 2f840123-221c-49dc-ae26-e5bfe0952599, System software update cannot proceed: see alert log for condition(s) that are preventing upgrade, Minor alert

Getting Started with Servicing the Hardware 43

Page 44

Viewing Entry Details (CLI)

To view a specific group of log entries at one time, use the following CLI commands:

hostname:maintenance logs> select audit hostname:maintenance logs audit> list -5

ENTRY TIME SUMMARY entry-721 2013-2-18 22:02:02 <system>, <system>, Request to create replication package entry-722 2013-2-18 22:02:08 <system>, <system>, Request to modify replication package entry-723 2013-2-18 23:33:32 root, 10.135.69.243, User logged in entry-724 2013-2-19 00:48:51 root, 10.135.69.243, Browser session timed out entry-725 2013-2-19 23:30:11 root, User logged in via CLI

Viewing Entry Details (CLI)

Use the following procedure to view entry details using the CLI.

To view details of a log entry, use the following CLI commands:

hostname:maintenance logs> select audit hostname:maintenance logs audit> select entry-000 show Properties: timestamp = 2013-10-15 00:59:37 user = root address = <console> summary = Enabled datalink:igb0 service annotation =

Note - The "annotation" is the session annotation, which can be enabled when configuring

users. For information about user configuration, see “Configuring Users” in Oracle ZFS

Storage Appliance Administration Guide, Release OS8.6.0.

Exporting Logs (BUI)

Use the following procedure to export logs using the BUI. You can export any combination of log types to your local machine by using the BUI Collect button.

To view logs, navigate to the Maintenance > Logs screen.

44 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 45

Click Collect.

Exporting Logs (BUI)

FIGURE 1

In the Collect Logs dialog box, choose the log types you want to export.

Logs Screen

To export the logs, click Apply. A <stdout> gz file containing all the requested log information is downloaded to your local machine.

Getting Started with Servicing the Hardware 45

Page 46

Exporting Logs (CLI)

Use the following procedure to export logs using the CLI.

To export logs, use the following CLI commands. A <stdout> gz file containing all the requested log information is downloaded to your local machine.

hostname:maintenance logs> show Logs:

LOG ENTRIES LAST alert 3458 2013-2-18 23:02:29 audit 731 2013-2-20 16:13:04 fltlog 2 2013-2-3 06:29:02 scrk 0 system 44 2013-1-14 18:19:59

hostname:maintenance logs> select fltlog hostname:maintenance logs fltlog> collect

SUNW-MSG-ID: AK-8000-86, TYPE: Defect, VER: 1, SEVERITY: Major EVENT-TIME: Wed Nov 13 03:50:15 UTC 2013 PLATFORM: Sun-Fire-X4240, CSN: 0000000000, HOSTNAME: hpc-iwashi-01 SOURCE: ak-diagnosis, REV: 1.0 EVENT-ID: d83655cd-c03d-efff-efde-9c78dd47259e DESC: The service processor needs to be reset to ensure proper functioning. AUTO-RESPONSE: None. IMPACT: Service Processor-controlled functionality, including LEDs, fault management, and the serial console, may not work correctly. REC-ACTION: Click the initiate repair button.

SUNW-MSG-ID: FMD-8000-4M, TYPE: Repair, VER: 1, SEVERITY: Major EVENT-TIME: Wed Nov 13 03:50:15 UTC 2013 PLATFORM: Sun-Fire-X4240, CSN: 0000000000, HOSTNAME: hpc-iwashi-01 SOURCE: ak-diagnosis, REV: 1.0 EVENT-ID: d83655cd-c03d-efff-efde-9c78dd47259e DESC: All faults associated with an event id have been addressed. AUTO-RESPONSE: Some system components offlined because of the original fault may have been brought back online. IMPACT: Performance degradation of the system due to the original fault may have been recovered. REC-ACTION: No action is required.

hostname:maintenance logs fltlog>

46 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 47

Using Workflows

A workflow is a script that is uploaded to and managed by the Oracle ZFS Storage Appliance by itself. For information on scripting, see “Working with CLI Scripting” in Oracle ZFS

Storage Appliance Administration Guide, Release OS8.6.0. Workflows can be parameterized

and executed in a first-class fashion from either the browser user interface (BUI) or the command line interface (CLI). For information on maintenance workflows, see “Maintenance

Workflows” in Oracle ZFS Storage Appliance Administration Guide, Release OS8.6.0.

Workflows may also be optionally executed as alert actions or at a designated time. As such, workflows allow for the Oracle ZFS Storage Appliance to be extended in ways that capture specific policies and procedures, and can be used to formally encode best practices for a particular organization or application. For information on alert actions, see “Configuring Alerts”

in Oracle ZFS Storage Appliance Administration Guide, Release OS8.6.0

Using Workflows

Getting Started with Servicing the Hardware 47

Page 48

48 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 49

Prerequisites for Servicing Hardware

This section describes the prerequisites for servicing hardware. See the followings for details:

■

“Safety Information” on page 49

■

“Required Tools and Information” on page 50

■

“Powering Off the Controller” on page 50

■

“Extending the Storage Controller from the Rack” on page 51

■

“Removing the Top Cover” on page 52

■

“Powering Off the Disk Shelf” on page 53

Safety Information

This section contains safety information that you must follow when servicing the storage system. For your protection, observe the following safety precautions when setting up your equipment:

■

Do not remove the side panels, or run the storage system with the side panels removed. Hazardous voltage is present that could cause injury. The covers and panels must be in place for proper air flow to prevent equipment damage.

■

Follow all cautions, warnings, and instructions marked on the equipment and described in Important Safety Information for Oracle's Hardware Systems included with your system.

■

Ensure that the voltage and frequency of your power source match the voltage inscribed on the electrical rating label.

■

Follow the electrostatic discharge safety practices. Electrostatic discharge (ESD) sensitive devices, such as PCI cards, HDDs, SSDs, and memory cards, require special handling. Circuit boards and HDDs contain electronic components that are extremely sensitive to static electricity. Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards. Do not touch the components without using antistatic precautions, especially along the connector edges.

Prerequisites for Servicing Hardware 49

Page 50

Required Tools and Information

Note - The Critical status indicator on the front panel flashes red when powering off and

powering on the ZS3-2. After the controller is powered on, the Power/OK status indicator is steady green, and the Red Critical status indicator is off.

Required Tools and Information

The following tools are needed to service the CRUs:

■

Antistatic wrist strap - Wear an antistatic wrist strap and use an antistatic mat when handling components such as HDDs or PCI cards. When servicing or removing storage controller components, attach an antistatic strap to your wrist and then to a metal area on the chassis. Following this practice equalizes the electrical potentials between you and the storage controller.

■

Antistatic mat - Place static-sensitive components on an antistatic mat.

■

No. 2 Phillips screwdriver

■

Nonconducting, No.1 flat-blade screwdriver or equivalent

■

Nonconducting stylus or pencil (to power on the storage controller)

Powering Off the Controller

Some replacement tasks require you to power off the controller. Note that there will be a loss of access to the storage unless the system is in a clustered configuration. The recommended methods for powering off a controller depend on whether you have completed initial configuration. Use a power off method from one of the following two situations.

If you have completed initial configuration, power off the controller using one of the following methods:

■

SSH in to the appliance and enter the maintenance system poweroff command.

■

Use a pen or non-conducting pointed object to press and release the Power button on the front panel.

If you have not completed the initial configuration steps, power off the controller using one of the following methods:

■

Use a pen or non-conducting pointed object to press and release the Power button on the front panel.

50 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 51

Extending the Storage Controller from the Rack

■

SSH or use a serial connection to log in to the service processor (SP), and then issue the command stop /SYS.

■

To initiate an emergency shutdown during which all applications and files will be closed abruptly without saving, press and hold the Power button for at least four seconds until the Power/OK status indicator on the front panel flashes, indicating that the storage controller is in standby power mode. To completely remove power, disconnect the AC power cords from the rear panel of the storage controller.

Related Topics

■

“Powering Off the Disk Shelf” on page 53

Extending the Storage Controller from the Rack

Use the following procedure to extend the storage controller from the rack. If you cannot extend the controller without removing power, first see “Powering Off the Controller” on page 50.

Disconnect all cables from the back of the storage controller. Be sure the cables are labeled for proper connection later.

From the front of the storage controller, release the two slide release latches.

Prerequisites for Servicing Hardware 51

Page 52

Removing the Top Cover

While squeezing the slide release latches, slowly pull the storage controller forward until the slide rails latch.

Removing the Top Cover

To access internal controller components for servicing, remove the top cover. Use the following procedure to remove the top cover.

Fully loosen the two captive screws at the rear of the top cover (1).

52 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 53

Powering Off the Disk Shelf

Slide the top cover rearward 0.5 in (13 mm) and lift it straight up and off the chassis (2), and set the top cover aside.

Powering Off the Disk Shelf

Most disk shelf components are hot-swappable; you do not need to remove power when replacing them. Do not remove a component if you do not have an immediate replacement. The disk shelf must not be operated without all components in place.

If a disk shelf is a part of a storage pool, powering off or removing all SAS chains from a disk shelf will trigger a kernel panic on the controller(s) to prevent data loss, unless the shelf is part of an NSPF (no single point of failure) data pool. To avoid this, shut down the controller(s) before decommissioning the shelf, as described in “Powering Off the

Controller” on page 50.

Stop all input and output to and from the disk shelf.

Wait approximately two minutes until all disk activity indicators have stopped flashing.

Place the power supply on/off switches to the "O" off position.

Prerequisites for Servicing Hardware 53

Page 54

Powering Off the Disk Shelf

Disconnect the power cords from the external power source for the cabinet.

Note - All power cords must be disconnected to completely remove power from the disk shelf.

Related Topics

■

“Powering Off the Controller” on page 50

■

For details on NSPF profiles, see “Storage Pool Concepts” in Oracle ZFS Storage

Appliance Administration Guide, Release OS8.6.0

54 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 55

Servicing the ZS4-4 Controller

Be sure to read sections “Getting Started with Servicing the Hardware” on page 15 and

“Prerequisites for Servicing Hardware” on page 49.

Most hardware faults clear after replacing the affected component. If a fault does not self-clear, see “Repairing Active Problems” on page 37.

To replace ZS4-4 hardware components, use the following tasks:

■

“Replacing a ZS4-4 HDD or SSD” on page 55

■

“Replacing a ZS4-4 Power Supply” on page 57

■

“Replacing a ZS4-4 Fan Module” on page 58

■

“Identifying a Faulted ZS4-4 Memory Module” on page 60

■

“Replacing a ZS4-4 DIMM” on page 62

■

“Replacing a ZS4-4 Memory Riser Card” on page 64

■

“Replacing a ZS4-4 PCIe Card” on page 66

■

“Replacing a ZS4-4 Battery” on page 71

To understand the ZS4-4 controller, use the following topics:

■

“ZS4-4 Base Configuration” on page 74

■

“ZS4-4 Front and Rear Panel Components” on page 75

■

“ZS4-4 PCIe I/O Cards” on page 79

■

“ZS4-4 Internal Components” on page 83

■

“ZS4-4 Attached Storage” on page 88

Replacing a ZS4-4 HDD or SSD

HDDs and SSDs are hot-swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities. Use the following procedure to replace a ZS4-4 HDD or SSD. For information on disk compatibility, see the Oracle Systems

Handbook.

Servicing the ZS4-4 Controller 55

Page 56

Replacing a ZS4-4 HDD or SSD

Note - If you have more than one failed drive, replace only one drive at a time. Removing more

than one drive in quick succession will cause a hardware/pool fault. Also, do not perform a pool configuration operation while a disk firmware upgrade is occurring. To check if an upgrade is in progress, navigate to Maintenance > System in the BUI, or maintenance system updates in the CLI.

Identify the failed HDD or SSD.

If you are not physically at the system, go to the Maintenance > Hardware section of the BUI, click the right-arrow icon for the controller, and note the ID of the faulted drive, which has an amber icon. To highlight the location of the drive in the controller picture, click on the drive ID. To view details about the faulted drive, click its information icon and then click Active Problems.

If you are physically at the system, the amber Service action required indicator on the HDD

or SSD should be lit. Additionally, you can click the locator icon in the BUI to flash the locator indicator on the faulted drive.

Verify that the Ready to Remove indicator is lit on the faulted drive.

On the drive you want to remove, push the drive release button (1) to open the latch.

Grasp the latch (2) and pull the drive out of the drive slot (3).

After at least 30 seconds, go to the Hardware > Maintenance screen, and click the right-arrow icon on the system controller to verify that the software has detected that the drive is not present.

56 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 57

Replacing a ZS4-4 Power Supply

On the replacement drive, open the release lever and slide the drive into the same slot (1) until it is fully seated. Close the latch (2) to lock the drive in place.

The Oracle ZFS Storage Appliance software automatically detects and configures the new drive.

Go to the Maintenance > Hardware screen of the BUI, click the right-arrow

icon for the controller, and verify that the status icon is green for the newly installed drive. To view details about the new drive, click its information icon.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Replacing a ZS4-4 Power Supply

Storage controllers are equipped with redundant, hot-swappable power supplies. If a power supply fails and you do not have a replacement available, leave the failed power supply installed to ensure proper air flow. A faulted power supply is indicated by an amber-colored status LED. Use the following procedure to replace a ZS4-4 power supply.

Gain access to the rear of the storage controller where the power supplies are located.

Disconnect the power cord from the faulted power supply.

To unlock the power supply handle, squeeze the release latch.

Servicing the ZS4-4 Controller 57

Page 58

Replacing a ZS4-4 Fan Module

Rotate the release latch fully downward, to disengage the power supply from the internal power supply backplane connector.

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Ensure that the replacement power supply handle is in the open position.

Align the replacement power supply with the empty power supply bay.

Slide the power supply into the bay (1) until it stops.

To seat the power supply, rotate the release handle upward (2) until it locks in place. This action of rotating the handle upward draws the power supply into the controller and engages it with the internal connector.

Note - The latch at the hinge of the handle must engage the slot at the bottom of the power

supply bay.

10.

Connect the power cord to the power supply.

11.

Verify that the green AC Present status indicator is lit.

12.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller, click PSU, and verify that the status icon is green for the newly installed power supply. To view details about the new power supply, click its information icon.

13.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Replacing a ZS4-4 Fan Module

Caution - Operating a controller for an extended period of time with fans removed reduces the

effectiveness of the cooling system. For this reason, the replacement fan should be unpacked in advance and ready to insert into the controller chassis as soon as the faulted fan is removed.

58 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 59

Replacing a ZS4-4 Fan Module

Fan modules cool the motherboard and its components. The fan modules are arranged in stacked pairs (a front row and a back row) to provide redundancy. If you remove a failed fan module, replace it immediately. Fan modules are hot-swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities. Use the following procedure to replace a ZS4-4 fan module.

To flash the locator LED on the chassis to service, go to Maintenance >

Hardware in the BUI and click the associated locator icon , or go to the service processor (SP) prompt and enter set /SYS/LOCATE status=on

To locate the faulted component, go to Maintenance > Hardware > Fan and click the row with an amber indicator.

The component is highlighted in the diagram.

At the back of the controller, verify that cables have sufficient length and clearance to extend the controller from the rack.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Remove the top cover as described in “Removing the Top Cover” on page 52.

Lift the green handle at the top of the fan module to unlock the fan module, and pull the fan module straight up and out.

Caution - Excessive movement or rocking of the fan module during removal can cause damage

to the internal connector on the fan module board. When removing a fan module, do not rock it back and forth.

Position the replacement fan module so the connector on the bottom of the module is aligned with the connector on the inside of the slot and all keying and

Servicing the ZS4-4 Controller 59

Page 60

Identifying a Faulted ZS4-4 Memory Module

labeling are correctly positioned. The fan modules are keyed to ensure that they are installed in the correct orientation.

Slide the fan module into the slot until it stops.

Press downward on the top of the fan module, on the Press Here to Latch label, until the fan module is fully seated.

10.

Replace the top cover on the chassis.

11.

Return the controller to the normal rack position by pushing the release tabs on the side of each rail while slowly pushing the storage controller into the rack.

12.

Verify that the green OK indicator is lit.

13.

Verify that the Top Fan indicator and the Service Action Required indicator on the front panel are not lit.

14.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller, click Fan, and verify that the status icon is green for the newly installed fan module.

15.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Identifying a Faulted ZS4-4 Memory Module

Replacing a DIMM is a cold-swap procedure, and you must remove power from the storage controller. Use the following procedure to identify a faulted ZS4-4 memory module.

To identify a general memory fault, go to the Maintenance > Hardware screen of the BUI, and click the right-arrow icon for the controller. Then click DIMM and note the ID of the faulted module, which has an amber icon. To highlight the

60 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 61

Identifying a Faulted ZS4-4 Memory Module

location of the memory module in the controller picture, click on the module ID. To view details about the faulted module, click its information icon and then click Active Problems.

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Disconnect the AC power cords from the rear panel of the storage controller.

Caution - Because 3.3 VDC standby power is always present in the system, you must unplug

the power cords before accessing any cold-serviceable components.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Remove the top cover as described in “Removing the Top Cover” on page 52.

To locate the faulted component, first press and hold the Fault Remind button to verify that the remind circuitry is usable. The Fault Remind button is located on the divider between cooling zone 1 and cooling zone 2. The Power LED, next to the button, is green when the remind circuitry is usable.

■

If a memory riser card has failed, see “Replacing a ZS4-4 Memory Riser

Card” on page 64 for removal and installation instructions.

■

If a DIMM has failed, see “Replacing a ZS4-4 DIMM” on page 62 for removal and installation instructions.

■

If a CPU has failed, the LEDs for both memory riser cards associated with the failed CPU turn on. The following example, shows the Fault indicators for memory riser cards, P0/MR0 and P0/MR1 are lit, as is the Fault indicator for CPU, P0.

Servicing the ZS4-4 Controller 61

Page 62

Replacing a ZS4-4 DIMM

Caution - A CPU is a field replaceable unit (FRU) and must be replaced by a trained Oracle

service technician.

Replacing a ZS4-4 DIMM

Use the following procedure to replace a ZS4-4 DIMM.

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Disconnect the AC power cords from the rear panel of the storage controller.

Caution - Because 3.3 VDC standby power is always present in the system, you must unplug

the power cords before accessing any cold-serviceable components.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Remove the top cover as described in “Removing the Top Cover” on page 52.

Remove the memory riser card with the lighted Fault indicators (P0/MR0 and P0/MR1), and locate the failed DIMM. Use the DIMM Fault Remind circuit to locate the failed DIMM. See “Identifying a Faulted ZS4-4 Memory

Module” on page 60.

62 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 63

Replacing a ZS4-4 DIMM

To remove the failed DIMM, rotate both DIMM slot ejector levers outward as far as they will go.

Carefully lift the DIMM straight up to remove it from the socket.

Repeat the previous steps for any other faulty DIMMs.

Ensure that the DIMM ejector levers at both ends of the DIMM slot are in a fully open position.

10.

Align the replacement DIMM with the empty slot.

The DIMM is keyed with a notch that must align with a protrusion in the DIMM slot. The keying ensures that the DIMM is installed correctly.

11.

Gently and evenly push the DIMM into the slot until the ejector levers rise.

The levers rise as the DIMM is pushed further into the slot.

12.

Ensure that the levers have risen completely and have locked the DIMM in the slot.

Caution - Open DIMM ejector levers can break off during the installation of the memory riser

card. All populated and unpopulated DIMM slot ejector levers on the memory riser card must be in the fully closed position before installing the card in the controller. Ensure that all levers are closed and locked.

Servicing the ZS4-4 Controller 63

Page 64

Replacing a ZS4-4 Memory Riser Card

13.

Replace the memory riser card. See “Replacing a ZS4-4 Memory Riser

Card” on page 64.

14.

Replace the top cover on the chassis.

15.

Return the controller to the normal rack position by pushing the release tabs on the side of each rail while slowly pushing the storage controller into the rack.

16.

Reconnect all cables to their original ports, referring to the labels previously added as a guide. For cabling details, see the Oracle ZFS Storage Appliance

Cabling Guide.

17.

Reconnect the power cords to the power supplies.

18.

Verify that standby power is on, indicated by the Power/OK status indicator flashing on the front panel about two minutes after the power cords are plugged in.

19.

Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel.

The Power/OK status indicator next to the Power button lights and remains lit.

20.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller, click DIMM, and verify that the status icon is green for the newly installed DIMM. To view details about the new DIMM, click its information icon.

21.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Replacing a ZS4-4 Memory Riser Card

Use the following procedure to replace a ZS4-4 memory riser card.

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Disconnect the AC power cords from the rear panel of the storage controller.

Caution - Because 3.3 VDC standby power is always present in the system, you must unplug

the power cords before accessing any cold-serviceable components.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

64 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 65

Replacing a ZS4-4 Memory Riser Card

Remove the top cover as described in “Removing the Top Cover” on page 52.

To remove the memory riser card, pull the handles upward to disengage the connector from the motherboard, and carefully lift the memory riser straight up and out of the controller. The handles act as levers against the sidewalls to extract the card connector from the connector on the motherboard.

On the replacement memory riser card, ensure that all populated and unpopulated DIMM slot ejector levers are in the closed and locked position.

Caution - Open DIMM ejector levers can break off during the installation of the memory riser

card. All populated and unpopulated DIMM slot ejector levers on the memory riser card must be in the fully closed and locked position before installing the card in the controller.

In the controller, ensure that the memory riser card ejector levers are in the closed position.

Note - The ejector levers are only used to remove a memory riser card and are not used to

install a card.

Position the memory riser card over the slot on the motherboard.

The DIMMs on the card must face to the left (when positioned at the front of the controller).

Lower the memory riser card into the controller and set it on the slot on the motherboard.

Servicing the ZS4-4 Controller 65

Page 66

Replacing a ZS4-4 PCIe Card

10.

Ensure that the memory riser card connector is aligned with the slot.

11.

Firmly push down on the metal bracket on top of the card, to seat the card within the connector on the motherboard.

12.

Replace the top cover on the chassis.

13.

Return the controller to the normal rack position by pushing the release tabs on the side of each rail while slowly pushing the storage controller into the rack.

14.

Reconnect all cables to their original ports, referring to the labels previously added as a guide. For cabling details, see the Oracle ZFS Storage Appliance

Cabling Guide.

15.

Reconnect the power cords to the power supplies.

16.

Verify that standby power is on, indicated by the Power/OK status indicator flashing on the front panel about two minutes after the power cords are plugged in.

17.

Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel.

18.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller and then click DIMM to verify that the status icon is green for the newly installed memory riser card.

19.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Replacing a ZS4-4 PCIe Card

Replacing a PCIe card is a cold-swap procedure, and you must remove power from the storage controller. If you are installing a newly released HBA, upgrade your system software before installing the HBA. All HBAs must be of the same type. Use the following procedure to replace a ZS4-4 PCle card.

Go to the Maintenance > Hardware screen of the BUI, click the right-arrow icon for the controller, click Slot, and note the ID of the faulted PCIe card, which has an amber icon. To highlight the location of the card in the controller picture, click on the card ID. To view details about the faulted card, click its information icon and then click Active Problems.

66 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 67

Replacing a ZS4-4 PCIe Card

Caution - This procedure requires that you handle components that are sensitive to static

discharge, which can cause the component to fail. To avoid damage, wear an antistatic wrist strap and use an antistatic mat when handling components.

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Disconnect the AC power cords from the rear panel of the storage controller.

Caution - Because 3.3 VDC standby power is always present in the system, you must unplug

the power cords before accessing any cold-serviceable components.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Remove the top cover as described in “Removing the Top Cover” on page 52.

Locate the PCIe card position in the storage controller.

Servicing the ZS4-4 Controller 67

Page 68

Replacing a ZS4-4 PCIe Card

To disengage the PCIe card retaining bar, push down on the bar, move it away from the controller back wall, and lift it to an upright position.

Carefully remove the PCIe card from the PCIe card slot.

Caution - Whenever you remove a PCIe card, you should replace it with another PCIe card or a

filler panel; otherwise, the controller might overheat due to improper airflow.

68 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 69

Ensure the PCIe card retaining bar is open.

Replacing a ZS4-4 PCIe Card

Servicing the ZS4-4 Controller 69

Page 70

Replacing a ZS4-4 PCIe Card

10.

Install the replacement PCIe card into the PCIe card slot.

11.

Return the retaining bar to its closed and locked position.

12.

Replace the top cover on the chassis.

13.

Return the controller to the normal rack position by pushing the release tabs on the side of each rail while slowly pushing the storage controller into the rack.

14.

Reconnect all cables to their original ports, referring to the labels previously added as a guide. For cabling details, see the Oracle ZFS Storage Appliance

Cabling Guide.

70 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 71

Replacing a ZS4-4 Battery

15.

Reconnect the power cords to the power supplies.

16.

Verify that standby power is on, indicated by the Power/OK status indicator flashing on the front panel about two minutes after the power cords are plugged in.

17.

Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel. The Power/OK status indicator next to the Power button lights and remains lit.

18.

Connect data cables to the PCIe card, routing them through the cable management arm, if used.

19.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller, click Slot, and verify that the status icon is green for the newly installed card. To view details about the new card, click its information icon.

20.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

21.

If you replaced an HBA, connect the disk shelves to it as described in

“Connecting to Attached Storage” on page 272. If you installed a new

HBA, install the disk shelf as described in “Installation Prerequisites and

Hardware Overviews” in Oracle ZFS Storage Appliance Installation Guide

and connect the expansion storage as described in “Connecting to Attached

Storage” on page 272.

Replacing a ZS4-4 Battery

Replacing a system battery is a cold-swap procedure, and you must remove power from the storage controller. You might need to replace the battery if the storage controller fails to maintain the proper time when powered off and disconnected from the network. You will need a small (No.1 flat-blade) non-metallic screwdriver or equivalent. Use the following procedure to replace a ZS4-4 system battery.

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Disconnect the AC power cords from the rear panel of the storage controller.

Caution - Because 3.3 VDC standby power is always present in the system, you must unplug

the power cords before accessing any cold-serviceable components.

Servicing the ZS4-4 Controller 71

Page 72

Replacing a ZS4-4 Battery

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Remove the top cover as described in “Removing the Top Cover” on page 52.

The battery is located toward the back of the controller between PCIe slot 1 and the power supply side wall.

If necessary, remove the PCIe card in slot 1 to access the battery. See

“Replacing a ZS4-4 PCIe Card” on page 66.

Remove the battery from the battery holder by pushing the back of the battery, negative (-) side, toward the metal tab, on the positive (+) side, and lifting the battery up and out of the battery holder.

Take care not to bend the tab when pushing against it.

Caution - Do not deform the metal tab on the positive (+) side of the battery. The metal tab

maintains the positive battery connection and secures the battery in the holder.

72 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 73

Replacing a ZS4-4 Battery

Press the new battery into the battery holder with the positive (+) side facing the metal tab that holds it in place.

If you removed the PCIe card from slot 1, re-install it. See Replacing a ZS4-4 PCIe Card.

Replace the top cover on the chassis.

10.

Return the controller to the normal rack position by pushing the release tabs on the side of each rail while slowly pushing the storage controller into the rack.

11.

Reconnect all cables to their original ports, referring to the labels previously added as a guide. For cabling details, see the Oracle ZFS Storage Appliance

Cabling Guide.

12.

Connect the power cords to the power supplies.

13.

Verify that standby power is on, indicated by the Power/OK status indicator flashing on the front panel about two minutes after the power cords are plugged in.

Servicing the ZS4-4 Controller 73

Page 74

ZS4-4 Controller Hardware Overview

14.

Use a pen, or other pointed object, to press and release the recessed Power button on the storage controller front panel. The Power/OK status indicator next to the Power button lights and remains lit.

15.

If you re-installed a PCIe card, connect the data cables to the PCIe card, routing them through the cable management arm, if used.

16.

When the system has finished booting, log in and set the time using the steps in the BUI Clock task. For more information, see “Setting Clock Synchronization

(BUI)” in Oracle ZFS Storage Appliance Administration Guide, Release OS8.6.0.

ZS4-4 Controller Hardware Overview

To understand the ZS4-4 controller, use the following topics:

■

“ZS4-4 Base Configuration” on page 74

■

“ZS4-4 Front and Rear Panel Components” on page 75

■

“ZS4-4 PCIe I/O Cards” on page 79

■

“ZS4-4 Internal Components” on page 83

■

“ZS4-4 Attached Storage” on page 88

ZS4-4 Base Configuration

The ZS4-4 controller can be configured as a standalone controller or clustered controllers to create a high-availability cluster configuration. The following table describes the base configuration.

TABLE 15

Component Description

CPU Four Intel Xeon 15-core, 2.8 GHz

Memory 1.5TB 16GB DDR3 LV RDIMM

Boot Disks Two 2.5-inch SAS-2 HDDs

Read Flash Up to four optional 2.5-inch SAS-2 SSDs

HBAs Two 4x4-port SAS-2 (base configuration)

PCIe slots 11 (4 base configuration, 7 expansion slots)

74 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

ZS4-4 Controller Base Configuration

Page 75

ZS4-4 Controller Hardware Overview

Refer to the Oracle ZFS Storage ZS4-4 data sheet at http://www.oracle.com/goto/zs4-4 for the most recent component specification and physical, electrical, and environmental specifications.

For information about supported disks and capacities, refer to the Oracle Systems Handbook.

ZS4-4 Front and Rear Panel Components

Front Panel Components - The ZS4-4 controller drive slots and front panel components are shown in the following figure.

FIGURE 2

Figure Legend Figure Legend

1 Locator LED/button (white) 9 USB 2.0 connectors (2)

2 Service action required LED (amber) 10 DB-15 video port

3 Power/OK LED (green) 11 Boot drive 0 (required)

4 Power button 12 Boot drive 1 (required)

5 Service Processor (SP) OK (green)/Fault (amber) LED 13 Solid state drive 2 (optional)

6 Fan/CPU/Memory Service action required LED

(amber)

ZS4-4 Controller Front Panel

14 Solid state drive 3 (optional)

Servicing the ZS4-4 Controller 75

Page 76

ZS4-4 Controller Hardware Overview

Figure Legend Figure Legend

7 Power Supply (PS) Service action required LED (amber)

8 Over temperature warning LED (amber) 16 Solid state drive 5 (optional)

Rear Panel Components - The ZS4-4 rear panel is shown in the following figure. Base

configuration PCIe cards are not depicted in this illustration.

15 Solid state drive 4 (optional)

FIGURE 3

Figure Legend Figure Legend

1 Power supply unit (PSU) 0 indicator panel 7 Network (NET) 10 GbE ports: NET0NET3

2 PSU 0 AC inlet 8 USB 2.0 connectors (2)

3 PSU 1 indicator panel 9 PCIe card slots 7-11

4 PSU 1 AC inlet 10 Network management (NET MGT) 10/100/1000

5 System status indicator panel 11 Serial management (SER MGT) RJ-45 port

6 PCIe card slots 1-6 12 DB-15 video port

ZS4-4 Controller Rear Panel

BASE-T Ethernet port

Cluster Interface Card Compatibility - The ZS4-4 controller can contain a Version 2 or Version 3 cluster interface card. For clustered controllers, ensure you are using the same version in each controller because these two versions are not compatible.

76 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 77

ZS4-4 Controller Hardware Overview

For information about how to connect cables to form a cluster, see “Connecting Cluster Cables”

in Oracle ZFS Storage Appliance Cabling Guide.

Ethernet Ports

The ZS4-4 has four RJ-45 10-Gigabit Ethernet (10GbE) network connectors on the rear panel, labeled NET 0, NET 1, NET 2, and NET 3 (bottom left to top right), as shown in the figure below. Use these ports to connect the appliance to the network.

The LEDs located above the NET ports, labeled 2, 0, 3, 1 (left to right) are Link/Activity indicators.

LED Status

OFF (1) No Link

ON (0) Link and no activity

Blink Link and activity

Note - Speed is not indicated for the NET ports.

FIGURE 4

ZS4-4 Ethernet Ports

Network Management Port

The network management connector (NET MGT), shown in the figure below, is an RJ-45 10/100/1000 BASE-T Ethernet port and provides an alternate terminal interface to the service processor (SP) console.

Servicing the ZS4-4 Controller 77

Page 78

ZS4-4 Controller Hardware Overview

FIGURE 5

ZS4-4 Network Management Port

Serial Management Port

The serial management connector (SER MGT), shown in the figure below, is an RJ-45 port and provides a terminal connection to the SP console.

FIGURE 6

ZS4-4 Serial Management Port

4x4 SAS-2 HBA

The 4x4 SAS-2 HBA provides connectivity to external DE2-24 disk shelves. The HBA ports are numbered 3-0, top to bottom, as shown in the following figure.

78 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 79

ZS4-4 Controller Hardware Overview

FIGURE 7

4x4 SAS-2 HBA Port Numbers

ZS4-4 PCIe I/O Cards

The ZS4-4 base configuration contains the following PCIe cards:

■

One 8-port SAS-2 internal HBA (slot 2)

■

Two 4-port (4x4) SAS-2 external HBAs (slot 6 and slot 7)

■

One cluster interface card (slot 4)

Servicing the ZS4-4 Controller 79

Page 80

ZS4-4 Controller Hardware Overview

The following figure shows the PCIe I/O slot numbers.

FIGURE 8

ZS4-4 PCIe I/O Slot Numbers

PCIe Slot Order - Install optional PCIe cards in the following order:

■

Install additional 4x4 SAS-2 HBAs in slot 9, then slot 3.

■

Install InfiniBand CX3 HCAs into the first available client-option slot starting with slot 11, then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.

■

Install 16Gb FC HBAs into the first available client-option slot starting with slot 11, then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.

■

Install 10Gb Ethernet Optical NICs into the first available client-option slot starting with slot 11, then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.

■

Install 10Gb Ethernet Copper NICs into the first available client-option slot starting with slot 11, then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.

ZS4-4 PCIe Base and Optional Configurations - The following table describes the PCIe base and optional slot assignments for ZS4-4 standalone and cluster configurations. PCIe slot numbering begins with slot 1.

See the legend for a description of the interconnect types and option codes shown in the Type column.

Slot Description Max Type Note

1 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

1 2-port 10Gb Ethernet Optical

NIC

1 2-port 10Gb Ethernet Copper

NIC

80 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

4 C Optional recommended front-

4 D Optional recommended front-

end

Page 81

ZS4-4 Controller Hardware Overview

Slot Description Max Type Note

1 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

(Backup)

2 8-port SAS-2 Internal HBA 1 F Base configuration

3 4-port (4x4) SAS-2 External

4 E Optional back-end

HBA

3 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

end

3 2-port 10Gb Ethernet Optical

NIC

3 2-port 10Gb Ethernet Copper

NIC

4 C Optional recommended front-

end

4 D Optional recommended front-

end

3 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

(Backup)

4 Cluster Interface (second

1 G Base configuration

generation)

5 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

end

5 2-port 10Gb Ethernet Optical

NIC

5 2-port 10Gb Ethernet Copper

NIC

4 C Optional recommended front-

end

4 D Optional recommended front-

end

5 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

(Backup)

6 4-port (4x4) SAS-2 External

4 E Base configuration

HBA

7 4-port (4x4) SAS-2 External

4 E Base configuration

HBA

8 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

end

8 2-port 10Gb Ethernet Optical

NIC

8 2-port 10Gb Ethernet Copper

NIC

4 C Optional recommended front-

end

4 D Optional recommended front-

end

8 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

(Backup)

9 4-port (4x4) SAS-2 External

4 E Base configuration

HBA

9 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

end

9 2-port 10Gb Ethernet Optical

NIC

9 2-port 10Gb Ethernet Copper

NIC

4 C Optional recommended front-

end

4 D Optional recommended front-

end

Servicing the ZS4-4 Controller 81

Page 82

ZS4-4 Controller Hardware Overview

Slot Description Max Type Note

9 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

10 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

10 2-port 10Gb Ethernet Optical

10 2-port 10Gb Ethernet Copper

10 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

11 2-port InfiniBand CX3 HCA 4 A Optional recommended front-

11 2-port 10Gb Ethernet Optical

11 2-port 10Gb Ethernet Copper

11 16GB Dual Universal FC HBA 4 B Optional FC target or initiator

NIC

(Backup)

end

4 C Optional recommended front-

end

4 D Optional recommended front-

end

(Backup)

end

4 C Optional recommended front-

end

4 D Optional recommended front-

end

(Backup)

The legend describes the interconnect types and option codes shown in the Type column.

Legend for Interconnect Types and Options

A InfiniBand QDR QSFP+

- QSFP Direct Copper Cable Connect

- Optical Transceiver QSFP Short Range 40Gbs

B Fibre Channel 16Gb SFP+

- Optical Transceiver SFP Short Range 16Gbs

C GbE NIC Multi-mode Fiber SFP+

10GBase-SR/LR

- Optical Transceiver SFP Short Range 10Gbs

- Optical Transceiver SFP Long Range 10Gbs

D GbE NIC UTP 10GBase-T

E Storage Array 4-port External SAS-2

Mini-SAS HD

- SFF-8644 Mini-SAS HD to MiniSAS Copper

F HDD 8-port Internal SAS-2 Mini-

SAS

-SFF-8087 Mini-SAS to Mini-SAS Copper

G Server Heartbeat 2-port RS-232 1-

port 1GBase-T

-Copper RJ-45 Serial RS-232

-Copper RJ-45 Unshielded Twisted Pair

82 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 83

ZS4-4 Controller Hardware Overview

Legend for Interconnect Types and Options

- Copper RJ-45 Unshielded Twisted Pair

ZS4-4 Internal Components

The ZS4-4 chassis contains both customer-replaceable units (CRUs) and field-replaceable units (FRUs) as shown in the following figure. FRUs must be replaced by trained Oracle service technicians.

FIGURE 9

Figure Legend

1 Power supplies (CRU) 8 Memory riser card (CRU)

2 Power supply backplane (FRU) 9 Motherboard (FRU)

3 SP card (FRU) 10 System drive (CRU)

4 HBA/PCIe cards (CRU) 11 Fan module (CRU)

5 CPU (FRU) 12 Fan board (FRU)

6 Heatsink (FRU) 13 Drive backplane (FRU)

7 Cover 14 Chassis

Internal Components (Exploded View)

Servicing the ZS4-4 Controller 83

Page 84

ZS4-4 Controller Hardware Overview

ZS4-4 System Drive - The ZS4-4 controller has two system boot drives in slots 0 and 1, configured as a mirrored pair. Read-optimized cache devices can be installed in controller slots 2 through 5, or installed in DE2 disk shelf slots 20 through 23.

Note - Read cache devices cannot be installed in both controller and disk shelf slots at the

same time. For software requirements and slot configuration rules, see Table 49, “Disk Shelf

Configurations,” on page 286.

A filler panel must be installed in empty drive slots. The system drive LEDs are shown in the following figure.

FIGURE 10

System Drive LEDs

For information about supported disks and capacities, refer to the Oracle Systems Handbook.

TABLE 16

Figure Legend

1 Blue (Ready to remove) 2 Service action required (amber) 3 OK/Activity (green)

System Drive LEDs

ZS4-4 CPU and Memory - The ZS4-4 controller has four Intel Xeon E7-8895 v2 15-core

2.8 GHz CPUs and eight memory riser cards as shown in the following figure. The memory configuration is 16GB DDR3 DIMMs to accommodate up to 1.5TB (ninety-six 16GB). All ZS4-4 DIMM risers are fully populated to accommodate this offering.

84 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 85

ZS4-4 Controller Hardware Overview

FIGURE 11

ZS4-4 CPU and Memory

Figure Legend

1 Memory riser card P3/MR1 7 Memory riser card P0/MR1

2 Memory riser card P3/MR0 8 Memory riser card P0/MR0

3 Memory riser card P2/MR1 9 CPU P3

4 Memory riser card P2/MR0 10 CPU P2

5 Memory riser card P1/MR1 11 CPU P1

6 Memory riser card P1/MR0 12 CPU P0

Servicing the ZS4-4 Controller 85

Page 86

ZS4-4 Controller Hardware Overview

Each memory riser card contains twelve DIMM slots, four DDR3 channels, and two memory buffer ASICs. Each each memory buffer has two channels (A and B) and links to three DIMM slots per channel. Each memory buffer is connected to the processor's built-in memory controller by an SMI-2 link.

DIMM names in appliance logs and the Maintenance > Hardware view are displayed with the full name, such as /SYS/MB/P0/D7.

For more information about memory layout and procedures for replacing DIMMs, see

“Replacing a ZS4-4 DIMM” on page 62.

ZS4-4 Cooling Subsystem - The ZS4-4 internal components are cooled by air that is pulled in through the front of the controller and exhausted out the back of the controller. Cooling occurs in two areas of the chassis: the power supply area and the motherboard area.

The following figure shows the cooling zones and the approximate location of the temperature sensors. The accompanying legend table provides sensor NAC names and sensor motherboard designations.

86 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 87

ZS4-4 Controller Hardware Overview

FIGURE 12

ZS4-4 Cooling Subsysem

Figure Legend

0 Cooling zone 0 6 Temperature sensor TS_ZONE2

(U4505)

1 Cooling zone 1 7 Temperature sensor TS_OUT

(U4506)

2 Cooling zone 2 8 Temperature sensor TS_TVL_1

(U4002)

3 Cooling zone 3 (power supply

backplane area)

9 Temperature sensor TS_TVL_0

(U4302)

4 Temperature sensor TS_PS (U4603) 10 Temperature sensor TS_ZONE0_B

(U4509)

Servicing the ZS4-4 Controller 87

Page 88

ZS4-4 Controller Hardware Overview

Figure Legend

5 Temperature sensor TS_ZONE1

ZS4-4 Attached Storage

The ZS4-4 standalone and cluster controller configurations allow up to 36 disk shelves, consisting of up to six chains of one to six DE2-24 disk shelves. Any combination of disk-only, or disk, log, and read cache devices, can be combined within the chain in any order. The Sun Disk Shelf is not supported. For more information, see “Disk Shelf

Configurations” on page 286.

Note - Read cache devices must be contained in either the controller or disk shelf, not both.

Related Topics

■

“Connecting to Attached Storage” on page 272

■

“Disk Shelf Hardware Overview” on page 272

■

Oracle ZFS Storage Appliance Cabling Guide

(U4507)

11 Temperature sensor TS_ZONE0_A

(U4508)

88 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 89

Servicing the ZS3-4 Controller

Be sure to read sections “Getting Started with Servicing the Hardware” on page 15 and

“Prerequisites for Servicing Hardware” on page 49.

Most hardware faults clear after replacing the affected component. If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Oracle DE2-24s and Sun Disk Shelves can be used together with standalone and clustered controllers after certain hardware modifications. Contact Oracle Service to upgrade your controllers for disk shelf interoperability.

To service ZS3-4 hardware components, use the following tasks.

■

“Replacing a ZS3-4 HDD or SSD” on page 89

■

“Replacing a ZS3-4 Fan Module” on page 91

■

“Replacing a ZS3-4 Power Supply” on page 93

■

“Replacing a ZS3-4 DIMM” on page 95

■

“Replacing a ZS3-4 PCIe Card or Riser” on page 104

■

“Replacing a ZS3-4 Battery” on page 108

To understand the ZS3-4, see the following topics:

■

“ZS3-4 Base Configuration” on page 111

■

“ZS3-4 Specifications” on page 111

■

“ZS3-4 Front and Rear Panel Components” on page 112

■

“ZS3-4 Internal Components” on page 116

■

“ZS3-4 PCIe I/O Cards” on page 121

■

“ZS3-4 Attached Storage” on page 125

Replacing a ZS3-4 HDD or SSD

HDDs and SSDs are hot-swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities. Use the following procedure

Servicing the ZS3-4 Controller 89

Page 90

Replacing a ZS3-4 HDD or SSD

to replace a ZS3-4 HDD or SSD. For information on disk compatibility, see the Oracle Systems

Handbook.

Note - If you have more than one failed drive, replace only one drive at a time. Removing more

than one drive in quick succession causes a hardware/pool fault. Also, do not perform a pool configuration operation while a disk firmware upgrade is occurring. To check if an upgrade is in progress, navigate to Maintenance > System in the BUI, or maintenance system updates in the CLI.

Identify the failed HDD or SSD by going to the Maintenance > Hardware section

of the BUI and clicking the drive information icon . If you are physically at the system, the amber Service Required indicator on the HDD or SSD should be illuminated.

If you are not physically at the system, turn on the locator indicator by clicking

the locator icon .

Push the release button on the HDD or SSD to open the latch.

Grasp the latch (2), and pull the drive out of the drive slot.

After at least 30 seconds, navigate to the Hardware > Maintenance screen, and

click the right-arrow icon on the system controller to verify that the software has detected that the drive is not present.

90 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 91

Slide the replacement drive into the slot until it is fully seated.

Close the latch to lock the drive in place.

Replacing a ZS3-4 Fan Module

The Oracle ZFS Storage system software automatically detects and configures the new drive. The device appears in the BUI Maintenance > Hardware screen when you view details for the controller or drive shelf.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Replacing a ZS3-4 Fan Module

Caution - Operating a controller for an extended period of time with fans removed reduces the

effectiveness of the cooling system. For this reason, the replacement fan should be unpacked in advance and ready to insert into the controller chassis as soon as the faulted fan is removed.

Fan modules are hot-swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities. Use the following procedure to replace a ZS3-4 fan module.

To flash the locator LED on the chassis to service, go to Maintenance >

Hardware in the BUI and click the associated locator icon , or go to the service processor (SP) prompt and enter set /SYS/LOCATE status=on

Servicing the ZS3-4 Controller 91

Page 92

Replacing a ZS3-4 Fan Module

To locate the faulted component, go to Maintenance > Hardware > Fan and click the row with an amber indicator.

The component is highlighted in the diagram.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Lift the latch at the top of the fan module to unlock the fan module, and pull the fan module out.

Unlock and insert the fan module.

Apply firm pressure to fully seat the fan module.

On the replaced fan module, verify that the Fan OK indicator is lit, and that the fault indicator is not lit.

Verify that the Top Fan status indicator, the Service Required status indicators, and the Locator status indicator/Locator button are not lit.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller and then click Fan to verify that the online icon is green

for the newly installed fan module.

10.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

11.

Push the release tabs on the side of each rail and slowly slide the storage controller into the rack.

92 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 93

Replacing a ZS3-4 Power Supply

Storage controllers are equipped with redundant hot-swappable power supplies. If a power supply fails and you do not have a replacement, leave the failed power supply installed to ensure proper air flow. A faulted power supply is indicated by an amber colored status LED. Use the following procedure to replace a ZS3-4 power supply.

Gain access to the rear of the storage controller where the faulted power supply is located.

If a cable management arm (CMA) is installed, press and hold the CMA release tab and rotate the arm out of the way.

Replacing a ZS3-4 Power Supply

Disconnect the power cord from the faulted power supply.

Servicing the ZS3-4 Controller 93

Page 94

Replacing a ZS3-4 Power Supply

Grasp the power supply handle and press the release latch to remove the power supply.

Align the replacement power supply with the empty power supply chassis bay.

94 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 95

Replacing a ZS3-4 DIMM

Slide the power supply into the bay until it is fully seated. The following figure shows the power supply.

Connect the power cord to the power supply.

Verify that the green AC Present status indicator is lit.

Close the CMA, inserting the CMA into the rear left rail bracket.

10.

Go to the Maintenance > Hardware screen of the BUI. Click the right-arrow icon

for the controller and then click PSU to verify that the online icon is green

for the newly installed power supply.

11.

If a fault does not self-clear, see “Repairing Active Problems” on page 37.

Replacing a ZS3-4 DIMM

Caution - This procedure requires that you handle components that are sensitive to static

discharge, which can cause the component to fail. To avoid damage, wear an antistatic wrist strap and use an antistatic mat when handling components.

Servicing the ZS3-4 Controller 95

Page 96

Replacing a ZS3-4 DIMM

To identify a specific memory module that has faulted, you must open the storage controller and use the amber status LEDs on the motherboard. To identify a general memory fault, go to

the Maintenance > Hardware screen of the BUI, and click on the right-arrow icon on the controller. Then click DIMM to locate the faulted component, indicated by the warning icon

Shut down the controller using one of the power-off methods described in

“Powering Off the Controller” on page 50.

Disconnect the AC power cords from the rear panel of the storage controller.

Caution - Because 3.3 VDC standby power is always present in the system, you must unplug

the power cords before accessing any cold-serviceable components.

Extend the controller from the rack as described in “Extending the Storage

Controller from the Rack” on page 51.

Remove the top cover as described in “Removing the Top Cover” on page 52.

To locate the DIMM you want to service, press the Fault Remind Button on the storage controller.

96 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 97

Replacing a ZS3-4 DIMM

The following illustration shows the Fault remind button on the ZS3-4 controller.

Servicing the ZS3-4 Controller 97

Page 98

Replacing a ZS3-4 DIMM

Identify the memory riser that hosts the faulted DIMM by the Service Required status indicator. Lift the memory riser straight up to remove it from the motherboard, and place it on an antistatic mat.

Rotate both DIMM slot ejectors outward as far as they will go and carefully lift the faulted DIMM straight up to remove it from the socket.

98 Oracle ZFS Storage Appliance Customer Service Manual • July 2016

Page 99

Replacing a ZS3-4 DIMM

Line up the replacement DIMM with the connector, aligning the notch with the key to ensure that the component is oriented correctly.

Push the DIMM into the connector until the ejector tabs lock the component in place.

10.

To replace the cover:

Servicing the ZS3-4 Controller 99

Page 100

Replacing a ZS3-4 DIMM

Push the memory riser module into the associated CPU memory riser slot until the riser module locks in place.

Place the top cover on the chassis so that it is forward of the rear of the storage controller by about an inch (2.5 cm).

100 Oracle ZFS Storage Appliance Customer Service Manual • July 2016