Western Digital F3x00, E3000 User Manual

User Guide

OpenFlex™ F3x00 and E3000 1ET2212 Version 1.1 December 2020

User Guide Table of Contents

Revision History.................................................................................................................................... iv

Notices....................................................................................................................................................v

Points of Contact.................................................................................................................................. vi

Chapter 1. Overview.........................................................................................................1

OpenFlex F3x00 and E3000................................................................................................................ 2

Servicing Features.........................................................................................................................3

Composable Infrastructures......................................................................................................... 3

NVMe-oF........................................................................................................................................ 4

Supported SKUs............................................................................................................................4

System Architecture Overview........................................................................................................... 5

System High Speed Data Ingest Architecture........................................................................... 5

System Thermal and Cooling...................................................................................................... 5

Electrical Specifications....................................................................................................................... 7

Environmental Specifications.............................................................................................................. 8

Mechanical Specifications................................................................................................................... 8

Performance Specifications................................................................................................................. 9

Physical Design.................................................................................................................................... 11

LEDs.............................................................................................................................................. 13

Cables...........................................................................................................................................16

Restrictions and Limitations............................................................................................................... 16

Site Requirements............................................................................................................................... 17

Power Requirements................................................................................................................... 17

Rack Requirements...................................................................................................................... 17

Thermal and Cooling Requirements..........................................................................................18

Installation and Servicing Requirements................................................................................... 18

Chapter 2. Components............................................................................................... 20

Chassis................................................................................................................................................. 21

User Guide Table of Contents

Chassis Specifications................................................................................................................. 21

PSU...................................................................................................................................................... 22

PSU Specifications...................................................................................................................... 22

Fan Module......................................................................................................................................... 23

Fan Module Specifications......................................................................................................... 23

BMC Module....................................................................................................................................... 24

BMC Module Specifications....................................................................................................... 24

Rails......................................................................................................................................................25

Rails Specifications..................................................................................................................... 25

OpenFlexF3x00................................................................................................................................... 26

OpenFlexF3x00 Specifications...................................................................................................26

OpenFlex E3000 Fabric Device Blank...............................................................................................27

E3000 Fabric Device Blank Specifications...............................................................................27

List of Field/Customer Replaceable Units....................................................................................... 28

Chapter 3. Management............................................................................................... 29

OCGUI................................................................................................................................................. 30

Compatible Browsers.................................................................................................................30

Login Page.................................................................................................................................. 30

Dashboard.................................................................................................................................... 31

Storage Device Page..................................................................................................................36

Chassis Device Page.................................................................................................................. 43

Basic Operational Functions...................................................................................................... 50

Maintenance................................................................................................................................ 96

Firmware Upgrade.................................................................................................................... 106

Part Replacement.............................................................................................................................. 113

Fan Module Replacement..........................................................................................................113

PSU Replacement.......................................................................................................................116

BMC Module Replacement.......................................................................................................120

User Guide Table of Contents

OpenFlexF3x00 Replacement...................................................................................................123

Rails Replacement..................................................................................................................... 127

Chassis Replacement................................................................................................................ 145

Chapter 4. Safety......................................................................................................... 166

Electrostatic Discharge.....................................................................................................................167

Optimizing Location..........................................................................................................................167

Power Connections...........................................................................................................................167

Power Cords......................................................................................................................................167

Rackmountable Systems.................................................................................................................. 168

Restricted Access Location............................................................................................................. 168

Safety and Service............................................................................................................................ 168

Safety Warnings and Cautions........................................................................................................ 169

Chapter 5. Regulatory................................................................................................. 170

Europe (CE Declaration of Conformity)........................................................................................... 171

FCC Class A Notice...........................................................................................................................171

ICES-003 Class A Notice—Avis NMB-003, Classe A....................................................................... 171

Japanese Compliance Statement, Class A ITE................................................................................171

KCC Notice (Republic of Korea Only), Class A ITE........................................................................ 172

Taiwan Warning Label Statement, Class A ITE...............................................................................172

iii

User Guide Revision History

Revision History

Date Revision Comment

July 2020 1.0 Initial Release

December 2020 1.1 Added UK Import Representation Contact

User Guide Notices

Notices

Western Digital Technologies, Inc. or its affiliates' (collectively “Western Digital”) general policy does not recommend the use of its products in life support applications wherein a failure or malfunction of the product may directly threaten life or injury. Per Western Digital Terms and Conditions of Sale, the user of Western Digital products in life support applications assumes all risk of such use and indemnifies Western Digital against all damages.

This document is for information use only and is subject to change without prior notice. Western Digital assumes no responsibility for any errors that may appear in this document, nor for incidental or consequential damages resulting from the furnishing, performance or use of this material.

Absent a written agreement signed by Western Digital or its authorized representative to the contrary, Western Digital explicitly disclaims any express and implied warranties and indemnities of any kind that may, or could, be associated with this document and related material, and any user of this document or related material agrees to such disclaimer as a precondition to receipt and usage hereof.

Each user of this document or any product referred to herein expressly waives all guaranties and warranties of any kind associated with this document any related materials or such product, whether expressed or implied, including without limitation, any implied warranty of merchantability or fitness for a particular purpose or non-infringement. Each user of this document or any product referred to herein also expressly agrees Western Digital shall not be liable for any incidental, punitive, indirect, special, or consequential damages, including without limitation physical injury or death, property damage, lost data, loss of profits or costs of procurement of substitute goods, technology, or services, arising out of or related to this document, any related materials or any product referred to herein, regardless of whether such damages are based on tort, warranty, contract, or any other legal theory, even if advised of the possibility of such damages.

This document and its contents, including diagrams, schematics, methodology, work product, and intellectual property rights described in, associated with, or implied by this document, are the sole and exclusive property of Western Digital. No intellectual property license, express or implied, is granted by Western Digital associated with the document recipient's receipt, access and/or use of this document or the products referred to herein; Western Digital retains all rights hereto.

Western Digital, the Western Digital logo, and OpenFlex are registered trademarks or trademarks of Western Digital Corporation or its affiliates in the US and/or other countries. All other marks are the property of their respective owners. Product specifications subject to change without notice. Pictures shown may vary from actual products. Not all products are available in all regions of the world.

Western Digital 5601 Great Oaks Parkway San Jose, CA 95119

User Guide Points of Contact

Points of Contact

For further assistance with a Western Digital product, contact Western Digital Datacenter Platforms technical support. Please be prepared to provide the following information: part number (P/N), serial number (S/N), product name and/or model number, and a brief description of the issue.

Email:

support@wdc.com

Website:

https://portal.wdc.com/Support/s/

UK Import Representation Contact

Western Digital UK Limited Hamilton House, Regent Park, Kingston Road Leatherhead, Surrey KT22 7PL, GB,

United Kingdom

Telephone: +44 1372 366000

Western Digital

Overview

In This Chapter:

- OpenFlex F3x00 and E3000..........................2

- System Architecture Overview.....................5

- Electrical Specifications.................................7

- Environmental Specifications........................8

- Mechanical Specifications............................. 8

- Performance Specifications...........................9

- Physical Design.............................................. 11

- Restrictions and Limitations........................ 16

- Site Requirements.........................................17

User Guide

1.1 OpenFlex F3x00 and E3000

The OpenFlex F3x00 and E3000 is a 3U rack mounted data storage enclosure built on the OpenFlex platform. OpenFlex is Western Digital’s architecture that supports Open Composable Infrastructure (OCI). The OpenFlex F3x00 and E3000 are fabric devices that leverage this OCI approach in the form of disagreggated data storage using NVMe-over-Fabrics (NVMe-oF™). NVMe-oF is a networked storage protocol that allows storage to be disaggregated from compute to make that storage widely available to multiple applications and servers. By enabling applications to share a common pool of storage capacity, data can be easily shared between applications, or needed capacity can be allocated to an application regardless of location. Exploiting NVMe™ device-level performance, NVMeoF promises to deliver the lowest end-to-end latency from application to shared storage. NVMe-oF enables composable infrastructures to deliver the data locality benefits of NVMe DAS (low latency, high performance) while providing the agility and flexibility of sharing storage and compute.

The maximum data storage capacity is 614TB1 when leveraging a full set of 10 F3x00 fabric devices. The enclosure runs on an input voltage of 200V - 240V and consumes ~1400W of power under typical conditions. It requires a maximum of 1600W at full load.

1. Overview

1.1 OpenFlex F3x00 and E3000

Composable Infrastructure

An emerging category of datacenter infrastructure that seeks to disaggregate compute, storage, and networking fabric resources into shared resource pools that can be available for on-demand allocation (i.e., “composable”). Composability occurs at the software level, disaggregation occurs at the hardware level using NVMe™over-Fabric—will vastly improve compute and storage utilization, performance, and agility in the data center.

• 614TB

• 12GBps NVMe-oF over QSFP28

• 68.5 kg (151.1 lbs.)

Cables

OpenFlex

OpenFlex is Western Digital’s architecture that supports Open Composable Infrastructure through storage disaggregation – both disk and flash natively attached to a scalable fabric. OpenFlex does not rule out multiple fabrics, but whenever possible, Ethernet will be used as a unifying connect for both flash and disk because of its broad applicability and availability.

• 200V - 240V Input Voltages

• 3U Form Factor

• Hot-swappable PSUs and Fans

Open Composable API

Western Digital's new Open Composable API is designed for data center composability. It builds upon existing industry standards utilizing the best features of those standards as well as practices from proprietary management protocols.

• Dual 1600W PSUs

• Operational Temperature: 5°C to 35°C

• Dynamic Provisioning Supported

1. Max storage capacity depends on device version and device configuration.

User Guide

1.1.1 Servicing Features

The OpenFlex F3x00 and E3000 are equipped with several features that make servicing simpler and safer.

• Every CRU component has been designed with toolless removal features.

• The BMC Module, Fan Module, PSU, and F3x00 devices are all hotswappable components.

This document provides full instructions on how these features operate in the Management (page 29) section.

1.1.2 Composable Infrastructures

An emerging category of datacenter infrastructure that seeks to disaggregate compute, storage, and networking fabric resources into shared resource pools that can be available for on-demand allocation (i.e., “composable”). Composable occurs at the software level, disaggregation occurs at the hardware level.

Western Digital’s vision for Open Composable Infrastructures is based on four key pillars:

• Open

1. Overview

1.1 OpenFlex F3x00 and E3000

◦ Open in both API and form factor ◦ Designed for robust interoperability of multi-vendor solutions

• Scalable

◦ Ability to compose solutions at the width of the network ◦ Enable self-organizing systems of composable elements that communicate horizontally

• Disaggregated

◦ Pools of resources available for any use case that is defined at run time ◦ Independent scaling of compute & storage elements to maximize efficiency & agility

Extensible

•

◦ Inclusive of both disk and flash ◦ Entire ecosystem of composable elements managed & orchestrated using a common API

framework

◦ Prepared for yet-to-come composable elements – e.g., memory, accelerators

1.1.2.1 Open Composable

The Western Digital Open Composable Infrastructure (OCI) uses a common API to manage and coordinate with all fabric-attached storage including pools of flash and disk. The infrastructure also supports the management of networking and compute resources. The API is used for all managed elements to accelerate the ability to use disaggregated resources where components are no longer subcomponents, but core elements connected to the network.

1.1.2.2 Open Composable API (OCAPI)

Western Digital’s Open Composable API is a RESTful interface for OpenFlex that enables a Unified Fabric Control Plane for Storage Fabric Devices. This allows for composing disaggregated storage resources— with compute, networking, and memory—into virtual systems in the future. These virtual systems will be dynamically provided to the right application at the right time, ensuring SLAs can be met automatically.

User Guide

• Volume management (create / modify / delete namespaces, format media)

• Monitor hardware sensors (temperatures, voltages, fan speeds, hardware state)

• Configure hardware (update firmware, reboot individual components or systems, assert LEDs)

• Monitor performance (statistics, bandwidth, IOPS, latency)

• Capture inventory data (serial number, part number, etc.)

• Capture log information

• Configure policies (user access lists, authentication, LUN masking, HTTPS/TLS encryption/security

• Self-discovery of other locally-available resources configurable using the Open Composable API for

1.1.2.3 OCGUI

The Open Composable Graphical User Interface (OCGUI) is the graphical representation of all of the data shared up to the fabric by the OCAPI. This GUI is presented to the user by browsing to the IP address of any device on the fabric. The GUI has a "command-center" design layout that presents all vital health, utilization, and performance statistics related to devices on the network at a glance.

1. Overview

1.1 OpenFlex F3x00 and E3000

with certificate/key settings)

OpenFlex

1.1.3 NVMe-oF

Non-Volatile Memory Express over Fabric (NVMe-oF) is one of the primary enabling technologies for the OpenFlex platform. NVMe-oF enables the high-speed, low-latency storage performance of NVMe over a fabric switching network. OpenFlex products drive network communications across the fabric using 100Gb/s Ethernet protocol. This allows for a complex network of computing devices to all share the same storage resources with very high performance.

1.1.4 Supported SKUs

The following table lists the versions of Western Digital product that are supported by this document.

Table 1: Devices

Device Volume Bandwidth3Drive Writes Encryption

OpenFlex F3000 Fabric Device 12.8TB 12GB/s RI-3DW/D ISE 1EX1906

OpenFlex F3000 Fabric Device 12.8TB 12GB/s RI-3DW/D SE 1EX1910

OpenFlex F3100 Fabric Device 12.8TB 12GB/s RI-2DW/D ISE 1EX2413

OpenFlex F3200 Fabric Device 12.8TB 12GB/s RI-2DW/D ISE 1EX2269

OpenFlex F3000 Fabric Device 15.2TB 12GB/s RI-1DW/D ISE 1EX1907

OpenFlex F3000 Fabric Device 15.2TB 12GB/s RI-1DW/D SE 1EX1911

Part

Number

OpenFlex F3100 Fabric Device 15.36TB 12GB/s RI-0.8DW/D ISE 1EX2416

OpenFlex F3200 Fabric Device 15.36TB 12GB/s RI-0.8DW/D ISE 1EX2272

OpenFlex F3000 Fabric Device 25.6TB 12GB/s RI-3DW/D ISE 1EX1908

OpenFlex F3000 Fabric Device 25.6TB 12GB/s RI-3DW/D SE 1EX1912

3. Bandwidth obtained by sequential read

User Guide

1. Overview

1.2 System Architecture Overview

Device Volume Bandwidth3Drive Writes Encryption

OpenFlex F3100 Fabric Device 25.6TB 12GB/s RI-2DW/D ISE 1EX2414

OpenFlex F3200 Fabric Device 25.6TB 12GB/s RI-2DW/D ISE 1EX2270

OpenFlex F3000 Fabric Device 30.72TB 12GB/s RI-1DW/D ISE 1EX1909

OpenFlex F3000 Fabric Device 30.72TB 12GB/s RI-1DW/D SE 1EX1913

OpenFlex F3100 Fabric Device 30.72TB 12GB/s RI-0.8DW/D ISE 1EX2417

OpenFlex F3200 Fabric Device 30.72TB 12GB/s RI-0.8DW/D ISE 1EX2273

OpenFlex F3100 Fabric Device 51.2TB 12GB/s RI-2DW/D ISE 1EX2415

OpenFlex F3200 Fabric Device 51.2TB 12GB/s RI-2DW/D ISE 1EX2271

OpenFlex F3000 Fabric Device 61.44TB 12GB/s RI-1DW/D SE 1EX1914

OpenFlex F3100 Fabric Device 61.44TB 12GB/s RI-0.8DW/D ISE 1EX2418

OpenFlex F3200 Fabric Device 61.44TB 12GB/s RI-0.8DW/D ISE 1EX2274

1.2 System Architecture Overview

1.2.1 System High Speed Data Ingest Architecture

Part

Number

The system main data ingest architecture uses two separate 50G Ethernet connections each on a dual QSFP28 connector on the rear I/O of the chassis. This completes the connection from the device that is inserted into a chassis slot, through the backplane into the QSFP connectors. The architecture supports the hot swap nature of the devices and does not require any sort of shut down or disconnection before servicing. Each 100G Ethernet connection is split in half at the QSFP28 connectors resulting in 50G per connector allowing for dual port functionality with the device.

Figure 2: System High Speed Data Ingest

1.2.2 System Thermal and Cooling

3. Bandwidth obtained by sequential read

User Guide

The following image displays the thermal zones as viewed from the top of the enclosure. When viewing the enclosure from the front, the right device zone is on the right-hand side and the left device zone is on the left-hand side. When viewing the enclosure from the rear, the order is reversed. Each of the two thermal zones contains major components that are thermally maintained within their specific zone.

Figure 3: System Thermal Zones

1. Overview

1.2 System Architecture Overview

The E3000 uses a base algorithm, called a thermal algorithm, to control the overall thermal environment of the system. The system is mechanically separated into two thermal zones to support efficient cooling of the system components in order to achieve the intended performance of the system. The thermal zones are split into device zones (the left and right side of the enclosure) and center zone when standing at the front of the system. The device thermal zones contain up to ten devices or device blanks each that are cooled by four fan modules that are located directly behind the devices. The four fan modules behind the devices maintain the cooling for devices contained within the device slot installed into A through J. The center thermal zone contains the BMC module that is cooled by the fans contained in the redundant PSUs the are located directly behind the BMC module. The different thermal zones are designed to maintain proper thermal cooling across the entire system. During servicing the system increases the speed of the fan modules and PSU fans to maintain a balanced thermal load.

1.2.2.1 System Thermal Algorithm

The System Thermal Algorithm is designed to use temperature sensors and defined thresholds to determine if the algorithm will select critical, increase, decrease, or no change as the device decision.

The System Thermal Algorithm uses the concept of priorities to ensure the proper function of the system. As a result, any critical fault results in the system ramping the fans to maximum RPMs to protect the hardware for the duration of the fault. The fault will remain in a critical state until the fault is fully resolved and by bringing the temperature back within the specified defined thresholds.

User Guide

The System Thermal Algorithm contains minimum and maximum thresholds related to the ambient temperature of the system. The system is designed to maintain a maximum ambient temperature of 35°C. Exceeding 35°C may result in damage to the hardware and potentially void the product warranty. These thresholds allow for the best possible operating conditions. If the system goes outside of the minimum or maximum threshold window, the BMC will adjust the fan speed to accommodate the issue and a fault will be reported. The following table lists the Thermal Algorithm Thresholds along with the related fault levels and threshold values.

The following table lists the different severity of thermal algorithm critical faults that are reported to the user.

Table 2: Thermal Algorithm Critical Faults

1. Overview

1.3 Electrical Specifications

Component Critical Faults

Device Any sensor >= critical

BMC

The following table lists the device decision values that the system chooses from during operation. It describes how the pulse width modulation (PWM) of the power that is being distributed to the cooling fans reacts to different fault types.The fault will be reported based on the severity of the thermal issue.

Not installed FAN

Not installed PSU

Not installed Device or Blank

Not installed BMC

Any sensor >= criticalPSU

Any critical status on SMBus

Table 3: Thermal Algorithm Device Decision for Fan Control

Fault Sensor Value Fan Response

Critical Sensor >= Critical 100% PWM

Increase Sensor >= Max PWM + increase step

Decrease Sensor <= Min PWM - decrease step

No Change Min < Sensor < Max Hold PWM

1.3 Electrical Specifications

Table 4: Electrical Specifications

Specification Value

Max Power Consumption 1600W

Typical Power Consumption ~1400W

Input Voltage 200V - 240V

PSU Connector Type C16

User Guide

Specification Value

Inrush Current Maximum (per PSU) AC line inrush current shall not exceed 40A peak,

PSU Efficiency 80 PLUS Platinum

1.4 Environmental Specifications

Table 5: Environmental Specifications

Specification Non-Operational Operational

Temperature -30°C to 60°C 5°C to 35°C

Temperature Gradient 30°C per hour max 20°C per hour max

Temperature De-rating 1°C per 300m above 3000m 1°C per 175m above 950m

1. Overview

1.4 Environmental Specifications

for up to one-quarter of the AC cycle after which, the input current should be no more than the specified maximum input current.

Relative Humidity 5-95% Non-Condensing 8-85% Non-Condensing

Relative Humidity Gradient 30% per hour maximum 30% per hour maximum

Altitude -300m to 12,000m / -984 ft. to

39,370 ft

Cooling N/A 4 Fan Modules (N+1 Supported),

1.5 Mechanical Specifications

Table 6: Mechanical Specifications

Specification Non-Operational Operational

Shock 20G, 7ms half sine; 3 positive

and 3 negative pulses in each axis Z and Y. X axis- 15G, 7ms half sine, 3 positive and 3 negative pulses

Vibration Linear Random: 0.17G, 0 -

peak swept sine; 5 - 500Hz; 1 complete sweep @ 1/2 octave per minute, ~13 minutes each axis in X, Y, and Z Linear Random: 0.50Grms; 5-500Hz; 10 minutes each axis in X, Y, and Z Linear Random: 0.54Grms; 1-200Hz; 60 minutes each axis in X, Y, and Z

-300m to 3048m / -984 ft. to 10,000 ft.

containing two fans per module

5G, 11ms half sine; 3 positive and 3 negative pulses in each axis X, Y, and Z. Minimum 6 seconds between shocks

Linear Random: 0.15Grms; 5 500Hz; 10 minutes each axis in X, Y, and Z Swept Sine:0.17 G, 0 - peak, 5-500 Hz 0.5 octaves/min, approx. 13 minutes each axis

User Guide

Weight 107.04 kg / 236 lbs.

Dimensions W: 448.8 mm x L: 1000 mm x H: 176.7 mm / W: 17.6 in. x L: 39.4 in. x

System Installation Length 1000 mm / 39.4 in. from the front rack chassis mounts to the rear of

Required Rack Depth 1000 mm / 39.4 in. of usable rack space, frame to frame

Required Rack Width 450mm (17.72in.) with 465mm (18.31in.) ± 1.5mm nominal hole

Rack Units (U) 4U

Vertical Rack Rail Spacing 718 mm – 850 mm / 28.26 in. – 33.46 in.

1. Overview

1.6 Performance Specifications

Specification Non-Operational Operational

Swept Sine: 0.50G, 0 - peak

swept sine; 5 - 500Hz; 1 complete sweep @ 1/2 octave per minute

H: 6.9 in.

the system

spacing. See EIA-310 Rack Standard

Table 7: Acoustic Specifications

Quantities Declared

A-weighted sound power level1, L

WAd

{1 B = 10

Idle

Mode

8.5 8.5 9.6 9.6

dB}

Average A-weighted emission sound pressure level2, L

(dB) {bystander position3}

pAm

66.0 66.0 74.9 76.2

Statistical adder for verification4, K (dB) 2.5 2.5 2.5 2.5

Notes on Acoustic Testing Methodology and Environment:

• Background noise: <7dBA

• Environmental test conditions: ~23° C, 57% RH, 101.3 kPa

• Tested configuration: 3U NVMeOF VBOF device with 10 devices installed.

Declared A-weighted sound power level for a single machine, calculated per section 4.4.2 of ISO

9296-1988 and measured per ISO 3744

Declared A-weighted sound pressure level for a single machine, calculated per section 4.4.4 of

ISO-9296-1988 and measured per ISO 3744

The front and rear of the UUT were selected for the bystander location due to access typically from the cool or hot isle in a data center. This does not meet the four bystander positions as specified in ECMA-74 2012, but the microphones were adjusted to the preferred location.

The statistical adder, K, accounts for random measurement error, and is equal to 2.5 dB, which is appropriate for a 5% risk of rejection for SR = 1.5 dB per section 4.4.2 of ISO 9296-1988.

At steady state condition, system/PSU fans reached 100% pwm speeds during fan fail mode testing.

Operating

Mode

Fan Fail

Mode

Max Fan

Mode

1.6 Performance Specifications

User Guide

Table 8: Performance Specifications

Number of Device Slots 10 Dual-port Fabric Device Bays

Data Transfer Rates 12GBps NVMe-oF

Max Raw Data Storage Capacity 614TB

Ethernet Ports 20 x 50Gbps QSFP28 ( 2 per Fabric Device)

Table 9: F3000 Performance Specifications by CRU P/N

1. Overview

1.6 Performance Specifications

Specification Value

1 x 10/100/1G Ethernet

CRU P/N

Capacity/

Endurance

Random Read

(4KB, QD=1024)

Random Write

(4KB, QD=1024)

Random Mixed

70R/30W

(4KB, QD=1024)

Sequential Read

(128KB, QD=320)

Sequential Write (128KB, QD=320)

Random Write Latency

(4KB, QD=1, 99.99%)

Queue depth for 61.4TB device optimized at 1536, not 1024 as stated for other capacities

Notes on Performance Testing:

• Latency measured through a single Mellanox SN2700 switch

• K IOPs = IOPs x 1000

• Devices pre-conditioned with 2 full sequential fills

1EX1906/

1EX1910

12.8TB

3DWPD

2021K

IOPs

918K IOPs

1570K

IOPs

11.9 GB/s 11.9 GB/s 11.9 GB/s 11.9 GB/s 11.9 GB/s

9.3 GB/s 9.3 GB/s 10.2 GB/s 10.2 GB/s 11.7 GB/s

107 µs 114 µs 75.3 µs 76.3 µs 48.9 µs

1EX1907/

1EX1911

15.4TB

1DWPD

2021K

IOPs

919K IOPs

1537K

IOPs

1EX1908/

1EX1912

25.6TB

3DWPD

2004K

IOPs

609K

IOPs

1194K

IOPs

1EX1909/

1EX1913

30.7TB

1DWPD

2002K

IOPs

608K

IOPs

1194K

IOPs

1EX1914

61.4TB

1DWPD

2220K

IOPs

2420K

IOPs

2253K

IOPs

Table 10: F3100 Performance Specifications by CRU P/N

CRU P/N 1EX2413 1EX2416 1EX2414 1EX2417 1EX2415 1EX2418

Capacity/

Endurance

Random Read

(4KB, QD=1024)

4. Max storage capacity depends on device version and device configuration.

12.8TB

2DWPD

2199K

IOPs

15.4TB

0.8DWPD

2111K

IOPs

25.6TB

2DWPD

2164K

IOPs

30.7TB

0.8DWPD

2160K

IOPs

51.2TB

2DWPD

2176K

IOPs

61.4TB

0.8DWPD

2191K

IOPs

User Guide

1. Overview

1.7 Physical Design

CRU P/N 1EX2413 1EX2416 1EX2414 1EX2417 1EX2415 1EX2418

Random Write

(4KB, QD=1024)

1493K

IOPs

1433K

IOPs

1431K

IOPs

1397K

IOPs

1464K

IOPs

1400K

IOPs

Random Mixed

70R/30W

(4KB, QD=1024)

Sequential Read

(128KB, QD=320)

Sequential Write (128KB, QD=320)

Random Write

Latency

(4KB, QD=1, 99.99%)

Notes on Performance Testing:

• K IOPs = IOPs x 1000

• Devices pre-conditioned with 2 full sequential fills

2199K

IOPs

11.8 GB/s 11.7 GB/s 11.7 GB/s 11.7 GB/s 11.7 11.7 GB/s

9.9 GB/s 9.9 GB/s 9.9 GB/s 9.4 GB/s 9.4 GB/s 9.9 GB/s

33.9 µs 33.7 µs 33.7 µs 33.9 µs 33.7 µs 33.5 µs

2137K

IOPs

2183K

IOPs

Table 11: F3200 Performance Specifications by CRU P/N

CRU P/N 1EX2513 1EX2516 1EX2514 1EX2517 1EX2515 1EX2518

Capacity/

Endurance

Random Read

(4KB, QD=1024)

12.8TB

2DWPD

2218K

IOPs

15.4TB

0.8DWPD

2209K

IOPs

25.6TB

2DWPD

2212K

IOPs

2188K

IOPs

30.7TB

0.8DWPD

2210K

IOPs

2227K

IOPs

51.2TB

2DWPD

2221K

IOPs

2251K

IOPs

61.4TB

0.8DWPD

2214K

IOPs

Random Write

(4KB, QD=1024)

Random Mixed

70R/30W

(4KB, QD=1024)

Sequential Read

(128KB, QD=320)

Sequential Write (128KB, QD=320)

Random Write

Latency

(4KB, QD=1, 99.99%)

Notes on Performance Testing:

• K IOPs = IOPs x 1000

• Devices pre-conditioned with 2 full sequential fills

1.7 Physical Design

2130K

IOPs

2292K

IOPs

11.5 GB/s 11.5 GB/s 11.5 GB/s 11.5 GB/s 11.5 GB/s 11.5 GB/s

38.5 µs 38.1 µs 38.1 µs 38.0 µs 46.7 µs 38.1 µs

1998K

IOPs

882K

IOPs

2124K

IOPs

2256K

IOPs

2129K

IOPs

2252K

IOPs

1930K

IOPs

2308K

IOPs

1907K

IOPs

2285K

IOPs

User Guide

The OpenFlex F3x00 and E3000 physical design emphasizes easy access to hot-swappable components, maximization of data storage capacity in the rack, and bold aesthetic design.

Figure 4: OpenFlex F3x00 and E3000 Layout

1. Overview

1.7 Physical Design

The enclosure measures 447.2 mm/17.6 in. wide by 828.04 mm/32.6 in. long. It's height is 130.9 mm/5.2 in. or 3U. The chassis installation length is 778 mm (30.6 in.) from front rack chassis mounts to rear of system.

Figure 5: OpenFlex F3x00 and E3000 Dimensions

User Guide

1.7.1 LEDs

Chassis LEDs

Figure 6: Chassis Rear IO LEDs

1. Overview

1.7 Physical Design

Table 12: Chassis Rear IO LED Flash Patterns

LED Name Color Behavior

Ethernet Link/Activity Green Solid On: Connected

Blink: Activity Off: No Connection

Ethernet Speed

Identification Blue

QSFP28 LED

Green Solid On : Operating at 100 Mbps

Off: Operating at 10 Mbps

Amber Solid On: Operating at 1 Gbps

Off: Operating at 10 Mbps

Blink @ 1 Hz: Enclosure or any component being

identified (fans, PSUs, etc.)

Off: Enclosure/components not being identified

Green

Amber

Solid On: Link operating at maximum speed

Blink @ 3 Hz: Link activity

Off: Default state

Solid On: Link operating at a lower speed, 50G

or less

Blink @ 3 Hz: Low speed link activity

Blink @ 1 Hz: On/Off - Identify

Power Green Solid On: Enclosure is powered on

User Guide

PSU LED

1. Overview

1.7 Physical Design

LED Name Color Behavior

Off: Enclosure is powered off

Fault Amber Solid On: Enclosure is not ready or BMC is not

present

Blink @ 2 Hz: Enclosure has a fault Off: Enclosure has no fault

Figure 7: PSU LED

Fan Module LED

Figure 8: Fan Module LED

Table 13: PSU LED Flash Patterns

LED Name Color Behavior

Multi-

function LED

Table 14: Fan Module LED Flash Patterns

LED Name Color Behavior

LED Amber Solid On: Fan reporting

Green Solid On: PSU on and

reporting no faults Off: PSU disconnected from power

Amber Solid On: PSU

disconnected from power or critical fault, causing shutdown failure Blink @ 1 Hz: PSU reporting fault Off: PSU reporting no fault

fault Blink @ 1 Hz: Fan being identified Off: Fan on and reporting no fault

User Guide

BMC Module LEDs

1. Overview

1.7 Physical Design

Figure 9: BMC Module LEDs

OpenFlex F3x00 LEDs

Figure 10: OpenFlex F3x00 LEDs

Table 15: BMC Module LED Flash Patterns

LED Name Color Behavior

Identification Blue Blink @ 1 Hz: BMC or

any component being

identified (fans, PSUs, etc.)

Fault Amber Blink @ 2 Hz:

Enclosure reporting fault Off: Default State

Power Green Solid: Powered On

Table 16: OpenFlex F3x00 LED Flash Patterns

LED Name Color Behavior

Device Status White Solid On: Powered on

and connected Off: Not ready or powered down

Power Green Solid On: Powered on

Off: Powered down

Fault Amber Solid On: Device

powered down Blink @ 1 Hz: Device reporting fault Off: Device has no fault

Identification Blue Solid On: Device

shutting down or powering on Blink @ 1 Hz: Device being idendified Off: Device not being identified

User Guide

1.7.2 Cables

Approved Power Cables

Table 17: Compatible Passive Cables

1. Overview

1.7 Physical Design

Type Part Number Length

IEC C14 to IEC C15 Heavy

Duty 15A Power Cable

Note: The following Ethernet cables are compatible with the OpenFlex F3x00 and E3000 but are not provided by Western Digital.

Manufacturer Model Length Speed

Mellanox

MCP1600-C001 Direct Attach Cable 1m 100Gb/s

MCP1600-C002 Direct Attach Cable 2m 100Gb/s

(Provided in Accessory Kit) 6 ft.

MCP1600-C002E26N Direct Attach Cable 2m 100Gb/s

MCP1600-C002E30N Direct Attach Cable 2m 100Gb/s

MCP1600-C003 Direct Attach Cable 3m 100Gb/s

MCP7H00-G001 Direct Attach Splitter Cable 1m 2 x 50Gb/s

MCP7H00-G003 Direct Attach Splitter Cable 3m 2 x 50Gb/s

Amphenol

NDARHG-0001 Direct Attach Cable 1m 100Gb/s

NDARHG-0002 Direct Attach Cable 2m 100Gb/s

NDARHG-0004 Direct Attach Cable 4m 100Gb/s

NDARHJ-0003 Direct Attach Cable 3m 100Gb/s

NDARHJ-0005 Direct Attach Cable 5m 100Gb/s

Table 18: Compatible Active Cables

Manufacturer Model Length Speed

Mellanox

MFA1A00-C005 Active Optical Cable 5m 100Gb/s

MFA7A20-C003 Active Optical Cable 3m 100Gb/s

MFA7A20-C010 Active Optical Cable 10m 100Gb/s

User Guide

1.8 Restrictions and Limitations

The OpenFlex F3x00 and E3000 have the following restrictions and limitations on functionality:

• The E3000 chassis must be full for proper airflow. If there is only one F3x00 storage device installed, the rest of the slots must be filled with device blanks.

• If the BMC Module is not installed in the E3000 chassis, F3x00 devices cannot be added or slotswapped. In the absence of BMC Module, the LEDs may not function as designed.

• Only hot-swap a single device at a time. Never remove more than one at a time. Both PSUs must be functional in order to detect new hot-plugged F3x00s.

• The F3x00 storage device does not support RoCE v1; it supports RoCE v2 only.

• Powering on an F3x00 storage device must be accomplished through the E3000 chassis (via its GUI)

that contains it, but powering off an F3x00 can be accomplished from the storage device itself (via its GUI).

• The F3x00 storage device data ports support IPv4 networking only.

• The E3000 chassis device management port supports both IPv4 and IPv6 networking.

• The E3000 chassis device does not support DNS configuration or administration.

• Only one browser session is allowed to an individual F3x00 storage device at a time.

•

Enforced maximum limits on configuration:

◦ 256 Volumes ◦ 256 Hosts ◦ 2048 Queue Pairs ◦ 128 Queue Pairs (1 Admin, 127 IO) per connection ◦ 64 max. queue depth per IO Queue Pair ◦ 8 VLANs per F3x00 storage device

1. Overview

1.8 Restrictions and Limitations

1.9 Site Requirements

1.9.1 Power Requirements

The E3000 is equipped with redundant power supply units. The PSUs are hot-swappable and are located at the rear of the chassis. The following is a power specification summary. Note that the system does not support low-line voltage.

Table 19: Power Specification Summary

Specification Value

Power Output 1600W

Input Voltage 200V - 240V

80 PLUS Standard Platinum

Connector Type C16

1.9.2 Rack Requirements

User Guide

The E3000 is designed to be installed into a rack that meets the EIA-310 standard with a minimum of 1000 mm (39.4 in.) of usable rack space, frame to frame. The vertical rack rails must be set between 718 mm (28.26 in.) – 850 mm (33.46 in.) to support the enclosure. It requires 3U of rack space, and it should be installed into the rack at the lowest possible U height to keep the load on the rack balanced.

Table 20: Required Rack Specifications

1. Overview

1.9 Site Requirements

Parameter Requirement

Rack Depth 1000 mm (39.4 in.) of usable rack space, frame to

frame

Rack Width 450mm (17.72in.) with 465mm (18.31in.) ± 1.5mm

nominal hole spacing. See EIA-310 Rack Standard

Rack Units (U) 3U

Vertical Rack Rail Spacing 718 mm (28.26 in.) – 850 mm (33.46 in.)

Static Load Rating Rack meets ISTA 3E or 3B test requirements and

regulations when mounted to the shipping pallet

Dynamic Load Rating Rack meets ISTA 3E or 3B test requirements and

regulations when mounted to the shipping pallet

1.9.3 Thermal and Cooling Requirements

The thermal output of the enclosure depends on the number of F3x00s that are populated in the E3000 chassis. Use the following table to determine how many BTUs of heat will exhaust from the rear of the unit.

Table 21: BTU Exhaust per Component

Condition Typical BTU Output Max BTU Output

Single E3000 with BMC Module 768 BTU/hr = 225 W

(25C/77F, fans at 50%)

Single F3x00 409 BTU/hr = 120 W

(25C/77F, Workload: Random Read/Write, queue depth 32, 80% internal processors dedicated to workload)

E3000 Fully Populated with F3x00s

5244 BTU/hr = 1537 W (25C/77F, fans at 50%)

1177 BTU/hr = 345 W (35C/95F, fans at 100%)

546 BTU/hr = 160 W (35C/95F, Workload: Highest throughput to drives – Seq Write, queue depth 32, all internal processors at 100%)

5884 BTU/hr = 1725 W (35C/95F, fans at 100%)

1.9.4 Installation and Servicing Requirements

User Guide

Space Requirements

Installation of the OpenFlex F3x00 and E3000 requires two people and enough space in front of the rack for a safe installation. The recommended forward clearance is 889 mm / 35 in. from the front of the rack and 609.6 mm / 24 in. on both sides of the enclosure. It is also recommended to make considerations for any carts or lift equipment that might be used to perform the installation.

Servicing of the OpenFlex F3x00 and E3000 requires one person and a minimum of 889 mm / 35 in. of space in front of the rack to allow enough clearance to remove an F3x00 from the enclosure.

The following diagram provides measurement details for installation and servicing:

Figure 11: Installation and Servicing Space

1. Overview

1.9 Site Requirements

5. The weight of the enclosure during installation will vary, depending on the number of F3x00 devices and blanks contained in the E3000. In some situations, carts or lift equipment may be required.

Western Digital

Components

In This Chapter:

- Chassis........................................................... 21

- PSU.................................................................22

- Fan Module....................................................23

- BMC Module..................................................24

- Rails................................................................25

- OpenFlex F3x00............................................26

- OpenFlex E3000 Fabric Device Blank.........27

- List of Field/Customer Replaceable

Units................................................................ 28

User Guide

2.1 Chassis

The OpenFlex E3000 chassis is the primary housing that contains and connects all of the system components that comprise the OpenFlex F3x00 and E3000 . The chassis contains ten device slots and one BMC Module slot on the front, and the rear contains slots for the redundant PSUs and four Fan Modules. The OpenFlex F3x00 and E3000 is installed and secured onto shelf style rail mounts. The rear IO houses the primary connections such as power and QSFP28 ports and the status LEDs for all of the components.

2.1.1 Chassis Specifications

2. Components

2.1 Chassis

Table 22: Chassis Specification Summary

Specification Value

Rack Units 3U

Number of

Device Slots

Number of BMC Slots 1

Part Number 1EX1919

Hot Swappable? No

Dimensions

Weight 17.32 kg / 38.2 lbs

W: 447.2 mm x

L: 828.04 mm

x H: 130.9 mm

W: 17.6 in. x L:

32.6 in. x H: 5.2 in.

User Guide

2.2 PSU

The OpenFlex E3000 chassis contains toolless redundant 1600W Power Supply Units (PSU). Each PSU requires an input voltage of between 200V - 240V. The PSUs are certified 80 PLUS Platinum and use the C16 connector type. Due to the redundant nature of the PSUs, they may be serviced or replaced, one at a time, while the enclosure is powered on.

2.2.1 PSU Specifications

2. Components

Table 23: PSU Specification Summary

Specification Value

Power Output 1600W

Input Voltage 200V - 240V

80 PLUS Standard Platinum

Connector Type C16

2.2 PSU

Internal Fan Speed Up to 25,000 RPM

Number per Enclosure 2

Part Number 1EX1916

Hot Swappable? Yes

Service Window 5 minutes

Dimensions

Weight 1.05 kg / 2.31 lbs.

W: 54.5 mm x L: 321.5

mm x H: 40.1 mm

W: 2.15 in. x L: 12.66

in. x H: 1.58 in.

User Guide

2.3 Fan Module

The OpenFlex E3000 contains four toolless Fan Modules to maintain the cooling across the entire system. The Fan Modules are connected to the rear of the Chassis using a handle to seat the module in place and a captive screw to secure the Fan Module into the fan bay. Due to the redundant nature of the Fan Modules, they may be serviced or replaced, one at a time, while the enclosure is powered on. The Fan Modules will accelerate to the maximum RPM while the BMC Module or devices are removed during servicing or replacement.

2.3.1 Fan Module Specifications

2. Components

2.3 Fan Module

Table 24: Fan Module Specification Summary

Specification Value

Rated Voltage 12V

Fan Speed 12000 RPM

Acoustics 76 dB measured at 1

meter from the fan inlet

Number per Enclosure 4 Fan Modules

containing two

rotors per module

Part Number 1EX1915

Hot Swappable? Yes

Service Window 5 minutes

Dimensions

Weight 0.8 kg / 1.76 lbs.

W: 92.74 mm x L: 127.2

mm x H: 85.2 mm

W: 3.65 in x L:

5.01 in x H: 3.35 in

+ 149 hidden pages