IBM x3850 X6, x3950 X6 Planning And Implementation Manual

Front cover

IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

Covers the sixth generation Enterprise X-Architecture servers

Provides technical information about all server features

Explains what planning you need to do

ibm.com/redbooks

David Watts

Rani Doughty

Ilya Solovyev

International Technical Support Organization

IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

September 2014

SG24-8208-00

Note: Before using this information and the product it supports, read the information in “Notices” on page vii.

First Edition (September 2014)

This edition applies to IBM System x3850 X6 and x3950 X6, machine type 3837, with Intel Xeon Processor E7-4800 v2 and Intel Xeon Processor E7-8800 v2 processors.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Summary of changes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

26 September 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

25 June 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

17 June 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Authors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Target workloads. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.2 Business analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.3 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.4 Enterprise applications: ERP and CRM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Key features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 Positioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Storage versus in-memory data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.5 Flash storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.6 Energy efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.7 Services offerings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.8 What this book contains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Chapter 2. Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1 Modular design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.1 Compute Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1.2 Storage Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.3 Primary I/O Book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.4 Additional I/O Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.5 Power supplies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2 System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.1 x3850 X6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.2 x3950 X6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3 Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.3.1 Intel Xeon processor E7-4800/8800 v2 product family . . . . . . . . . . . . . . . . . . . . . 24

2.3.2 Compute Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.4 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4.1 Operational modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.4.2 Memory mirroring and rank sparing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.4.3 Chipkill . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.4.4 Redundant bit steering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.4.5 IBM Advanced Page Retire. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.5 PCIe 3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.6 Internal storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.6.1 Storage Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.6.2 IBM eXFlash memory-channel storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.6.3 IBM eXFlash SSD technology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.6.4 IBM High IOPS adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.6.5 IBM FlashCache Storage Accelerator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.7 UEFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.8 Integrated Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.9 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Chapter 3. Product information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.1 Product features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.1.1 Fast application performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.1.2 Agile system design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.1.3 Resilient platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.2 Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.3 Standard models of X6 servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.4 Physical design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.5 Ports and controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.5.1 The front operator panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.5.2 LCD system information panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.5.3 Rear ports and LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.6 Compute Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.6.1 Compute Book design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.6.2 Compute Book population order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

3.7 Processor options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.8 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.8.1 Memory options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.8.2 Memory population order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.9 IBM eXFlash memory-channel storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.10 Storage subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.10.1 Storage Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.10.2 Backplanes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3.10.3 RAID controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.10.4 Disk drive options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.10.5 IBM High IOPS adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.10.6 External disk storage expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.11 I/O subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.12 Primary I/O Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3.13 Half-length I/O Books and Full-length I/O Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.13.1 Half-length I/O Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

3.13.2 Full-length I/O Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

3.14 Hot-swap adapter support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3.15 Network adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.16 Storage host bus adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

3.17 GPU adapters and co-processors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

3.18 Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.19 Standard onboard features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.19.1 Integrated Management Module II (IMM2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.19.2 UEFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.19.3 Integrated Trusted Platform Module (TPM). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.19.4 Light path diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3.20 Integrated virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.21 Hot-swap capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

iv IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

3.22 Power subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.23 Fans and cooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3.24 Upgrading to an 8-socket X6 server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Chapter 4. Infrastructure planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.1 Physical and electrical specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4.2 Rack selection and rack options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.3 Floor clearance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.4 Use of the Rear Door Heat eXchanger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.5 Power advice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4.5.1 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4.5.2 Power supply redundancy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

4.5.3 Rules for achieving redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

4.5.4 Power supply installation order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

4.5.5 Power policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.5.6 Additional power settings in the IMM2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

4.5.7 Examples of power connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.6 Cooling advice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4.7 Uninterruptible Power Supply units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4.8 PDU and line cord selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4.8.1 Server to PDU power cord options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4.8.2 PDU and line cord options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Chapter 5. Preparing the hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

5.1 Configuring the IMM2 settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

5.1.1 IMM2 virtual presence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

5.1.2 IMM2 network access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

5.1.3 Configuring the IMM2 network interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

5.1.4 IMM2 dedicated versus shared ML2 Ethernet port . . . . . . . . . . . . . . . . . . . . . . . 135

5.1.5 x3950 X6 IMM2 communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

5.1.6 IMM2 communications troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

5.1.7 IMM2 functions to diagnose and manage the server . . . . . . . . . . . . . . . . . . . . . 137

5.2 UEFI settings for performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.2.1 Operating modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

5.3 UEFI common settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

5.3.1 System power settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

5.3.2 Processor settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

5.3.3 Memory settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

5.3.4 ServeRAID M5210 RAID controller configuration . . . . . . . . . . . . . . . . . . . . . . . . 151

5.4 PCIe adapter placement advice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

5.5 Hot-swap procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

5.5.1 Hot-swapping a power supply. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

5.5.2 Hot-swapping an I/O Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

5.6 Partitioning the x3950 X6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

5.6.1 Partitioning an x3950 X6 via the IMM2 web interface . . . . . . . . . . . . . . . . . . . . . 164

5.7 Updating firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

5.7.1 Firmware tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

5.7.2 Updating firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

5.8 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

5.8.1 Integrated Management Module (IMM). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

5.8.2 LCD system information panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

5.8.3 System event log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

5.8.4 POST event log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Contents v

5.8.5 IBM Electronic Service Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

5.8.6 Problem Determination and Service Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Chapter 6. Operating system installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

6.1 Installing without a local optical drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

6.1.1 IMM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

6.1.2 Local USB port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

6.1.3 Preboot eXecution Environment (PXE). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

6.2 IBM ServerGuide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

6.3 IBM ServerGuide Scripting Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

6.4 Use of embedded VMware ESXi. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

6.5 Booting from SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Chapter 7. Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

7.2 Integrated Management Module II (IMM2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

7.2.1 Configuring IMM2 for out-of-band-management . . . . . . . . . . . . . . . . . . . . . . . . . 201

7.2.2 Configuring IMM2 in-band configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

7.3 Remote control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

7.3.1 Accessing the remote control feature in the IMM2 . . . . . . . . . . . . . . . . . . . . . . . 205

7.4 IBM Systems Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

7.4.1 Discovering the IMM2 of an x3850 X6 out-of-band via IBM Systems Director . . 207

7.4.2 Service and Support Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

7.5 Upward Integration Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

7.5.1 Machine Check Architecture (MCA) error recovery . . . . . . . . . . . . . . . . . . . . . . 221

7.6 Advanced Settings Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

7.6.1 Using ASU to configure settings in IMM2-based servers . . . . . . . . . . . . . . . . . . 222

7.6.2 Command examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

7.7 MegaRAID Storage Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

7.7.1 MegaRAID Storage Manager installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

7.7.2 Drive states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

7.7.3 Virtual drive states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

7.7.4 MegaCLI utility for storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

7.8 IBM Electronic Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

7.9 Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

7.9.1 Enabling SoL in UEFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

7.9.2 Enabling SoL in the operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

7.9.3 How to start a SoL connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

vi IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION

PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

AIX® BladeCenter® Calibrated Vectored Cooling™ DB2® Electronic Service Agent™ FlashSystem™ Global Technology Services® IBM®

IBM FlashSystem™ IBM Flex System® IBM Systems Director Active Energy

Manager™ Intelligent Cluster™ PureFlex® Redbooks® Redbooks (logo) ®

ServerProven® System Storage® System x® System z® Tivoli® X-Architecture®

The following terms are trademarks of other companies:

Evolution, and Kenexa device are trademarks or registered trademarks of Kenexa, an IBM Company.

Intel, Intel Xeon, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Java, and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

Other company, product, or service names may be trademarks or service marks of others.

viii IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

Summary of changes

This section describes the technical changes made in this edition of the book and in previous editions. This edition might also include minor corrections and editorial changes that are not identified.

26 September 2014

Changed information:

򐂰 All processors support eXFlash DIMMs 򐂰 The RAID 1 feature of eXFlash DIMMs is currently not supported

25 June 2014

New information:

򐂰 NVIDIA GPUs support a maximum of 1 TB of system memory, page 95 򐂰 Information about the cable management kit shipped with the server, page 53

Changed information: 򐂰 Corrected the depth dimensions, page 53 and page 108

17 June 2014

This revision reflects the addition, deletion, or modification of new and changed information described below.

New information: 򐂰 Added Intel I350 Ethernet adapters

Changed information:

򐂰 Certain processors do not support eXFlash DIMMs 򐂰 The eXFlash DIMM driver does not support RAID 򐂰 VMware vSphere 5.1 supports a maximum of 160 concurrent threads

x IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

Preface

The increasing demand for cloud computing and business analytical workloads by enterprises to meet business needs drives innovation to find new ways to build informational systems. Clients are looking for cost-optimized fit-for-purpose IT solutions that manage large amounts of data, easily scale performance, and provide reliable real-time access to actionable information.

Built on decades of innovation, IBM® introduces its sixth generation of IBM Enterprise X-Architecture® technology, IBM X6 servers. IBM X6 servers are designed to be and

򐂰 Fast application performance means immediate access to actionable information. 򐂰 Agile system design helps to reduce acquisition costs and provide the ability to host

򐂰 Resilient platforms maximize application uptime and promote easy integration in virtual

IBM X6 servers continue to lead the way as the shift toward mission-critical scalable databases, business analytics, virtualization, enterprise applications, and cloud applications accelerates.

This IBM Redbooks® publication covers product information as well as planning and implementation information. In the first few chapters, we provide detailed technical information about the four-socket x3850 X6 and eight-socket x3950 X6. This information is most useful in designing, configuring, and planning to order a server solution. In the later chapters of the book, we provide detailed configuration and setup information to get your server operational.

fast, agile,

resilient:

multiple generations of technology in a single server.

environments.

This book is aimed at clients, IBM Business Partners, and IBM employees that want to understand the features and capabilities of the IBM X6 portfolio of servers and want to learn how to install and configure the servers for use in production.

Authors

This book was produced by a team of specialists from around the world working at the International Technical Support Organization, Raleigh Center.

David Watts is a Consulting IT Specialist at the IBM ITSO Center in Raleigh. He manages residencies and produces IBM Redbooks publications on hardware and software topics related to IBM Flex System®, IBM System x®, and IBM BladeCenter® servers. He has authored over 250 books, papers, and Product Guides. He holds a Bachelor of Engineering degree from the University of Queensland (Australia), and has worked for IBM in both the United States and Australia since 1989. David is an IBM Certified IT Specialist, and a member of the IT Specialist Certification Review Board.

Rani Doughty is a hardware specialist and data center consultant with a background in System x, BladeCenter, and Flex System. She currently works with the world-wide Data Center Services (DCS) team in IBM Lab Services as a developer of the IBM Power Configurator. She has 10 years of technical experience in the x86 field. She holds an honors degree in IT from the University of Ballarat (Australia). She has written and presented world-wide extensively on IBM Systems Director, pre and post sale tools, and infrastructure planning.

Ilya Solovyev is a Technical Consultant for IBM STG Lab Services based in Moscow. He currently provides technical consulting services for System x, Flex System, BladeCenter, IBM System Storage®, and Systems Software. His areas of expertise also include Linux systems, virtualization and cloud solutions, IBM Systems solution for SAP HANA. Ilya is a certified Red Hat Engineer and has a Bachelor degree in Math from the Volgograd State University.

Thanks to the following people for their contributions to this project:

IBM System x marketing:

򐂰 Jacqueline Gutierrez 򐂰 Kyle Hampton 򐂰 Kathy Holomon 򐂰 Randy Lundin 򐂰 Iliyas Pathan 򐂰 Steve Simmons 򐂰 Randi Wood

IBM System x development:

򐂰 David Brenchley 򐂰 David Fritz 򐂰 Josh Lowry 򐂰 Nina Newton

xii IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

򐂰 Loc Nguyen 򐂰 Bill Stevens 򐂰 Tim Schlude

IBM Redbooks:

򐂰 Deana Coble 򐂰 Rich Conway 򐂰 Tam ikia B arrow 򐂰 Ilya Krutov 򐂰 Debbie Willmschen

Others who helped us:

򐂰 Matthew Archibald, Data Center Services 򐂰 Simon Casey, IBM 򐂰 Chris Cook, Diablo Technologies 򐂰 Chris Noonan, SanDisk

Now you can become a published author, too!

Here’s an opportunity to spotlight your skills, grow your career, and become a published author—all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base.

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Comments welcome

Your comments are important to us!

We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks publications in one of the following ways:

򐂰 Use the online Contact us review Redbooks form found at:

ibm.com/redbooks

򐂰 Send your comments in an email to:

redbooks@us.ibm.com

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

Preface xiii

Stay connected to IBM Redbooks

򐂰 Find us on Facebook:

http://www.facebook.com/IBMRedbooks

򐂰 Follow us on Twitter:

http://twitter.com/ibmredbooks

򐂰 Look for us on LinkedIn:

http://www.linkedin.com/groups?home=&gid=2130806

򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks

weekly newsletter:

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

򐂰 Stay current on recent Redbooks publications with RSS Feeds:

http://www.redbooks.ibm.com/rss.html

xiv IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

Chapter 1. Introduction

The IBM X6 family of scalable rack servers consists of two servers:

򐂰 IBM System x3850 x6, a four-socket 4U rack-mount server 򐂰 IBM System x3950 X6, an eight-socket 8U rack-mount server

These servers are the sixth generation of servers built upon the IBM Enterprise X-Architecture. Enterprise X-Architecture is the culmination of bringing generations of IBM technology and innovation derived from our experience in high-end enterprise servers.

The IBM X6 servers deliver innovation with enhanced scalability, reliability, availability, and serviceability features to enable optimal break-through performance ideal for mission-critical scalable databases, business analytics, virtualization, enterprise applications, and cloud applications.

The IBM X6 generation servers pack numerous fault-tolerant and high-availability features into a high-density, rack-optimized, chassis-like package where all serviceable components are front and rear accessible, significantly reducing the space needed to support massive network computing operations and simplify servicing.

These servers can be expanded on demand, offering flexible modular scalability in processing, I/O, and memory dimensions so that you can provision what you need now and expand the system to meet future requirements. X6 is fast, agile, and resilient.

This chapter contains the following topics:

򐂰 1.1, “Target workloads” on page 2 򐂰 1.2, “Key features” on page 5 򐂰 1.3, “Positioning” on page 6 򐂰 1.4, “Storage versus in-memory data” on page 7 򐂰 1.5, “Flash storage” on page 8 򐂰 1.6, “Energy efficiency” on page 9 򐂰 1.7, “Services offerings” on page 10 򐂰 1.8, “What this book contains” on page 10

1.1 Target workloads

The IBM X6 servers introduce new levels of fault tolerance with advanced reliability, availability, and serviceability (RAS) features implemented in hardware and software, simplified servicing and upgrades with a bookshelf concept and lid-less design, and dramatic improvements in response time with stretched memory speeds and innovative flash storage offerings, while leveraging proven technologies of the previous generations of Enterprise X-Architecture.

These servers provide those looking for the highest level of scalable performance, the maximum memory capacity, and the richest set of RAS features for maximum productivity. They are designed for mission-critical, scalable workloads, including large databases, and ERP/CRM systems to support online transaction processing, business analytics, virtualization, and enterprise applications.

This section describes how IBM X6 technology helps to address challenges clients are facing in these mission-critical enterprise environments.

1.1.1 Databases

Leadership performance, scalability, and large memory support means that X6 systems can be highly utilized, yielding the best return for database applications such as these:

򐂰 SAP Business Suite on X6 򐂰 Microsoft SQL Data Warehouse on X6 򐂰 SAP HANA on X6 򐂰 IBM DB2® BLU on X6

X6 is well suited for Online transaction processing (OLTP) workloads. OLTP workloads are characterized by small, interactive transactions that generally require subsecond response times. For most OLTP systems, the processor, memory, and I/O subsystem in a server are well balanced and are not considered performance bottlenecks.

The major source of performance issues in OLTP environments is typically related to the storage I/O. The speed of traditional hard disk drive (HDD)-based storage systems does not match the processing capabilities of the server. As a result, often a situation occurs where a powerful processor sits idle, waiting for storage I/O requests to complete, negatively impacting the user and business productivity. This is not the case with X6.

The OLTP workload optimization goal for IBM X6 systems is to address storage I/O bottlenecks. The possible choices are in-memory data and the use of flash storage:

򐂰 In-memory data:

The main memory is the fastest storage type that can hold a significant amount of data. Data in main memory can be addressed more than a hundred times faster than data on a spinning hard disk.

For more information about in-memory data, see 1.4, “Storage versus in-memory data” on page 7.

򐂰 Flash storage as a main data store:

When flash storage is used as a main OLTP data storage, entire database structures are placed onto solid-state storage logical volumes. Solid-state storage has significantly better IOPS performance characteristics compared to traditional spinning hard disk drives.

2 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

In X6, flash-based storage choices are as follows:

– IBM eXFlash DIMMs: Where flash based storage modules are installed in memory

DIMM sockets, thereby having the lowest possible latency and maximized performance.

– IBM eXFlash SSDs: Based on packs of 1.8-inch solid-state drives and standard

SAS/SATA storage connectivity. – External IBM FlashSystem™ storage systems: Based on Fibre Channel connectivity. – IBM FlashCache Storage Accelerator: Intelligent, application-level caching software

that transforms IBM High IOPS Adapters and qualified SSDs into a transparent and

dynamic flash cache for “hot” data.

For more information about flash storage, see 1.5, “Flash storage” on page 8.

1.1.2 Business analytics

Data warehouses are commonly used with online analytical processing (OLAP) workloads in decision support systems, such as financial analysis. Unlike OLTP, where transactions are typically relatively simple and deal with small amounts of data, OLAP queries are more complex and process larger volumes of data.

For OLAP workloads, transactional delays can significantly increase business and financial risks. Usually, decision making is stalled or delayed because of lack of accurate, real-time operational data for analytics, which can mean missed opportunities.

These transactional delays come primarily from batch data loads and performance issues due to handling heavy complex queries and massive amounts of data (frequently referred to as big data) that use I/O resources. For OLAP workloads, a fast response time is critical to ensure that strategic business decisions can be made quickly in dynamic market conditions.

In general, clients might experience the following challenges with OLAP environments:

򐂰 Slow query execution and response times, which delay business decision making. 򐂰 Dramatic growth in data, which requires deeper analysis.

IBM X6 systems can help to make businesses more agile and analytics-driven by providing up-to-the-minute analytics based on real-time data. As with OLTP workloads, in-memory databases or flash storage are used for workload optimization (see 1.4, “Storage versus in-memory data” on page 7 and 1.5, “Flash storage” on page 8).

Using IBM X6 technology, we help address challenges in OLAP environments in the following ways:

򐂰 Dramatically boosting the performance of OLAP workloads with distributed scale-out

architecture, providing almost linear and virtually unlimited performance and capacity scalability.

򐂰 Significantly improving response time for better and timely decision making.

1.1.3 Virtualization

Virtualization commonly increases effectiveness in resource usage, reduce capital expenses, and software licensing fees, and reduce operational and management costs.

The first wave of server consolidation focused on lightly loaded servers that easily tapped into a hypervisor’s ability to share processor and memory resources across applications.

Chapter 1. Introduction 3

Hypervisors struggle to manage and share the heavy I/O loads typical of performance-intensive workloads. As a result, performance-intensive databases used for core enterprise workloads, such as CRM, ERP and SCM, are left to run on physical, non-virtual servers.

The next wave of server virtualization with IBM X6 will expand virtualization footprint to the workhorse applications of enterprise IT, namely those performance-intensive databases.

IBM X6 makes virtualization of mission-critical, performance-intensive workloads possible in a number of ways:

򐂰 IBM FlashCache Storage Accelerator intelligent caching software makes it possible to

virtualize high-performance databases and applications and increases per server VM density. Customers deploying FlashCache Storage Accelerator can recognize up to a 10X improvement in IOPS performance, up to 5X faster rich-media web page loads, and reduces I/O load and increased performance in primary storage. For more information about FlashCache Accelerator see 1.5, “Flash storage” on page 8.

򐂰 IBM X6 support for an integrated hypervisor: All x3850 X6 and x3950 X6 models support

the addition of an internal USB key with VMware ESXi installed.

򐂰 Workload optimized models have been designed to take the guesswork out of deciding

what the best components are for a given workload.

򐂰 Processor support: The Intel Xeon Processor E7-4800/8800 v2 series support Intel

Virtualization Technology (Intel VT) Flex Priority and Intel VT Flex migration.

򐂰 Large VM support for enterprise applications:

– Virtualized SAP HANA on X6 – VMware vCloud Suite on X6 – Microsoft Hyper-V Private Cloud on X6

1.1.4 Enterprise applications: ERP and CRM

Enterprise applications, such as Enterprise Resource Planning (ERP) or Customer Relationship Management (CRM), represent a mixed workload where both transaction processing and certain level of real-time reporting exist. In a 2-tier implementation, both database server and application modules reside on a same server. The key performance metric is response time, as with OLTP and OLAP workloads.

IBM X6 offerings provide low latency, extreme performance, and efficient transaction management to accommodate mixed workload requirements. IBM X6 in-memory and flash storage offerings can help to deliver the following benefits for enterprise applications:

򐂰 Dramatically boosting the performance of existing applications and lowering cost per IOPS

ratio without a need to redesign the application architecture.

򐂰 Increasing user productivity with better response times, improving business efficiency. 򐂰 Increasing data availability by using advanced system-level high availability and reliability

technologies, reducing the number of solution components and shortening batch processing and backup times.

򐂰 Increasing storage performance and capacity while decreasing power, cooling and space

requirements.

4 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

1.2 Key features

The IBM X6 system is fast, agile, and resilient and makes meeting the business needs of your enterprise easier:

򐂰 Fast application performance means immediate access to actionable information.

IBM X6 delivers fast application performance, thanks to an innovative scalable design and new storage technology that is designed to optimize overall solution performance.

This business-critical, enterprise-class server leverages unique X6 technology to deliver this level of performance and value to clients:

– IBM eXFlash DIMM: An innovative flash-based memory-channel storage device that

leverages the memory bus to deliver faster database performance while reducing

storage infrastructure costs. – IBM FlashCache Storage Accelerator: An advanced intelligent caching software for

IBM System x X6 servers that enables Flash and hard disk drive storage to

transparently work together to maximize performance and minimize cost.

򐂰 Agile system design helps reduce acquisition costs by up to 40% compared to previous

generations of X-Architecture. IBM X6 delivers a unique and adaptive modular design that allows you to grow on demand

with the new bookshelf design. Scale from 2-way to 4-way to 8-way and grow your memory and I/O to meet your needs and at the same time, realize infrastructure cost reductions by up to 40% without compromises in capacity or performance.

With X6 technology, you can realize benefits such as these:

– Supporting multiple generations of Intel processor technology allows for easily

swapping out Compute Books as new ones become available. – Adding I/O capability, such as extra network adapters or storage devices, while the

server is still running, allows for upgrades without minimal application downtime. – The majority of components used in an four-socket x3850 X6 can be reused when

upgrading to an eight-socket x3950 X6.

򐂰 A resilient platform maximizes application uptime and promote easy integration in virtual

environments. This new server is designed not only to continue operating in case of a component failure

but also to help you reduce planned and even unplanned downtime. The reliability, availability, and serviceability features of the new IBM X6 servers include

these capabilities:

– Predict failures before they happen:

Predictive Failure Analysis (PFA) allows the server to monitor the status of critical

subsystems and to notify the system administrator when components appear to be

degrading. Thanks to this information, in most cases, replacement of failing parts can

be performed as part of planned maintenance activity. This reduces the need for

unscheduled outages and so your system continues to run. – Find failed components fast:

Light path diagnostics allows systems engineers and administrators to easily and

quickly diagnose hardware problems. The LCD display on the front of the server gives

you more information about the problem at hand than LEDs, so failures can now be

evaluated in seconds and costly downtime can be reduced or avoided altogether.

Chapter 1. Introduction 5

– Survive a processor failure:

The server is designed to recover from a failed processor and restart automatically.

Even if the primary processor (the one used for booting the operating system) fails, the

X6 system is designed so it can boot from another processor using redundant links to

key resources. – Survive memory failures:

The combination of IBM Chipkill and Redundant Bit Steering (RBS, also known as

Double Device Data Correction or DDDC) allows the server to tolerate two sequential

DRAM memory chip failures without affecting overall system performance. – Survive an adapter failure and replace it while the server is running:

The new servers have up to six adapter slots that support hot-swapping. This means

the I/O Books can be removed and any failed adapters can be replaced without any

server downtime. – Swap components easily with the server’s lidless design:

There is no need to pull this server in or out of the rack to service it because all

components can be accessed either from the front or from the rear. This design allows

for faster maintenance by simplifying service procedures. This concept is similar to

what we have with BladeCenter and Flex System.

These built-in technologies drive the outstanding system availability and uninterrupted application performance needed to host mission-critical applications.

1.3 Positioning

The IBM System x3850 X6 and x3950 X6 servers are the next generation of X-Architecture following on from the highly successful eX5 server. IBM X6 servers include a number of new features when compared to the previous generation of eX5 including support for more memory and I/O in a modular design.

When compared to the 4-socket x3750 M4 server, the X6 servers fill the demand for enterprise workloads that require 4-socket and 8-socket performance, high availability, and advanced RAS features.

Table 1-1 shows a high-level comparison between the 4 socket x3750 M4, the eX5-based x3850 and x3950 X5, and the X6-based x3850 and x3950 X6.

Table 1-1 Maximum configurations for the X6 systems

Maximum configurations x3750 M4 x3850/x3950 X5 x3850/x3950 X6

Form factor 4 socket 2U 4U 4U

Processors 1-node 4 4 4

Cores 1-node 32 40 60

8 socket Not available 8U 8U

2-node Not available 8 8

6 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

2-node Not available 80 120

Maximum configurations x3750 M4 x3850/x3950 X5 x3850/x3950 X6

Memory 1-node 48 DIMM slots

1-node with MAX5

2-node Not available 128 DIMM slots

Not available 96 DIMM slots

64 DIMM slots

96 DIMM slots

Not available

192 DIMM slots

2.5-inch drive bays

1.8-inch SSDs drive bays

Standard 1Gb Ethernet interfaces

Standard 10Gb Ethernet interface

USB ports 1-node 4 USB 2.0 8 USB 2.0 6 USB 2.0, 2 USB

Power supplies 1-node 2 x 1400W 2 x 1975W 4 x 900W or

a. Requires all processors to be installed in order to use all memory slots. b. Model dependent. c. Mixing of power supplies in pairs is supported.

1-node 16 8 8

2-node Not available 16 16

1-node 32 16 16

2-node Not available 32 32

1-node 2 2

2-node Not available 4 8 (Optional)

1-node 2 (Optional) 2 2 (Optional)

2-node Not available 4 4 (Optional)

2-node Not available 16 USB 2.0 12 USB 2.0, 4

2-node Not available 4 x 1975W 8 x 900W or

4 (Optional)

3.0

USB 3.0

1400W

1.4 Storage versus in-memory data

Main memory (RAM) is the fastest storage type that can hold a significant amount of data. Data in main memory can be accessed more than a hundred thousand times faster than data on a spinning hard disk, and even flash technology storage is about a thousand times slower than main memory.

Main memory is connected directly to the processors through a high-speed bus, whereas hard disks are connected through a chain of buses (QPI, PCIe, SAN) and controllers (I/O hub, RAID controller or SAN adapter, and storage controller).

Compared to keeping data on disk, keeping the data in main memory can dramatically improve database performance just by the advantage in access time. However, there is one potential drawback. In a database transaction that has been committed, the transaction cannot stay committed.

Chapter 1. Introduction 7

In database technology, atomicity, consistency, isolation, and durability (ACID) is a set of requirements that guarantees that database transactions are processed reliably:

򐂰 A transaction must be atomic. That is, if part of a transaction fails, the entire transaction

has to fail and leave the database state unchanged.

򐂰 The consistency of a database must be preserved by the transactions that it performs. 򐂰 Isolation ensures that no transaction interferes with another transaction. 򐂰 Durability means that after a transaction is committed, it will remain committed.

Although the first three requirements are not affected by the in-memory concept, durability is a requirement that cannot be met by storing data in main memory alone. This is because main memory is volatile storage. That is, it loses its content when no electrical power is present. To make data persistent, it must reside on non-volatile storage. Therefore, some sort of permanent storage is still needed such as hard drives, solid-state drives (SSDs), or flash devices to form a hybrid solution that uses both in-memory and disk technology together.

The advantage of a hybrid solution can mean flexibility by being able to balance the performance, cost, and persistence and form factor, in the following ways:

򐂰 Performance: Use in-memory technology to enhance performance of sorting, storing, and

retrieving specified data rather than going to disk

򐂰 Persistence and form factor: Memory cannot approach the density of a small hard drive 򐂰 Cost: Less costly hard-disks can be substituted for more memory

In the next section, we discuss IBM technologies for flash storage.

1.5 Flash storage

IBM flash storage offerings for X6 servers combine extreme IOPS performance and low response time for transactional database workloads. The flash technologies used in the X6 servers include IBM eXFlash DIMMs, IBM eXFlash SSDs, and IBM FlashCache Storage Accelerator.

򐂰 IBM eXFlash memory-channel storage:

IBM eXFlash DIMMs represent innovative technology that utilizes DDR3 memory channels to connect flash storage modules. IBM is the first company that will deploy memory-channel storage technology in their industry-standard servers.

These are key features of eXFlash DIMMs:

– eXFlash DIMMs will be accessed through standard DDR3 memory channels. – eXFlash DIMMs will be interoperable with standard RDIMMs in the same channel. – eXFlash DIMMs will be supported by the major operating systems through software

drivers. – The projected capacity and performance of the eXFlash DIMMs will allow up to 25 TB

of a DDR3-based flash storage in a single x3950 X6 with up to 10 million random read

IOPS and more than 4 million random write IOPS.

򐂰 IBM eXFlash SSDs:

IBM eXFlash solid-state drives have an innovative high-density design of the drive cages and the performance-optimized storage controllers with the reliable high-speed solid-state drive technology.

8 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

򐂰 IBM FlashCache Storage Accelerator:

IBM FlashCache Storage Accelerator is an all-in-one flash-caching product that leverages the speed, management, capacity, and breadth of the IBM High IOPS adapters and qualified SSDs and integrates them into a high speed server-side caching service that seamlessly accelerates the most important data with little or minimal IT overhead in both physical and virtual servers.

These new technologies will allow IBM X6 servers to deliver break-through performance for targeted workloads by offering significantly lower latency and higher performance compared to traditional solid-state drives.

For additional information about IBM flash storage solutions, see the following sections:

򐂰 2.6.3, “IBM eXFlash SSD technology” on page 40 򐂰 2.6.2, “IBM eXFlash memory-channel storage” on page 38 򐂰 2.6.5, “IBM FlashCache Storage Accelerator” on page 42

1.6 Energy efficiency

The x3850 X6 and x3950 X6 offer the following energy-efficiency features to save energy, reduce operational costs, increase energy availability, and contribute to the green environment:

򐂰 Energy-efficient electronic components help lower operational costs. 򐂰 Highly efficient 900 W AC and 1400 W AC power supplies have 80 PLUS Platinum

certification.

򐂰 Intel Xeon processor E7-4800/8800 v2 product families offer significantly better

performance over the previous generation while fitting into the same thermal design power (TDP) limits.

򐂰 Intel Intelligent Power Capability powers individual processor elements on and off as

needed, to reduce power draw.

򐂰 Low-voltage Intel Xeon processors draw less energy to satisfy the demands of power and

thermally constrained data centers and telecommunication environments.

򐂰 Low-voltage 1.35 V DDR3 memory RDIMMs consume 15% less energy compared to 1.5

V DDR3 RDIMMs.

򐂰 Solid state drives (SSDs) consume as much as 80% less power than traditional spinning

2.5-inch HDDs.

򐂰 The server uses hexagonal ventilation holes, which is a part of IBM Calibrated Vectored

Cooling™ technology. Hexagonal holes can be grouped more densely than round holes, providing more efficient airflow through the system.

򐂰 IBM Systems Director Active Energy Manager™ provides advanced data center power

notification and management to help achieve lower heat output and reduced cooling needs.

Chapter 1. Introduction 9

1.7 Services offerings

The X6 systems fit into the services offerings that are already available from IBM Global Technology Services® for System x and BladeCenter. More information about these services is available at the following website:

http://www.ibm.com/systems/services/gts/systemxbcis.html

In addition to the existing offerings for asset management, information infrastructure, service management, security, virtualization, and consolidation, and business and collaborative solutions; IBM Systems Lab Services and Training has six offerings specifically for X6:

򐂰 Virtualization Enablement 򐂰 Database Enablement 򐂰 Enterprise Application Enablement 򐂰 Migration Study 򐂰 Virtualization Health Check 򐂰 Rapid! Migration Tool

IBM Systems Lab Services and Training is a worldwide services team of deeply skilled, experienced consultants and instructors who assist clients in the acceleration of adopting new IBM products and offerings, maximizing the performance of their IBM Systems and solutions, and enabling operational excellence through the transfer of skills and knowledge to the clients' staffs. The services offerings are designed around having the flexibility to be customized to meet your needs and can provide preconfigured services, custom services, expert skills transfer, off-the-shelf training, and online/classroom courses for X6.

For more information, send an email to this address:

mailto:stgls@us.ibm.com

Also, more information is available at the following website:

http://www.ibm.com/systems/services/labservices

1.8 What this book contains

In this book, readers get a general understanding of X6 technology, which sets it apart from previous models, and the architecture that makes up this product line. This book is broken down into several chapters:

The first three chapters give an in-depth look at the X6 hardware and architecture:

򐂰 Chapter 1: A high-level overview of X6 and its target workloads/markets 򐂰 Chapter 2: New technology in X6 򐂰 Chapter 3: An in-depth look at key components, such as memory, CPU, storage, and I/O

The last four chapters describe preparing and implementing the X6 server. We describe power and cooling considerations, operating installations, systems management, and firmware update tools:

򐂰 Chapter 4: Rack, power, and cooling considerations for X6. 򐂰 Chapter 5: Preparing the X6 hardware for use. 򐂰 Chapter 6: Operating system installation guidelines. 򐂰 Chapter 7: How to manage your X6 hardware.

10 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

Chapter 2. Technology

In this chapter, we give an overview of the technologies that IBM includes in the IBM System x3850 X6 and x3950 X6 servers. We describe system architecture and new chassis design with modular structure and the latest Intel Xeon E7-4800 v2 and E7-8800 v2 processors.

Then we describe the current memory options and features of the storage subsystem, including innovative memory-channel storage technology implemented as IBM eXFlash DIMMs. We discuss other advanced technology in the servers and describe X6 scaling and partitioning capabilities.

This chapter contains the following topics:

򐂰 2.1, “Modular design” on page 12 򐂰 2.2, “System architecture” on page 19 򐂰 2.3, “Processors” on page 23 򐂰 2.4, “Memory” on page 30 򐂰 2.5, “PCIe 3.0” on page 37 򐂰 2.6, “Internal storage” on page 38 򐂰 2.7, “UEFI” on page 44 򐂰 2.8, “Integrated Management Module” on page 44 򐂰 2.9, “Scalability” on page 45

2.1 Modular design

IBM X6 rack family consists of the new flagship servers of the IBM x86 server family:

򐂰 IBM System x3850 X6: 4U rack-optimized server scalable to four sockets 򐂰 IBM System x3950 X6: 8U rack-optimized server scalable to eight sockets

Figure 2-1 shows the IBM System x3850 X6.

Figure 2-1 IBM System x3850 X6

The x3950 X6 looks like two x3850 X6 servers where one is placed on top of the other. However, unlike eX5 servers, x3950 X6 employs a single chassis with a single midplane design without any external connectors and cables.

Figure 2-2 shows the IBM System x3950 X6.

Figure 2-2 IBM System x3950 X6

12 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

The X6 systems offer a new “bookshelf” design concept that is based on a fixed chassis mounted in a standard rack cabinet. There is no need to pull the chassis in or out of the rack to access components because all components can be accessed either from the front or from the rear like pulling books from a bookshelf.

Figure 2-3 shows the x3850 X6 server with one of the four Compute Books partially removed.

Figure 2-3 IBM x3850 X6 server with a Compute Book partially removed

The modular component that can be installed in a chassis is called a book. There are several types of books available:

򐂰 Compute Books:

A Compute Book contains one processor, 24 DIMM slots, and 2 hot-swap fan modules. It is accessible from the front of the server.

The x3850 X6 supports up to four Compute Books. The x3950 X6 supports up to eight Compute Books.

򐂰 Storage Books:

The Storage Book contains standard 2.5-inch drives or IBM eXFlash 1.8-inch hot-swap SSD drives. It also provides front USB and video ports, and it has two PCIe slots reserved for internal storage adapters. The Storage Book is accessible from the front of the server.

The x3850 X6 has one Storage Book. The x3950 X6 has two Storage Books.

򐂰 I/O Books:

An I/O Book is a container that provides PCIe expansion capabilities. I/O Books are accessible from the rear of the server.

There are three types of I/O Books:

– Primary I/O Book. This book provides core I/O connectivity, including the ML2 unique

slot for an onboard network, three standard PCIe 3.0 slots, Integrated Management

Module II, hot-swap fan modules and USB, video, serial, and systems management

ports.

Chapter 2. Technology 13

– Full-length I/O Book. This hot-swap book provides three optional full-length PCIe slots,

Storage Book Compute Books

Primary I/O Book

Additional I/O Books

Power supplies

and two of them are capable of hosting a Graphics Processing Unit (GPU) or

co-processor adapters with a total power consumption of 300 W. – Half-length I/O Book. This hot-swap book provides three optional half-length PCIe

slots.

The x3850 X6 has one Primary I/O Book and supports one or two of the full or half-length I/O Books (one of each or two of either). The x3950 X6 has two Primary I/O Books and supports up to four of the full or half-length I/O Books (any combination).

2.1.1 Compute Books

Overall, the x3850 X6 server has up to four Compute Books, one Storage Book, one Primary I/O Book, and up to two optional I/O Books. In addition, the 4U chassis supports up to four power supplies and up to ten hot-swap dual-motor fans (eight fans on the front and two fans on the rear).

Figure 2-4shows the front of the x3850 X6 server where you can see the four Compute Books and the Storage Book.

Figure 2-4 x3850 X6 front view

Figure 2-5 shows the rear view of the x3850 X6 server where you can see the Primary I/O Book, additional I/O Books, and power supplies.

Figure 2-5 x3850 X6 rear view

14 IBM System x3850 X6 and x3950 X6 Planning and Implementation Guide

+ 232 hidden pages

IBM x3850 X6, x3950 X6 Planning And Implementation Manual

Specifications and Main Features

Frequently Asked Questions

User Manual