IBM Power 780, Power 770 Technical Overview And Introduction

Download

Front cover

IBM Power 770 and 780

Technical Overview and Introduction

Features the 9117-MMD and 9179-MHD based on the latest POWER7+ processor technology

Describes support of up to 20 LPARS per processor core

Discusses new I/O cards and drawers

ibm.com/redbooks

An Ding Chen

Dave Freeman

Breno Henrique Leitão

Redpaper

International Technical Support Organization

IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

February 2013

REDP-4924-00

Note: Before using this information and the product it supports, read the information in “Notices” on page vii.

First Edition (February 2013)

This edition applies to the IBM Power 770 (9117-MMD) and Power 780 (9179-MHD) Power Systems servers..

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

The team who wrote this paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x

Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Chapter 1. General description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Systems overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 IBM Power 770 server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.2 IBM Power 780 server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Operating environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Physical package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.4 System features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4.1 Power 770 system features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4.2 Power 780 system features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.4.3 Minimum features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4.4 Power supply features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.4.5 Processor card features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.4.6 Summary of processor features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.4.7 Memory features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.5 Disk and media features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.6 I/O drawers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.6.1 PCI-DDR 12X Expansion Drawers (FC 5796) . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.6.2 12X I/O Drawer PCIe (FC 5802 and FC 5877) . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.6.3 EXP12S SAS Drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.6.4 EXP 24S SFF Gen2-bay Drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.6.5 EXP30 Ultra SSD I/O Drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.6.6 I/O drawers and usable PCI slot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.7 Comparison between models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

1.8 Build to order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

1.9 IBM editions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

1.10 Model upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

1.10.1 Power 770 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

1.10.2 Power 780 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

1.11 Management consoles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.11.1 HMC models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.11.2 IBM SDMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.12 System racks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.12.1 IBM 7014 model T00 rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.12.2 IBM 7014 model T42 rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.12.3 IBM 7014 model S25 rack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.12.4 IBM 7953 model 94Y rack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.12.5 Feature code 0555 rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

1.12.6 Feature code 0551 rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

1.12.7 Feature code 0553 rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

1.12.8 The AC power distribution unit and rack content . . . . . . . . . . . . . . . . . . . . . . . . 32

1.12.9 Rack-mounting rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

1.12.10 Useful rack additions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Chapter 2. Architecture and technical overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.1 The IBM POWER7+ processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.1.1 POWER7+ processor overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.1.2 POWER7+ processor core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.1.3 Simultaneous multithreading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.1.4 Memory access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.1.5 On-chip L3 cache innovation and Intelligent Cache . . . . . . . . . . . . . . . . . . . . . . . 45

2.1.6 POWER7+ processor and Intelligent Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.1.7 Comparison of the POWER7+ and POWER6 processors . . . . . . . . . . . . . . . . . . 47

2.2 POWER7+ processor card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.2.2 Processor interconnects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3 Memory subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3.1 Fully buffered DIMM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3.2 Memory placement rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3.3 Memory activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

2.3.4 Memory throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.3.5 Active Memory Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.3.6 Special Uncorrectable Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.4 Capacity on Demand. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.4.1 Capacity Upgrade on Demand (CUoD). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.4.2 On/Off Capacity on Demand (On/Off CoD). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.4.3 Utility Capacity on Demand (Utility CoD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.4.4 Trial Capacity on Demand (Trial CoD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.4.5 Software licensing and CoD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.5 CEC drawer interconnection cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.6 System bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

2.6.1 I/O buses and GX++ card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

2.6.2 Service processor bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

2.7 Internal I/O subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2.7.1 Blind-swap cassettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2.7.2 System ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2.8 PCI adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

2.8.1 PCI Express (PCIe). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

2.8.2 PCI-X adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

2.8.3 IBM i IOP adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.8.4 PCIe adapter form factors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.8.5 LAN adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

2.8.6 Graphics accelerator adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.8.7 SCSI and SAS adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.8.8 iSCSI adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.8.9 Fibre Channel adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

2.8.10 Fibre Channel over Ethernet (FCoE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.8.11 InfiniBand host channel adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.8.12 Asynchronous and USB adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

2.8.13 Cryptographic Coprocessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

2.9 Internal storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

2.9.1 Dual split backplane mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

2.9.2 Triple split backplane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

2.9.3 Dual storage I/O Adapter (IOA) configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

iv IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.9.4 DVD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

2.10 External I/O subsystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

2.10.1 PCI-DDR 12X Expansion Drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

2.10.2 12X I/O Drawer PCIe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

2.10.3 Dividing SFF drive bays in 12X I/O drawer PCIe . . . . . . . . . . . . . . . . . . . . . . . . 85

2.10.4 12X I/O Drawer PCIe and PCI-DDR 12X Expansion Drawer 12X cabling . . . . . 88

2.10.5 12X I/O Drawer PCIe and PCI-DDR 12X Expansion Drawer SPCN cabling . . . 90

2.11 External disk subsystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

2.11.1 EXP30 Ultra SSD I/O Drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

2.11.2 EXP12S SAS Expansion Drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

2.11.3 EXP24S SFF Gen2-bay drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

2.11.4 TotalStorage EXP24 disk drawer and tower . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

2.11.5 IBM TotalStorage EXP24 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

2.11.6 IBM System Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

2.12 Hardware Management Console (HMC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

2.12.1 HMC mode and RAID 1 support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

2.12.2 HMC functional overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

2.12.3 HMC code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

2.12.4 HMC connectivity to the POWER7+ processor-based systems . . . . . . . . . . . . 104

2.12.5 High availability by using the HMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

2.12.6 HMC code level. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

2.13 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

2.13.1 Virtual I/O Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

2.13.2 IBM AIX operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

2.13.3 IBM i operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

2.13.4 Linux operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

2.13.5 Java versions that are supported . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

2.13.6 Boosting performance and productivity with IBM compilers . . . . . . . . . . . . . . . 112

2.14 Energy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

2.14.1 IBM EnergyScale technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

2.14.2 Thermal power management device (TPMD) card . . . . . . . . . . . . . . . . . . . . . . 117

Chapter 3. Virtualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

3.1 POWER Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

3.2 POWER processor modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

3.3 Active Memory Expansion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

3.4 PowerVM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.4.1 PowerVM editions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

3.4.2 Logical partitions (LPARs). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

3.4.3 Multiple shared processor pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

3.4.4 Virtual I/O Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

3.4.5 PowerVM Live Partition Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

3.4.6 Active Memory Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

3.4.7 Active Memory Deduplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

3.4.8 Dynamic Platform Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

3.4.9 Operating system support for PowerVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

3.4.10 Linux support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

3.5 System Planning Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

3.6 POWER Version 2.2 enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Chapter 4. Continuous availability and manageability . . . . . . . . . . . . . . . . . . . . . . . . 155

4.1 Reliability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

4.1.1 Designed for reliability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Contents v

4.1.2 Placement of components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

4.1.3 Redundant components and concurrent repair. . . . . . . . . . . . . . . . . . . . . . . . . . 158

4.2 Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

4.2.1 Partition availability priority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

4.2.2 General detection and deallocation of failing components . . . . . . . . . . . . . . . . . 160

4.2.3 Memory protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

4.2.4 Active Memory Mirroring for Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

4.2.5 Cache protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

4.2.6 Special uncorrectable error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

4.2.7 PCI-enhanced error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

4.2.8 POWER7 I/O chip freeze behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

4.3 Serviceability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

4.3.1 Detecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

4.3.2 Diagnosing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

4.3.3 Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

4.3.4 Notifying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

4.3.5 Locating and servicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

4.4 Manageability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

4.4.1 Service user interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

4.4.2 IBM Power Systems firmware maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

4.4.3 Electronic Services and Electronic Service Agent . . . . . . . . . . . . . . . . . . . . . . . 192

4.5 POWER7+ RAS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

4.6 PORE in POWER7+: Assisting Energy Management and providing RAS capabilities 194

4.7 Operating system support for RAS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

vi IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION

PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

Active Memory™ AIX® AS/400® BladeCenter® DS8000® Electronic Service Agent™ EnergyScale™ Focal Point™ IBM Flex System™ IBM Systems Director Active Energy

Manager™ IBM® Micro-Partitioning® POWER Hypervisor™

Power Systems™ Power Systems Software™ POWER6+™ POWER6® POWER7+™ POWER7® PowerHA® PowerPC® PowerVM® POWER® pSeries® PureFlex™ Rational Team Concert™ Rational®

Real-time Compression™ Redbooks® Redpaper™ Redbooks (logo) ® RS/6000® Storwize® System Storage® System x® System z® Tivoli® Workload Partitions Manager™ XIV®

The following terms are trademarks of other companies:

Intel Xeon, Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

LTO, Ultrium, the LTO Logo and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.

Microsoft, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Java, and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Other company, product, or service names may be trademarks or service marks of others.

viii IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Preface

This IBM® Redpaper™ publication is a comprehensive guide covering the IBM Power 770 (9117-MMD) and Power 780 (9179-MHD) servers that support IBM AIX®, IBM i, and Linux operating systems. The goal of this paper is to introduce the major innovative Power 770 and 780 offerings and their prominent functions:

򐂰 The IBM POWER7+™ processor, available at frequencies of 3.8 GHz and 4.2 GHz for the

򐂰 The specialized IBM POWER7+ Level 3 cache that provides greater bandwidth, capacity,

򐂰 The 1 Gb or 10 Gb Integrated Multifunction Card that provides two USB ports, one

򐂰 The IBM Active Memory™ Mirroring (AMM) for Hypervisor feature, which mirrors the main

򐂰 IBM PowerVM® virtualization, including PowerVM Live Partition Mobility and PowerVM

򐂰 Active Memory Expansion that provides more usable memory than what is physically

򐂰 IBM EnergyScale™ technology that provides features such as power trending,

򐂰 Enterprise-ready reliability, serviceability, and availability

Power 770 and 3.7 GHz and 4.4 GHz for the Power 780

and reliability

serial port, and four Ethernet connectors for a processor enclosure and does not require a PCI slot

memory that is used by the firmware

Active Memory Sharing

installed on the system

power-saving, capping of power, and thermal measurement

򐂰 Dynamic Platform Optimizer 򐂰 High-performance SSD drawer

This publication is for professionals who want to acquire a better understanding of IBM Power Systems™ products. The intended audience includes the following areas:

򐂰 Clients 򐂰 Sales and marketing professionals 򐂰 Technical support professionals 򐂰 IBM Business Partners 򐂰 Independent software vendors

This paper expands the current set of IBM Power Systems documentation by providing a desktop reference that offers a detailed technical description of the Power 770 and Power 780 systems.

This paper does not replace the latest marketing materials and configuration tools. It is intended as an additional source of information that, together with existing sources, can be used to enhance your knowledge of IBM server solutions.

The team who wrote this paper

This paper was produced by a team of specialists from around the world working at the International Technical Support Organization, Poughkeepsie Center.

An Ding Chen is a Power Systems Product Engineer in Shanghai, China, who provides level 3 hardware and firmware support in all Asia Pacific countries and Japan. He has twelve years of experience on AIX, UNIX, IBM RS/6000®, IBM pSeries®, and Power Systems products. After university, he passed the CATE certification on pSeries systems and IBM AIX 5L. He joined IBM in 2006.

Dave Freeman has worked for IBM since 1985. He is currently a Systems Service Representative (SSR) with IBM UK. He has worked extensively with Power Systems and Storage systems for the last 10 years. Prior to this role, Dave was an IT Systems Engineer, providing presales technical support to IBM sales and IBM Business Partners in the small and medium business sector, primarily on IBM i (IBM AS/400®). He has a degree in Information Technology from the Polytechnic of Central London.

Breno Henrique Leitão is an Advisory Software Engineer at the Linux Technology Center in Brazil. He has 14 years of experience with Linux. Breno is also a Master Inventor in Brazil and holds a degree in Computer Science from Universidade de Sao Paulo. His areas of expertise include operating systems performance, virtualization, and networking. He has written extensively about Linux, mainly about networking and debugging.

The project that produced this publication was managed by:

Scott Vetter

Executive Project Manager, PMP

Thanks to the following people for their contributions to this project:

Ron Arroyo, Tamikia Barrow, Louis Bellanger, James Hermes, Volker Haug, Daniel Hurliman, Benjamim Mashak, Camille Mamm, Duc Nguyen, Rakesh Sharma, Phil G Williams, Jacobo Vargas

IBM U.S.A

Now you can become a published author, too!

Here’s an opportunity to spotlight your skills, grow your career, and become a published author—all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base.

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

x IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Comments welcome

Your comments are important to us!

We want our papers to be as helpful as possible. Send us your comments about this paper or other IBM Redbooks® publications in one of the following ways:

򐂰 Use the online Contact us review Redbooks form found at:

ibm.com/redbooks

򐂰 Send your comments in an email to:

redbooks@us.ibm.com

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

Stay connected to IBM Redbooks

򐂰 Find us on Facebook:

http://www.facebook.com/IBMRedbooks

򐂰 Follow us on Twitter:

http://twitter.com/ibmredbooks

򐂰 Look for us on LinkedIn:

http://www.linkedin.com/groups?home=&gid=2130806

򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks

weekly newsletter:

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

򐂰 Stay current on recent Redbooks publications with RSS Feeds:

http://www.redbooks.ibm.com/rss.html

Preface xi

xii IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Chapter 1. General description

The IBM Power 770 (9117-MMD) and IBM Power 780 servers (9179-MHD) use the latest POWER7+ processor technology that is designed to deliver unprecedented performance, scalability, reliability, and manageability for demanding commercial workloads.

The innovative IBM Power 770 and Power 780 servers with POWER7+ processors are symmetric multiprocessing (SMP), rack-mounted servers. These modular-built systems use one to four enclosures. Each enclosure is four EIA units (4U) tall and is housed in a 19-inch rack.

The 9117-MMD and 9179-MHD servers introduce a processor card that houses four P7+ processors. The P7+ technology introduces increased performance over previous P7 processors.

1.1 Systems overview

You can find detailed information about the Power 770 and Power 780 systems within the following sections.

1.1.1 IBM Power 770 server

The Power 770 processor card features 64-bit architecture designed with four single-chip module (SCM) POWER7+ processors. Each POWER7+ SCM enables up to either three or four active processor cores. The 3-core SCM has 756 KB of L2 cache (256 KB per core) and 30 MB of L3 cache (10 MB per core). The 4-core SCM has 1 MB of L2 cache (256 KB per core) and 40 MB of L3 cache (10 MB per core).

A Power 770 server using 3-core SCM processors will enable up to 48 processor cores across four enclosures, running at frequencies of 4.22 GHz. The Power 770 server is available starting as low as four active cores and incrementing one core at a time through built-in capacity on demand (CoD) functions to a maximum of 48 active cores.

A system using 4-core SCM processors will enable up to 64 processor cores across four CEC enclosures running at frequencies of 3.8 GHz. The server can be specified starting with only four active cores and incrementing one core at a time through built-in CoD functions, to a maximum of 64 active cores.

A single Power 770 CEC enclosure is equipped with 16 DIMM slots running at 1066 MHz. A system configured with four drawers and 64 GB DDR3 DIMMs supports up to a maximum of

4.0 TB of DDR3 memory. All POWER7+ DDR3 memory uses memory architecture that provides increased bandwidth and capacity. This increase enables operating at a higher data rate for large memory configurations.

The Power 770 has two integrated POWER7+ I/O controllers that enhance I/O performance while supporting a maximum of six internal Peripheral Component Interconnect Express (PCIe) adapters and six internal small form-factor SAS DASD bays.

The Power 770 supports Active Memory Mirroring (AMM) for the hypervisor, which is available as an optional feature. AMM guards against system-wide outages as a result of any uncorrectable error associated with firmware. When featured, it can be enabled, disabled, or re-enabled depending on the user’s requirements.

Also available as an option is Active Memory Expansion, which enhances memory capacity.

2 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Figure 1-1 shows a Power 770 with the maximum four enclosures, and the front views of a

Power 770 with 4 enclosures

Power 780 single enclosure

Power 770 single enclosure

single-enclosure Power 770 and single-enclosure 780 CEC.

Figure 1-1 Four-enclosure Power 770, and single-enclosure Power 770 and Power 780

1.1.2 IBM Power 780 server

The Power 780 processor card comprises four SCM POWER7+ processors, each designed with 64-bit architecture. The Power 780 POWER7+ SCM enables either up to four or eight active processor cores. Each 4-core has 1 MB of L2 cache and 40 MB of L3 cache; each 8-core has 2 MB of L2 cache and 80 MB of L3 cache (256 KB L2 and10 MB L3 per core).

For the Power 780, each POWER7+ SCM processor is available at frequencies of 4.42 GHz with four cores, or 3.72 GHz with eight cores. The Power 780 server is available starting as low as four active cores for the 4-core SCM and eight active cores for the 8-core SCM. Built-in capacity on demand (CoD) functionality allows incremental activation of one core at a time, up to a maximum of 64 or 128 active cores.

TurboCore mode: TurboCore mode is no longer supported on the 780 processor card.

A single Power 780 CEC enclosure is equipped with 16 DIMM slots running at 1066 MHz. A system configured with four drawers and 64 GB DDR3 DIMMs supports up to a maximum of

The Power 780 has two integrated POWER7+ I/O controllers that enhance I/O performance while supporting a maximum of six internal PCIe adapters and six internal small form-factor SAS DASD bays.

The Power 780 supports AMM for the hypervisor. It is a standard no-additional-charge feature. AMM guards against system-wide outages as a result of any uncorrectable error associated with firmware. You have the option to enable, disable, or re-enable this feature depending on your needs. Also available as an option is Active Memory Expansion, which enhances memory capacity.

Chapter 1. General description 3

1.2 Operating environment

Table 1-1 lists the operating environment specifications for the servers.

Table 1-1 Operating environment for Power 770 and Power 780 (for one enclosure only)

Description Operating Non-operating

Temperature 5 - 35 degrees C

(41 - 95 degrees F)

Relative humidity 20 - 80% 8 - 80%

Maximum dew point 29 degrees C

(84 degrees F)

Operating voltage 200 - 240 V ac Not applicable

Operating frequency 50 - 60 +/- 3 Hz Not applicable

Power consumption Power 770:

1,600 watts maximum (per enclosure with 16 cores active)

Power 780: 1,900 watts maximum (per enclosure with 24 cores active)

Power source loading Power 770:

1.649 kVA maximum (per enclosure with 16 cores active)

Power 780:

1.959 kVA maximum (per enclosure with 24 cores active)

Thermal output Power 770:

5,461 Btu/hr maximum (per enclosure with 16 cores active)

5 - 45 degrees C (41 - 113 degrees F)

28 degrees C (82 degrees F)

Not applicable

Maximum altitude 3048 m

Noise level for one enclosure Power 770 (one enclosure with 16 active cores):

Noise level for four enclosures Power 770 (four enclosure with 64 active cores):

4 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Power 780: 6,485 Btu/hr maximum (per enclosure with 24 cores active)

Not applicable

(10,000 ft)

򐂰 7.1 bels (operating or idle) 򐂰 6.6 bels (operating or idle) with acoustic rack doors

Power 780 (one enclosure with 24 active cores):

򐂰 7.1 bels (operating or idle) 򐂰 6.6 bels (operating or idle) with acoustic rack doors

򐂰 7.6 bels (operating or idle) 򐂰 7.1 bels (operating or idle) with acoustic rack doors

Power 780 (four enclosure with 96 active cores):

򐂰 7.6 bels (operating or idle) 򐂰 7.1 bels (operating or idle) with acoustic rack doors

The IBM Systems Energy Estimator is a web-based tool for estimating power requirements

GX++ Bus

Integrated

Ports

Power

Supplies

FSP

connectors

P C

SPCN

Ports

Serial

Port

USB

Ports

Power 770 and Power 780 enclosure rear view

HMC Ports

for IBM Power Systems. You can use this tool to estimate typical power requirements (watts) for a specific system configuration under normal operating conditions:

http://www-912.ibm.com/see/EnergyEstimator/

1.3 Physical package

Table 1-2 lists the physical dimensions of an individual enclosure. Both servers are available only in a rack-mounted form factor. They are modular systems that can be constructed from one to four building-block enclosures. Each of these enclosures can take 4U (EIA units) of rack space. Thus, a two-enclosure system requires 8U, three enclosures require 12U, and four enclosures require 16U.

Table 1-2 Physical dimensions of a Power 770 and Power 780 enclosure

Dimension Power 770 (Model 9117-MMD)

Width 483 mm (19.0 in) 483 mm (19.0 in)

Depth 863 mm (32.0 in) 863 mm (32.0 in)

Height 174 mm (6.85 in), 4U (EIA units) 174 mm (6.85 in), 4U (EIA units)

Weight 70.3 kg (155 lb) 70.3 kg (155 lb)

single enclosure

Power 780 (Model 9179-MHD) single enclosure

Figure 1-2 shows the rear view of the Power 770 and Power 780.

Figure 1-2 Rear view of the Power 770 and Power 780

Chapter 1. General description 5

1.4 System features

Both Power 770 and Power 780 processor card features 64-bit architecture designed with four single-chip module (SCM) POWER7+ processors.

1.4.1 Power 770 system features

The following features are available on the Power 770:

򐂰 A 4U 19-inch rack-mount system enclosure 򐂰 One to four system enclosures: 16U maximum system size 򐂰 One processor card feature per enclosure (includes the voltage regulator):

– 0 to 12-core, 4.22 GHz processor card (FC EPM0) – 0 to 16-core, 3.8 GHz processor card (FC EPM1)

򐂰 POWER7+ DDR3 Memory DIMMs (16 DIMM slots per CEC enclosure):

– 0 - 32 GB (4 X 8 GB), 1066 MHz (FC EM40) – 0 - 64 GB (4 X 16 GB), 1066 MHz (FC EM41) – 0 - 128 GB (4 X 32 GB), 1066 MHz (FC EM42) – 0 - 256 GB (4 X 64 GB), 1066 MHz (FC EM44)

򐂰 Six hot-swappable, 2.5-inch, small form factor, SAS disk or SSD bays per enclosure 򐂰 One hot-plug, slim-line, SATA media bay per enclosure (optional) 򐂰 Redundant hot-swap AC power supplies in each enclosure 򐂰 Choice of Integrated Multifunction Card options; maximum one per enclosure:

– Dual 10 Gb Copper and Dual 1 Gb Ethernet (FC 1768) – Dual 10 Gb Optical and Dual 1 Gb Ethernet (FC 1769)

򐂰 One serial port included on each Integrated Multifunction Card 򐂰 Two USB ports included on each Integrated Multifunction Card, plus another USB port on

each enclosure (maximum nine usable per system)

Additional considerations: 򐂰 The Ethernet port of the Integrated Multifunction Card cannot be used for an IBM i

console. Use separate Ethernet adapters that can be directly controlled by IBM i without the Virtual I/O server for IBM i LAN consoles if desired. Alternatively, an HMC can also be used for an IBM i console.

򐂰 The first CEC enclosure must contain one Integrated Multifunction Card (FC 1768 or

FC 1769). The Integrated Multifunction Card is optional for the second, third, or fourth CEC enclosure.

򐂰 Each Integrated Multifunction Card has four Ethernet ports, two USB ports, and one

serial port. Usage of the serial port by AIX or Linux is supported for MODEM call home, TTY console, and snooping even if an HMC is attached to the server. Usage by the serial port to communicate with a UPS is not supported.

򐂰 The first and second CEC enclosures each have two HMC ports on the service

processor (FC EU09). If there are two CEC enclosures, the HMC must be connected to both service processor cards.

򐂰 Two HMC ports per enclosure (maximum four per system)

6 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

򐂰 Eight I/O expansion slots per enclosure (maximum 32 per system)

– Six Gen2 PCIe 8x slots plus two GX++ slots per enclosure

򐂰 Dynamic LPAR support, Processor and Memory Capacity Upgrade on Demand (CUoD) 򐂰 PowerVM (optional)

– IBM Micro-Partitioning® technology – Virtual I/O Server (VIOS) – Automated CPU and memory reconfiguration support for dedicated and shared

processor logical partition groups (dynamic LPAR)

– Support for manual provisioning of resources, namely PowerVM Live Partition

Migration (PowerVM Enterprise Edition)

򐂰 Optional IBM PowerHA® for AIX, IBM i, and Linux 򐂰 A 12X I/O drawer with PCI slots

– Up to 16 PCIe I/O drawers (FC 5802 or FC 5877) – Up to 32 PCI-X DDR I/O drawers (7314-G30 or FC 5796)

򐂰 Disk-only I/O drawers

– Up to 56 EXP24S SFF SAS I/O drawers on external SAS controller (FC 5887) – Up to 110 EXP12S SAS DASD/SSD I/O drawers on SAS PCI controllers (FC 5886) – Up to 60 EXP24 SCSI DASD Expansion drawers on SCSI PCI controllers (7031-D24) – Enhanced EXP30 Ultra Solid-State Drive (SSD) I/O Drawer holding up to 30 SSDs (FC

EDR1). The EXP30 Ultra SSD I/O Drawer is attached directly to the Power 770 (9117-MMD) and Power 780 (9179-MHD) GX++ slot for higher bandwidth.

򐂰 IBM Systems Director Active Energy Manager™

The Power 770 operator interface controls, located on the front panel of the primary I/O drawer, consist of a power ON/OFF button with an IBM POWER® indicator, an LCD display for diagnostic feedback, a RESET button, and a disturbance or system attention LED.

1.4.2 Power 780 system features

The following features are available on the Power 780:

򐂰 A 4U 19-inch rack-mount system enclosure 򐂰 One to four system enclosures: 16U maximum system size 򐂰 One processor card feature per enclosure (includes the voltage regulator):

– 0 to 16 core, 4.42 GHz processor card (FC EPH0) – 0 to 32 core, 3.72 GHz processor card (FC EPH2)

򐂰 POWER7+ DDR3 Memory DIMMs (16 DIMM slots per processor card):

– 0 - 32 GB (4 X 8 GB), 1066 MHz (FC EM40) – 0 - 64 GB (4 X 16 GB), 1066 MHz (FC EM41) – 0 - 128 GB (4 X 32 GB), 1066 MHz (FC EM42) – 0 - 256 GB (4 X 64 GB), 1066 MHz (FC EM44)

Chapter 1. General description 7

򐂰 Choice of Integrated Multifunction Card options; maximum one per enclosure:

– Dual 10 Gb Copper and Dual 1 Gb Ethernet (FC 1768) – Dual 10 Gb Optical and Dual 1 Gb Ethernet (FC 1769)

򐂰 One serial port included on each Integrated Multifunction Card 򐂰 Two USB ports included on each Integrated Multifunction Card plus another USB port on

each enclosure (maximum nine usable per system)

Additional considerations: 򐂰 The Ethernet ports of the Integrated Multifunction Card cannot be used for an IBM i

console. Separate Ethernet adapters that can be directly controlled by IBM i without the Virtual I/O server should be used for IBM i LAN consoles if desired. Alternatively, an HMC can also be used for an IBM i console.

򐂰 The first CEC enclosure must contain one Integrated Multifunction Card (FC 1768 or

FC 1769). The Integrated Multifunction Card is optional for the second, third, or fourth CEC enclosure.

򐂰 Each Integrated Multifunction Card has four Ethernet ports, two USB ports, and one

򐂰 The first and second CEC enclosures each have two HMC ports on the service

processor (FC EU09). If there are two CEC enclosures, the HMC must be connected to both service processor cards.

򐂰 Two HMC ports per enclosure (maximum four per system) 򐂰 Eight I/O expansion slots per enclosure (maximum 32 per system)

– Six Gen2 PCIe 8x slots plus two GX++ slots per enclosure

򐂰 Dynamic LPAR support, Processor and Memory CUoD 򐂰 PowerVM (optional)

– Micro-Partitioning – Virtual I/O Server (VIOS) – Automated CPU and memory reconfiguration support for dedicated and shared

processor logical partition (LPAR) groups

– Support for manual provisioning of resources partition migration (PowerVM

Enterprise Edition)

򐂰 Optional PowerHA for AIX, IBM i, and Linux 򐂰 A 12X I/O drawer with PCI slots

– Up to 16 PCIe I/O drawers (FC 5802 or FC 5877) – Up to 32 PCI-X DDR I/O drawers (7314-G30 or feature FC 5796)

򐂰 Disk-only I/O drawers

򐂰 IBM Systems Director Active Energy Manager

8 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

The Power 780 operator interface controls, located on the front panel of the primary I/O drawer, consist of a power ON/OFF button with a POWER indicator, an LCD display for diagnostic feedback, a RESET button, and a disturbance or system attention LED.

1.4.3 Minimum features

Each system has a minimum feature set in order to be valid. Table 1-3 shows the minimum system configuration for a Power 770.

Table 1-3 Minimum features for Power 770 system

Power 770 minimum features Additional notes

1x CEC enclosure (4U) 򐂰 1x System Enclosure with IBM Bezel (FC EB85) or OEM

Bezel (FC EB86)

򐂰 1x Service Processor (FC EU09) 򐂰 1x DASD Backplane (FC 5652) 򐂰 2x Power Cords (two selected by customer)

– 2x A/C Power Supply (FC 5532)

򐂰 1x Operator Panel (FC EC53) 򐂰 1x Integrated Multifunction Card options (one of these):

– Dual 10 Gb Copper and Dual 1 Gb Ethernet (FC 1768) – Dual 10 Gb Optical and Dual 1 Gb Ethernet (FC 1769)

1x primary operating system (one of these)

1x Processor Card 򐂰 0 to 12-core, 4.42 GHz processor card (FC EPM0)

4x Processor Activations (quantity of four for one of these)

2x DDR3 Memory DIMMs (one of these)

32x Activations of 1 GB DDR3 POWER7+ memory

32x Activations of 100 GB DDR3 POWER7+ memory

For AIX and Linux: 1x disk drive For IBM i: 2x disk drives

1X Language Group (selected by the customer)

򐂰 AIX (FC 2146) 򐂰 Linux (FC 2147) 򐂰 IBM i (FC 2145)

򐂰 0 to 16-core, 3.72 GHz processor card (FC EPM2)

򐂰 One Processor Activation for processor feature FC EPM0

(FC EPMA)

򐂰 One Processor Activation for processor feature FC EPM2

(FC EPMB)

򐂰 0 - 32 GB (4 X 8 GB), 1066 MHz (FC EM40) 򐂰 0 - 64 GB (4 X 16 GB), 1066 MHz (FC EM41) 򐂰 0 - 128 GB (4 X 32 GB), 1066 MHz (FC EM42) 򐂰 0 - 256 GB (4 X 64 GB), 1066 MHz (FC EM44)

FC EMA2

FC EMA3

Formatted to match the system Primary O/S indicator selected, or if using a Fibre Channel attached SAN (indicated by FC 0837) a disk drive is not required.

Chapter 1. General description 9

Power 770 minimum features Additional notes

1x Removable Media Device (FC 5771)

1x HMC Required for every Power 770 (9117-MMD)

Considerations:

򐂰 A minimum number of four processor activations must be ordered per system. 򐂰 The minimum activations ordered with MES orders of memory features EM40, EM41, EM42, and

EM44 depend on the total installed capacity of features EM40, EM41, EM42, and EM44. This allows newly ordered mem ory to be purchased with less than 50% activations when the currently installed capacity exceeds 50% of the existing features EM40, EM41, EM42 and EM44 capacity.

򐂰 The minimum activations ordered with all initial orders of memory features EM40, EM41, EM42,

and EM44 must be 50% of their installed capacity.

Optionally orderable, a stand-alone system (not network attached) would required this feature.

Table 1-4 shows the minimum system configuration for a Power 780 system.

Table 1-4 Minimum features for Power 780 system

Power 780 minimum features Additional notes

1x CEC enclosure (4U) 򐂰 1x System Enclosure with IBM Bezel (FC EB95) or OEM

Bezel (FC EB96)

򐂰 1x Service Processor (FC EU09) 򐂰 1x DASD Backplane (FC 5652) 򐂰 2x Power Cords (two selected by customer)

– 2x A/C Power Supply (FC 5532)

򐂰 1x Operator Panel (FC EC53) 򐂰 1x Integrated Multifunction Card options (one of these):

– Dual 10 Gb Copper and Dual 1 Gb Ethernet (FC 1768) – Dual 10 Gb Optical and Dual 1 Gb Ethernet (FC 1769)

1x primary operating system (one of these)

1x Processor Card (one of these)

4x Processor Activations for Processor Feature FC EPH0 or FC EPH2

2x DDR3 Memory DIMM (one of these)

32x Activations of 1 GB DDR3 POWER7+ memory

32x Activations of 100 GB DDR3 POWER7+ memory

For AIX and Linux: 1x disk drive For IBM i: 2x disk drives

1X Language Group (selected by the customer)

򐂰 AIX (FC 2146) 򐂰 Linux (FC 2147) 򐂰 IBM i (FC 2145)

򐂰 0 to 16-core, 4.42 GHz processor card (FC EPH0) 򐂰 0 to 32-core, 3.72 GHz processor card (FC EPH2)

򐂰 0 to 16-core, 4.42 GHz processor card (FC EPH0) requires

FC EPHC

򐂰 0 to 32-core, 3.72 GHz processor card (FC EPH2) requires

FC EPHD

򐂰 0 - 32 GB (4 X 8 GB), 1066 MHz (FC EM40) 򐂰 0 - 64 GB (4 X 16 GB), 1066 MHz (FC EM41) 򐂰 0 - 128 GB (4 X 32 GB), 1066 MHz (FC EM42) 򐂰 0 - 256 GB (4 X 64 GB), 1066 MHz (FC EM44)

FC EMA2

FC EMA3

Formatted to match the system Primary O/S indicator selected, or if using a Fibre Channel attached SAN (indicated by FC 0837) a disk drive is not required.

10 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Power 780 minimum features Additional notes

1x Removable Media Device (FC 5771)

1x HMC Required for every Power 780 (9179-MHD)

Considerations:

򐂰 A minimum number of four processor activations must be ordered per system. 򐂰 The minimum activations ordered with MES orders of memory features EM40, EM41, EM42, and

򐂰 The minimum activations ordered with all initial orders of memory features EM40, EM41, EM42,

and EM44 must be 50% of their installed capacity.

1.4.4 Power supply features

Two system AC power supplies are required for each CEC enclosure. The second power supply provides redundant power for enhanced system availability. To provide full redundancy, the two power supplies must be connected to separate power distribution units (PDUs).

A CEC enclosure will continue to function with one working power supply. A failed power supply can be hot-swapped but must remain in the system until the replacement power supply is available for exchange. The system requires one functional power supply in each CEC enclosure to remain operational.

Optionally orderable, a stand-alone system (not network attached) requires this feature.

Each Power 770 or Power 780 server with two or more CEC enclosures must have one Power Control Cable (FC 6006 or similar) to connect the service interface card in the first enclosure to the service interface card in the second enclosure.

1.4.5 Processor card features

Each of the four system enclosures contains one powerful POWER7+ processor EPH0 card feature, consisting of four single-chip module processors. Each of the POWER7+ processors in the server has a 64-bit architecture.

The Power 770 has two types of processor cards, offering the following features: 򐂰 Four 3-core POWER7+ SCMs with 120 MB of L3 cache (12-cores per processor card,

each core with 10 MB of L3 cache) at 4.22 GHz (FC EPM0)

򐂰 Four 4-core POWER7+ SCMs with 160 MB of L3 cache (16-cores per processor card,

each core with 10 MB of L3 cache) at 3.72 GHz (FC EPM1)

The Power 780 has two types of processor cards (note that the TurboCore feature is no longer offered on 780):

򐂰 Four 4-core POWER7+ SCMs with 160 MB of L3 cache (16-cores per processor card,

each core with 10 MB of L3 cache) at 4.42 GHz (FC EPH0)

򐂰 Four 8-core POWER7+ SCMs with 320 MB of L3 cache (32-cores per processor card,

each core with 10 MB of L3 cache) at 3.72 GHz (FC EPH2)

Chapter 1. General description 11

Figure 1-3 shows the top view of the Power 770 and Power 780 system with four SCMs

POWER7

Memory

TPMD

PCIe Slot #1

PCIe Slot #2

PCIe Slot #3

PCIe Slot #4

PCIe Slot #5

PCIe Slot #6

F a n s

installed. The four POWER7+ SCMs and the system memory reside on a single processor card feature.

Figure 1-3 Top view of a Power 770 or 780 system with four SCMs

TurboCore: TurboCore mode is no longer supported on the Power 780.

Several types of capacity on demand (CoD) processor options are available on the Power 770 and Power 780 servers to help meet changing resource requirements in an on-demand environment by using resources installed on the system but not activated. CoD allows you to purchase additional permanent processor or memory capacity and dynamically activate it when needed.

More detailed information about CoD is in 2.4, “Capacity on Demand” on page 58.

1.4.6 Summary of processor features

Table 1-5 summarizes the processor feature codes for the Power 770.

Table 1-5 Summary of processor features for the Power 770

Feature

Description OS

code

EPM0 0 to 12-core 4.22 GHz POWER7+ processor card:

12-core 4.22 GHz POWER7+ CUoD processor planar contains four 3-core processors. Each processor has 756 KB of L2 cache (256 KB per core) and 40 MB of L3 cache (10 MB per core). There are 16 DDR3 DIMM slots on the processor planar (8 DIMM slots per processor), which can be used as Capacity on Demand (CoD) memory without activating the processors. The voltage regulators are included in this feature code.

support

AIX IBM i Linux

12 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Feature code

Description OS

support

EPMA One processor activation for processor FC EPM0:

Each occurrence of this feature permanently activates one processor on Processor Card FC EPM0. One processor activation for processor feature FC EPM0 with inactive processors.

EPMW Processor CoD utility billing for FC EPM0, 100 processor-minutes:

Provides payment for temporary use of processor feature FC EPM0 with supported AIX or Linux operating systems. Each occurrence of this feature will pay for 100 minutes of usage. The purchase of this feature occurs after the customer has 100 minutes of use on processor cores in the shared processor pool that are not permanently active.

EPMX Processor CoD utility billing for FC EPM0, 100 processor-minutes:

Provides payment for temporary use of processor feature FC EPM0 with supported IBM i operating systems. Each occurrence of this feature will pay for 100 minutes of usage. The purchase of this feature occurs after the customer has 100 minutes of use on processor cores in the shared processor pool that are not permanently active.

EPME One processor-day on/off billing for FC EPM0:

After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/Off Processor Core Day Billing features and bill you. One FC EPME must be ordered for each billable processor core day of feature FC EPM0 used by a supported AIX or Linux operating system.

AIX IBM i Linux

AIX Linux

IBM i

AIX Linux

EPMF One processor-day on/off billing for FC EPM0:

IBM i After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/Off Processor Core Day Billing features and the client will be charged. One FC EPMF must be ordered for each billable processor core day of feature FC EPM0 used by a supported IBM i operating system.

EPMN On/off processor CoD billing, 100 processor-days, for FC EPM0 AIX

Linux

EPMP On/off processor CoD billing, 100 processor-days, for FC EPM0 IBM i

EP9T 90-day temporary on/off processor enablement AIX

IBM i

Linux

EPM1 0 to 16-core 3.8 GHz POWER7+ processor card:

16-core 3.8 GHz POWER7+ CUoD processor planar containing two 8-core processors. Each processor has 2 MB of L2 cache (256 KB per

AIX

IBM i

Linux core) and 32 MB of L3 cache (4 MB per core). There are 16 DDR3 DIMM slots on the processor planar (8 DIMM slots per processor), which can be used as capacity on demand (CoD) memory without activating the processors. The voltage regulators are included in this feature code.

Chapter 1. General description 13

Feature code

Description OS

support

EPMB One processor activation for processor FC EPM1:

Each occurrence of this feature will permanently activate one processor on Processor Card FC EPM1. One processor activation for processor feature FC EPM1 with inactive processors.

EPMY Processor CoD utility billing for FC EPM1, 100 processor-minutes:

Provides payment for temporary use of processor feature FC EPM1 with supported AIX or Linux operating systems. Each occurrence of this feature will pay for 100 minutes of usage. The purchase of this feature occurs after the customer has 100 minutes of use on processor cores in the shared processor pool that are not permanently active.

EPMZ Processor CoD utility billing for FC EPM1, 100 processor-minutes:

Provides payment for temporary use of processor feature FC EPM1 with supported IBM i operating systems. Each occurrence of this feature will pay for 100 minutes of usage. The purchase of this feature occurs after the customer has 100 minutes of use on processor cores in the shared processor pool that are not permanently active.

EPMG One processor-day on/off billing for FC EPM1:

After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/Off Processor Core Day Billing features and the client will be charged. One FC EPMG must be ordered for each billable processor core day of feature FC EPM1 used by a supported AIX or Linux operating system.

AIX

IBM i

Linux

AIX

Linux

IBM i

AIX

Linux

EPMH One processor-day on/off billing for FC EPM1:

IBM i After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/Off Processor Core Day Billing features and the client will be charged. One FC EPMH must be ordered for each billable processor core day of feature FC EPM1 used by a supported IBM i operating system.

EPMQ On/off processor CoD billing, 100 processor-days, for FC EPM1 AIX

Linux

EPMR On/off processor CoD billing, 100 processor-days, for FC EPM1 IBM i

7951 On/Off Processor Enablement:

This feature can be ordered to enable your server for On/Off capacity on demand. After it is enabled, you can request processors on a

AIX

Linux

IBM i temporary basis. You must sign an On/Off Capacity on Demand contract before you order this feature.

Note: To renew this feature after the allowed 360 processor days have been used, this feature must be removed from the system configuration file and reordered by placing a miscellaneous equipment specification (MES) order.

14 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Table 1-6 summarizes the processor feature codes for the Power 780.

Table 1-6 Summary of processor features for the Power 780

Feature

Description OS

code

EPH0 0 to 16-core 4.42 GHz POWER7+ processor card:

16-core 4.42 GHz POWER7+ CUoD processor card containing four 4-core processors. Each processor has 1 MB of L2 cache (256 KB per core) and 40 MB of L3 cache (4 MB per core). There are 16 DDR3 DIMM slots on the processor planar (8 DIMM slots per processor), which can be used as capacity on demand (CoD) memory without activating the processors. The voltage regulators are included in this feature code.

EPHA 1-core activation for processor feature FC EPH0:

Each occurrence of this feature will permanently activate one processor core on Processor Card FC EPH0.

EPHN 100 on/off processor days of CoD billing for processor FC EPH0:

After the On/off Processor function is enabled in a system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is provided to your sales channel. The sales channel will place an order on your behalf for the quantity of this feature that matches your reported use. One FC EP2L provides 100 days of on/off processor billing for POWER7+ CoD Processor Book FC EPH0 for AIX/Linux.

EPHP 100 on/off processor days of CoD billing for processor FC EPH0:

After the On/off Processor function is enabled in a system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is provided to your sales channel. The sales channel will place an order on your behalf for the quantity of this feature that matches your reported use. One FC EP2M provides 100 days of on/off processor billing for POWER7+ CoD Processor Book FC EPH0 for IBM i.

support

AIX IBM i Linux

AIX Linux

IBM i

EPHE One processor day on/off billing for FC EPH0:

After an On/Off Processor Enablement feature is ordered and the

AIX

Linux associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/Off Processor Core Day Billing features and the client will be charged. One FC 5342 must be ordered for each billable processor core day of feature FC EPH0 used by a supported AIX or Linux operating system.

EPHF One processor day on/off billing for FC EPH0:

IBM i After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/ Off Processor Core Day Billing features and the client will be charged. One FC 5343 must be ordered for each billable processor core day of feature FC EPH0 used by a supported IBM i operating system.

EPHU Processor CoD utility billing for FC EPH0, 100 processor-minutes AIX

Linux

EPHV Processor CoD utility billing for FC EPH0, 100 processor-minutes IBM i

Chapter 1. General description 15

Feature code

Description OS

support

EP9T 90 day temporary on/off processor enablement AIX

IBM i

Linux

EPH2 0 to 32-core 3.72 GHz POWER7+ processor card:

32-core 3.72 GHz POWER7+ CUoD processor planar containing four 8-core processors. Each processor has 2 MB of L2 cache (256 KB per core) and 80 MB of L3 cache (10 MB per core). There are 16 DDR3 DIMM slots on the processor planar (eight DIMM slots per processor), which can be used as CoD memory without activating the processors. The voltage regulators are included in this feature code.

EPHC 1-core activation for processor feature FC EPH2:

Each occurrence of this feature will permanently activate one processor core on Processor Card FC EPH2.

EPHS 100 on/off processor days of CoD billing for processor FC EPH2:

After the On/off Processor function is enabled in a system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is provided to your sales channel. The sales channel will place an order on your behalf for the quantity of this feature that matches your reported use. One FC EPHS provides 100 days of on/off processor billing for POWER7+ CoD Processor Book FC EPH2 for AIX/Linux.

EPHT 100 on/off processor days of CoD billing for processor FC EPH2:

After the On/off Processor function is enabled in a system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is provided to your sales channel. The sales channel will place an order on your behalf for the quantity of this feature that matches your reported use. One FC EP2P provides 100 days of on/off processor billing for POWER7+ CoD Processor Book FC EPH2 for IBM i.

AIX

IBM i

Linux

AIX

Linux

IBM i

AIX

Linux

IBM i

EPHJ One processor day on/off billing for FC EPH2:

After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/ Off Processor Core Day Billing features and the client will be charged. One FC EP27 must be ordered for each billable processor core day of feature FC EPH2 used by a supported AIX or Linux operating system.

EPHK One processor day on/off billing for FC EPH2:

After an On/Off Processor Enablement feature is ordered and the associated enablement code is entered into the system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is then provided to your sales channel. The sales channel will place an order for a quantity of On/ Off Processor Core Day Billing features and the client will be charged. One FC EP29 must be ordered for each billable processor core day of feature FC EPH2 used by a supported IBM i operating system.

EPHY Processor CoD utility billing for FC EPH0, 100 processor-minutes AIX

EPHZ Processor CoD utility billing for FC EPH0, 100 processor-minutes IBM i

16 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

AIX

Linux

IBM i

Linux

Feature code

Description OS

support

FC 7951 On/Off Processor Enablement:

This feature can be ordered to enable your server for On/Off capacity on demand. After it is enabled, you can request processors on a temporary basis. You must sign an On/Off Capacity on Demand contract before you order this feature.

Note: To renew this feature after the allowed 360 processor days have been used, this feature must be removed from the system configuration file and reordered by placing an MES order.

AIX

Linux

IBM i

Chapter 1. General description 17

1.4.7 Memory features

P7+

CP1

P7+

CP0

Regulator #1

Regulator #4

Regulator #5

M0  A SN DDR3 DIMM #5

M0 - B SN DDR3 DIMM #6

M0 - C SN DDR3 DIMM #7

M0 - D SN DDR3 DIMM #8

GX1

GX0

GX1

GX0

MC0

M0  A SN DDR3 DIMM #1

M0 - B SN DDR3 DIMM #2

M0 - C SN DDR3 DIMM #3

M0 - D SN DDR3 DIMM #4

M0 - A SN DDR3 DIMM #13

M0 - B SN DDR3 DIMM #14

M0 - C SN DDR3 DIMM #15

M0 - D SN DDR3 DIMM #16

M0 - A SN DDR3 DIMM #9

M0 - B SN DDR3 DIMM #10

M0 - C SN DDR3 DIMM #11

M0 - D SN DDR3 DIMM #12

MC0

Regulator #7

Regulator #8

TPMD Slot

Regulator #3

Regulator #2

P7+

CP2

GX0

GX1

MC0

P7+

CP3

GX0

GX1

MC0

Regulator #6

GX1

to IO Planar

P5IOC2-B

P7IOC-B

GX0

to IO Planar

P5IOC2-A

P7IOC-A

GX3

to GX Slot lower

(P1-C3-T1/T2)

GX2

to GX Slot upper

(P1-C2-T1/T2)

Power

Connector

In POWER7+ systems, DDR3 memory is used throughout. There are four separate capacity DIMMs: 8 GB, 16 GB, 32 GB, or 64 GB. The POWER7+ DDR3 memory has been redesigned to provide greater bandwidth and capacity. The 16, 32 and 64 GB DIMMs use 4 GB DRAMs. This enables operating at a higher data rate for large memory configurations. All processor cards have 16 memory DIMM slots (eight per processor) running at speeds up to 1066 MHz and must be populated with POWER7+ DDR3 Memory DIMMs.

Figure 1-4 outlines the general connectivity of an 8-core POWER7+ processor and DDR3 memory DIMMS. The figure shows the eight memory channels (four per memory controller).

Figure 1-4 Outline of POWER7+ processor connectivity to DDR3 DIMMs in Power 770 and Power 780

On each processor card for the Power 770 and Power 780 there is a total of 16 DDR3 memory DIMM slots to be connected. Each of the four SCMs on the card accesses four DIMM slots.

The quad-high (96 mm) DIMM cards are connected to the POWER7+ processor memory controller through an advanced memory buffer ASIC. For each DIMM, there is a corresponding memory buffer. Each memory channel into the POWER7+ memory controllers is driven at 6.4 GHz.

Each DIMM (except the 64 GB DIMM) contains DDR3 x8 DRAMs in a configuration, with 10 DRAMs per rank, and plugs into a 276-pin DIMM slot connector. The 64 GB DIMM is an 8-rank DIMM using x4 parts (1024Kx4). The x4 DIMMs are 20 DRAMs per rank.

DDR2 DIMMs: DDR2 DIMMs (used in IBM POWER6® based systems) are not supported

18 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

in POWER7+-based systems.

The Power 770 and Power 780 have memory features in 32 GB, 64 GB, 128 GB, and 256 GB capacities. Table 1-7 summarizes the capacities of the memory features and highlights other characteristics.

Table 1-7 Summary of memory features

Feature code

FC EM40 DDR3 32 GB 1066 MHz 4 x 8 GB DIMMs 4

FC EM41 DDR3 64 GB 1066 MHz 4 x 16 GB DIMMs 4

FC EM42 DDR3 128 GB 1066 MHz 4 x 32 GB DIMMs 4

FC EM44 DDR3 256 GB 1066 MHz 4 x 64 GB DIMMs 4

Memory technology

Capacity Access

rate

DIMMs DIMM slots

used

None of the memory in these features is active. FC EMA2 or FC EMA3 must be purchased to activate the memory. Table 1-8 outlines the memory activation feature codes and corresponding memory capacity activations.

Table 1-8 CoD system memory activation features

Feature code

FC EMA2 1 GB Activation of 1 GB of DDR3 POWER7+ memory. Each

FC EMA3 100 GB Activation of 100 GB of DDR3 POWER7+ memory.

Activation capacity

Additional information OS

support

AIX occurrence of this feature permanently activates 1 GB of DDR3 POWER7+ memory.

Each occurrence of this feature permanently activate 100 GB of DDR3 POWER7+ memory.

IBM i

Linux

AIX

IBM i

Linux

FC 7954 N/A On/Off Memory Enablement: This feature can be

ordered to enable your server for On/Off Capacity on Demand. After it is enabled, you can request memory on a temporary basis. You must sign an On/Off Capacity on Demand contract before this feature is ordered. To renew this feature after the allowed 999 GB days have been used, this feature must be removed from the system configuration file and reordered by placing an MES order.

FC 4710 N/A On/Off 999 GB-Days, Memory Billing POWER7+:

After the ON/OFF Memory function is enabled in a system, you must report your on/off usage to IBM at least monthly. This information, used to compute your billing data, is provided to your sales channel. The sales channel will place an order on your behalf for the quantity of this feature that matches your reported use. One FC 4FC 4710 feature must be ordered for each 999 billable days for each 1 GB increment of POWER7+ memory that was used.

AIX

IBM i

Linux

AIX

IBM i

Linux

Chapter 1. General description 19

Feature code

Activation capacity

Additional information OS

support

FC 7377 N/A On/Off, 1 GB-1Day, Memory Billing POWER7+:

After the ON/OFF Memory function is enabled in a system you must report the client’s on/off usage to IBM on a monthly basis. This information is used to compute IBM billing data. One FC 7377 feature must be ordered for each billable day for each 1 GB increment of POWER7+ memory that was used.

Note that inactive memory must be available in the system for temporary use.

Notes: 򐂰 All POWER7+ memory features must be purchased with sufficient permanent memory activation

features so that the system memory is at least 50% active.

򐂰 The minimum activations ordered with MES orders of memory features EM40, EM41, EM42, and

򐂰 The minimum activations ordered with all initial orders of memory features EM40, EM41, EM42,

and EM44 must be 50% of their installed capacity.

AIX

IBM i

Linux

Moving memory: Memory CoD activations activate memory hardware only for the system serial number for which they are purchased. If memory hardware is moved to another system, the memory might not be functional in that system until arrangements are made to move the memory activations or purchase additional memory activations.

1.5 Disk and media features

Each system building block features two SAS DASD controllers with six hot-swappable

2.5-inch small form-factor (SFF) disk bays and one hot-plug, slim-line media bay per enclosure. The SFF SAS disk drives and solid state drive (SSD) are supported internally. In a full configuration with four connected building blocks, the combined system supports up to 24 disk bays. SAS drives and SSD drives can share the same backplane.

Table 1-9 shows the available disk drive feature codes that each bay can contain.

Table 1-9 Disk drive feature code description

Feature code

1917 146 GB 15 K RPM SAS SFF-2 Disk Drive AIX, Linux

1886 146 GB 15 K RPM SFF SAS Disk Drive AIX, Linux

1775 177 GB SFF-1 SSD with eMLC AIX, Linux

1793 177 GB SFF-2 SSD with eMLC AIX, Linux

1995 177 GB SSD Module with eMLC AIX, Linux

1925 300 GB 10 K RPM SAS SFF-2 Disk Drive AIX, Linux

Description OS

support

20 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Feature code

1885 300 GB 10 K RPM SFF SAS Disk Drive AIX, Linux

1880 300 GB 15 K RPM SAS SFF Disk Drive AIX, Linux

1953 300 GB 15 K RPM SAS SFF-2 Disk Drive AIX, Linux

ES02 387GB 1.8" SAS SSD for AIX/Linux with eMLC AIX, Linux

ES0A 387 GB SFF-1 SSD with eMLC AIX, Linux

ES0C 387 GB SFF-2 SSD eMLC AIX, Linux

1790 600 GB 10 K RPM SAS SFF Disk Drive AIX, Linux

1964 600 GB 10 K RPM SAS SFF-2 Disk Drive AIX, Linux

1751 900 GB 10 K RPM SAS SFF-1 Disk Drive AIX, Linux

1752 900 GB 10 K RPM SAS SFF-2 Disk Drive AIX, Linux

1947 139 GB 15 K RPM SAS SFF-2 Disk Drive IBM i

1888 139 GB 15 K RPM SFF SAS Disk Drive IBM i

1787 177 GB SFF-1 SSD with eMLC IBM i

1794 177 GB SFF-2 SSD with eMLC IBM i

Description OS

support

1996 177 GB SSD Module with eMLC IBM i

1956 283 GB 10 K RPM SAS SFF-2 Disk Drive IBM i

1911 283 GB 10 K RPM SFF SAS Disk Drive IBM i

1879 283 GB 15 K RPM SAS SFF Disk Drive IBM i

1948 283 GB 15 K RPM SAS SFF-2 Disk Drive IBM i

ES0B 387 GB SFF-1 SSD eMLC IBM i

ES0D 387 GB SFF-2 SSD eMLC IBM i

1916 571 GB 10 K RPM SAS SFF Disk Drive IBM i

1962 571 GB 10 K RPM SAS SFF-2 Disk Drive IBM i

1737 856 GB 10 K RPM SAS SFF-1 Disk Drive IBM i

1738 856 GB 10 K RPM SAS SFF-2 Disk Drive IBM i

Certain adapters are available for order in large quantities. Table 1-10 lists the disk drives available in a quantity of 150.

Table 1-10 Available disk drives in quantity of 150

Feature code

Description OS

support

EQ51 Quantity 150 of 1751 (900 GB SFF-2 disk) AIX, Linux

EQ52 Quantity 150 of 1752 (900 GB SFF-2 disk) AIX, Linux

EQ0A Quantity 150 of ES0A (387GB SAS SFF SSD) AIX, Linux

EQ0C Quantity 150 of ES0C (387GB SAS SFF SSD) AIX, Linux

Chapter 1. General description 21

Feature code

EQ51 Quantity 150 of FC 1751 (900 GB 15 K RPM SAS SFF-1 Disk Drive) AIX, Linux

EQ52 Quantity 150 of FC 1752 (900 GB 15 K RPM SAS SFF-2 Disk Drive) AIX, Linux

7550 Quantity 150 of FC 1790 (600 GB 10 K RPM SAS SFF Disk Drive) AIX, Linux

1887 Quantity 150 of FC 1793 (177 GB SAS SSD) AIX, Linux

1928 Quantity 150 of FC 1880 (300 GB 15 K RPM SAS SFF Disk Drive) AIX, Linux

7547 Quantity 150 of FC 1885 (300 GB 10 K RPM SFF SAS Disk Drive) AIX, Linux

7548 Quantity 150 of FC 1886 (146 GB 15 K RPM SFF SAS Disk Drive) AIX, Linux

1866 Quantity 150 of FC 1917 (146 GB 15 K RPM SAS SFF-2 Disk Drive) AIX, Linux

1869 Quantity 150 of FC 1925 (300 GB 10 K RPM SAS SFF-2 Disk Drive) AIX, Linux

1929 Quantity 150 of FC 1953 (300 GB 15 K RPM SAS SFF-2 Disk Drive) AIX, Linux

1818 Quantity 150 of FC 1964 (600 GB 10 K RPM SAS SFF-2 Disk Drive) AIX, Linux

7578 Quantity 150 of FC 1775 (177 GB SAS SFF SSD) AIX. Linux

EQ37 Quantity 150 of 1737 (856 GB SFF-2 disk) IBM i

EQ38 Quantity 150 of 1738 (856 GB SFF-2 disk) IBM i

Description OS

support

EQ0B Quantity 150 of ES0B (387GB SAS SFF SSD) IBM i

EQ0D Quantity 150 of ES0D (387GB SAS SFF SSD) IBM i

EQ37 Quantity 150 of FC 1737 (856 GB 10 K RPM SAS SFF-1 Disk Drive) IBM i

EQ38 Quantity 150 of FC 1738 (856 GB 10 K RPM SAS SFF-2 Disk Drive) IBM i

7582 Quantity 150 of FC 1787 (177 GB SAS SFF SSD) IBM i

1958 Quantity 150 of FC 1794 (177 GB SAS SSD) IBM i

1926 Quantity 150 of FC 1879 (283 GB 15 K RPM SAS SFF Disk Drive) IBM i

7544 Quantity 150 of FC 1888 (139 GB 15 K RPM SFF SAS Disk Drive) IBM i

7557 Quantity 150 of FC 1911(283 GB 10 K RPM SFF SAS Disk Drive) IBM i

7566 Quantity 150 of FC 1916 (571 GB 10 K RPM SAS SFF Disk Drive) IBM i

1868 Quantity 150 of FC 1947 (139 GB 15 K RPM SAS SFF-2 Disk Drive) IBM i

1927 Quantity 150 of FC 1948 (283 GB 15 K RPM SAS SFF-2 Disk Drive) IBM i

1844 Quantity 150 of FC 1956 (283 GB 10 K RPM SAS SFF-2 Disk Drive) IBM i

1817 Quantity 150 of FC 1962 (571 GB 10 K RPM SAS SFF-2 Disk Drive) IBM i

The Power 770 and Power 780 support both 2.5-inch and 3.5-inch SAS SFF hard disks. The

3.5-inch DASD hard disk can be attached to the Power 770 and Power 780 but must be located in a feature FC 5886 EXP12S I/O drawer, whereas 2.5-inch DASD hard files can be mounted either internally or in the EXP24S SFF Gen2-bay Drawer (FC 5887).

If you need more disks than are available with the internal disk bays, you can attach additional external disk subsystems. For more detailed information about the available external disk subsystems, see 2.11, “External disk subsystems” on page 93.

22 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

SCSI disks are not supported in the Power 770 and 780 disk bays. However, if you want to use SCSI disks, you can attach existing SCSI disk subsystems.

The disk/media backplane feature FC 5652 provides six SFF disk slots and one SATA media slot. In a full configuration with four connected building blocks, the combined system supports up to four media devices with Media Enclosure and Backplane FC 5652. The SATA Slimline DVD-RAM drive is the only supported media device option. It was refreshed, and the feature code was changed from FC 5762 to FC 5771.

1.6 I/O drawers

The system has eight I/O expansion slots per enclosure, including two dedicated GX++ slots. If more PCI slots are needed, such as to extend the number of LPARs, up to 32 PCI-DDR 12X Expansion Drawers (FC 5796) and up to 16 12X I/O Drawer PCIe features (FC 5802 and FC 5877) can be attached.

The Power 770 and the Power 780 servers support the following 12X attached I/O drawers, providing extensive capability to expand the overall server expandability and connectivity:

򐂰 Feature FC 5802 provides PCIe slots and SFF SAS disk slots. 򐂰 Feature FC 5877 provides PCIe slots. 򐂰 Feature FC 5796 provides PCI-X slots. 򐂰 The 7314-G30 drawer provides PCI-X slots (supported, but no longer orderable).

Disk-only I/O drawers are also supported, providing large storage capacity and multiple partition support:

򐂰 Feature FC 5886 EXP12S holds a 3.5-inch SAS disk or SSD. 򐂰 Feature FC 5887 EXP 24S SFF Gen2-bay Drawer for high-density storage holds SAS

Hard Disk drives.

򐂰 The 7031-D24 holds a 3.5-inch SCSI disk (supported, but no longer orderable). 򐂰 The 7031-T24 holds a 3.5-inch SCSI disk (supported, but no longer orderable).

1.6.1 PCI-DDR 12X Expansion Drawers (FC 5796)

The PCI-DDR 12X Expansion Drawer (FC 5796) is a 4U tall (EIA units) drawer and mounts in a 19-inch rack. Feature FC 5796 takes up half the width of the 4U (EIA units) rack space. Feature FC 5796 requires the use of a FC 7314 drawer mounting enclosure. The 4U vertical enclosure can hold up to two FC 5796 drawers mounted side by side in the enclosure. A maximum of four FC 5796 drawers can be placed on the same 12X loop.

The I/O drawer has the following attributes:

򐂰 A 4U (EIA units) rack-mount enclosure (FC 7314) holding one or two FC 5796 drawers 򐂰 Six PCI-X DDR slots: 64-bit, 3.3 V, 266 MHz (blind-swap) 򐂰 Redundant hot-swappable power and cooling units

1.6.2 12X I/O Drawer PCIe (FC 5802 and FC 5877)

The FC 5802 and FC 5877 expansion units are 19-inch, rack-mountable, I/O expansion drawers that are designed to be attached to the system by using 12X double data rate (DDR) cables. The expansion units can accommodate 10 generation-3 cassettes. These cassettes can be installed and removed without removing the drawer from the rack.

Chapter 1. General description 23

A maximum of two FC 5802 drawers can be placed on the same 12X loop. Feature FC 5877 is the same as FC 5802, except it does not support disk bays. Feature FC 5877 can be on the same loop as FC 5802. Feature FC 5877 cannot be upgraded to FC 5802.

The I/O drawer has the following attributes:

򐂰 Eighteen SAS hot-swap SFF disk bays (only FC 5802) 򐂰 Ten PCI Express (PCIe) based I/O adapter slots (blind-swap) 򐂰 Redundant hot-swappable power and cooling units

Mixing: Mixing FC 5802 or 5877 and FC 5796 on the same loop is not supported.

1.6.3 EXP12S SAS Drawer

The EXP12S SAS Drawer (FC 5886) is a 2 EIA drawer and mounts in a 19-inch rack. The drawer can hold either SAS disk drives or SSD. The EXP12S SAS drawer has twelve 3.5-inch SAS disk bays with redundant data paths to each bay. The SAS disk drives or SSDs that are contained in the EXP12S are controlled by one or two PCIe or PCI-X SAS adapters that are connected to the EXP12S with SAS cables.

1.6.4 EXP 24S SFF Gen2-bay Drawer

The EXP24S SFF Gen2-bay Drawer is an expansion drawer that supports up to twenty-four

2.5-inch hot-swap SFF SAS HDDs on POWER6 or POWER7+ servers in 2U of 19-inch rack space. The EXP24S bays are controlled by SAS adapters/controllers attached to the I/O drawer by SAS X or Y cables.

The SFF bays of the EXP24S are different from the SFF bays of the POWER7+ system units or 12X PCIe I/O drawers (FC 5802 and FC 5803). The EXP24S uses Gen2 or SFF-2 SAS drives that physically do not fit in the Gen1 or SFF-1 bays of the POWER7+ system unit or 12X PCIe I/O Drawers, or vice versa.

1.6.5 EXP30 Ultra SSD I/O Drawer

The enhanced EXP30 Ultra SSD I/O Drawer (FC EDR1) provides the IBM Power POWER7+ 770 and 780 up to 30 solid-state drives (SSD) in only 1U of rack space without any PCIe slots. The drawer provides up to 480,000 IOPS and up to 12.6.2 TB of capacity for AIX or Linux clients. Plus up to 48 additional hard disk drives (HDDs) can be directly attached to the Ultra Drawer (still without using any PCIe slots) providing up to 43.2 TB additional capacity in only 4U additional rack space for AIX clients. This ultra-dense SSD option is similar to the Ultra Drawer (FC 5888), which remains available to the Power 710, 720, 730, and 740. The EXP30 attaches to the 770 or 780 server with a GX++ adapter, FC 1914.

1.6.6 I/O drawers and usable PCI slot

The I/O drawer model types can be intermixed on a single server within the appropriate I/O loop. Depending on the system configuration, the maximum number of I/O drawers that is supported differs.

24 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Table 1-11 summarizes the maximum number of I/O drawers supported and the total number of PCI slots that are available when expansion consists of a single drawer type.

Table 1-11 Maximum number of I/O drawers supported and total number of PCI slots

System drawers

1 drawer 8 4 48 6 0 46

2 drawers 16 8 96 12 0 92

3 drawers 24 12 144 18 0 138

4 drawers 32 16 192 24 0 184

Maximum FC 5796 drawers

Maximum FC 5802 and FC 5877 drawers

Table 1-12 summarizes the maximum number of disk-only I/O drawers supported.

Table 1-12 Maximum number of disk only I/O drawers supported

Server Maximum FC 5886 drawers Maximum FC 5887 drawers

Power 770 110 56

Power 780 110 56

1.7 Comparison between models

Total number of slots

FC 5796 FC 5802 and FC 5877

PCI-X PCIe PCI-X PCIe

The Power 770 offers configuration options, where the POWER7+ processor card can have one of two processor speeds installed. In either case, the processor card is populated with four single chip modules (SCMs). The card will contain one of the following processor configurations:

򐂰 Four 3-core SCMs running at 4.22 GHz 򐂰 Four 4-core SCMs running at 3.8 GHz.

Both of these Power 770 models are available starting as low as four active cores, and incrementing one core at a time through built-in CoD functions to a maximum of 48 active cores, with the 4.22 GHz processor or 64 active cores with the 3.8 GHz processor.

The Power 780 also offers a four-socket POWER7+ processor card with one of two processor configurations installed. These processor cards have the following specifications:

򐂰 Four 4-core SCMs running at 4.42 GHz 򐂰 Four 8-core SCMs running at 3.72 GHz

Both of these Power 780 models are available starting as low as four active cores, and incrementing one core at a time through built-in CoD functions to a maximum of 64 active cores, with the 4.42 GHz processor or 128 active cores with the 3.72 GHz processor.

Chapter 1. General description 25

Table 1-13 summarizes the processor core options and frequencies, and matches them to the L3 cache sizes for the Power 770 and Power 780.

Table 1-13 Summary of processor core counts, core frequencies, and L3 cache sizes

System Cores per

POWER7+ SCM

Frequency (GHz)

L3 cache

Enclosure summation

System maximum (cores)c

Power 770 3 4.22 30 MB 12-cores and

120 MB L3 cache

Power 770 4 3.8 40 MB 16-cores and

160 MB L3 cache

Power 780 4 4.42 40 MB 16-cores and

160 MB L3 cache

Power 780 8 3.72 80 MB 32-cores and

320 MB L3 cache

a. The total L3 cache available on the POWER7+ SCM, maintaining 10 MB per processor core b. The total number of processor cores and L3 cache within a populated enclosure c. The maximum number of cores with four CEC enclosures and all cores activated

1.8 Build to order

You can do a build to order (also called a la carte) configuration by using the IBM Configurator for e-business (e-config). With it, you specify each configuration feature that you want on the system.

This method is the only configuration method for the IBM Power 770 and Power 780 servers.

128

1.9 IBM editions

IBM edition offerings are not available for the IBM Power 770 and Power 780 servers.

1.10 Model upgrades

The following sections describe the various upgrades that are available.

1.10.1 Power 770

You can upgrade the 9117-MMA, 9117-MMB, or 9117-MMC with 9117-MMD processors. For upgrades from 9117-MMA, 9117-MMB, or 9117-MMC systems, IBM will install new CEC enclosures to replace your current CEC enclosure. The current CEC enclosures are returned to IBM in exchange for the financial consideration identified under the applicable feature conversions for each upgrade.

Clients taking advantage of the model upgrade offer from a 9117-MMA or 9117-MMB/MMC system are required to return all components of the serialized MT model that were not ordered through feature codes. Any feature for which a feature conversion is used to obtain a new part must be returned to IBM also. You may keep and reuse any features from the CEC enclosures that were not involved in a feature conversion transaction.

26 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

1.10.2 Power 780

You can upgrade the 9117-MMA, 9179-MHB or 9179-MHC with 9179-MHD processors. For upgrades from 9117-MMA, 9179-MHB, or 9179-MHC processor-based systems, IBM will install new CEC enclosures to replace the enclosures you currently have. Your current CEC enclosures are returned to IBM in exchange for the financial considerations that are identified under the applicable feature conversions for each upgrade.

Clients taking advantage of the model upgrade offer from 9117-MMA, 9179-MHB, or 9179-MHC processor-based system are required to return all components of the serialized MT model that were not ordered through feature codes. Any feature for which a feature conversion is used to obtain a new part must be returned to IBM also. You may keep and reuse any features from the CEC enclosures that were not involved in a feature conversion transaction.

Upgrade considerations

Feature conversions are set up for the following items:

򐂰 POWER6, IBM POWER6+™ and POWER7 processors to POWER7+ processors 򐂰 DDR2 memory DIMMS to DDR3 memory DIMMS 򐂰 New trim kits upgrading from 9117-MMA, 9117-MMB or 9179-MHB to 9179-MHD (existing

trim kits are only functional for one-drawer configurations or for racks holding only I/O and no Power 770 or 780 processor enclosures)

򐂰 PowerVM (Standard to Enterprise) 򐂰 Drawer/Bezel 򐂰 PCIe Crypto Gen3 򐂰 PCIx 1.5 GB RAID

The following features that are present on the current system can be moved to the new system:

򐂰 DDR3 memory DIMMs (FC 5600, FC 5601, FC 5602 and FC 5564) 򐂰 Active Memory Expansion Enablement (FC 4791) 򐂰 FSP/Clock Pass Through Card (FC 5665) 򐂰 Service Processor (FC 5664) 򐂰 175 MB Cache RAID - Dual IOA Enablement Card (FC 5662) 򐂰 Operator Panel (FC 1853) 򐂰 Disk/Media Backplane (FC 5652) 򐂰 PCIe adapters with cables, line cords, keyboards, and displays 򐂰 PowerVM Standard edition (FC 7942) or PowerVm Enterprise edition (FC 7995) 򐂰 I/O drawers (FC 5786, FC 5796, FC 5802, FC 5877, and FC 5886) 򐂰 Racks (FC 0551, FC 0553, and FC 0555) 򐂰 Doors (FC 6068, FC 6069, FC 6248, FC 6249, and FC 6858) 򐂰 SATA DVD-RAM (FC 5762)

The Power 770 and Power 780 can support the following drawers:

򐂰 FC 5802 and FC 5877 PCIe 12X I/O drawers 򐂰 FC 5797 and FC 7413-G30 PCI-X (12X) I/O Drawer 򐂰 FC 5786 and FC 7031-D24 TotalStorage EXP24 SCSI Disk Drawer 򐂰 FC 5886 EXP12S SAS Disk Drawer 򐂰 FC EDR1 EXP30 Ultra SSD I/O Drawer

Chapter 1. General description 27

The Power 770 and Power 780 support only the SAS DASD SFF hard disks, internally. The existing 3.5-inch DASD hard disks can be attached to Power 770 and Power 780, but must be located in an I/O drawer such as FC 5886.

For POWER6, POWER6+, or POWER7 processor-based systems that have the On/Off CoD function enabled, you must reorder the on/off enablement features (FC 7951and FC 7954) when placing the upgrade MES order for the new Power 770 or 780 system to keep the On/Off CoD function active. To initiate the model upgrade, the on/off enablement features should be removed from the configuration file before the MES order is started. Any temporary use of processors or memory owed to IBM on the existing system must be paid before installing the Power 770 model MMD or Power 780 model MHD.

Features FC 8FC 8018 and FC 8030 are available to support migration of the PowerVM features FC 7942 or FC 7995 during the initial order and build of the MMD or MHD upgrade MES order. Customers can add feature FC 8018 or FC 8030 to their upgrade orders in a quantity not to exceed the quantity of feature FC 7942 or FC 7995, obtained for the system being upgraded. Feature FC 7942 or FC 7995 must be migrated to the new configuration report in a quantity that equals feature FC 8018 or FC 8030. Additional FC 7942 or FC 7995 features can be ordered during the upgrade.

Clients can add feature FC 8018 to their upgrade orders in a quantity not to exceed the quantity of feature FC 7942, obtained for the system being upgraded. Feature FC 7942 must be migrated to the new configuration report in a quantity that equals feature FC 8018. Additional FC 7942 features can be ordered during the upgrade.

Features 8527 and 8528 are available to support migration of DDR3 memory activations 1812 or 1813 during the initial order and build of the MHD upgrade MES order. You can add features 8527 and 8528 to your upgrade orders in a quantity not to exceed the quantity of feature 1812 or 1813 obtained for the system being upgraded. The 1812 or 1813 features should be migrated to the new configuration report in a quantity that equals feature 1812 and

1813. Additional 1812 or 1813 features can be ordered during the upgrade.

1.11 Management consoles

This section discusses the supported management interfaces for the servers.

1.11.1 HMC models

The Hardware Management Console (HMC) is required for managing the IBM Power 770 and Power 780. It has a set of functions that are necessary to manage the system:

򐂰 Creating and maintaining a multiple partition environment 򐂰 Displaying a virtual operating system session terminal for each partition 򐂰 Displaying a virtual operator panel of contents for each partition 򐂰 Detecting, reporting, and storing changes in hardware conditions 򐂰 Powering managed systems on and off 򐂰 Acting as a service focal point for service representatives to determine an appropriate

service strategy

In 2012, IBM announced a new HMC model, machine type 7042-CR7. Hardware features on the CR7 model include a second HDD (FC 1998) for RAID 1 data mirroring, and the option of a redundant power supply. At the time of writing, the latest version of HMC code was V7.6.0.

28 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

This code level is required for new LPAR function support, which allows the HMC to manage more LPARs per processor core; a core can now be partitioned in up to 20 LPARs (0.05 of a core).

The IBM Power 770 and Power 780 are not supported by the Integrated Virtualization Manager (IVM).

Several HMC models are supported to manage POWER7+ based systems. Two models (7042-CR6 and 7042-CR7) are available for ordering at the time of writing, but you can also use one of the withdrawn models listed in Table 1-14.

Table 1-14 HMC models supporting POWER7+ processor technology-based servers

Type-model Availability Description

7310-C05 Withdrawn IBM 7310 Model C05 Desktop Hardware Management Console

7310-C06 Withdrawn IBM 7310 Model C06 Deskside Hardware Management Console

7042-C06 Withdrawn IBM 7042 Model C06 Deskside Hardware Management Console

7042-C07 Withdrawn IBM 7042 Model C07 Deskside Hardware Management Console

7042-C08 Withdrawn IBM 7042 Model C08 Deskside Hardware Management Console

7310-CR3 Withdrawn IBM 7310 Model CR3 Rack-Mounted Hardware Management Console

7042-CR4 Withdrawn IBM 7042 Model CR4 Rack-Mounted Hardware Management Console

7042-CR5 Withdrawn IBM 7042 Model CR5 Rack-Mounted Hardware Management Console

7042-CR6 Withdrawn IBM 7042 Model CR6 Rack mounted Hardware Management Console

7042-CR7 Available IBM 7042 Model CR7 Rack mounted Hardware Management Console

At the time of writing, base Licensed Machine Code Version 7 Revision 7.6.0 or later is required to support the Power 770(9117-MMD) and Power 780(9179-MHD).

Fix Central: You can download or order the latest HMC code from the Fix Central website:

http://www.ibm.com/support/fixcentral

Existing HMC models 7310 can be upgraded to Licensed Machine Code Version 7 to support environments that might include POWER5, POWER5+, POWER6, POWER6+, and POWER7 and POWER7+ processor-based servers. Licensed Machine Code Version 6 (FC 0961) is not available for 7042 HMCs.

If you want to support more than 254 partitions in total, the HMC might require a memory upgrade to 4 GB.

1.11.2 IBM SDMC

IBM withdrew the SDMC product. Customers should migrate to the HMC platform by upgrading their 7042_CR6 server to the latest HMC code. See IBM Power Systems: SDMC to HMC Migration Guide (RAID1), REDP-4872.

Chapter 1. General description 29

1.12 System racks

The Power 770 and its I/O drawers are designed to be mounted in the following existing IBM racks: 7014-T00, 7014-T42, 7014-B42, 7014-S25, FC 0551, FC 0553, or FC 0555.

In addition, a 42U slim rack is now available: the 7953-94Y (FC ER05). The Power 780 and I/O drawers can be ordered only with the 7014-T00, 7014-T42 or 7953-94Y racks. These are built to the 19-inch EIA standard. An existing 7014-T00, 7014-B42, 7014-S25, 7014-T42, FC 0551, FC 0553, or FC 0555 rack can be used for the Power 770 and Power 780 if sufficient space and power are available.

The 36U (1.8-meter) rack (FC 0551) and the 42U (2.0-meter) rack (FC 0553) are available for order only on MES upgrade orders. For initial system orders, the racks must be ordered as machine type 7014, models T00, B42, S25, or T42; or machine type 7953, model 94Y.

If a system is to be installed in a rack or cabinet that is not IBM, it must meet requirements.

Responsibility: The client is responsible for ensuring that the installation of the drawer in the preferred rack or cabinet results in a configuration that is stable, serviceable, safe, and compatible with the drawer requirements for power, cooling, cable management, weight, and rail security.

1.12.1 IBM 7014 model T00 rack

The 1.8-meter (71-inch) model T00 is compatible with past and present IBM Power systems. The features of the T00 rack are as follows:

򐂰 It has 36U (EIA units) of usable space. 򐂰 It has optional removable side panels. 򐂰 It has an optional highly perforated front door. 򐂰 It has optional side-to-side mounting hardware for joining multiple racks. 򐂰 It has standard business black or optional white color in OEM format. 򐂰 It has increased power distribution and weight capacity. 򐂰 It supports both AC and DC configurations. 򐂰 The rack height is increased to 1926 mm (75.8 in.) if a power distribution panel is fixed to

the top of the rack.

򐂰 Up to four power distribution units (PDUs) can be mounted in the PDU bays (see

Figure 1-5 on page 33), but others can fit inside the rack. See 1.12.8, “The AC power distribution unit and rack content” on page 32.

򐂰 Weights are as follows:

– T00 base empty rack: 244 kg (535 lb) – T00 full rack: 816 kg (1795 lb) – Maximum Weight of Drawers is 572 kg (1260 lb) – Maximum Weight of Drawers in a zone 4 earthquake environment is 490 kg (1080 lb).

This equates to 13.6 kg (30 lb)/EIA.

30 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Important: If additional weight is added to the top of the rack, for example add feature code 6117, the 490 kg (1080 lb) must be reduced by the weight of the addition. As an example, feature code 6117 weighs approximately 45 kg (100 lb) so the new Maximum Weight of Drawers the rack can support in a zone 4 earthquake environment is 445 kg (980 lb). In the zone 4 earthquake environment the rack should be configured starting with the heavier drawers at the bottom of the rack.

1.12.2 IBM 7014 model T42 rack

The 2.0-meter (79.3-inch) Model T42 addresses the client requirement for a tall enclosure to house the maximum amount of equipment in the smallest possible floor space. The following features differ in the model T42 rack from the model T00:

򐂰 The T42 rack has 42U (EIA units) of usable space (6U of additional space). 򐂰 The model T42 supports AC power only. 򐂰 Weights are as follows:

– T42 base empty rack: 261 kg (575 lb) – T42 full rack: 930 kg (2045 lb)

򐂰 The FC ERG7 feature provides an attractive black full height rack door. The door is steel,

with a perforated flat front surface. The perforation pattern extends from the bottom to the top of the door to enhance ventilation and provide some visibility into the rack.

High end: A special door (FC 6250) and side panels (FC 6238) are available to make the rack appear as a high-end server (but in a 19-inch rack format instead of a 24-inch rack).

1.12.3 IBM 7014 model S25 rack

The 1.3-meter (49-inch) model S25 rack has the following features:

򐂰 25U (EIA units) 򐂰 Weights:

– Base empty rack: 100.2 kg (221 lb) – Maximum load limit: 567.5 kg (1250 lb)

The S25 racks do not have vertical mounting space that accommodate FC 7188 PDUs. All PDUs required for application in these racks must be installed horizontally in the rear of the rack. Each horizontally mounted PDU occupies 1U of space in the rack, and therefore reduces the space available for mounting servers and other components.

S25 or B25 rack: The Power 780 cannot be ordered with a S25 or B25 rack.

1.12.4 IBM 7953 model 94Y rack

The 2.0-meter (79.3 inch) model 94Y rack has the following features:

򐂰 42U (EIA units) 򐂰 Weights:

– Base empty rack: 187 kg (221 lb) – Maximum load limit: 664 kg (1460 lb)

Chapter 1. General description 31

The IBM 42U Slim Rack (7953-94Y) differs from the IBM 42U enterprise rack (7014-B42 or 7014-T42) in several aspects. Both provide 42U of vertical space, are 1100 mm deep, and have an interior rail-to-rail depth of 715 mm. However, the IBM 42U Slim Rack is 600 mm wide; the B42/T42 is 645 mm wide with side covers. For clients with 2-foot floor tiles, the extra 45 mm (1.77-inch) width of the enterprise rack can sometimes cause challenges when cutting holes in the floor tiles for cabling.

The 42U Slim Rack has a lockable perforated front steel door, providing ventilation, physical security, and visibility of indicator lights in the installed equipment within. In the rear, either a lockable perforated rear steel door (FC EC02) or a lockable rear door heat exchanger (RDHX; 1164-95X) is used. Lockable optional side panels (FC EC03) increase the rack’s aesthetics, help control airflow through the rack, and provide physical security. Multiple 42U Slim Racks can be bolted together to create a rack suite (indicate feature EC04).

Up to six optional 1U PDUs can be placed vertically in the sides of the rack. Additional PDUs can be located horizontally, but they each will use 1U of space in this position.

1.12.5 Feature code 0555 rack

The 1.3-meter rack (FC 0555) is a 25U (EIA units) rack. The rack that is delivered as FC 0555 is the same rack delivered when you order the 7014-S25 rack. The included features might differ. The FC 0555 is supported, but it is no longer orderable.

1.12.6 Feature code 0551 rack

The 1.8-meter rack (FC 0551) is a 36U (EIA units) rack. The rack that is delivered as FC 0551 is the same rack delivered when you order the 7014-T00 rack. The included features might differ. Several features that are delivered as part of the 7014-T00 must be ordered separately with the FC 0551.

1.12.7 Feature code 0553 rack

The 2.0-meter rack (FC 0553) is a 42U (EIA units) rack. The rack that is delivered as FC 0553 is the same rack delivered when you order the 7014-T42 or B42 rack. The included features might differ. Several features that are delivered as part of the 7014-T42 or B42 must be ordered separately with the FC 0553.

1.12.8 The AC power distribution unit and rack content

For rack models T00, T42 and the slim 94Y, 12-outlet PDUs are available. These include PDUs Universal UTG0247 Connector (FC 9FC 9188 and FC 7188) and Intelligent PDU+ Universal UTG0247 Connector (FC 7109).

Four PDUs can be mounted vertically in the back of the T00 and T42 racks. Six PDUs can be mounted vertically in the 94Y. Figure 1-5 on page 33 shows the placement of the four vertically mounted PDUs. In the rear of the rack, two additional PDUs can be installed horizontally in the T00 rack and three in the T42 rack. The four vertical mounting locations will be filled first in the T00 and T42 racks. Mounting PDUs horizontally consumes 1U per PDU and reduces the space available for other racked components. When mounting PDUs horizontally, use fillers in the EIA units occupied by these PDUs to facilitate proper air flow and ventilation in the rack.

32 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Rack Rear View

Circuit breaker reset

Status LED

Figure 1-5 PDU placement and PDU view

For the Power 770 and Power 780 installed in IBM 7014 or FC 055x racks, the following PDU rules apply:

򐂰 For PDU FC 7188 and FC 7109 when using power cord FC 6654, FC 6655, FC 6656,

FC 6657, or FC 6658, each pair of PDUs can power up to two Power 770 and Power 780 CEC enclosures.

򐂰 For PDU FC 7188 and FC 7109 when using power cord FC 6489, 6491, FC 6492, or FC

6653, each pair of PDUs can power up to 4-5 Power 770 and Power 780 CEC enclosures.

For detailed power cord requirements and power cord feature codes, see the IBM Power Systems Hardware Information Center website:

http://publib.boulder.ibm.com/infocenter/systems/scope/hw/index.jsp

Power cord: Ensure that the appropriate power cord feature is configured to support the power being supplied.

The Base/Side Mount Universal PDU (FC 9188) feature, the optional and additional Universal PDU (FC 7188) feature, and the Intelligent PDU+ options (FC 7109) support a wide range of country requirements and electrical power specifications. The PDU receives power through a UTG0247 power line connector. Each PDU requires one PDU-to-wall power cord. Various power cord features are available for countries and applications by varying the PDU-to-wall power cord, which must be ordered separately. Each power cord provides the unique design characteristics for the specific power requirements. To match new power requirements and

Chapter 1. General description 33

save previous investments, these power cords can be requested with an initial order of the rack or with a later upgrade of the rack features.

The PDU has 12 client-usable IEC 320-C13 outlets. There are six groups of two outlets fed by six circuit breakers. Each outlet is rated up to 10 amps, but each group of two outlets is fed from one 15-amp circuit breaker.

The Universal PDUs are compatible with previous models.

Power cord and PDU: Based on the power cord that is used, the PDU can supply a range of 4.8 - 19.2 kVA. The total kilovolt ampere (kVA) of all the drawers that are plugged into the PDU must not exceed the power cord limitation.

Each system drawer to be mounted in the rack requires two power cords, which are not included in the base order. For maximum availability, be sure to connect power cords from the same system to two separate PDUs in the rack, and to connect each PDU to independent power sources.

1.12.9 Rack-mounting rules

The system consists of one to four CEC enclosures. Each enclosure occupies 4U of vertical rack space. When mounting the system into a rack account for the following primary considerations:

򐂰 For configurations with two, three, or four drawers, all drawers must be installed together in

the same rack, in a contiguous space of 8U, 12U, or 16U within the rack. The uppermost enclosure in the system is the base enclosure. This enclosure will contain the active service processor and the operator panel. If a second CEC enclosure is part of the system, the backup service processor is contained in the second CEC enclosure.

򐂰 Model 7014-T42, 7014-B42, or FC 0553 rack is constructed with a small flange at the

bottom of EIA location 37. When a system is installed near the top of 7014-T42, 7014-B42, or FC 0553 rack, no system drawer can be installed in EIA positions 34, 35, or 36. This approach is to avoid interference with the front bezel or with the front flex cable, depending on the system configuration. A two-drawer system cannot be installed above position 29. A three-drawer system cannot be installed above position 25. A four-drawer system cannot be installed above position 21. (The position number refers to the bottom of the lowest drawer.)

򐂰 When a system is installed in a model 7014-T00, 7014-T42, 7014-B42, FC 0551, or

FC 0553 rack that has no front door, a Thin Profile Front Trim Kit must be ordered for the rack. The required trim kit for the 7014-T00 or FC 0551 rack is FC 6263. The required trim kit for the 7014-T42, 7014-B42, or FC 0553 rack is FC 6272. When upgrading from a 9117-MMA, trim kits FC 6263 or FC 6272 can be used for one drawer enclosures only.

򐂰 The design of the Power 770 and Power 780 is optimized for use in a 7014-T00,

7014-T42, -7014B42, -S25, FC 0551, or FC 0553 rack. Both the front cover and the processor flex cables occupy space on the front left side of an IBM 7014, FC 0551, and FC 0553 rack that might not be available in typical non-IBM racks.

򐂰 Acoustic door features are available with the 7014-T00, 7014-B42, 7014-T42, FC 0551,

and FC 0553 racks to meet the lower acoustic levels identified in the specification section of this document. The acoustic door feature can be ordered on new T00, B42, T42, FC 0551, and FC 0553 racks or ordered for the T00, B42, T42, FC 0551, and FC 0553 racks that you already own.

34 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

1.12.10 Useful rack additions

This section highlights several solutions for IBM Power Systems rack-based systems.

IBM 7214 Model 1U2 SAS Storage Enclosure

The IBM System Storage® 7214 Tape and DVD Enclosure Express is designed to mount in one EIA unit of a standard IBM Power Systems 19-inch rack and can be configured with one or two tape drives, or either one or two Slim DVD-RAM or DVD-ROM drives in the right-side bay.

The two bays of the 7214 Express can accommodate the following tape or DVD drives for IBM Power servers:

򐂰 DAT72 36 GB Tape Drive: Up to two drives 򐂰 DAT72 36 GB Tape Drive: Up to two drives 򐂰 DAT160 80 GB Tape Drive: Up to two drives 򐂰 LTO Ultrium 4 Half-High 800 GB Tape Drive: Up to two drives 򐂰 DVD-RAM Optical Drive: Up to two drives 򐂰 DVD-ROM Optical Drive: Up to two drives

IBM System Storage 7214 Tape and DVD Enclosure

The IBM System Storage 7214 Tape and DVD Enclosure is designed to mount in one EIA unit of a standard IBM Power Systems 19-inch rack and can be configured with one or two tape drives, or either one or two Slim DVD-RAM or DVD-ROM drives in the right-side bay.

The two bays of the IBM System Storage 7214 Tape and DVD Enclosure can accommodate the following tape or DVD drives for IBM Power servers:

IBM System Storage 7216 Multi-Media Enclosure

The IBM System Storage 7216 Multi-Media Enclosure (Model 1U2) is designed to attach to the Power 770 and the Power 780 through a USB port on the server or through a PCIe SAS adapter. The 7216 has two bays to accommodate external tape, removable disk drive, or DVD-RAM drive options.

The following optional drive technologies are available for the 7216-1U2:

򐂰 DAT160 80 GB SAS Tape Drive (FC 5619 򐂰 DAT320 160 GB SAS Tape Drive (FC 1402) 򐂰 DAT320 160 GB USB Tape Drive (FC 5673) 򐂰 LTO Ultrium 5 Half-High 1.5 TB SAS Tape Drive (FC 8247) 򐂰 DVD-RAM - 9.4 GB SAS Slim Optical Drive (FC 1420 and FC 1422) 򐂰 RDX Removable Disk Drive Docking Station (FC 1103)

Unavailable: The DAT320 160 GB SAS Tape Drive (FC 1402) and the DAT320 160 GB USB Tape Drive (FC 5673) are no longer available as of July 15, 2011.

Chapter 1. General description 35

To attach a 7216 Multi-Media Enclosure to the Power 770 and Power 780, consider the following cabling procedures:

򐂰 Attachment by an SAS adapter

A PCIe Dual-X4 SAS adapter (FC 5901) or a PCIe LP 2-x4-port SAS Adapter 3 Gb (FC

5278) must be installed in the Power 770 and Power 780 server to attach to a 7216 Model 1U2 Multi-Media Storage Enclosure. Attaching a 7216 to a Power 770 and Power 780 through the integrated SAS adapter is not supported.

For each SAS tape drive and DVD-RAM drive feature installed in the 7216, the appropriate external SAS cable will be included.

An optional Quad External SAS cable is available by specifying (FC 5544) with each 7216 order. The Quad External Cable allows up to four 7216 SAS tape or DVD-RAM features to attach to a single System SAS adapter.

Up to two 7216 storage enclosure SAS features can be attached per PCIe Dual-X4 SAS adapter (FC 5901) or the PCIe LP 2-x4-port SAS Adapter 3 Gb (FC 5278).

򐂰 Attachment by a USB adapter

The Removable RDX HDD Docking Station features on 7216 only support the USB cable that is provided as part of the feature code. Additional USB hubs, add-on USB cables, or USB cable extenders are not supported.

For each RDX Docking Station feature installed in the 7216, the appropriate external USB cable will be included. The 7216 RDX Docking Station feature can be connected to the external, integrated USB ports on the Power 770 and Power 780 or to the USB ports on 4-Port USB PCI Express Adapter (FC 2728).

The 7216 DAT320 USB tape drive or RDX Docking Station features can be connected to the external, integrated USB ports on the Power 770 and Power 780.

The two drive slots of the 7216 enclosure can hold the following drive combinations:

򐂰 One tape drive (DAT160 SAS or LTO Ultrium 5 Half-High SAS) with second bay empty 򐂰 Two tape drives (DAT160 SAS or LTO Ultrium 5 Half-High SAS) in any combination 򐂰 One tape drive (DAT160 SAS or LTO Ultrium 5 Half-High SAS) and one DVD-RAM SAS

drive sled with one or two DVD-RAM SAS drives

򐂰 Up to four DVD-RAM drives 򐂰 One tape drive (DAT160 SAS or LTO Ultrium 5 Half-High SAS) in one bay, and one RDX

Removable HDD Docking Station in the other drive bay

򐂰 One RDX Removable HDD Docking Station and one DVD-RAM SAS drive sled with one

or two DVD-RAM SAS drives in the bay on the right

򐂰 Two RDX Removable HDD Docking Stations

36 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Figure 1-6 shows the 7216 Multi-Media Enclosure.

Figure 1-6

FC 7216 Multi-Media Enclosure

In general, the 7216-1U2 is supported by the AIX, IBM i, and Linux operating systems. IBM i, from Version 7.1, now fully supports the internal 5.25 inch RDX SATA removable HDD docking station, including boot support (no VIOS support). This support provides a fast, robust, high-performance alternative to tape backup/restore devices.

IBM System Storage 7226 Model 1U3 Multi-Media Enclosure

IBM System Storage 7226 Model 1U3 Multi-Media Enclosure can accommodate up to two tape drives, two RDX removable disk drive docking stations, or up to four DVD RAM drives. The 7226 offers SAS, USB, and FC electronic interface drive options. The 7226 Storage Enclosure delivers external tape, removable disk drive, and DVD RAM drive options that allow data transfer within similar system archival storage and retrieval technologies installed in existing IT facilities. The 7226 offers an expansive list of drive feature options.

IBM 7226 Multi-Media Enclosure options are as follows: 򐂰 DAT160 80 GB Tape Drives: With SAS or USB interface options and a data transfer rate of

up to 24 MBps, the DAT160 drive is read-write compatible with DAT160, DAT72, and DDS4 data cartridges.

򐂰 LTO Ultrium 5 Half-High 1.5 TB SAS and FC Tape Drive: With data transfer rates of up to

280 MBps, HHLT05 is read-write compatible with LTO Ultrium 5 and LTO Ultrium 4 data cartridges, and read-only compatible with Ultrium 3 data cartridges.

򐂰 DVD-RAM: 9.4 GB SAS Slim Optical Drive with SAS and USB interface option is

compatible with most standard DVD disks.

򐂰 RDX removable disk drives: The RDX USB docking station is compatible with most RDX

removable disk drive cartridges when used in the same operating system. The 7226 offers the following RDX removable drive capacity options:

– 320 GB – 500 GB – 1.0 TB

Chapter 1. General description 37

Removable RDX drives are in a rugged cartridge that inserts in a RDX removable (USB) disk docking station (FC 1103). RDX drives are compatible with docking stations installed internally in IBM POWER6 and POWER7 servers.

Media used in the 7226 DAT160 SAS and USB tape drive features are compatible with DAT160 tape drives installed internally in IBM POWER6 and POWER7 servers, and in IBM BladeCenter® systems.

Media used in LTO Ultrium 5 Half-High 1.5 TB tape drives is compatible with HHLTO5 tape drives installed in the IBM TS2250 and TS2350 external tape drives, IBM LTO5 tape libraries, and HHLTO5 tape drives installed internally in IBM POWER6 and IBM POWER7 servers.

The 7226 offers customer-replaceable unit (CRU) maintenance service to help make installation or replacement of new drives efficient. Other 7226 components are also designed for CRU maintenance.

The IBM System Storage 7226 Multi-Media Enclosure is compatible with most IBM POWER6 and POWER7 systems, and also IBM BladeCenter models (PS700, PS701, PS702, PS703, and PS704) that offer current level AIX, IBM i, and LINUX operating systems.

The IBM i operating system does not support 7226 USB devices.

For a complete list of host software versions and release levels that support the 7226, see the following System Storage Interoperation Center (SSIC) website:

http://www.ibm.com/systems/support/storage/config/ssic/index.jsp

Flat panel display options

The IBM 7316 Model TF3 is a rack-mountable flat panel console kit consisting of a 17-inch

337.9 mm x 270.3 mm flat panel color monitor, rack keyboard tray, IBM Travel Keyboard, support for IBM keyboard, video, mouse (KVM) switches, and language support. The IBM 7316-TF3 Flat Panel Console Kit offers the following features:

򐂰 Slim, sleek, lightweight monitor design that occupies only 1U (1.75 inches) in a 19-inch

standard rack

򐂰 A 17-inch, flat screen TFT monitor with truly accurate images and virtually no distortion 򐂰 The ability to mount the IBM Travel Keyboard in the 7316-TF3 rack keyboard tray 򐂰 Support for IBM keyboard/video/mouse (KVM) switches that provide control of as many as

128 servers, and support of both USB and PS/2 server-side keyboard and mouse connections

38 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Chapter 2. Architecture and technical

overview

The IBM Power 770 offers a 4-socket CEC enclosure, populated with 3-core or 4-core POWER7+ processors. A fully-configured 4-drawer system has either 48 or 64 cores, depending on which processor is specified.

The IBM Power 780 offers the same 4-socket CEC enclosure, populated with 4-core or 8-core POWER7+ processors. This architecture offers a maximum system configuration of 64 or 128 cores, depending on the processor option chosen.

This chapter provides an overview of the system architecture and its major components. The bandwidths that are provided are theoretical maximums used for reference.

The speeds shown are at an individual component level. Multiple components and application implementation are key to achieving the best performance.

Always do the performance sizing at the application workload environment level and evaluate performance by using real-world performance measurements and production workloads.

Figure 2-1 shows the logical system diagram of the 4-socket Power 770 and Power 780.

DASD Backplane

Op Panel RAID card

Write Cache

Write

SATA

Media

usb2

EXP

GX++ Card

P710C

GX++ Card

I/O Drawer

Six PCI-e Gen2 8x Slots

PCI-e Gen2 8x

CI-

e G

I-

e G

GX ++ Buses

PCI-e Slot

Passthru

Card

(Drawers 3 & 4)

Passthru A Conn

I/O Backplane

GX ++ Buses

4 socket CPU Card

Mid-Plane

PSI

8 SN DIMMs

WXZ Buses

P7IOC

(A)

GX++ Slot

2 HMC

2 SPCN

D2D Conns

FSP1 Card

(Drawers 1 & 2)

TPMD

DRAM

B Buses

P7+

B Buses

8 SN DIMMs

DRAM

P7+

GX++ Slot

SAS

Controller

SAS

Controller

SAS

Controller

Anchor

Card

DASD

PCI-X 32

Serial

On-board Ethernet Card

Ext SAS

BE3

Ethernet

Controller

2x 10 Gb 2x 1 Gb Eth

2x USB

P7IOC

(B)

PLX

Serial

PLX

Figure 2-1 IBM Power 770 or 780 logical system diagram

40 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.1 The IBM POWER7+ processor

The IBM POWER7+ processor represents a leap forward in technology achievement and associated computing capability. The multi-core architecture of the POWER7+ processor has been matched with innovation across a wide range of related technologies to deliver leading throughput, efficiency, scalability, and reliability, availability, and serviceability (RAS).

Although the processor is an important component in delivering outstanding servers, many elements and facilities must be balanced on a server to deliver maximum throughput. As with previous generations of systems based on POWER processors, the design philosophy for POWER7+ processor-based systems is one of system-wide balance in which the POWER7+ processor plays an important role.

In many cases, IBM is innovative to achieve required levels of throughput and bandwidth. Areas of innovation for the POWER7+ processor and POWER7+ processor-based systems include (but are not limited to) the following items:

򐂰 On-chip L3 cache implemented in embedded dynamic random access memory (eDRAM) 򐂰 Cache hierarchy and component innovation 򐂰 Advances in memory subsystem 򐂰 Advances in off-chip signaling 򐂰 Advances in I/O cards throughput and latency 򐂰 Advances in RAS features, as power-on reset and L3 cache dynamic column repair

The superscalar POWER7+ processor design also provides a variety of other capabilities:

򐂰 Binary compatibility with the prior generation of POWER processors 򐂰 Support for PowerVM virtualization capabilities, including PowerVM Live Partition Mobility

to and from POWER6 and POWER6+ processor-based systems

Chapter 2. Architecture and technical overview 41

Figure 2-2 shows the POWER7+ processor die layout, with the major areas identified:

򐂰 Processor cores 򐂰 L2 cache 򐂰 L3 cache and chip interconnection 򐂰 Simultaneous multiprocessing links 򐂰 Memory controllers. 򐂰 I/O links

Figure 2-2 POWER7+ processor die with key areas indicated

2.1.1 POWER7+ processor overview

The POWER7+ processor chip is fabricated with IBM 32 nm Silicon-On-Insulator (SOI) technology using copper interconnect, and implements an on-chip L3 cache using eDRAM.

The POWER7+ processor chip is 567 mm (transistors). Up to eight processor cores are on the chip, each with 12 execution units, 256 KB of L2 cache per core, and access to up to 80 MB of shared on-chip L3 cache per SCM.

For memory access, the POWER7+ processor includes one double data rate 3 (DDR3) memory controllers, each with four memory channels. To be able to scale effectively, the POWER7+ processor uses a combination of local and global SMP links with high coherency bandwidth and takes advantage of the IBM dual-scope broadcast coherence protocol.

42 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

and is built by using 2.1 billion components

Table 2-1 summarizes the technology characteristics of the POWER7+ processor.

Table 2-1 Summary of POWER7+ processor technology

Technology POWER7+ processor

Die size 567 mm

Fabrication technology 򐂰 32 nm lithography

Processor cores 3, 4, or 8

Maximum execution threads core/chip 4/32

Maximum L2 cache core/chip 256 KB/2 MB

Maximum On-chip L3 cache core/chip 10 MB/80 MB

DDR3 memory controllers 1

SMP design-point 32 sockets with IBM POWER7+ processors

Compatibility With prior generation of POWER processor

2.1.2 POWER7+ processor core

򐂰 Copper interconnect 򐂰 Silicon-on-Insulator 򐂰 eDRAM

Each POWER7+ processor core implements aggressive out-of-order (OoO) instruction execution to drive high efficiency in the use of available execution paths. The POWER7+ processor has an Instruction Sequence Unit that is capable of dispatching up to six instructions per cycle to a set of queues. Up to eight instructions per cycle can be issued to the instruction execution units. The POWER7+ processor has a set of 12 execution units:

򐂰 Two fixed point units 򐂰 Two load store units 򐂰 Four double precision floating point units 򐂰 One vector unit 򐂰 One branch unit 򐂰 One condition register unit 򐂰 One decimal floating point unit

The following caches are tightly coupled to each POWER7+ processor core:

򐂰 Instruction cache: 32 KB 򐂰 Data cache: 32 KB 򐂰 L2 cache: 256 KB, implemented in fast SRAM

Chapter 2. Architecture and technical overview 43

2.1.3 Simultaneous multithreading

2004 2-way SMT

FX0 FX1 FP0 FP1 LS0 LS1 BRX CRL

1995 Single thread out of order

FX0 FX1 FP0 FP1 LS0 LS1 BRX CRL

1997 Hardware multi-thread

FX0 FX1 FP0 FP1 LS0 LS1 BRX CRL

2010 4-way SMT

Thread 1 ExecutingThread 0 Executing Thread 3 ExecutingThread 2 Executing

No Thread Executing

Multi-threading evolution

POWER7+ processors support SMT1, SMT2, and SMT4 modes to enable up to four instruction threads to execute simultaneously in each POWER7+ processor core. The processor supports the following instruction thread execution modes:

򐂰 SMT1: Single instruction execution thread per core 򐂰 SMT2: Two instruction execution threads per core 򐂰 SMT4: Four instruction execution threads per core

SMT4 mode enables the POWER7+ processor to maximize the throughput of the processor core by offering an increase in processor-core efficiency. SMT4 mode is the latest step in an evolution of multithreading technologies introduced by IBM. Figure 2-3 shows the evolution of simultaneous multithreading in the industry.

44 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Figure 2-3 Evolution of simultaneous multithreading

The various SMT modes offered by the POWER7+ processor allow flexibility, enabling users to select the threading technology that meets an aggregation of objectives such as performance, throughput, energy use, and workload enablement.

Intelligent Threads

The POWER7+ processor features Intelligent Threads that can vary based on the workload demand. The system either automatically selects (or the system administrator can manually select) whether a workload benefits from dedicating as much capability as possible to a single thread of work, or if the workload benefits more from having capability spread across two or four threads of work. With more threads, the POWER7+ processor can deliver more total capacity as more tasks are accomplished in parallel. With fewer threads, those workloads that need fast individual tasks can get the performance that they need for maximum benefit.

2.1.4 Memory access

Memory Channel Description

P7+

SCM

DIMMCPU

Downstream Link

13 + 2 + 1

Upstream Link

20 + 2 + 1

SN DIMM 1

SN DIMM 2

SN DIMM 3

SN DIMM 4

MEM Cntl 0

B C

Each POWER7+ processor chip has one DDR3 memory controller, which uses four memory channels to connect to its quad of DIMMs. Each channel operates at 1066 MHz and can address up to 64 GB of memory. Thus, each POWER7+ processor chip is capable of addressing up to 256 GB of memory. The whole system can address up to 4TB of total memory.

Figure 2-4 gives a simple overview of the POWER7+ processor memory access structure.

Figure 2-4 Overview of POWER7+ memory access structure

2.1.5 On-chip L3 cache innovation and Intelligent Cache

A breakthrough in material engineering and microprocessor fabrication enabled IBM to implement the L3 cache in eDRAM and place it on the POWER7+ processor die. L3 cache is critical to a balanced design, as is the ability to provide good signaling between the L3 cache and other elements of the hierarchy, such as the L2 cache or SMP interconnect.

The on-chip L3 cache is organized into separate areas with differing latency characteristics. Each processor core is associated with a Fast Local Region of L3 cache (FLR-L3) but also has access to other L3 cache regions as shared L3 cache. Additionally, each core can negotiate to use the FLR-L3 cache associated with another core, depending on reference patterns. Data can also be cloned to be stored in more than one core’s FLR-L3 cache, again depending on reference patterns. This processor to optimize the access to L3 cache lines and minimize overall cache latencies.

Intelligent Cache management enables the POWER7+

Chapter 2. Architecture and technical overview 45

Figure 2-5 shows the FLR-L3 cache regions for each of the cores on the POWER7+

Core Core Core Core

Core

L2 Cache

Core

L2 Cache

Core

L2 Cache

Core

L2 Cache

Mem Ctrl Mem Ctrl

L3 Cache and Chip Interconnect

Local SMP Links

Remote SMP + I/O Links

Fast local L3

Cache Region

Fast local L3

Cache Region

Fast local L3

Cache Region

Fast local L3

Cache Region

L2 Cache L2 Cache L2 Cache L2 Cache

Fast local L3

Cache Region

Fast local L3

Cache Region

Fast local L3

Cache Region

Fast local L3

Cache Region

processor die.

Figure 2-5 Fast local regions of L3 cache on the POWER7+ processor

The innovation of using eDRAM on the POWER7+ processor die is significant for several reasons:

򐂰 Latency improvement

A six-to-one latency improvement occurs by moving the L3 cache on-chip compared to L3 accesses on an external (on-ceramic) ASIC.

򐂰 Bandwidth improvement

A 2x bandwidth improvement occurs with on-chip interconnect. Frequency and bus sizes are increased to and from each core.

򐂰 No off-chip driver or receivers

Removing drivers or receivers from the L3 access path lowers interface requirements, conserves energy, and lowers latency.

򐂰 Small physical footprint

The performance of eDRAM when implemented on-chip is similar to conventional SRAM but requires far less physical space. IBM on-chip eDRAM uses only a third of the components used in conventional SRAM, which has a minimum of six transistors to implement a 1-bit memory cell.

򐂰 Low energy consumption

The on-chip eDRAM uses only 20% of the standby power of SRAM.

46 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.1.6 POWER7+ processor and Intelligent Energy

Energy consumption is an important area of focus for the design of the POWER7+ processor, which includes Intelligent Energy features that help to dynamically optimize energy usage and performance so that the best possible balance is maintained. Intelligent Energy features, such as EnergyScale, work with IBM Systems Director Active Energy Manager to dynamically optimize processor speed based on thermal conditions and system utilization.

2.1.7 Comparison of the POWER7+ and POWER6 processors

Table 2-2 shows comparable characteristics between the generations of POWER7+ and POWER6 processors.

Table 2-2 Comparison of technology for the POWER7+ processor and the prior generation

POWER7+ POWER7 POWER6+

Technology

Die size

Maximum cores

Maximum SMT threads per core

Maximum frequency

L2 Cache

L3 Cache

Memory support

I/O bus

Enhanced cache mode (TurboCore)

Sleep and nap mode

a. Not supported on the Power 770 and Power 780 4-socket systems. b. For more information about sleep and nap modes, see 2.14.1, “IBM EnergyScale technology”

on page 115.

32 nm 45 nm 65 nm

567 mm

882

4 threads 4 threads 2 threads

4.4 GHz 4.25 GHz 5.0 GHz

256 KB per core 256 KB per core 4 MB per core

10 MB of FLR-L3 cache per core with each core having access to the full 80 MB of L3 cache, on-chip eDRAM

DDR3 DDR3 DDR2

Two GX++ Two GX++ One GX++

No Yes

Both Both Nap only

567 mm

4MB or 8MB of FLR-L3 cache per core with each core having access to the full 32 MB of L3 cache, on-chip eDRAM

341 mm

32 MB off-chip eDRAM ASIC

Chapter 2. Architecture and technical overview 47

2.2 POWER7+ processor card

Connector to Mid plane

2x P7+ SCMs

2xP7+ SCMs

Ext Fabric Conn

Pluggable Regulator

Front

DDR 3 DIMMs

Pluggable Regulator

TPMD Card

IBM Power 770 and Power 780 servers are modular systems that are built with one to four CEC enclosures. The processor and memory subsystem in each CEC enclosure is contained on a single processor card. The processor card contains four processor sockets and 16 fully buffered DDR3 memory DIMMs.

2.2.1 Overview

The IBM Power 770 processor card is populated with 3-core or 4-core POWER7+ processors. This way enables a maximum system configuration of 64-cores, built from four CEC enclosures.

The IBM Power 780 shares the same 4-socket processor card. The 780 processor cards are populated with 4-core or 8-core POWER7+ processors, enabling a maximum system configuration of 128 cores.

Figure 2-6 illustrates the major components of the 4-socket processor card.

Figure 2-6 IBM Power 770 or 780 4-socket processor card

48 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.2.2 Processor interconnects

POWER7

chip

B Bus - 8 bytes

2 GX ++ Buses - 4 bytes each

WXZ Buses

8 bytes

4 Memory

channels

POWER7 Chip

M C

The POWER7+ processor uses one memory controller, MC0, to access four DIMMs. The processor uses two GX++ buses (Figure 2-7).

Figure 2-7 Processor interconnects on 4-socket processor card

Each POWER7+ SCM has two serial EPROMs that contain the module’s vital product data (VPD).

2.3 Memory subsystem

On the Power 770 and Power 780 servers, each enclosure houses 16 DDR3 DIMM slots. The DIMM cards for the Power 770 and Power 780 are 96 mm tall, fully buffered, and placed in one of the 16 DIMM slots on the processor card.

2.3.1 Fully buffered DIMM

Fully buffered DIMM technology is used to increase reliability, speed, and density of memory subsystems. Conventionally, data lines from the memory controllers have to be connected to the data lines in every DRAM module. This effect traditionally degrades either the memory access times or memory density. Fully buffered DIMMs overcome this effect by implementing an advanced buffer between the memory controllers and the DRAMs with two independent signaling interfaces. This technique decouples the DRAMs from the bus and memory controller interfaces, allowing efficient signaling between the buffer and the DRAM.

2.3.2 Memory placement rules

The minimum DDR3 memory capacity for the Power 770 and Power 780 systems is 64 GB of installed memory, of which 32 GB must be activated.

All the memory DIMMs for the Power 770 and Power 780 are capable of capacity upgrade on demand and must have a minimum of 50% of its physical capacity activated. For example, the minimum installed memory for both servers is 64 GB RAM, whereas they can have a minimum of 32 GB RAM active.

Chapter 2. Architecture and technical overview 49

Unsupported: DDR2 memory (used in POWER6 processor-based systems) is not

P3-C28

P3-C29

P3-C27

P3-C26

P3-C25

P3-C24

P3-C23

P3-C22

M0 - D SN DDR3 DIMM #16

M0 - C SN DDR3 DIMM #15

M0 - B SN DDR3 DIMM #14

M0 - A SN DDR3 DIMM #13

Regulator 8

TPMD Card

Regulator 7

Regulator 6

Regulator 5

M0 - D SN DDR3 DIMM #8

M0 - C SN DDR3 DIMM #7

M0 - B SN DDR3 DIMM #6

M0 - A SN DDR3 DIMM #5

M0 - D SN DDR3 DIMM #12

M0 - C SN DDR3 DIMM #11

M0 - B SN DDR3 DIMM #10

M0 - A SN DDR3 DIMM #9

M0 - D SN DDR3 DIMM #4

M0 - C SN DDR3 DIMM #3

M0 - B SN DDR3 DIMM #2

M0 - A SN DDR3 DIMM #1

Loc Code Conn Ref

P3-C1 J1A

P3-C2 J2A

P3-C3 J3A

P3-C4 J4A

P3-C5

FSI 3

FSI 5

FSI 13

FSI 15

FSI 6

FSI 4

FSI 16

FSI 14

P3-C6

P3-C7 J5A

P3-C8 J6A

P3-C9 J7A

P3-C10 J8A

P3-C11 J1B

P3-C12 J2B

P3-C13 J3B

P3-C14 J4B

P3-C15

P3-C16

P3-C17

P3-C18 J5B

P3-C19 J6B

P3-C20 J7B

P3-C21 J8B

Midplane Connector

Regulator 1

Regulator 2

Regulator 3

Regulator 4

P7+ CP2

SNFSI 0, 4 SNFSI 1, 5 SNFSI 2, 6 SNFSI 3, 7

P7+ CP1

FSI0 FSI1 MC0

SNFSI 0, 4 SNFSI 1, 5 SNFSI 2, 6 SNFSI 3, 7

P7+ CP3

MC0 FSI0 FSI1

SNFSI 0, 4 SNFSI 1, 5 SNFSI 2, 6 SNFSI 3, 7

P7+ CP0

SNFSI 0, 4 SNFSI 1, 5 SNFSI 2, 6 SNFSI 3, 7

FSI0 FSI1 MC0

MC0 FSI0 FSI1

supported in POWER7+ processor-based systems.

Figure 2-8 shows the physical memory DIMM topology for Power 770 and Power 780 with four P7+ single-chip-modules (SCMs).

Figure 2-8 Physical memory DIMM topology for the Power 780 with four SCMs

50 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

For the POWER 770 and POWER 780, 16 buffered DIMM slots are available: 򐂰 DIMM slots J1A to J4A are connected to the memory controller on POWER7+

processor CP0.

򐂰 DIMM slots J5A to J8A are connected to the memory controller on POWER7+

processor CP2.

򐂰 DIMM slots J1B to J4B are connected to the memory controller on POWER7+

processor CP3.

򐂰 DIMM slots J5B to J8B are connected to the memory controller on POWER7+

processor CP1.

The memory-placement rules are as follows: 򐂰 Plug sequence will always allow for memory mirroring (for example, no feature code needs

to be specified for memory mirroring). The highlighted (green) cells in the following tables indicate the Active Memory Mirroring (AMM) base configuration.

򐂰 DIMMs must be installed by 4x DIMMs at a time, referred to as a DIMM-quad. 򐂰 DIMM-quads must be homogeneous; only DIMMs of the same feature code (FC) or

custom card identification number (CCIN) are allowed on the same quad.

򐂰 Minimum requirement is two quads of identical memory (that is, the same feature

code/CCIN) per enclosure.

򐂰 A DIMM-quad is the minimum installable unit for subsequent upgrades. 򐂰 Although each drawer can have different capacity memory DIMMs, for maximum memory

performance, the total memory capacity on each memory controller should be equivalent.

򐂰 The DIMM-quad placement rules for a single enclosure are as follows:

– Quad 1: J1A, J2A, J5A, J6A (mandatory minimum for each enclosure) – Quad 2: J3A, J4A, J7A, J8A (mandatory minimum for each enclosure) – Quad 3: J1B, J2B, J5B, J6B – Quad 4: J3B, J4B, J7B, J8B

Table 2-3 shows the optimal placement of each DIMM-quad within a single enclosure system. The enclosure

must have at least two DIMM-quads installed in slots J1A, J2A, J5A, J6A, and

J5A, J6A, J7A, and J8A, as shown with the highlighted color.

Table 2-3 Optimum DIMM-quad placement for a single enclosure system

Enclosure 1

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q1 Q1 Q2 Q2 Q1 Q1 Q2 Q2 Q3 Q3 Q4 Q4 Q3 Q3 Q4 Q4

Quads Q1 and Q2 must be identical to each other. Note: For maximum memory performance, the total memory capacity on each memory controller must be equivalent.

When populating a multi-enclosure system with DIMM-quads, each enclosure must have at least two DIMM-quads installed in slots J1A, J2A, J5A, J6A, J3AB, J4A, J7A, and J8A. After the mandatory requirements and memory-plugging rules are followed, there is an optimal approach to populating the systems.

Chapter 2. Architecture and technical overview 51

Table 2-4 shows the optimal placement of each DIMM-quad within a dual-enclosure system. Each enclosure must have at least two DIMM-quads installed.

Table 2-4 Optimum DIMM-quad placement for a dual enclosure system

Enclosure 1

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q1 Q1 Q2 Q2 Q1 Q1 Q2 Q2 Q5 Q5 Q8 Q8 Q5 Q5 Q8 Q8

Enclosure 2

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q3 Q3 Q4 Q4 Q3 Q3 Q4 Q4 Q6 Q6 Q7 Q7 Q6 Q6 Q7 Q7

Quads Q1 and Q2 must be identical to each other. Quads Q3 and Q4 must be identical to each other. Note: For maximum memory performance, the total memory capacity on each memory controller must be equivalent.

52 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Table 2-5 shows the optimal placement of each DIMM-quad within a three-enclosure system. Each enclosure

Table 2-5 Optimum DIMM-quad placement for a three-enclosure system

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory Controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q1 Q1 Q2 Q2 Q1 Q1 Q2 Q2 Q7 Q7 Q12 Q12 Q7 Q7 Q12 Q12

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q3 Q3 Q4 Q4 Q3 Q3 Q4 Q4 Q8 Q8 Q11 Q11 Q8 Q8 Q11 Q11

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q5 Q5 Q6 Q6 Q5 Q5 Q6 Q6 Q9 Q9 Q10 Q10 Q9 Q9 Q10 Q10

must have at least two DIMM-quads installed.

Enclosure 1

Enclosure 2

Enclosure 3

Quads Q1 and Q2 must be identical to each other. Quads Q3 and Q4 must be identical to each other. Quads Q5 and Q6 must be identical to each other. Note: For maximum memory performance, the total memory capacity on each memory controller must be equivalent.

Chapter 2. Architecture and technical overview 53

Table 2-6 shows the optimal placement of each DIMM-quad within a four-enclosure system. Each enclosure must have at least two DIMM-quads installed.

Table 2-6 Optimum DIMM-quad placement for a four-enclosure system

Enclosure 1

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q1 Q1 Q2 Q2 Q1 Q1 Q2 Q2 Q9 Q9 Q16 Q16 Q9 Q9 Q16 Q16

Enclosure 2

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q3 Q3 Q4 Q4 Q3 Q3 Q4 Q4 Q10 Q10 Q15 Q15 Q10 Q10 Q15 Q15

Enclosure 3

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller 1 Memory controller 0 Memory controller 1 Memory controller 0

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q5 Q5 Q6 Q6 Q5 Q5 Q6 Q6 Q11 11 Q14 Q14 11 Q11 Q14 Q14

CPU 1 CPU 3 CPU 4 CPU 2

Memory controller Memory controller Memory controller Memory controller

J1A J2A J3A J4A J5A J6A J7A J8A J1B J2B J3B J4B J5B J6B J7B J8B

Q7 Q7 Q8 Q8 Q7 Q7 Q8 Q8 Q12 Q12 Q13 Q13 Q12 Q12 Q13 Q13

Quads Q1 and Q2 must be identical to each other. Quads Q3 and Q4 must be identical to each other. Quads Q5 and Q6 must be identical to each other. Quads Q7 and Q8 must be identical to each other. Note: For maximum memory performance, the total memory capacity on each memory controller must be equivalent.

2.3.3 Memory activation

The minimum amount of memory activation for the Power 770 and Power 780 servers is 50% of the installed memory. For example, the minimum amount of memory that can be installed is 8 x 8 GB, of which 32 GB will be active. On an exception basis, a request for price quotation (RPQ) may be requested so that memory activation may go down to 25% of the installed memory.

Enclosure 4

54 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.3.4 Memory throughput

POWER7+ has exceptional cache, memory, and interconnect bandwidths. Table 2-7 shows the bandwidth estimate for the Power 770 system running at 3.8 GHz.

Table 2-7 Power 770 memory bandwidth estimates for POWER7+ cores running at 3.8 GHz

Memory Bandwidth

L1 (data) cache 182.78 GBps

L2 cache 182.78 GBps

L3 cache 121.85 GBps

System memory: 4x enclosures:

Inter-node buses (four enclosures) 158.02 GBps

Intra-node buses (four enclosures) 1075.2 GBps

68.22 GBps per socket

1091.58 GBps

With an increase in frequency, the Power 780 running at 4.42 GHz generates higher cache bandwidth (Table 2-8).

Table 2-8 Power 780 memory bandwidth estimates for POWER7+ cores running at 4.42 GHz

Memory Bandwidth

L1 (data) cache 212.35 GBps

L2 cache 212.35GBps

L3 cache 141.56 GBps

System memory: 4x enclosures:

Inter-node buses (four enclosures) 158.02 GBps

Intra-node buses (four enclosures) 1075.2 GBps

68.22 GBps per socket

1091.58 GBps

2.3.5 Active Memory Mirroring

Power 770 and Power 780 servers have the ability to provide mirroring of the hypervisor code across multiple memory DIMMs. If a DIMM that contains the hypervisor code develops an uncorrectable error, its mirrored partner will enable the system to continue to operate uninterrupted.

Active Memory Mirroring (AMM) is included with all Power 780 systems at no additional charge. It can be enabled, disabled, or re-enabled depending on the user’s requirements.

On the Power 770, AMM is optional and must be ordered using feature code 4797. It can be enabled, disabled, or re-enabled depending on the user's requirements.

The hypervisor code, which resides on the initial DIMMs (J1A to J8A), will be mirrored on the same group of DIMMs to allow for more usable memory. Table 2-3 on page 51 shows the DIMMs involved on the memory mirroring operation.

Chapter 2. Architecture and technical overview 55

Figure 2-9 shows how Active Memory Mirroring uses DIMM-quads.

P3-C28

P3-C29

P3-C27

P3-C26

P3-C25

P3-C24

P3-C23

P3-C22

M0 - D SN DDR3 DIMM #16

M0 - C SN DDR3 DIMM #15

M0 - B SN DDR3 DIMM #14

M0 - A SN DDR3 DIMM #13

Regulator 8

TPMD Card

Regulator 7

Regulator 6

Regulator 5

DIMM Quad 2

Mirrored Hypervisor code

M0 - D SN DDR3 DIMM #12

M0 - C SN DDR3 DIMM #11

M0 - B SN DDR3 DIMM #10

M0 - A SN DDR3 DIMM #9

DIMM Quad 2

Hypervisor code

Loc Code Conn Ref

P3-C1 J1A

P3-C2 J2A

P3-C3 J3A

P3-C4 J4A

P3-C5

FSI 3

FSI 5

FSI 13

FSI 15

FSI 6

FSI 4

FSI 16

FSI 14

P3-C6

P3-C7 J5A

P3-C8 J6A

P3-C9 J7A

P3-C10 J8A

P3-C11 J1B

P3-C12 J2B

P3-C13 J3B

P3-C14 J4B

P3-C15

P3-C16

P3-C17

P3-C18 J5B

P3-C19 J6B

P3-C20 J7B

P3-C21 J8B

Midplane Connector

Regulator 1

Regulator 2

Regulator 3

Regulator 4

P7+ CP3

SNFSI 0, 4 SNFSI 1, 5 SNFSI 2, 6 SNFSI 3, 7

P7+ CP1

P7+ CP3

P7+ CP1

SNFSI 0, 4 SNFSI 1, 5 SNFSI 2, 6 SNFSI 3, 7

FSI0 FSI1 MC0

MC0 FSI0 FSI1

Figure 2-9 Active Memory Mirroring in a single CEC

To enable AMM feature, the server must have at least eight DIMMs of the same size populated on slots J1A to J8A. It is also mandatory that the server has enough free memory to accommodate the mirrored memory pages.

Besides the hypervisor code itself, other components that are vital to the server operation are also mirrored:

򐂰 Hardware page tables (HPTs), responsible for tracking the state of the memory pages

assigned to partitions

򐂰 Translation control entities (TCEs), responsible for providing I/O buffers for the

partition’s communications

򐂰 Memory used by the hypervisor to maintain partition configuration, I/O states, virtual I/O

information, and partition state

56 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

It is possible to check whether the Active Memory Mirroring option is enabled and change its current status through HMC, under the Advanced Tab on the CEC Properties panel (Figure 2-10).

Figure 2-10 CEC Properties panel on an HMC

After a failure on one of the DIMMs containing hypervisor data occurs, all the server operations remain active and flexible service processor (FSP) will isolate the failing DIMMs. Because there are no longer eight functional DIMMs behind a memory controller, Active Memory Mirroring are not available until this DIMM is replaced. Systems stay in the partially mirrored state until the failing DIMM is replaced.

There are components that are not mirrored because they are not vital to the regular server operations and require a larger amount of memory to accommodate its data:

򐂰 Advanced Memory Sharing Pool 򐂰 Memory used to hold the contents of platform dumps

Partition data: Active Memory Mirroring will not mirror partition data. It was designed to mirror only the hypervisor code and its components, allowing this data to be protected against a DIMM failure

With AMM, uncorrectable errors in data that are owned by a partition or application are handled by the existing Special Uncorrectable Error handling methods in the hardware, firmware, and operating system.

2.3.6 Special Uncorrectable Error handling

Special Uncorrectable Error (SUE) handling prevents an uncorrectable error in memory or cache from immediately causing the system to terminate. Rather, the system tags the data and determines whether it will ever be used again. If the error is irrelevant, it does not force a checkstop. If the data is used, termination can be limited to the program/kernel or hypervisor owning the data, or freeze of the I/O adapters controlled by an I/O hub controller if data is to be transferred to an I/O device.

Chapter 2. Architecture and technical overview 57

2.4 Capacity on Demand

Several types of capacity on Demand (CoD) offerings are optionally available on the Power 770 and 780 servers to help meet changing resource requirements in an on-demand environment, by using resources that are installed on the system but that are not activated.

2.4.1 Capacity Upgrade on Demand (CUoD)

With the CUoD offering, you can purchase additional permanent processor or memory capacity and dynamically activate them when needed, without requiring you to restart your server or interrupt your business.

2.4.2 On/Off Capacity on Demand (On/Off CoD)

With the On/Off CoD offering, you can temporarily activate and deactivate processor cores and memory units to help meet the demands of business peaks such as seasonal activity, period-end, or special promotions. When you order an On/Off CoD feature, you receive an enablement code that allows a system operator to make requests for additional processor and memory capacity in increments of one processor day or 1 GB memory day. The system monitors the amount and duration of the activations. Both prepaid and post-pay options are available.

Charges are based on usage reporting that is collected monthly. Processors and memory may be activated and turned off an unlimited number of times, when additional processing resources are needed.

This offering provides a system administrator an interface at the HMC to manage the activation and deactivation of resources. A monitor that resides on the server records the usage activity. This usage data must be sent to IBM on a monthly basis. A bill is then generated based on the total amount of processor and memory resources utilized, in increments of Processor and Memory (1 GB) Days.

New to both Power 770 model MMD and Power 780 model MHD are 90-day temporary On/Off CoD processor and memory enablement features. These features enable a system to temporarily activate all inactive processor and memory CoD resources for a maximum of 90 days before ordering another temporary on/off enablement feature code is required. Also announced for Power 770 model MMD are high density memory DIMMs using 4 GB technology. This technology provides memory DIMMs for 64 GB, 128 GB, and 256 GB DDR3 memory feature codes. IBM continues to offer the 32 GB, 2 GB memory feature.

Before using temporary capacity on your server, you must enable your server. To enable, an enablement feature (MES only) must be ordered and the required contracts must be in place.

If a Power 770 or Power 780 server uses the IBM i operating system in addition to any other supported operating system on the same server, the client must inform IBM which operating system caused the temporary On/Off CoD processor usage so that the correct feature can be used for billing.

The features that are used to order enablement codes and support billing charges on the Power 770 and 780 are described in 1.4, “System features” on page 6 and 1.4.7, “Memory features” on page 18.

58 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

The On/Off CoD process consists of three steps: enablement, activation, and billing. 򐂰 Enablement

Before requesting temporary capacity on a server, you must enable it for On/Off CoD. To do this, order an enablement feature and sign the required contracts. IBM will generate an enablement code, mail it to you, and post it on the web for you to retrieve and enter on the target server.

processor enablement code allows you to request up to 360 processor days of

temporary capacity. If the 360 processor-day limit is reached, place an order for another processor enablement code to reset the number of days that you can request back to 360.

memory enablement code lets you request up to 999 memory days of temporary

capacity. If you reach the limit of 999 memory days, place an order for another memory enablement code to reset the number of allowable days you can request back to 999.

򐂰 Activation requests

When On/Off CoD temporary capacity is needed, use the HMC menu for On/Off CoD. Specify how many inactive processors or gigabytes of memory are required to be temporarily activated for some number of days. You are billed for the days requested, whether the capacity is assigned to partitions or remain in the shared processor pool.

At the end of the temporary period (days that were requested), you must ensure that the temporarily activated capacity is available to be reclaimed by the server (not assigned to partitions), or you are billed for any unreturned processor days.

򐂰 Billing

The contract, signed by the client before receiving the enablement code, requires the On/Off CoD user to report billing data at least once a month (whether or not activity occurs). This data is used to determine the proper amount to bill at the end of each billing period (calendar quarter). Failure to report billing data for use of temporary processor or memory capacity during a billing quarter can result in default billing equivalent to 90 processor days of temporary capacity.

For more information about registration, enablement, and usage of On/Off CoD, visit the following location:

http://www.ibm.com/systems/power/hardware/cod

2.4.3 Utility Capacity on Demand (Utility CoD)

Utility CoD automatically provides additional processor performance on a temporary basis within the shared processor pool.

With Utility CoD, you can place a quantity of inactive processors into the server’s shared processor pool, which then becomes available to the pool's resource manager. When the server recognizes that the combined processor utilization within the shared processor pool exceeds 100% of the level of base (purchased and active) processors that are assigned across uncapped partitions, then a Utility CoD processor minute is charged and this level of performance is available for the next minute of use.

If additional workload requires a higher level of performance, the system automatically allows the additional Utility CoD processors to be used, and the system automatically and continuously monitors and charges for the performance needed above the base (permanent) level.

Chapter 2. Architecture and technical overview 59

Registration and usage reporting for utility CoD is made using a public website and payment is based on reported usage. Utility CoD requires PowerVM Standard Edition or PowerVM Enterprise Edition to be active.

If a Power 770 or Power 780 server uses the IBM i operating system in addition to any other supported operating system on the same server, the client must inform IBM which operating system caused the temporary Utility CoD processor usage so that the correct feature can be used for billing.

For more information regarding registration, enablement, and use of Utility CoD, visit the following location:

http://www.ibm.com/systems/support/planning/capacity/index.html

2.4.4 Trial Capacity on Demand (Trial CoD)

A standard request for Trial CoD requires you to complete a form including contact information and vital product data (VPD) from your Power 770 or Power 780 system with inactive CoD resources.

A standard request activates two processors or 4 GB of memory (or both two processors and 4 GB of memory) for 30 days. Subsequent standard requests can be made after each purchase of a permanent processor activation. An HMC is required to manage Trial CoD activations.

exception request for Trial CoD requires you to complete a form including contact

information and VPD from your Power 770 or Power 780 system with inactive CoD resources. An exception request will activate all inactive processors or all inactive memory (or all inactive processor and memory) for 30 days. An exception request can be made only one time over the life of the machine. An HMC is required to manage Trial CoD activations.

To request either a Standard or an Exception Trial, visit the following location:

https://www-912.ibm.com/tcod_reg.nsf/TrialCod?OpenForm

2.4.5 Software licensing and CoD

For software licensing considerations with the various CoD offerings, see the most recent revision of the Power Systems Capacity on Demand User’s Guide:

http://www.ibm.com/systems/power/hardware/cod

2.5 CEC drawer interconnection cables

IBM Power 770 or 780 systems can be configured with more than one system enclosure. The connection between the processor cards in the separate system enclosures requires a set of processor drawer interconnect cables. Each system enclosure must be connected to each other through a flat flexible SMP cable. These cables are connected on the front of the drawers.

Furthermore, service processor cables are needed to connect the components in each system enclosure to the active service processor for system functions monitoring. These flexible cables connect at the rear of each enclosure and are required for two-drawer, three-drawer, and four-drawer configurations.

60 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

The star fabric bus topology that connects the processors together in separate drawers is contained on SMP flex cables that are routed external to the drawers. These flex cables attach directly to the CPU cards at the front of the drawer and are routed behind the front covers of the drawers.

The SMP and flexible service processor (FSP) cable features described in Table 2-9 are required to connect the processors together when system configuration is made of two-drawer, three-drawer, or four-drawer system enclosures.

Table 2-9 Required flex cables feature codes

Enclosure MMB/MMC SMP cables MMD/MHD SMP cables FSP cables

Two-drawer 3711, 3712 3715, 3716 3671

Three-drawer 3712, 3713 3716, 3717 3671, 3672

Four-drawer 3712, 3713, 3714 3716, 3717, 3718 3671, 3672, 3673

The cables are designed to support hot-addition of system enclosure up to the maximum scalability. When adding a new drawer, existing cables remain in place and new cables are added. The only exception is for cable 3711 or 3715, which is replaced when growing from a two-drawer to three-drawer configuration.

The cables are also designed to allow the concurrent maintenance of the Power 770 or Power 780 in case the IBM service representative needs to extract a system enclosure from the rack. The design of the flexible cables allows each system enclosure to be disconnected without any impact on the other drawers.

Chapter 2. Architecture and technical overview 61

To allow such concurrent maintenance operation, plugging the SMP Flex cables in the order of their numbering is extremely important. Each cable is numbered, as shown in Figure 2-11. Note that the 2- and 3-Drawer cabling has changed in this model.

Figure 2-11 SMP cables installation

Figure 2-12 and Figure 2-13 shows the changed SMP cabling for 2- and 3-drawer configurations. There is some overlap of the cables, so some are hidden from view.

62 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Figure 2-12 Three link 2-Drawer SMP cabling

Figure 2-13 Two link 3-Drawer SMP cabling

Chapter 2. Architecture and technical overview 63

Similarly, the FSP flex cables must be installed in the correct order (see Figure 2-14), as follows:

1. Install a second node flex cable from node 1 to node 2.

2. Add a third node flex cable from node 1 and node 2 to node 3.

3. Add a fourth node flex cable from node 1 and node 2 to node 4.

Figure 2-14 FSP flex cables

The design of the Power 770 and Power 780 is optimized for use in an IBM 7014-T00 or 7014-T42 rack. Both the front cover and the external processor fabric cables occupy space on the front left and right sides of an IBM 7014 rack; racks that are not from IBM might not offer the same room. When a Power 770 or Power 780 is configured with two or more system enclosures in a 7014-T42 or 7014-B42 rack, the CEC enclosures must be located in EIA 36 or below to allow space for the flex cables.

64 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

The total width of the server, with cables installed, is 21 inches, as shown in Figure 2-15.

SMP

Cable

Drawer 1  B

Left

Drawer 3  A

Left

SMP

Cable

Drawer 2  B

Left

Drawer 3  B

Left

SMP

Cable

Drawer 1  A

Left

Drawer 4  A

Left

SMP

Cable

Drawer 3  A

Right

Drawer 4  A

Right

SMP

Cable

Drawer 2  A

Right

Drawer 4  B

Right

SMP

Cable

Drawer 1  B

Right

Drawer 2  B

Right

Figure 2-15 Front view of the rack with SMP cables overlapping the rack rails

Chapter 2. Architecture and technical overview 65

In the rear of the rack, the FSP cables require only some room in the left side of the racks, as

Four Drawer Cable

Three Drawer Cable

Two Drawer Cable

Figure 2-16 shows.

Figure 2-16 Rear view of rack with detail of FSP flex cables

66 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.6 System bus

This section provides additional information related to the internal buses.

2.6.1 I/O buses and GX++ card

Each CEC enclosure of the Power 770 and Power 780 contains one POWER7+ processor card. Each processor card comprises 3-core, 4-core, or 8-core single-chip module POWER7+ processors, with different frequencies depending on the configuration, such as 4.228Ghz,

3.724GHz, or 3.808Ghz

Within a CEC enclosure a total of two GX++ buses are available for I/O connectivity and expansion. Each GX++ bus is routed through the midplane to the I/O backplane and drive to two different POWER7+ processors. The two POWER7+ processors are responsible for the other two IO chips, also routed through the midplane. Table 2-10 shows the I/O bandwidth for available processors cards.

Table 2-10 External GX++ I/O bandwidth

Processor card Slot description Frequency GX++ Bandwidth

(maximum theoretical)

򐂰 4.228 GHz 򐂰 3.808 GHz 򐂰 4.424 GHz 򐂰 3.7 GHz

Single enclosure (data portion is on average 2/3 total) 26.283 GBps

Total (4x enclosures) 157.696 GBps

Bandwidth: Technically, all the other interfaces, such as daughter card, asynchronous port, DVD, USB, and the PCI Express slots are connected to two other internal GX++ ports through two P7IOC chipsets. So, theoretically, if all the ports, devices, and PCIe slots are considered, the total I/O bandwidth for a system with four CECs is 315.392 GBps.

2.6.2 Service processor bus

The flexible service processor (FSP) flex cable, which is located at the rear of the system, is used for service processor (SP) communication between the system drawers. A SP card (FC EU09) is installed in system drawer 1 and system drawer 2. The FSP/Clock Pass-Through card (FC 5665 CCIN 2BBC) is installed in system drawers 3 and 4. These cards connect to drawers 1 and 2 through the FSP flex cable. The FSP cable was changed to point-to-point cabling similar to the processor drawer interconnect cable. When a system drawer is added, another FSP flex cable is added. A detailed cable configuration is discussed in 2.5, “CEC drawer interconnection cables” on page 60.

CPU Socket 0 (CP0) GX bus 1

CPU Socket 0 (CP0) GX bus 0

2.464 GHz 19.712 GBps

2.464

GHz 19.712 GBps

Chapter 2. Architecture and technical overview 67

2.7 Internal I/O subsystem

The internal I/O subsystem resides on the I/O planar, which supports six PCIe slots. All PCIe slots are hot-pluggable and enabled with enhanced error handling (EEH). In the unlikely event of a problem, EEH-enabled adapters respond to a special data packet that is generated from the affected PCIe slot hardware by calling system firmware, which examines the affected bus, allows the device driver to reset it, and continues without a system reboot. For more information about RAS on the I/O buses, see Chapter 4, “Continuous availability and manageability” on page 159

Table 2-11 lists the slot configuration of the Power 770 and Power 780.

Table 2-11 Slot configuration of the Power 770 and 780

Slot number

Slot 1 PCIe Gen2 x8 P2-C1 P7IOC A PCIe PHB5 Full length

Slot 2 PCIe Gen2 x8 P2-C2 P7IOC A PCIe PHB4 Full length

Slot 3 PCIe Gen2 x8 P2-C3 P7IOC A PCIe PHB3 Full length

Slot 4 PCIe Gen2 x8 P2-C4 P7IOC A PCIe PHB2 Full length

Slot 5 PCIe Gen2 x8 P2-C5 P7IOC B PCIe PHB5 Full length

Slot 6 PCIe Gen2 x8 P2-C6 P7IOC B PCIe PHB4 Full length

Description Location

code

PCI host bridge (PHB)

Maximum card size

Slot 7 GX++ P1-C2 - -

Slot 8 GX++ P1-C3 - -

2.7.1 Blind-swap cassettes

The Power 770 and Power 780 uses fourth-generation blind-swap cassettes to manage the installation and removal of adapters. This mechanism requires an interposer card that allows the PCIe adapters to plug in vertically to the system, allows more airflow through the cassette, and provides more room under the PCIe cards to accommodate the GX++ multifunctional host bridge chip heat-sink height. Cassettes can be installed and removed without removing the CEC enclosure from the rack.

2.7.2 System ports

Each CEC enclosure is equipped with an integrated multifunction card. This integrated card provides two USB ports, one serial port, and four Ethernet connectors for a processor enclosure and does not require a PCIe slot. When ordering a Power 770 or Power 780, you may select the following options:

򐂰 Dual 10 Gb Copper and Dual 1 Gb Ethernet (FC 1768 CCIN 2BF3) 򐂰 Dual 10 Gb Optical and Dual 1 Gb Ethernet (FC 1769 CCIN 2BF4) 򐂰 Dual 10 Gb RJ45 and Dual SFP+ 10Gb Twinaxial LOM (FC EN10 򐂰 Dual 10 Gb RJ45 and Dual SFP+ 10Gb Optical-SR LOM(FC EN11

CCIN 2C4C)

CCIN 2C4D)

All of the connectors are on the rear bulkhead of the CEC, and one integrated multifunction card can be placed in an individual CEC enclosure. An integrated multifunction card is

Feature codes EN10 and EN11 are currently not supported. Support is planned for middle of 2013.

68 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

required in CEC enclosure 1, but it is not required in CEC enclosures 2, 3, or 4. On a multi-enclosure system, the integrated multifunction card features can differ.

On the port type and cable size, the copper twinaxial ports support up to 5 m cabling distances. The RJ-45 ports support up to 100 m cabling distance using a CAT5e cable. The optical ports only support the 850 nm optic cable (multi-mode cable) and support up to 300 m cabling distances.

The Power 770 and Power 780 each support one serial port in the rear of the system. This connector is a standard 9-pin male D-shell, and it supports the RS232 interface. Because the Power 770 and Power 780 are managed by an HMC, this serial port is always controlled by the operating system, and therefore is available in any system configuration. It is driven by the integrated PLX Serial chip, and it supports any serial device that has an operating system device driver. The FSP virtual console will be on the HMC.

2.8 PCI adapters

This section covers the types and functions of the PCI cards supported by IBM Power 770 and Power 780 systems.

2.8.1 PCI Express (PCIe)

PCIe uses a serial interface and allows for point-to-point interconnections between devices (using a directly wired interface between these connection points). A single PCIe serial link is a dual-simplex connection that uses two pairs of wires, one pair for transmit and one pair for receive, and can transmit only one bit per cycle. These two pairs of wires are called a PCIe link can consist of multiple lanes. In such configurations, the connection is labelled as x1, x2, x8, x12, x16, or x32, where the number is effectively the number of lanes.

lane. A

Two generations of PCIe interface are supported in Power 770 and Power 780 models: 򐂰 Gen1: Capable of transmitting at the extremely high speed of 2.5 Gbps, which gives a

capacity of a peak bandwidth of 2 GBps simplex on an 8-lane interface

򐂰 Gen2: Double the speed of the Gen1 interface, which gives a capacity of a peak

bandwidth of 4 GBps on an 8-lane interface

PCIe Gen1 slots support Gen1 adapter cards and also most of the Gen2 adapters. In this case, when a Gen2 adapter is used in a Gen1 slot, the adapter will operate at PCIe Gen1 speed. PCIe Gen2 slots support both Gen1 and Gen2 adapters. In this case, when a Gen1 card is installed into a Gen2 slot, it operates at PCIe Gen1 speed with a slight performance enhancement. When a Gen2 adapter is installed into a Gen2 slot, it operates at the full PCIe Gen2 speed.

The IBM Power 770 and Power 780 CEC enclosure is equipped with six PCIe 8x Gen2 slots.

2.8.2 PCI-X adapters

IBM offers PCIe adapter options for the Power 770 and Power 780 CEC enclosure. If a PCI-extended (PCI-X) adapter is required, a PCI-X DDR 12X I/O Drawer (FC 5796) can be attached to the system by using a GX++ adapter loop. PCIe adapters use a different type of slot than PCI and PCI-X adapters. If you attempt to force an adapter into the wrong type of slot, you might damage the adapter or the slot. All adapters support Extended Error Handling (EEH) on PCIe and PCI-X slots. For more information about RAS on I/O devices, see Chapter 4, “Continuous availability and manageability” on page 159.

Chapter 2. Architecture and technical overview 69

2.8.3 IBM i IOP adapters

IBM i IOP adapters are not supported with the Power 770 and Power 780, which has the following results:

򐂰 Existing PCI adapters that require an IOP are affected. 򐂰 Existing I/O devices are affected, such as certain tape libraries or optical drive libraries, or

any HVD SCSI device.

򐂰

Twinaxial displays or printers cannot be attached except through an OEM protocol converter.

Before adding or rearranging adapters, use the System Planning Tool to validate the new adapter configuration. See the System Planning Tool website:

http://www.ibm.com/systems/support/tools/systemplanningtool/

If you are installing a new feature, ensure that you have the software that is required to support the new feature, and determine whether there are any existing PTF prerequisites to install. See the IBM Prerequisite website for information:

https://www-912.ibm.com/e_dir/eServerPreReq.nsf

2.8.4 PCIe adapter form factors

IBM POWER7 and POWER7+ processor-based servers are able to support two form factors of PCIe adapters:

򐂰 PCIe low-profile (LP) cards, which are used on the Power 710 and Power 730 PCIe slots.

Low profile adapters are also supported in the PCIe riser card slots of the Power 720 and Power 740 servers.

򐂰 PCIe full-height and full-high cards, which are plugged into the following servers slots:

– Power 720 and Power 740 (Within the base system, five PCIe half-length slots

are supported.) –Power750 –Power755 –Power760 –Power770 –Power780 –Power795 – PCIe slots of external drawers, as FC 5802 and FC 5877

Low-profile PCIe adapter cards are supported only in low-profile PCIe slots, and full-height and full-high cards are supported only in full-high slots.

Figure 2-17 on page 71 lists the PCIe adapter form factors.

70 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Figure 2-17 PCIe adapter form factors

Low Profile PCIe Slots

 Power 710 / 730  Power 720 / 740

- PC I e riser c ar d

Low Profile Full Height Full High

Full High PCIe Slots

 Power 720 / 740 / 750 / 760 / 770 / 780 / 795  12X PCIe I/O Drawer

- FC 5802 / FC 5877 for 19-inch rack

- FC 5803 / FC 5873 for 24-inch rack

Many of the full-height card features are also available in low-profile format. For example, the PCIe RAID and SSD SAS Adapter 3 Gb is available as a low-profile adapter or as a full-height adapter, each one having a different feature code. As expected, they have equivalent functional characteristics.

Table 2-12 is a list of low-profile adapter cards and their equivalents in full height.

Table 2-12 Equivalent adapter cards

Low profile Adapter description Full height

Feature

CCIN Feature

code

2053 57CD PCIe RAID and SSD SAS adapter 3 Gb 2054 or

5269 5269 PCIe POWER GXT145 Graphics Accelerator 5748 5748

5270 2B3B 10 Gb FCoE PCIe Dual Port adapter 5708 2B3B

5271 5271 4-Port 10/100/1000 Base-TX PCI-Express adapter 5717 5271

5272 5272 10 Gigabit Ethernet-CX4 PCI Express adapter 5732 5732

5273 577D 8 Gigabit PCI Express Dual Port Fibre Channel

adapter

5274 5768 2-Port Gigabit Ethernet-SX PCI Express adapter 5768 5768

5275 5275 10 Gb ENet Fibre RNIC PCIe 8x adapter 5769 5275

5276 5774 4 Gigabit PCI Express Dual Port Fibre Channel

adapter

5277 57D2 4-Port Sync EIA-232 PCIe adapter 5785 57D2

5278 57B3 SAS Controller PCIe 8x adapter 5901 57B3

5280 2B44 PCIe2 LP 4-Port 10 Gb Ethernet &1 Gb Ethernet

SR & RJ45 adapter

CCIN

code

57CD

2055

5735 577D

5774 5774

5744 2B44

EN0B 577F PCIe2 16 Gb 2-Port Fibre Channel adapter EN0A 577F

EN0J 2B93 PCIe2 4-Port (10 Gb FCOE & 1 Gb Ethernet)

EN0H 2B93

SR & RJ45 adapter

Chapter 2. Architecture and technical overview 71

Before adding or rearranging adapters, use the System Planning Tool to validate the new adapter configuration. See the System Planning Tool website for detailed information:

http://www.ibm.com/systems/support/tools/systemplanningtool/

If you are installing a new feature, ensure that you have the required software to support the new feature and determine whether there are any existing update prerequisites to install. To do this, use the IBM Prerequisite website:

https://www-912.ibm.com/e_dir/eServerPreReq.nsf

The following sections discuss the supported adapters and provide tables of orderable feature codes. The tables indicate operating system support (AIX, IBM i, and Linux) for each of the adapters.

2.8.5 LAN adapters

To connect a Power 770 and Power 780 to a local area network (LAN), you can use the integrated multifunction card, or a dedicated adapter. For more information, see 2.7.2, “System ports” on page 68.

LPARs: The integrated multifunction card can be shared by LPARS that use VIOS, so each LPAR is able to access it without a dedicated network card.

Other LAN adapters are supported in the CEC enclosure PCIe slots or in I/O enclosures that are attached to the system by using a PCI-X or PCIe slot. Table 2-13 lists the additional LAN adapters that are available.

Table 2-13 Available LAN adapters

Feature code

5287 5287 2-port 10 GbE SR PCIe adapter PCIe Low profile

5288 5288 2-Port 10 GbE SFP+ Copper

5706 5706 IBM 2-Port 10/100/1000 Base-TX

5708 2B3B 2-Port 10Gb NIC/FCoE Adapter PCIe Full height

5717 5271 4-Port 10/100/1000 Base-TX PCI

5732 5732 10 Gigabit Ethernet-CX4 PCI

5740 4-Port 10/100/1000 Base-TX

5744 2B44 PCIe2 4-Port 10 GbE&1 GbE SR

5745 2B43 PCIe2 4-Port 10 GbE&1 GbE

CCIN Adapter description Slot Size OS

Short

PCIe Full height

adapter

PCI-X Full height

Ethernet PCI-X adapter

PCIe Full height

Express adapter

PCIe Full height

Express adapter

PCI-X Full height

PCI-X adapter

PCIe Full height Linux

RJ45 adapter

PCIe Short Linux

SFP+Copper&RJ45 adapter

Short

support

AIX, Linux

AIX, IBM i, Linux

AIX, Linux

72 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Feature code

CCIN Adapter description Slot Size OS

support

5767 5767 2-Port 10/100/1000 Base-TX

Ethernet PCI Express adapter

5768 5768 2-Port Gigabit Ethernet-SX PCI

Express adapter

5769 5769 10 Gigabit Ethernet-SR PCI

Express adapter

5772 576E 10 Gigabit Ethernet-LR PCI

Express adapter

5899 576F 4-Port 1Gb Ethernet Adapter PCIe Full height AIX, IBM i,

EC28 EC27 2-Port 10Gb Ethernet/FCoE SFP+

Adapter with Optics

EC30 EC29 2-Port 10Gb Ethernet/FCoE SFP+

Adapter with Optics

EN0H 2B93 PCIe2 4-port (10Gb FCoE and

1GbE) SR&RJ4 Adapter

2.8.6 Graphics accelerator adapters

The IBM Power 770 and Power 780 support up to eight graphics adapters (Table 2-14). They can be configured to operate in either 8-bit or 24-bit color modes. These adapters support both analog and digital monitors, and do not support hot-plug installation. The total number of graphics accelerator adapters in any one partition cannot exceed four.

PCIe Full height

Short

PCIe Full height

Short

PCIe Full height

Short

PCIe Full height

Short

PCIe Low profile AIX, Linux

PCIe Full height AIX, Linux,

AIX, IBM i, Linux

AIX, Linux

AIX, IBM i, Linux

Linux

IBM i

Table 2-14 Available graphics accelerator adapters

Feature code

2849

5748 5748 POWER GXT145 PCI Express

a. Supported, but is no longer orderable.

CCIN Adapter description Slot Size OS

2849 POWER GXT135P Graphics

Accelerator with Digital Support

Graphics Accelerator

support

PCI-X Short AIX,

Linux

PCIe Short AIX,

Linux

Chapter 2. Architecture and technical overview 73

2.8.7 SCSI and SAS adapters

The Power 770 and Power 780 do not support SCSI adapters and SCSI disks. SAS adapters are supported and Table 2-15 lists the available SAS adapters.

Table 2-15 Available SCSI and SAS adapters

Feature code

CCIN Adapter description Slot Size OS

support

2055 57CD PCIe RAID and SSD SAS Adapter

5805 574E PCIe 380MB Cache Dual - x4 3 Gb

5901 57B3 PCIe Dual-x4 SAS adapter PCIe Short AIX, IBM i,

5903

5908 575C PCI-X DDR 1.5 GB Cache SAS

5912 572A PCI-X DDR Dual - x4 SAS adapter PCI-X Short AIX, IBM i,

5913

ESA1 57B4 PCIe2 RAID SAS Adapter

a. A pair of adapters is required to provide mirrored write cache data and adapter redundancy.

2.8.8 iSCSI adapters

3 Gb with Blind Swap Cassette

SAS RAID Adapter

574E PCIe 380MB Cache Dual - x4 3 Gb

SAS RAID adapter

RAID adapter (BSC)

57B5 PCIe2 1.8 GB Cache RAID SAS

adapter Tri-port 6 Gb

Dual-port 6 Gb

PCIe Short AIX, IBM i,

Linux

PCIe Full height AIX IBM i,

Linux

PCIe Short AIX, IBM i,

Linux

PCI-X Long AIX, IBM i,

Linux

PCIe Full height AIX, IBM i,

Linux

PCIe Full height AIX, IBM i,

Linux

The iSCSI adapters in Power Systems provide the advantage of increased bandwidth through the hardware support of the iSCSI protocol. The 1 Gigabit iSCSI TCP/IP Offload Engine (TOE) PCI-X adapters support hardware encapsulation of SCSI commands and data into TCP, and transports them over the Ethernet using IP packets. The adapter operates as an iSCSI TOE. This offload function eliminates host protocol processing and reduces CPU utilization and interrupts. The adapter uses a small form factor LC type fiber optic connector or a copper RJ45 connector.

Table 2-16 lists the orderable iSCSI adapters.

Table 2-16 Available iSCSI adapters

Feature code

5713 573B 1 Gigabit iSCSI TOE PCI-X on

5714

a. Supported, but is no longer orderable.

CCIN Adapter description Slot Size OS

PCI-X Short AIX, IBM i,

Copper Media Adapter

573C 1 Gigabit iSCSI TOE PCI-X on

Optical Media Adapter

PCI-X Short AIX, IBM i,

74 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

support

Linux

2.8.9 Fibre Channel adapter

The IBM Power 770 and Power 780 servers support direct or SAN connection to devices that use Fibre Channel adapters. Table 2-17 summarizes the available Fibre Channel adapters.

All of these adapters except FC 5735 have LC connectors. If you attach a device or switch with an SC type fiber connector, an LC-SC 50 Micron Fiber Converter Cable (FC 2456) or an LC-SC 62.5 Micron Fiber Converter Cable (FC 2459) is required.

Table 2-17 Available Fibre Channel adapters

Feature code

a b

5729

5735

5749 576B 4 Gbps Fibre Channel (2-Port) PCI-X Short IBM i

5758

CCIN Adapter description Slot Size OS

577D 8 Gigabit PCI Express Dual Port Fibre

1910 280D 280E

PCIe2 8 Gb 4-port Fibre Channel Adapter

Channel Adapter

4 Gb Single-Port Fibre Channel PCI-X

2.0 DDR Adapter

support

PCIe AIX, Linux

PCIe Short AIX, IBM i,

Linux

PCI-X Short AIX, Linux

5759 1910

5759

5774 2844 4 Gigabit PCI Express Dual Port Fibre

a. A Gen2 PCIe slot is required to provide the bandwidth for all four ports to operate at full speed. b. N_Port ID Virtualization (NPIV) capability is supported through VIOS. c. Supported, but is no longer orderable.

4 Gb Dual-Port Fibre Channel PCI-X

2.0 DDR Adapter

Channel Adapter

PCI-X Short AIX, Linux

PCIe Short AIX, IBM i,

Linux

Chapter 2. Architecture and technical overview 75

2.8.10 Fibre Channel over Ethernet (FCoE)

Ethernet and F ibre

Channel Cables

Ethernet

Cable

Fibre Channel

Cable

FC S w i t ch

Et he rnet S wit c h

CEC or I/O Drawer

Et he rn et

CEC or I/O Drawer

Rack

Fibre Channel (FC)

De vi ce or F C Swi tch

Ethernet Cables

Eth er ne t

Cable

Fibre Channel

Cable

FCoE Switch

CEC or I/O Drawer

Rack

Fibre Channel (FC)

De vi ce or F C Swi tch

FCoE

Ethernet Device/

Sw it c h

Eth e rn e t De v ic e/

Switch or FCoE

De vice/S witc h

FCoE allows for the convergence of Fibre Channel and Ethernet traffic onto a single adapter and converged fabric.

Figure 2-18 compares an existing Fibre Channel and network connection, and a FCoE connection.

Figure 2-18 Comparison between existing Fibre Channel and network connection and FCoE connection

Table 2-18 lists the available Fibre Channel over Ethernet adapter. It is a high-performance Converged Network Adapters (CNA) using SR optics. Each port can provide network interface card (NIC) traffic and Fibre Channel functions simultaneously.

Table 2-18 Available FCoE adapter

Feature code

5708 2B3B 10 Gb FCoE PCIe Dual Port adapter PCIe Full height

CCIN Adapter description Slot Size OS

For more information about FCoE, see An Introduction to Fibre Channel over Ethernet, and Fibre Channel over Convergence Enhanced Ethernet, REDP-4493.

2.8.11 InfiniBand host channel adapter

The InfiniBand architecture (IBA) is an industry-standard architecture for server I/O and inter-server communication. It was developed by the InfiniBand Trade Association (IBTA) to provide the levels of reliability, availability, performance, and scalability necessary for present and future server systems with levels significantly better than can be achieved by using bus-oriented I/O structures.

InfiniBand (IB) is an open set of interconnect standards and specifications. The main IB specification is published by the InfiniBand Trade Association and is available at the following location:

http://www.infinibandta.org/

Short

support

AIX, Linux

76 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

InfiniBand is based on a switched fabric architecture of serial point-to-point links, where these IB links can be connected to either host channel adapters (HCAs), used primarily in servers, or to target channel adapters (TCAs), used primarily in storage subsystems.

The InfiniBand physical connection consists of multiple byte lanes. Each individual byte lane is a four-wire, 2.5, 5.0, or 10.0 Gbps bidirectional connection. Combinations of link width and byte-lane speed allow for overall link speeds from 2.5 Gbps to 120 Gbps. The architecture defines a layered hardware protocol, and also a software layer to manage initialization and the communication between devices. Each link can support multiple transport services for reliability and multiple prioritized virtual communication channels.

For more information about InfiniBand, read HPC Clusters Using InfiniBand on IBM Power Systems Servers, SG24-7767.

IBM offers the GX++ 12X DDR Adapter that plugs into the system backplane (GX++ slot). There are two GX++ slots in each CEC enclosure. By attaching a 12X to 4X converter cable (FC 1828), an IB switch can be attached.

Table 2-19 lists the available InfiniBand adapters.

Table 2-19 Available InfiniBand adapters

Feature code

1808 2BC3 GX++ 12X DDR adapter, Dual-port GX++ Standard

5285 58E2 2-Port 4X IB QDR adapter 40 Gb PCIe Full

CCIN Adapter description Slot Size OS

2.8.12 Asynchronous and USB adapters

Asynchronous PCI adapters provide connection of asynchronous EIA-232 or RS-422 devices. Table 2-20 lists the available asynchronous and USB adapters.

Table 2-20 Available asynchronous adapters

Feature code

2728 57D1 4-port USB PCIe adapter PCIe Short AIX, Linux

5289 57D4 2-Port Async EIA-232 PCIe

5785 57D2 4-Port Asynchronous EIA-232

CCIN Adapter description Slot Size OS

adapter, 2-Port RJ45 Async

PCIe adapter

support

AIX, IBM i,

size

height

PCIe Short AIX, Linux

Linux

AIX, IBM i, Linux

support

Heartbeats: PowerHA releases no longer support heartbeats over serial connections.

Chapter 2. Architecture and technical overview 77

2.8.13 Cryptographic Coprocessor

The Cryptographic Coprocessor cards provide both cryptographic coprocessor and cryptographic accelerator functions in a single card.

The IBM PCIe Cryptographic Coprocessor adapter highlights the following features:

򐂰 Integrated Dual processors that operate in parallel for higher reliability 򐂰 Supports IBM Common Cryptographic Architecture or PKCS#11 standard 򐂰 Ability to configure adapter as coprocessor or accelerator 򐂰 Support for smart card applications using Europay, MasterCard and Visa 򐂰 Cryptographic key generation and random number generation 򐂰 PIN processing: generation, verification, translation 򐂰 Encrypt and Decrypt using AES and DES keys

See the following site for the most recent firmware and software updates:

http://www.ibm.com/security/cryptocards/

Table 2-23 on page 85 lists the cryptographic adapter that is available for the server.

Table 2-21 Available cryptographic adapters

Feature code

CCIN Adapter description Slot Size OS

support

4808 4765 PCIe Crypto Coprocessor with GEN3

4809 4765 PCIe Crypto Coprocessor with GEN4

2.9 Internal storage

Serial-attached SCSI (SAS) drives the Power 770 and Power 780 internal disk subsystem. SAS provides enhancements over parallel SCSI with its point-to-point high frequency connections. SAS physical links are a set of four wires used as two differential signal pairs. One differential signal transmits in one direction. The other differential signal transmits in the opposite direction. Data can be transmitted in both directions simultaneously.

The Power 770 and Power 780 CEC enclosures have an extremely flexible and powerful backplane for supporting hard disk drives (HDD) or solid-state drives (SSD). The six small form factor (SFF) bays can be configured in three ways to match your business needs. Two integrated SAS controllers can be optionally augmented with a 175 MB Cache RAID - Dual IOA Enablement Card (Figure 2-19 on page 80). These two controllers provide redundancy and additional flexibility. The optional 175 MB Cache RAID - Dual IOA Enablement Card enables dual 175 MB write cache and provides dual batteries for protection of that write cache.

Blindswap Cassette 4765-001

PCIe Full height AIX,

IBM i

PCIe Full height AIX,

IBM i

There are two PCIe integrated SAS controllers under the POWER7 I/O chip and also the SAS controller that is directly connected to the DVD media bay (Figure 2-19 on page 80).

78 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Power 770 and Power 780 supports various internal storage configurations:

򐂰 Dual split backplane mode: The backplane is configured as two sets of three bays (3/3). 򐂰 Triple split backplane mode: The backplane is configured as three sets of two bays (2/2/2). 򐂰 Dual storage IOA configuration using internal disk drives (Dual RAID of internal drives

only): The backplane is configured as one set of six bays.

򐂰 Dual storage IOA configuration using internal disk drives and external enclosure (Dual

RAID of internal drives and external drives).

Configuration options can vary depending on the controller options and the operating system that is selected. The controllers for the dual split backplane configurations are always the two embedded controllers. But if the triple split backplane configuration is used, the two integrated SAS controllers run the first two sets of bays and require a SAS Controller adapter (FC 5901) located in a PCIe slot in a CEC enclosure. This adapter controls the third set of bays. By having three controllers, you can have three boot drives supporting three partitions.

Rules: The following SSD or HDD configuration rules apply:

򐂰 You can mix SSD and HDD drives when configured as one set of six bays. 򐂰 If you want to have both SSDs and HDDs within a dual split configuration, you must use

the same type of drive within each set of three. You cannot mix SSDs and HDDs within a subset of three bays.

򐂰 If you want to have both SSDs and HDDs within a triple split configuration, you must

use the same type of drive within each set of two. You cannot mix SSDs and HDDs within a subset of two bays. The FC 5901 PCIe SAS adapter that controls the remaining two bays in a triple split configuration does not support SSDs.

You can configure the two embedded controllers together as a pair for higher redundancy or you can configure them separately. If you configure them separately, they can be owned by separate partitions or they can be treated independently within the same partition. If configured as a pair, they provide controller redundancy and can automatically switch over to the other controller if one has problems. Also, if configured as a pair, both can be active at the same time (active/active) assuming that two or more arrays are configured, providing additional performance capability and also redundancy. The pair controls all six small form factor (SFF) bays and both see all six drives. The dual split (3/3) and triple split (2/2/2) configurations are not used with the paired controllers. RAID 0 and RAID 10 are supported, and you can also mirror two sets of controller/drives using the operating system.

Power 770 and Power 780, with more than one CEC enclosure, support enclosures with different internal storage configurations.

Adding the optional 175 MB Cache RAID - Dual IOA Enablement Card (FC 5662) causes the pair of embedded controllers in that CEC drawer to be configured as dual controllers, accessing all six SAS drive bays. With this feature you can get controller redundancy, additional RAID protection options, and additional I/O performance. RAID 5 (a minimum of three drives is required) and RAID 6 (a minimum of four drives is required) are available when configured as dual controllers with one set of six bays. Dual IOA Enablement Card (FC 5662) plugs in to the disk or media backplane and enables a 175 MB write cache on each of the two embedded RAID adapters by providing two rechargeable batteries with associated charger circuitry.

The write cache can provide additional I/O performance for attached disk or solid-state drives, particularly for RAID 5 and RAID 6. The write cache contents are mirrored for redundancy between the two RAID adapters, resulting in an effective write cache size of 175 MB. The

Chapter 2. Architecture and technical overview 79

batteries provide power to maintain both copies of write-cache information in the event that

P7IOC

Integrated

SAS Adapter

P7IOC

DVD

DASD

VSES

Redriver

Integrated

SAS Adapter

Integrated

SAS Adapter

Redriver

SAS Port Exp.

Optional

Battery

Optional

Battery

External Port

power is lost.

Without the Dual IOA Enablement Card, each controller can access only two or three SAS drive bays.

Another expansion option is an SAS expansion port (FC 1819). The SAS expansion port can add more SAS bays to the six bays in the system unit. A DASD expansion drawer (FC 5886) is attached using a SAS port on the rear of the processor drawer, and its two SAS bays are run by the pair of embedded controllers. The pair of embedded controllers is now running 18 SAS bays (six SFF bays in the system unit and twelve 3.5-inch bays in the drawer). The disk drawer is attached to the SAS port with a SAS YI cable, and the embedded controllers are connected to the port using a FC 1819 cable assembly. In this 18-bay configuration, all drives must be HDDs.

IBM i supports configurations that use one set of six bays but does not support logically splitting the backplane into split (dual or triple). Thus, the Dual IOA Enablement card (FC 5662) is required if IBM i is to access any of the SAS bays in that CEC enclosure. AIX and Linux support configurations using two sets of three bays (3/3) or three sets of two bays (2/2/2) without the dual IOA enablement card. With FC 5662, they support dual controllers running one set of six bays.

Figure 2-19 shows the internal SAS topology overview.

Figure 2-19 Internal SAS topology overview

The system backplane also includes a third embedded controller for running the DVD-RAM drive in the CEC enclosure. Because the controller is independent from the two other SAS disk or SSD controllers, it allows the DVD to be switched between multiple partitions without affecting the assignment of disks or SSDs in the CEC drawer.

80 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Table 2-22 summarizes the internal storage combination and the feature codes that are required for any combination.

Table 2-22 SAS configurations summary

SAS subsystem configuration

Two-way split backplane

Three-way split backplane

Dual storage IOA with internal disk

Dual storage IOA with internal disk and external disk enclosure

FC 5662 CCIN 2BC2

No None None N/A IBM i does not support

No Dual x4 SAS

Yes None None N/A Internal SAS port

Yes Requires an

External SAS components

adapter (FC 5901 CCIN 57B3)

external disk enclosure (FC 5886)

SAS port cables

Internal SAS port (FC 1815) SAS cable for three-way split backplane

Internal SAS port (FC 1819) SAS cable assembly for connecting to an external SAS drive enclosure

SAS cables

AI cable (FC 3679) Adapter to internal drive (1 meter)

FC 3686 or FC 3687

Notes

this combination. Connecting to an external disk enclosure is not supported.

IBM i does not support this combination. An I/O adapter can be located in another enclosure of the system.

cable (FC 1815) cannot be used with this or high availability RAID configuration.

1-meter cable is FC 3686. 3-meters cable is FC 3687.

Chapter 2. Architecture and technical overview 81

2.9.1 Dual split backplane mode

D1 D2 D3

D4 D5 D6

Front view

3 way split backplane pin = 1, drven = 0

Morris and Williams Assembly

Battery card not populated

lanes 0,1

lanes 2,3

Raid = 0

VSES

DVD

Integrated Controller

SAS A

Port

Expander

SAS B

Port

Expander

P7IOC

PCI-e 8x

Integrated Obsidian

Controller

IO connections

Redriver

Dual split backplane mode offers two sets of three disks and is the standard configuration. If you want, one of the sets can be connected to an external SAS PCIe or PCI-X adapter if FC 1819 is selected. Figure 2-20 shows how the six disk bays are shared with the dual split backplane mode. Although solid-state drives (SSDs) are supported with a dual split backplane configuration, mixing SSDs and hard disk drives HDDs in the same split domain is not supported. Also, mirroring SSDs with HDDs is not possible, or vice versa.

Figure 2-20 Dual split backplane overview

82 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

2.9.2 Triple split backplane

Option cable FC 1815 for 3 way split

D1 D2 D3

D4 D5 D6

Front view

3 way split backplane pin = 1, drven = 0

Morris and Williams Assembly

Battery card not populated

lanes 0,1

lanes 2,3

Raid = 0

VSES

DVD

Integrated

Controller-E

Integrated

Controller

SAS A

Port

Expander

SAS B

Port

Expander

P7IOC

PCI-e 8x

Integrated Obsidian

Controller

IO connections

Redriver

SAS 4x connector Key = 2

Cadet-E

AI cable FC3679

The triple split backplane mode offers three sets of two disk drives each. This mode requires an internal SAS Cable (FC 1815), a SAS cable (FC 3679), and a SAS controller, such as SAS Controller FC 5901. Figure 2-21 shows how the six disk bays are shared with the triple split backplane mode. The PCI adapter that drives two of the six disks can be located in the same Power 770 (or Power 780) CEC enclosure as the disk drives or adapter, even in a different system enclosure or external I/O drawer.

Figure 2-21 Triple split backplane overview

2.9.3 Dual storage I/O Adapter (IOA) configurations

Although SSDs are supported with a triple split backplane configuration, mixing SSDs and HDDs in the same split domain is not supported. Also, mirroring SSDs with HDDs is not possible.

The dual storage IOA (FC 1819) configurations are available with either internal or external disk drives from another I/O drawer. SSDs are not supported with this mode.

Chapter 2. Architecture and technical overview 83

If this IOA is selected for an enclosure, selecting SAS cable FC 3686 or FC 3687 to support RAID internal and external drives is necessary (Figure 2-22 on page 84). If this IOA is not selected for the enclosure, the RAID supports only internal enclosure disks.

This configuration increases availability by using dual storage IOA or high availability (HA) to

lanes 0,1

DVD

Integrated

Obsidian-E

Integrated

Obsidian-E

SAS A

Port Expander VSC7158

SAS B

Port Expander VSC7158

P7IOC

PCI-e 8x

Integrated Obsidian

Redriver

Obsidian-E

IO connections

SAS 4x connector

Option pins (2x) Link active pin (1x)

Battery charge

Battery

VSES

lanes 2,3

Redrive

connect multiple adapters to a common set of internal disk drives. It also increases the performance of RAID arrays. The following rules apply to this configuration:

򐂰 This configuration uses the 175 MB Cache RAID - Dual IOA Enablement Card (FC 5662). 򐂰 Using the dual IOA enablement card, the two embedded adapters can connect to each

other and to all six disk drives, and also the 12 disk drives in an external disk drive enclosure, if one is used.

򐂰 The disk drives are required to be in RAID arrays. 򐂰 There are no separate SAS cables required to connect the two embedded SAS RAID

adapters to each other. The connection is contained within the backplane.

򐂰 RAID 0, 10, 5, and 6 support up to six drives. 򐂰 SSDs and HDDs can be used, but can never be mixed in the same disk enclosure. 򐂰 To connect to the external storage, you need to connect to the FC 5886 disk drive

enclosure.

Figure 2-22 shows the topology of the RAID mode.

Figure 2-22 RAID mode (external disk drives option)

2.9.4 DVD

The DVD media bay is directly connected to the integrated SAS controller on the I/O backplane and has a specific chip (VSES) for controlling the DVD LED and power. The VSES appears as a separate device to the device driver and operating systems (Figure 2-19 on page 80).

84 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

Because the integrated SAS controller is independent from the two SAS disk or SSD controllers, it allows the DVD to be switched between multiple partitions without impacting the assignment of disks or SSDs in the CEC enclosure.

2.10 External I/O subsystems

This section describes the external 12X I/O subsystems that can be attached to the Power 770 and Power 780, listed as follows:

򐂰 PCI-DDR 12X Expansion Drawer (FC 5796) 򐂰 12X I/O Drawer PCIe, small form factor (SFF) disk (FC 5802) 򐂰 12X I/O Drawer PCIe, No Disk (FC 5877)

Table 2-23 provides an overview of all the supported I/O drawers.

Table 2-23 I/O drawer capabilities

Feature code

5796 0 6 x PCI-X GX++ adapter card

5802 18 x SAS disk drive bays 10 x PCIe GX++ adapter card

5877 0 10 x PCIe GX++ adapter card

DASD PCI slots Requirements for the

The two GX++ buses from the second processor card feed two GX++ adapter slots. An optional GX++ 12X DDR Adapter, Dual-port (FC 1808), which is installed in GX++ adapter slot, enables the attachment of a 12X loop, which runs at either SDR or DDR speed, depending on the 12X I/ O drawers that are attached.

2.10.1 PCI-DDR 12X Expansion Drawer

The PCI-DDR 12X Expansion Drawer (FC 5796) is a 4U (EIA units) drawer and mounts in a 19-inch rack. It is 224 mm (8.8 in.) wide and takes up half the width of the 4U (EIA units) rack space. The 4U enclosure can hold up to two PCI-DDR 12X Expansion Drawer drawers mounted side-by-side in the enclosure. The drawer is 800 mm (31.5 in.) deep and can weigh up to 20 kg (44 lb).

Power 770 and Power 780

(FC 1808 CCIN 2BC3)

The PCI-DDR 12X Expansion Drawer has six 64-bit, 3.3 V, PCI-X DDR slots, running at 266 MHz, that use blind-swap cassettes and support hot-plugging of adapter cards. The drawer includes redundant hot-plug power and cooling.

Two interface adapters are available for use in the drawer:

򐂰 Dual-Port 12X Channel Attach Adapter Long Run (FC 6457 CCIN 520A) 򐂰 Dual-Port 12X Channel Attach Adapter Short Run (FC 6446 CCIN 520B)

The adapter selection is based on how close the host system or the next I/O drawer in the loop is physically located. The drawer attaches to a host system CEC enclosure with a 12X adapter in a GX++ slot through SDR or DDR cables (or both SDR and DDR cables). A maximum of four drawers can be placed on the same 12X loop. Mixing drawers as FC 5802, FC 5877, and FC 5796 on the same loop is not supported.

Chapter 2. Architecture and technical overview 85

A minimum configuration of two 12X cables (either SDR or DDR), two AC power cables, and

12X Port 1 (P1-C7-T2)

P1-C8-T3 P1-C1

P1-C2

P1-C3

P1-C4

P1-C5

P1-C6

12X Port 0

(P1-C7-T1)

SPCN 0 (P1-C8-T1)

SPCN 1 (P1-C8-T2)

two SPCN cables is required to ensure proper redundancy.

Figure 2-23 shows the rear view of the expansion unit.

Figure 2-23 PCI-X DDR 12X Expansion Drawer rear view

2.10.2 12X I/O Drawer PCIe

The 12X I/O Drawer PCIe is a 19-inch I/O and storage drawer. It provides a 4U-tall (EIA units) drawer containing 10 PCIe-based I/O adapter slots and 18 SAS hot-swap small form factor disk bays, which can be used for either disk drives or SSD (FC 5802). The adapter slots use blind-swap cassettes and support hot-plugging of adapter cards.

A maximum of two drawers can be placed on the same 12X loop. The 12X I/O drawer (FC 5877) is the same as this drawer (FC 5802) except that it does not support any disk bays. Drawer FC 5877 can be on the same loop as drawer FC 5802. Drawer (FC 5877) cannot be upgraded to a drawer that contains the disks also, that is, drawer FC 5802.

The physical dimensions of the drawer are 444.5 mm (17.5 in.) wide by 177.8 mm (7.0 in.) high by 711.2 mm (28.0 in.) deep for use in a 19-inch rack.

A minimum configuration of two 12X DDR cables, two AC power cables, and two SPCN cables is required to ensure proper redundancy. The drawer attaches to the host CEC enclosure with a 12X adapter in a GX++ slot through 12X DDR cables that are available in various cable lengths:

򐂰 0.6 m (FC 1861) 򐂰 1.5 m (FC 1862) 򐂰 3.0 m (FC 1865) 򐂰 8 m (FC 1864)

The 12X SDR cables are not supported on this drawer.

86 IBM Power 770 and 780 (9117-MMD, 9179-MHD) Technical Overview and Introduction

IBM Power 780, Power 770 Technical Overview And Introduction

Specifications and Main Features

Frequently Asked Questions

User Manual