IBM z13s Technical Manual

Front cover

IBM z13s Technical Guide

Octavian Lascu Barbara Sannerud Cecilia A. De Leon Edzard Hoogerbrug Ewerson Palacio Franco Pinto Jin J. Yang John P. Troy Martin Soellig

In partnership with

IBM Academy of Technology

Redbooks

International Technical Support Organization

IBM z13s Technical Guide

June 2016

SG24-8294-00

Note: Before using this information and the product it supports, read the information in “Notices” on page xix.

First Edition (June 2016)

This edition applies to IBM z13s servers.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xx

IBM Redbooks promotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

Authors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv

Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi

Chapter 1. Introducing IBM z13s servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Overview of IBM z13s servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 z13s servers highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1 Processor and memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.2 Capacity and performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.3 I/O subsystem and I/O features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.4 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2.5 Reliability, availability, and serviceability design. . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3 z13s technical overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3.1 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.3.2 Model upgrade paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.3.3 Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.3.4 CPC drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.3.5 I/O connectivity: PCIe and InfiniBand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.3.6 I/O subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.3.7 Parallel Sysplex Coupling and Server Time Protocol connectivity . . . . . . . . . . . . 20

1.3.8 Special-purpose features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.3.9 Reliability, availability, and serviceability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

1.4 Hardware Management Consoles and Support Elements . . . . . . . . . . . . . . . . . . . . . . 27

1.5 IBM z BladeCenter Extension (zBX) Model 004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

1.5.1 Blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.5.2 IBM WebSphere DataPower Integration Appliance XI50 for zEnterprise . . . . . . . 28

1.6 IBM z Unified Resource Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.7 IBM Dynamic Partition Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.8 Operating systems and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.8.1 Supported operating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.8.2 IBM compilers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Chapter 2. Central processor complex hardware components . . . . . . . . . . . . . . . . . . 35

2.1 Frame and drawers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.1.1 The z13s frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.1.2 PCIe I/O drawer and I/O drawer features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.2 Processor drawer concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.2.1 CPC drawer interconnect topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.2 Oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.3 System control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.2.4 CPC drawer power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3 Single chip modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3.1 Processor units and system control chips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.3.2 Processor unit (PU) chip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.3.3 Processor unit (core). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3.4 PU characterization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

2.3.5 System control chip. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.3.6 Cache level structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.4 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.4.1 Memory subsystem topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

2.4.2 Redundant array of independent memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.4.3 Memory configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.4.4 Memory upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

2.4.5 Preplanned memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

2.5 Reliability, availability, and serviceability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2.5.1 RAS in the CPC memory subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2.5.2 General z13s RAS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

2.6 Connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

2.6.1 Redundant I/O interconnect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

2.7 Model configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2.7.1 Upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.7.2 Concurrent PU conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.7.3 Model capacity identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.7.4 Model capacity identifier and MSU values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

2.7.5 Capacity BackUp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.7.6 On/Off Capacity on Demand and CPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

2.8 Power and cooling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

2.8.1 Power considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

2.8.2 High-voltage DC power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

2.8.3 Internal Battery Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

2.8.4 Power capping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

2.8.5 Power Estimation tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

2.8.6 Cooling requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

2.9 Summary of z13s structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Chapter 3. Central processor complex system design . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.2 Design highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.3 CPC drawer design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.3.1 Cache levels and memory structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.3.2 CPC drawer interconnect topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.4 Processor unit design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.4.1 Simultaneous multithreading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

3.4.2 Single-instruction multiple-data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

3.4.3 Out-of-order execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.4.4 Superscalar processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

3.4.5 Compression and cryptography accelerators on a chip . . . . . . . . . . . . . . . . . . . . 95

3.4.6 Decimal floating point accelerator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.4.7 IEEE floating point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.4.8 Processor error detection and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.4.9 Branch prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3.4.10 Wild branch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3.4.11 Translation lookaside buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

iv IBM z13s Technical Guide

3.4.12 Instruction fetching, decoding, and grouping . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

3.4.13 Extended Translation Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

3.4.14 Instruction set extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.4.15 Transactional Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.4.16 Runtime Instrumentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.5 Processor unit functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.5.2 Central processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

3.5.3 Integrated Facility for Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3.5.4 Internal Coupling Facility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3.5.5 IBM z Integrated Information Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

3.5.6 System assist processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

3.5.7 Reserved processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.5.8 Integrated firmware processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.5.9 Processor unit assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.5.10 Sparing rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

3.5.11 Increased flexibility with z/VM mode partitions . . . . . . . . . . . . . . . . . . . . . . . . . 112

3.6 Memory design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

3.6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

3.6.2 Main storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

3.6.3 Expanded storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

3.6.4 Hardware system area (HSA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

3.7 Logical partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

3.7.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

3.7.2 Storage operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

3.7.3 Reserved storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

3.7.4 Logical partition storage granularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

3.7.5 LPAR dynamic storage reconfiguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.8 Intelligent Resource Director. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.9 Clustering technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

3.9.1 Coupling Facility Control Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

3.9.2 Coupling Thin Interrupts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

3.9.3 Dynamic CF dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

3.9.4 CFCC and Flash Express use. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Chapter 4. Central processor complex I/O system structure . . . . . . . . . . . . . . . . . . . 137

4.1 Introduction to the InfiniBand and PCIe for I/O infrastructure . . . . . . . . . . . . . . . . . . . 138

4.1.1 InfiniBand I/O infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.1.2 PCIe I/O infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.1.3 InfiniBand specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

4.1.4 PCIe Generation 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

4.2 I/O system overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

4.2.1 Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

4.2.2 Summary of supported I/O features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

4.3 I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

4.4 PCIe I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

4.5 PCIe I/O drawer and I/O drawer offerings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

4.6 Fanouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

4.6.1 PCIe Generation 3 fanout (FC 0173) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

4.6.2 HCA2-C fanout (FC 0162). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

4.6.3 Integrated Coupling Adapter (FC 0172) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

4.6.4 HCA3-O (12x IFB) fanout (FC 0171). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

4.6.5 HCA3-O LR (1x IFB) fanout (FC 0170). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Contents v

4.6.6 Fanout considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

4.7 I/O features (cards) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

4.7.1 I/O feature card ordering information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

4.7.2 Physical channel report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

4.8 Connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

4.8.1 I/O feature support and configuration rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

4.8.2 FICON channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

4.8.3 OSA-Express5S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

4.8.4 OSA-Express4S features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

4.8.5 OSA-Express for ensemble connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

4.8.6 HiperSockets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

4.9 Parallel Sysplex connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

4.9.1 Coupling links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

4.9.2 Migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

4.9.3 Pulse per second input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.10 Cryptographic functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.10.1 CPACF functions (FC 3863) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.10.2 Crypto Express5S feature (FC 0890) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.11 Integrated firmware processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.12 Flash Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

4.12.1 IBM Flash Express read/write cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

4.13 10GbE RoCE Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

4.14 zEDC Express. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Chapter 5. Central processor complex channel subsystem. . . . . . . . . . . . . . . . . . . . 187

5.1 Channel subsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

5.1.1 Multiple logical channel subsystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

5.1.2 Multiple subchannel sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

5.1.3 Channel path spanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

5.2 I/O configuration management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

5.3 Channel subsystem summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Chapter 6. Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

6.1 Cryptography in IBM z13 and z13s servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

6.2 Some fundamentals on cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

6.2.1 Modern cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

6.2.2 Kerckhoffs’ principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

6.2.3 Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

6.2.4 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

6.3 Cryptography on IBM z13s servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

6.4 CP Assist for Cryptographic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

6.4.1 Cryptographic synchronous functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

6.4.2 CPACF protected key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

6.5 Crypto Express5S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

6.5.1 Cryptographic asynchronous functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

6.5.2 Crypto Express5S as a CCA coprocessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

6.5.3 Crypto Express5S as an EP11 coprocessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

6.5.4 Crypto Express5S as an accelerator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

6.5.5 Management of Crypto Express5S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

6.6 TKE workstation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

6.6.1 Logical partition, TKE host, and TKE target . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

6.6.2 Optional smart card reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

6.6.3 TKE workstation with Licensed Internal Code 8.0. . . . . . . . . . . . . . . . . . . . . . . . 223

vi IBM z13s Technical Guide

6.6.4 TKE workstation with Licensed Internal Code 8.1. . . . . . . . . . . . . . . . . . . . . . . . 223

6.6.5 TKE hardware support and migration information. . . . . . . . . . . . . . . . . . . . . . . . 224

6.7 Cryptographic functions comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

6.8 Cryptographic software support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Chapter 7. Software support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

7.1 Operating systems summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

7.2 Support by operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

7.2.1 z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

7.2.2 z/VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

7.2.3 z/VSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

7.2.4 z/TPF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

7.2.5 Linux on z Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

7.2.6 KVM for IBM z Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

7.2.7 z13s function support summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

7.3 Support by function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

7.3.1 Single system image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

7.3.2 zIIP support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

7.3.3 Transactional Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

7.3.4 Maximum main storage size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

7.3.5 Flash Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

7.3.6 z Enterprise Data Compression Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

7.3.7 10GbE RoCE Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

7.3.8 Large page support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

7.3.9 Hardware decimal floating point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

7.3.10 Up to 40 LPARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

7.3.11 Separate LPAR management of PUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

7.3.12 Dynamic LPAR memory upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

7.3.13 LPAR physical capacity limit enforcement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

7.3.14 Capacity Provisioning Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

7.3.15 Dynamic PU add . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

7.3.16 HiperDispatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

7.3.17 The 63.75-K subchannels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

7.3.18 Multiple Subchannel Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

7.3.19 Three subchannel sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

7.3.20 IPL from an alternative subchannel set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

7.3.21 Modified Indirect Data Address Word facility . . . . . . . . . . . . . . . . . . . . . . . . . . 257

7.3.22 HiperSockets Completion Queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

7.3.23 HiperSockets integration with the intraensemble data network . . . . . . . . . . . . 258

7.3.24 HiperSockets Virtual Switch Bridge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

7.3.25 HiperSockets Multiple Write Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

7.3.26 HiperSockets IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

7.3.27 HiperSockets Layer 2 support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

7.3.28 HiperSockets network traffic analyzer for Linux on z Systems . . . . . . . . . . . . . 260

7.3.29 FICON Express16S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

7.3.30 FICON Express8S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

7.3.31 FICON Express8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

7.3.32 z/OS Discovery and Auto-Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

7.3.33 High-performance FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

7.3.34 Request node identification data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

7.3.35 32 K subchannels for the FICON Express16S . . . . . . . . . . . . . . . . . . . . . . . . . 266

7.3.36 Extended distance FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

7.3.37 Platform and name server registration in FICON channel . . . . . . . . . . . . . . . . 266

Contents vii

7.3.38 FICON link incident reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

7.3.39 FCP provides increased performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

7.3.40 N_Port ID Virtualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

7.3.41 OSA-Express5S 10-Gigabit Ethernet LR and SR . . . . . . . . . . . . . . . . . . . . . . . 268

7.3.42 OSA-Express5S Gigabit Ethernet LX and SX. . . . . . . . . . . . . . . . . . . . . . . . . . 268

7.3.43 OSA-Express5S 1000BASE-T Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

7.3.44 OSA-Express4S 10-Gigabit Ethernet LR and SR . . . . . . . . . . . . . . . . . . . . . . . 270

7.3.45 OSA-Express4S Gigabit Ethernet LX and SX. . . . . . . . . . . . . . . . . . . . . . . . . . 271

7.3.46 OSA-Express4S 1000BASE-T Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

7.3.47 Open Systems Adapter for IBM zAware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

7.3.48 Open Systems Adapter for Ensemble. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

7.3.49 Intranode management network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

7.3.50 Intraensemble data network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

7.3.51 OSA-Express5S and OSA-Express4S NCP support . . . . . . . . . . . . . . . . . . . . 274

7.3.52 Integrated Console Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

7.3.53 VLAN management enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

7.3.54 GARP VLAN Registration Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

7.3.55 Inbound workload queuing for OSA-Express5S and OSA-Express4S . . . . . . . 276

7.3.56 Inbound workload queuing for Enterprise Extender . . . . . . . . . . . . . . . . . . . . . 277

7.3.57 Querying and displaying an OSA configuration . . . . . . . . . . . . . . . . . . . . . . . . 277

7.3.58 Link aggregation support for z/VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

7.3.59 Multi-VSwitch Link Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

7.3.60 QDIO data connection isolation for z/VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

7.3.61 QDIO interface isolation for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

7.3.62 QDIO optimized latency mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

7.3.63 Large send for IPv6 packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

7.3.64 OSA-Express5S and OSA-Express4S checksum offload. . . . . . . . . . . . . . . . . 280

7.3.65 Checksum offload for IPv4and IPv6 packets when in QDIO mode. . . . . . . . . . 280

7.3.66 Adapter interruptions for QDIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

7.3.67 OSA Dynamic LAN idle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

7.3.68 OSA Layer 3 virtual MAC for z/OS environments . . . . . . . . . . . . . . . . . . . . . . . 281

7.3.69 QDIO Diagnostic Synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

7.3.70 Network Traffic Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

7.3.71 Program-directed re-IPL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

7.3.72 Coupling over InfiniBand and Integrated Coupling Adapter . . . . . . . . . . . . . . . 282

7.3.73 Dynamic I/O support for InfiniBand and ICA CHPIDs . . . . . . . . . . . . . . . . . . . . 283

7.3.74 Simultaneous multithreading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

7.3.75 Single Instruction Multiple Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

7.3.76 Shared Memory Communication - Direct Memory Access . . . . . . . . . . . . . . . . 284

7.4 Cryptographic support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

7.4.1 CP Assist for Cryptographic Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

7.4.2 Crypto Express5S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

7.4.3 Web deliverables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

7.4.4 z/OS Integrated Cryptographic Service Facility FMIDs. . . . . . . . . . . . . . . . . . . . 286

7.4.5 ICSF migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

7.5 GDPS Virtual Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

7.6 z/OS migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

7.6.1 General guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

7.6.2 Hardware configuration definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

7.6.3 Coupling links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

7.6.4 Large page support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

7.6.5 Capacity Provisioning Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

7.6.6 Decimal floating point and z/OS XL C/C++ considerations. . . . . . . . . . . . . . . . . 291

viii IBM z13s Technical Guide

7.7 IBM z Advanced Workload Analysis Reporter (zAware) . . . . . . . . . . . . . . . . . . . . . . . 291

7.7.1 z Appliance Container Infrastructure mode LPAR . . . . . . . . . . . . . . . . . . . . . . . 292

7.8 Coupling facility and CFCC considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

7.8.1 CFCC Level 21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

7.8.2 Flash Express exploitation by CFCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

7.8.3 CFCC Coupling Thin Interrupts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

7.9 Simultaneous multithreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

7.10 Single-instruction multiple-data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

7.11 The MIDAW facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

7.11.1 MIDAW technical description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

7.11.2 Extended format data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

7.11.3 Performance benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

7.12 IOCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

7.13 Worldwide port name tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

7.14 ICKDSF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

7.15 IBM z BladeCenter Extension (zBX) Model 004 software support . . . . . . . . . . . . . . 304

7.15.1 IBM Blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

7.15.2 IBM WebSphere DataPower Integration Appliance XI50 for zEnterprise . . . . . 304

7.16 Software licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

7.16.1 Software licensing considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

7.16.2 Monthly license charge pricing metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

7.16.3 Advanced Workload License Charges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

7.16.4 Advanced Entry Workload License Charge . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

7.16.5 System z New Application License Charges. . . . . . . . . . . . . . . . . . . . . . . . . . . 308

7.16.6 Midrange workload license charges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

7.16.7 Parallel Sysplex License Charges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

7.16.8 z Systems International Program License Agreement . . . . . . . . . . . . . . . . . . . 309

7.16.9 zBX licensed software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

7.17 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Chapter 8. System upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

8.1 Upgrade types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

8.1.1 Overview of upgrade types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

8.1.2 Terminology related to CoD for z13s systems . . . . . . . . . . . . . . . . . . . . . . . . . . 313

8.1.3 Permanent upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

8.1.4 Temporary upgrades. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

8.2 Concurrent upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

8.2.1 Model upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

8.2.2 Customer Initiated Upgrade facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

8.2.3 Summary of concurrent upgrade functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322

8.3 Miscellaneous equipment specification upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

8.3.1 MES upgrade for processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

8.3.2 MES upgrade for memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

8.3.3 Preplanned Memory feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

8.3.4 MES upgrades for the zBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

8.4 Permanent upgrade through the CIU facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328

8.4.1 Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330

8.4.2 Retrieval and activation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331

8.5 On/Off Capacity on Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

8.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

8.5.2 Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

8.5.3 On/Off CoD testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

8.5.4 Activation and deactivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

Contents ix

8.5.5 Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

8.5.6 IBM z/OS capacity provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

8.6 Capacity for Planned Event. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

8.7 Capacity BackUp. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

8.7.1 Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

8.7.2 CBU activation and deactivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347

8.7.3 Automatic CBU for Geographically Dispersed Parallel Sysplex . . . . . . . . . . . . . 349

8.8 Nondisruptive upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

8.8.1 Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8.8.2 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8.8.3 I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8.8.4 Cryptographic adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8.8.5 Special features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8.8.6 Concurrent upgrade considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8.9 Summary of capacity on-demand offerings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Chapter 9. Reliability, availability, and serviceability. . . . . . . . . . . . . . . . . . . . . . . . . . 355

9.1 The RAS strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

9.2 Availability characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

9.3 RAS functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

9.3.1 Unscheduled outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

9.3.2 Scheduled outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359

9.4 Enhanced Driver Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

9.5 RAS capability for the HMC and SE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362

9.6 RAS capability for zBX Model 004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363

BladeCenter components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363

zBX firmware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

9.6.1 zBX RAS and the IBM z Unified Resource Manager . . . . . . . . . . . . . . . . . . . . . 364

9.6.2 zBX Model 004: 2458-004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

9.7 Considerations for PowerHA in zBX environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . 366

9.8 IBM z Advanced Workload Analysis Reporter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

9.9 RAS capability for Flash Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

Chapter 10. Environmental requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369

10.1 IBM z13s power and cooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

10.1.1 Power and I/O cabling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

10.1.2 Power consumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

10.1.3 Internal Battery Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

10.1.4 Balanced Power Plan Ahead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

10.1.5 Emergency power-off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

10.1.6 Cooling requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

10.1.7 New environmental class for IBM z13s servers: ASHREA Class A3 . . . . . . . . 374

10.2 IBM z13s physical specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

10.2.1 Weights and dimensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

10.2.2 Four-in-one (4-in-1) bolt-down kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

10.3 IBM zBX environmental components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

10.3.1 IBM zBX configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

10.3.2 IBM zBX power components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

10.3.3 IBM zBX cooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377

10.3.4 IBM zBX physical specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

10.4 Energy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

10.4.1 Power estimation tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

10.4.2 Query maximum potential power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

x IBM z13s Technical Guide

10.4.3 System Activity Display and Monitors Dashboard. . . . . . . . . . . . . . . . . . . . . . . 382

10.4.4 IBM Systems Director Active Energy Manager . . . . . . . . . . . . . . . . . . . . . . . . . 383

10.4.5 Unified Resource Manager: Energy management . . . . . . . . . . . . . . . . . . . . . . 384

Chapter 11. Hardware Management Console and Support Elements . . . . . . . . . . . . 387

11.1 Introduction to the HMC and SE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388

11.2 HMC and SE enhancements and changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388

11.2.1 Driver Level 27 HMC and SE enhancements and changes . . . . . . . . . . . . . . . 389

11.2.2 Rack-mounted HMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392

11.2.3 New Support Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

11.2.4 New backup options for HMCs and primary SEs . . . . . . . . . . . . . . . . . . . . . . . 393

11.2.5 SE driver support with the HMC driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

11.2.6 HMC feature codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

11.2.7 Tree Style User Interface and Classic Style User Interface . . . . . . . . . . . . . . . 398

11.3 HMC and SE connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

11.3.1 Network planning for the HMC and SE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

11.3.2 Hardware prerequisite changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

11.3.3 RSF is broadband-only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

11.3.4 TCP/IP Version 6 on the HMC and SE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

11.3.5 Assigning addresses to the HMC and SE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

11.4 Remote Support Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

11.4.1 Security characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

11.4.2 RSF connections to IBM and Enhanced IBM Service Support System . . . . . . 404

11.4.3 HMC and SE remote operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

11.5 HMC and SE key capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

11.5.1 Central processor complex management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

11.5.2 Logical partition management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406

11.5.3 Operating system communication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

11.5.4 HMC and SE microcode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

11.5.5 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

11.5.6 Capacity on demand support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

11.5.7 Features on Demand support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416

11.5.8 Server Time Protocol support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416

11.5.9 NTP client and server support on the HMC . . . . . . . . . . . . . . . . . . . . . . . . . . . 420

11.5.10 Security and user ID management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422

11.5.11 System Input/Output Configuration Analyzer on the SE and HMC. . . . . . . . . 424

11.5.12 Automated operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

11.5.13 Cryptographic support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425

11.5.14 Installation support for z/VM using the HMC. . . . . . . . . . . . . . . . . . . . . . . . . . 426

11.5.15 Dynamic Partition Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

11.6 HMC in an ensemble. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

11.6.1 Unified Resource Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

11.6.2 Ensemble definition and management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432

11.6.3 HMC availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

11.6.4 Considerations for multiple HMCs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

11.6.5 HMC browser session to a primary HMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

11.6.6 HMC ensemble topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

Chapter 12. Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437

12.1 Performance information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438

12.2 LSPR workload suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

12.3 Fundamental components of workload capacity performance . . . . . . . . . . . . . . . . . 439

12.3.1 Instruction path length. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

Contents xi

12.3.2 Instruction complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440

12.3.3 Memory hierarchy and memory nest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440

12.4 Relative nest intensity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

12.5 LSPR workload categories based on relative nest intensity . . . . . . . . . . . . . . . . . . . 443

12.6 Relating production workloads to LSPR workloads . . . . . . . . . . . . . . . . . . . . . . . . . 444

12.7 Workload performance variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

12.7.1 Main performance improvement drivers with z13s . . . . . . . . . . . . . . . . . . . . . . 446

Appendix A. IBM z Appliance Container Infrastructure. . . . . . . . . . . . . . . . . . . . . . . . 449

A.1 What is zACI? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

A.2 Why use zACI? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

A.3 IBM z Systems servers and zACI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

12.7.2 Example: Deploying IBM zAware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

Appendix B. IBM z Systems Advanced Workload Analysis Reporter (IBM zAware). 453

B.1 Troubleshooting in complex IT environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

B.2 Introducing IBM zAware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

B.2.1 Hardware requirements overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456

B.2.2 Value of IBM zAware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456

B.2.3 IBM z/OS Solutions to improve problem diagnostic procedures. . . . . . . . . . . . . 457

B.3 Understanding IBM zAware technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458

B.3.1 Training period . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

B.3.2 Priming IBM zAware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

B.3.3 IBM zAware ignore message support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

B.3.4 IBM zAware graphical user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

B.3.5 IBM zAware complements your existing tools . . . . . . . . . . . . . . . . . . . . . . . . . . 464

B.4 IBM zAware prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

B.4.1 IBM zAware features and ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

12.7.3 Feature on Demand (FoD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

B.4.2 IBM zAware operating requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

B.5 Configuring and using IBM zAware virtual appliance . . . . . . . . . . . . . . . . . . . . . . . . . 468

Appendix C. Channel options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471

C.1 Channel options supported on z13s servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

C.2 Maximum unrepeated distance for FICON SX features . . . . . . . . . . . . . . . . . . . . . . . 473

Appendix D. Shared Memory Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475

D.1 Shared Memory Communications overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

D.2 Shared Memory Communication over RDMA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

D.2.1 RDMA technology overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

D.2.2 Shared Memory Communications over RDMA. . . . . . . . . . . . . . . . . . . . . . . . . . 477

D.2.3 Single Root I/O Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

D.2.4 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

D.2.5 10GbE RoCE Express feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

D.2.6 10GbE RoCE Express configuration example . . . . . . . . . . . . . . . . . . . . . . . . . . 481

D.2.7 Hardware configuration definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483

D.2.8 Software exploitation of SMC-R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484

D.2.9 SMC-R support overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

D.2.10 SMC-R use cases for z/OS to z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

D.2.11 Enabling SMC-R support in z/OS Communications Server . . . . . . . . . . . . . . . 488

D.3 Shared Memory Communications - Direct Memory Access . . . . . . . . . . . . . . . . . . . . 489

D.3.1 Internal Shared Memory technology overview . . . . . . . . . . . . . . . . . . . . . . . . . . 490

D.3.2 SMC-D over Internal Shared Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490

D.3.3 Internal Shared Memory - Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492

xii IBM z13s Technical Guide

D.3.4 Virtual PCI Function (vPCI Adapter). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492

D.3.5 Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495

D.3.6 Hardware configuration definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495

D.3.7 Sample IOCP FUNCTION statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496

D.3.8 Software exploitation of ISM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498

D.3.9 SMC-D over ISM prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498

D.3.10 Enabling SMC-D support in z/OS Communications Server . . . . . . . . . . . . . . . 499

D.3.11 SMC-D support overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500

Appendix E. IBM Dynamic Partition Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501

E.1 What is IBM Dynamic Partition Manager? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502

E.2 Why use DPM?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502

E.3 IBM z Systems servers and DPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

E.4 Setting up the DPM environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504

E.4.1 Defining partitions in DPM mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507

E.4.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510

Appendix F. KVM for IBM z Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511

F.1 Why KVM for IBM z Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512

F.1.1 Advantages of using KVM for IBM z Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 512

F.2 IBM z Systems servers and KVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513

F.2.1 Storage connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

F.2.2 Network connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

F.2.3 Hardware Management Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515

F.2.4 Open source virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515

F.2.5 What comes with KVM for IBM z Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516

F.3 Managing the KVM for IBM z Systems environment. . . . . . . . . . . . . . . . . . . . . . . . . . 518

F.3.1 Hypervisor Performance Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

F.4 Using IBM Cloud Manager with OpenStack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

Appendix G. Native Peripheral Component Interconnect Express . . . . . . . . . . . . . . 521

G.1 Design of native PCIe I/O adapter management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

G.1.1 Native PCIe adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

G.1.2 Integrated firmware processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

G.1.3 Resource groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

G.1.4 Management tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524

G.2 Native PCIe feature plugging rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524

G.3 Native PCIe feature definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525

G.3.1 FUNCTION identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527

G.3.2 Virtual function number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527

G.3.3 Physical network identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527

Appendix H. Flash Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529

H.1 Flash Express overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530

H.2 Using Flash Express. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532

H.3 Security on Flash Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535

H.3.1 Integrated Key Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535

H.3.2 Key serving topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537

H.3.3 Error recovery scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538

Appendix I. GDPS Virtual Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541

I.1 GDPS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542

I.2 Overview of GDPS Virtual Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544

I.3 GDPS Virtual Appliance recovery scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547

Contents xiii

I.3.1 Planned disk outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547

I.3.2 Unplanned disk outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548

I.3.3 Disaster recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549

Appendix J. IBM zEnterprise Data Compression Express . . . . . . . . . . . . . . . . . . . . . 551

J.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552

J.2 zEDC Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552

J.3 Software support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553

J.3.1 z/VM V6R3 support with PTFs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554

J.3.2 IBM SDK 7 for z/OS Java support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554

J.3.3 IBM z Systems Batch Network Analyzer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

xiv IBM z13s Technical Guide

Figures

2-1 z13s frame: Rear and front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2-2 z13s (one CPC drawer) I/O drawer configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2-3 z13s (two CPC Drawers) I/O Drawer Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2-4 Model N10 components (top view) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2-5 Model N20 components (top view) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2-6 CPC drawer (front view) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2-7 CPC drawer logical structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2-8 Drawer to drawer communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2-9 Oscillators cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2-10 Conceptual overview of system control elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2-11 Redundant DCAs and blowers for CPC drawers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2-12 Single chip modules (PU SCM and SC SCM) N20 CPC Drawer . . . . . . . . . . . . . . . . 47

2-13 PU Chip Floorplan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2-14 PU Core floorplan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

2-15 SC chip diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2-16 Cache level structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2-17 CPC drawer memory topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

2-18 RAIM configuration per node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2-19 Model N10 memory plug locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2-20 Model N20 one drawer memory plug locations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2-21 Model N20 two drawer memory plug locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2-22 Memory allocation diagram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2-23 Model N10 drawer: Location of the PCIe and IFB fanouts . . . . . . . . . . . . . . . . . . . . . 66

2-24 Model N20 two CPC drawer: Locations of the PCIe and IFB fanouts. . . . . . . . . . . . . 66

2-25 Redundant I/O interconnect for I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2-26 Redundant I/O interconnect for PCIe I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2-27 z13s upgrade paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3-1 z13s cache levels and memory hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3-2 z13s cache topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3-3 z13s and zBC12 cache level comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

3-4 z13s CPC drawer communication topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3-5 Point-to-point topology z13s two CPC drawers communication . . . . . . . . . . . . . . . . . . 88

3-6 Two threads running simultaneously on the same processor core. . . . . . . . . . . . . . . . 90

3-7 Schematic representation of add SIMD instruction with 16 elements in each vector . . 91

3-8 Floating point registers overlaid by vector registers . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3-9 z13s PU core logical diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

3-10 z13s in-order and out-of-order core execution improvements . . . . . . . . . . . . . . . . . . 94

3-11 Compression and cryptography accelerators on a core in the chip . . . . . . . . . . . . . . 96

3-12 PU error detection and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3-13 ICF options: Shared ICFs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

3-14 Logical flow of Java code execution on a zIIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

3-15 IRD LPAR cluster example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3-16 Sysplex hardware overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

3-17 Dynamic CF dispatching (shared CPs or shared ICF PUs) . . . . . . . . . . . . . . . . . . . 134

4-1 I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

4-2 I/O domains of an I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

4-3 PCIe I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4-4 z13s (N20) connectivity to PCIe I/O drawers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

4-5 PCIe I/O drawer with 32 PCIe slots and four I/O domains . . . . . . . . . . . . . . . . . . . . . 146

4-6 Infrastructure for PCIe and InfiniBand coupling links . . . . . . . . . . . . . . . . . . . . . . . . . 148

4-7 OM3 50/125 µm multimode fiber cable with MPO connectors . . . . . . . . . . . . . . . . . . 150

4-8 z13 Parallel Sysplex coupling connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

4-9 CPC drawer front view: Coupling links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

4-10 HCA3-O Fanouts: z13s versus z114 / zBC12 servers . . . . . . . . . . . . . . . . . . . . . . . 180

4-11 PCIe I/O drawer that is fully populated with Flash Express cards. . . . . . . . . . . . . . . 185

5-1 Multiple channel subsystems and multiple subchannel sets. . . . . . . . . . . . . . . . . . . . 188

5-2 Output for display ios,config(all) command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

5-3 z Systems CSS: Channel subsystems with channel spanning . . . . . . . . . . . . . . . . . . 194

5-4 CSS, LPAR, and identifier example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

6-1 Three levels of protection with three levels of speed. . . . . . . . . . . . . . . . . . . . . . . . . . 203

6-2 Cryptographic hardware supported on IBM z13s servers . . . . . . . . . . . . . . . . . . . . . . 204

6-3 z13s cryptographic support in z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

6-4 The cryptographic coprocessor CPACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

6-5 CPACF key wrapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

6-6 Customize Image Profiles: Crypto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

6-7 SE: View LPAR Cryptographic Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

7-1 Result of the display core command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

7-2 Simultaneous Multithreading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

7-3 IDAW usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

7-4 MIDAW format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

7-5 MIDAW usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

8-1 The provisioning architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

8-2 Example of temporary upgrade activation sequence . . . . . . . . . . . . . . . . . . . . . . . . . 321

8-3 Memory sizes and upgrades for the N10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

8-4 Memory sizes and upgrades for the single drawer N20 . . . . . . . . . . . . . . . . . . . . . . . 326

8-5 Memory sizes and upgrades for the two drawer N20 . . . . . . . . . . . . . . . . . . . . . . . . . 326

8-6 Feature on Demand window for zBX Blades features HWMs. . . . . . . . . . . . . . . . . . . 327

8-7 Permanent upgrade order example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

8-8 CIU-eligible order activation example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330

8-9 Machine profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331

8-10 IBM z13s Perform Model Conversion window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

8-11 Customer Initiated Upgrade Order Activation Number window. . . . . . . . . . . . . . . . . 332

8-12 Order On/Off CoD record window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

8-13 On/Off CoD order example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

8-14 The capacity provisioning infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

8-15 A Capacity Provisioning Domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

8-16 Example of C02 with three CBU features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

8-17 STSI output on z13s server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

9-1 Typical PowerHA cluster diagram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

9-2 Flash Express RAS components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

10-1 IBM z13s cabling options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

10-2 Top Exit I/O cabling feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

10-3 Recommended environmental conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

10-4 Rear Door Heat eXchanger (left) and functional diagram . . . . . . . . . . . . . . . . . . . . . 379

10-5 Top Exit Support for the zBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

10-6 Maximum potential power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

10-7 Power usage on the System Activity Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

11-1 Diagnostic sampling authorization control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

11-2 Change LPAR security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

11-3 Rack-mounted HMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392

11-4 SEs location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

xvi IBM z13s Technical Guide

11-5 Configure backup FTP server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

11-6 Backup Critical Console Data destinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

11-7 Backup Critical Data destinations of SEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

11-8 Scheduled Operation for HMC backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

11-9 Scheduled Operation for SEs backup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

11-10 HMC and SE connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

11-11 SE Physical connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

11-12 HMC connectivity examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

11-13 Change LPAR Group Controls - Group absolute capping . . . . . . . . . . . . . . . . . . . 407

11-14 Manage Trusted Signing Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

11-15 Import Remote Certificate example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

11-16 Configure 3270 Emulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

11-17 Microcode terms and interaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410

11-18 System Information: Bundle level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

11-19 HMC Monitor task group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412

11-20 Monitors Dashboard task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

11-21 Display the activity for an LPAR by processor type . . . . . . . . . . . . . . . . . . . . . . . . 413

11-22 Display the SMT usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

11-23 Monitors Dashboard: Crypto function integration . . . . . . . . . . . . . . . . . . . . . . . . . . 414

11-24 Monitors Dashboard - Flash Express function integration . . . . . . . . . . . . . . . . . . . 414

11-25 Environmental Efficiency Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

11-26 Customize Console Date and Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418

11-27 Timing Network window with Scheduled DST and Scheduled leap second offset . 419

11-28 HMC NTP broadband authentication support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421

11-29 Time coordination for zBX components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422

11-30 Cryptographic Configuration window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

11-31 Enabling Dynamic Partition Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427

11-32 HMC welcome window (CPC in DPM Mode) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

11-33 Unified Resource Manager functions and suites . . . . . . . . . . . . . . . . . . . . . . . . . . 429

11-34 Ensemble example with primary and alternate HMCs . . . . . . . . . . . . . . . . . . . . . . 435

12-1 z13s to zBC12, z114, z10 BC, and z9 BC performance comparison . . . . . . . . . . . . 438

12-2 Memory hierarchy on the z13s one CPC drawer system (two nodes) . . . . . . . . . . . 441

12-3 Relative Nest Intensity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442

12-4 The traditional factors that were used to categorize workloads . . . . . . . . . . . . . . . . 442

12-5 New z/OS workload categories defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

A-1 zACI Framework basic outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

A-2 IBM zAware Image Profile based on zACI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

A-3 zACI icon in the HMC interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

B-1 IBM zAware complements an existing environment . . . . . . . . . . . . . . . . . . . . . . . . . . 456

B-2 IBM zAware shortens the business impact of a problem . . . . . . . . . . . . . . . . . . . . . . 457

B-3 Basic components of the IBM zAware environment . . . . . . . . . . . . . . . . . . . . . . . . . . 459

B-4 HMC Image Profile for an IBM zAware LPAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

B-5 HMC Image Profile for an IBM zAware LPAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

B-6 IBM zAware Heat Map view analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

B-7 IBM zAware bar score with intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

B-8 Feature on Demand window for IBM zAware feature . . . . . . . . . . . . . . . . . . . . . . . . . 466

D-1 RDMA technology overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477

D-2 Dynamic transition from TCP to SMC-R. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

D-3 Shared RoCE mode concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

D-4 10GbE RoCE Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480

D-5 10GbE RoCE Express sample configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481

D-6 RNIC and OSD interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

D-7 Physical network ID example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484

Figures xvii

D-8 Reduced latency and improved wall clock time with SMC-R . . . . . . . . . . . . . . . . . . . 485

D-9 Sysplex Distributor before RoCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

D-10 Sysplex Distributor after RoCE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488

D-11 Connecting two LPARs on the same CPC using SMC-D. . . . . . . . . . . . . . . . . . . . . 490

D-12 Clustered systems: Multitier application solution. RDMA, and DMA . . . . . . . . . . . . 490

D-13 Dynamic transition from TCP to SMC-D by using two OSA-Express adapters . . . . 491

D-14 Concept of vPCI adapter implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493

D-15 SMC-D configuration that uses Ethernet to provide connectivity . . . . . . . . . . . . . . . 494

D-16 SMC-D configuration that uses HiperSockets to provide connectivity . . . . . . . . . . . 495

D-17 ISM adapters shared between LPARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496

D-18 Multiple LPARs connected through multiple VLANs. . . . . . . . . . . . . . . . . . . . . . . . . 497

D-19 SMCD parameter in GLOBALCONFIG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499

E-1 High-level view of DPM implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

E-2 Enabling DPM mode of operation from the SE CPC configuration options . . . . . . . . 504

E-3 Entering the OSA ports that will be used by the management network . . . . . . . . . . . 505

E-4 DPM mode welcome window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506

E-5 Traditional HMC Welcome window when no CPCs are running in DPM mode . . . . . 507

E-6 User Options when the HMC presents the DPM welcome window . . . . . . . . . . . . . . 507

E-7 DPM wizard welcome window options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509

F-1 KVM running in z Systems LPARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

F-2 Open source virtualization (KVM for IBM z Systems) . . . . . . . . . . . . . . . . . . . . . . . . . 516

F-3 KVM for IBM z Systems management interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518

F-4 KVM management by using the libvirt API layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519

G-1 I/O domains and resource groups managed by the IFP . . . . . . . . . . . . . . . . . . . . . . 523

G-2 Sample output of AO data or PCHID report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525

G-3 Example of IOCP statements for zEDC Express and 10GbE RoCE Express . . . . . . 527

H-1 z Systems storage hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530

H-2 Flash Express PCIe adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531

H-3 PCIe I/O drawer that is fully populated with Flash Express cards . . . . . . . . . . . . . . . 531

H-4 Sample SE/HMC window for Flash Express allocation to LPAR . . . . . . . . . . . . . . . . 533

H-5 Flash Express allocation in z/OS LPARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534

H-6 Integrated Key Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536

H-7 Key serving topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538

I-1 GDPS offerings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542

I-2 Positioning a virtual appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544

I-3 GDPS virtual appliance architecture overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546

I-4 GDPS Storage failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548

J-1 Relationships between the PCIe I/O drawer card slots, I/O domains, and resource

groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553

xviii IBM z13s Technical Guide

Notices

This information was developed for products and services offered in the US. This material might be available from IBM in other languages. However, you may be required to own a copy of the product or product version in that language in order to access it.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, MD-NC119, Armonk, NY 10504-1785, US

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.

IBM may use or distribute any of the information you provide in any way it believes appropriate without incurring any obligation to you.

The performance data and client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

Statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to actual people or business enterprises is entirely coincidental.

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are provided “AS IS”, without warranty of any kind. IBM shall not be liable for any damages arising out of your use of the sample programs.

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks or registered trademarks of International Business Machines Corporation, and might also be trademarks or registered trademarks in other countries.

AIX® CICS® Cognos® DataPower® DB2 Connect™ DB2® developerWorks® Distributed Relational Database

Architecture™ Domino® DRDA® DS8000® ECKD™ FICON® FlashCopy® GDPS® Geographically Dispersed Parallel

Sysplex™ HACMP™ HiperSockets™ HyperSwap® IBM Systems Director Active Energy

Manager™

IBM z Systems™ IBM z13™ IBM® IMS™ Language Environment® Lotus® MVS™ NetView® OMEGAMON® Parallel Sysplex® Passport Advantage® Power Systems™ POWER6® POWER7® PowerHA® PowerPC® PowerVM® PR/SM™ Processor Resource/Systems

Manager™ RACF® Redbooks® Redpaper™

Redpapers™ Redbooks (logo) ® Resource Link® Resource Measurement Facility™ RMF™ System Storage® System z10® System z9® System z® SystemMirror® Tivoli® VIA® VTAM® WebSphere® z Systems™ z/Architecture® z/OS® z/VM® z/VSE® z10™ z13™ z9® zEnterprise®

The following terms are trademarks of other companies:

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Java, and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Other company, product, or service names may be trademarks or service marks of others.

xx IBM z13s Technical Guide

IBM REDBOOKS PROMOTIONS

Find and read thousands of IBM Redbooks publications

Search, bookmark, save and organize favorites

Get personalized notiﬁcations of new content

Link to the latest Redbooks blogs and videos

Download

Now

Get the latest version of the Redbooks Mobile App

iOS

Android

Place a Sponsorship Promotion in an IBM Redbooks publication, featuring your business or solution with a link to your web site.

Qualiﬁed IBM Business Partners may place a full page promotion in the most popular Redbooks publications. Imagine the power of being seen by users who download millions of Redbooks publications each year!

Promote your business in an IBM Redbooks publication

ibm.com/Redbooks

About Redbooks Business Partner Programs

IBM Redbooks promotions

THIS PAGE INTENTIONALLY LEFT BLANK

Preface

Digital business has been driving the transformation of underlying information technology (IT) infrastructure to be more efficient, secure, adaptive, and integrated. IT must be able to handle the explosive growth of mobile clients and employees. It also must be able to process enormous amounts of data to provide deep and real-time insights to help achieve the greatest business impact.

This IBM® IBM z13s server. IBM z Systems servers are the trusted enterprise platform for integrating data, transactions, and insight. A data-centric infrastructure must always be available with a

99.999% or better availability, have flawless data integrity, and be secured from misuse. It needs to be an integrated infrastructure that can support new applications. It also needs to have integrated capabilities that can provide new mobile capabilities with real-time analytics delivered by a secure cloud infrastructure.

IBM z13s servers are designed with improved scalability, performance, security, resiliency, availability, and virtualization. The superscalar design allows z13s servers to deliver a record level of capacity over the prior single frame z Systems server. In its maximum configuration, the z13s server is powered by up to 20 client characterizable microprocessors (cores) running at 4.3 GHz. This configuration can run more than 18,000 millions of instructions per second (MIPS) and up to 4 TB of client memory. The IBM z13s Model N20 is estimated to provide up to 100% more total system capacity than the IBM zEnterprise® BC12 Model H13.

This book provides information about the IBM z13s server and its functions, features, and associated software support. Greater detail is offered in areas relevant to technical planning. It is intended for systems engineers, consultants, planners, and anyone who wants to understand the IBM z Systems™ functions and plan for their usage. It is not intended as an introduction to mainframes. Readers are expected to be generally familiar with existing IBM z Systems technology and terminology.

Redbooks® publication addresses the new IBM z Systems™ single frame, the

Authors

This book was produced by a team working at the International Technical Support Organization, Poughkeepsie Center.

Octavian Lascu is a Senior IT Consultant for IBM Romania with over 25 years of IT experience. He specializes in designing, implementing, and supporting complex IT infrastructure environments (cloud, systems, storage, and networking), including high availability and disaster recovery solutions and high-performance computing deployments. He has developed materials for and taught over 50 workshops for technical audiences around the world. He has written several IBM Redbook and IBM Redpaper™ publications.

Barbara Sannerud is a Worldwide Technical Enablement Manager for the IBM z Systems platform. She has 30 years of experience in services, strategy, marketing, and product management. Before her current role, she was Offering Manager for IBM z/OS®, and also held competitive marketing roles. She holds math and MBA degrees and joined IBM from the software and professional services industries where she specialized in performance, systems management, and security.

Cecilia A. De Leon is a Certified IT Specialist in the Philippines. She has 15 years of experience in the z Systems field. She has worked at IBM for 7 years. She holds a degree in Computer Engineering from Mapua Institute of Technology. Her areas of expertise include z Systems servers and operating system. In her current role as Client Technical Specialist, she supports mainframe clients and IBM sales representatives on technical sales engagements. She has also worked as a systems programmer for large banks in the Philippines.

Edzard Hoogerbrug is a System Support Representative in The Netherlands. During the past 26 years, he has worked in various roles within IBM, mainly with mainframes. He has 14 years experience working for EMEA L2 support for Z series after a 2.5 years assignment in Montpellier France. He holds a degree in electrotechnology. His areas of expertise include failure analysis on for z Systems hardware.

Ewerson Palacio is an IBM Distinguished Engineer and a Certified Consulting IT Specialist for Large Systems in Brazil. He has more than 40 years of experience in IBM large systems. Ewerson holds a Computer Science degree from Sao Paulo University. His areas of expertise include z Systems client technical support, mainframe architecture, infrastructure implementation, and design. He is an ITSO z Systems hardware official speaker who has presented technical ITSO seminars, workshops, and private sessions to IBM clients, IBM IT Architects, IBM IT Specialists, and IBM Business Partners around the globe. He has also been a z Systems Hardware Top Gun training designer, developer, and instructor for the last generations of the IBM high-end servers. Ewerson leads the Mainframe Specialty Services Area (MF-SSA), which is part of GTS Delivery, Technology and Engineering (DT&E). He is a member of the IBM Academy of Technology.

Franco Pinto is a Client Technical Specialist in IBM Switzerland. He has 20 years of experience in the mainframe and z/OS fields. His areas of expertise include z Systems technical pre-sales covering mainframe sizing and installation planning, and providing support on existing and new z Systems functions.

Jin J. Yang is a Senior System Service Representative in China. He joined IBM in 1999 to support z Systems products maintenance for clients in China. He has been working in the Technical Support Group to provide second-level support for z Systems clients as a country Top Gun since 2009. His areas of expertise include z Systems hardware, channel connectivity, IBM z/VM®, and Linux on z Systems.

John P. Troy is a z Systems and Storage hardware National Top Gun in the northeast area of the United States. He has 35 years of experience in the service field. His areas of expertise include z Systems server and high-end storage systems technical and customer support. John has been a z Systems hardware technical support course designer, developer, and instructor for the last six generations of IBM high-end servers.

Martin Soellig is a Consulting IT Specialist in Germany. He has 26 years of experience working in the z Systems field. He holds a degree in mathematics from University of Hamburg. His areas of expertise include z/OS and z Systems hardware, specifically in IBM Parallel Sysplex® and GDPS® environments.

Thanks to the following people for their contributions to this project:

William G. White International Technical Support Organization, Poughkeepsie Center

Robert Haimowitz Development Support Team Poughkeepsie

xxiv IBM z13s Technical Guide

Patty Driever Diana Henderson Anuja Deedwaniya Lisa Schloemer Christine Smith Martin Ziskind Romney White Jerry Stevens Anthony Sofia IBM Poughkeepsie

Leslie Geer III IBM Endicott

Monika Zimmerman Carl Mayer Angel Nunez Mencias IBM Germany

Parwez Hamid IBM Poughkeepsie

Now you can become a published author, too!

Here’s an opportunity to spotlight your skills, grow your career, and become a published author—all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base.

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Comments welcome

Your comments are important to us!

We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks publications in one of the following ways:

򐂰 Use the online Contact us review Redbooks form found at:

ibm.com/redbooks

򐂰 Send your comments in an email to:

redbooks@us.ibm.com

Preface xxv

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

Stay connected to IBM Redbooks

򐂰 Find us on Facebook:

http://www.facebook.com/IBMRedbooks

򐂰 Follow us on Twitter:

http://twitter.com/ibmredbooks

򐂰 Look for us on LinkedIn:

http://www.linkedin.com/groups?home=&gid=2130806

򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks

weekly newsletter:

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

򐂰 Stay current on recent Redbooks publications with RSS Feeds:

http://www.redbooks.ibm.com/rss.html

xxvi IBM z13s Technical Guide

Chapter 1. Introducing IBM z13s servers

Digital business has been driving the transformation of IT to be more efficient, secure, adaptive, and intelligent. Today’s technology foundation must be capable of increasing the value of data, drawing on the wealth of applications on the z Systems platform, and delivering new opportunities made possible by the API economy. z Systems is designed to harvest information from huge volumes of data to unlock real-time insights, and enable smarter decision making to deliver the greatest business advantage. The hybrid cloud infrastructure requires a secured and resilient platform that is capable of in-transaction analytics, and capable of providing customers with the agility and insight they need to succeed in today’s marketplace.

The IT infrastructure must always be on to serve customers in a 24 x 7 global business, have flawless data integrity, and be capable of thwarting cyber security threats. It needs to be resilient to support the introduction of new workloads and handle spikes in processing while meeting the most demanding service levels. It must support both traditional delivery models and new “as a service” cloud delivery models. It also must support the needs of traditional and mobile knowledge workers.

This chapter introduces the basic concepts of IBM z13s servers.

This chapter includes the following sections:

򐂰 Overview of IBM z13s servers 򐂰 z13s servers highlights 򐂰 z13s technical overview 򐂰 Hardware Management Consoles and Support Elements 򐂰 IBM z BladeCenter Extension (zBX) Model 004 򐂰 IBM z Unified Resource Manager 򐂰 IBM Dynamic Partition Manager 򐂰 Operating systems and software

1.1 Overview of IBM z13s servers

The IBM z13s server, like its predecessors, is designed from the chip level up for intense data serving and transaction processing, the core of business. IBM z13s servers are designed to provide unmatched support for data processing by providing these features:

򐂰 A strong, powerful I/O infrastructure 򐂰 Larger cache sizes on the chip to bring data close to processing power 򐂰 Enhanced cryptographic support and compression capabilities of the co-processors and

I/O features

򐂰 Simultaneous multithreading to drive throughput 򐂰 The 99.999%

application availability design of the IBM z Systems clustering technologies

z13s servers feature a newly designed eight-core

processor chip introduced and shared with the IBM z13™. This chip features innovations designed to deliver maximum performance, and up to 4 TB of available memory, representing an eight-fold increase in memory over the prior processor generation. More memory can help improve response time, minimize application constraints, speed up batch processing, and reduce processor cycles that are consumed for a specific workload. z13s servers also offer a 2.75x boost in the system I/O bandwidth, which when combined with new key I/O enhancements can reduce transfer time for the global enterprise. z13s servers also have more throughput per core (1.3 times the performance of a zBC12 based on workload and model), and more processing cores (50% over zBC12) to drive higher volumes of workloads.

Terminology: The remainder of the book uses the designation CPC to refer to the central

processor complex

The IBM operating systems participate in providing new advantages for the digital era. For instance, z/OS V2.2 running on z13s servers sets the groundwork for digital business by providing support for demanding workloads such as operational analytics and cloud along with traditional mission-critical applications. z/OS V2.2 continues to support the IBM z Integrated Information Processor (zIIP)

that can take advantage of the simultaneous multithreading (SMT) feature implemented in the IBM z Systems processor unit (PU) for greater throughput. Applications running under z/OS V2.2 can use vector processing and single-instruction, multiple-data (SIMD) for new analytics applications, Java processing and so on.

Operating system enhancements drive simplification and enables cost-saving consolidation opportunities. With enhancements to extend the reach of your skills, z/OS V2.2 and the entitled z/OS Management Facility can help extend the reach of administrators and other personnel, enabling them to handle configuration tasks with ease. Mobile Workload Pricing for z/OS can help reduce the cost of growth for mobile transactions that are processed by programs such as IBM CICS®, IBM IMS™, and IBM DB2® for z/OS.

IBM z/VM 6.3 has been enhanced to use the SMT and SIMD features offered on the new processor chip

A z/OS Parallel Sysplex is designed to provide 99.999% application availability. See:

http://www.ibm.com/systems/z/advantages/resiliency/datadriven/application.html

Only 6 or 7 active cores per PU Single Chip Module (SCM) in z13s servers.

zAAP workloads are now run on zIIP.

z/VM 6.3 SMT support is provided on Integrated Facility for Linux (IFL) processors only.

z/VM supports 20 processors on z13s servers and up to 40 threads when SMT enabled.

2 IBM z13s Technical Guide

, and also supports 64 threads5 for Linux workloads. With support for sharing

+ 558 hidden pages