IBM z10 User Guide

IBM System z10™
Enterprise Class

Hardware Overview

CMG – April 15, 2008
Tom Russell – IBM Canada

Thanks to Harv Emery, John Hughes, WSC

IBM Systems

© 2008 IBM Corporation

IBM System z

Trademarks

The following are trademarks of the International Business Machines Corporation in the United States and/or other countries.

AIX*
APPN*
Cell Broadband Engine
DB2*
DB2 Connect
DirMaint
DRDA*
Distributed Relational Database Architecture
e-business logo*
ECKD
Enterprise Storage Server*
ESCON*
FICON*
GDPS*

* Registered trademarks of IBM Corporation

The following are trademarks or registered trademarks of other companies.

InfiniBand® is a registered trademark of the InfiniBand Trade Association (IBTA).
Intel is a trademark of Intel Corporation in the United States, other countries, or both.
Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation.
Red Hat, the Red Hat "Shadow Man" logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries.
SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that

any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and
the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may

have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be

subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the

performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Geographically Dispersed Parallel Sysplex
HiperSockets
HyperSwap
IBM*
eServer
IBM logo*
IMS
InfoPrint*
Language Environment*
MQSeries*
Multiprise*
NetView*
On demand business logo

OS/390*

Parallel Sysplex*
POWER6
PR/SM
Processor Resource/Systems Manager
RACF*
Resource Link
RMF

S/390*
Sysplex Timer*
System z
System z9
System z10
TotalStorage*

Virtualization Engine
VSE/ESA
VTAM*
WebSphere*
z/Architecture
z/OS*
z/VM*
z/VSE
zSeries*

2

© 2008 IBM Corporation

IBM Systems

IBM System z

System z10 EC New Functions and Features

Five hardware models

Faster Processor Unit (PU)

Up to 64 customer PUs

36 CP Subcapacity Settings

Star Book Interconnect

Up to 1,520 GB memory

Fixed HSA as standard

Large Page (1 MB)

HiperDispatch

Enhanced CPACF SHA 512,
AES 192 and 256-bit keys

Hardware Decimal Floating Point

New Capacity on Demand
architecture and enhancements

SOD: PSIFB for z9 EC & BC for

non-dedicated CF Models*

6.0 GBps InfiniBand HCA to I/O
interconnect

FICON Enhancements

SCSI IPL included in Base LIC

OSA-Express3 10 GbE (2Q08)*

HiperSockets enhancements

InfiniBand Coupling Links (2Q08)*
STP using InfiniBand (2Q08)*

Standard ETR Attachment

FICON LX Fiber Quick Connect

Power Monitoring support

Scheduled Outage Reduction

72 New Instructions

Capacity Provisioning

No support for Japanese Compatibility Mode (JCM)
No support for MVS Assist instructions

3

© 2008 IBM Corporation

Improved RAS

* All statements regarding IBM's plans, directions, and intent are subject to change
or withdrawal without notice. Any reliance on these Statements of General Direction
is at the relying party's sole risk and will not create liability or obligation for IBM.

IBM Systems

IBM System z

z10 EC System Upgrades

E64

z9 EC

E56

 z10 EC to higher z10 EC model

E40

E26

Concurrent Upgrade

z990

4

© 2008 IBM Corporation

E12

► Concurrent upgrade of z10 EC

Models E26, E40 and E56.
Upgrade to E64 is disruptive

► When upgrading to z10 EC E64,

unlike the z9 EC, the first Book is
retained

 Any z9 EC to any z10 EC
 Any z990 to any z10 EC

IBM Systems

IBM System z

IBM System z10 EC Key Dates

 IBM System z10 Announce – February 26, 2008

► First Day Orders
► Resource Link
► Capacity Planning Tools (zPCR, zTPM, zCP3000)
► SAPR Guide (SA06-016-00) and SA Confirmation Checklist available

 Availability – February 26, 2008

► z10 EC all Models
► Upgrades from z990, z9 EC to z10 EC

 Availability – May 26, 2008

► Model upgrades within z10 EC
► Feature Upgrades within the z10 EC – May 26, 2008

™

support available

 Planned Availability* – 2Q 2008

► OSA Express3 10 GbE LR – the first of a new OSA generation
► InfiniBand Coupling Links for any z10 EC and ICF-only z9 EC and BC machines

 New ITSO Redbooks (Draft versions)

● z10 EC Technical Introduction, SG24-7515 - February 26, 2008

● z10 EC Technical Guide, SG24-7516 - February 26, 2008

● z10 EC Capacity on Demand, SG24-7504 - March, 2008

● Getting Started with InfiniBand on z10 EC and System z9, SG24-7539 – May, 2008

* All statements regarding IBM's future direction and intent are subj ect to change or withdrawal without notice, and represent goals and objectives only.

5

© 2008 IBM Corporation

IBM Systems

IBM System z

z10 EC Multi-Chip Module (MCM)

 96mm x 96mm MCM

► 103 Glass Ceramic layers
► 7 chip sites
► 7356 LGA connections
► 17 and 20 way MCMs

PU 1

S 2

PU 4 PU 3

S 3

 CMOS 11s chip Technology

► PU, SC, S chips, 65 nm
► 5 PU chips/MCM – Each up to 4 cores

● One memory control (MC) per PU chip

● 21.97 mm x 21.17 mm

● 994 million transistors/chip

● L1 cache/PU

– 64 KB I-cache
– 128 KB D-cache

● L1.5 cache/PU

– 3 MB

PU 0PU 2

● 4.4 GHz

● Approx 0.23 ns Cycle Time

● 6 Km of wire

► 2 Storage Control (SC) chip

● 21.11 mm x 21.71 mm

SC 0SC 1

● 1.6 billion transistors/chip

● L2 Cache 24 MB per SC chip (48 MB/Book)

● L2 access to/from other MCMs

● 3 Km of wire

S 0

► 4 SEEPROM (S) chips

● 2 x active and 2 x redundant

S 1

● Product data for MCM, chips and other engineering

information

► Clock Functions – distributed across PU and SC chips

● Master Time-of-Day (TOD) and 9037 (ETR)

functions are on the SC

6

© 2008 IBM Corporation

IBM Systems

IBM System z

z10 EC Chip Relationship to POWER6™

 Siblings, not identical twins
 Share lots of DNA

► IBM 65nm Silicon-On-Insulator (SOI) technology

► Design building blocks:

● Latches, SRAMs, regfiles, dataflow elements

► Large portions of Fixed Point Unit (FXU), Binary Floating-

point Unit. (BFU), Hardware Decimal Floating-point Unit
(HDFU), Memory Controller (MC), I/O Bus Controller (GX)

► Core pipeline design style

Enterprise Quad Core
z10 Processor Chip

● High-frequency, low-latency, mostly-in-order

► Many designers and engineers

 Different personalities

► Very different Instruction Set Architectures (ISAs)

● very different cores

► Cache hierarchy and coherency model

► SMP topology and protocol

► Chip organization

► IBM z Chip optimized for Enterprise Data Serving Hub

7

© 2008 IBM Corporation

POWER6 Dual Core Chip

IBM Systems

IBM System z

Orderable Processor Features

Books

/PUs

1/17

2/34

3/51

4/68

4/77

CPsModel

IFLs

uIFLs

0 - 12

0 - 12E12

0 - 11
0 - 26

0 - 26E26

0 - 25
0 - 40

0 - 40E40

0 - 39
0 - 56

0 - 56E56

0 - 55
0 - 64

0 - 64E64

0 - 63

zAAPs

ICFs

zIIPs

0 - 6

0-12

0 - 6

0 - 13

0-16

0 - 13
0 - 20

0-16

0 - 20
0 - 28

0-16

0 - 28
0 - 32

0-16

0 - 32

Opt

Saps

0-3

0-7

0-11

0-18

0-21

Std

Saps

3

6

9

10

11

Std

Spares

2

2

2

2

2

Note: A minimum of one CP, IFL, or ICF must be
purchased on every model.

Note: One zAAP and one zIIP may be purchased for
each CP purchased.

8

© 2008 IBM Corporation

Note: System z10 EC is designed not to require
Optional SAPs for production workloads except
sometimes for TPF or z/TPF workloads.

IBM Systems

IBM System z

z9 vs z10 EC CEC Structure

z10 ECz9 EC

SMP Configuration

Topology

Max Memory

Cache Levels

S54

4 books, 64 PUs

Dual Ring

One or Two Hops

Up to 512GB

-HSA?

L1 per PU

L2 per Book

E64

4 books, 77 PUs

Fully Connected

NoYesJumper Books

Up to 1,520 GB

+ 16 GB HSA

L1 and L1.5 per PU

L2 per Book

4 KB and 1 MB4 KBPage Sizes

9

© 2008 IBM Corporation

IBM Systems

IBM System z

z10 EC HiperDispatch

 HiperDispatch – z10 EC unique function

► Dispatcher Affinity (DA) - New z/OS Dispatcher

► Vertical CPU Management (VCM) - New PR/SM Support

 Hardware cache optimization occurs when a given unit of work is consistently

dispatched on the same physical CPU

► Up till now software, hardware, and firmware have had pride in the fact of how

independent they were from each other

► Non-Uniform-Memory-Access has forced a paradigm change

● CPUs have different distance-to-memory attributes

● Memory accesses can take a number of cycles depending upon cache level /

local or remote repository accessed

 The entire z10 EC hardware/firmware/OS stack now tightly collaborates to

obtain the hardware’s full potential

 All supported z/OS releases (z/OS 1.7 requires the zIIP web deliverable)

10

© 2008 IBM Corporation

IBM Systems

IBM System z

System z9 EC CP Subcapacity (12 or Fewer CPs)

CP Capacity

Relative to Full Speed

Subcapacity CP settings

7nn = 100%

6nn

5nn

4nn

¡

¡

¡

aa%

bb%

cc%

nn = 01 Through 12

1-CP through 12-CP only
Available on any hardware model

409

509

609

709

410

510

610

710

411

511

611

401

501

601

701

402

502

602

702

403

503

603

703

404

504

604

704

405

505

605

705

406

506

606

706

407

507

607

707

408

508

608

708

1-way 2-way 3-way 4-way 5-way 6-way

7-way 8-way

9-way

10-way

 The System z10 EC will offer 36 CP subcapacity settings with the first twelve or fewer

CPs (general purpose) engines.

► All CPs must be the same capacity within one z10 EC

► On machines with 13 or more CPs, all CPs must run at full speed

711

11-way

412

512

612

12-way

712

11

 The entry point is approximately xx% of the capacity of the full speed CP

 All specialty engines run at full speed. The one for one entitlement to purchase one

zAAP and one zIIP for each CP purchased is the same for CPs of any speed.

© 2008 IBM Corporation

IBM Systems

IBM System z

LSPR Ratios and MSU Values for System z10 EC

z10 EC to z9 EC

Ratios

z10 EC MSU

Values*

LSPR mixed workload average, multi-image for z/OS 1.8 with HiperDispatch
active on z10 EC!

Uni-processor

16-way z10 EC to 16-way z9 EC

32-way z10 EC to 32-way z9 EC

56-way z10 EC to 54-way z9 EC

64-way z10 EC to 54-way z9 EC

1.62

1.49

1.49

1.54

1.70

115 for 701

1,264 for 716

2,200 for 732

3,395 for 756

3,739 for 764

12

* Reflects Mainframe Charter Technology Dividend.

© 2008 IBM Corporation

IBM Systems

IBM System z

z10 EC Capacity Planning in a nutshell

GHz

MIPs

MSUs

tables

13

?

HiperDispatch

Don’t use “one number” capacity comparisons!

Work with IBM technical support for capacity planning!

Customers can now use zPCR

© 2008 IBM Corporation

IBM Systems

IBM System z

Evolution of System z Specialty Engines

Building on a strong track record of

technology innovation with specialty

engines – DB Compression, SORT,

Encryption, Vector Facility

Cell Broadband Engine™

IBM System z9

Integrated

Information

Processor (IBM

zIIP) 2006

System z Application

Eligible for zIIP:

Assist Processor

 DB2 remote

(zAAP) 2004

access and
BI/DW

Integrated Facility

for Linux (IFL)

2000

Internal Coupling

Facility (ICF) 1997

* All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

14

© 2008 IBM Corporation

*SOD: IBM plans to enhance z/VM in a future release to support the new
System z10 EC capability to allow any combination of CP, zIIP, zAAP, IFL,
and ICF processor-types to reside in the same z/VM LPAR

Eligible for zAAP:

 Java

™

execution

environment

 z/OS XML

 ISVs
 New! IPSec

encryption

 z/OS XML
 z/OS Global

Mirror*

IBM Systems

IBM System z

Large Page Support

 Issue: Translation Lookaside Buffer (TLB) Coverage shrinking as % of

memory size

► Over the past few years application memory sizes have dramatically increased due to

support for 64-bit addressing in both physical and virtual memory

► TLB sizes have remained relatively small due to low access time requirements and

hardware space limitations

► TLB coverage today represents a much smaller fraction of an applications working

set size leading to a larger number of TLB misses

► Applications can suffer a significant performance penalty resulting from an increased

number of TLB misses as well as the increased cost of each TLB miss

 Solution: Increase TLB coverage without proportionally enlarging the TLB

size by using large pages

► Large Pages allow for a single TLB entry to fulfill many more address translations
► Large Pages will provide exploiters with better TLB coverage

 Benefit:

► Designed for better performance by decreasing the number of TLB misses that an

application incurs

15

© 2008 IBM Corporation

IBM Systems