IBM BladeCenter PS700, BladeCenter PS701, BladeCenter PS702 Technical Overview And Introduction

Download

Front cover

IBM BladeCenter PS700, PS701, and PS702

Technical Overview and Introduction

Features the POWER7 processor providing advanced multi-core technology

Details the follow-on to the BladeCenter JS23 and JS43 servers

Includes product information and features

ibm.com/redbooks

David Watts

Kerry Anders

Berjis Patel

Redpaper

International Technical Support Organization

IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

May 2010

REDP-4655-00

Note: Before using this information and the product it supports, read the information in “Notices” on page vii.

First Edition (May 2010)

This edition applies to:

򐂰 IBM BladeCenter PS700, 8406-70Y 򐂰 IBM BladeCenter PS701, 8406-71Y 򐂰 IBM BladeCenter PS702, 8406-71Y + FC 8358

This document created or updated on July 6, 2012.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

The team who wrote this paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Chapter 1. Introduction and general description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Overview of PS700, PS701, and PS702 blade servers . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 IBM BladeCenter support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.1 Supported BladeCenter chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.2 Number of PS700, PS701, and PS702 blades in a chassis . . . . . . . . . . . . . . . . . 14

1.3 Operating environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.4 Physical package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.5 System features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5.1 PS700 system features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5.2 PS701 system features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.5.3 PS702 system features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.5.4 Minimum features for the POWER7 processor-based blade servers . . . . . . . . . . 21

1.5.5 Power supply features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1.5.6 Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1.5.7 Memory features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.5.8 I/O features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.5.9 Disk features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.5.10 Standard onboard features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.6 Supported BladeCenter I/O modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.6.1 Ethernet switch and intelligent pass through modules . . . . . . . . . . . . . . . . . . . . . 30

1.6.2 SAS I/O modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.6.3 Fibre Channel switch and pass-thru modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.6.4 Converged networking I/O modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

1.6.5 InfiniBand switch module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

1.6.6 Multi-switch Interconnect Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

1.6.7 Multi-switch Interconnect Module for BladeCenter HT . . . . . . . . . . . . . . . . . . . . . 34

1.7 Comparison between PS700, PS701, PS702, and 750 models . . . . . . . . . . . . . . . . . . 35

1.8 Building to order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

1.9 Model upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Chapter 2. Architecture and technical overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.2 The IBM POWER7 processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.2.1 POWER7 processor overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.2.2 POWER7 processor core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.2.3 Simultaneous multithreading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.2.4 Memory access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.2.5 Flexible POWER7 processor packaging and offerings . . . . . . . . . . . . . . . . . . . . . 42

2.2.6 On-chip L3 cache innovation and intelligent cache. . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.7 POWER7 processor and intelligent energy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.2.8 Comparison of the POWER7 and POWER6 processors . . . . . . . . . . . . . . . . . . . 45

2.3 POWER7 processor-based blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.4 Memory subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.4.1 Memory placement rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.5 Technical comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.6 Internal I/O subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.6.1 Peripheral Component Interconnect Express (PCIe) bus . . . . . . . . . . . . . . . . . . . 52

2.6.2 PCIe slots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.6.3 I/O expansion cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.6.4 Embedded SAS Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.6.5 HEA ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.6.6 Embedded USB controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.7 Integrated Virtual Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.7.1 IVE subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2.8 Service processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

2.8.1 Server console access by SOL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

2.9 Internal storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

2.9.1 Hardware RAID function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.9.2 External SAS connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.10 External disk subsystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.10.1 IBM BladeCenter S Disk Storage Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2.10.2 IBM System Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2.11 IVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.12 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.13 IBM EnergyScale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.13.1 IBM EnergyScale technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.13.2 EnergyScale device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Chapter 3. Virtualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.1 POWER Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.2 POWER processor modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.3 PowerVM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.3.1 PowerVM Editions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.3.2 Logical partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3.3.3 VIOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.3.4 PowerVM Lx86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.3.5 PowerVM Live Partition Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

3.3.6 Active Memory Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

3.3.7 N_Port ID Virtualization (NPIV). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.3.8 Supported PowerVM features by operating system . . . . . . . . . . . . . . . . . . . . . . . 98

Chapter 4. Continuous availability and manageability . . . . . . . . . . . . . . . . . . . . . . . . . 99

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.2 Reliability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.2.1 Designed for reliability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.2.2 Placement of components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.2.3 Redundant components and concurrent repair. . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.3 Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.3.1 Partition availability priority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.3.2 General detection and deallocation of failing components . . . . . . . . . . . . . . . . . 103

4.3.3 Memory protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

4.3.4 Cache protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.3.5 Special uncorrectable error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

iv IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

4.3.6 PCI extended error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4.4 Serviceability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.4.1 Detecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

4.4.2 Diagnosing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

4.4.3 Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4.4.4 Notifying the client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

4.4.5 Locating and servicing parts requiring service . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.5 Manageability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

4.5.1 Service user interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

4.5.2 IBM Power Systems firmware maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4.5.3 Electronic Service Agent tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4.5.4 BladeCenter Service Advisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

How to get Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Contents v

vi IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION

PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

AIX 5L™ AIX® BladeCenter® Calibrated Vectored Cooling™ DS4000® DS8000® Electronic Service Agent™ EnergyScale™ FlashCopy® Focal Point™ IBM Systems Director Active Energy

Manager™

IBM® Micro-Partitioning™ POWER Hypervisor™ Power Systems™ Power Systems Software™ POWER4™ POWER5™ POWER6+™ POWER6® POWER7™ PowerVM™ POWER®

pSeries® Redbooks® Redpaper™ Redbooks (logo) ® ServerProven® System p5® System p® System Storage® System x® System z® Tivoli® XIV®

The following terms are trademarks of other companies:

SnapManager, and the NetApp logo are trademarks or registered trademarks of NetApp, Inc. in the U.S. and other countries.

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

viii IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Preface

The IBM® BladeCenter® PS700, PS701, and PS702 are premier blades for 64-bit applications. They are designed to minimize complexity, improve efficiency, automate processes, reduce energy consumption, and scale easily. These blade servers are based on the IBM POWER7™ processor and support AIX®, IBM i, and Linux® operating systems. Their ability to coexist in the same chassis with other IBM BladeCenter blades servers enhances the ability to deliver rapid return of investment demanded by clients and businesses.

This IBM Redpaper™ is a comprehensive guide covering the IBM BladeCenter PS700, PS701, and PS702 servers. The goal of this paper is to introduce the offerings and their prominent features and functions.

The team who wrote this paper

This paper was produced by a team of specialists from around the world working at the International Technical Support Organization, Poughkeepsie Center.

David Watts is a Consulting IT Specialist at the IBM ITSO Center in Raleigh. He manages residencies and produces IBM Redbooks® publications on hardware and software topics related to IBM BladeCenter and IBM System x® servers and associated client platforms. He has authored over 80 books, papers, and Web docs. He has worked for IBM both in the US and Australia since 1989. David is an IBM Certified IT Specialist and a member of the IT Specialist Certification Review Board. He holds a Bachelor of Engineering degree from the University of Queensland (Australia)

Kerry Anders is a Consultant in System p® Lab Services for the IBM Systems and Technology Group, based in Austin, Texas. He supports clients in implementing IBM Power Systems™ blades using Virtual I/O Server, Integrated Virtualization Manager, and AIX. Prior IBM Redbooks publication projects include the IBM BladeCenter JS12 and JS22

Implementation Guide, SG24-7655 and the IBM BladeCenter JS23 and JS43 Implementation Guide, SG24-7740. Previously, he was the Systems Integration Test Team Lead for the IBM

BladeCenter JS21blade with IBM SAN storage using AIX and Linux. His prior work includes test experience with the JS20 blade, also using AIX and Linux in SAN environments. Kerry began his career with IBM in the Federal Systems Division supporting NASA at the Johnson Space Center as a Systems Engineer. He transferred to Austin in 1993.

Berjis Patel is a Senior IT Specialist with System Sales Implementation Services with IBM Global Technology Services in Canada. He has over 20 years of experience in the IT industry with more then 15 years with IBM UNIX® (AIX) solutions. He is a certified presales specialist for IBM System p and has multiple IBM Hundred Percent Club awards. His area of expertise is consulting, selling, and implementing services such as consolidation, virtualization, migration, high-availability, and systems management solutions on IBM Power Systems. He has worked in various IBM locations including India, the Middle East, the USA, and now in Canada with different roles since 1995.

The team (l-r): David, Berjis, and Kerry

Thanks to the following people for their contributions to this project:

From IBM Power Systems development:

򐂰 Chris Austen 򐂰 Larry Cook 򐂰 Jeff Franke 򐂰 Tom Flynn 򐂰 Kaena Freitas 򐂰 Ghadir Gholami 򐂰 Jim Jordan 򐂰 Richard Lary 򐂰 Gregory Mclntire 򐂰 Todd Rosedahl 򐂰 Steven Royer 򐂰 Mark Smolen 򐂰 Chris Sturgill 򐂰 Mike Stys

From IBM Power Systems marketing:

򐂰 John Biebelhausen 򐂰 Guy Paradise

From IBM Systems & Technology Group: 򐂰 Michael L. Nelson

Now you can become a published author, too!

Here's an opportunity to spotlight your skills, grow your career, and become a published author - all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base.

x IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Comments welcome

Your comments are important to us!

We want our papers to be as helpful as possible. Send us your comments about this paper or other IBM Redbooks publications in one of the following ways:

򐂰 Use the online Contact us review Redbooks form found at:

ibm.com/redbooks

򐂰 Send your comments in an e-mail to:

redbooks@us.ibm.com

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

Stay connected to IBM Redbooks

򐂰 Find us on Facebook:

http://www.facebook.com/IBMRedbooks

򐂰 Follow us on twitter:

http://twitter.com/ibmredbooks

򐂰 Look for us on LinkedIn:

http://www.linkedin.com/groups?home=&gid=2130806

򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks

weekly newsletter:

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

򐂰 Stay current on recent Redbooks publications with RSS Feeds:

http://www.redbooks.ibm.com/rss.html

Preface xi

xii IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Chapter 1. Introduction and general

description

This chapter provides an introduction and general description to the new IBM BladeCenter POWER7 processor-based blade servers. These new blades offer processor scalability from four cores to 16 cores:

򐂰 IBM BladeCenter PS700: Single-wide blade with a single-socket 4-core processor 򐂰 IBM BladeCenter PS701: Single-wide blade with a single-socket 8-core processor 򐂰 IBM BladeCenter PS702: Double-wide blade with two single-socket 8-core processors

The new PS700, PS701, and PS702 blades are premier blades for 64-bit applications. They are designed to minimize complexity, improve efficiency, automate processes, reduce energy consumption, and scale easily.

The POWER7 processor-based PS700, PS701, and PS702 blades support AIX, IBM i, and Linux operating systems. Their ability to coexist in the same chassis with other IBM BladeCenter blades servers enhances the ability to deliver rapid return of investment demanded by clients and businesses.

This chapter covers the following topics:

򐂰 1.1, “Overview of PS700, PS701, and PS702 blade servers” on page 2 򐂰 1.2, “IBM BladeCenter support” on page 4 򐂰 1.3, “Operating environment” on page 14 򐂰 1.4, “Physical package” on page 15 򐂰 1.5, “System features” on page 16 򐂰 1.6, “Supported BladeCenter I/O modules” on page 29 򐂰 1.7, “Comparison between PS700, PS701, PS702, and 750 models” on page 35 򐂰 1.8, “Building to order” on page 36 򐂰 1.9, “Model upgrades” on page 36

1.1 Overview of PS700, PS701, and PS702 blade servers

Figure 1-1 shows the IBM BladeCenter PS700, PS701, and PS702 blade servers.

Figure 1-1 The IBM BladeCenter PS702, BladeCenter PS701, and BladeCenter PS700

The PS700 blade server

The PS700 Blade Server (8406-70Y) is a single socket, single wide 4-core 3.0 GHz POWER7 processor-based server. The POWER7 processor is a 64-bit, 4-core with 256 KB L2 cache per core and 4 MB L3 cache per core.

The PS700 blade server has eight DDR3 memory DIMM slots. The industry standard VLP DDR3 Memory DIMMs are either 4 GB or 8 GB running at 1066 MHz. The memory is supported in pairs, thus the minimum memory required for PS700 blade server is 8 GB (two 4 GB DIMMs). The maximum memory that can be supported is 64 GB (eight 8 GB DIMMs).

It has two Host Ethernet Adapters (HEA) 1 GB integrated Ethernet ports that are connected to the BladeCenter chassis fabric (midplane). The PS700 has an integrated SAS controller that supports local (on-board) storage, integrated USB controller and Serial over LAN console access through the service processor, and the BladeCenter Advance Management Module.

It supports two on-board disk drive bays. The on-board storage can be one or two 2.5-inch SAS HDD. The integrated SAS controller supports RAID 0, RAID 1, and RAID 10 hardware when two HDDs are used.

The PS700 also supports one PCIe CIOv expansion card slot and one PCIe CFFh expansion card slot. See 1.5.8, “I/O features” on page 24 for supported I/O expansion cards.

2 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

The PS701 blade server

The PS701 blade server (8406-71Y) is a single socket, single-wide 8-core 3.0 GHz POWER7 processor-based server. The POWER7 processor is a 64-bit, 8-core with 256 KB L2 cache per core and 4 MB L3 cache per core.

The PS701 blade server has 16 DDR3 memory DIMM slots. The industry standard VLP DDR3 memory DIMMs are either 4 GB or 8 GB running at 1066 MHz. The memory is supported in pairs, thus the minimum memory required for PS701 blade server is 8 GB (two 4 GB DIMMs). The maximum memory that can be supported is 128 GB (16x 8 GB DIMMs).

The PS701 blade server has two Host Ethernet Adapters (HEA) 1 GB integrated Ethernet ports that are connected to the BladeCenter chassis fabric (midplane). The PS701 also has an integrated SAS controller that supports local (on-board) storage, integrated USB controller and Serial over LAN console access through the service processor, and the BladeCenter Advance Management Module.

The PS701 has one on-board disk drive bay. The on-board storage can be one 2.5-inch SAS HDD. The PS701 also supports one PCIe CIOv expansion card slot and one PCIe CFFh expansion card slot. See 1.5.8, “I/O features” on page 24 for supported I/O expansion cards.

The PS702 blade server

The PS702 blade server (8406-71Y +FC 8358) is a two socket, double-wide 16-core 3.0 GHz POWER7 processor-based server. The POWER7 processor is a 64-bit, 8-core with 256 KB L2 cache per core and 4 MB L3 cache per core. The PS702 combines a single-wide base blade (PS701) and an expansion unit (feature 8358), referred to as double-wide blade, which occupies two adjacent slots in the IBM BladeCenter chassis.

The PS702 blade server has 32 DDR3 memory DIMM slots. The industry standard VLP DDR3 memory DIMMs are either 4 GB or 8 GB running at 1066 MHz. The memory is supported in pairs, thus the minimum memory required for PS702 blade server is 8 GB (two 4 GB DIMMs). The maximum memory that can be supported is 256 GB (32x 8 GB DIMMs).

Note: The PS702 blade server can have a minimum of 8 GB memory based as per architecture, but we recommend a reasonable ratio between cores and memory.

The PS702 blade server has four Host Ethernet Adapter 1 GB integrated Ethernet ports that are connected to the BladeCenter chassis fabric (midplane). The PS702 also has an integrated SAS controller that supports local (on-board) storage, integrated USB controller and Serial over LAN console access through the service processor, and the BladeCenter Advance Management Module.

The PS702 blade server has two disk drive bays, one on the base blade and one on the expansion unit. The on-board storage can be one or two 2.5-inch SAS disk drives. The integrated SAS controller supports RAID 0, RAID 1 and RAID 10 hardware when two HDDs are used.

The PS702 supports two PCIe CIOv expansion card slot and two PCIe CFFh expansion card slots. See 1.5.8, “I/O features” on page 24 for supported I/O expansion cards.

Note: For the PS702 blade server, the service processor (FSP or just SP) in the expansion blade is set to IO mode, which provides control busses from IOs, but does not provide redundancy and backup operational support to the SP in the base blade.

Chapter 1. Introduction and general description 3

1.2 IBM BladeCenter support

Blade servers are thin servers that insert into a single rack-mounted chassis that supplies shared power, cooling, and networking infrastructure. Each server is an independent server with its own processors, memory, storage, network controllers, operating system, and applications. The IBM BladeCenter chassis is the container for the blade servers and shared infrastructure devices.

The IBM BladeCenter chassis can contain a mix of POWER®, Intel®, Cell and AMD processor-based blades. Depending on the IBM BladeCenter chassis selected, combinations of Ethernet, SAS, Fibre Channel, and FCoE I/O fabrics can also be shared within the same chassis.

All chassis can offer full redundancy for all shared infrastructure, network, and I/O fabrics. Having multiple power supplies, network switches, and I/O switches contained within a BladeCenter chassis eliminates single points of failure in these areas.

The following sections describe the BladeCenter chassis that support the PS700, PS701, and PS702 blades. For a comprehensive look at all aspects of BladeCenter products see the IBM Redbooks publication, IBM BladeCenter Products and Technology, SG24-7523, available from the following Web page:

http://www.redbooks.ibm.com/abstracts/sg247523.html

1.2.1 Supported BladeCenter chassis

The PS700, PS701, and PS702 blades are supported in the IBM BladeCenter chassis as listed in Table 1-1.

Table 1-1 The blade servers supported in each BladeCenter chassis

Blade Machine

type-model

PS700 8406-70Y 1 slot

PS701 8406-71Y 1 slot

PS702 8406-71Y 2 slot

a. BladeCenter E requires an Advanced Management Module and a minimum of two 2000 watt power supplies. b. Only specific models of the BladeCenter E support the PS700. See Table 1-2.

Blade width BC S

8886

Ye s Ye s

Ye s No No No Ye s Ye s Ye s

BC E 8677

BC T

8720

No No Ye s Ye s Ye s

BC T 8730

BC H 8852

BC HT 8740

A detailed description of each supported BladeCenter for the PS700, PS701, and PS702 blades is contained in the following sections.

򐂰 IBM BladeCenter E (PS700 only support) provides the greatest density and common

fabric support. It is the lowest entry cost option. See “BladeCenter E” on page 5 for details on the chassis.

Only specific models of the BladeCenter E models listed are supported as shown in Ta bl e 1 - 2 .

BC HT 8750

4 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Table 1-2 BladeCenter E models that support the PS700

BladeCenter E models

8677-3Xx No

8677-3Rx No

8677-E2x No

8677-3Sx

Supports the PS700

Ye s

8677-4Sx

8677-3Tx

8677-4Tx Ye s

a. x = country-specific letter (for example, EMEA MTM is 8677-4SG,

and the US MTM is 8677-4SU).

b. The 3Sx and 3Tx models are supported but only with upgraded

(2320W) power supplies

Ye s

򐂰 IBM BladeCenter H delivers high performance, extreme reliability, and ultimate flexibility

for the most demanding IT environments. See “BladeCenter H” on page 7.

򐂰 IBM BladeCenter HT models are designed for high-performance flexible

telecommunications environments by supporting high-speed internet working technologies (such as 10G Ethernet). They provide a robust platform for NGNs. See “BladeCenter HT” on page 9.

򐂰 IBM BladeCenter S combines the power of blade servers with integrated storage, all in an

easy-to-use package designed specifically for the office and distributed enterprise environments. See “BladeCenter S” on page 12.

Note: The number of blade servers that can be installed into chassis is dependent on the power supply configuration, power supply input (110V/208V BladeCenter S only) and power domain configuration options. See 1.2.2, “Number of PS700, PS701, and PS702 blades in a chassis” on page 14 for more information.

BladeCenter E

IBM designed the IBM BladeCenter E (machine type 8677) to be a highly modular chassis to accommodate a range of diverse business requirements. BladeCenter supports not only blade servers, but also a wide range of networking modules, including Gigabit Ethernet, Fibre Channel, and SAS for connectivity to the client’s existing network environment. BladeCenter E also supports a redundant pair of Management Modules for comprehensive systems management.

Providing a wide selection of integrated switching options, BladeCenter systems lower the total cost of ownership (TCO) by eliminating the need to purchase additional keyboards, videos, and mice (KVM), Ethernet and Fibre Channel switches, or the cumbersome and expensive cabling required by the switches. BladeCenter is a leader in the industry in providing flexibility and a variety of integration choices with components that fit your infrastructure and deliver a comprehensive blade solution.

BladeCenter E’s superior density and feature set are made possible by the BladeCenter E innovative chassis architecture. Because BladeCenter E uses super energy-efficient components and shared infrastructure architecture, clients realize lower power consumption when compared to their most likely alternative (that is, non-blade server designs).

Chapter 1. Introduction and general description 5

BladeCenter E’s lower power consumption and Calibrated Vectored Cooling™ allow more servers to fit in a tight power or cooling environment.

Figure 1-2 displays the front view of an IBM BladeCenter E.

Figure 1-2 BladeCenter E front view

Figure 1-3 displays the rear view of an IBM BladeCenter E.

Figure 1-3 BladeCenter E rear view

The key features of IBM BladeCenter E chassis are as follows:

򐂰 A rack-optimized, 7 U modular design enclosure for up to 14 hot-swap blades 򐂰 A high-availability mid-plane that supports hot-swap of individual blades 򐂰 For 8677-3Sx, two 2,000-watt, hot-swap power modules and support for two optional

2,000-watt power modules, offering redundancy and power for robust configurations

6 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

򐂰 For 8677-4Sx, two 2,320-watt, hot-swap power modules and support for two optional

2,320-watt power modules, offering higher power and performance than previous models, extending support to a wider range of blades

򐂰 Two hot-swap blowers 򐂰 An Advanced Management Module that provides chassis-level solutions, simplifying

deployment and management of your installation

򐂰 Support for up to four network or storage switches or pass-through modules 򐂰 A light path diagnostic panel, and USB 2.0 port 򐂰 Support for UltraSlim enhanced SATA DVD-ROM and multi-burner drives 򐂰 IBM Systems Director and Tivoli® Provisioning Manager for OS Deployments for easy

installation and management

򐂰 Energy-efficient design and innovative features to maximize productivity and reduce

power usage

򐂰 Extreme density and integration to ease data center space constraints 򐂰 Help in protecting your IT investment through IBM BladeCenter family longevity,

compatibility, and innovation leadership in blades

򐂰 Support for the latest generation of IBM BladeCenter blades, providing investment

protection

BladeCenter H

IBM BladeCenter H delivers high performance, extreme reliability, and ultimate flexibility to even the most demanding IT environments. In 9 U of rack space, the BladeCenter H chassis can contain up to 14 blade servers, 10 switch modules, and four power supplies to provide the necessary I/O network switching, power, cooling, and control panel information to support the individual servers.

The chassis supports up to four traditional fabrics using networking switches, storage switches, or pass through devices. The chassis also supports up to four high-speed fabrics for support of protocols such as 4X InfiniBand or 10 Gigabit Ethernet. The built-in media tray includes light path diagnostics, two front USB inputs, and a optical drive.

Figure 1-4 displays the front view of an IBM BladeCenter H.

Chapter 1. Introduction and general description 7

Figure 1-4 BladeCenter H front view

Figure 1-5 displays the rear view of an IBM BladeCenter H.

Figure 1-5 BladeCenter H rear view

The key features of IBM BladeCenter H chassis are as follows: 򐂰 A rack-optimized, 9 U modular design enclosure for up to 14 hot-swap blades

8 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

򐂰 A high-availability mid-plane that supports hot-swap of individual blades 򐂰 Two 2,900-watt, hot-swap power modules and support for two optional 2,900-watt power

modules, offering redundancy and power for robust configurations

򐂰 Two hot-swap redundant blowers, and six or 12 supplemental fans with power supplies 򐂰 An Advanced Management Module that provides chassis-level solutions, simplifying

deployment and management of your installation

򐂰 Support for up to four network or storage switches or pass-through modules 򐂰 Support for up to four bridge modules 򐂰 A light path diagnostic panel, and two USB 2.0 ports 򐂰 Serial port breakout connector 򐂰 Support for UltraSlim Enhanced SATA DVD-ROM and Multi-Burner Drives 򐂰 IBM Systems Director and Tivoli Provisioning Manager for OS Deployments for easy

installation and management

򐂰 Energy-efficient design and innovative features to maximize productivity and reduce

power usage

򐂰 Density and integration to ease data center space constraints 򐂰 Help in protecting your IT investment through IBM BladeCenter family longevity,

compatibility, and innovation leadership in blades

򐂰 Support for the latest generation of IBM BladeCenter blades, helping provide investment

protection

BladeCenter HT

The IBM BladeCenter HT is a 12-server blade chassis designed for high-density server installations, typically for telecommunications use. It offers high performance with the support of 10 G Ethernet installations. This 12 U high chassis with DC or AC power supplies provides a cost-effective, high-performance, high-availability solution for telecommunication networks and other rugged non-telecommunications environments. IBM BladeCenter HT chassis is positioned for expansion, capacity, redundancy, and carrier-grade NEBS level 3/ETSI compliance in DC models.

BladeCenter HT provides a solid foundation for next-generation networks (NGN) enabling service providers to become on demand providers. Coupled with technological expertise within the enterprise data center, IBM makes use of the industry know-how of key business partners to deliver added value within service provider networks.

Figure 1-6 shows the front view of the BladeCenter HT.

Chapter 1. Introduction and general description 9

Figure 1-6 BladeCenter HT front view

10 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 1-7 shows the rear view of the BladeCenter HT.

Figure 1-7 BladeCenter HT rear view

BladeCenter HT delivers rich telecommunications features and functionality, including integrated servers, storage and networking, fault-tolerant features, optional hot swappable redundant DC or AC power supplies and cooling, and built-in system management resources. The result is a Network Equipment Building Systems (NEBS-3) and ETSI-compliant server platform optimized for next-generation networks.

The following BladeCenter HT applications are suited for these servers: 򐂰 Network management and security

– Network management engine – Internet cache engine – RSA encryption –Gateways – Intrusion detection

򐂰 Network infrastructure

– Softswitch – Unified messaging – Gateway/Gatekeeper/SS7 solutions – VOIP services and processing – Voice portals – IP translation database

Chapter 1. Introduction and general description 11

The key features of the BladeCenter HT are as follows: 򐂰 Support for up to 12 blade servers, compatible with the other chassis in the BladeCenter

family

򐂰 Four standard and four high-speed I/O module bays, compatible with the other chassis in

the BladeCenter family

򐂰 A media tray at the front with light path diagnostics, two USB 2.0 ports, and optional

compact flash memory module support

򐂰 Two hot-swap management-module bays (one management module standard) 򐂰 Four hot-swap power-module bays (two power modules standard) 򐂰 New serial port for direct serial connection to installed blades 򐂰 Compliance with the NEBS 3 and ETSI core network specifications

BladeCenter S

The BladeCenter S chassis can hold up to six blade servers, and up to 12 hot-swap 3.5-inch SAS or SATA disk drives in just 7 U of rack space. It can also include up to four C14 950-watt / 1450-watt power supplies. The BladeCenter S offers the necessary I/O network switching, power, cooling, and control panel information to support the individual servers.

The IBM BladeCenter S is one of five chassis in the BladeCenter family. The BladeCenter S provides an easy IT solution to the small and medium office and to the distributed enterprise. Figure 1-8 shows the front view of IBM BladeCenter S.

Figure 1-8 The front of the BladeCenter S chassis

12 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 1-9 shows the rear view of the chassis.

Figure 1-9 The rear of the BladeCenter S chassis

The key features of IBM BladeCenter S chassis are as follows:

򐂰 A rack-optimized, 7 U modular design enclosure for up to six hot-swap blades 򐂰 Two optional Disk Storage Modules for HDDs, six 3.5-inch SAS/SATA drives each 򐂰 High-availability mid-plane that supports hot-swap of individual blades 򐂰 Two 950/1450-watt, hot-swap power modules and support for two optional 950/1450-watt

power modules, offering redundancy and power for robust configurations

򐂰 Four hot-swap redundant blowers, plus one fan in each power supply 򐂰 An Advanced Management Module that provides chassis-level solutions, simplifying

deployment and management of your installation

򐂰 Support for up to four network or storage switches or pass-through modules 򐂰 A light path diagnostic panel, and two USB 2.0 ports 򐂰 Support for optional UltraSlim Enhanced SATA DVD-ROM and Multi-Burner Drives 򐂰 Support for SAS RAID Controller Module makes it easy for clients to buy the all-in-one

BladeCenter S solution

򐂰 IBM Systems Director, Storage Configuration Manager (SCM), Start Now Advisor, and

Tivoli Provisioning Manager for OS Deployments support for easy installation and management

򐂰 Energy-efficient design and innovative features to maximize productivity and reduce

power usage

򐂰 Help in protecting your IT investment through IBM BladeCenter family longevity,

compatibility, and innovation leadership in blades

򐂰 Support for the latest generation of IBM BladeCenter blades, helping provide investment

protection

Chapter 1. Introduction and general description 13

1.2.2 Number of PS700, PS701, and PS702 blades in a chassis

The number of POWER7 processor based blades that can be installed in a BladeCenter chassis depends on several factors:

򐂰 BladeCenter chassis type 򐂰 Number of power supplies installed 򐂰 Power supply voltage option (BladeCenter S only) 򐂰 BladeCenter power domain configuration

Table 1-3 shows the maximum number of PS700, PS701, and PS702 blades running in a maximum configuration (memory, disk, expansion cards) for each supported BladeCenter chassis that can be installed with fully redundant power and without performance reduction. IBM blades that are based on processor types other than POWER7 might reduce these numbers.

Table 1-3 PS700, PS701, and PS702 blades per chassis type

BladeCenter E

14 Slots Total 14 Slots Total 12 Slots Total 6 Slots Total

2 PS 4 PS 2 PS 4 PS 2 PS 4 PS 2 PS 4 PS 2 PS 4 PS

PS7006147146122626

BladeCenter H BladeCenter HT BladeCenter S

110VAC 208VAC

PS701 None None 7 14 6 12 2 6 2 6

PS702NoneNone37361313

a. BladeCenter E requires 2000 or 2300 watt power supplies

When mixing blades of different processor types in the same BladeCenter, the BladeCenter Power Configurator tool helps determine if the combination desired is valid. It is expected that this tool will be updated to include the PS700, PS701, and PS702 blade configurations. For more information about this update, see the following Web page:

http://www.ibm.com/systems/bladecenter/powerconfig

1.3 Operating environment

In this section, we list the operating environment specifications for the PS700, PS701, and PS702 blade servers and BladeCenter H and S.

IBM Blade Server PS700, PS701, and PS702

򐂰 Operating temperature

– 10 to 35 °C (50 to 95 °F) at 0 to 914 meters altitude (0 to 3000 feet) – 10 to 32 °C (50 to 90 °F) at 914 to 2133 meters altitude (3000 to 7000 feet)

򐂰 Relative Humidity 8% to 80% 򐂰 Maximum Altitude 2133 meters (7000 ft.)

14 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

IBM BladeCenter H

򐂰 Operating temperature

– 10.0 to 35 °C (50 to 95 °F) at 0 to 914 m (0 to 3000 ft.) – 10.0 to 32 °C (50 to 90 °F) at 914 to 2,133 m (3000 to 7000 ft.)

򐂰 Relative humidity 8% to 80% 򐂰 Maximum altitude: 2,133 meters (7000 ft.)

IBM BladeCenter S

򐂰 – Operating Temperature:

– 10 to 35 °C (50° to 95°F) at 0 to 914 m (0 to 3000 ft.) – 10 to 32°C (50° to 90°F) at 914 to 2,133 m (3000 to 7000 ft.)

򐂰 Relative humidity: 8% to 80% 򐂰 Maximum altitude: 2,133 meters (7000 ft.)

IBM BladeCenter E

򐂰 Operating temperature

– 10.0 to 35.0 °C (50 to 95 °F) at 0 to 914 m (0 to 3000 ft.) – 10.0 to 32.0 °C (50 to 90 °F) at 914 to 2133 m (3000 to 7000 ft.)

򐂰 Relative humidity: 8% to 80% 򐂰 Maximum altitude: 2133 meters (7000 ft.)

BladeCenter HT

򐂰 Operating temperature

– 5 to 40 °C (41 to 104 °F) at -60 to 1800 m (-197 to 6000 ft.) – 5 to 30 °C (41 to 86 °F) at 1800m to 4000m (6000 to 13000 ft.)

򐂰 Relative humidity 5% to 85% 򐂰 Maximum altitude: 4000 meters (13000 ft.)

1.4 Physical package

The PS700, PS701 and the PS702 Blade Servers are supported in BladeCenter H, HT and S. Bladecenter E supports PS700 Blade Servers only.

This section describes the physical dimension of the POWER7 Blade Servers and the supported BladeCenter chassis only. Table 1-4 shows the physical dimensions of the PS700, PS701, and PS702 blade servers.

Table 1-4 Physical dimensions of PS700, PS701, and PS702 Blade Servers

Dimension PS700 blade server PS701 blade server PS702 blade server

Height 9.65 inch (245 mm) 9.65 inch (245 mm) 9.65 inch (245 mm)

Width 1.14 inch (29 mm)

Single-wide blade

1.14 inch (29 mm) Single-wide blade

2.32 inch (59 mm) Double-wide blade

Depth 17.55 inch (445 mm) 17.55 inch (445 mm) 17.55 inch (445 mm)

Weight 9.6 lbs (4.35 kg) 9.6 lbs (4.35 kg) 19.2 lbs (8.7 kg)

Chapter 1. Introduction and general description 15

Table 1-5 shows the physical dimension of the BladeCenter chassis that supports the POWER7 processor based Blade Servers.

Table 1-5 Physical dimension of Supported BladeCenter chassis

Dimension BladeCenter H BladeCenter S BladeCenter E

Height 15.75“ (400 mm) 12” (305 mm) 12” (305 mm) 21“ (528 mm)

Width 17.4” (442 mm) 17.5” (445 mm) 17.5” (445 mm) 17.4” (442 mm)

Depth 28” (711 mm) 28.9” (734 mm) 28” (711 mm) 27.8” (706 mm)

a. PS700 only. The PS701 and PS702 are not supported in BladeCenter E chassis

1.5 System features

The PS700, PS701, and PS702 blade servers are 4-core, 8-core and 16-core POWER7 processor-based blade servers.This section describes the features on each of the POWER7 blade server. The following topics are covered:

򐂰 1.5.1, “PS700 system features” on page 16 򐂰 1.5.2, “PS701 system features” on page 18 򐂰 1.5.3, “PS702 system features” on page 20 򐂰 1.5.4, “Minimum features for the POWER7 processor-based blade servers” on page 21 򐂰 1.5.5, “Power supply features” on page 22 򐂰 1.5.6, “Processor” on page 22 򐂰 1.5.7, “Memory features” on page 23 򐂰 1.5.8, “I/O features” on page 24 򐂰 1.5.9, “Disk features” on page 28 򐂰 1.5.10, “Standard onboard features” on page 28

BladeCenter HT

1.5.1 PS700 system features

The BladeCenter PS700, model 8406-70Y, is shown in Figure 1-10 on page 17. The features of the server are as follows:

򐂰 Machine type and model number

8406-70Y

򐂰 Form factor

Single-wide (30 mm) blade

򐂰 Processors:

– Single-socket 4-core 64-bit POWER7 processor operating at a 3.0 GHz clock speed. – Based on CMOS 12S 45 nm SOI (silicon-on-insulator) technology, – Power consumption is 150w/socket – Single-wide (SW) Blade package

򐂰 Memory

–8 DIMM Slots – Minimum capacity 8 GB, maximum capacity 64 GB – Industry standard VLP DDR3 DIMMs

16 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

POWER7 4-core processor

Two disk drive bays

Eight DIMMs sockets

CIOv connector

CFFh connector

Figure 1-10 Top view of PS700 blade server

򐂰 Disk:

– Two disk drive bays support one or two SAS HDD – Hardware mirroring RAID 0, RAID 1, and RAID 10

򐂰 On-board integrated features:

– Service processor (SP) – Two 1 GB Ethernet ports (HEA) – SAS Controller – USB Controller that routes to the USB 2.0 port on the media tray – One Serial over LAN (SOL) Console through SP

򐂰 Expansion Card I/O Options:

– One CIOv expansion card slot (PCIe) – One CFFh expansion card slot (PCIe)

Chapter 1. Introduction and general description 17

1.5.2 PS701 system features

POWER7 8-Core processor

16 DIMM sockets

Disk drive bay

CIOv connector

CFFh connector

Connector for expansion blade (FC 8358)

The BladeCenter PS701 is shown in Figure 1-11.

Figure 1-11 Top view of the PS701 blade server

The features of the server are as follows: 򐂰 Machine type and model number

8406-71Y

򐂰 Form factor

Single-wide (30 mm) blade

򐂰 Processors:

– Single-socket 8-core 64-bit POWER7 processor operating at a 3.0 GHz clock speed – Based on CMOS 12S 45 nm SOI (silicon-on-insulator) technology – Power consumption is 150w/socket – Single-wide (SW) Blade package

򐂰 Memory

– 16 DIMM slots – Minimum capacity 8 GB, maximum capacity 128 GB – Industry standard VLP DDR3 DIMMs

򐂰 Disk

– One disk drive bays supports one SAS HDD (hard disk drive).

18 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

򐂰 On-board integrated features:

– Service processor (SP) – Two 1 GB Ethernet ports (HEA) – SAS Controller – USB Controller which routes to the USB 2.0 port on the media tray. – 1 Serial over LAN (SOL) Console through SP

򐂰 Expansion Card I/O Options:

– One CIOv expansion card slot (PCIe) – One CFFh expansion card slot (PCIe)

Chapter 1. Introduction and general description 19

1.5.3 PS702 system features

CIOv connector

CFFh connector

Screw-down point to attach to PS702 base blade

32 DIMM sockets (16 in each blade)

Connector to join the blades together

PS702 expansion blade (FC 8358)

PS702 base blade

Two POW ER7 8-core processors

CIOv connector

CFFh connector

Disk drive bay

The two halves of the BladeCenter PS702 are shown in Figure 1-12.

Figure 1-12 Top view of PS702 blade server

20 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

The features of the server are as follows: 򐂰 Machine type and model number

8406-71Y + FC 8358

򐂰 Form factor

Double-wide (60 mm) blade

򐂰 Processors:

– Two-socket 16-core 64-bit POWER7 processor operating at a 3.0 GHz clock speed – Based on CMOS 12S 45 nm SOI (silicon-on-insulator) technology – Power consumption is 150w/socket – Double-wide (SW) Blade package

򐂰 Memory

– 16 DIMM Slots – Minimum capacity 8 GB, maximum capacity 256 GB – Industry Standard VLP DDR3 DIMMs

򐂰 Disk

– Two disk drive bays (one on each blade) supports one or two SAS HDD – Hardware mirroring RAID 0, RAID 1, RAID 10

򐂰 On-board integrated features:

– Service processor (one on each blade

) – Four 1 GB Ethernet ports (HEA) – SAS Controller – USB Controller which routes to the USB 2.0 port on the media tray – 1 Serial over LAN (SOL) Console through FSP

򐂰 Expansion Card I/O Options:

– One CIOv expansion card slot (PCIe) – One CFFh expansion card slot (PCIe)

Note: The PS702 is 16-core POWER7 processor-based blade server that is a combination of a single-socket 8-core blade, model 8406-71Y (PS701), and a PS702 expansion blade, feature code #8358.

1.5.4 Minimum features for the POWER7 processor-based blade servers

At the minimum PS700, PS701, and PS702 requires a BladeCenter chassis and one processor socket per blade (four core single socket in PS700, eight core single socket in PS701 and two eight core single socket in PS702 blade servers), minimum memory (8 GB) and zero or one DASD, and a Language Group Specify (mandatory to order voltage nomenclature/language).

Each system has a minimum feature set to be valid. The minimum system configuration for a PS700 or PS701 is shown in Table 1-6 on page 22.

The service processor (or flexible service processor) on the expansion unit provides control but does not offer redundancy with the SP on the base unit.

Chapter 1. Introduction and general description 21

Table 1-6 Minimum features for PS700 PS701 and PS702 Blade Server

Category Minimum features required

BladeCenter chassis Supported BladeCenter chassis (refer to 1.2.1, “Supported BladeCenter chassis”

on page 4)

Processor 򐂰 4-core 3.0 GHz PS700 Blade (#8406-70Y)

򐂰 8-core 3.0 GHz PS701Blade (#8406-71Y 򐂰 16-core 3.0 GHz PS702 Blade(#8406-71Y + FC 8358)

Memory Two DDR3 Memory DIMM:

򐂰 For P700 and P701 8 GB (2 x 4 GB) Memory DIMMs, 1066 MHz (#8208) 򐂰 For P702 8 GB (2 x 4 GB) Memory DIMMs, 1066 MHz (#8208) on each base

board

Storage For AIX and Linux: 1x disk drive

For IBM i, 2x disk drives

1x Language Group Country specific (selected by the customer)

Operating system 1x primary operating system (one of these)

AIX/Linux/Virtual I/O Server:

򐂰 300 GB SAS 2.5-inch HDD (#8274)OR 򐂰 600 GB SAS 2.5 inch HDD (#8276)

IBM i (Required VIOS partition)

򐂰 300 GB SAS 2.5-inch HDD (#8274)OR ( 򐂰 600 GB SAS 2.5 inch HDD (#8276)

If Boot from SAN 8 GB Fibre Channel HBA is selected with FC #8240, #8242 or #8271 or Fibre Channel over Ethernet Adapter FC #8275 must be ordered

򐂰 AIX (#2146) 򐂰 Linux (#2147) 򐂰 IBM i (#2145) plus IBM i 6.1.1 (#0566)

1.5.5 Power supply features

The power consumption for each PS700, PS701, and PS702 blade server is 12V at 350 watts maximum, which is provided by the BladeCenter power supply modules. The maximum measured value is the worst case power consumption expected from a fully populated server under intensive workload. The maximum measured value also accounts for component tolerance and non-ideal operating conditions. Power consumption and heat load vary greatly by server configuration and use.

Use the IBM Systems Energy Estimator to obtain a heat output estimate based on a specific configuration. The Estimator is available from the following Web page:

http://www-912.ibm.com/see/EnergyEstimator

For information about power supply requirements for each of the BladeCenter chassis supported by POWER7 blade servers and the number of POWER7 blades supported, see

1.2.2, “Number of PS700, PS701, and PS702 blades in a chassis” on page 14.

1.5.6 Processor

The POWER7 3.0 GHz 64-bit POWER7 processor for blade servers is available in four-core (PS700), eight-core (PS701) or two eight-core (PS702) configurations. They are optimized to achieve maximum performance for both the system and its virtual machines. Couple that performance with PowerVM™ and you are now enabled for massive workload consolidation to drive maximum system use, predictable performance, and cost efficiency.

22 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

POWER7 Intelligent Threads Technology enables workload optimization by selecting the most suitable threading mode (Single thread (per core) or Simultaneous Multi-thread 2 or 4 modes also called 2-SMT and 4-SMT). The Intelligent Threads Technology can provide improved application performance. In addition, POWER7 processors can maximize cache access to cores, improving performance, using Intelligent Cache technology

POWER7 offers Intelligent Energy Management features that can dramatically and dynamically conserve power and further improve energy efficiency. These features enable the POWER7 processor to operate at a higher frequency if environmental conditions permit, for increased performance and performance per watt. Alternatively, if user settings permit, these features allow the processor to operate at a reduced frequency for significant energy savings.

The key processor feature on each of the P700 blade server are as follows: 򐂰 The PS700 blade server contains one four-core, 64-bit POWER7 3.0 GHz processor with

256 KB per processor core L2 cache and 4 MB per processor core L3 cache. No processor options are available.

򐂰 The PS701 blade server contains one eight-core, 64-bit POWER7 3.0 GHz processor with

256 KB per processor core L2 cache and 4 MB per processor core L3 cache. No processor options are available.

򐂰 The PS702 blade server is a double-wide that supports two eight-core, 64-bit POWER7

3.0 GHz processor with 256 KB per processor core L2 cache and 4 MB per processor core L3 cache. No processor options are available.

1.5.7 Memory features

The PS700, PS701, and PS702 blade servers uses industry standard VLP DDR3 memory DIMMs. Memory DIMMs must be installed in matched pairs with the same size and speed. For details about memory subsystem and layout, see 2.4, “Memory subsystem” on page 46.

The PS700, PS701, and PS702 blade serves have eight, 16, and 32 DIMM slots respectively. Memory is available in 4 GB or 8 GB DIMMs, both operating at a memory speed of 1066 MHz. The memory sizes can be mixed within a system. You can use pairs of 4 GB DIMMs with pairs of 8 GB DIMMs.

The POWER7 DDR3 memory uses a new memory architecture to provide greater bandwidth and capacity. This enables operating at a higher data rate for larger memory configurations. For details, see 2.4, “Memory subsystem” on page 46. Table 1-7 shows the DIMM features.

Table 1-7 Memory DIMM options

Feature code DIMM size Quantity Speed

8208 4 GB 2 1066 MHz

8209 8 GB 2 1066 MHz

Notes:

The DDR2 DIMMs used in JS23 and JS43 blade servers are not supported in the Power7 blade servers.

The announcement letter for the POWER7 processor-based blades incorrectly lists the memory speed of the 8 GB DIMMs to be 800 MHz.

Chapter 1. Introduction and general description 23

1.5.8 I/O features

The PS700 and PS701 have one CIOv PCIe expansion card slot and one CFFh PCIe high-speed expansion card slot. The PS702 blade server has two CIOv expansion card slots and two CFFh expansion card slots.

Table 1-8 shows the supported CIOv and CFFh expansion cards in POWER7 processor-based servers.

Table 1-8 Supported I/O Expansion Card on POWER7 Blades

Card Description Feature Code

CIOv

QLogic 8 Gb Fibre Channel Expansion Card (CIOv) 8242

QLogic 4 Gb Fibre Channel Expansion Card (CIOv) 8241

Emulex 8 Gb Fibre Channel Expansion Card (CIOv) 8240

3 GB SAS Passthrough Expansion Card (CIOv) 8246

CFFh

QLogic 8 GB Fibre Channel Expansion Card (CFFh) 8271

QLogic Ethernet and 4 Gb Fibre Channel Expansion Card (CFFh) 8252

QLogic 2-port 10 Gb Converged Network Adapter (CFFh) 8275

4X InfiniBand DDR Expansion Card (CFFh) 8258

QLogic 8 Gb Fibre Channel Expansion Card (CIOv)

The QLogic 8 Gb Fibre Channel Expansion Card (CIOv) for IBM BladeCenter, feature #8242, enables high-speed access for IBM blade servers to connect to a Fibre Channel storage area network (SAN). When compared to the previous-generation 4 Gb adapters, the new adapter doubles the throughput speeds for Fibre Channel traffic. As a result, you can manage increased amounts of data and possibly benefit from a reduced hardware expense.

The card has the following features:

򐂰 CIOv form factor 򐂰 QLogic 2532 8 Gb ASIC 򐂰 PCI Express 2.0 host interface 򐂰 Support for two full-duplex Fibre Channel ports at 8 Gbps maximum per channel 򐂰 Support for Fibre Channel Protocol Small Computer System Interface (FCP-SCSI) and

Fibre Channel Internet Protocol (FC-IP)

򐂰 Support for Fibre Channel service (class 3) 򐂰 Support for switched fabric, point-to-point, and Fibre Channel Arbitrated Loop (FC-AL)

connections

򐂰 Support for NPIV when installed in the PS700, PS701, and PS702 Blade Servers

For more information, see the IBM Redbooks at-a-glance guide at the following Web page:

http://www.redbooks.ibm.com/abstracts/tips0692.html?Open

24 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

QLogic 4 Gb Fibre Channel Expansion Card (CIOv)

The QLogic 4 Gb Fibre Channel Expansion Card (CIOv) for BladeCenter, feature #8241, enables you to connect the BladeCenter servers with CIOv expansion slots to a Fibre Channel SAN. Pick any Fibre Channel storage solution from the IBM System Storage® DS3000, DS4000®, DS5000, and DS8000® series, and begin accessing data over a high-speed interconnect. This card is installed into the PCI Express CIOv slot of a supported blade server. It provides connections to Fibre Channel-compatible modules located in bays 3 and 4 of a supported BladeCenter chassis. A maximum of one QLogic 4 Gb Fibre Channel Expansion Card (CIOv) is supported per single-wide (30 mm) blade server.

The card has the following features:

򐂰 CIOv form factor 򐂰 PCI Express 2.0 host interface 򐂰 Support for two full-duplex Fibre Channel ports at 4 Gbps maximum per channel 򐂰 Support for Fibre Channel Protocol SCSI (FCP-SCSI) and Fibre Channel Internet Protocol

(FC-IP)

򐂰 Support for Fibre Channel service (class 3) 򐂰 Support for switched fabric, point-to-point, and Fibre Channel Arbitrated Loop (FC-AL)

connections

For more information, see the IBM Redbooks at-a-glance guide at the following Web page:

http://www.redbooks.ibm.com/abstracts/tips0695.html?Open

Emulex 8 Gb Fibre Channel Expansion Card (CIOv)

The Emulex 8 Gb Fibre Channel Expansion Card (CIOv) for IBM BladeCenter, feature #8240, enables high-performance connection to a SAN. The innovative design of the IBM BladeCenter midplane enables this Fibre Channel adapter to operate without the need for an optical transceiver module. This saves significant hardware costs. Each adapter provides dual paths to the SAN switches to ensure full redundancy. The exclusive firmware-based architecture allows firmware and features to be upgraded without taking the server offline or rebooting and without the need to upgrade the driver.

The card has the following features:

򐂰 Support of the 8 Gbps Fibre Channel standard 򐂰 Use of the Emulex "Saturn" 8 Gb Fibre Channel I/O Controller (IOC) chip 򐂰 Enablement of high-speed and dual-port connection to a Fibre Channel SAN 򐂰 Can be combined with a CFFh card on the same blade server 򐂰 Comprehensive virtualization capabilities with support for N_Port ID Virtualization (NPIV)

and Virtual Fabric

򐂰 Simplified installation and configuration using common HBA drivers 򐂰 Efficient administration by using HBAnyware for HBAs anywhere in the SAN 򐂰 Common driver model that eases management and enables upgrades independent of

HBA firmware

򐂰 Support of BladeCenter Open Fabric Manager 򐂰 Support for NPIV when installed in the PS700, PS701, and PS702 blade servers

For more information, see the IBM Redbooks at-a-glance guide at the following Web page:

http://www.redbooks.ibm.com/abstracts/tips0703.html?Open

Chapter 1. Introduction and general description 25

3 Gb SAS Passthrough Expansion Card (CIOv)

This card, feature #8246, is an expansion card that offers the ideal way to connect the supported BladeCenter servers to a wide variety of SAS storage devices. The SAS connectivity card can connect to the Disk Storage Modules in the BladeCenter S. The card routes the pair of SAS channels from the blade's onboard SAS controller to the SAS switches installed in the BladeCenter chassis.

Tip: This card is also known as the SAS Connectivity Card (CIOv) for IBM BladeCenter.

This card is installed into the CIOv slot of the supported blade server. It provides connections to SAS modules located in bays 3 and 4 of a supported BladeCenter chassis.

The card has the following features:

򐂰 CIOv form factor 򐂰 Provides external connections for the two SAS ports of blade server's onboard SAS

controller

򐂰 Support for two full-duplex SAS ports at 3 Gbps maximum per channel 򐂰 Support for SAS, SSP, and SMP protocols 򐂰 Connectivity to SAS storage devices

For more information, see the IBM Redbooks at-a-glance guide at the following Web page:

http://www.redbooks.ibm.com/abstracts/tips0701.html?Open

QLogic 8 Gb Fibre Channel Expansion Card (CFFh)

The QLogic 8 Gb Fibre Channel Expansion Card (CFFh) for IBM BladeCenter, feature #8271, is installed in the blade server and allows connectivity to high-speed switch bays. This expansion card provides flexibility for connecting the blade server to the horizontally oriented BladeCenter H modules in bays 7 and 8 or bays 9 and 10 when using the Multi-Switch Interconnect Module (MSIM). This card is used in conjunction with MSIM on the chassis and requires that a Fibre Channel capable I/O module is installed in the right position of the MSIM. It can be combined with a CFFv I/O card on the same high-speed blade server.

The card has the following features: 򐂰 Support for Fibre Channel protocol SCSI (FCP-SCSI) and Fibre Channel Internet protocol

(FCP-IP)

򐂰 Support for point-to-point fabric connection (F-port fabric login) 򐂰 Support for Fibre Channel service (classes 2 and 3) 򐂰 Support for NPIV when installed in PS701, PS700 and PS702 Blade Servers 򐂰 Support for remote startup (boot) operations 򐂰 Support for BladeCenter Open Fabric Manager 򐂰 Support for Fibre Device Management Interface (FDMI) standard (VESA standard) 򐂰 Fibre Channel 8 Gbps, 4 Gbps, or 2 Gbps auto-negotiation

Note: This card is also known as the QLogic Ethernet and 8 Gb Fibre Channel Expansion Card (CFFh) for IBM BladeCenter. However, the Ethernet ports were not supported in the POWER7 processor-based blades at the time of writing.

For more information, see the IBM Redbooks at-a-glance guide at the following Web page:

26 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

http://www.redbooks.ibm.com/abstracts/tips0690.html?Open

QLogic Ethernet and 4 Gb Fibre Channel Expansion Card (CFFh)

The QLogic Ethernet and 4 Gb Fibre Channel Expansion Card, feature #8252, is a CFFh high speed blade server expansion card with two 4 Gb Fibre Channel ports and two 1 Gb Ethernet ports. It provides QLogic 2432M PCI-Express x4 ASIC for 4 Gb 2-port Fibre Channel and Broadcom 5715S PCI-Express x4 ASIC for 1 Gb 2-port Ethernet. This card is used in conjunction with the Multi-Switch Interconnect Module and is installed in the left position of the MSIM and a Fibre Channel capable I/O module is installed in the right position of the MSIM. Both switches do not need to be present at the same time because the Fibre Channel and Ethernet networks are separate and distinct. It can be combined with a CFFv I/O card on the same high-speed blade server.

The card has the following features: 򐂰 Support for Fibre Channel protocol SCSI (FCP-SCSI) and Fibre Channel Internet protocol

(FCP-IP)

򐂰 Support for point-to-point fabric connection (F-port fabric login). Support for remote startup

(boot) operations

򐂰 Support for BladeCenter Open Fabric Manager

For more detail see the IBM Redbooks publication IBM BladeCenter Products and Technology, SG24-7523, available at the following Web page:

http://www.redbooks.ibm.com/abstracts/sg247523.html?Open

QLogic 2-port 10 Gb Converged Network Adapter (CFFh)

The QLogic 2-port 10 Gb Converged Network Adapter (CFFh) for IBM BladeCenter, feature #8275, offers robust 8 Gb Fibre Channel storage connectivity and 10 Gb networking over a single Converged Enhanced Ethernet (CEE) link. Because this adapter combines the functions of a network interface card and a host bus adapter on a single converged adapter, clients can realize potential benefits in cost, power, and cooling, and data center footprint by deploying less hardware.

The card has the following features:

򐂰 CFFh PCI Express 2.0 x8 adapter 򐂰 Communication module: QLogic ISP8112 򐂰 Support for up to two CEE HSSMs in a BladeCenter H or HT chassis 򐂰 Support for 10 Gb Converged Enhanced Ethernet (CEE) 򐂰 Support for Fibre Channel over Converged Enhanced Ethernet (FCoCEE) 򐂰 Full hardware offload for FCoCEE protocol processing 򐂰 Support for IPv4 and IPv6 򐂰 Support for SAN boot over CEE, PXE boot, and iSCSI boot 򐂰 Support for Wake on LAN

For more information, see the IBM Redbooks at-a-glance guide at the following Web page:

http://www.redbooks.ibm.com/abstracts/tips0716.html?Open

4X InfiniBand DDR Expansion Card (CFFh)

The InfiniBand 4X DDR Expansion Card for IBM BladeCenter delivers low-latency and high-bandwidth for performance-driven server and storage clustering applications

Chapter 1. Introduction and general description 27

The card has the following features:

򐂰 1.2us MPI ping latency 򐂰 20 Gbps InfiniBand ports 򐂰 CPU offload of transport operations 򐂰 End-to-end QoS and congestion control 򐂰 Hardware-based I/O virtualization 򐂰 TCP/UDP/IP stateless offload

For more information, see the IBM Redbooks publication IBM BladeCenter Products and Technology, SG24-7523, available at the following Web page:

http://www.redbooks.ibm.com/abstracts/sg247523.html?Open

1.5.9 Disk features

The PS700 blade servers has two disk bays:

򐂰 In the first bay it can have one 2.5 inch SAS HDD 򐂰 In second bay it can have one 2.5 inch SAS HDD

The PS701 blade servers has one disk bay. In this bay it can have one 2.5 inch SAS HDD.

The PS702 blade servers have two disk bays (one on each of the blade):

򐂰 On the base card it can have one 2.5 inch SAS HDD. 򐂰 On the expansion unit it can have one 2.5 inch SAS HDD.

Table 1-6 lists the supported disk features on the PS700, PS701 and PS701 blade servers

Table 1-9 Supported disk drives

Feature code Description

8274 300 GB 10K SFF SAS HDD

8276 600 GB 10K SFF SAS HDD

1.5.10 Standard onboard features

In this section, we describe the standard on-board features.

Service processor

The service processor (or flexible service processor, FSP) is the main integral part of the blade server. It monitors and manages system hardware, resources, and devices. It does the system initialization, configuration, and thermal/power management. It takes corrective action if required.

The PS700 and PS701 have only one service processor. The PS702 blade server has two FSPs (one on each blade). However, the second service processor is only in IO mode and is not redundant to the one on the base blade.

For more details about service processors, see 2.10, “External disk subsystems” on page 68.

Host Ethernet Adapter (HEA)

The integrated IO Hub provides two 1 GB Ethernet ports also called HEAs (host Ethernet adapters). HEA is part of the Integrated Virtual Ethernet subsystem (IVE). Each HEA has their own MAC address and can have a maximum of 16 logical ports. These logical ports can be

28 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

used to communicate to the multiple Lpars, which helps in virtualization/sharing of the Ethernet port without using the Ethernet bridge on the Virtual IO Server.

The PS700 and PS701 blade servers have two 1 GB HEA. The PS702 has four 1 GB HEAs (two on each system board).

For more details about HEA and IVE subsystems, see 2.7, “Integrated Virtual Ethernet” on page 61.

SAS Controller

The integrated SAS controller is used to drive the local SAS storage.

The 3 GB SAS Passthrough expansion card can be used to connect to the BladeCenter SAS switch, which can be connected to the external storage. This SAS passthrough expansion card can also be used to connect to BladeCenter S internal drive SAS drives. See “3 Gb SAS Passthrough Expansion Card (CIOv)” on page 26 for more information.

The blades servers each have one integrated SAS controller. The SAS controller host PCI-X interface to P5IOC2 I/O Hub is 64 bits wide and operates at 133 MHz. The integrated SAS controller supports hardware mirroring RAID 0, RAID 1 or RAID 10 when two HDDs are used in P701 or P702 blade servers

For more information, see “SAS adapter” on page 58 and 2.9, “Internal storage” on page 65

USB controller

The USB controller connects the USB bus to the midplane, which is then routed to the media tray in the BladeCenter chassis to connect to USB devices (such as an optical drive or diskette drive).

For more information, see 2.6.6, “Embedded USB controller” on page 60.

Serial over LAN (SOL)

The integrated SOL function routes the console data stream over standard dual 1 GB Ethernet ports to the Advance Management Module. The PS700, PS701, and PS702 do not have on-board video chips and do not support KVM connections. Console access is only by SOL connection. Each blade can have a single SOL session, however there can be multiple telnet or ssh sessions to the BladeCenter AMM each acting as a SOL connection to a different blade.

For more information, see 2.8.1, “Server console access by SOL” on page 63.

1.6 Supported BladeCenter I/O modules

With IBM BladeCenter, the switches and other I/O modules are installed in the chassis rather than as discrete devices installed in the rack.

The BladeCenter chassis supports a wide variety and range of I/O switch modules. These switch modules are matched to the type, slot location, and form factor of the expansion cards installed in a blade server. For more information, see 1.5.8, “I/O features” on page 24 and 2.6, “Internal I/O subsystem” on page 52.

The I/O switch modules described in the following sections are matched with the on-board HEAs and supported expansion cards in the PS700, PS701, and PS702 blades. In general, the integrated ports on the blades and the additional ports on the expansion cards can

Chapter 1. Introduction and general description 29

function with a single supporting I/O switch module. However, I/O switch modules should be added in pairs to eliminate single points of failure.

For the latest and most current information about blade, expansion card, switch module, and chassis compatibility and interoperability see the IBM BladeCenter Interoperability Guide at the following Web page:

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016

1.6.1 Ethernet switch and intelligent pass through modules

Various types of Ethernet switch and pass through modules from several manufacturers are available for BladeCenter, and they support different network layers and services. These I/O modules provide external and chassis blade-to-blade connectivity.

The HEAs are on-blade ports that are part of the IVE subsystem that is a standard part of the PS700, PS701, and PS702 blades. For more information, see 2.7, “Integrated Virtual Ethernet” on page 61. There are two physical ports on the PS700 and PS701 and four physical ports on the PS702. The data traffic from these on-blade 1 Gb Ethernet adapters is directed to I/O switch bays 1 and 2 on all BladeCenter chassis except BladeCenter S. On the BladeCenter S the connections for all blade HEA ports are wired to I/O switch bay 1.

To provide external network connectivity and a SOL system console through the BladeCenter Advanced Management Module, at least one Ethernet I/O module is required in switch bay 1. For more information, see 2.8.1, “Server console access by SOL” on page 63.

In addition to the HEA ports, the QLogic Ethernet and 4 Gb Fibre Channel Expansion Card (CFFh) card can provide two additional 1 Gb Ethernet ports per card.

A list of available Ethernet I/O modules that support the on-blade HEA ports and expansion card are shown in Table 1-10 on page 30. Not all switches are supported in every configuration of BladeCenter. Complete compatibility matrixes are available on the following Web pages:

򐂰 ServerProven®:

http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html

򐂰 BladeCenter Interoperability Guide

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016

Table 1-10 Ethernet switch modules

Part number

43W4395 5450 Cisco Catalyst Switch Module 3012 4 x Gigabit Ethernet Layer 2/3

41Y8523 2989 Cisco Catalyst Switch Module 3110G 4 x Gigabit Ethernet,

41Y8522 2988 Cisco Catalyst Switch Module 3110X 1 x 10 Gb Ethernet,

39Y9324 1484 IBM Server Connectivity Module 6 x Gigabit Ethernet Layer 2

Feature Code

Option description Number x type

of external ports

2 x StackWise Plus

Network layers

Layer 2/3

32R1860 1495 BNT L2/3 Copper Gigabit Ethernet Switch Module 6 x Gigabit Ethernet Layer 2/3

32R1861 1496 BNT L2/3 Fibre Gigabit Ethernet Switch Module 6 x Gigabit Ethernet Layer 2/3

32R1859 1494 BNT Layer 2-7 Gigabit Ethernet Switch Module 4 x Gigabit Ethernet Layer 2/7

30 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Part number

Feature Code

Option description Number x type

of external ports

Network layers

44W4404 1590 BNT 1/10 Gb Uplink Ethernet Switch Module 3 x 10 Gb Ethernet,

6 x Gigabit Ethernet

44W4483 5452 Intelligent Copper Pass-thru Module 14 x Gigabit Ethernet -

1.6.2 SAS I/O modules

SAS I/O modules provide affordable storage connectivity for BladeCenter chassis using SAS technology to create simple fabric for external shared or non-shared storage attachments. A SAS module can also perform RAID controller functions inside the BladeCenter S chassis for HDDs installed into Disk Storage Module (DSM) and external EXP3000 expansions. The SAS RAID Controller Module and DSMs in a BladeCenter S provides RAID 0, 1, 5, and 10 support

In the PS700, PS701, and PS702 blades, the 3 Gb SAS Passthrough Expansion Card (CIOv) is required for external SAS connectivity. The SAS expansion card requires SAS I/O modules in switch bays 3 and 4 of all supported BladeCenters.

Table 1-11 lists the SAS I/O modules and support matrix.

Table 1-11 SAS I/O modules supported by the SAS pass through card

Feature

Part number

code Description

3 Gb SAS

pass-thru

card

BC-E

BC-H

BC-HT

BC-S

Layer 2/3

MSIM

MSIM-HT

39Y9195 2980 SAS Connectivity Module Ye s Ye s Ye s Ye s Ye s N o N o

43W3584 3734 SAS RAID Controller Module

Yes NoNoNoYe s N o N o

1.6.3 Fibre Channel switch and pass-thru modules

Fibre Channel I/O modules are available from several manufacturers. These I/O modules can provide full SAN fabric support up to 8 Gb.

The following 4 Gb and 8 Gb Fibre Channel cards are CIOv form factor and require a Fibre Channel switch or Intelligent Pass Through module in switch bays 3 and 4 of all supported BladeCenters. The CIOv expansion cards are as follows:

򐂰 Emulex 8 Gb Fibre Channel Expansion Card (CIOv) 򐂰 QLogic 4 Gb Fibre Channel Expansion Card (CIOv) 򐂰 QLogic 8 Gb Fibre Channel Expansion Card (CIOv)

Additional 4 Gb and 8 Gb Fibre Channel ports are also available in the CFFh form factor expansion cards. These cards require the use of the MSIM in a BladeCenter H or a MSIM-HT in a BladeCenter HT plus Fibre Channel I/O modules. The CFFh Fibre Channel cards are as follows:

򐂰 QLogic Ethernet and 4 Gb Fibre Channel Expansion Card (CFFh) 򐂰 QLogic 8 Gb Fibre Channel Expansion Card (CFFh)

A list of available Fibre Channel I/O modules that support the CIOv and CFFh expansion cards is shown in Table 1-12. Not all modules are supported in every configuration of BladeCenter. Complete compatibility matrixes are available on the following Web pages:

Chapter 1. Introduction and general description 31

򐂰 ServerProven:

http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html

򐂰 BladeCenter Interoperability Guide

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016

Table 1-12 Fibre Channel I/O modules

Part number

32R1812 1569 Brocade 20-port SAN Switch Module 6 4 Gbps 32R1813 1571 Brocade 10-port SAN Switch Module 42C1828 5764 Brocade Enterprise 20-port 8 Gb SAN Switch Module 6 8 Gbps 44X1920 5481 Brocade 20-port 8 Gb SAN Switch Module 6 8 Gbps 44X1921 5483 Brocade 10-port 8 Gb SAN Switch Module 6 8 Gbps 39Y9280 2983 Cisco Systems 20-port 4 Gb FC Switch Module 6 4 Gbps 39Y9284 2984 Cisco Systems 10-port 4 Gb FC Switch Module 26R0881 1560 QLogic 20-port 4 Gb Fibre Channel Switch Module 6 4 Gbps 43W6725 2987 QLogic 20-port 4 Gb SAN Switch Module 6 4 Gbps 43W6724 2986 QLogic 10-port 4 Gb SAN Switch Module 43W6723 2985 QLogic 4 Gb Intelligent Pass-thru Module 44X1905 5478 QLogic 20-Port 8 Gb SAN Switch Module 6 8 Gbps 44X1907 5482 QLogic 8 Gb Intelligent Pass-thru Module 46M6172 4799 QLogic Virtual Fabric Extension Module 6 8 Gbps

a. Only 10 ports are activated on these switches. An optional upgrade to 20 ports (14 internal + 6 external) is

available.

b. Can be upgraded to full fabric switch

Feature Code

Description Number of

external ports

6 4 Gbps

6 4 Gbps 6 4 gbps

6 8 Gbps

Port interface bandwidth

1.6.4 Converged networking I/O modules

There are two basic solutions to implement Fibre Channel over Ethernet (FCoE) over a converged network with a BladeCenter.

򐂰 The first solution uses a top-of-rack FCoE capable switch in conjunction with converged

network capable 10 Gb Ethernet I/O modules in the BladeCenter. The FCoE-capable top-of-rack switch provides connectivity to the SAN.

򐂰 The second BladeCenter H solution uses a combination of converged network capable 10

Gb Ethernet switch modules and fabric extension modules to provide SAN connectivity, all contained within the BladeCenter H I/O bays.

Implementing either solution with the PS700, PS701, and PS702 blades requires the QLogic 2-port 10 Gb Converged Network Adapter (CFFh). The QLogic Converged Network Adapter (CNA) provides 10 Gb Ethernet and 8 Gb Fibre Channel connectivity over a single CEE link. This card is a CFFh form factor with connections to BladeCenter H and HT I/O module bays 7 and 9.

Table 1-13 shows the currently available I/O modules that are available to provide a FCoE solution.

32 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Table 1-13 Converged network modules supported by the QLogic CNA

Part Number

46C7191 1639 BNT Virtual Fabric 10 Gb Switch Module for IBM BladeCenter

46M6181 1641 10 Gb Ethernet Pass-Thru Module for BladeCenter

46M6172 4799 QLogic Virtual Fabric Extension Module for IBM BladeCenter

46M6071

a. Used for top-of-rack solution. b. Use with Fabric Extension Module for self contain BladeCenter solution. c. Also requires BNT Virtual Fabric 10 Gb Switch Module. d. BladeCenter H only. e. Support is planned.

Feature Code

0072

Description Number of

Cisco Nexus 4001I Switch Module for IBM BladeCenter

a e

1.6.5 InfiniBand switch module

The Voltaire 40 Gb InfiniBand Switch Module for BladeCenter provides InfiniBand QDR connectivity between the blade server and external InfiniBand fabrics in non-blocking designs, all on a single device. Voltaire's high speed module also accommodates performance-optimized fabric designs using a single BladeCenter chassis or stacking multiple BladeCenter chassis without requiring an external InfiniBand switch.

c d

a b

external ports

10 x 10 Gb SFP+

14 x 10 Gb SFP+

6 x 8 Gb FC SFP

6 x 10 Gb SFP+

The InfiniBand switch module offers 14 internal ports, one to each server, and 16 ports out of the chassis per switch.

The module's HyperScale architecture also provides a unique interswitch link or mesh capability to form highly scalable, cost-effective, and low latency fabrics. Because this switch has 16 uplink ports, they can create a meshed architecture and still have unblocked access to data using the 14 uplink ports. This solution can scale from 14 to 126 nodes and offers latency of less than 200 nanoseconds, allowing applications to operate at maximum efficiency.

The PS700, PS701, and PS702 blades connect to the Voltaire switch through the CFFh form factor 4X InfiniBand DDR Expansion Card (CFFh). The card is only supported in a BladeCenter H and the two ports are connected to high speed I/O switch bays 7/8 and 9/10.

The Voltaire 40 Gb InfiniBand Switch Module for the BladeCenter H is shown in Table 1-14.

Table 1-14 InfiniBand switch module for IBM BladeCenter

Part number

46M6005 0057 Voltaire 40 Gb InfiniBand Switch Module

a. BladeCenter H only

Feature code

Description Number of

1.6.6 Multi-switch Interconnect Module

Type of external

external ports

16 4X QDR (40 Gbps)

ports

The MSIM is a switch module container that fits in the high speed switch bays (bays 7 and 8 or bays 9 and 10) of the BladeCenter H chassis. Up to two MSIMs can be installed in the BladeCenter H. The MSIM supports most standard switch modules.I/O module. I/O module to MSIM compatibility matrixes can be reviewed at the following Web pages:

򐂰 ServerProven:

Chapter 1. Introduction and general description 33

http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html

Right bay for Fibre Channel Switch Modules

Left bay for Ethernet Switch Modules

򐂰 BladeCenter Interoperability Guide

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016

With PS700, PS701, and PS702 blades, the following expansion cards require a MSIM in a BladeCenter H chassis:

򐂰 QLogic Ethernet and 4 Gb FibreChannel Expansion Card (CFFh) 򐂰 QLogic 8 Gb Fibre Channel Expansion Card (CFFh)

The MSIM is shown in Figure 1-13.

Note: The MSIM comes standard without any I/O modules installed. They need to be ordered separately. In addition, the use of MSIM modules requires that all four power modules be installed in the BladeCenter H chassis.

Figure 1-13 Multi-switch Interconnect Module

Table 1-15 shows MSIM ordering information.

Table 1-15 MSIM ordering information

Description Part Number Feature Code

MSIM for IBM BladeCenter 39Y9314 1465

1.6.7 Multi-switch Interconnect Module for BladeCenter HT

The Multi-switch Interconnect Module for BladeCenter HT (MSIM-HT) is a switch module container that fits in the high-speed switch bays (bays 7 and 8 or bays 9 and 10) of the BladeCenter HT chassis. Up to two MSIM s can be installed in the BladeCenter HT. The MSIM-HT accepts two supported standard switch modules as shown in Figure 1-14 on page 35.

The MSIM-HT has a reduced number of supported standard I/O modules compared to the MSIM. I/O module to MSIM-HT compatibility matrixes can be viewed at the following Web pages.

34 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

For the latest support information, see one of the following resources: 򐂰 ServerProven:

http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html

򐂰 BladeCenter Interoperability Guide

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016

With PS700, PS701, and PS702 blades the QLogic Ethernet and 4 Gb FibreChannel Expansion Card (CFFh) requires a MSIM-HT in a BladeCenter HT chassis.

Note: The MSIM-HT comes standard without any I/O modules installed. They need to be ordered separately. In addition, the use of MSIM-HT modules requires that all four power modules be installed in the BladeCenter HT chassis.

Figure 1-14 Multi-switch Interconnect Module for BladeCenter HT

Table 1-16 shows MSIM-HT ordering information.

Table 1-16 MSIM-HT ordering information

Description Part Number Feature Code

Multi-switch Interconnect Module for BladeCenter HT 44R5913 5491

1.7 Comparison between PS700, PS701, PS702, and 750 models

This section describes the difference between the POWER7 Blade servers and the entry POWER7 Rack Server (Power 750). This helps to better position the POWER7 processor Blade Servers.

The POWER7 Blade Server configuration offers three blade servers. The PS700 is 4-core, the PS701 is 8-Core, and the PS702 is a 16-core Power7 based processor running at 3.0 GHz

The Power 750 offers a a 6-core or 8-core configuration. The 6-core Power7 processor runs at 3.3 GHz and the 8-Core runs at 3.0 GHz, 3.3 GHz or 3.5 GHz systems.

The POWER7 processor has 4 MB L3 cache per core and 256 KB L2 cache per core.

Chapter 1. Introduction and general description 35

Table 1-17 compares the processor core options and frequencies, L3 cache sizes between the P& Blade servers and entry Power7 rack server Power 750.

Table 1-17 Comparison of P7 Blade Server and P750 Server

System Cores per

processor

PS700 blade 4 3.0 16 MB 4 / 4 4 GB / 64 GB Single-wide

PS701 blade 8 3.0 32 MB 8 / 8 16 GB / 128 GB Single-wide

PS702 blade 8 3.0 32 MB 16 / 16 32 GB / 256 GB Double-wide

Power 750 6 3.3 24 MB 6 / 24 8 GB / 512 GB Rack

Power 750 8 3.0 / 3.3 / 3.55 32 MB 8 / 32 8 GB / 512 GB Rack

Frequency in (GHz)

L3 cache per processor

Minimum / Maximum cores

Minimum / Maximum memory

Form factor

For a detailed comparison, see 2.5, “Technical comparison” on page 51.

1.8 Building to order

You can perform a build to order configuration using the IBM Configurator for e-business (e-config) where you specify each configuration feature that you want on the system. You build on top of the base-required features.

The configurator allows you to select an pre-configured Express model or to build a system to order.

The recommendation is to start with one of several available starting configurations, such as the IBM Editions. These solutions are available at initial system-order time with a starting configuration that is ready to run.

1.9 Model upgrades

The PS700, PS701, and PS702 are new serial-number blade servers. There are no upgrades from POWER5™ or POWER6® blade servers to POWER7 blade servers, which retain the serial number.

However, you can upgrade a PS701 server to a PS702 with the feature code #8358. Feature code 8358 delivers an additional eight-core 3.0 GHz processor, a second set of 16 DIMM slots, and an additional disk bay to the PS701 blade server. Thus you can have an upgrade of two eight-core 3.0 GHz POWER7 processor, with a maximum of 256 GB memory and two 300 GB or 600 GB SAS HDD.

36 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Chapter 2. Architecture and technical

overview

This chapter discusses the overall system architecture of the POWER7 processor-based blade servers and provides details about each major subsystem and technology.

The topics covered are:

򐂰 2.1, “Architecture” on page 38 򐂰 2.2, “The IBM POWER7 processor” on page 38 򐂰 2.3, “POWER7 processor-based blades” on page 46 򐂰 2.4, “Memory subsystem” on page 46 򐂰 2.5, “Technical comparison” on page 51 򐂰 2.6, “Internal I/O subsystem” on page 52 򐂰 2.7, “Integrated Virtual Ethernet” on page 61 򐂰 2.8, “Service processor” on page 63 򐂰 2.9, “Internal storage” on page 65 򐂰 2.10, “External disk subsystems” on page 68 򐂰 2.11, “IVM” on page 71 򐂰 2.12, “Operating system support” on page 72 򐂰 2.13, “IBM EnergyScale” on page 74

Note: The bandwidths that are provided throughout the chapter are theoretical maximums used for reference.

2.1 Architecture

DIMM P1-C 7 DIMM P1-C 8

Memory

buffer

DIMM P1-C 2

DIMM P1-C 1

Memory

buffer

DIMM P1-C 6

DIMM P1-C 5

DIMM P1-C 3 DIMM P1-C 4

DIMM P1-C15 DIMM P1-C16

Memory

buffer

DIMM P1-C10

DIMM P1-C9

DIMM P1-C14

DIMM P1-C13

DIMM P1-C11 DIMM P1-C12

Battery

Vital

product

data card

SMP

Connector

SAS Ports to

SMP Connector

SAS Ports

NVRAM SDRAM

NANO

Flash

SDRAM NVRAM

GX+

BUS

CFFh

Connector

P1-C20

P7 processor

chip

Memory

buffer

GX+ Bridge

PCIe

HEA

MII/RMII

Switch

Enet

FSP

RS485

MUX

USB

Enet

Switch

CIOv P1-C19

SAS P1-D1

2.5" HDD

SAS

Controller

PCIX

1Gb

Blade Conn.

This chapter discuses the overall system architecture represented by Figure 2-1, with its major components described in the following sections.

Figure 2-1 PS701 logical data flow

2.2 The IBM POWER7 processor

The IBM POWER7 processor represents a leap forward in technology achievement and associated computing capability. The multi-core architecture of the POWER7 processor has been matched with innovation across a wide range of related technologies to deliver leading throughput, efficiency, scalability, and reliability, availability, and serviceability (RAS).

Although the processor is an important component in delivering outstanding servers, many elements and facilities have to be balanced across a server to deliver maximum throughput. As with previous generations of systems based on POWER processors, the design philosophy for POWER7 processor-based systems is one of system-wide balance in which the POWER7 processor plays an important role.

38 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

IBM has been innovative to achieve required levels of throughput and bandwidth. Areas of

Core

4MB L3

Memory Controller 1

L2 C1

Core

4MB L3

Memory Controller 0

Core

4MB L3

Core

4MB L3

Core

4MB L3

Core

4MB L3

L2 C1

Core

4MB L3

L2 C1

Core

4MB L3

SMP

GX+ Bridge

Memory buffers

innovation for the POWER7 processor and POWER7 processor-based systems include (but are not limited to) the following elements:

򐂰 On-chip L3 cache implemented in embedded dynamic random access memory (eDRAM) 򐂰 Cache hierarchy and component innovation 򐂰 Advances in memory subsystem 򐂰 Advances in off-chip signalling 򐂰 Exploitation of long-term investment in coherence innovation

The superscalar POWER7 processor design also provides a variety of other capabilities:

򐂰 Binary compatibility with the prior generation of POWER processors 򐂰 Support for PowerVM virtualization capabilities, including PowerVM Live Partition Mobility

to and from POWER6 and POWER6+™ processor-based systems

Figure 2-2 shows the POWER7 processor die layout with the major areas identified: processor cores, L2 cache, L3 cache and chip interconnection, simultaneous multiprocessing (SMP) links, and memory controllers.

Figure 2-2 POWER7 processor architecture

2.2.1 POWER7 processor overview

The POWER7 processor chip is fabricated with the IBM 45 nm Silicon-On-Insulator (SOI) technology using copper interconnects, and implements an on-chip L3 cache using eDRAM.

The POWER7 processor chip is 567 mm (transistors). Eight processor cores are on the chip, each with 12 execution units, 256 KB of L2 cache, and access to up to 32 MB of shared on-chip L3 cache.

For memory access, the POWER7 processor includes two DDR3 (Double Data Rate 3) memory controllers, each with four memory channels. To scale effectively, the POWER7 processor uses a combination of local and global SMP links with high coherency bandwidth and makes use of the IBM dual-scope broadcast coherence protocol.

and is built using 1.2 billion components

Chapter 2. Architecture and technical overview 39

Table 2-1 summarizes the technology characteristics of the POWER7 processor.

Table 2-1 Summary of POWER7 processor technology

Technology POWER7 processor

Die size 567 mm

Fabrication technology 򐂰 45 nm lithography

򐂰 Copper interconnect 򐂰 Silicon-on-Insulator 򐂰 eDRAM

Components 1.2 billion components (transistors) offering the equivalent

function of 2.7 billion (For further details, see 2.2.6, “On-chip L3 cache innovation and intelligent cache” on page 43)

Processor cores 8

Max execution threads core/chip 4/32

L2 cache core/chip 256 KB / 2 MB

On-chip L3 cache core/chip 4 MB / 32 MB

DDR3 memory controllers 2

SMP design-point Up to 32 sockets with IBM POWER7 processors

Compatibility With prior generation of POWER processor

2.2.2 POWER7 processor core

Each POWER7 processor core implements aggressive out-of-order (OoO) instruction execution to drive high efficiency in the use of available execution paths. The POWER7 processor has an instruction sequence unit that is capable of dispatching up to six instructions per cycle to a set of queues. Up to eight instructions per cycle can be issued to the instruction execution units. The POWER7 processor has a set of twelve execution units as follows:

򐂰 2 fixed point units 򐂰 2 load store units 򐂰 4 double precision floating point units 򐂰 1 vector unit 򐂰 1 branch unit 򐂰 1 condition register unit 򐂰 1 decimal floating point unit

The caches that are tightly coupled to each POWER7 processor core are as follows:

򐂰 Instruction cache: 32 KB 򐂰 Data cache: 32 KB 򐂰 L2 cache: 256 KB, implemented in fast SRAM

40 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

2.2.3 Simultaneous multithreading

Multi-threadi ng Evol uti on

Thre ad 1 Ex ecuting

Thre ad 0 Ex ecuting

No Th re ad E x ecut ing

FX0 FX1 FP0

FP1 LS0 LS1

BRX CRL

1995 Single t hread out of order

FX0 FX1

FP0 FP1 LS0

LS1 BRX CRL

1997 Hardware muti-thread

FX0 FX1 FP0 FP1

LS0 LS1 BRX CRL

2003 2 Way SMT

FX0 FX1

FP0 FP1 LS0 LS1

BRX CRL

2009 4 W ay SMT

T hread 3 E x ec utin g

Thre ad 2 Ex ecuting

An enhancement in the POWER7 processor is the addition of the SMT4 mode to enable four instruction threads to execute simultaneously in each POWER7 processor core. Thus, the instruction thread execution modes of the POWER7 processor are as follows:

򐂰 SMT1: single instruction execution thread per core 򐂰 SMT2: two instruction execution threads per core 򐂰 SMT4: four instruction execution threads per core

SMT4 mode enables the POWER7 processor to maximize the throughput of the processor core by offering an increase in processor-core efficiency. SMT4 mode is the latest step in an evolution of multithreading technologies introduced by IBM. Figure 2-3 shows the evolution of simultaneous multithreading.

Figure 2-3 Evolution of simultaneous multithreading

The various SMT modes offered by the POWER7 processor allow flexibility, enabling users to select the threading technology that meets an aggregation of objectives (such as performance, throughput, energy use, and workload enablement).

Intelligent threads

The POWER7 processor features intelligent threads that can vary based on the workload demand. The system either automatically selects (or the system administrator can manually select) whether a workload benefits from dedicating as much capability as possible to a single thread of work, or if the workload benefits more from having capability spread across two or four threads of work. With more threads, the POWER7 processor can deliver more total capacity as more tasks are accomplished in parallel. With fewer threads, workloads that need fast individual tasks can get the performance they need for maximum benefit.

Chapter 2. Architecture and technical overview 41

2.2.4 Memory access

Advanced

Buffer

ASIC Chip

Memory

Controller

Memory

Controller

POWER7 processor chip

Dual integrated DDR3 memory controllers

 High channel and DIMM utilization  Advanced energy management  RAS advances

Eight high-speed 6.4 GHz channels

 New low-power differential signalling

New DDR3 buffer chip architecture

 Larger capacity support (32 GB/core)  Energy management support  RAS enablement

DDR3 DRAMs

P7 Core P7 Core P7 Core P7 Core

Each POWER7 processor chip has two DDR3 memory controllers, each with four memory channels (enabling eight memory channels per POWER7 processor). Each channel operates at 6.4 Gbps and can address up to 32 GB of memory. Thus, each POWER7 processor chip is capable of addressing up to 256 GB of memory.

Note: In certain POWER7 processor-based systems (including the PS700, PS701, and PS702 blades) only one memory controller is active.

Figure 2-4 gives a simple overview of the POWER7 processor memory access structure.

Figure 2-4 Overview of POWER7 memory access structure

2.2.5 Flexible POWER7 processor packaging and offerings

42 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

POWER7 processors have the unique ability to optimize to various workload types. For example, database workloads typically benefit from fast processors that handle high transaction rates at high speeds. Web workloads typically benefit more from processors with many threads that allow the breakdown of Web requests into many parts and handle them in parallel. POWER7 processors have the unique ability to provide leadership performance in either case.

POWER7 processor 4-core and 6-core offerings

The base design for the POWER7 processor is an 8-core processor with 32 MB of on-chip L3 cache (4 MB per core). However, the architecture allows for differing numbers of processor cores to be active: 4-cores or 6-cores, as well as the full 8-core version.

The L3 cache associated with the implementation is dependant on the number of active cores. For a 6-core version, this typically means that 6 x 4 MB (24 MB) of L3 cache is available. Similarly, for a 4-core version, the L3 cache available is 16 MB.

Optimized for servers

The POWER7 processor forms the basis of a flexible compute platform and can be offered in a number of guises to address differing system requirements.

The POWER7 processor can be offered with a single active memory controller with four channels for servers where higher degrees of memory parallelism are not required.

Similarly, the POWER7 processor can be offered with a variety of SMP bus capacities appropriate to the scaling-point of particular server models.

Figure 2-5 shows the physical packaging options that are supported with POWER7 processors.

Figure 2-5 Outline of the POWER7 processor physical packaging

2.2.6 On-chip L3 cache innovation and intelligent cache

A breakthrough in material engineering and microprocessor fabrication has enabled IBM to implement the L3 cache in eDRAM and place it on the POWER7 processor die. L3 cache is critical to a balanced design, as is the ability to provide good signalling between the L3 cache and other elements of the hierarchy such as the L2 cache or SMP interconnect.

The on-chip L3 cache is organized into separate areas with differing latency characteristics. Each processor core is associated with a Fast Local Region of L3 cache (FLR-L3) but also has access to other L3 cache regions as shared L3 cache. Additionally, each core can negotiate to use the FLR-L3 cache associated with another core, depending on reference patterns. Data can also be cloned to be stored in more than one core's FLR-L3 cache, again depending on reference patterns. This processor to optimize the access to L3 cache lines and minimize overall cache latencies.

intelligent cache management enables the POWER7

Chapter 2. Architecture and technical overview 43

Figure 2-6 shows the FLR-L3 cache regions for the cores on the POWER7 processor die.

Figure 2-6 FLR-L3 cache regions on the POWER7 processor

The innovation of using eDRAM on the POWER7 processor die is significant for several reasons:

򐂰 Latency improvement

A six-to-one latency improvement occurs by moving the L3 cache on-chip compared to L3 accesses on an external (on-ceramic) ASIC.

򐂰 Bandwidth improvement

A 2x bandwidth improvement occurs with on-chip interconnect. Frequency and bus sizes are increased to and from each core.

򐂰 No off-chip driver or receivers

Removing drivers and receivers from the L3 access path lowers interface requirements, conserves energy, and lowers latency.

򐂰 Small physical footprint

The performance of eDRAM when implemented on-chip is similar to conventional SRAM but requires far less physical space. IBM on-chip eDRAM uses only a third of the components used in conventional SRAM, which has a minimum of six transistors to implement a 1-bit memory cell.

򐂰 Low energy consumption

The on-chip eDRAM uses only 20% of the standby power of SRAM.

44 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

2.2.7 POWER7 processor and intelligent energy

Energy consumption is an important area of focus for the design of the POWER7 processor which includes intelligent energy features that help to optimize energy usage and performance dynamically, so that the best possible balance is maintained. Intelligent energy features (such as EnergyScale™) work with the BladeCenter Advanced Management Module (AMM) and IBM Systems Director Active Energy Manager™ to optimize processor speed dynamically, based on thermal conditions and system use.

2.2.8 Comparison of the POWER7 and POWER6 processors

Table 2-2 shows comparable characteristics between the generations of POWER7 and POWER6 processors.

Note: This shows the characteristics of the POWER7 processors in general, but not necessarily as implemented in the POWER7 processor-based blade servers.

Table 2-2 Comparison of technology for the POWER7 processor and the prior generation

Feature POWER7 POWER6+ POWER6

Technology 45 nm 65 nm 65 nm

Die size 567 mm

341 mm

Maximum cores 8 2 2

Maximum SMT threads per core

Maximum frequency 4.14 GHz 5.0 GHz 4.7 GHz

L2 Cache 256 KB per core 4 MB per core 4 MB per core

L3 Cache 4 MB of FLR-L3 cache

Memory support DDR3 DDR2 DDR2

I/O Bus Two GX+ One GX+ One GX+

Enhanced Cache Mode (TurboCore)

Sleep & Nap Mode Both Nap only Nap only

4 threads 2 threads 2 threads

32 MB off-chip eDRAM per core with each core having access to the full 32 MB of L3 cache, on-chip eDRAM

Ye s N o N o

ASIC

32 MB off-chip eDRAM ASIC

Chapter 2. Architecture and technical overview 45

2.3 POWER7 processor-based blades

The PS700 blade contains a single processor socket with a four-core processor and eight DDR3 memory DIMM slots. The PS701 blade contains a single processor socket with an eight-core processor and 16 DDR3 memory DIMM slots. The PS702 blade contains two processor sockets, each with a eight-core processor and a total of 32 DDR3 memory DIMM slots. The cores in all these blades run at 3.0 GHz.

POWER7 processor-based blades support POWER7 processors with various processors core counts. Table 2-3 summarizes the POWER7 processors for the PS700, PS701, and PS702 blades.

Table 2-3 Summary of POWER7 processor options for the PS700, PS701, and PS702 blades

Blade Model Cores per

POWER7 processor

PS700 4 1 3.0 16

PS701 8 1 3.0 32

PS702 8 2 3.0 32

Number of POWER7 processors

2.4 Memory subsystem

Frequency (GHz)

L3 cache size per POWER7 processor (MB)

The PS700 4-core processor contains one integrated DDR3 memory controller and two memory buffers that can interface with a total of eight DDR3 DIMMS. The PS701 single 8-core processor, and the PS702’s two 8-core processors chips also use a single memory controller per processor chip but use four memory buffers that can access a total of 16 or 32 DDR3 DIMMS respectively.

Industry standard DDR3 Registered DIMM (RDIMM) technology is used to increase reliability, speed, and density of memory subsystems.

2.4.1 Memory placement rules

The minimum DDR3 memory capacity for the PS700, PS701, and PS702 8 GB (2 x 4 GB DIMMs). The maximum memory supported is as follows:

򐂰 PS700 64 GB (8 x 8 GB) 򐂰 PS701 128 GB (16 x 8 GB) 򐂰 PS702 256 GB (32 x 8 GB)

Note: DDR2 memory (used in POWER6 processor-based systems) is not supported in POWER7 processor-based systems.

Figure 2-7 on page 47 show the PS701 physical memory DIMM topology. Figure 2-8 on page 47 shows the PS701 and PS702 base blade.

46 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 2-7 Memory DIMM topology for the PS700

DIMM P1-C7 DIMM P1-C8

Buffer

DIMM P1-C2

DIMM P1-C1

Buffer

DIMM P1-C6

DIMM P1-C5

DIMM P1-C3 DIMM P1-C4

P7 processor

chip

DIMM P1-C7 DIMM P1-C8

Buffer

DIMM P1-C2

DIMM P1-C1

Buffer

DIMM P1-C6

DIMM P1-C5

DIMM P1-C3 DIMM P1-C4

DIMM P1-C15 DIMM P1-C16

Buffer

DIMM P1-C10

DIMM P1-C9

DIMM P1-C14

DIMM P1-C13

DIMM P1-C11 DIMM P1-C12

P7 processor

chip

Buffer

Figure 2-8 Memory DIMM topology for the PS701 and PS702 base blade

There are eight buffered DIMM slots on the PS700, and 16 on the PS701 and PS702 base blade with an additional 16 slots on the PS702 expansion unit. The PS700 DIMM slots are numbered P1-C1 through P1-C8 as shown in Figure 2-7. The PS701 and the PS702 base blade have slots labelled P1-C1 through P1-C16 as shown in Figure 2-8. For the PS702 expansion unit the numbering is the same except for the reference to the second planar board. The numbering is from P2-C1 through P2-C16.

Chapter 2. Architecture and technical overview 47

The memory-placement rules are as follows:

򐂰 Memory is installed in DIMM-pairs (as in two DIMMs) 򐂰 DIMM pairs must be matched in size (that is, two 4 GB DIMMs or two 8 GB DIMMs). 򐂰 Minimum memory requirement are as follows:

– PS700 8 GB (2 x 4 GB DIMMs) – PS701 8 GB (2 x 4 GB DIMMs) – PS702 8 GB (2 x 4 GB DIMMs)

Note: The stated memory DIMM numbers are the minimums supported by the architecture but might not indicate the minimum order amounts.

򐂰 Mixing of DIMM capacity between pairs is permitted.

DIMMs should be installed in specific DIMM sockets depending on the number of DIMMs to install. This is described in the following three tables.

For the PS700, Table 2-4 shows the required placement of memory DIMMs depending on the number of DIMMs installed (2, 4, 6, or 8).

Table 2-4 PS700 DIMM placement rules

DIMM socket: PS700

Number of DIMMs to install:

2468

P1-C1

P1-C2

P1-C3

P1-C4

P1-C5

P1-C6

P1-C7

P1-C8

x x x x

x x

x x x x

x x

x x x

48 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

For the PS701, Table 2-5 shows the required placement of memory DIMMs depending on the number of DIMMs installed.

Table 2-5 PS701 DIMM placement rules

DIMM socket: PS701

Number of DIMMs to install:

2 4 6 8 10 12 14 16

P1-C1

P1-C2

P1-C3

P1-C4

P1-C5

P1-C6

P1-C7

P1-C8

P1-C9

P1-C10

P1-C11

P1-C12

P1-C13

P1-C14

P1-C15

P1-C16

x x x x x x x x

x x x x

x x x x x x x x

x x x x

x x x

x x x x x x

x x x

x x x x x x

x x x x x

x x

x x x x x x x

x x

x x x x x x x

Chapter 2. Architecture and technical overview 49

For the PS702, Table 2-6 shows the required placement of memory DIMMs depending on the number of DIMMs installed.

Table 2-6 PS702 DIMM placement rules

DIMM socket:

P1-C1

PS702 Number of DIMMs to install 2 4 6 8 101214161820222426283032

x x x x x x x x x x x x x x x x

P1-C2 P1-C3 P1-C4 P1-C5 P1-C6 P1-C7 P1-C8 P1-C9 P1-C10 P1-C11 P1-C12 P1-C13 P1-C14 P1-C15 P1-C16 P2-C1 P2-C2 P2-C3

x x x x x x x x

x x x x x x x x x x x x x x x x

x x x x x x x x

x x x x x x

x x x x x x x x x x x x

x x x x x x

x x x x x x x x x x x x

x x x x x x x x x x

x x

x x x x x x x x x x

x x

x x x x

x x x x x x x x x x x x x x

x x x x

x x x x x x x x x x x x x x

x x x x x x x x x x x x x x x

x x x x x x x

x x x x x x x x x x x x x x x

P2-C4 P2-C5 P2-C6 P2-C7 P2-C8 P2-C9 P2-C10 P2-C11 P2-C12 P2-C13 P2-C14 P2-C15 P2-C16

50 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

x x x x x x x

x x x x x

x x x x x x x x x x x

x x x x x

x x x x x x x x x x x

x x x x x x x x x

x x x

x x x x x x x x x x x x x

x x x

x x x x x x x x x x x x x

2.5 Technical comparison

Table 2-7 shows a comparison of the technical aspects of the PS700, PS701, and PS702 blade compared to a Power 750 Express.

Table 2-7 Comparison of technical characteristics between PS blades and the Power 750 Express

Systems characteristic PS700 PS701 PS702 Power 750 Express

Processor 4-cores at 3.0

GHz

Pluggable processor cards Not Applicable Not Applicable Not Applicable 1–4

Min./Max. processor cores 4 8 16 6/24 (6-core) or 8/32 (8-core)

L3 cache On-chip

eDRAM

Max memory slots and type 8 DDR3 16 DDR3 32 DDR3 8 slots per processor card

Memory chipkill Yes Yes Yes Yes

Memory spare No No No Yes

Memory hotplug No No No No

EnergyScale device Yes Yes Yes Yes

PCIe x8 slots 2 2 4 3

PCI-X 2.0 slots 0 0 0 2

PCIe and PCI-X hot plug No No No Yes

Integrated Virtual Ethernet Ports / Speed

Integrated 2 /1 Gb

8-cores at 3.0 GHz

On-chip eDRAM

Integrated 2 / 1 Gb

16-cores at 3.0 GHz

On-chip eDRAM

Integrated 4 / 1 Gb

6-cores at 3.3 GHz 8-cores at 3.0 GHz, 3.3 GHz,

3.55 GHz

On-chip eDRAM

(32 slots max.), DDR3

daughter card quad port / 1 Gb or dual port / 10 Gb

PowerVM support Yes Yes Yes Yes

Capacity on Demand No No No No

Redundant hotplug power Yes through

DASD bays 2 1 2 8 (hot-plug, front access,

GX slot (GX+ slot does not support RIO2)

Yes through

chassis

Not applicable Not applicable Not applicable 1 x GX+ slot and 1 x GX++

chassis

Yes through chassis

Chapter 2. Architecture and technical overview 51

Ye s

SFF)

slot (not hot pluggable)

2.6 Internal I/O subsystem

Each POWER7 processor as implemented in the POWER7 processor-based blades utilizes a single GX+ bus which is used to connect to the I/O subsystem. The I/O subsystem is a GX+ multifunctional host bridge chip which provides the following major interfaces:

򐂰 One GX+ primary interface to the processor 򐂰 Two 64-bit PCI-X 2.0 buses, one 64-bit PCI-X 1.0 bus, and one 32-bit PCI-X 1.0 bus 򐂰 Four x8 PCI Express links 򐂰 Two 10 Gbps Ethernet ports: Each port is individually configurable to function as two 1

Gbps ports

The PS702 with two POWER7 processors also has two GX+ multifunctional host bridge chips. Unless otherwise noted, references to slots, embedded controllers and so forth are assumed to be doubled for the PS702.

Note: Table 2-2 on page 45 indicates there are two GX+ buses in the POWER7 processor however only one of them is active in the PS700 and PS701, and each processor in the PS702.

2.6.1 Peripheral Component Interconnect Express (PCIe) bus

PCIe uses a serial interface and allows for point-to-point interconnections between devices using a directly wired interface between these connection points. A single PCIe serial link is a dual-simplex connection using two pairs of wires, one pair for transmit and one pair for receive, and can only transmit one bit per cycle. It can transmit at the extremely high speed of

2.5 Gbps, which equates to a burst mode of 320 MBps on a single connection. These two pairs of wires is called a lane. A PCIe link might be comprised of multiple lanes. In such configurations, the connection is labeled as x1, x2, x8, x12, x16, or x32, where the number is the number of lanes.

The PCIe expansion card options for the PS700, PS701, and PS702 blades support Extended Error Handling (EEH). The card ports are routed through the BladeCenter mid-plane to predetermined I/O switch bays. The switches installed in these switch bays must match the type of expansion card installed, Ethernet, Fibre Channel, and so forth.

2.6.2 PCIe slots

The two PCIe slots are connected to the four x8 PCIe links on the GX+ multifunctional host bridge chip. One of the links supports the CIOv connector and the other three links support the CFFh connector on the blade. All PCIe slots are Enhanced Error Handling (EEH). PCI EEH-enabled adapters respond to a special data packet generated from the affected PCIe slot hardware by calling system firmware, which examines the affected bus, allows the device driver to reset it, and continues without a system reboot. For Linux, EEH support extends to the majority of frequently used devices, although various third-party PCI devices might not provide native EEH support.

Expansion card form factors

There are two PCIe card form factors supported on the PS700, PS701, and PS702 blades:

򐂰 CIOv 򐂰 CFFh

52 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

CIOv form factor

A CIOv expansion card uses the PCI Express 2.0 x8 160 pin connector. A CIOv adapter requires compatible switch modules to be installed in bay 3 and bay 4 of the BladeCenter chassis. The CIOv card can be used in any BladeCenter that supports the PS700, PS701, and PS702 blades.

CFFh form factor

The CFFh expansion card attaches to the 450 pin PCIe Express connector of the blade server. In addition, the CFFh adapter can only be used in servers that are installed in the BladeCenter H, BladeCenter HT, or BladeCenter S chassis.

A CFFh adapter requires that either: 򐂰 A Multi-Switch Interconnect Module (MSIM) or MSIM-HT (BladeCenter HT chassis) is

installed in bays 7 and 8, bays 9 and 10, or both.

򐂰 A high speed switch module be installed in bay 7 and bay 9. 򐂰 In the BladeCenter S, a compatible switch module is installed in bay 2.

The requirement of either the MSIM, MSIM-HT, or high-speed switch modules depends on the type of CFFh expansion card installed. The MSIM or MSIM-HT must contain compatible switch modules. See 1.6.6, “Multi-switch Interconnect Module” on page 33, or 1.6.7, “Multi-switch Interconnect Module for BladeCenter HT” on page 34, for more information about the MSIM or MSIM-HT.

The CIOv expansion card can be used in conjunction with a CFFh card in BladeCenter H, HT and in certain cases a BladeCenter S chassis, depending on the expansion card type.

Table 2-8 lists the slot types, locations, and supported expansion card form factor types of the PS700, PS701, and PS702 blades.

Table 2-8 Slot configuration of the PS700, PS701, and PS702 blades

Card location Form factor PS700 location PS701 location PS702 location

Base blade

Expansion blade CIOv Not present Not present

Expansion blade CFFh Not present Not present

CIOv P1-C11 P1-C19 P1-C19

CFFh P1-C12 P1-C20 P1-C20

P2-C19

P2-C20

Chapter 2. Architecture and technical overview 53

Figure 2-9 shows the locations of the PCIe CIOv and CFFh connectors and the physical

CFFh connector - P1-C12 CIOv connector - P1-C11

CFFh connector - P1-C20 CIOv connector - P1-C19

location codes for the PS700.

Figure 2-9 PS700 location codes for PCIe expansion cards

Figure 2-10 shows the locations of the PCIe CIOv and CFFh connectors for the PS701 and PS702 base planar and the physical location codes. The expansion unit for the PS702 uses the prefix P2 for the slots on the second planar.

Figure 2-10 PS701 and PS702 base location codes for PCIe expansion cards

54 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 2-11 shows the locations of the PCIe CIOv and CFFh connectors for the PS702

CFFh connector - P2-C20 CIOv connector - P2-C19

expansion blade (feature code 8358) and the physical location codes.

Figure 2-11 PS702 expansion blade location codes for PCIe expansion cards

BladeCenter I/O topology

There are no externally accessible ports on the PS700, PS701, and PS702 blades, All I/O is routed through a BladeCenter midplane to the I/O modules bays.

The I/O ports on all expansion cards are typically set up to provide a redundant pair of ports. Each port has a separate path through the mid-plane of the BladeCenter chassis to a specific I/O module bay. Figure 2-12 on page 56 through Figure 2-15 on page 57 show the four supported BladeCenter chassis and the I/O topology for each.

Chapter 2. Architecture and technical overview 55

Blade Server 14

Blade Server 1 On-Boar d 1GbE

Expansion card

I/O Bay 3I/O Bay 4 I/O Bay 2 I/O Bay 1

Standard I/O bays connections

Legend

Mid-Plane

CIOv

Blade Server 14

Blade Server 1 On-Boar d 1GbE

CFFv

CFFh

Expans ion cards

I/O Bay 7

I/O Bay 9

I/O Bay 8

I/O Bay 10

I/O Bay 5I/O Bay 6 I/O Bay 3I/O Bay 4 I/O Bay 2 I/O Bay 1

Standard I/O bays connections High-speed I/O bays connections Br idge modules I/O bays connections

Legend

Mid-Plane

CIOv

Figure 2-12 BladeCenter E I/O topology

56 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 2-13 BladeCenter H I/O topology

Blade Server 12

Blade Server 1 On-Boar d 1GbE

CFFh

Expans ion cards

Standard I/O bays connections High-speed I/O bays connections Br idge modules I/O bays connections Standard I/O bays inter-switch links High-speed I/O bays inter-swi tch links

Legend

Mid-Plane

I/O Bay 7

I/O Bay 8

I/O Bay 10

I/O Bay 9

I/O Bay 3I/O Bay 4 I/O Bay 2 I/O Bay 1

CIOv

Blade Server 6

Blade Server 1 On-Board 1GbE

CFFh

Expans ion cards

I/O Bay 3I/O Bay 4 I/O Bay 1

Standard I/O bays connections x4 SAS DSM connections I/O bays 3 & 4 Ethernet connections

Legend

Mid-Plane

DSM1DSM2

I/O Bay 2

CIOv

Figure 2-14 BladeCenter HT I/O topology

Figure 2-15 BladeCenter S I/O topology

Chapter 2. Architecture and technical overview 57

2.6.3 I/O expansion cards

The following I/O expansion cards provide additional resources that can be used by a native operating system, the Virtual I/O Server (VIOS), or assigned directly to a LPAR by the VIOS.

See 1.5.8, “I/O features” on page 24 for details about each supported card.

LAN adapters

In addition to the onboard HEA ports, Ethernet ports can be added with LAN expansion card adapters. The current LAN adapters for the PS700, PS701, and PS702 blades are available in the CFFh form factor type. The QLogic Ethernet and 4 Gb Fibre Channel Expansion Card has two 1 Gb Ethernet ports and two 4 Gb Fibre Channel ports.

The Ethernet ports on CFFh expansion cards (BladeCenter H and HT) are connected to switch bays 7, and 9 and the Fibre Channel ports to switch bays 8, and 10. In the BladeCenter S only the Ethernet ports are usable and the connection is to Bay 2.

SAS adapter

To connect to external SAS devices, including the BladeCenter S storage modules, the 3 Gb SAS Passthrough Expansion Card and BladeCenter SAS Connectivity Modules are required.

The 3 Gb SAS Passthrough Expansion Card is a two port PICe CIOv form factor card. The output from the ports on this card are routed through the BladeCenter mid-plane to I/O switch bays 3 and 4.

Fibre Channel adapters

The PS700, PS701, and PS702 support direct or SAN connection to devices using Fibre Channel adapters and the appropriate pass-through or Fibre Channel switch modules in the BladeCenter chassis. Fibre Channel expansion cards are available in both form factors and in 4 Gb and 8 Gb data rates.

The two ports on CIOv form factor expansion cards are connected to BladeCenter I/O switch module bays 3 and 4. The two Fibre Channel ports on a CFFh expansion card connect to BladeCenter H or HT I/O switch bays 8 and 10. The Fibre Channel ports on a CFFh form factor adapters are not supported for use in a BladeCenter S chassis.

Fibre Channel over Ethernet (FCoE)

Fibre Channel over Ethernet (FCoE), is being developed within T11 as part of the Fibre Channel Backbone 5 (FC-BB-5) project. It is not meant to displace or replace FC. FCoE is an enhancement that expands FC into the Ethernet by combining two leading-edge technologies (FC and the Ethernet). This evolution with FCoE makes network consolidation a reality by the combination of Fibre Channel and Ethernet. This network consolidation maintains the resiliency, efficiency, and seamlessness of the existing FC-based data center.

Figure 2-16 on page 59 shows a configuration using BladeCenter FCoE components.

58 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 2-16 FCoE connections in IBM BladeCenter

IBM POWER7 blades each with a Converged Network Adapter

BNT Virtual Fabr ic 10Gb Switch Module with 10Gb Ethernet ports

QLogic Virtual Fabric Extension Module with 8Gb Fibre Channel ports

BladeCenter H

LAN

SAN

Internal connections

For more information about FCoE, read An Introduction to Fibre Channel over Ethernet, and Fibre Channel over Convergence Enhanced Ethernet, REDP-4493, available from the

following Web page:

http://www.redbooks.ibm.com/abstracts/redp4493.html

The QLogic 2-port 10 Gb Converged Network Adapter is a CFFh form factor card. The ports on this card are connected to BladeCenter H and HT I/O switch module bays 7 and 9. In these bays a pass through or FCoE capable I/O module can provide connectivity to a top-of-rack switch. A combination of the appropriate I/O switch module in these bays and the proper Fibre Channel capable modules in bays 3 and 5 can eliminate the top-of-rack switch requirement. See 1.6, “Supported BladeCenter I/O modules” on page 29.

InfiniBand Host Channel adapter

The InfiniBand Architecture (IBA) is an industry-standard architecture for server I/O and interserver communication. It was developed by the InfiniBand Trade Association (IBTA) to provide the levels of reliability, availability, performance, and scalability necessary for present and future server systems with levels significantly better than can be achieved using bus-oriented I/O structures.

InfiniBand is an open set of interconnected standards and specifications. The main InfiniBand specification has been published by the InfiniBand Trade Association and is available at the following Web page:

http://www.infinibandta.org/

InfiniBand is based on a switched fabric architecture of serial point-to-point links. These InfiniBand links can be connected to either host channel adapters (HCAs), used primarily in servers, or target channel adapters (TCAs), used primarily in storage subsystems.

The InfiniBand physical connection consists of multiple byte lanes. Each individual byte lane is a four-wire, 2.5, 5.0, or 10.0 Gbps bi-directional connection. Combinations of link width and byte lane speed allow for overall link speeds of 2.5–120 Gbps. The architecture defines a layered hardware protocol as well as a software layer to manage initialization and the communication between devices. Each link can support multiple transport services for reliability and multiple prioritized virtual communication channels.

Chapter 2. Architecture and technical overview 59

For more information about InfiniBand, read HPC Clusters Using InfiniBand on IBM Power Systems Servers, SG24-7767, available from the following Web page:

http://www.redbooks.ibm.com/abstracts/sg247767.html

The 4X InfiniBand DDR Expansion Card is a 2 port CFFh form factor card and is only supported in a BladeCenter H chassis. The two ports are connected to the BladeCenter H I/O switch bays 7 and 8, and 9 and 10. These switch bays require a supported InfiniBand switch module to provide either external or blade to blade within the same chassis communication.

2.6.4 Embedded SAS Controller

The embedded SAS controller is connected to one of the 64-bit PCI-X 2.0 buses on the GX+ multifunctional host bridge chip. The PS702 uses a single embedded SAS controller. More information about the SAS I/O subsystem can be found in 2.9, “Internal storage” on page 65.

2.6.5 HEA ports

Each HEA port has its own connection to the GX+ multifunctional host bridge chip. The connections are configured for 1 Gb operation. The HEA ports are part of the Integrated Virtual Ethernet (IVE) subsystem. The IVE subsystem is described in 2.7, “Integrated Virtual Ethernet” on page 61.

2.6.6 Embedded USB controller

The USB controller is connected to the 64-bit PCI-X 1.0 bus of the GX+ multifunctional host bridge chip. This embedded USB controller provides support for four USB root ports. These ports are connected to four USB busses on the BladeCenter midplane through two connectors. Two of these ports are routed to the two AMM bays. The other two USB ports are directed to the BladeCenter Media Tray, as shown in Figure 2-17 on page 61. The PS702 uses a single embedded USB controller.

60 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 2-17 System overview of USB connections

USB HUB2

IDE/USB Conv.

IDE CD USB FDD

GX+ host

bridge chip

FET

Switch

FET

Switch

FET

Switch

FET

Switch

BladeCenter

Media

Tray

Processor Blade

BladeCenter

Midplane

Management

Module

USB2 USB1

USB3 USB4

Keyboard/

Mouse

Keyboard/

Mouse

USB Demux

Note: The PS700, PS701, and PS702 blades do not support the KVM function from the AMM.

BladeCenter Media Tray

The BladeCenter Media Tray, depending on the BladeCenter chassis used, can contain up to two USB ports, one optical drive and system status LEDs. For information about the different media tray options available by BladeCenter model see IBM BladeCenter Products and Technology, SG24-7523 available from the following Web page:

http://www.redbooks.ibm.com/abstracts/sg247523.html

The media tray is a shared resources that can be assigned to any blade slot.

2.7 Integrated Virtual Ethernet

Introduced with POWER6, POWER7 processor-based servers continue the use of IVE. The terms IVE and HEA are sometimes used interchangeably, however, IVE encompasses all the hardware parts including the HEA and the integration of several technologies.

IVE enables the ability to manage the sharing of the integrated HEA physical ports. The

PS700, PS701, and PS702 blades include two 1 Gb HEA ports on the base blade. The PS702 has an additional two HEA ports on the expansion unit for a total of four physical ports.

Chapter 2. Architecture and technical overview 61

IVE provides logical Ethernet ports that can communicate to logical partitions (LPARs)

Hypervisor

Hosting

Partition

AIX AIX Linux

Virtual Ethernet Switch

Network Adapters

Virtual

Ethernet

Driver

Virtual

Ethernet

Driver

Virtual

Ethernet

Driver

Packet

Forwarder

Using Virtual I/O Server Shared Ethernet Adapter

AIX AIX Linux

Virtual

Ethernet

Driver

Virtual

Ethernet

Driver

Virtual

Ethernet

Driver

Integrated Virtual Ethernet

Using Integrated Virtual Ethernet

LAN, WAN, ...

reducing the use of IBM POWER Hypervisor™. The design provides a logical connection for multiple LPARs to a physical port, allowing LPARs to access external networks through the HEA without using a Shared Ethernet Adapter (Ethernet bridge) through the Virtual I/O Server. This eliminates the need to move packets (using Virtual Ethernet Adapters) between partitions and then through a Shared Ethernet Adapter (SEA) to an physical Ethernet port. LPARs can share HEA ports with improved performance.

Figure 2-18 shows the difference between IVE and SEA implementations.

Figure 2-18 IVE compared to Virtual I/O Server Shared Ethernet Adapter

IVE design meets general market requirements for better performance and better virtualization for Ethernet. It offers the following benefits:

򐂰 Either two 1 Gbps HEA ports (PS700 and PS701) or four 1 Gbps HEA ports (PS702) 򐂰 Logical ports assigned to LPARs for external network connectivity as an option to a

Virtual I/O Server provided Shared Ethernet Adapter (SEA)

򐂰 Industry standard hardware acceleration, loaded with flexible configuration possibilities 򐂰 The speed and performance of the GX+ bus 򐂰 Great improvement of latency for short packets that are ideal for messaging applications

(such as distributed databases) that require low latency communication for synchronization and short transactions

For more information about IVE features readIntegrated Virtual Ethernet Adapter Technical Overview and Introduction, REDP-4340, available at the following Web page:

http://www.redbooks.ibm.com/abstracts/redp4340.html

2.7.1 IVE subsystem

One of the key design goals of the IVE architecture is the capability to integrate up to two 10 Gbps Ethernet ports or four 1 Gbps Ethernet ports into the P5IOC2 chip, with the effect of a low cost Ethernet solution for low-end and mid-range server platforms. Any 10 Gbps, 1 Gbps, 100 Mbps, or 10 Mbps speeds share the same I/O.

62 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

The IVE implementation on the PS700, PS701, and PS702 blades use a maximum rate of 1 Gbps and the HEA ports are integrated onto the base blades and expansion unit. The two physical ports on the PS700 and PS701 are associated to a single logical port group. The two additional physical ports on the PS702 expansion unit are associated to a second port group. Each port group can address up to 16 logical ports, A maximum of 16 MAC addresses are assigned to a port group. A maximum of one logical port per physical port can be given to a LPAR.

The HEA ports are connected to the BladeCenter I/O switch module bays 1 and 2, with the exception of the BladeCenter S. The BladeCenter S connects all HEA ports to I/O switch module bay 1.

Note: On PS700, PS701, and PS702 blades, as of this writing, Virtual I/O Server must be installed on the blade to configure IVE logical ports through Integrated Virtualization Manager (IVM). Native operating system installations can only use the physical ports.

IVE does not have flash memory for its open firmware but it is stored in the service processor flash and then passed to POWER Hypervisor control. Flash code update, therefore, is done by the POWER Hypervisor.

Important: The HEA port implementation on the PS700, PS701, and PS702 blades always shows a link status of up, and should be considered when implementing network failover scenarios.

2.8 Service processor

The service processor (previously known as the Flexible Service Processor or FSP) is used to monitor and manage the system hardware resources and devices. In a POWER7-based blade implementation the external network connection for the service processor is routed through an on-blade Ethernet switch, through the BladeCenter midplane, chassis switches and to the AMM. The Serial over LAN (SOL) connection for a system console uses this same connection.When the blade is in standby power mode the service processor responds to AMM instructions and can detect Wake-on-LAN (WOL) packets.

The PS700 and PS701 each have a single service processor. The PS702 has a second service processor in the expansion unit. However, it is only used for controlling and managing the hardware on this second planar.

2.8.1 Server console access by SOL

The PS700, PS701, and PS702 blades do not have an on-board video chip and do not support KVM connections. Server console access is obtain by a SOL connection.

SOL provides a means to manage servers remotely by using a command-line interface (CLI) over a Telnet or secure shell (SSH) connection. SOL is required to manage servers that do not have KVM support. SOL provides console redirection for both System Management Services (SMS) and the blade server operating system. The SOL feature redirects server serial-connection data over a LAN without requiring special cabling. The SOL connection enables blade servers to be managed from any remote location with network access.

Chapter 2. Architecture and technical overview 63

SOL offers the following advantages:

Enet Switch

Port-14 MM Ports

GX+ bridge chip

External

Networks

Blade Center

Chassis Switches

10/100

Internal Mana gem e nt

Fabric

Management

Module

10/100

Management

Network

1 Gb

Fabrics

to 2nd MM to 2nd MM

SOL Telnet

Server

Enet

SP1

PSI

eNet0 MDIO

GX Intf

P7 Blade

HEA Port0A

1Gb Enet

HEA Port0B

1Gb Enet

Enet Switch

Port 4

Port 2

Port 3

Port 0

Mgmt

Port

Port 1

Enet Switch

Port-14 MM Ports

2 1

SOL traffic System traffic

򐂰 Remote administration without keyboard, video, or mouse (headless servers) 򐂰 Reduced cabling and without requiring a serial concentrator 򐂰 Standard Telnet interface, eliminating the requirement for special client software

The IBM BladeCenter AMM CLI provides access to the text-console command prompt on each blade server through a SOL connection, enabling the blade servers to be managed from a remote location.

In the BladeCenter environment, the SOL console data stream from a blade is routed from the blades’s service processor to the AMM through the on-blade switch to the network infrastructure of the BladeCenter unit, including an Ethernet-compatible I/O module that supports SOL communication. Figure 2-19 shows the SOL data stream flow.

Figure 2-19 SOL service processor to AMM connection

BladeCenter components are configured for SOL operation through the BladeCenter AMM. The AMM also acts as a proxy in the network infrastructure to couple a client running a Telnet or SSH session with the management module to an SOL session running on a blade server, enabling the Telnet or SSH client to interact with the serial port of the blade server over the network.

Because all SOL traffic is controlled by and routed through the AMM, administrators can segregate the management traffic for the BladeCenter unit from the data traffic of the blade servers. To start an SOL connection with a blade server, perform the following steps:

64 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

1. Start a Telnet or SSH CLI session with the AMM.

2. Start a remote-console SOL session with any blade server in the BladeCenter unit that is set up and enabled for SOL operation.

You can establish up to 20 separate Web-interface, Telnet, or SSH sessions with a BladeCenter AMM. For a BladeCenter unit, this step enables you to have 14 simultaneous SOL sessions active (one for each of up to 14 blade servers) with six additional CLI sessions available for BladeCenter unit management.

With a BladeCenter S unit you have six simultaneous SOL sessions active (one for each of up to six blade servers) with 14 additional CLI sessions available for BladeCenter unit management. If security is a concern, you can use Secure Shell (SSH) sessions, or connections made through the serial management port that is available on the AMM, to establish secure Telnet CLI sessions with the BladeCenter management module before starting an SOL console-redirect session with a blade server.

SOL has the following requirements: 򐂰 An Ethernet switch module or Intelligent Pass-Thru Module is installed in bay 1 of a

BladeCenter

򐂰 SOL is enabled for those blades that you want to connect to with SOL. 򐂰 The Ethernet switch module must be set up correctly.

For details about setting up SOL, see the BladeCenter Serial Over LAN Setup Guide, which can be found at the following Web page:

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-54666

This guide contains an example of how to establish a Telnet or SSH connection to the management module and then an SOL console.

2.9 Internal storage

PS700, PS701 and PS702 blades use an integrated SAS controller. The controller’s PCI-X interface to the GX+ multifunctional host bridge chip is 64 bits wide and operates at 133 MHz.

This controller provides ports for the internal drives, and ports through the 3 Gb SAS Passthrough Expansion Card to the BladeCenter SAS switch modules. The SAS controller ports used for the internal disk drives can support a single 2.5” SAS hard disk drive (HDD) at each DASD bay location, as shown in Figure 2-20 on page 66, Figure 2-21 on page 66. and Figure 2-22 on page 67.

Note: Solid state drives (SSDs) are not supported.

Chapter 2. Architecture and technical overview 65

Figure 2-20 PS700 SAS configuration

CIOv

SAS Card

PS700

P1-D1

SAS Controller

SA S HDD

SAS

Swit ch

Bay4

SAS

Switch

Bay3

SAS HDD

P1-D2

CIOv

SAS Card

PS701 and PS702 base

P1-D1

SAS Controller

SAS HDD

SAS

Swit ch

Bay4

SAS

Switch

Bay3

66 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Figure 2-21 PS701 SAS configuration

Figure 2-22 PS702 SAS configuration

CIOv

SAS Card

CIOv

SAS

Card

PS701 and PS702 base

P S7 02 expansi on

unit only

P2-D1

P1-D1

SMP Connector

SAS Controller

SAS HDD

SAS

Swit ch

Bay4

SAS

Switch

Bay3

SAS

Swit ch

Bay4

SAS

Switch

Bay3

SAS HDD

P1-D1

P1-D2

Figure 2-23 show the physical locations and codes for the HDDs in the PS700

Figure 2-23 HDD location and physical location code PS700

Chapter 2. Architecture and technical overview 67

Figure 2-24 show the physical location and code for a HDD in a PS701. The PS702

P1-D1

expansion unit locates the HDD is the same location with a physical location code of P2-D1.

Figure 2-24 HDD location and physical location code PS701

2.9.1 Hardware RAID function

The PS700 and PS702 have support for RAID functions across a blades’s internal when more than one storage drive is installed in the system through the SAS controller. RAID 0 and RAID 1 are supported. If there is only one drive, there is no RAID function. The PS701 only supports one drive so RAID is not offered.

The configuration of the RAID array the blade’s internal disks is performed by booting the system from the AIX Diagnostic Utilities disk prior to installing the operating system.

2.9.2 External SAS connections

The onboard SAS controller in the PS700, PS701, and PS702 blades does not provide a direct access external SAS port. However, by using a 3 Gb SAS Passthrough Expansion Card and BladeCenter SAS Connectivity Modules, two ports on the SAS controller (four in the PS702 with a second SAS card on the expansion unit) are expanded, providing access to BladeCenter S Disk Storage Modules (DSM) or an external SAS disk sub-system.

2.10 External disk subsystems

This section describes the external disk subsystems, supported IBM System Storage family of products.

For up-to-date compatibility information for Power blades and IBM Storage, go to the the Storage System Interoperability Center at the following link:

http://ibm.com/systems/support/storage/config/ssic

For N Series Storage compatibility with Power blades, go to:

http://ibm.com/systems/storage/network/interophome.html

68 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

2.10.1 IBM BladeCenter S Disk Storage Modules

The BladeCenter S supports up to two storage modules, These modules provide integrated SAS storage functionality to the BladeCenter S chassis.

The storage module’s collection of disk drives are made accessible to blade servers through a SAS Connectivity Module or SAS RAID Controller Module installed in the BladeCenter S chassis and SAS expansion cards installed in the blades. The SAS RAID Controller Module provides RAID 0, 1, 5, and 10 support.

Each of the two storage modules contain up to six 3.5 inch hot-swap hard drives, for a total of 12 internal drives. The storage module supports SAS, SATA, and Near Line SAS (NL SAS) drives. Intermixing SAS and SATA or SAS and NL SAS drives within the same storage module is supported.

2.10.2 IBM System Storage

The IBM System Storage Disk Systems products and offerings provide compelling storage solutions with superior value for all levels of business.

IBM System Storage N series

IBM N series unified system storage solutions can provide customers with the latest technology to help them improve performance, virtualization manageability, and system efficiency at a reduced total cost of ownership. Several enhancements have been incorporated to the N series product line, to complement and reinvigorate this portfolio of solutions:

򐂰 The new SnapManager® for Hyper-V provides extensive management for backup,

restoration, and replication for Microsoft® Hyper-V environments

򐂰 The new N series Software Packs provides the benefits of a broad set of N series

solutions at a reduced cost.

򐂰 An essential component to this launch is Fibre Channel over Ethernet access and 10 Gb

Ethernet, to help integrate Fibre Channel and Ethernet flow into a unified network, and take advantage of current Fibre Channel installations.

For more information, see the following Web page:

http://www.ibm.com/systems/storage/network

IBM System Storage DS3000 family

The IBM System Storage DS3000 is an entry-level storage system designed to meet the availability and consolidation needs for a wide range of users. New features, including larger capacity 450 GB SAS drives, increased data protection features (such as RAID 6), and more FlashCopy® images per volume provide a reliable virtualization platform with the support of Microsoft Windows® Server 2008 with HyperV.

For more information, see the following Web page:

http://www.ibm.com/systems/storage/disk/ds3000/

IBM System Storage DS5020 Express

Optimized data management requires storage solutions with high data availability, strong storage management capabilities and powerful performance features. IBM offers the IBM System Storage DS5020 Express, designed to provide lower total cost of ownership, high

Chapter 2. Architecture and technical overview 69

performance, robust functionality, and unparalleled ease of use. As part of the IBM DS series, the DS5020 Express offers the following features:

򐂰 High-performance 8 Gbps capable Fibre Channel connections 򐂰 Optional 1 Gbps iSCSI interface 򐂰 Up to 112 TB of physical storage capacity with 112 1 TB SATA disk drives 򐂰 Powerful system management, data management, and data protection features

For more information, see the following Web page:

http://www.ibm.com/systems/storage/disk/ds5020/

IBM System Storage DS5000

New DS5000 enhancements help reduce costs by reducing power per performance by introducing SSD drives. Also, with the new EXP5060 expansion unit supporting 60 1-TB SATA drives in a 4 U package, you can see up to a one-third reduction in floor space over standard enclosures. With the addition of 1 Gbps iSCSI host-attach, you can reduce cost for less demanding applications and continue providing high performance where necessary by using the 8 Gbps FC host ports. With DS5000, you get consistent performance from a smarter design, that simplifies your infrastructure, improves your total cost of ownership (TCO), and reduces costs.

For more information, see the following Web page:

http://www.ibm.com/systems/storage/disk/ds5000

IBM XIV Storage System

IBM is introducing a mid-sized configuration of its self-optimizing, self-healing, resilient disk solution, the IBM XIV® Storage System. Organizations with mid-sized capacity requirements can take advantage of the latest technology from IBM for their most demanding applications with as little as 27 TB of usable capacity and incremental upgrades.

For more information, see the following Web page:

http://www.ibm.com/systems/storage/disk/xiv/

IBM System Storage DS8700

The IBM System Storage DS8700 is the most advanced model in the IBM DS8000 lineup and introduces dual IBM POWER6 based controllers that usher in a new level of performance for the company’s flagship enterprise disk platform. The new DS8700 supports the most demanding business applications with its superior data throughput, unparalleled resiliency features and five-nines availability. In today’s dynamic, global business environment, where organizations such as yours need information be reliably available around the clock and with minimal delay, can you really afford not to run your business on the DS8000 series? With its tremendous scalability, flexible tiered storage options, broad server support, and support for advanced IBM duplication technology, the DS8000 can help simplify the storage environment by consolidating multiple storage systems onto a single system, and provide the availability and performance you have come to trust for your most important business applications.

For more information, see the following Web page:

http://www.ibm.com/systems/storage/disk/ds8000/

70 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

2.11 IVM

IVM is a simplified hardware management solution that is part of the PowerVM implementation on the PS700, PS701, and PS702 blades. POWER processor-based blades do not include an option for attachment to a Hardware Management Console (HMC).

IVM inherits most of the HMC features and capabilities and enables the exploitation of PowerVM technology. It manages a single server, avoiding the need for an independent appliance. It is designed to provide a solution that enables the administrator to reduce system setup time and to make hardware management easier, at a lower cost.

IVM is an addition to the Virtual I/O Server, the product that enables I/O virtualization in the family of POWER processor-based systems. The IVM functions are provided by software executing within the Virtual I/O Server partition installed on the server to manage. See Ta bl e 2 - 9 .

For a complete description of the possibilities offered by IVM, see Integrated Virtualization Manager on IBM System p5, REDP-4061, available the following Web page:

http://www.redbooks.ibm.com/abstracts/redp4061.html

Table 2-9 Comparison of IVM and HMC

Characteristic IVM HMC

General characteristics

Delivery vehicle Integrated into the server A desktop or rack-mounted appliance

Footprint Runs in 60 MB memory and requires

minimal CPU as it runs stateless.

Installation Installed with the Virtual I/O Server

(optical or network). Preinstall option available on certain systems.

Multiple system support One IVM per server One HMC can manage multiple servers

User interface Web browser (no local graphical display)

and telnet session

Scripting and automation VIOS command-line interface (CLI) and

HMC compatible CLI.

RAS characteristics

Redundancy and HA of manager

Multiple VIOS No, single VIOS Yes

Fix or update process for manager

Adapter microcode updates Inventory scout through RMC Inventory scout through RMC

Only one IVM per server Multiple HMCs can manage the same

VIOS fixes and updates HMC e-fixes and release updates

2-Core x86, 2 GB RAM, 80 GB HD

Appliance is preinstalled. Reinstall through optical media or network is supported.

(48 CECs / 1024 LPARS)

Web browser (local or remote)

HMC CLI

system for HMC redundancy.

Firmware updates Inband through OS; not concurrent Service Focal Point™ with concurrent

firmware updates

Chapter 2. Architecture and technical overview 71

Characteristic IVM HMC

I/O concurrent maintenance (not available on POWER based blades)

Serviceable event management

PowerVM function

Full PowerVM Capability Partial Full

Capacity on Demand Entry of PowerVM codes only Full Support

I/O Support for IBM i Virtual Only Virtual and Direct

Multiple Shared Processor Pool

Workload Management (WLM) Groups Supported

Support for multiple profiles per partition

SysPlan Deploy & mksysplan

VIOS support for slot and device level concurrent maintenance through the diag hot plug support.

Service Focal Point Light: Consolidated management of firmware- and management partition-detected errors

No, default pool only Yes

One 254

No Yes

No No

Guided support in the “Repair and Verify” function on the HMC.

Service Focal Point support for consolidated management of operating system- and firmware-detected errors

2.12 Operating system support

The IBM POWER7 processor-based systems supports three families of operating systems:

򐂰 AIX 򐂰 IBM i 򐂰 Linux

In addition, the Virtual I/O Server can be installed in special partitions that provide support to the other operating systems for using features such as virtualized I/O devices, PowerVM Live Partition Mobility, or PowerVM Active Memory™ Sharing.

Note: For details about the software available on IBM POWER servers, see Power Systems Software™ at the following Web page:

http://www.ibm.com/systems/power/software/index.html

The PS700, PS701, and PS702 blades support the following operating system versions.

Virtual I/O Server

Virtual I/O Server 2.1.3.0 or later

IBM regularly updates the Virtual I/O Server code. To find information about the latest updates, see the Virtual I/O Server at the following Web page:

http://www14.software.ibm.com/webapp/set2/sas/f/vios/documentation/home.html

72 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

IBM AIX Version 5.3

AIX Version 5.3 with the 5300-12 Technology Level or later.

A partition using AIX Version 5.3 executes in POWER6 or POWER6+ compatibility mode.

IBM periodically releases maintenance packages (service packs or technology levels) for the AIX 5L operating system. Information about these packages, downloading, and obtaining the CD-ROM is on the Fix Central Web page:

http://www.ibm.com/eserver/support/fixes/fixcentral/main/pseries/aix

The Service Update Management Assistant can help you to automate the task of checking and downloading operating system downloads, and is part of the base operating system. For more information about the suma command functionality, go to the following Web page:

http://www14.software.ibm.com/webapp/set2/sas/f/genunix/suma.html

AIX Version 6.1

AIX 6.1 with the 6100-05 Technology Level or later

For information regarding AIX V6.1 maintenance and support, go to the Fix Central Web page:

http://www.ibm.com/eserver/support/fixes/fixcentral/main/pseries/aix

IBM i

Virtual I/O Server is required to install IBM i in a LPAR on PS700, PS701, and PS702 blades and all I/O must be virtualized.

򐂰 IBM i 6.1 with i 6.1.1 machine code, or later 򐂰 IBM i 7.1 or later

For a detailed guide on installing and operating IBM i with Power Blades, see the following Web page:

http://ibm.com/systems/resources/systems_power_hardware_blades_i_on_blade_readme.pdf

Linux

Linux is an open source operating system that runs on numerous platforms from embedded systems to mainframe computers. It provides a UNIX-like implementation in many computer architectures.

At the time of this writing, the supported versions of Linux on POWER7 processor technology based servers are as follows:

򐂰 SUSE Linux Enterprise Server 10 with SP3 and the latest maintenance, in POWER6

Compatibility mode

򐂰 SUSE Linux Enterprise Server 11 with SP1 or later, supporting POWER6 or POWER7

mode

򐂰 Red Hat RHEL 5.5 in POWER6 Compatibility mode

Linux operating system licenses are ordered separately from the hardware. You can obtain Linux operating system licenses from IBM, to be included with your POWER7 processor technology-based servers, or from other Linux distributors.

Chapter 2. Architecture and technical overview 73

For information about the features and external devices supported by Linux, go to the following Web page:

http://www.ibm.com/systems/p/os/linux/

For information about SUSE Linux Enterprise Server, go to the following Web page:

http://www.novell.com/products/server

For information about Red Hat Enterprise Linux Advanced Server, go to the following Web page:

http://www.redhat.com/rhel/features

Supported virtualization features are listed in 3.3.8, “Supported PowerVM features by operating system” on page 98.

2.13 IBM EnergyScale

IBM EnergyScale technology provides functions to help the user understand and dynamically optimize the processor performance versus processor power and system workload, to control IBM Power Systems power and cooling usage.

The BladeCenter AMM and IBM Systems Director Active Energy Manager exploit EnergyScale technology, enabling advanced energy management features to conserve power and improve energy efficiency. Intelligent energy optimization capabilities enable the POWER7 processor to operate at a higher frequency for increased performance and performance per watt, or reduce frequency to save energy.

2.13.1 IBM EnergyScale technology

This section describes IBM EnergyScale design features, and hardware and software requirements.

IBM EnergyScale consists of the following elements: 򐂰 A built-in EnergyScale device (formally known as Thermal Power Management Device or

TPMD)

򐂰 Power executive software. IBM Systems Director Active Energy Manager, an IBM

Systems Directors plug-in and BladeCenter AMM.

IBM EnergyScale functions include the following elements: 򐂰 Energy trending

EnergyScale provides continuous collection of real-time server energy consumption. This function enables administrators to predict power consumption across their infrastructure and to react to business and processing needs. For example, administrators might use such information to predict data center energy consumption at various times of the day, week, or month.

򐂰 Thermal reporting

IBM Systems Director Active Energy Manager can display measured ambient temperature and calculated exhaust heat index temperature. This information can help identify data center hot spots that require attention.

74 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

򐂰 Power Saver Mode

Power Saver Mode reduces the processor frequency and voltage by a fixed amount, reducing the energy consumption of the system and still delivering predictable performance. This percentage is predetermined to be within a safe operating limit and is not user configurable. The server is designed for a fixed frequency drop of 50% from nominal. Power Saver Mode is not supported during boot or reboot operations, although it is a persistent condition that is sustained after the boot when the system starts executing instructions.

򐂰 Dynamic Power Saver Mode

Dynamic Power Saver Mode varies processor frequency and voltage based on the use of the POWER7 processors. The user must configure this setting from the BladeCenter AMM or IBM Director Active Energy Manager. Processor frequency and use are inversely proportional for most workloads, implying that as the frequency of a processor increases, its use decreases, given a constant workload. Dynamic Power Saver Mode takes advantage of this relationship to detect opportunities to save power, based on measured real-time system use. When a system is idle, the system firmware lowers the frequency and voltage to Power Saver Mode values. When fully used, the maximum frequency varies, depending on whether the user favors power savings or system performance. If an administrator prefers energy savings and a system is fully-used, the system can reduce the maximum frequency to 95% of nominal values. If performance is favored over energy consumption, the maximum frequency will be at least 100% of nominal. Dynamic Power Saver Mode is mutually exclusive with Power Saver mode. Only one of these modes can be enabled at a given time.

򐂰 Power capping

Power capping enforces a user-specified limit on power usage. Power capping is not a power saving mechanism. It enforces power caps by throttling the processors in the system, degrading performance significantly. The idea of a power cap is to set a limit that should never be reached but frees up margined power in the data center. The margined power is the amount of extra power that is allocated to a server during its installation in a datacenter. It is based on the server environmental specifications that usually are never reached because server specifications are always based on maximum configurations and worst case scenarios. The user must set and enable an energy cap from the BladeCenter AMM or IBM Systems Director Active Energy Manager user interface.

򐂰 Soft power capping

Soft power capping extends the allowed energy capping range further, beyond a region that can be guaranteed in all configurations and conditions. If an energy management goal is to meet a particular consumption limit, soft power capping is the mechanism to use.

򐂰 Processor Core Nap

The IBM POWER7 processor uses a low-power mode called Nap that stops processor execution when there is no work to do on that processor core. The latency of exiting Nap falls within a partition dispatch (context switch) such that the POWER Hypervisor can use it as a general purpose idle state. When the operating system detects that a processor thread is idle, it yields control of a hardware thread to the POWER Hypervisor. The POWER Hypervisor immediately puts the thread into Nap mode. Nap mode allows the hardware to clock-off most of the circuits inside the processor core. Reducing active energy consumption by turning off the clocks allows the temperature to fall, which further reduces leakage (static) power of the circuits causing a cumulative effect. Unlicensed cores are kept in core Nap until they are licensed and return to core Nap when they are unlicensed again.

Chapter 2. Architecture and technical overview 75

򐂰 Processor folding

Processor folding is a consolidation technique that dynamically adjusts, over the short-term, the number of processors available for dispatch to match the number of processors demanded by the workload. As the workload increases, the number of processors made available increases. As the workload decreases, the number of processors made available decreases. Processor folding increases energy savings during periods of low to moderate workload because unavailable processors remain in low-power idle states longer.

򐂰 EnergyScale for I/O

IBM POWER processor-based systems automatically power off pluggable, PCI adapter slots that are empty or not being used. System firmware automatically scans all pluggable PCI slots at regular intervals, looking for those that meet the criteria for being not in use and powering them off. This support is available for all POWER processor-based servers, and the expansion units that they support.

In addition to the normal EnergyScale functions, the EnergyScale device in the PS700, PS701, and PS702 blades incorporate the following BladeCenter functions:

򐂰 Transition from over-subscribed power consumption to nominal power consumption when

commanded by the BladeCenter AMM. This transition is signaled by the AMM as a result of a redundant power supply failure in the BladeCenter.

򐂰 Report blade power consumption to the AMM through the service processor 򐂰 Report blade system voltage levels to the AMM through the service processor 򐂰 Accommodate BladeCenter/AMM defined thermal triggers such as warning temperature,

throttle temperature, and critical temperature

2.13.2 EnergyScale device

The EnergyScale device dynamically optimizes the processor performance depending on processor power and system workload.

The IBM POWER7 chip is a significant improvement in power and performance over the IBM POWER6 chip. POWER7 has more internal hardware, and power and thermal management functions to interact with:

򐂰 More hardware: Eight cores versus two cores, four threads versus two threads per core,

and asynchronous processor core chiplet

򐂰 Advanced Idle Power Management functions 򐂰 Advanced Dynamic Power Management (DPM) functions in all units in hardware

(processor cores, processor core chiplet, chip-level nest unit level, and chip level)

򐂰 Advanced Actuators/Control 򐂰 Advanced Accelerators

The new EnergyScale device has a more powerful microcontroller, more A/D channels and more busses to handle the increase workload, link traffic, and new power and thermal functions.

76 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Chapter 3. Virtualization

IBM Advance POWER Virtualization (PowerVM) is a feature use to consolidate workload to deliver cost savings and improve infrastructure responsiveness. As we look for ways to maximize the return on your IT infrastructure investments, consolidating workloads and increasing server use becomes an attractive proposition.

IBM Power Systems, combined with PowerVM technology, are designed to help you consolidate and simplify your IT environment. The following list details key capabilities:

򐂰 Improve server use by consolidating diverse sets of applications. 򐂰 Share CPU, memory, and I/O resources to reduce total cost of ownership. 򐂰 Improve business responsiveness and operational speed by dynamically re-allocating

resources to applications as needed, to better anticipate changing business needs.

򐂰 Simplify IT infrastructure management by making workloads independent of hardware

resources, enabling you to make business-driven policies to deliver resources based on time, cost, and service-level requirements.

򐂰 Move running workloads between servers to maximize availability and avoid planned

downtime

This chapter discusses the virtualization technologies and features on IBM POWER7 processor-based blade servers:

򐂰 3.1, “POWER Hypervisor” on page 78 򐂰 3.2, “POWER processor modes” on page 82 򐂰 3.3, “PowerVM” on page 83

3.1 POWER Hypervisor

Combined with features designed into the POWER7 processors, the POWER Hypervisor delivers functions that enable capabilities, including dedicated processor partitioning, micro-partitioning, virtual processors, IEEE VLAN compatible virtual switch, and virtual SCSI adapters, virtual Fibre Channel adapters, and virtual consoles.

The POWER Hypervisor technology is integrated with all IBM POWER servers including the POWER7 processor-based blade servers. The hypervisor orchestrates and manages system virtualization, including creating logical partitions and dynamically moving resources across multiple operating environments. The POWER Hypervisor is a basic component of the system firmware that is layered between the hardware and operating system. POWER Hypervisor offers the following functions:

򐂰 Provides an abstraction layer between the physical hardware resources and the logical

partitions using them

򐂰 Enforces partition integrity by providing a security layer between logical partitions 򐂰 Controls the dispatch of virtual processors to physical processors and saves and restores

all processor state information during a logical processor context switch

򐂰 Controls hardware I/O interrupt management facilities for logical partitions 򐂰 Provides virtual Ethernet switch between logical partitions that help to reduce the need for

physical Ethernet adapters for interpartition communication

򐂰 Monitors the service processor and performs a reset or reload if it detects the loss of the

service processor, notifying the operating system if the problem is not corrected

򐂰 Uses micro-partitioning to allow multiple instances of operating system to run on POWER6

and POWER7 processor-based servers or Blades

The POWER Hypervisor is always installed and activated, regardless of system configuration. The POWER Hypervisor does not own any physical I/O devices. All physical I/O devices in the system are owned by logical partitions or the Virtual I/O Server.

Memory is required to support the resource assignment to the logical partitions on the server. The amount of memory required by the POWER Hypervisor firmware varies according to several factors. The following factors influence POWER Hypervisor memory requirements:

򐂰 Number of logical partitions 򐂰 Number of physical and virtual I/O devices used by the logical partitions 򐂰 Maximum memory values specified in the logical partition profiles

The minimum amount of physical memory to create a partition is the size of the system’s logical memory block (LMB). The default LMB size varies according to the amount of memory configured in the system, as shown in Table 3-1.

Table 3-1 Configured CEC memory-to-default LMB size

Configurable memory in the system Default Logical Memory Block

Less than 4 GB 16 MB

Greater than 4 GB up to 8 GB 32 MB

Greater than 8 GB up to 16 GB 64 MB

Greater than 16 GB up to 32 GB 128 MB

Greater than 32 GB 256 MB

78 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Physical memory assigned to partitions are in increments of LMB.

The POWER Hypervisor provides the following types of virtual I/O adapters:

򐂰 Virtual SCSI 򐂰 Virtual Ethernet 򐂰 Virtual Fibre Channel 򐂰 Virtual (TTY) console

Virtual I/O adapters are defined by system administrators during logical partition definition. Configuration information for the adapters is presented to the partition operating system.

Virtual SCSI

The POWER Hypervisor provides a virtual SCSI mechanism for virtualization of storage devices. Virtual SCSI allows secure communications between a logical partition and the IO Server (VIOS). The storage virtualization is accomplished by pairing two adapters: a virtual SCSI server adapter on VIOS and a virtual SCSI client adapter on IBM i, Linux, or AIX partitions. The combination of Virtual SCSI and VIOS provides the opportunity to share physical disk adapters in a flexible and reliable manner.

Virtual Ethernet

The POWER Hypervisor provides an IEEE 802.1Q VLAN-style virtual Ethernet switch that allows partitions on the same server to use a fast and secure communication without any need for physical connection.

Virtual Ethernet support starts with AIX Version 5.3, or the appropriate level of Linux supporting virtual Ethernet devices (see 3.3.8, “Supported PowerVM features by operating system” on page 98). The virtual Ethernet is part of the base system configuration.

Virtual Ethernet has the following major features: 򐂰 The virtual Ethernet adapters can be used for both IPv4 and IPv6 communication and can

transmit packets with a size up to 65408 bytes. Therefore, the maximum MTU for the corresponding interface can be up to 65394 (=65408 -14 for the header) in non-VLAN case and to 65390 (=65408-14- 4) if VLAN tagging is used).

򐂰 The POWER Hypervisor presents itself to partitions as a virtual 802.1Q compliant switch.

The maximum number of VLANs is 4096. Virtual Ethernet adapters can be configured as either untagged or tagged (following the IEEE 802.1Q VLAN standard).

򐂰 An AIX partition supports 256 virtual Ethernet adapters for each logical partition. Besides

a default port VLAN ID, the number of additional VLAN ID values that can be assigned per Virtual Ethernet adapter is 20, which implies that each Virtual Ethernet adapter can be used to access 21 virtual networks.

򐂰 Each operating system partition detects the virtual local area network (VLAN) switch as an

Ethernet adapter without the physical link properties and asynchronous data transmit operations.

Any virtual Ethernet can also have connectivity outside of the server if a layer-2 bridge to a physical Ethernet adapter is set in one VIOS partition (see 3.3.3, “VIOS” on page 88 for more details about shared Ethernet). This is also known as a Shared Ethernet Adapter.

Note: Virtual Ethernet is based on the IEEE 802.1Q VLAN standard. No physical I/O adapter is required when creating a VLAN connection between partitions, and no access to an outside network is required for inter-partition communication.

Chapter 3. Virtualization 79

Virtual Fibre Channel

A virtual Fibre Channel adapter is a virtual adapter that provides client logical partitions with a Fibre Channel connection to a storage area network through the VIOS logical partition. The VIOS logical partition provides the connection between the virtual Fibre Channel adapters on the VIOS logical partition and the physical Fibre Channel adapters on the managed system.

NPIV is a standard technology for Fibre Channel networks. It enables you to connect multiple logical partitions to one physical port of a physical Fibre Channel adapter. Each logical partition is identified by a unique WWPN, which means that you can connect each logical partition to independent physical storage on a SAN.

Note: To enable NPIV on a managed system, we need VIOS to be at version 2.1 or later. Also, check if the Fibre Channel adapter on managed system supports NPIV.

You can only configure virtual Fibre Channel adapters on client logical partitions that run the following operating systems:

򐂰 AIX version 6.1 Technology Level 2, or later 򐂰 AIX 5.3 Technology Level 9 򐂰 IBM i version 6.1.1, or later 򐂰 SUSE Linux Enterprise Server 11, or later

For details on which expansion card support NPIV see 3.3.7, “N_Port ID Virtualization (NPIV)” on page 96

On systems that are managed by the Integrated Virtualization Manager (IVM), you can dynamically add and remove worldwide port names (WWPNs) to and from logical partitions, and you can dynamically change the physical ports to which the WWPNs are assigned. You can also view information about the virtual and physical Fibre Channel adapters and the WWPNs by using the lsmap and lsnports commands.

For more information about how virtual Fibre Channel is managed on IVM see the following Web page:

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/arecu/ar ecukickoff.htm

Figure 3-1 on page 81 depicts the connections between the client partition virtual Fibre Channel adapters and the external storage.

80 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

VIRTUAL I/O SERVER

Client Logical

Partition 2

Physical

Disk 1

Physical Fibre

Channel Adapter

Server Virtual Fiber

Channel Adapter

Server Virtual Fiber

Channel Adapter

Physical

Disk 2

Client Virtual

Fiber Channel

Adapter

Client Logical

Partition 1

Client Virtual

Fiber Channel

Adapter

HYPERVISOR

STORAGE AREA NETWORK

Figure 3-1 Connectivity between virtual Fibre Channels adapters and external SAN devices

Virtual Serial Adapters (TTY) console

Virtual serial adapters provide a point-to-point connection from one logical partition to another, or from the Hardware Management Console (HMC) to each logical partition on the managed system. Virtual serial adapters are used primarily to establish terminal or console connections to logical partitions.

Each partition needs to have access to a system console. Tasks such as operating system installation, network setup, and certain problem analysis activities require a dedicated system console. The POWER Hypervisor provides the virtual console using a virtual TTY or serial adapter and a set of Hypervisor calls to operate on them. Virtual TTY does not require the purchase of any additional features or software such as the PowerVM Edition features.

The operating system console can be provided by the IVM virtual TTY, using the SOL feature.

Chapter 3. Virtualization 81

3.2 POWER processor modes

Although, strictly speaking, not a virtualization feature, POWER modes are described in this section because they affect certain virtualization features.

On Power System servers, partitions can be configured to run in several modes, including: 򐂰 POWER6 compatibility mode

This execution mode is compatible with v2.05 of the Power Instruction Set Architecture (ISA). For more information, see:

http://www.power.org/resources/reading/PowerISA_V2.05.pdf

򐂰 POWER6+ compatibility mode

This mode is similar to POWER6, with 8 additional Storage Protection Keys.

򐂰 POWER7 mode

This is the native mode for POWER7 processors, implementing v2.06 of the Power Instruction Set Architecture. For more information, see:

http://www.power.org/resources/downloads/PowerISA_V2.06_PUBLIC.pdf

Figure 3-2 shows how to choose the processor compatibility mode by editing the partition properties of a logical partition from the IVM.

Figure 3-2 Configuring partition profile compatibility mode from IVM

Table 3-2 on page 83 lists the differences between these modes.

82 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Table 3-2 Differences between POWER6 and POWER7 mode

POWER6 and POWER6+ mode POWER7 mode Customer value

2-thread SMT 4-thread SMT Throughput performance, processor

core utilization

VMX (Vector Multimedia Extension or AltiVec)

Affinity OFF by default 3-tier memory, micro-partition Affinity Improved system performance for

򐂰 Barrier Synchronization 򐂰 Fixed 128-byte Array; Kernel

Extension Access

64-core and 128-thread scaling 򐂰 32-core and 128-thread scaling

EnergyScale CPU Idle EnergyScale CPU Idle and Folding with

VSX (Vector Scalar Extension) High performance computing for

graphic and scientific workload.

system images spanning sockets and nodes.

򐂰 Enhanced Barrier Synchronization 򐂰 Variable Sized Array; User Shared

Memory Access

򐂰 64-core and 256-thread scaling 򐂰 256-core and 1024-thread scaling

NAP and SLEEP

High performance computing parallel programming synchronization facility

Performance and Scalability for Large Scale-Up Single System Image Workloads (such as OLTP, ERP scale-up, WPAR consolidation).

Improved Energy Efficiency

3.3 PowerVM

The PowerVM platform is the family of technologies, capabilities and offerings that deliver industry-leading virtualization on the IBM Power Systems. It is the new umbrella branding term for PowerVM (Logical Partitioning, Micro-Partitioning™, Power Hypervisor, VIOS, Live Partition Mobility, Workload Partitions, and so on). As with Advanced Power Virtualization in the past, PowerVM is a combination of hardware enablement and value-added software.

3.3.1 PowerVM Editions

This section provides information about the PowerVM Editions on POWER7 processor-based blade servers.

򐂰 PowerVM Express Edition

This edition is intended for evaluations, pilots, and proof of concepts, generally in single-server projects. This edition supports up to three partitions per system (VIOS, AIX, Linux, and IBM i) that share processors and I/O. It allows users to try out the Integrated Virtualization Manager (IVM) and the VIOS.

򐂰 PowerVM Standard Edition

This edition is intended for production deployments, and server consolidation. This edition makes the POWER7 systems an ideal platform for consolidation of AIX, Linux, and IBM i operating system applications, helping clients reduce infrastructure complexity and cost.

Offering an intuitive, Web-based interface for managing virtualization within a single blade, the IVM component of VIOS allows the small business IT manager to set up and manage logical partitions (LPARs) quickly and easily. It also enables Virtual I/O and Virtual Ethernet so that storage and communications adapters can be shared among all the LPARs running on the PS700, PS701, and PS702 Blade Servers. Ultimately, IBM micro-partitioning technology allows each processor core to be subdivided into as many as 10 virtual servers. Because the PS700, PS701, and PS702 is built with POWER7

Chapter 3. Virtualization 83

technology, other advanced virtualization functions such as Shared Dedicated Capacity can be exploited.

򐂰 PowerVM Enterprise Edition

The Enterprise edition is suitable for large server deployments such as multi-server deployments and cloud infrastructure. This edition includes all the features of PowerVM Standard Edition plus a new capability called Live Partition Mobility. Live Partition Mobility allows for the movement of a running AIX or Linux partition from one POWER7 processor-based server to another with no application downtime, resulting in better system use, improved application availability, and potential energy savings. With Live Partition Mobility, planned application downtime due to regular server maintenance can be a thing of the past.

For each VIOS license ordered, an order for either the one-year (5771-VIO) or three-year (5773-VIO) Software Maintenance (SWMA) is also submitted. You must purchase a license for each active processor on the server.

Note: PowerVM Express Edition, PowerVM Standard Edition, and PowerVM Enterprise Edition are optional when running AIX or Linux. PowerVM Express Edition, PowerVM Standard Edition or PowerVM Enterprise Edition is required when running the IBM i operating system on the PS 700, PS701 and PS702 Blade Servers

Table 3-3 lists the PowerVM Edition available on each model of POWER7 processor-based blade servers with their feature code:

Table 3-3 PowerVM Edition and feature codes

Blade Servers Power VM Express Power VM Standard PowerVM Enterprise

PS700 #5225 #5227 #5228

PS701 #5225 #5227 #5228

PS702 #5225 #5227 #5228

Note: It is possible to upgrade from the Express Edition to the Standard or Enterprise Edition, and from Standard to Enterprise Editions.

Table 3-4 lists the offerings of the three PowerVM editions for Power7 blades.

Table 3-4 PowerVM capabilities by edition for POWER7-based blades

PowerVM Offerings Express Standard Enterprise

Micro-partitions

Maximum LPARs up to 3 per server 10 per core 10 per core

Management IVM IVM IVM

Virtual I/O Server

NPIV

Live Partition Mobility

Active Memory Sharing

Ye s Ye s Ye s

No No Ye s

LX86

84 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

Ye s Ye s Ye s

The PowerVM Editions Web site also contains useful information:

http://publib.boulder.ibm.com/infocenter/systems/scope/hw/index.jsp?topic=/arecu/a recukickoff.htm

3.3.2 Logical partitions

Logical partitions (LPARs) and virtualization increase use of system resources and add a new level of configuration possibilities. This section provides details and configuration specifications about this topic. A logical partition can be regarded as a logical server, capable of booting an operating system and running a workload.

Dynamic logical partitioning

LPAR was introduced with the POWER4™ processor-based product line and the IBM AIX Version 5.1 operating system. This technology offered the capability to divide a pSeries® system into multiple logical partitions, allowing each logical partition to run an operating environment on dedicated attached devices, such as processors, memory, and I/O components.

Later, dynamic logical partitioning increased the flexibility, allowing selected system resources (such as processors, memory, and I/O components) to be added and deleted from logical partitions as they are executing. IBM AIX Version 5.2, with necessary enhancements to enable dynamic LPAR, was introduced in 2002. The ability to reconfigure dynamic LPARs encourages system administrators to redefine available system resources dynamically to reach the optimum capacity for each defined dynamic LPAR.

Micro-partitioning

Virtualization of physical processors in POWER5, POWER6, and POWER7 systems introduces an abstraction layer that is implemented in POWER Hypervisor. Micro-partitioning is the ability to distribute the processing capacity of one or more physical processors among one or more logical partitions. Thus, processors are shared amongst logical partitions. Micro-partitioning technology allows you to allocate fractions of processors to a logical partition.

The POWER Hypervisor abstracts the physical processors and presents a set of virtual processors to the operating system within the micro-partitions on the system. The operating system sees only the virtual processors and dispatches runable tasks to them in the normal course of running a workload.

From an operating system perspective, a virtual processor cannot be distinguished from a physical processor, unless the operating system has been enhanced to be made aware of the difference. Physical processors are abstracted into virtual processors that are available to partitions. The meaning of the term

When defining a shared processor partition, several options have to be defined: 򐂰 Processing Units

The minimum, desired, and maximum processing units. Processing units are defined as processing power, or the fraction of time that the partition is dispatched on physical processors. Processing units define the capacity entitlement of the partition.

򐂰 Cap or Uncap partition

physical processor in this section is a processor core.

Select whether or not the partition can access extra processing power to “fill up” its virtual processors beyond its capacity entitlement, selecting either to cap or uncap your partition. If spare processing power is available in the processor pool or other partitions are not

Chapter 3. Virtualization 85

using their entitlement, an uncapped partition can use additional processing units if its entitlement is not enough to satisfy its application processing demand.

򐂰 Weight

The weight (preference) is in the case of an uncapped partition.

򐂰 Virtual processors

The minimum, desired, and maximum number of virtual processors. A virtual processor is a depiction or a representation of a physical processor that is presented to the operating system running in a micro-partition

The POWER Hypervisor calculates a partition’s processing power based on minimum, desired, and maximum values, processing mode and on other active partitions’ requirements. The actual entitlement is never smaller than the processing units desired value but can exceed that value in the case of an uncapped partition and can be up to the number of virtual processors allocated.

A partition can be defined with a processor capacity as small as 0.10 processing units. This represents 0.1 of a physical processor. Each physical processor can be shared by up to 10 shared processor partitions and the partition’s entitlement can be incremented fractionally by as little as 0.01 of the processor. The shared processor partitions are dispatched and time-sliced on the physical processors under control of the POWER Hypervisor. The shared processor partitions are created and managed by the HMC or Integrated Virtualization Management.

Partitioning maximums on the POWER7-based blades is as follows:

򐂰 The PS700 can have four dedicated partitions or up to 40 micro-partitions 򐂰 The PS701 can have eight dedicated partitions or up to 80 micro-partitions 򐂰 The PS702 can have 16 dedicated partitions or up to 160 micro-partitions

It is important to point out that the maximums stated are supported by the hardware, but the practical limits depend on the application workload demands.

The following list details additional information about virtual processors: 򐂰 A virtual processor can be running (dispatched) either on a physical processor or as

standby waiting for a physical processor to became available.

򐂰 Virtual processors do not introduce any additional abstraction level. They are only a

dispatch entity. On a physical processor, virtual processors run at the same speed as the physical processor.

򐂰 Each partition’s profile defines CPU entitlement, which determines how much processing

power any given partition should receive. The total sum of CPU entitlement of all partitions cannot exceed the number of available physical processors in the pool.

򐂰 The number of virtual processors can be changed dynamically through a dynamic LPAR

operation.

Processor mode

When you create a logical partition, you can assign entire processors for dedicated use, or you can assign partial processor units from a shared processor pool. This setting defines the processing mode of the logical partition.

Figure 3-3 shows a diagram of the concepts discussed in the remaining sections.

86 IBM BladeCenter PS700, PS701, and PS702 Technical Overview and Introduction

IBM BladeCenter PS700, BladeCenter PS701, BladeCenter PS702 Technical Overview And Introduction

Specifications and Main Features

Frequently Asked Questions

User Manual