IBM 440 User Manual

Front cover

IBM xSeries
440 Planning and
Installation Guide

Describes the technical details of
the x440 models

Helps you prepare for and
perform an installation

Covers key IBM Director
management tools

ibm.com/redbooks

David Watts

Reza Fanaei Aghdam

Duncan Furniss

Jason King

International Technical Support Organization

IBM ^ xSeries 440 Planning and Installation
Guide

October 2002

SG24-6196-00

Note: Before using this information and the product it supports, read the information in
“Notices” on page vii.

First Edition (October 2002)

This edition applies to the IBM ^ xSeries 440, machine type 8687.

© Copyright International Business Machines Corporation 2002. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix

Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Chapter 1. Technical description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 The x440 product line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 System partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 IBM XA-32 chipset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.4 Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4.1 Intel Xeon Processor MP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.4.2 Intel Xeon Processor DP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5 SMP Expansion Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.6 IBM XceL4 Server Accelerator Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.7 System memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.8 PCI subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.9 Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.10 Light Path Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

1.11 Remote Supervisor Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

1.12 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.13 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Chapter 2. Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.1 xSeries 440 application solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.1.1 Server consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.1.2 Enterprise applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.1.3 Infrastructure applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.1.4 Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.2 Why choose the x440 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.1 IBM XA-32 chipset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.2 Intel Xeon MP and DP processors . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.2.3 XceL4 Server Accelerator Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.2.4 High-performance memory subsystem . . . . . . . . . . . . . . . . . . . . . . . 46

2.2.5 Active PCI-X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.2.6 XpandOnDemand scalability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.2.7 System Partition Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

© Copyright IBM Corp. 2002. All rights reserved. iii

2.3 The benefits of system partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.4 Server consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.4.1 Types of server consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.4.2 Why consolidate servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.4.3 Benefits from server consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Chapter 3. Planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.1 System hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.1.1 Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.1.2 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.1.3 PCI slot configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.1.4 Broadcom Gigabit Ethernet controller . . . . . . . . . . . . . . . . . . . . . . . . 72

3.2 Cabling and connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.2.1 SMP Expansion Module connectivity . . . . . . . . . . . . . . . . . . . . . . . . 74

3.2.2 Remote Supervisor Adapter connectivity . . . . . . . . . . . . . . . . . . . . . 77

3.2.3 Remote Expansion Enclosure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.2.4 Serial connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.3 Storage considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.3.1 xSeries storage solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.3.2 Disk subsystem performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.3.3 Tape backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

3.4 Server partitioning and consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

3.5 Operating system considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

3.5.1 Windows 2000 Datacenter Server . . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.5.2 Microsoft Windows NT 4.0 Enterprise Edition. . . . . . . . . . . . . . . . . . 95

3.5.3 Microsoft Windows 2000 Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.5.4 Microsoft Windows .NET Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.5.5 Novell NetWare. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.5.6 Red Hat/SuSE Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.5.7 VMware ESX Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3.6 Application considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.6.1 Scalability and performance considerations . . . . . . . . . . . . . . . . . . 100

3.6.2 SMP and server types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.7 Rack installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

3.8 Power considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3.9 Solution Assurance Review. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Chapter 4. Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.1 System BIOS settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4.1.1 Updating BIOS and firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4.1.2 Enabling memory mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4.1.3 Enabling Hyper-Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.2 Device drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

iv IBM ^ xSeries 440 Planning and Installation Guide

4.3 Operating system installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.3.1 Microsoft Windows 2000 Server and Advanced Server . . . . . . . . . 112

4.3.2 Red Hat Linux installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

4.3.3 NetWare installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.3.4 VMware ESX Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

4.4 Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Chapter 5. Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.1 Active PCI Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

5.1.1 Using Active PCI Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

5.1.2 Adding adapters to the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

5.1.3 Analyzing an existing configuration. . . . . . . . . . . . . . . . . . . . . . . . . 144

5.2 System Partition Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

5.3 Process Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

5.3.1 Process alias rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

5.3.2 Process execution rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

5.3.3 Group process execution rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Contents v

vi IBM ^ xSeries 440 Planning and Installation Guide

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that
does not infringe any IBM intellectual property right may be used instead. However, it is the user's
responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document.
The furnishing of this document does not give you any license to these patents. You can send license
inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION

PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may
make improvements and/or changes in the product(s) and/or the program(s) described in this publication at
any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm
the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on
the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the
sample programs are written. These examples have not been thoroughly tested under all conditions. IBM,
therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,
modify, and distribute these sample programs in any form without payment to IBM for the purposes of
developing, using, marketing, or distributing application programs conforming to IBM's application
programming interfaces.

© Copyright IBM Corp. 2002. All rights reserved. vii

Trademarks

The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:

Active Memory™
Active™ PCI-X
Chipkill™
DB2®
Electronic Service Agent™
Enterprise Storage Server™
ESCON®
FlashCopy®
IBM®
Informix®
iSeries™
Memory ProteXion™
Netfinity®

The following terms are trademarks of International Business Machines Corporation and Lotus Development
Corporation in the United States, other countries, or both:

Domino™ Lotus® Notes®

The following terms are trademarks of other companies:

ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United
States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun
Microsystems, Inc. in the United States, other countries, or both.

PowerPC®
PowerPC 750™
Predictive Failure Analysis®
pSeries™
Redbooks(logo)™
RETAIN®
S/390®
ServeRAID™
ServerProven®
SP™
SP1®
SP2®
ThinkPad®

Tivoli®
TotalStorage™
Wake on LAN®
WebSphere®
X-Architecture™
XA-32™
XceL4™
XpandOnDemand™
xSeries™
zSeries™

C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure
Electronic Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.

viii IBM ^ xSeries 440 Planning and Installation Guide

Preface

The IBM ^ xSeries 440 is IBM’s flagship industry standard server and is
the first full implementation of the 32-bit IBM XA-32 chipset, code named
“Summit”, as part of the Enterprise X-Architecture strategy. The x440 provides
new levels of high availability and price performance, and offers scalability from
two-way to 16-way SMP, from 2 GB to 128 GB of memory, and up to 24 PCI slots,
all in one single system image.

This redbook is a comprehensive resource on the technical aspects of the server,
and is divided into five key subject areas:

򐂰 Chapter 1, “Technical description” introduces the server and its subsystems

򐂰 Chapter 2, “Positioning” examines the types of applications that would be

򐂰 Chapter 3, “Planning” describes the aspects of planning to purchase and

򐂰 Chapter 4, “Installation” goes through the process of installing Windows 2000,

and describes the key features and how they work.

used on a server such as the x440, including server consolidation,
line-of-business application, and infrastructure applications. It reviews the
features that make the x440 such a powerful system.

planning to install the x440. It covers such topics as configuration, operating
system specifics, scalability, and physical site planning.

Red Hat Linux, NetWare, and VMware ESX Server. It describes what BIOS
and drivers updates are appropriate and when to install them.

򐂰 Chapter 5, “Management” describes how to use the key IBM Director

extensions designed for the x440: System Partition Manager, Active PCI
Manager, and Process Control.

A partner redbook is

and VMware ESX Server

Server Consolidation with the IBM

, SG24-6852.

^

xSeries 440

The team that wrote this redbook

This redbook was produced by a team of specialists from around the world
working at the International Technical Support Organization, Raleigh Center.

David Watts is a Consulting IT Specialist at the International Technical Support
Organization in Raleigh. He manages residencies and produces IBM
Redbooks on hardware and software topics related to IBM xSeries systems
and associated client platforms. He has authored over 20 redbooks; his most

© Copyright IBM Corp. 2002. All rights reserved. ix

recent books include

Solutions

Bachelor of Engineering degree from the University of Queensland (Australia)
and has worked for IBM for over 13 years. He is an IBM ^ Certified
Specialist for xSeries and an IBM Certified IT Specialist.

Reza Fanaei Aghdam is a Senior IT Specialist working in Zurich, Switzerland.
He has 10 years of experience in support of computer, software and
programming. He has a Bachelor of Computer Sciences degree from the
Fachhochschule Konstanz and a Bachelor of Information Management from the
University of Konstanz. His areas of expertise include xSeries servers, IBM
Director, IBM FAStT solutions, and database programming. He is a Microsoft
MCSE, Microsoft Certified Cluster Specialist, Novell MCNE, Citrix CCA, and an
IBM ^Certified Expert for xSeries.

Duncan Furniss is an Advisory IT Specialist for IBM Canada, and is the senior
xSeries product specialist for western Canada. He has 14 years of professional
experience with Intel-based hardware, networking, and storage technologies,
more than 11 of them at IBM. His areas of expertise include systems design and
implementation, performance tuning, and systems management. He currently
writes, consults, and presents on these and related topics regularly in the course
of his work. He is an IBM ^ Certified Specialist for xSeries. He was
co-author of the redbook

Jason King is a Service Engineer working for W J Moncrieff in Perth, Western
Australia. He has seven years of experience working with xSeries and Netfinity
hardware. He is a Microsoft Certified Professional and an IBM ^
Certified Specialist for xSeries. His areas of expertise include IBM xSeries
servers, Windows NT 4.0, Windows 2000, and IBM Director.

and

Implementing IBM Director Management Solutions

Integrating IBM Director with Enterprise Management

. He has a

High Availability without Clustering

.

x IBM ^ xSeries 440 Planning and Installation Guide

The team (l-r): David, Duncan, Reza, Jason

Thanks to the following people for their contributions to this project:

Alfredo Aldereguia, Lead Engineer, SS16 System Development, Raleigh
Kenny Bain, EMEA Advanced Technical Support, Greenock
Patrick de Broux, IT Consultant, ATS Product Introduction Centre, Hursley
Donn Bullock, Global Brand Manager, Enterprise X-Architecture, Raleigh
Alex Candelaria, Staff Engineer, Enterprise Support Group, Seattle
Michael Cannon, xSeries Sales & Technical Education, Raleigh
Mark Chapman, xSeries Marketing Communications, Raleigh
Henry Chung, Technical Project Manager, Datacenter Offerings, Seattle
Peter Escue, Americas Advanced Technical Support, Dallas
Dottie Gardner, Technical Project Manager, Information Development, Raleigh
Roger Hellman, xSeries Global Product Marketing Manager, Raleigh
Ron Humphrey, Technical Project Manager, Active PCI Manager, Seattle
Koichi Kii, Development Manager, Active PCI Manager, Seattle
Grace Lennil, IBM Center for Microsoft Technologies, Seattle
David A McIntosh, Technical Specialist, xSeries Techline, Greenock
John McAbel, World Wide Cluster Offering Product Manager, Beaverton
Gregg McKnight, Distinguished Engineer, xSeries Performance, Raleigh
Robert Moon, Team Lead, xSeries Techline, Greenock
Michael Parris, WW Technical Support Marketing, Raleigh
Kiron Rakkar, Manager, WebSphere Beta Programs, Raleigh
Paul Shaw, Active PCI Manager Development, Seattle
Gary Turner, Technical Project Manager, System Partition Manager, Seattle

Preface xi

Damon West, Course Developer, xSeries Education, Raleigh

Thanks also to the team that wrote the redbook

IBM

^

Keith Olsen, Gabriel Sallah, and Chandrasekhara Seetharaman.

xSeries 440 and VMware ESX Server

Become a published author

Join us for a two- to six-week residency program! Help write an IBM Redbook
dealing with specific products or solutions, while getting hands-on experience
with leading-edge technologies. You'll team with IBM technical professionals,
Business Partners and/or customers.

Your efforts will help increase product acceptance and customer satisfaction. As
a bonus, you'll develop a network of contacts in IBM development labs, and
increase your productivity and marketability.

Find out more about the residency program, browse the residency index, and
apply online at:

ibm.com/redbooks/residencies.html

Comments welcome

Your comments are important to us!

Server Consolidation with the

, SG24-6852: Steve Russell,

We want our Redbooks to be as helpful as possible. Send us your comments
about this or other Redbooks in one of the following ways:

򐂰 Use the online Contact us review redbook form found at:

ibm.com/redbooks

򐂰 Send your comments in an Internet note to:

[e-mail]

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization
Dept. HZ8 Building 662
P.O. Box 12195
Research Triangle Park, NC 27709-2195

xii IBM ^ xSeries 440 Planning and Installation Guide

Chapter 1. Technical description

The IBM ^ xSeries 440 is the latest IBM top-of-the-range server and is
the first full implementation of the 32-bit IBM XA-32 chipset, code named
“Summit” as part of the Enterprise X-Architecture strategy. The x440 provides
new levels of high availability and price performance, and offers scalability
beyond a single server.

The following are the key features of the x440:
򐂰 Two-way Intel Xeon processor MP models, upgradable to four-way and

eight-way

1

򐂰 Two-way Intel Xeon processor DP models, upgradable to four-way Xeon DP

or four-way (and beyond) Xeon MP

򐂰 Ability to connect two x440s together to form a single eight-way (4+4), 12-way

(4+8) or 16-way (8+8) SMP system image

򐂰 Physical system partitioning, controlled by IBM Director and the Remote

Supervisor Adapter, to consolidate servers or set up high-speed clustering
configurations

򐂰 4U rack-dense design
򐂰 32 MB XceL4 Server Accelerator Cache providing an extra level of cache
򐂰 2 GB or 4 GB RAM standard, up to 64 GB total using 2 GB ECC SDRAM

DIMMs

© Copyright IBM Corp. 2002. All rights reserved. 1

򐂰 Memory enhancement such as memory mirroring, Chipkill, and Memory

ProteXion

򐂰 Six Active PCI-X slots: two 64-bit 133 MHz, two 64-bit 100 MHz, two 64-bit

66 MHz

򐂰 Connectivity to an RXE-100 external PCI-X enclosure for an additional 12

PCI-X slots

򐂰 Integrated dual-channel Ultra160 SCSI controller
򐂰 Two hot-swap 1” drive bays
򐂰 Support for major storage subsystems, including Fibre Channel and

ServeRAID

򐂰 Light Path Diagnostics and the Remote Supervisor Adapter for systems

management

򐂰 Integrated 10/100/1000 Mbps Ethernet controller

The ability to connect multiple systems together and to partition them is the
implementation of the concept of

XpandOnDemand represents the first industry-standard implementation of true
“pay-as-you-grow” servers. New levels of scalability are achieved using a building
block design that allows more cost-effective scalability. These technologies,
powered by the XA-32 chipset, will provide scalability from two-way up to 16-way
systems using “scalable enterprise nodes”, the x440s being each of those nodes,
and, optionally, one or more external remote I/O enclosures.

XpandOnDemand.

Each scalable enterprise node contains processors, memory, I/O support,
storage and other devices and operates as an independent system. Each node
may run a different operating system from the other nodes, or if desired multiple
nodes can be assigned to one operating system image via system partitioning.
Nodes are attached to one another through dedicated high-speed
interconnections, called SMP Expansion Ports. This offers the flexibility to run
several hardware nodes as either a single complex of nodes or as two or more
smaller units to support multiple operating systems and/or clustered
configurations. The nodes can even be rearranged later into other configurations,
as needed.

1.1 The x440 product line

The models of the x440 are being made available throughout 2002. This is
because the complexity associated with developing the new IBM XA-32 chipset,
formerly known by its code name “Summit”, has meant additional development
and testing being required for introducing the x440 above that required of other

2 IBM ^ xSeries 440 Planning and Installation Guide

products. Additional testing pertains directly to the complexity of multiple SMP
configurations and the time commitment required for testing the ServerProven list
against each of these configurations.

All of the capabilities of the x440, including 16-way SMP capability and remote
I/O sharing, were announced in March 2002, but as a result of this additional
configuration development and testing, the x440 configurations will be introduced
in multiple phases during 2002 and 2003 as testing is completed.

Important: This document covers the products as of November 2002 in detail,
and only introduces the likely features of the follow-on models.

The models available as of November 2002 are listed in Table 1-1.

Table 1-1 Models available from November 2002

Model Standard processors Max SMP L2 cache L3 cache Std memory

8687-1RX 2x 1.4 GHz Intel Xeon MP 8-way 256 KB 512 KB 2 GB (4x 512 MB)

8687-2RX 2x 1.5 GHz Intel Xeon MP 8-way 256 KB 512 KB 2 GB (4x 512 MB)

8687-3RX 2x 1.6 GHz Intel Xeon MP 8-way 256 KB 1 MB 2 GB (4x 512 MB)

8687-4RX 2x 1.5 GHz Intel Xeon MP 8-way 256 KB 1 MB 2 GB (4x 512 MB)

8687-5RX 2x 1.9 GHz Intel Xeon MP 8-way 256 KB 1 MB 2 GB (4x 512 MB)

8687-6RX 4x 1.9 GHz Intel Xeon MP 8-way 256 KB 1 MB 4 GB (4x 1 GB)

8687-7RX 4x 2.0 GHz Intel Xeon MP 8-way 256 KB 2 MB 2 GB (4x 512 MB)

8687-3RY 2x 2.4 GHz Intel Xeon DP 4-way 512 KB 0 2 GB (4x 512 MB)

8687-4RY 4x 2.4 GHz Intel Xeon DP 4-way 512 KB 0 4 GB (8x 512 MB)

The x440 models that have Xeon MP processors installed currently only support
processor configurations of two, four and eight processors. The x440 models that
have Xeon DP processors only support processor configurations of two or four
processors, but can be upgraded to eight Xeon MP processors if desired.

Figure 1-1 on page 4 shows the available single-node configurations and the
CPU and memory options.

Chapter 1. Technical description 3

One RXE expansion
connection

xSeries 440

Two Xeon DP processors, 2-32 GB
Four Xeon DP processors, 4-64 GB
Two Xeon MP processors, 2-32 GB
Four Xeon MP processors, 2-64 GB
Eight Xeon MP proecessors, 4-64 GB

Figure 1-1 x440 configurations currently available

RXE-100

6 PCI-X slots
12 PCI-X slots

The attachment of a single RXE-100 Remote Expansion Enclosure is also
supported, as shown in Figure 1-1. The RXE-100 has six PCI-X slots standard,
upgradable to 12 PCI-X slots, giving the customer up to a total of 12 PCI-X or 18
PCI-X slots respectively.

In addition to the single-node configurations, three additional two-node
configurations are possible:

򐂰 A single 16-way system comprised of two eight-way x440 nodes, as shown in

Figure 1-2 on page 5. This will be available in November 2002.

򐂰 A single 12-way system comprised of an eight-way and a four-way x440, as

shown in Figure 1-3 on page 5. This will be available in early 2003.

򐂰 A single eight-way system comprised of two four-way x440 nodes, as shown

in Figure 1-3 on page 5. This will be available in early 2003.

Each of these configurations can optionally also have an RXE-100 attached (see
Figure 1-2 on page 5 for an example).

4 IBM ^ xSeries 440 Planning and Installation Guide

RXE expansion
connections

SMP expansion
connections

One 16-way complex

Each xSeries 440 has:

Eight CPUs
4-64 GB memory

Figure 1-2 16-way server configuration using two eight-way x440 nodes

RXE-100

6 PCI-X slots
12 PCI-X slots

One eight-way complex

x440 node 1:

Four CPUs
2-32 GB memory

SMP expansion
connections

x440 node 2:

Four CPUs
2-32 GB memory

Figure 1-3 Eight-way and 12-way two-node configurations

One 12-way complex

x440 node 1:

Eight CPUs
4-64 GB memory

SMP expansion
connections

x440 node 2:

Four CPUs
2-32 GB memory

Chapter 1. Technical description 5

1.2 System partitioning

Partitioning is the ability to divide a system to support multiple operating system
images simultaneously. The benefits of system partitioning include:

򐂰 Hardware consolidation
򐂰 Software migration and coexistence
򐂰 Version control
򐂰 Development, testing and maintenance
򐂰 Workload isolation
򐂰 Resource optimization around a particular application and operating system

combination

򐂰 Independent backup and recovery on a partition basis

There are two types of system partitioning: physical partitioning
(hardware-based, but not yet available) and logical partitioning (software-based,
enabled with VMware ESX Server):

򐂰 Logical partitioning

Using logical partitioning, administrators can partition a multinode complex at
the individual processor level (with associated memory, I/O and other required
resources) or even lower (that is, multiple partitions per processor) without
shutting down and restarting the hardware and software.

VMware ESX Server V1.5 supports one to eight partitions per CPU, up to a
maximum total of 64 partitions. For example, in an eight-way server, you can
have between eight partitions and 64 partitions. In V1.5, a partition cannot
span multiple CPUs, but a partition can be allocated a fraction of a CPU,
down to 1/8th of a CPU.

ESX Server virtualizes the resources of the x440 and is the closest that
Intel-based servers have come to date to the LPAR implementation of zSeries
mainframes.

When workload demands change, you can reassign resources from one
logical partition to another without having to shut down and restart the
system. ESX Server does not, however, support hot-adding of hardware
(such as disks and adapters).

For more information on ESX Server, see the redbook

with the IBM

3.5.7, “VMware ESX Server” on page 98.

򐂰 Physical partitioning

This form of partitioning is available in 4Q 2002 with the release of System
Partition Manager, a plug-in for IBM Director.

6 IBM ^ xSeries 440 Planning and Installation Guide

^

xSeries 440 and VMware ESX Server

Server Consolidation

, SG24-6852 and

With physical partitioning, a single multinode server complex can
simultaneously run multiple instances of one operating system in separate
partitions, as well as multiple versions of an operating system or even
different types of operating systems. The components of the server (for
example memory, CPUs, and I/O) are physically divided, under the control of
the server’s firmware and IBM Director.

The server can have up to two nodes, each capable of running its own
operating system and applications, all running simultaneously. A partition can
also span nodes, even to the point of having all four nodes serving one
operating system. Each node can be managed independently by IBM
Director.

See 5.2, “System Partition Manager” on page 150 for details.

1.3 IBM XA-32 chipset

The IBM XA-32 chipset is the product name describing the chipset developed
under the code name “Summit” and implemented on the IA-32 platform. A
product of the IBM Microelectronics Division in Austin, Texas, the XA-32 chipset
is fabricated using the latest in copper technology and is composed of the
following components:

򐂰 Memory controllers — one memory controller, code named “Cyclone”, per

four-way located within the SMP Expansion Module

򐂰 Processor/cache controllers — one processor and cache controller, code

named “Tw i s t er ”, per eight-way located within the SMP Expansion Module

򐂰 PCI bridges — two PCI bridges, code named “Winnipeg”, per x440 located on

the centerplane and the I/O board that control both the PCI-X and Remote I/O

Figure 1-4 on page 8 shows the various IBM XA-32 components in a four-way
x440 configuration.

Chapter 1. Technical description 7

CEC 1

CPU 1 CPU 2 CPU 3 CPU 4

IBM XA-32
core chipset

RXE

Expansion

Port A

(1 GBps)

Ultra160

SCSI

Gigabit

Ethernet

400 MHz

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

PCI bridge PCI bridge

3.2 GBps

3.2 GBps

3.2
GBps

100MHz
4-way
interleave

33 MHz66 MHz

Video

USB

Kbd/Ms

RSA

3.2 GBps

Processor &

cache controller

3.2 GBps

Memory

controller

2 GBps

Bus A66 MHz

64-bit

66 MHz

SMP Expansion
Ports (3.2GBps)

2 GBps

B-100

64-bit

100 MHz

133 MHz

Figure 1-4 xSeries 440 system block diagram — one SMP Expansion Module

D-133C-133

64-bit

The component that contains the CPUs, processor/cache controller, memory
controller, memory, and cache is called the SMP Expansion Module (or central
electronics complex—CEC). The Xeon MP-based models of the x440 ship with
one SMP Expansion Module with two or four CPUs and 2 GB or 4 GB of RAM.
The Xeon DP-based models have either two CPUs in one SMP Expansion
Module or four CPUs in two SMP Expansion Modules.

Tip: The terms central electronics complex, CEC, and SMP Expansion

Module

are used interchangeably in relation to the x440. We use SMP

Expansion Module in this redbook.

8 IBM ^ xSeries 440 Planning and Installation Guide

The CPUs are connected together with a 100 MHz frontside bus, but supply data
at an effective rate of 400 MHz using the “quad-pump” design of the Intel
NetBurst architecture as described in 1.4.1, “Intel Xeon Processor MP” on
page 13. To ensure the processors are optimally used, the x440 has a 32 MB
XceL4 Server Accelerator Cache, comprised of 200 MHz DDR memory. This L4
system cache services all CPUs in an SMP Expansion Module.

Memory used in the x440 is standard 133 MHz ECC SDRAM DIMMs; however,
the 133 MHz DIMMs are run at 100 MHz (for parts availability reasons). With
2 GB DIMMs, up to 32 GB can be installed using all 16 DIMM sockets. The
memory is four-way interleaved so that the memory subsystem can supply data
fast enough to match the throughput of the CPUs. Four-way interleaving means
that DIMMs must be installed in matched fours and in specific DIMM sockets (see

3.1.2, “Memory” on page 65).

The second SMP Expansion Module can be installed when more than four Xeon
MP processors, or two Xeon DP processors, are required. This also enables the
system to have up to 64 GB of RAM, using 2 GB DIMMs. The block diagram with
two SMP Expansion Modules is shown in Figure 1-5 on page 10.

Note: When Xeon DP processors are used, only two CPUs can be installed in
each SMP Expansion Module. The processors are installed in CPU positions
1 and 4. Positions 2 and 3 must hold air baffles to maintain proper air flow.

Chapter 1. Technical description 9

CEC 1

CEC 2

CPU 1 CPU 2 CPU 3 CPU 4

400 MHz

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

RXE Expansion

Port A (1 GBps)

3.2 GBps

3.2 GBps

3.2

GBps

100 MHz

PCI bridge

Ultra160

SCSI

Gigabit

Ethernet

3.2 GBps

Processor &

cache controller

3.2 GBps

Memory

controller

2 GBps

33 MHz66 MHz

Video

USB

Kbd/Ms

RSA

SMP Expansion
Ports (3.2GBps)

RXE

Expansion

Port B

(1 GBps)

Bus A66 MHz

cache controller

controller

2 GBps

64-bit

66 MHz

3.2 GBps

Processor &

3.2 GBps

Memory

PCI bridge

B-100

64-bit

100 MHz

3.2 GBps

3.2 GBps

3.2
GBps

100 MHz

64-bit

133 MHz

CPU 1CPU 2CPU 3CPU 4

400 MHz

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

D-133C-133

IBM XA-32
core chipset

Figure 1-5 xSeries 440 system block diagram — two SMP Expansion Modules

When two SMP Expansion Modules are installed, they are connected together
using two 3.2 GBps SMP Expansion Ports. The third scalability port is not used in
this single-node eight-way configuration.

The two PCI bridges in the XA-32 chipset provide support for 33, 66, 100, and
133 MHz devices using four PCI-X buses (labeled A-D in Figure 1-5). This is
discussed further in 1.8, “PCI subsystem” on page 23.

The PCI bridge also has a 1 GBps bi-directional Remote Expansion I/O port
(RXE port) for connectivity to the RXE-100 enclosure. This port is labeled “RXE
Expansion Port A” in both Figure 1-4 on page 8 (four-way) and Figure 1-5
(eight-way). The RXE-100 provides up to an additional 12 PCI-X slots. When the
second SMP Expansion Module is installed to form an eight-way system
(Figure 1-5), the second RXE port, labeled “RXE Expansion Port B”, connects to
the memory controller of the second SMP Expansion Module.

10 IBM ^ xSeries 440 Planning and Installation Guide

As of November 2002, you can connect two x440 servers together to form one
16-way complex. The two x440 nodes are connected together using all three
SMP Expansion Ports as shown in Figure 1-6.

CPU 1 CPU 2 CPU 3 CPU 4

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

x440 Node 1

CPU 1 CPU 2 CPU 3 CPU 4

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

Processor &

cache controller

Memory

controller

PCI bridge

Processor &

cache controller

Memory

controller

CEC 1 CEC 2

1
2
3

CEC 1

1
2
3

CEC 2

SMP Expansion
Ports (3.2GBps)

1
2
3

1
2
3

Processor &

cache controller

Memory

controller

PCI bridge

Processor &

cache controller

Memory

controller

CPU 1CPU 2CPU 3CPU 4

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

CPU 1CPU 2CPU 3CPU 4

32 MB

L4 cache

SDRAM

SDRAM

SDRAM

SDRAM

x440 Node 2

PCI bridge

Figure 1-6 16-way configuration (four SMP Expansion Modules)

The rear panel of the x440, indicating the location of the SMP Expansion Ports
and RXE Expansion Ports, is shown in Figure 1-7 on page 12.

PCI bridge

Chapter 1. Technical description 11

Figure 1-7 Rear panel of the xSeries 440 (one SMP Expansion Module installed)

1.4 Processors

The x440 models use one of the following processors:

򐂰 Xeon Processor MP (“Gallatin”)
򐂰 Xeon Processor MP (“Foster”)
򐂰 Xeon Processor DP (“Prestonia”)

The Xeon MP models of the x440 come with two or four processors installed in
the standard SMP Expansion Module. Up to four processors are supported in the
standard module and, with the addition of a second SMP Expansion Module, up
to eight processors can be installed in an x440.

The x440 entry-level systems can be ordered with either two Xeon DP
processors in a single SMP Expansion Module or with four Xeon DP processors
in two SMP Expansion Modules. There is no further upgrade beyond four Xeon
DP processors, other than replacing them with Xeon MP processors.

See 3.1.1, “Processors” on page 64 for further discussion about what you should
consider before implementing an x440 solution.

12 IBM ^ xSeries 440 Planning and Installation Guide

1.4.1 Intel Xeon Processor MP

The Xeon Processor MP (code named “Foster” or “Gallatin”) returns to the ZIF
socket design of the original Pentium processor, instead of the Slot 2 cartridge
design of the Pentium III Xeon processors. This smaller form factor means that
the x440 can have up to eight processors in a 4U node.

The Xeon MP processor has three levels of cache, all of which are on the
processor die:

򐂰 Level 3 cache is equivalent to L2 cache on the Pentium III Xeon. Foster

processors in the x440 models contain either 512 KB or 1 MB of L3 cache.
Gallatin processors contain either 1 MB or 2 MB or L3 cache.

򐂰 Level 2 cache is equivalent to L1 cache on the Pentium III Xeon and is 256 KB

in size. The L2 cache implements the Advanced Transfer Cache technology,
which means L2-to-processor transfers occur across a 256-bit bus in only one
clock cycle.

򐂰 A new level 1 cache, 12 KB in size, is “closest” to the processor and is used to

store micro-operations (that is, decoded executable machine instructions) and
serves those to the processor at rated speed. This additional level of cache
saves decode time on cache hits. There is an additional 8 KB for data related
to those instructions, for a total of 20KB.

The x440 also implements a Level 4 cache as described in 1.6, “IBM XceL4
Server Accelerator Cache” on page 19.

Intel has also introduced a number of features associated with its newly
announced NetBurst micro-architecture. These are available in the x440,
including:

򐂰 400 MHz frontside bus

The Pentium III Xeon processor has a 100 MHz frontside bus that equates to
a burst throughput of 800 MBps. With protocols such as TCP/IP, this has been
shown to be a bottleneck in high-throughput situations. The Xeon Processor
MP improves on this by using two 100 MHz clocks, out of phase with each
other by 90° and using both edges of each clock to transmit data. This is
shown in Figure 1-8.

100 MHz clock A

100 MHz clock B

Figure 1-8 Quad-pumped frontside bus

Chapter 1. Technical description 13

This increases the performance of the frontside bus without the difficulty of
high-speed clock signal integrity issues. The end result is an effective burst
throughput of 3.2 GBps, which can have a substantial impact, especially on
TCP/IP-based LAN traffic.

򐂰 Hyper-Threading

Hyper-Threading technology enables a single physical processor to execute
two separate code streams (threads) concurrently. To the operating system, a
processor with Hyper-Threading appears as two
which has its own architectural state - that is, its own data, segment, and
control registers and its own advanced programmable interrupt controller
(APIC).

For example, Figure 1-9 shows a 16-way x440 complex running Datacenter
Server with Hyper-Threading enabled.

logical processors, each of

Figure 1-9 Datacenter sees 32 processors when Hyper-Threading is enabled on a 16-way configuration

14 IBM ^ xSeries 440 Planning and Installation Guide

Each logical processor can be individually halted, interrupted, or directed to
execute a specified thread, independently from the other logical processor on
the chip. Unlike a traditional two-way SMP configuration that uses two
separate physical processors, the logical processors share the execution
resources of the processor core, which include the execution engine, the
caches, the system bus interface, and the firmware.

Note: Hyper-Threading is disabled by default on the x440. This is because
of a known bug in Windows 2000 Advanced Server. If Hyper-Threading is
enabled on an eight-way server, then the Windows 2000 Advanced Server
will trap (blue screen) during installation. This problem does not affect other
supported operating systems.

Hyper-Threading technology is designed to improve server performance by
exploiting the multi-threading capability of operating systems, such as
Windows .NET and Linux, and server applications, in such a way as to
increase the use of the on-chip execution resources available on these
processors.

Fewer or slower processors usually yield the best gains from
Hyper-Threading because there is a greater likelihood that the software can
spawn sufficient numbers of threads to keep both paths busy. The following
performance gains are likely:

– Two physical processors: 15-25% performance gain
– Four physical processors: 1-13% gain
– Eight physical processors: 0-5% gain

Tests have found that software often limits SMP scalability, but customers
should expect improved results as software matures. Best-case applications
today are:

– Databases
– Java
– Web servers
– E-mail

Note: Microsoft licensing of the Windows Server operating systems is by
number of processors (four-way for Server, eight-way for Advanced Server,
32-way for Datacenter Server). Therefore, the appearance of twice as many
logical processors can potentially affect the installation of the operating
system. See 1.12, “Operating system support” on page 28 for details.

For more information about Hyper-Threading, see:

http://www.intel.com/technology/hyperthread/

Chapter 1. Technical description 15

򐂰 Advanced Dynamic Execution

The Pentium III Xeon processor has a 10-stage pipeline. However, the large
number of transistors in each pipeline stage means that the processor is
limited to speeds under 1 GHz, due to latency in the pipeline.

The Xeon Processor MP has a 20-stage pipeline, which can hold up to 126
concurrent instructions in flight and up to 48 reads and 24 writes active in the
pipeline. The lower complexity of each stage also means that future clock
speed increases are possible.

It is important to note, however, that the longer pipeline means that it now
takes more clock cycles to execute the same instruction when compared to
the Pentium III Xeon.

Comparing the Xeon Processor MP with the Pentium III Xeon and current
operating systems (Windows 2000, Linux with 2.4 kernel), good rules of
thumb are:

– 1.5 GHz Xeon Processor MP/512 KB L3

MB L2 Xeon

– 1.6 GHz Xeon Processor MP/1 MB L3

MB L2 Xeon

The next generations of operating systems will likely improve performance of
the MP processor as they take advantage of the NetBurst architecture. These
include Windows .NET and the Linux 2.5/2.6 kernels.

For more information about the features of the Xeon Processor MP, go to:

http://www.intel.com/design/xeon/xeonmp/prodbref

1.4.2 Intel Xeon Processor DP

The Xeon DP is similar to the Xeon MP and is also based on the Intel NetBurst
micro-architecture. The Xeon DP was designed by Intel to be suitable only in
uniprocessor and two-way SMP processor systems. However, with the use of the
IBM XA-32 chipset, the x440 can have up to four Xeon DP processors installed.
The Xeon DP models of the x440 models use 2.4 GHz processors, part
37L3533.

The key differences between the processors are listed in Table 1-2.

Table 1-2 Differences between the Xeon DP and the Xeon MP

≈ 5-20% faster than 900 MHz 2

≈ 15-35% faster than 900 MHz 2

Feature Xeon Processor DP Xeon Processor MP

Maximum CPUs per SMP Expansion Module Two Four

Maximum CPUs per x440 node Four Eight

16 IBM ^ xSeries 440 Planning and Installation Guide