HP COMPAQ PROLIANT 8000 SERVER User Manual

June 2000

Compaq Computer Corporation

ISSD Technology Communications

ONTENTS

C

Introduction....................... 3

High-Performance

Technologies ..................... 3

ProLiant 8000 Architecture......4

Pentium III Xeon

Processor Technology...........10

High-Availability

Technologies .................... 11

PCI Hot Plug Technology ......11

Redundant Array

Controller Technology........... 1 2

Redundant Network Interface

Controller Technology........... 1 2

Redundant Hot-Plug Fans .....13

Redundant Processor

Power Modules.....................15

Redundant Hot-Plug

Power Supplie s ....................15

Hot-Plug Hard Disk Drives .....18

Storage

Technologies .................... 18

Hard Drive Technology..........18

Array Controller Technology..18

Management

Technologies .................... 19

Remote-Flash

Redundant ROM ................... 19

Auto-Default

ROM Configura tion ...............20

Integrated Management

Display................................. 21

Integrated Remote Console... 21 Remote Insigh t

Board/PCI Optio n .................21

Serviceability

Features ............................22

Conclusion........................ 22

T

ECHNOLOGY

.

Compaq ProLiant 8000 Server Technology

.

This technology brief describes the high-performance, high-availability, storage, and

.

management technologies built into the Compaq ProLiant 8000 server. The high-

.

performance technologies include a new 8-way system architecture driven by the

.

Profusion chipset codeveloped by Compaq, Intel, and Corollary, as well as Intel’s

.

Pentium III Xeon processors. In addition, new disk drive technology and a new extended

.

SCSI controller with redundancy capability allows the ProLiant 8000 server to deliver

.

maximum highly available internal storage capacity.

.

The high-availability technologies include redundant internal array controllers; PCI Hot

.

Plug technology; redundant hot-plug power supplies, drives, and fans; and redundant

.

processor power modules. The ProLiant 8000 design also contains easy-access

.

components that reduce downtime for service or upgrades. Key management

.

technologies, such as the Integrated Remote Console, Integrated Management Display,

.

Remote-flash Redundant ROM, and auto-default ROM configuration, further enhance

.

availability.

.

The intended audience for this paper is engineers and system administrators familiar

.

with existing Compaq technology and servers. For those less familiar with Compaq

.

technology, please see the related technology briefs referenced in this document. For

.

more information about the ProLiant 8000 server, see the complete list of features at

.

http://www.compaq.com/products/servers/ProLiant8000/quickspecs.html.

.

Please direct comments regarding this communication to the ISSD Technology Communications Group at this Internet

.

address: TechCom@compaq.com

.

B

RIEF

TC000603TB

1

ECHNOLOGY BRIEF

T

.

OTICE

N

.

The information in this publication is subject to change without notice and is provided “AS IS”

.

WITHOUT WARRANTY OF ANY KIND. THE ENTIRE RISK ARISING OUT OF T HE USE

.

OF THIS INFORMATION REMAINS WITH RECIP IENT. IN NO EVENT SHALL COMPAQ

.

BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, SPECIAL, PUNITIVE

.

OR OTHER DAMAGES WHATSOEVER (INCLUDING WITHOUT LIMITATION,

.

DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION OR LOSS OF

.

BUSINESS INFORMATION), EVEN IF COMPAQ HAS BEEN ADVISED OF THE

.

POSSIBILITY OF SUCH DAMAGES.

.

The limited warranties for Compaq products are exclusively set forth in the documentation

.

accompanying such products. Nothing herein should be construed as constituting a further or

.

additional warranty.

.

This publication does not constitute an endorsement of the product or products that were tested.

.

The configuration or configurations tested or described may or may not be the only available

.

solution. This test is not a determination of product quality or correctness, nor does it ensure

.

compliance with any federal state or local requirements.

.

Compaq, Contura, Deskpro, Fastart, Compaq Insight Manager, LTE, PageMarq, Systempro,

.

Systempro/LT, ProLiant, TwinTray, ROMPaq, LicensePaq, QVision, SLT, ProLinea, SmartStart,

.

NetFlex, DirectPlus, QuickFind, RemotePaq, BackPaq, TechPaq, SpeedPaq, QuickBack, PaqFax,

.

Presario, SilentCool, CompaqCare (design), Aero, SmartStation, MiniStation, and PaqRap,

.

ProSignia, Concerto, Vocalyst, and MediaPilot are registered with the United States Patent and

.

Trademark Office.

.

Change is Good, Compaq Capital, Colinq, Armada, SmartQ, Counselor, CarePaq, Netelligent,

.

Smart Uplink, Extended Repeater Architecture, Scalable Clock Architecture, QuickChoice,

.

Systempro/XL, Net1, LTE Elite, PageMate, SoftPaq, FirstPaq, SolutionPaq, EasyPoint, EZ Help,

.

MaxLight, MultiLock, QuickBlank, QuickLock, UltraView, Innovate logo, and Compaq PC Card

.

Solution logo are trademarks and/or service marks of Compaq Computer Corporation.

.

Microsoft, Windows, Windows NT, Windows NT Advanced Server, SQL Server for Windows NT

.

are trademarks and/or registered trademarks of Microsoft Corporation.

.

NetWare and Novell are registered trademarks and IntranetWare, NDS, and Novell Directory

.

Services are trademarks of Novell, Inc.

.

Pentium is a registered trademark of Intel Corporation.

.

Other product names mentioned herein may be trademarks and/or registered trademarks of their

.

respective companies.

.

Compaq ProLiant 8000 Server Technology

.

Second Edition (June 2000)

.

Document Number TC000603TB

.

(cont.)

TC000603TB

2

PCI – peripheral component interconnect

SCSI – small computer system interface

ROM – read only memory

ECHNOLOGY BRIEF

T

.

NTRODUCTION

I

.

In today’s business environment, enterprise customers require powerful servers with fault-tolerant

.

features to keep their businesses r unning 24 hours a day, 7 days a week. They also need servers

.

that are easy to manage and will scale to meet future requirements.

.

Providing breakthrough performance, t he ProLiant 8000 server offers outstanding scalability from

.

one to eight Intel Pentium III Xeon processors, up to 16 GB of synchronous dynamic random

.

access memory (SDRAM), and 11 PCI slots. The ProLiant 8000 server delivers massive internal

.

storage, with capacity for more than 380 GB of internal Wide Ultra-3 SCSI hot-plug storage. A

.

new extended SCSI array controller provides cable-free connections to the drive cages and allows

.

all three drive cages to be configured as a single contiguous array or as separate arrays. High-

.

availability features, including optional redundant array controllers, push-button PCI Hot Plug, hot-

.

plug redundant fans, redundant hot-plug power supplies, Compaq Re mot e-flash Redundant RO M,

.

and redundant processor power modules (PPMs), keep the ProLiant 8000 server up and running in

.

the most demanding 7x2 4 environments.

.

The ProLiant 8000 server provides all of the high-availability and manageability features of the

.

ProLiant 7000 server with added performance and scalability—all in a similar price class. In

.

addition, Compaq delivers on its commitment to superior investment protection by offering a

.

ProLiant 7000 in-chassis upgrade to 8-way architecture. Customers can leverage their ProLiant

.

7000 investment by keeping the chassis and serial number and replacing only select components for

.

the upgrade. For more information, see the 8-way upgrade website at

.

http://www.compaq.com/upgrade/8-way.

.

This technology brief explains the features of the ProLiant 8000 server and describes the

.

management and storage technologies that enhance performance and availability. Many of these

.

technologies are covered in more detail in separate technology briefs, which are referenced in this

.

document.

.

IGH-PERFORMANCE TECHNOLOGIES

H

.

The Compaq ProLiant 8000 server is a powerful, industry-leading technology solution for

.

distributed enterprise computing. A revolutionary new 8-way system architecture allows the

.

ProLiant 8000 server to deliver unsurpassed performance. This new architecture is the first

.

implementation in the industry of the Profusion chipset and features Pentium III Xeon processors.

.

Compaq servers using this architecture provide new levels of performance without requiring

.

modifications or special releases of operating systems (OSs) or applications. The Compaq 8-way

.

architecture provides nearly linear scalab ility for up to eight processors running under Microsoft

.

Windows NT 4.0, Microsoft Windows 2000, Novell NetWare, or SCO UnixWare. The 8-way

.

architecture delivers superior performance and price:performance for processor-intensive

.

applications such as Microsoft Terminal Server and Oracle databases; for memory-intensive

.

applications such as Lotus Notes, Microsoft Exchange, and SAP solutions; and for input/output

.

(I/O) intensive applications such as Microsoft SQL Server.

.

(cont.)

TC000603TB

3

ECHNOLOGY BRIEF

T

.

ProLiant 8000 Architecture

.

The system architecture is the key to the high level of performance offered by the ProLiant 8000

.

server. Compaq worked closely with Intel and Corollary for three years to perfect the 8-way

.

symmetric multiprocessing (SMP) architecture so that the processors, memory, and I/O subsystem

.

work in harmony to deliver breakthrough levels of performance. Figure 1 illustrates the

.

architecture used in the ProLiant 8000 server. The essential features of the architecture include:

.

• Profusion five-point crossbar switch

.

• Dual 100-MHz processor buses

.

• Dual 100-MHz memory buses, each with its own memory controller

.

• Dedicated 100-MHz I/O bus

.

• Support for up to eight Pentium III Xeon processors

.

• Support for up to 16 GB of two-way, cache-line interleaved SDRAM

.

• Dual cache accelerators

.

• I/O filter

.

• Three Compaq designed host-to-PCI bridges

.

Pentium III

.

Xeon

.

Figure 1: Block diagram of the ProLiant 8000 system architecture.

.

Pentium III

Xeon

800 MB/s GTL+ left bus 800 MB/s GTL+ right bus

32-bit, 33-MHz PCI

133 MB/s

(cont.)

Pentium III

Xeon

Left Cache

Accelerator

800 MB/s GTL I/O bus

Compaq

Host-to-PCI

Bridge

2-Way Cache-Line Interleaved

SDRAM

Pentium III

Xeon

64-bit, 33-MHz PCI

266 MB/s

SDRAM

800 MB/s

Profusion

Compaq

Host-to-PCI

Bridge

Pentium III

Xeon

Right Cache

Accelerator

Pentium III

Xeon

Compaq

Host-to-PCI

Bridge

Pentium III

Xeon

64-bit, 66-MHz PCI

533 MB/s

Pentium III

Xeon

TC000603TB

4

ECHNOLOGY BRIEF

T

.

Profusion Chipset

.

At the heart of the 8-way architecture is the Profusion chipset. The chipset uses a five-point

.

crossbar switch (Figure 2) to connect processor buses, memory ports, and the I/O bus. The

.

crossbar switch contains static random access memory (SRAM) with ten ports—five read and five

.

write—that appear as five bidirectional ports, one for each of the processor, memory, and I/O

.

buses. The switch may connect two ports directly or may store data from the originating bus in the

.

SRAM before it is transferred to the destination bus. This nonblocking design allows simultaneous

.

read and write accesses from all five buses, which results in better system performance.

.

Figure 2: Block diagram of the Profusion crossbar switch.

.

The Profusion crossbar switch provides direct paths from each memory bus to each processor bus

.

and to the I/O bus. Depending on the status of the system, the direct paths can be used to bypass

.

the SRAM, thus reducing latency and improving performance. The direct paths are used only to

.

read data. To improve processor utilization and performance, write data is always posted to the

.

SRAM and written to main memory later.

.

The Profusion crossbar switch consists of two physical chips—the memory address controller

.

(MAC) and the data interface buffer (DIB). This functional partitioning of the application-specific

.

integrated circuit (ASIC), as shown in Figure 3, improves system performance. For every

.

transaction from a processor or an I/O controller, the address and command portions are routed

.

through the MAC and the data is routed through the DIB. The MAC manages the external cache

.

accelerators and tracks the information stored in the DIB. The DIB allows simultaneous data

.

transfer on all five ports, has 64-cache-line buffers, and uses error-correcting code to maintain data

.

integrity. The cache-line buffers can be used by any transaction for any device on any bus; and

.

since there are no dedicated queues between buses, the efficiency of the buffers is high. This

.

improves system performance.

.

Left Processor Bus

(cont.)

Right Memory PortLeft Memory Port

Memory Interface Memory Interface

Processor Interface

10-Port SRAM

I/O Interface

I/O Bus

Processor Interface

Right Processor Bus

TC000603TB

5

ECHNOLOGY BRIEF

T

.

Processor bus

I/O bus

Figure 3: ASIC partitioning in the Profusion chipset.

Processor and I/O Buses

The ProLiant 8000 server includes two 64-bit processor buses and a dedicated 64-bit I/O bus. All three are 100-MHz advanced Gunning transceiver logic plus (AGTL+) buses with a theoretical maximum throughput of 800 MB/s. AGTL+ buses facilitate higher clock speeds without significantly reducing the bus length or number of electrical loads. Each 100-MHz AGTL+ bus can support a maximum of five loads per bus. This allows four processors and one connection to the memory controller on each processor bus. The use of two processor buses enables the ProLiant 8000 server to support up to eight Pentium III Xeon processors.

The Profusion chipset joins the two processor buses, the I/O bus, and the two memory ports. The otherwise independent processor and I/O buses are joined by a logical connection that is made only when required to transfer data. Each of the three AGTL+ buses has independent access to the two memory ports. This architecture prevents I/O traffic from consuming bandwidth on the processor bus. In addition, the use of 100-MHz buses and five independent paths allows the crossbar switch to deliver an aggregate instantaneous peak throughp ut of 4 GB/s—unprecedented high per formance for customers.

Memory Subsystem

The ProLiant 8000 memory subsystem includes dual 100-MHz buses, each with its own memory controller. The use of two memory buses increases memory bandwidth, reduces access conflicts, and increases the quantity of memory supported. Memory attached to each of the buses is cacheline interleaved, which means the buses share a common address range. One memory bus responds to even-numbered cache lines, and the other bus responds to odd-numbered cache lines. This configuration allows simultaneous use of both memory buses, which theoretically doubles throughput. It is especially advantageous for applicatio ns that access memory randomly. In random accesses, roughly half the requests at any one time are even-numbered lines, while the other half are odd-numbered lines.

(cont.)

address

data

Cache

Accelerator

Memory Access

Controller (MAC)

A

Memory

Array

DD

Data Inter face

Buffer (DIB)

Cache

Accelerator

A

Memory

Array

address

data

address

Processor bus

TC000603TB

6

ECHNOLOGY BRIEF

T

.

The memory subsystem uses uniform memory access, which reduces latency and gives all

.

processors equal access times to either memory bus. In systems using nonuniform memory access

.

architectures, a processor has quick access to one memory bus but incurs a lag time (or latency)

.

when accessing a second memory bus.

.

The ProLiant 8000 server supports up to 16 GB of error checking and correcting SDRAM that

.

corrects all single-bit errors and detects double-bit errors. Memory is divided into eight banks,

.

each consisting of two dual inline memory modules.

.

Although the Profusion chipset supports up to 32 GB of memory, industry-standard OSs provide

.

only minimal support and scalability for this memory capacity, and enhanced support will not be

.

available for an extended period. Also, in discussions with customers, Compaq learned that very

.

few server implementations are fully configured with memory. With this in mind, Compaq used the

.

internal server space to provide additional drive capacity in the ProLiant 8000 server. As customer

.

requirements and OS capabilities increase in the future, Compaq will continue to modify servers to

.

match these requirements.

.

Cache Accelerators

.

One of the main challenges of designing an efficient SMP architecture is maintaining cache

.

coherency. To allow faster access to memory, most processors write data to cache memory rather

.

than to main memory. When a processor writes data to its cache, the cache has a newer copy of the

.

data than main memory. Cache coherency ensures that the most recent copy of the data is read by

.

any device that requests it. The cache coherency protocol essentially makes the cache look like

.

main memory. Cache coherency is critical for the proper operation of an SMP architecture, and the

.

performance and scalability of the architecture is affected by how efficiently it maintains cache

.

coherency.

.

With multiple processor buses and a separate I/O bus, it is extremely challenging to maintain cache

.

coherency in the 8-way architecture. Each memory access must look at, or snoop, the caches on its

.

local processor bus and snoop all caches on the remote processor bus and the I/O bus. The amount

.

of snoop traffic can significantly impact the scalability of the system.

.

The ProLiant 8000 architecture uses cache accelerators to minimize snoop traffic to the remote

.

processor bus and I/O bus. The cache accelerators store the address and state of the data for all

.

caches on their respective buses. The Profusion crossbar switch uses this information to determine

.

whether to snoop the remote processor and I/O buses. Depending on how often a software

.

application shares data, the reduction in snoop traffic can significantly improve overall system

.

performance and scalability.

.

I/O Filter

.

The ProLiant 8000 server also includes three Compaq host-to-PCI bridges with prefetch buffers, so

.

they act as caching bridges. The Profusion chipset contains a built-in I/O filter for the caching

.

bridges on the I/O bus. The I/O filter enhances performance by reducing snoop traffic on the I/O

.

bus. This I/O filter is designed to work with all three of the Compaq host-to-PCI bridges. When a

.

processor requests a cache line with the intent to modify it, the MAC performs a lookup into the

.

I/O filter to determine if that line resides in one of the caching bridges. If it does reside there, the

.

MAC initiates a transaction on the I/O bus to invalidate that cache line. If the cache line is not

.

present in one of the bridges, then no transaction is run on the bus.

.

(cont.)

TC000603TB

7

HP COMPAQ PROLIANT 8000 SERVER User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

ProLiant 8000 Architecture