Hp COMPAQ PROLIANT 5000 Configuring for Peak Performance

TECHNOLOGY BRIEF

.

July 1997

Compaq Computer Corporation

CONTENTS

Introduction....................3

ProLiant 5000

Overview........................4

Processors...........................4

I/O ...................................... 4

Memory............................... 5

System Architecture ..............6

Storage ............................... 6

Power..................................7

ProLiant 5000

Architecture ................... 7

Performance Implications

of the EISA Bus ....................9

Configuration

Recommendations .............. 10

PCI Peripherals.................. 10

Example

Configurations.............. 13

Conclusion................... 15

Appendix ...................... 16

1679A/0697/ECG

.

Configuring the Compaq ProLiant 5000

.

Server for Peak Performance

.

EXECUTIVE SUMMARY

.

The development of the Intel Pentium Pro processor paved the way for the next technology leap in

.

capturing, processing, and managing business-critical data with industry-standard platforms.

.

Compaq has designed a high performance, high availability platform to exploit the full power of

.

the Pentium Pro and the latest in high-performance options. The Compaq ProLiant 5000 Server

.

provides midrange-class performance and reliability for running business-critical applications in

.

the data center, work group, or department.

.

The ProLiant 5000 is based on the Intel 82450GX chip set. The key to realizing peak

.

performance from any Pentium Pro server, including the ProLiant 5000, is optimal configuration

.

of the I/O subsystems. The ProLiant 5000 contains architectural changes and enhancements,

.

including use of the Intel 82450GX chip set, that are not available in the Compaq ProLiant 4500

.

Server. Because of these enhancements, achieving peak performance from the ProLiant 5000

.

system requires careful planning of initial system setup and configuration.

.

This brief explains important characteristics of Pentium Pro servers and provides technical

.

information for configuring the ProLiant 5000 to optimize overall system performance—initially

.

and later, as user needs expand.

.

Please direct comments regarding this communication to the ECG Technology Communications Group at this internet address:

.

tech_com@bangate.compaq.com

.

TECHNOLOGY BRIEF (cont.)

.

NOTICE

.

The information in this publication is subject to change without notice and is provided “AS IS”

.

679A/0697/ECG

.

WITHOUT WARRANTY OF ANY KIND. THE ENTIRE RISK ARISING OUT OF THE USE

.

OF THIS INFORMATION REMAINS WITH RECIPIENT. IN NO EVENT SHALL COMPAQ

.

BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, SPECIAL, PUNITIVE

.

OR OTHER DAMAGES WHATSOEVER (INCLUDING WITHOUT LIMITATION,

.

DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION OR LOSS OF

.

BUSINESS INFORMATION), EVEN IF COMPAQ HAS BEEN ADVISED OF THE

.

POSSIBILITY OF SUCH DAMAGES.

.

The limited warranties for Compaq products are exclusively set forth in the documentation

.

accompanying such products. Nothing herein should be construed as constituting a further or

.

additional warranty.

.

This publication does not constitute an endorsement of the product or products that were tested.

.

The configuration or configurations tested or described may or may not be the only available

.

solution. This test is not a determination or product quality or correctness, nor does it ensure

.

compliance with any federal state or local requirements.

.

Compaq, Compaq Insight Manager, PageMarq, ProLiant, ROMPaq, and QVision are registered

.

with the United States Patent and Trademark Office.

.

Netelligent, Systempro/XL, Net1, Vocalyst, PageMate, and SoftPaq are trademarks and/or

.

service marks of Compaq Computer Corporation.

.

Microsoft, Windows NT are trademarks and/or registered trademarks of Microsoft Corporation.

.

Other product names mentioned herein may be trademarks and/or registered trademarks of their

.

respective companies.

.

Configuring the Compaq ProLiant 5000 Server for Peak Performance

.

Fourth Edition (July 1997)

.

Document Number 679A/0697/ECG

.

2

TECHNOLOGY BRIEF (cont.)

.

INTRODUCTION

.

The Compaq ProLiant 5000 Server is a highly successful member of the ProLiant family of high

.

availability/high performance servers. It is the midrange alternative for the distributed enterprise,

.

679A/0697/ECG

.

delivering midrange-class performance and reliability. Its architecture is significantly different

.

from that of its predecessor, the Compaq ProLiant 4500 server, because the ProLiant 5000 is

.

designed to exploit performance advantages and unique characteristics of Pentium Pro processors,

.

plus the latest in high-performance options. With proper system configurations, the quad-

.

processor ProLiant 5000 will typically outperform an equivalent Pentium-based system by a

.

significant margin.

.

The key to realizing peak performance from any Pentium Pro server, including the ProLiant

.

5000, is optimal configuration of the I/O subsystems. Because of architectural changes and

.

enhancements incorporated into the ProLiant 5000, careful consideration must be given to the

.

initial system setup and configuration. Specifically:

.

• The Intel Pentium Pro processor is optimized for performance using Peripheral Component

.

3

Interconnect (PCI) peripherals. Therefore, the ProLiant 5000 achieves best performance with PCI peripherals. The ProLiant 5000 supports dual-peer PCI buses that provide aggregate I/O throughput capability as high as 267 MB/s. The dual-peer PCI buses increase configuration flexibility and allow higher levels of overall performance. However, attaining peak performance requires carefully balanced I/O loading across both PCI buses. This means careful planning of which expansion slot to use for each device in the initial configuration, as well as planning for future expansion.

• The ProLiant 5000 also includes an Extended Industry Standard Architecture (EISA) bus for

backward compatibility with installed EISA option boards. The EISA bus is bridged off one of the PCI buses. Since the ProLiant 5000 is optimized for PCI and the EISA bus is not the primary bus architecture in the system, achieving optimal overall system performance requires care in selecting and configuring any EISA peripheral to be used.

• Compaq uses the Intel 82450GX chip set in the ProLiant 5000. To ensure that its customers

do not experience the Intel-documented limitations of this chip set, Compaq has developed its own system board, and unique workarounds for many of these limitations are designed into Compaq hardware. For the rest, the only workaround is careful system configuration. Fortunately, most of the Intel chip set limitations are related to lower performance I/O adapters, so use of high-performance PCI peripherals and careful system configuration can minimize the impact of the Intel 82450GX chip set limitations on performance.

The configuration challenges cited above are not limitations specific to Compaq or the ProLiant 5000 platform. They are applicable to all 82450GX-based Pentium Pro systems. Compaq systems embody features specifically engineered to overcome most of these chip set issues.

The purpose of this brief is to explain some important characteristics of Pentium Pro servers and to provide technical information and recommendations for configuring the ProLiant 5000 to optimize overall system performance—initially and later as user needs expand.

TECHNOLOGY BRIEF (cont.)

.

679A/0697/ECG

PROLIANT 5000 OVERVIEW

.

Compaq continually strives to provide the highest levels of performance, scalability, and

.

reliability for our customers. For example, the Pentium-based ProLiant 4500 is so highly

.

optimized for EISA that it outperforms most competitors’ PCI-based Pentium servers in file and

.

print benchmarks, as well as in database benchmarks. Providing further significant performance

.

improvements for customers deploying business-critical database applications required a new

.

approach for the next generation Compaq server—the ProLiant 5000.

.

This section provides a brief overview of key subsystems in the ProLiant 5000.

.

Processors

.

The ProLiant 5000 uses the Intel 82450GX chip set, which supports up to four Pentium Pro

.

processors. The ProLiant 5000 accommodates up to four Pentium Pro processors, two per

.

processor board.

.

Customers may choose between these Pentium Pro processor options:

.

• 200-MHz processors, each with an integrated 16-KB L1 cache (8 KB for data and 8 KB for

.

4

instructions) and an integrated 256-KB L2 cache.

• 166-MHz processors, each with the same L1 cache and an integrated 512-KB L2 cache.

• 200-MHz processors, each with the same L1 cache and an integrated 512-KB L2 cache.

The L2 cache on each of these processors runs at processor speed and is protected by Error Correcting Code (ECC). The L2 caches are 4-way set associative for significant improvement in cache hit rate as compared to 2-way set associative caches in Pentium systems, including the ProLiant 4500. In typical applications, the higher hit rate allows the Pentium Pro L2 cache to outperform 2-way caches that are twice as large.

With its faster L2 cache, the 166-MHz processor provides a data throughput rate of more than 1 GB/s. That is 2.5 times the throughput rate of a 166-MHz Pentium processor.

To select a ProLiant 5000 Server with the right Pentium Pro processor option, two key factors must be understood:

• The demands of the computing environment.

• The relationship between processor speed and the size and speed of the cache.

In a uniprocessor ProLiant 5000 Server, the 200-MHz Pentium Pro processor with 256-KB L2 cache is the better choice, particularly in a file and print environment where I/O capability and the total memory available have the greatest effect on performance. However, for scaling, particularly in applications or database environments, cache size is more important than CPU clock speed. Therefore, the 166-MHz Pentium Pro processor with 512-KB L2 cache scales better for multiprocessor transaction processing. It follows, then, that a ProLiant 5000 with four 200MHz processors, each with a 512-KB L2 cache, provides the best performance in all environments.

I/O The ProLiant 5000 is optimized for PCI devices. There are several reasons for the shift from

EISA to PCI technology: First, I/O requirements for application servers are growing and will continue to grow into the

future. PCI technology will support growth well beyond the capabilities of EISA. Because of their high levels of processing power, initial Pentium Pro quad-processor servers will require I/O bandwidth that is close to the maximum bandwidth available with EISA.

TECHNOLOGY BRIEF (cont.)

.

Increased use of higher bandwidth peripherals will continue to raise I/O performance

.

requirements. At maximum bandwidth:

.

• Drive arrays could require up to 35 MB/s.

.

679A/0697/ECG

.

• FDDI requires up to 15 MB/s.

.

• Full duplex 100baseTX requires up to 20 MB/s.

.

• An ATM card requires up to 20 MB/s.

.

While the maximum bandwidth of an EISA bus is 33 MB/s, a single PCI bus provides a

.

maximum bandwidth of 133 MB/s.

.

Second, future needs will require increased I/O expansion. Slot expansion requirements will grow

.

with increasing processing power. While EISA is logically limited to 15 expansion slots, PCI can

.

be logically expanded to hundreds of slots or virtual slots by means of hierarchical bus expansion

.

designs using PCI-to-PCI bridges.

.

Third, PCI is a young and developing technology. Further performance enhancements are already

.

being specified, including a 64-bit implementation.

.

The optimal time for customers to transition to the Pentium Pro/PCI-based ProLiant 5000 will

.

vary, depending upon their computing needs and current technology. Customers using processor-

.

and I/O-intensive applications such as databases will benefit most by a prompt transition to the

.

ProLiant 5000.

.

Memory

.

Compaq believes that the memory subsystem significantly differentiates the ProLiant 5000 from

.

competitors’ Pentium Pro servers. In designing memory for the ProLiant 5000, Compaq is leading

.

the industry in the move from Single Inline Memory Module (SIMM) to Dual Inline Memory

.

Module (DIMM) technology.

.

The ProLiant 5000 contains 16 sockets for DIMMs. The system comes standard with 64 MB or

.

128 MB of RAM and is expandable to 4 GB of ECC-protected memory using 256 MB EDO

.

DIMMs.

.

For midrange-class servers, DIMMs offer these advantages over SIMMs:

.

• DIMMs offer a more flexible and cost effective upgrade path. A 4-way interleaved system

.

5

(which corresponds to a 256-bit-wide memory subsystem and is used in ProLiant 5000) provides maximum performance and maximum flexibility. Although using a 2-way interleaved system (which corresponds to a 128-bit-wide memory subsystem) will allow memory upgrade increments of 2 DIMMs or 4 SIMMs, it compromises performance.

• In a 4-way interleaved system, a memory upgrade requires either 8 SIMMs per bank or

4 DIMMs per bank. Upgrades using DIMMs, therefore, consume fewer memory slots and allow the highest total memory capacity.

• DIMMs consume significantly less power. SIMMs operate at 5 V. Because DIMMs operate

at 3.3 V, they consume approximately 50% less power and dissipate significantly less heat than SIMMs, thereby improving overall system reliability and availability.

• DIMMs provide larger memory capacities. If the ProLiant 5000 had been designed using

SIMM technology, the total system memory capacity would be limited to 1 GB. The switch to DIMMs increased total system memory capacity to 4 GB.

TECHNOLOGY BRIEF (cont.)

.

Some might argue that reuse of existing SIMMs for memory upgrades justifies using SIMMs in

.

Pentium Pro servers. For optimal performance, however, 82450GX-based Pentium Pro servers

.

must use DIMMs or SIMMs with an access time of 60 ns or faster. The access time of the

.

679A/0697/ECG

majority of SIMMs now in the field, however, is 70 ns or slower.

.

System Architecture

.

Figure 1 illustrates the ProLiant 5000 Server architecture. The system features a Gunning

.

Transceiver Logic processor bus (referred to in this paper as the host bus) that supports high-

.

speed, low voltage transfers. It is a 64-bit split-transaction bus operating at 66 MHz, with a

.

throughput capacity of 540 MB/s. Data integrity is ECC protected.

.

The ProLiant 5000 supports concurrent cache, memory, and I/O bus operation to optimize

.

performance for a wide variety of processor- and I/O-intensive applications. The 4-way set

.

associative L2 cache integrated into each Pentium Pro processor is accessed at full processor

.

speed through a separate, dedicated processor-to-cache bus. These 4-way L2 caches minimize

.

wait states by keeping data flowing to the processor. To fully exploit the power and scalability of

.

Pentium Pro in four-processor-capable servers, the ProLiant 5000 also has a 256-bit memory bus.

.

The ProLiant 5000 design includes dual-peer PCI buses, each operating at 33 MHz and providing

.

an aggregate bandwidth of 267 MB/s. The system also includes an EISA bus that provides a

.

maximum bandwidth of 33 MB/s.

.

Storage

.

The ProLiant 5000 contains three removable media drive bays. A 3.5-inch, 1.44-MB diskette

.

drive and a CD-ROM drive are standard. One half-height removable storage device, such as a

.

tape drive, can be installed in the third bay. The server features hot-pluggable hard drive bays. In

.

tower models using the 9.1-GB drives available today, four hot-pluggable drives provide

.

maximum internal storage of 36.4 GB. In rack models, five hot-pluggable drives provide

.

maximum internal storage of 45.5 GB. As larger capacity drives become available, maximum

.

internal storage will increase.

.

Optional mass storage devices can be connected to the ProLiant 5000 via the External Fast-Wide

.

SCSI-2 port on the back of the unit or the Fast-Wide SCSI-2 port on the optional Fast-Wide

.

SCSI-2 Array Controllers. When combined with the dual bus Compaq ProLiant Storage

.

6

Aggregate PCI Throughput

(267 MB/s)

64-Bit Host Bus

(540 MB/s + ECC)

PCI Bus (133 MB/s)

EISA Bus (33 MB/s)

256-Bit Memory Bus

Figure 1. Compaq ProLiant 5000 Server Architecture

TECHNOLOGY BRIEF (cont.)

.

System/F, a complete ProLiant 5000 system provides a total storage capacity of more than 750

.

GB.

.

Power

.

679A/0697/ECG

.

The ProLiant 5000 contains a 490/540-W power supply that provides nominal output of 3.3 V,

.

5 V, and 12 V. The trend today is toward use of lower-voltage components. System memory in the

.

ProLiant 5000, for example, operates at 3.3 V.

.

The standard ProLiant 5000 power supply has a dual rating: 490 W at 100/115 V and 540 W at

.

208/230 V. When the system is fully configured with four processors, four internal drives, and

.

more than six I/O boards, Compaq strongly recommends operating the ProLiant 5000 at

.

208/230 V.

.

Intel Pentium Pro processors have precise voltage requirements; therefore, Pentium Pro-based

.

systems need to provide dedicated regulators for each CPU (and for the processor support chips).

.

If a regulator fails, the processor fails, and the system fails.

.

A unique feature of the ProLiant 5000 server is Compaq Processor Power Modules. These

.

modules determine the precise voltage requirement of processors and the support chip set. Then

.

they ensure that the proper voltage is delivered to each component. The I/O system board comes

.

standard with one of these modules but will accommodate two for redundancy. Each ProLiant

.

5000 server comes standard with one Processor Power Module per processor. An optional N+1

.

redundancy scheme supports up to three Processor Power Modules for every two processors and

.

provides a highly available solution.

.

PROLIANT 5000 ARCHITECTURE

.

For all the reasons mentioned previously, Compaq chose to optimize the ProLiant 5000 for PCI.

.

There are two primary bus expansion schemes for PCI systems: PCI-to-PCI bridge architecture

.

and dual-peer PCI bus architecture.

.

Figure 2 is a block diagram showing PCI-to-PCI bridge architecture. This scheme provides extra

.

slots for additional PCI devices and is the lower cost implementation. However, it has two

.

significant limitations for high-end systems:

.

• It supports only one PCI bus directly and independently linked to the host bus.

.

• It provides no additional system level bus bandwidth.

.

Because of these limitations, optimizing the ProLiant 5000 I/O system for PCI meant using the

.

dual-peer PCI bridge architecture instead. Dual-peer PCI bus architecture significantly increases

.

the configuration flexibility of the system, allowing optimization levels that are not possible with

.

PCI-to-PCI bridge architecture.

.

Figure 3 is a block diagram showing the dual-peer PCI bus architecture Compaq chose for the

.

ProLiant 5000. This expansion scheme supports two PCI buses directly and independently linked

.

to the high speed Pentium Pro processor bus via dual Host-to-PCI bridges. The two PCI buses

.

provide aggregate I/O bandwidth totaling 267 MB/s and extra slots for PCI devices. The ProLiant

.

5000 also includes an EISA bus to maintain backward compatibility with installed EISA and ISA

.

peripherals.

.

7

TECHNOLOGY BRIEF (cont.)

.

679A/0697/ECG

.

Pentium Pro

.

8

Pentium Pro

Processors

Slots

Figure 3. Block diagram of the dual-peer PCI bridge architecture used in the Compaq ProLiant 5000 Server

Using high-bandwidth burst for all data transfers, PCI achieves a 133 MB/s peak throughput for reads and writes. Arbitration time in the ProLiant 5000 has been minimized so that more time can be spent effectively moving data. The high-performance PCI bridges support full concurrency of the host bus, the PCI bus, and the expansion bus in simultaneous operation. PCI also provides fast access. It is possible to access a PCI device in as little as 60 ns using a 33-MHz PCI clock.

Figure 4 depicts the I/O buses (primary and secondary PCI buses and the EISA bus) and slots in the ProLiant 5000. The primary PCI bus supports the following:

• PCI on dedicated slots 5, 6, 7, and 8.

• EISA bridge.

Processors Memory

Figure 2. Block diagram showing PCI-to-PCI bridge architecture

Bridge

133 MB/s 133 MB/s

Secondary PCI Bus

PCI-to-EISA

Host Bus

Host-to-PCI

Bridge

PCI-to-PCI

Bridge

Host Bus

540 MB/s

Host-

to-PCI

PCI Bus

133 MB/s

PCI Bus

Bridge

Slots

133 MB/s

Primary PCI Bus

Slots

Memory

33 MB/s

EISA Bus

Slots

TECHNOLOGY BRIEF (cont.)

.

679A/0697/ECG

• EISA on dedicated slot 1 and on shared slots 2 and 3.

.

• Embedded EISA peripherals including video, timers, keyboard, and mouse.

.

The secondary bus supports the following:

.

• PCI on dedicated slot 4 and on shared slots 2 and 3.

.

• An embedded Fast-Wide SCSI controller.

.

Pentium Pro

.

Processors

.

Performance Implications of the EISA Bus

.

Limitations of the Intel 82450GX chip set require that an EISA bus in a Pentium Pro server be

.

bridged off a PCI bus, which precludes directly buffering EISA to the host bus or to memory.

.

Essentially, EISA is one stage removed from the host bus. This can affect system performance in

.

two ways:

.

• Because EISA is bridged off a PCI bus in Pentium Pro servers, EISA peripherals consume

.

9

PCI bandwidth. In fact, EISA peripherals can consume a disproportionate amount of PCI bandwidth due to the indirect buffering to memory. A heavily loaded EISA peripheral can

significantly limit the performance of the entire system. Low- to medium-bandwidth embedded EISA peripherals, however, do not significantly affect PCI performance.

• For all 82450GX-based Pentium Pro systems, the result of the architectural focus on PCI is

lower EISA performance in Pentium Pro systems than in Pentium systems optimized for EISA (including the ProLiant 4500). Because Pentium Pro servers are optimized for PCI performance and EISA is bridged off PCI, low-level EISA performance in a Pentium Pro server using high-bandwidth EISA peripherals will be one half or less than in a system like the ProLiant 4500 that is optimized for EISA.

Primary PCI Bus

Secondary

PCI Bus

Memory

Controller

8 6 5

7

Figure 4. ProLiant 5000 I/O architecture

Memory

DIMMs

Bridge

4

EISA

3

Shared

3

Slots

2

Shared

2

Slots

Embedded

EISA

Peripherals

1

Embedded SCSI

Fast-Wide Controller