Hp COMPAQ PROLIANT 8500, COMPAQ PROLIANT 6400R Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Parallel Database Cluster Model
PDC/O1000 for Oracle8i and
Windows 2000
Administrator Guide
Second Edition (June 2001)
Part Number 225083-002
Compaq Computer Corporation
Notice
© 2001 Compaq Computer Corporation
Compaq, the Compaq logo, Compaq Insight Manager, SmartStart, Rompaq, ProLiant, and
StorageWorks Registered in U.S. Patent and Trademark Office. ActiveAnswers is a trademark of
Compaq Information Technologies Group, L.P. in the United States and other countries.
Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States
and other countries.
All other product names mentioned herein may be trademarks of their respective companies.
Compaq shall not be liable for technical or editorial errors or omissions contained herein. The
information in this document is provided “as is” without warranty of any kind and is subject to change
without notice. The warranties for Compaq products are set forth in the express limited warranty
statements accompanying such products. Nothing herein should be construed as constituting an
additional warranty.
Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000
Second Edition (June 2001)
Part Number 225083-002
Contents
About This Guide
Purpose .................................................................................................................... xiii
Audience.................................................................................................................. xiii
Scope ........................................................................................................................xiv
Referenced Manuals ..................................................................................................xv
Supplemental Documents .........................................................................................xvi
Text Conventions.....................................................................................................xvii
Symbols in Text.......................................................................................................xvii
Symbols on Equipment.......................................................................................... xviii
Rack Stability ...........................................................................................................xix
Getting Help .............................................................................................................xix
Compaq Technical Support ...............................................................................xix
Compaq Website.................................................................................................xx
Compaq Authorized Reseller..............................................................................xx
Chapter 1
Clustering Overview
Clusters Defined ...................................................................................................... 1-2
Availability .............................................................................................................. 1-3
Scalability ................................................................................................................ 1-3
Compaq Parallel Database Cluster Overview.......................................................... 1-4
iv Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Chapter 2
Architecture
Compaq ProLiant Servers ........................................................................................ 2-2
High-Availability Features of ProLiant Servers ............................................... 2-3
Shared Storage Components.................................................................................... 2-3
RA4000 Array................................................................................................... 2-4
RA4100 Array................................................................................................... 2-4
RA4000 Array Controller ................................................................................. 2-5
Fibre Channel SAN Switch............................................................................... 2-5
Storage Hub ...................................................................................................... 2-6
FC-AL Switch................................................................................................... 2-7
Fibre Host Adapters.......................................................................................... 2-7
Gigabit Interface Converter-Shortwave Modules ............................................. 2-8
Fibre Channel Cables........................................................................................ 2-8
Availability Features of the Shared Storage Components................................. 2-9
I/O Path Configurations in a Non-Redundant Fibre Channel Fabric ....................... 2-9
Overview of Fibre Channel Fabric SAN Topology .......................................... 2-9
Non-Redundant Fibre Channel Fabric............................................................ 2-10
Using Multiple Non-Redundant Fibre Channel Fabrics ................................. 2-11
Maximum Distances Between Cluster Nodes and Shared Storage
Subsystem Components in a Non-Redundant Fibre Channel Fabric.............. 2-13
I/O Data Paths for a Non-Redundant Fibre Channel Fabric ...........................2-13
I/O Path Configurations in a Non-Redundant Fibre Channel Arbitrated Loop...... 2-16
Overview of FC-AL SAN Topology .............................................................. 2-16
Non-Redundant Fibre Channel Arbitrated Loop............................................. 2-16
Using Multiple Non-Redundant Fibre Channel Arbitrated Loops.................. 2-18
Maximum Distances Between Cluster Nodes and Shared Storage
Subsystem Components in a Non-Redundant FC-AL .................................... 2-20
I/O Data Paths for a Non-Redundant FC-AL.................................................. 2-20
Cluster Interconnect Options.................................................................................. 2-23
Ethernet Cluster Interconnect..........................................................................2-23
Local Area Network........................................................................................ 2-29
Chapter 3
Cluster Software Components
Overview of the Cluster Software............................................................................ 3-1
Microsoft Windows 2000 Advanced Server............................................................ 3-2
Compaq Software .................................................................................................... 3-2
Compaq SmartStart and Support Software....................................................... 3-2
Compaq System Configuration Utility ............................................................. 3-3
Compaq Array Configuration Utility................................................................ 3-3
Fibre Channel Fault Isolation Utility................................................................ 3-3
Compaq Insight Manager ................................................................................. 3-4
Compaq Insight Manager XE ........................................................................... 3-4
Compaq Options ROMPaq............................................................................... 3-4
Compaq Operating System Dependent Modules.............................................. 3-5
Oracle Software ....................................................................................................... 3-5
Oracle8i Server Enterprise Edition................................................................... 3-5
Oracle8i Server................................................................................................. 3-6
Oracle8i Parallel Server Option........................................................................ 3-6
Oracle8i Enterprise Manager............................................................................ 3-6
Oracle8i Certification ....................................................................................... 3-7
Application Failover and Reconnection Software ................................................... 3-7
Chapter 4
Cluster Planning
Site Planning............................................................................................................ 4-2
Capacity Planning for Cluster Hardware ................................................................. 4-3
Compaq ProLiant Servers................................................................................. 4-3
Planning Shared Storage Components for Non-Redundant Fibre Channel
Fabrics .............................................................................................................. 4-4
Planning Shared Storage Components for Non-Redundant Fibre Channel
Arbitrated Loops............................................................................................... 4-5
Planning Cluster Interconnect and Client LAN Components........................... 4-6
Planning Cluster Configurations for Non-Redundant Fibre Channel Fabrics ......... 4-7
Sample Midsize Cluster with One Non-Redundant Fibre Channel Fabric....... 4-8
Sample Large Cluster with One Non-Redundant Fibre Channel Fabric........... 4-9
Planning Cluster Configurations for Non-Redundant Fibre Channel Arbitrated
Loops ..................................................................................................................... 4-10
Sample Midsize Cluster with One Non-Redundant FC-AL ........................... 4-10
Sample Large Cluster with One Non-Redundant FC-AL............................... 4-11
RAID Planning ...................................................................................................... 4-12
Supported RAID Levels ................................................................................. 4-14
Raw Data Storage and Database Size............................................................. 4-15
Selecting the Appropriate RAID Levels......................................................... 4-16
Planning the Grouping of Physical Disk Storage Space ........................................ 4-17
Contents v
vi Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Cluster Planning
continued
Disk Drive Planning............................................................................................... 4-18
Nonshared Disk Drives................................................................................... 4-18
Shared Disk Drives ......................................................................................... 4-18
Network Planning .................................................................................................. 4-19
Windows 2000 Advanced Server Hosts Files for an Ethernet Cluster
Interconnect ....................................................................................................4-19
Client LAN ..................................................................................................... 4-20
Chapter 5
Installation and Configuration
Installation Overview............................................................................................... 5-2
Installing the Hardware............................................................................................ 5-3
Setting Up the Nodes ........................................................................................ 5-3
Installing the Fibre Host Adapters .................................................................... 5-4
Installing GBIC-SW Modules for the Fibre Host Adapters.............................. 5-4
Cabling the Fibre Host Adapters to the Storage Hub, FC-AL Switch, or
Fibre Channel SAN Switch............................................................................... 5-5
Installing the Cluster Interconnect Adapters..................................................... 5-6
Installing the Client LAN Adapters .................................................................. 5-7
Setting Up the RA4000/RA4100 Arrays........................................................... 5-8
Installing GBIC-SW Modules for the RA4000 Array Controller ..................... 5-9
Cabling the Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch to
the RA4000 Array Controllers.......................................................................... 5-9
Installing Additional Fibre Channel Fabrics or FC-ALs................................. 5-10
Cabling the Ethernet Cluster Interconnect...................................................... 5-11
Cabling the Client LAN.................................................................................. 5-16
Installing Operating System Software and Configuring the RA4000/RA4100
Arrays..................................................................................................................... 5-17
Guidelines for Clusters ...................................................................................5-17
Automated Installation Using SmartStart ....................................................... 5-18
Installing Compaq OSDs ....................................................................................... 5-22
Verifying Cluster Communications ................................................................ 5-22
Mounting Remote Drives and Verifying Administrator Privileges ................ 5-23
Installing the Ethernet OSDs .......................................................................... 5-24
Installing Oracle Software ..................................................................................... 5-36
Configuring Oracle Software ................................................................................. 5-36
Installing Object Link Manager ............................................................................. 5-36
Additional Notes on Configuring Oracle Software......................................... 5-37
Installation and Configuration
continued
Verifying the Hardware and Software Installation ................................................ 5-38
Cluster Communications ................................................................................ 5-38
Access to Shared Storage from All Nodes...................................................... 5-38
OSDs .............................................................................................................. 5-38
Other Verification Tasks ................................................................................ 5-39
Power Distribution and Power Sequencing Guidelines ......................................... 5-39
Server Power Distribution .............................................................................. 5-40
RA4000/RA4100 Array Power Distribution .................................................. 5-40
Power Sequencing .......................................................................................... 5-41
Chapter 6
Cluster Management
Cluster Management Concepts ................................................................................ 6-2
Powering Off a Node Without Interrupting Cluster Services ........................... 6-2
Managing a Cluster in a Degraded Condition................................................... 6-2
Managing Network Clients Connected to a Cluster ......................................... 6-3
Cluster Events................................................................................................... 6-3
Management Applications ....................................................................................... 6-4
Monitoring Server and Network Hardware ...................................................... 6-4
Managing Shared Drives .................................................................................. 6-5
Monitoring Non-Redundant Fibre Channel Fabrics ......................................... 6-5
Monitoring Non-Redundant Fibre Channel Arbitrated Loops.......................... 6-6
Monitoring the Database .................................................................................. 6-6
Remotely Managing a Cluster .......................................................................... 6-7
Software Maintenance for Oracle8i......................................................................... 6-8
Deinstalling the OSDs ...................................................................................... 6-8
Upgrading Oracle8i Server ............................................................................. 6-11
Upgrading the OSDs....................................................................................... 6-11
Deinstalling a Partial OSD Installation........................................................... 6-13
Upgrading Oracle8i Server ............................................................................. 6-14
Managing Changes to Shared Storage Components.............................................. 6-14
Replacing a Failed Disk.................................................................................. 6-14
Adding Disk Drives to Increase Shared Storage Capacity ............................. 6-15
Adding an RA4000/RA4100 Array................................................................ 6-16
Replacing a Failed Fibre Host Adapter........................................................... 6-17
Replacing a Cluster Node ...................................................................................... 6-18
Removing the Node........................................................................................ 6-18
Adding the Replacement Node....................................................................... 6-19
Contents vii
viii Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Cluster Management
continued
Adding a Cluster Node........................................................................................... 6-21
Preparing the New Node................................................................................. 6-22
Preparing the Existing Cluster Nodes ............................................................. 6-23
Installing the Cluster Software for Oracle8i ................................................... 6-23
Monitoring Cluster Operation................................................................................ 6-25
Tools Overview............................................................................................... 6-25
Chapter 7
Troubleshooting
Basic Troubleshooting Tips ..................................................................................... 7-2
Power ................................................................................................................ 7-2
Physical Connections........................................................................................ 7-2
Access to Cluster Components .........................................................................7-3
Software Revisions ........................................................................................... 7-3
Firmware Revisions .......................................................................................... 7-4
Troubleshooting Oracle8i and OSD Installation Problems and Error Messages ..... 7-5
Potential Difficulties Installing the OSDs with the Oracle Universal
Installer ............................................................................................................. 7-5
Unable to Start OracleCMService..................................................................... 7-6
Unable to Start OracleNMService .................................................................... 7-7
Unable to Start the Database............................................................................. 7-7
Initialization of the Dynamic Link Library NM.DLL Failed............................ 7-8
Troubleshooting Node-to-Node Connectivity Problems.......................................... 7-8
Nodes Are Unable to Communicate with Each Other ...................................... 7-8
Unable to Ping the Cluster Interconnect or the Client LAN ............................. 7-9
Node or Nodes Unable to Rejoin the Cluster.................................................. 7-10
Troubleshooting Client-to-Cluster Connectivity Problems.................................... 7-10
A Network Client Cannot Communicate with the Cluster.............................. 7-10
Troubleshooting Shared Storage Problems............................................................ 7-11
Verifying Connectivity to a Non-Redundant Fibre Channel Fabric ............... 7-11
Verifying Connectivity to a Non-Redundant Fibre Channel Arbitrated
Loop................................................................................................................ 7-12
Shared Disks in the RA4000/RA4100 Arrays Are Not Recognized By One
or More Nodes ................................................................................................ 7-12
A Cluster Node Cannot Connect to the Shared Drives ................................... 7-14
Appendix A
Diagnosing and Resolving Shared Disk Problems
Introduction .............................................................................................................A-1
Run Object Link Manager On All Nodes ................................................................A-3
Restart All Affected Nodes in the Cluster ...............................................................A-4
Rerun and Validate Object Link Manager On All Affected Nodes .........................A-4
Run Disk Management On All Nodes .....................................................................A-5
Run and Validate the Array Configuration Utility On All Nodes............................A-5
Perform Cluster Software and Firmware Checks ....................................................A-6
Perform Cluster Hardware Checks ..........................................................................A-6
Contact Your Compaq Support Representative.......................................................A-7
Glossary
Index
Contents ix
x Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
List of Figures
Figure 1-1. Example of a six-node Compaq Parallel Database Cluster
Model PDC/O1000............................................................................................. 1-2
Figure 2-1. Fibre Host Adapters in a two-node PDC/O1000 cluster....................... 2-7
Figure 2-2. Two-node PDC/O1000 cluster with one non-redundant Fibre
Channel Fabric ................................................................................................. 2-11
Figure 2-3. PDC/O1000 cluster with two non-redundant Fibre Channel
Fabrics.............................................................................................................. 2-12
Figure 2-4. Maximum distances between cluster nodes and shared
storage components in a non-redundant Fibre Channel Fabric ........................ 2-13
Figure 2-5. Fibre Host Adapter-to-Fibre Channel SAN Switch I/O data
paths ................................................................................................................. 2-14
Figure 2-6. Fibre Channel SAN Switch-to-array controller I/O data path ............ 2-15
Figure 2-7. Two-node PDC/O1000 cluster with one non-redundant FC-AL........ 2-17
Figure 2-8. PDC/O1000 cluster with two non-redundant FC-ALs ....................... 2-19
Figure 2-9. Maximum distances between cluster nodes and shared
storage components in a non-redundant FC-AL............................................... 2-20
Figure 2-10. Fibre Host Adapter-to-Storage Hub/FC-AL Switch I/O data
paths ................................................................................................................. 2-21
Figure 2-11. Storage Hub/FC-AL Switch-to-array controller I/O data path ......... 2-22
Figure 2-12. Non-redundant Ethernet cluster interconnect using a
crossover cable ................................................................................................. 2-26
Figure 2-13. Non-redundant Ethernet cluster using an Ethernet switch or
hub.................................................................................................................... 2-27
Figure 2-14. Redundant Ethernet cluster interconnect for a two-node
PDC/O1000 cluster .......................................................................................... 2-28
Figure 4-1. Midsize PDC/O1000 cluster with one non-redundant Fibre
Channel Fabric ................................................................................................... 4-8
Figure 4-2. Larger PDC/O1000 cluster with one non-redundant Fibre
Channel Fabric ................................................................................................... 4-9
Figure 4-3. Midsize PDC/O1000 cluster with one non-redundant FC-AL ........... 4-10
Figure 4-4. Larger PDC/O1000 cluster with one non-redundant FC-AL.............. 4-11
Figure 4-5. RA4000/RA4100 Array disk grouping for a PDC/O1000 cluster...... 4-17
Figure 5-1. Connecting Fibre Host Adapters to a Storage Hub, FC-AL
Switch, or Fibre Channel SAN Switch............................................................... 5-6
Figure 5-2. Cabling a Storage Hub, FC-AL Switch, or Fibre Channel SAN
Switch to an RA4000 Array Controller.............................................................. 5-9
Figure 5-3. PDC/O1000 cluster with two non-redundant Fibre Channel
Fabrics or non-redundant FC-ALs.................................................................... 5-11
Figure 5-4. Non-redundant Ethernet cluster interconnect using a crossover
cable................................................................................................................. 5-13
Figure 5-5. Non-redundant Ethernet cluster interconnect using an Ethernet
switch or hub.................................................................................................... 5-14
Figure 5-6. Redundant Ethernet cluster interconnect for a two-node
PDC/O1000 cluster .......................................................................................... 5-15
Figure 5-7. Server power distribution in a three-node cluster............................... 5-40
Figure A-1. Tasks for diagnosing and resolving shared disk problems..................A-2
List of Tables
Table 2-1 High-Availability Components of ProLiant Servers ............................... 2-3
Contents xi
Purpose
Audience
About This Guide
This administrator guide provides information about the planning, installation,
configuration, implementation, management, and troubleshooting of the
Compaq Parallel Database Cluster Model PDC/O1000 running Oracle8i
software on the Microsoft Windows 2000 Advanced Server operating system.
The expected audience of this guide consists primarily of MIS professionals
whose jobs include designing, installing, configuring, and maintaining
Compaq Parallel Database Clusters.
The audience of this guide must have a working knowledge of Microsoft
Windows 2000 Advanced Server and of Oracle databases or have the
assistance of a database administrator.
This guide contains information for network administrators, database
administrators, installation technicians, systems integrators, and other
technical personnel in the enterprise environment for the purpose of cluster
planning, installation, implementation, and maintenance.
IMPORTANT: This guide contains installation, configuration, and maintenance
information that can be valuable for a variety of users. If you are installing the PDC/O1000
but will not be administering the cluster on a daily basis, please make this guide available
to the person or persons who will be responsible for the clustered servers after you have
completed the installation.
xiv Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Scope
This guide offers significant background information about clusters as well as
basic concepts associated with designing clusters. It also contains detailed
product descriptions and installation steps.
This administrator guide is designed to assist you in the following objectives:
■ Understanding basic concepts of clustering technology
■ Recognizing and using the high-availability features of the PDC/O1000
■ Planning and designing a PDC/O1000 cluster configuration to meet your
business needs
■ Installing and configuring PDC/O1000 hardware and software
■ Managing the PDC/O1000
■ Troubleshooting the PDC/O1000
The following summarizes the contents of this guide:
■ Chapter 1, “Clustering Overview,” provides an introduction to
clustering technology features and benefits.
■ Chapter 2, “Cluster Architecture,” describes the hardware components
of the PDC/O1000 and provides detailed information about I/O path
configurations and cluster interconnect options.
■ Chapter 3, “Cluster Software Components,” describes software
components used with the PDC/O1000.
■ Chapter 4, “Cluster Planning,” outlines approaches to planning and
designing PDC/O1000 cluster configurations that meet your business
needs.
■ Chapter 5, “Installation and Configuration,” outlines the steps you will
take to install and configure the PDC/O1000 hardware and software.
■ Chapter 6, “Cluster Management,” includes techniques for managing
and maintaining the PDC/O1000.
■ Chapter 7, “Troubleshooting,” contains troubleshooting information for
the PDC/O1000.
■ Appendix A, “Diagnosing and Resolving Shared Disk Problems,”
describes procedures to diagnose and resolve shared disk problems.
■ Glossary contains definitions of terms used in this guide.
Some clustering topics are mentioned, but not detailed, in this guide. For
example, this guide does not describe how to install and configure Oracle8i on
a PDC/O1000 cluster. For information about these topics, see the documents
referenced in the guide sections or refer to the documentation provided with
the Oracle8i software.
Referenced Manuals
For additional information, refer to documentation related to the specific
hardware and software components of the Compaq Parallel Database Cluster.
These related manuals include, but are not limited to:
■ Documentation related to the ProLiant servers you are clustering (for
example, guides, posters, and performance and tuning guides)
■ Compaq StorageWorks documentation
G Compaq StorageWorks RAID Array 4000 User Guide
G Compaq StorageWorks RAID Array 4100 User Guide
G Compaq StorageWorks Fibre Channel SAN Switch 8 Installation
and Hardware Guide
G Compaq StorageWorks Fibre Channel SAN Switch 16 Installation
and Hardware Guide
About This Guide xv
G Compaq StorageWorks Fibre Channel SAN Switch Management
Guide provided with the Fibre Channel SAN Switch
G Compaq StorageWorks Fibre Channel Arbitrated Loop Switch
(FC-AL Switch) User Guide
G Compaq StorageWorks Fibre Channel Storage Hub 7 Installation
Guide
G Compaq StorageWorks Fibre Channel Storage Hub 12 Installation
Guide
G Compaq StorageWorks Fibre Channel Host Bus Adapter Installation
Guide
G Compaq StorageWorks 64-Bit/66-MHz Fibre Channel Host Adapter
Installation Guide
■ Microsoft Windows 2000 Advanced Server documentation
G Microsoft Windows 2000 Advanced Server Administrator’s Guide
■ Oracle8i documentation, including:
G Oracle8i Parallel Server Setup and Configuration Guide
xvi Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
G
Oracle8i Parallel Server Concepts
G Oracle8i Parallel Server Administration, Deployment, and
Performance
G Oracle Enterprise Manager Administrator’s Guide
G Oracle Enterprise Manager Configuration Guide
G Oracle Enterprise Manager Concepts Guide
Supplemental Documents
The following technical documents contain important supplemental
information for the Compaq Parallel Database Cluster Model PDC/O1000:
■ Supported Ethernet Interconnects for Compaq Parallel Database
Clusters Using Oracle Parallel Server (ECG062/0299), at
www.compaq.com/support/techpubs/whitepapers
■ Compaq Parallel Database Cluster Model PDC/O1000 Certification
Matrix for Windows 2000, at
www.compaq.com/enterprise/ha-pdc.html
■ Various technical white papers on Oracle and cluster sizing, which are
available from Compaq ActiveAnswers website, at
www.compaq.com/activeanswers
Text Conventions
This document uses the following conventions to distinguish elements of text:
User Input, GUI
Selections
About This Guide xvii
Text a user types or enters appears in boldface.
Items a user selects from a GUI, such as tabs,
buttons, or menu items, also appear in boldface.
User input and GUI selections can appear in
uppercase and lowercase letters.
File Names, Command
Names, Directory
Names, Drive Names
Menu Options, Dialog
Box Names
Type When you are instructed to type information, type
Enter When you are instructed to enter information, type
Symbols in Text
These symbols may be found in the text of this guide. They have the following
meanings:
These elements can appear in uppercase and
lowercase letters.
These elements appear in initial capital letters and
may be bolded for emphasis.
the information without pressing the Enter key.
the information and then press the Enter key.
WARNING: Text set off in this manner indicates that failure to follow directions
in the warning could result in bodily harm or loss of life.
CAUTION: Text set off in this manner indicates that failure to follow directions
could result in damage to equipment or loss of information.
IMPORTANT: Text set off in this manner presents clarifying information or specific
instructions.
NOTE: Text set off in this manner presents commentary, sidelights, or interesting points
of information.
xviii Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Symbols on Equipment
These icons may be located on equipment in areas where hazardous conditions
may exist.
Any surface or area of the equipment marked with these symbols
indicates the presence of electrical shock hazards. Enclosed area
contains no operator serviceable parts.
WARNING: To reduce the risk of injury from electrical shock hazards,
do not open this enclosure.
Any RJ-45 receptacle marked with these symbols indicates a Network
Interface Connection.
WARNING: To reduce the risk of electrical shock, fire, or damage to
the equipment, do not plug telephone or telecommunications
connectors into this receptacle.
Any surface or area of the equipment marked with these symbols
indicates the presence of a hot surface or hot component. If this
surface is contacted, the potential for injury exists.
WARNING: To reduce the risk of injury from a hot component, allow
the surface to cool before touching.
Power Supplies or Systems marked with these symbols
indicate the equipment is supplied by multiple sources of
power.
WARNING: To reduce the risk of injury from electrical shock,
remove all power cords to completely disconnect power from
the system.
Rack Stability
WARNING: To reduce the risk of personal injury or damage to the equipment,
be sure that:
■ The leveling jacks are extended to the floor.
■ The full weight of the rack rests on the leveling jacks.
■ The stabilizing feet are attached to the rack if it is a single rack
installations.
■ The racks are coupled together in multiple rack installations.
■ Only one component is extended at a time. A rack may become unstable if
more than one component is extended for any reason.
Getting Help
If you have a problem and have exhausted the information in this guide, you
can get further information and other help in the following locations.
Compaq Technical Support
About This Guide xix
In North America, call the Compaq Technical Phone Support Center at
1-800-OK-COMPAQ. This service is available 24 hours a day, 7 days a week.
For continuous quality improvement, calls may be recorded or monitored.
Outside North America, call the nearest Compaq Technical Support Phone
Center. Telephone numbers for worldwide Technical Support Centers are
listed on the Compaq website. Access the Compaq website by logging on to
the Internet at
www.compaq.com
Be sure to have the following information available before you call Compaq:
■ Technical support registration number (if applicable)
■ Product serial number
■ Product model name and number
■ Applicable error messages
■ Add-on boards or hardware
■ Third-party hardware or software
■ Operating system type and revision level
xx Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Compaq Website
The Compaq website has information on this product as well as the latest
drivers and Flash ROM images. You can access the Compaq website by
logging on to the Internet at
www.compaq.com
Compaq Authorized Reseller
For the name of your nearest Compaq Authorized Reseller:
■ In the United States, call 1-800-345-1518.
■ In Canada, call 1-800-263-5868.
■ Elsewhere, see the Compaq website for locations and telephone
numbers.
Chapter 1
Clustering Overview
For many years, companies have depended on clustered computer systems to
fulfill two key requirements: to ensure users can access and process
information that is critical to the ongoing operation of their business, and to
increase the performance and throughput of their computer systems at minimal
cost. These requirements are known as availability and scalability,
respectively.
Historically, these requirements have been fulfilled with clustered systems
built on proprietary technology. Over the years, open systems have
progressively and aggressively moved proprietary technologies into
industry-standard products. Clustering is no exception. Its primary features,
availability and scalability, have been moving into client/server products for
the last few years.
The absorption of clustering technologies into open systems products is
creating less expensive, non-proprietary solutions that deliver levels of
functionality commonly found in traditional clusters. While some uses of the
proprietary solutions will always exist—such as those controlling stock
exchange trading floors and aerospace mission controls—many critical
applications can reach the desired levels of availability and scalability with
non-proprietary client/server-based clustering.
These clustering solutions use industry-standard hardware and software,
thereby providing key clustering features at a lower price than proprietary
clustering systems. Before examining the features and benefits of the
Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the
PDC/O1000), it is helpful to understand the concepts and terminology of
clustered systems.
1-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i on Windows 2000 Administrator Guide
Clusters Defined
A cluster is an integration of software and hardware products that enables a set
of loosely coupled servers and shared storage subsystem components to
present a single system image to clients and to operate as a single system. As a
cluster, the group of servers and shared storage subsystem components offers a
level of availability and scalability far exceeding that obtained if each cluster
node operated as a standalone server.
The PDC/O1000 uses Oracle8i Parallel Server, which is a parallel database
that can distribute its workload among the cluster nodes.
Figure 1-1 shows a PDC/O1000 cluster that contains:
■ Six cluster nodes (ProLiant
■ One Compaq StorageWorks
TM
servers)
TM
RAID Array 4000 (RA4000 Array) or
4100 (RA4100 Array)
■ One Compaq StorageWorks Fibre Channel Storage Hub (Storage Hub),
Compaq StorageWorks FC-AL Switch (FC-AL Switch), or
Compaq StorageWorks Fibre Channel SAN Switch
(Fibre Channel SAN Switch)
■ An Ethernet cluster interconnect
■ A client local area network (LAN)
RA4000/4100 Array
Node 2 Node 4 Node 6
Client LAN
Clients
Node 1Node 3Node 5
Storage Hub/Switch
(Cluster Interconnect)
Switch
Figure 1-1. Example of a six-node Compaq Parallel Database Cluster
Model PDC/
O1000
The PDC/O1000 uses non-redundant Fibre Channel Fabric Storage Area
Network (SAN) and non-redundant Fibre Channel Arbitrated Loop (FC-AL)
SAN topologies for its shared storage I/O data paths. These two SAN
topologies support the use of multiple non-redundant fabrics and loops,
respectively. In the example shown in Figure 1-1, the clustered nodes are
connected to the database on the shared storage subsystems through a
non-redundant Fibre Channel Fabric or non-redundant FC-AL. Clients access
the database through the client LAN, and the cluster nodes communicate
across an Ethernet cluster interconnect.
Availability
When computer systems experience outages, the amount of time the system is
unavailable is referred to as downtime. Downtime has several primary causes:
hardware faults, software faults, planned service, operator error, and
environmental factors. Minimizing downtime is a primary goal of a cluster.
Simply defined, availability is the measure of how well a computer system can
continuously deliver services to clients.
Availability is a system-wide endeavor. The hardware, the operating system,
and the applications must be designed for availability. Clustering requires
stability in these components, then couples them in such a way that failure of
one item does not render the system unusable. By using redundant components
and mechanisms that detect and recover from faults, clusters can greatly
increase the availability of applications critical to business operations.
Clustering Overview 1-3
Scalability
Simply defined, scalability is a computer system characteristic that enables
improved performance or throughput when supplementary hardware resources
are added. Scalable systems allow increased throughput by adding components
to an existing system without the expense of adding a new system.
In a stand-alone server configuration, scalable systems allow increased
throughput by adding processors or more memory. In a cluster configuration,
this result is usually obtained by adding cluster nodes.
Not only must the hardware benefit from additional components, but also
software must be constructed in such a way as to take advantage of the
additional processing power. Oracle8i Parallel Server distribute the workload
among the cluster nodes. As more nodes are added to the cluster, cluster-aware
applications can use the parallel features of Oracle8i Parallel Server to
distribute workload among more servers, thereby obtaining greater throughput.
1-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i on Windows 2000 Administrator Guide
Compaq Parallel Database Cluster Overview
As traditional clustering technology has moved into the open systems of
client/server computing, Compaq has provided innovative, customer-focused
solutions. The PDC/O1000 moves client/server computing one step closer to
the capabilities found in expensive, proprietary cluster solutions, at a fraction
of the cost.
The PDC/O1000 combines the Microsoft Windows 2000 Advanced Server
operating system and the industry-leading Oracle8i Parallel Server with
award-winning Compaq ProLiant servers and shared storage subsystems.
Together, these hardware and software components provide improved
performance through a truly scalable parallel application and improved
availability using clustering software that rapidly recovers from detectable
faults. These components also provide improved availability through
concurrent multinode database access using Oracle8i Parallel Server.
Chapter 2
Architecture
The Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as
the PDC/O1000) is an integration of a number of different hardware and
software products. This chapter discusses how each of the hardware products
plays a role in bringing a complete clustering solution to your computing
environment.
The hardware products include:
■ Compaq Proliant servers
■ Shared storage components
G Compaq StorageWorks RAID Array 4100s (RA4100 Arrays) or
Compaq StorageWorks RAID Array 4000s (RA4000 Arrays)
G Compaq StorageWorks RAID Array 4000 Controller (RA4000
Array Controller) installed in each RA4000 Array or RA4100 Array
G Compaq StorageWorks Fibre Channel SAN Switch (Fibre Channel
SAN Switch) for each non-redundant Fibre Channel Fabric
G Compaq StorageWorks Storage Hub (Storage Hub) or
Compaq StorageWorks FC-AL Switch (FC-AL Switch) for each
non-redundant Fibre Channel Arbitrated Loop
G Compaq StorageWorks 64-bit/66 MHz Fibre Channel Host Adapter
or Compaq StorageWorks Fibre Channel Host Adapter/P (Fibre Host
Adapter) installed in each server
2-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
Gigabit Interface Converter-Shortwave (GBIC-SW) modules
G
G Fibre Channel cables
■ Cluster interconnect components
G Ethernet NIC adapters
G Ethernet cables
G Ethernet switches or hubs
The software products include:
■ Microsoft Windows 2000 Advanced Server with Service Pack 1 or later
■ Compaq drivers and utilities
■ Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option
Refer to Chapter 3, “Cluster Software Components,” for a description of the
software products used with the PDC/O1000.
Compaq ProLiant Servers
A primary component of any cluster is the server. Each PDC/O1000 consists
of two or more cluster nodes. Each node is a Compaq ProLiant server.
With some exceptions, all nodes in a PDC/O1000 cluster must be identical in
model. In addition, all components common to all nodes in a cluster, such as
memory, number of CPUs, and the interconnect adapters, must be identical
and identically configured.
NOTE: Certain restrictions apply to the server models and server configurations that are
supported by the PDC/O1000. For a current list of PDC-certified servers and details on
supported configurations, refer to the Compaq Parallel Database Cluster Model
PDC/O1000 for Windows 2000 Certification Matrix at
www.compaq.com/solutions/enterprise/ha-pdc.html
High-Availability Features of ProLiant Servers
In addition to the increased application and data availability enabled by
clustering, ProLiant servers include many reliability features that provide a
solid foundation for effective clustered server solutions. The PDC/O1000 is
based on ProLiant servers, most of which offer excellent reliability through
redundant power supplies, redundant cooling fans, and Error Checking and
Correcting (ECC) memory. The high-availability features of ProLiant servers
are a critical foundation of Compaq clustering products. Table 2-1 lists the
high-availability features found in many ProLiant servers.
Table 2-1
High-Availability Components of ProLiant Servers
Hot-pluggable hard drives Redundant power supplies
Digital Linear Tape (DLT) Array (optional) ECC-protected processor-memory bus
Uninterruptible power supplies (optional) Redundant processor power modules
ECC memory PCI Hot Plug slots (in some servers)
Offline backup processor Redundant cooling fans
Architectur e 2-3
Shared Storage Components
The PDC/O1000 is based on a cluster architecture known as “shared storage
clustering,” in which clustered nodes share access to a common set of shared
disk drives. For the PDC/O1000, the shared storage includes these hardware
components:
■ RA4000 Arrays or RA4100 Arrays
■ One RA4000 Array Controller in each RA4000 Array or RA4100 Array
■ One Fibre Channel SAN Switch for each non-redundant Fibre Channel
Fabric
■ One Storage Hub or FC-AL Switch for each non-redundant Fibre
Channel Arbitrated Loop
■ Fibre Host Adapters
■ GBIC-SW modules
■ Fibre Channel cables
2-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
RA4000 Array
The RA4000 Array is one shared storage solution for the PDC/O1000. Each
non-redundant Fibre Channel Fabric or non-redundant Fibre Channel
Arbitrated Loop (FC-AL) can contain one or more RA4000 Arrays. Each
RA4000 Array contains one single-port RA4000 Array Controller. The array
controller connects the RA4000 Array to one Storage Hub, FC-AL Switch, or
Fibre Channel SAN Switch, which in turn is connected to one Fibre Host
Adapter in each cluster node.
The RA4000 Array can hold up to twelve 1-inch or eight 1.6-inch Wide-Ultra
SCSI drives. The drives must be mounted on Compaq hot-pluggable drive
trays. SCSI IDs are assigned automatically according to their drive location,
allowing 1-inch and 1.6-inch drives to be intermixed within the same RA4000
Array.
The RA4000 Array comes in either a rack-mountable or a tower model.
For more information about the RA4000 Array, refer to the
Compaq StorageWorks RAID Array 4000 User Guide.
RA4100 Array
The RA4100 Array is another shared storage solution for the PDC/O1000.
Each non-redundant Fibre Channel Fabric or non-redundant Fibre Channel
Arbitrated Loop (FC-AL) can contain one or more RA4100 Arrays. Each
RA4100 Array contains one single-port RA4000 Array Controller. The array
controller connects the RA4100 Array to one Storage Hub, FC-AL Switch, or
Fibre Channel SAN Switch, which in turn is connected to one Fibre Host
Adapter in each cluster node.
The RA4100 Array can hold up to twelve 1-inch Compaq Hot Plug Ultra2
Disk Drives. The drives must be mounted on Compaq hot-pluggable drive
trays. SCSI IDs are assigned automatically according to their drive location.
The RA4100 Array comes in a rack-mountable model.
For more information about the RA4100 Array, refer to the
Compaq StorageWorks RAID Array 4100 User Guide.
RA4000 Array Controller
One single-port RA4000 Array Controller is installed in each RA4000 Array
or RA4100 Array. If the array controller fails, the cluster nodes cannot access
the shared storage disks in that array.
From the perspective of the cluster nodes, the RA4000 Array Controller is
simply another device connected to one of the cluster’s I/O paths.
Consequently, each node sends its I/O requests to the RA4000 Array
Controller just as it would to any SCSI device. The RA4000 Array Controller
receives the I/O requests from the nodes and directs them to the shared storage
disks to which it has been configured. Because the array controller processes
the I/O requests, the cluster nodes are not burdened with the I/O processing
tasks associated with reading and writing data to multiple shared storage
devices.
For more information about the RA4000 Array Controller, refer to
the Compaq StorageWorks RAID Array 4000 User Guide or the
Compaq StorageWorks RAID Array 4100 User Guide.
Fibre Channel SAN Switch
Architectur e 2-5
One Fibre Channel SAN Switch is installed between cluster nodes and shared
storage arrays in a PDC/O1000 cluster to create a non-redundant Fibre
Channel Fabric.
An 8-port or 16-port Fibre Channel SAN Switch can be used. The choice of an
8-port or 16-port Fibre Channel SAN Switch is determined by your hardware
requirements. For example, a non-redundant Fibre Channel Fabric with four
Fibre Host Adapters and five or more RA4000/RA4100 Arrays would require
a 16-port Fibre Channel SAN Switch.
The Fibre Channel SAN Switch provides full 100 MBps bandwidth on every
port. Adding new devices to ports on the Fibre Channel SAN Switch increases
the aggregate bandwidth.
Fibre Channel SAN Switch is used to connect one Fibre Host Adapter in each
cluster node to the array controller in the RA4000/RA4100 Arrays. The Fibre
Host Adapter in each node, the Fibre Channel SAN Switch, and the
RA4000/RA41000 Arrays to which they are connected belong to the same
non-redundant Fibre Channel Fabric.
2-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide
For further information, refer to these manuals provided with the Fibre
Channel SAN Switch:
■ Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and
Hardware Guide
■ Compaq StorageWorks Fibre Channel SAN Switch 16 Installation and
Hardware Guide
■ Compaq StorageWorks Fibre Channel SAN Switch Management Guide
provided with the Fibre Channel SAN Switch
Storage Hub
One Storage Hub can be installed between cluster nodes and shared storage
arrays in a PDC/O1000 cluster to create a non-redundant Fibre Channel
Arbitrated Loop (FC-AL).
The Storage Hub is one method for connecting one Fibre Host Adapter in each
node with the array controller in the RA4000/RA4100 Array. The Fibre Host
Adapter in each node, the Storage Hub, and the RA4000/RA41000 Arrays to
which they are connected belong to the same non-redundant FC-AL.
On the Storage Hub, one port is used by a Fibre Host Adapter in each node
and one port is used to connect to the array controller in each
RA4000/RA4100 Array.
The PDC/O1000 allows the use of either the Storage Hub 7 (with 7 ports) or
the Storage Hub 12 (with 12 ports). Using the Storage Hub 7 limits the total
number of nodes and RA4000/RA4100 Arrays you can install in a
non-redundant. For example, a non-redundant FC-AL with four Fibre Host
Adapters and four or more RA4000/RA4100 Arrays requires a Storage Hub
with at least 8 ports (a Storage Hub 12). In your selection of a Storage Hub,
you should also consider the likelihood of cluster growth.
Refer to the Compaq StorageWorks Fibre Channel Storage Hub 7 Installation
Guide and the Compaq StorageWorks Fibre Channel Storage Hub 12
Installation Guide for further information about these products.
FC-AL Switch
One FC-AL Switch can also be installed between cluster nodes and shared
storage arrays in a PDC/O1000 cluster to create a non-redundant Fibre
Channel Arbitrated Loop (FC-AL).
The FC-AL Switch is another device for connecting one Fibre Host Adapter in
each node with the array controller in the RA4000/RA4100 Array. The Fibre
Host Adapter in each node, the FC-AL Switch, and the RA4000/RA41000
Arrays to which they are connected belong to the same non-redundant FC-AL.
The FC-AL Switch 8 supports eight ports. With the addition of the 3-port
Expansion Module (PEM), the switch supports 11 ports.
For further information, refer to the Compaq StorageWorks Fibre Channel
Arbitrated Loop Switch (FC-AL Switch) User Guide.
Fibre Host Adapters
Fibre Host Adapters are the interface between the cluster nodes (servers) and
the RA4000/RA4100 Arrays to which they are connected. As Figure 2-1
shows, a Fibre Channel cable runs from one Fibre Host Adapter in each cluster
node to a port on the Fibre Channel SAN Switch, FC-AL Switch, or
Storage Hub.
Architectur e 2-7
RA4000/4100
Array
Fibre
Host Adapter
ProLiant
Server
Figure 2-1. Fibre Host Adapters in a two-node PDC/O1000 cluster
Node 1
Client LAN
RA4000/4100
Array
Storage Hub/Switch
Switch
(Cluster Interconnect)
RA4000/4100
Array
Node 2
Fibre
Host Adapter
ProLiant
Server
Loading...