Compaq AA-RHGWC-TE User Manual
TruCluster Server
Hardware Configuration
Part Number: AA-RHGWC-TE
August 2000
Product Version: TruCluster Server Version 5.1
Operating System and Version: Tru64 UNIX Version 5.1
This manual describes howto configure the hardware for a TruCluster™
Server environment. TruCluster Server Version 5.1 runs on the
Tru64™ UNIX operating system.
Compaq Computer Corporation
Houston, Texas
© 2000 Compaq Computer Corporation
COMPAQ and the Compaq logo Registered in U.S. Patent and Trademark Office. Alpha, AlphaServer,
StorageWorks, TruCluster, and Tru64 are trademarks of Compaq Information Technologies Group, L.P.
Microsoft and Windows are trademarks of Microsoft Corporation. UNIX and The Open Group are
trademarks of The Open Group. All other product names mentioned herein may be trademarks or
registered trademarks of their respective companies.
®
Cover photo: Digital imagery
copyright 1999 PhotoDisc, Inc.
Confidential computer software. Valid license from Compaq required for possession, use, or copying.
Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software
Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under
vendor’s standard commercial license.
Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information
in this document is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
KIND. THE ENTIRE RISK ARISING OUT OF THE USE OF THIS INFORMATION REMAINS WITH
RECIPIENT. IN NO EVENT SHALL COMPAQ BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL,
INCIDENTAL, SPECIAL, PUNITIVE, OR OTHER DAMAGES WHATSOEVER (INCLUDING WITHOUT
LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION OR LOSS
OF BUSINESS INFORMATION), EVEN IF COMPAQ HAS BEEN ADVISED OF THE POSSIBILITY
OF SUCH DAMAGES AND WHETHER IN AN ACTION OF CONTRACT OR TORT, INCLUDING
NEGLIGENCE.
The limited warranties for Compaq products are exclusively set forth in the documentation
accompanying such products. Nothing herein should be construed as constituting a further or
additional warranty.
About This Manual
1 Introduction
1.1
1.2
1.3
1.3.1
1.3.1.1
1.3.1.2
1.3.1.3
1.3.1.4
1.4
1.5
1.5.1
1.5.2
1.5.3
1.5.4
1.5.5
1.6
The TruCluster Server Product ..................................... 1–1
Memory Requirements ...............................................
Minimum Disk Requirements ...................................... 1–2
Disks Needed for Installation .................................. 1–2
Tru64 UNIX Operating System Disk ..................... 1–2
Clusterwide Disk(s) ......................................... 1–3
Member Boot Disk .......................................... 1–3
Quorum Disk .................................................
Generic Two-NodeCluster ........................................... 1–4
Growing a Cluster from Minimum Storage to a NSPOF
Cluster ..................................................................
Two-Node Clusters Using an UltraSCSI BA356 Storage
Shelf and Minimum Disk Configurations .................... 1–7
Two-Node Clusters Using UltraSCSI BA356 Storage Units
with Increased Disk Configurations .......................... 1–9
Two-Node Configurations with UltraSCSI BA356 Storage
Units and Dual SCSI Buses .................................... 1–11
Using Hardware RAID to Mirror the Clusterwide Root
File System and Member System Boot Disks ................ 1–12
Creating a NSPOF Cluster ..................................... 1–14
Overview of Setting Up the TruCluster Server Hardware
Configuration .......................................................... 1–16
Contents
1–2
1–4
1–6
2 Hardware Requirements and Restrictions
2.1
2.2
2.3
2.3.1
2.3.2
2.3.3
2.4
2.5
TruCluster Server Member System Requirements .............. 2–1
Memory Channel Restrictions ...................................... 2–3
Host Bus Adapter Restrictions ..................................... 2–5
Fibre Channel Requirements and Restrictions .............. 2–5
KZPSA-BB SCSI Adapter Restrictions ....................... 2–8
KZPBA-CB SCSI Bus Adapter Restrictions ................. 2–9
Disk Device Restrictions ............................................. 2–9
RAID Array Controller Restrictions ............................... 2–9
Contents iii
2.6
2.7
2.8
2.9
SCSI Signal Converters ..............................................
DS-DWZZH-03 and DS-DWZZH-05 UltraSCSI Hubs ........... 2–11
SCSI Cables ............................................................
SCSI Terminators and Trilink Connectors ........................ 2–14
3 Shared SCSI Bus Requirements and Configurations Using
UltraSCSI Hardware
3.1
3.2
3.2.1
3.2.2
3.2.3
3.2.4
3.3
3.4
3.5
3.6
3.6.1
3.6.1.1
3.6.1.2
3.6.1.2.1
3.6.1.2.2
3.6.1.2.3
3.6.1.2.4
3.6.1.2.5
3.6.1.3
3.7
3.7.1
3.7.1.1
3.7.1.2
Shared SCSI Bus Configuration Requirements .................. 3–2
SCSI Bus Performance ...............................................
SCSI Bus Versus SCSI Bus Segments ........................ 3–3
Transmission Methods .......................................... 3–4
Data Path .........................................................
Bus Speed .........................................................
SCSI Bus Device Identification Numbers ........................ 3–5
SCSI Bus Length ......................................................
Terminating the Shared SCSI Bus When Using UltraSCSI
Hubs ....................................................................
UltraSCSI Hubs .......................................................
Using a DWZZH UltraSCSI Hub in a Cluster
Configuration .....................................................
DS-DWZZH-03 Description ................................ 3–9
DS-DWZZH-05 Description ................................ 3–10
DS-DWZZH-05 Configuration Guidelines .......... 3–10
DS-DWZZH-05 Fair Arbitration ..................... 3–12
DS-DWZZH-05 Address Configurations ............ 3–13
SCSI Bus Termination Power ......................... 3–15
DS-DWZZH-05 Indicators ............................. 3–15
Installing the DS-DWZZH-05 UltraSCSI Hub .......... 3–15
Preparing the UltraSCSI Storage Configuration ................ 3–16
Configuring Radially Connected TruCluster Server
Clusters with UltraSCSI Hardware ........................... 3–17
Preparing an HSZ70 or HSZ80 for a Shared SCSI Bus
Using Transparent Failover Mode ........................ 3–18
Preparing a Dual-Redundant HSZ70 or HSZ80 for a
Shared SCSI Bus Using Multiple-Bus Failover ........ 3–22
2–10
2–12
3–3
3–5
3–5
3–6
3–7
3–8
3–8
4 TruCluster Server System Configuration Using UltraSCSI
Hardware
4.1
4.2
4.3
iv Contents
Planning Your TruCluster Server Hardware Configuration ... 4–2
Obtaining the Firmware Release Notes ........................... 4–4
TruCluster Server Hardware Installation ........................ 4–5
4.3.1
4.3.2
4.3.3
4.3.3.1
4.3.3.2
4.3.3.3
Installation of a KZPBA-CB Using Internal Termination
for a Radial Configuration ......................................
Displaying KZPBA-CB Adapters with the show Console
Commands ........................................................
Displaying Console Environment Variables and Setting
the KZPBA-CB SCSI ID ........................................
Displaying KZPBA-CB pk* or isp* Console
Environment Variables ..................................... 4–15
Setting the KZPBA-CB SCSI ID .......................... 4–17
KZPBA-CB Termination Resistors ........................ 4–17
5 Setting Up the Memory Channel Cluster Interconnect
5.1
5.1.1
5.1.2
5.2
5.3
5.4
5.5
5.5.1
5.5.1.1
5.5.1.2
5.5.2
5.5.2.1
5.5.2.2
5.5.2.3
5.5.2.4
5.6
5.7
Setting the Memory Channel Adapter Jumpers ................ 5–2
MC1 and MC1.5 Jumpers ....................................... 5–2
MC2 Jumpers .....................................................
Installing the Memory Channel Adapter .......................... 5–6
Installing the MC2 Optical Converter in the Member System 5–6
Installing the Memory Channel Hub .............................. 5–7
Installing the Memory Channel Cables ........................... 5–7
Installing the MC1 or MC1.5 Cables .......................... 5–7
Connecting MC1 or MC1.5 Link Cables in Virtual Hub
Mode ...........................................................
Connecting MC1 Link Cables in Standard Hub Mode . 5–8
Installing the MC2 Cables ...................................... 5–9
Installing the MC2 Cables for Virtual Hub Mode
Without Optical Converters ............................... 5–9
Installing MC2 Cables in Virtual Hub Mode Using
Optical Converters .......................................... 5–10
Connecting MC2 Link Cables in Standard Hub Mode
(No Fiber Optics) ............................................ 5–10
Connecting MC2 Cables in Standard Hub Mode Using
Optical Converters .......................................... 5–10
Running Memory Channel Diagnostics ........................... 5–11
Upgrading Memory Channel Adapters ............................ 5–14
4–7
4–10
4–14
5–3
5–8
6 Using Fibre Channel Storage
6.1
6.2
6.2.1
6.2.2
6.2.2.1
Procedure for Installation Using Fibre Channel Disks ......... 6–2
Fibre Channel Overview ............................................. 6–4
Basic Fibre Channel Terminology ............................. 6–5
Fibre Channel Topologies ....................................... 6–6
Point-to-Point ................................................ 6–6
Contents v
6.2.2.2
6.2.2.3
6.3
6.3.1
6.3.2
6.4
6.4.1
6.4.2
6.5
6.5.1
6.5.1.1
6.5.1.2
6.5.1.2.1
6.5.1.2.2
6.5.1.2.3
6.5.1.2.4
6.5.1.2.5
6.5.2
6.5.2.1
6.5.2.2
6.5.2.3
6.5.3
6.5.3.1
6.6
6.6.1
6.6.1.1
6.6.1.2
6.6.1.3
6.6.1.4
6.6.1.5
Fabric .........................................................
Arbitrated Loop Topology ..................................
6–7
6–8
Example Fibre Channel Configurations Supported by
TruCluster Server .....................................................
6–9
Fibre Channel Cluster Configurations for Transparent
Failover Mode .....................................................
6–10
Fibre Channel Cluster Configurations for Multiple-Bus
Failover Mode .....................................................
6–11
Zoning and Cascaded Switches ..................................... 6–14
Zoning ..............................................................
Cascaded Switches ...............................................
6–14
6–15
Installing and Configuring Fibre Channel Hardware ........... 6–16
Installing and Setting Up the Fibre Channel Switch ...... 6–16
Installing the Switch ........................................ 6–17
Managing the Fibre Channel Switches .................. 6–18
Using the Switch Front Panel ........................ 6–19
Setting the Ethernet IP Address and Subnet Mask
from the Front Panel ................................... 6–19
Setting the DS-DSGGB-AA and DS-DSGGC-AA
Ethernet IP Address and Subnet Mask from a PC
or Terminal ..............................................
6–21
Logging In tothe Switch with a Telnet Connection 6–22
Setting the Switch Name via TelnetSession ....... 6–23
Installing and Configuring the KGPSA PCI-to-Fibre
Channel Adapter Module ....................................... 6–23
Installing the KGPSA PCI-to-Fibre Channel Adapter
Module ........................................................ 6–23
Setting the KGPSA-BC or KGPSA-CA to Run on a
Fabric ......................................................... 6–24
Obtaining the Worldwide Names of KGPSA Adapters 6–27
Setting Up the HSG80 Array Controller for Tru64 UNIX
Installation ........................................................ 6–28
Obtaining the WorldwideNames of HSG80 Controller 6–33
Installing Tru64 UNIX and TruCluster Server Software Using
Fibre Channel Storage ............................................... 6–34
Before You Install ................................................ 6–36
Configure the HSG80 Storagesets ........................ 6–36
Adding Units and Identifiers to the HSG80
Storagesets ................................................... 6–40
Set the Device Unit Number ............................... 6–43
Displaying the Available Boot Devices ................... 6–48
Set the bootdef_dev Console Environment Variable for
Tru64 UNIX Installation ................................... 6–49
vi Contents
6.6.2
6.6.3
6.6.4
6.6.5
6.6.6
6.6.7
6.7
6.7.1
6.7.2
6.8
6.8.1
6.8.2
Install the Tru64 UNIX Operating System .................. 6–51
Determining /dev/disk/dskn to Use for a Cluster
Installation ........................................................
Label the Disks to Be Used to Create the Cluster ..........
Install the TruCluster Server Software and Create the
First Cluster Member ...............
Reset the bootdef_dev Console Environment Variable ..... 6–53
Add Additional Systems to the Cluster ....................... 6–55
Converting the HSG80 from Transparent to Multiple-Bus
Failover Mode .........................................................
Overview ..........................................................
Procedure to Convert from Transparent to Multiple-bus
Failover Mode .....................................................
Using the emx Manager to Display Fibre Channel Adapter
Information ............................................................
Using the emxmgr Utility to Display Fibre Channel
Adapter Information .............................................
Using the emxmgr Utility Interactively ...................... 6–65
............................
7 Using GS80, GS160, or GS320 Hard Partitions in a TruCluster
Server Configuration
7.1
7.2
7.3
7.3.1
7.4
7.5
7.5.1
Overview ...............................................................
Hardware Requirements for a Hard Partition in a Cluster .... 7–2
Configuring Partitioned GS80, GS160, or GS320 Systems in a
TruCluster Configuration ............................................
Repartitioning a Single-Partition AlphaServer
GS80/160/320 as Two Partitions in a Cluster ................ 7–7
Determining AlphaServer GS80/160/320 System
Configuration .......................................................... 7–14
Updating GS80/160/320 Firmware ................................. 7–18
Updating AlphaServer GS80/160/320 Firmware ............ 7–18
6–51
6–53
6–53
6–57
6–58
6–59
6–62
6–62
7–1
7–7
8 Configuring a Shared SCSI Bus for Tape Drive Use
8.1
8.1.1
8.1.2
8.1.3
8.1.4
8.2
8.2.1
Preparing the TZ88 for Shared Bus Usage ....................... 8–1
Setting the TZ88N-VA SCSI ID ............................... 8–2
Cabling the TZ88N-VA .......................................... 8–3
Setting the TZ88N-TA SCSI ID ............................... 8–5
Cabling the TZ88N-TA .......................................... 8–5
Preparing the TZ89 for Shared SCSI Usage ...................... 8–6
Setting the DS-TZ89N-VW SCSI ID .......................... 8–6
Contents vii
8.2.2
8.2.3
8.2.4
8.3
8.3.1
8.3.2
8.4
8.4.1
8.4.2
8.5
8.5.1
8.5.2
8.6
8.6.1
8.6.2
8.7
8.7.1
8.7.2
8.8
8.8.1
8.8.2
8.8.2.1
8.8.2.2
8.8.2.3
8.8.2.4
8.9
8.9.1
8.9.2
8.9.3
8.9.4
8.10
8.10.1
8.10.2
8.10.3
8.10.4
Cabling the DS-TZ89N-VW Tape Drives ..................... 8–8
Setting the DS-TZ89N-TA SCSI ID ........................... 8–9
Cabling the DS-TZ89N-TA Tape Drives ...................... 8–9
Compaq 20/40 GB DLT Tape Drive ................................
8–10
Setting the Compaq 20/40 GB DLT Tape Drive SCSI ID .. 8–10
Cabling the Compaq 20/40 GB DLT Tape Drive .............
8–11
Compaq 40/80-GB DLT Drive ....................................... 8–14
Setting the Compaq 40/80-GB DLT Drive SCSI ID ........ 8–14
Cabling the Compaq 40/80-GB DLT Drive ................... 8–15
Preparing the TZ885 for Shared SCSI Usage .................... 8–17
Setting the TZ885 SCSI ID .................................... 8–17
Cabling the TZ885 Tape Drive ................................ 8–18
Preparing the TZ887 for Shared SCSI Bus Usage ............... 8–20
Setting the TZ887 SCSI ID .................................... 8–20
Cabling the TZ887 Tape Drive ................................ 8–21
Preparing the TL891 and TL892 DLT MiniLibraries for
Shared SCSI Usage ...................................................
8–23
Setting the TL891 or TL892 SCSI ID ......................... 8–23
Cabling the TL891 or TL892 MiniLibraries ................. 8–25
Preparing the TL890 DLT MiniLibrary Expansion Unit ....... 8–29
TL890 DLT MiniLibrary Expansion Unit Hardware ....... 8–29
Preparing the DLT MiniLibraries for Shared SCSI Bus
Usage ..............................................................
8–29
Cabling the DLT MiniLibraries ........................... 8–30
Configuring a Base Module as a Slave ................... 8–33
Powering Up the DLT MiniLibrary ....................... 8–34
Setting the TL890/TL891/TL892 SCSI ID ............... 8–35
Preparing the TL894 DLT Automated Tape Library for Shared
SCSI Bus Usage ....................................................... 8–36
TL894 Robotic Controller Required Firmware .............. 8–37
Setting TL894 Robotics Controller and Tape Drive SCSI
IDs .................................................................. 8–37
TL894 Tape Library Internal Cabling ........................ 8–39
Connecting the TL894 Tape Library to the Shared SCSI
Bus ................................................................. 8–41
Preparing the TL895 DLT Automated Tape Library for Shared
SCSI Bus Usage ....................................................... 8–43
TL895 Robotic Controller Required Firmware .............. 8–44
Setting the TL895 Tape Library SCSI IDs ................... 8–44
TL895 Tape Library Internal Cabling ........................ 8–45
Upgrading a TL895 .............................................. 8–47
viii Contents
8.10.5
8.11
Preparing the TL893 and TL896 Automated Tape Libraries
for Shared SCSI Bus Usage .............
8.11.1
8.11.2
8.11.3
8.11.4
8.11.5
8.11.6
8.12
Preparing the TL881 and TL891 DLT MiniLibraries for
Shared Bus Usage ....................................................
8.12.1
8.12.1.1
8.12.1.2
8.12.1.3
8.12.1.4
8.12.2
8.12.2.1
8.12.2.1.1
8.12.2.1.2
8.12.2.2
8.12.2.2.1
8.12.2.2.2
8.12.2.2.3
8.12.2.2.4
8.13
Compaq ESL9326D Enterprise Library ........................... 8–73
8.13.1
8.13.2
8.13.3
Connecting the TL895 Tape Library to the Shared SCSI
Bus .................................................................
............................
8–48
8–48
Communications with the Host Computer ................... 8–50
MUC Switch Functions .........................................
8–50
Setting the MUC SCSI ID ...................................... 8–51
Tape Drive SCSI IDs ............................................
8–51
TL893 and TL896 Automated Tape Library Internal
Cabling .............................................................
8–52
Connecting the TL893 and TL896 Automated Tape
Libraries to the Shared SCSI Bus ............................. 8–54
8–57
TL881 and TL891 DLT MiniLibraries Overview ............ 8–57
TL881 and TL891 DLT MiniLibrary Tabletop Model .. 8–57
TL881 and TL891 MiniLibrary Rackmount
Components .................................................
8–58
TL881 and TL891 Rackmount Scalability ............... 8–59
DLT MiniLibrary Part Numbers .......................... 8–60
Preparing a TL881 or TL891 MiniLibrary for Shared SCSI
Bus Use ............................................................
8–61
Preparing a Tabletop Model or Base Unit for
Standalone Shared SCSI Bus Usage ..................... 8–61
Setting the Standalone MiniLibrary Tape Drive
SCSI ID ..................................................
8–62
Cabling the TL881 or TL891 DLT MiniLibrary .... 8–63
Preparing a TL881 or TL891 Rackmount MiniLibrary
for Shared SCSI Bus Usage ................................ 8–67
Cabling the Rackmount TL881 or TL891 DLT
MiniLibrary ............................................. 8–67
Configuring a Base Unit as a Slave to the
Expansion Unit ......................................... 8–70
Powering Up the TL881/TL891 DLT MiniLibrary . 8–71
Setting the SCSI IDs for a Rackmount TL881 or
TL891 DLT MiniLibrary ............................... 8–72
General Overview ................................................ 8–73
ESL9326D Enterprise Library Overview ..................... 8–74
Preparing the ESL9326D Enterprise Library for Shared
SCSI Bus Usage .................................................. 8–74
Contents ix
8.13.3.1
8.13.3.2
8.13.3.3
8.13.3.4
ESL9326D Enterprise Library Robotic and Tape Drive
Required Firmware .........................................
Library Electronics and Tape Drive SCSI IDs .......... 8–75
ESL9326D Enterprise Library Internal Cabling ....... 8–75
Connecting the ESL9326D Enterprise Library to the
Shared SCSI Bus ............................................
8–75
8–77
9 Configurations Using External Termination or Radial Connections
to Non-UltraSCSI Devices
9.1
9.1.1
9.1.2
9.1.2.1
9.1.2.2
9.2
9.3
9.3.1
9.3.2
9.3.2.1
9.3.2.2
9.4
9.4.1
9.4.1.1
9.4.1.2
9.4.1.3
9.4.2
9.4.2.1
9.4.2.2
9.4.2.3
9.4.3
9.4.3.1
Using SCSI Bus Signal Converters ................................ 9–2
Types of SCSI Bus Signal Converters ......................... 9–2
Using the SCSI Bus Signal Converters ....................... 9–3
DWZZA and DWZZB Signal Converter Termination .. 9–3
DS-BA35X-DA Termination ............................... 9–4
Terminating the Shared SCSI Bus ................................. 9–5
Overview of Disk Storage Shelves .................................. 9–8
BA350 Storage Shelf .............................................
BA356 Storage Shelf .............................................
Non-UltraSCSI BA356 Storage Shelf .................... 9–10
UltraSCSI BA356 Storage Shelf .......................... 9–13
Preparing the Storage for Configurations Using External
Termination ............................................................
Preparing BA350, BA356, and UltraSCSI BA356 Storage
Shelves for an Externally Terminated TruCluster Server
Configuration .....................................................
Preparing a BA350 Storage Shelf for Shared SCSI
Usage .......................................................... 9–15
Preparing a BA356 Storage Shelf for Shared SCSI
Usage .......................................................... 9–16
Preparing an UltraSCSI BA356 Storage Shelf for a
TruCluster Configuration .................................. 9–17
Connecting Storage Shelves Together ........................ 9–17
Connecting a BA350 and a BA356 for Shared SCSI
Bus Usage .................................................... 9–18
Connecting Two BA356s for Shared SCSI Bus Usage . 9–20
Connecting Two UltraSCSI BA356s for Shared SCSI
Bus Usage .................................................... 9–22
Cabling a Non-UltraSCSI RAID Array Controller to an
Externally Terminated Shared SCSI Bus .................... 9–24
Cabling an HSZ40 or HSZ50 in a Cluster Using
External Termination ....................................... 9–25
9–9
9–10
9–14
9–15
x Contents
9.4.3.2
9.4.4
Cabling an HSZ20 in a Cluster Using External
Termination ..................................................
Cabling an HSZ40 or HSZ50 RAID Array Controller in a
Radial Configuration with an UltraSCSI Hub ..............
10 Configuring Systems for External Termination or Radial
Connections to Non-UltraSCSI Devices
10.1
10.1.1
10.1.2
10.1.3
10.1.4
10.1.4.1
10.1.4.2
10.1.4.3
10.1.4.4
10.1.4.5
TruCluster Server Hardware Installation Using PCI SCSI
Adapters ................................................................
Radial Installation of a KZPSA-BB or KZPBA-CB Using
Internal Termination ............................................
Installing a KZPSA-BB or KZPBA-CB Using External
Termination .......................................................
Displaying KZPSA-BB and KZPBA-CB Adapters with the
show Console Commands ....................................... 10–9
Displaying Console Environment Variables and Setting
the KZPSA-BB and KZPBA-CB SCSI ID ..................... 10–13
Displaying KZPSA-BB and KZPBA-CB pk* or isp*
Console Environment Variables ........................... 10–13
Setting the KZPBA-CB SCSI ID .......................... 10–16
Setting KZPSA-BB SCSI Bus ID, Bus Speed, and
Termination Power .......................................... 10–17
KZPSA-BB and KZPBA-CB Termination Resistors .... 10–18
Updating the KZPSA-BB Adapter Firmware ........... 10–18
9–28
9–28
10–1
10–2
10–6
A Worldwide ID-to-Disk Name Conversion Table
Index
Examples
4–1
4–2
4–3
4–4
4–5
4–6
4–7
Displaying Configuration on an AlphaServer DS20 ............. 4–10
Displaying Devices on an AlphaServer DS20 .................... 4–12
Displaying Configuration on an AlphaServer 8200 .............. 4–13
Displaying Devices on an AlphaServer 8200 ..................... 4–13
Displaying the pk* Console Environment Variables on an
AlphaServer DS20 System .......................................... 4–15
Displaying Console Variables for a KZPBA-CB on an
AlphaServer 8x00 System ........................................... 4–16
Setting the KZPBA-CB SCSI Bus ID .............................. 4–17
Contents xi
5–1
6–1
6–2
6–3
6–4
6–5
6–6
6–7
7–1
7–2
7–3
7–4
10–1
10–2
10–3
10–4
10–5
10–6
10–7
10–8
10–9
Running the mc_cable Test ..............
Determine HSG80 Connection Names ............................
Setting Up the Mirrorset ............................................
Adding Units and Identifiers to the HSG80 Storagesets ....... 6–40
Displaying the UDID and WorldwideNames of Devices Known
to the Console .........................................................
Using the wwidmgr quickset Command to Set the Device Unit
Number .................................................................
Sample Fibre Channel Device Names ............................. 6–49
Setting Device Unit Number for Additional Member System . 6–55
Defining Hard Partitions with SCM Environment Variables .. 7–10
Turning Partition Power On ........................................ 7–12
Displaying AlphaServer GS160 System Information ........... 7–14
Displaying Console Serial Bus Information ...................... 7–16
Displaying Configuration on an AlphaServer 4100 .............. 10–9
Displaying Devices on an AlphaServer 4100 ..................... 10–10
Displaying Configuration on an AlphaServer 8200 .............. 10–11
Displaying Devices on an AlphaServer 8200 ..................... 10–12
Displaying the pk* Console Environment Variables on an
AlphaServer 4100 System ........................................... 10–13
Displaying Console Variables for a KZPBA-CB on an
AlphaServer 8x00 System ........................................... 10–15
Displaying Console Variables for a KZPSA-BB on an
AlphaServer 8x00 System ........................................... 10–15
Setting the KZPBA-CB SCSI Bus ID .............................. 10–16
Setting KZPSA-BB SCSI Bus ID and Speed ...................... 10–17
............................
5–13
6–31
6–37
6–45
6–47
Figures
1–1
1–2
1–3
1–4
1–5
1–6
1–7
xii Contents
Two-Node Cluster with Minimum Disk Configuration and No
Quorum Disk .......................................................... 1–5
Generic Two-Node Cluster with Minimum Disk Configuration
and Quorum Disk ..................................................... 1–6
Minimum Two-Node Cluster with UltraSCSI BA356 Storage
Unit ..................................................................... 1–8
Two-Node Cluster with Two UltraSCSI DS-BA356 Storage
Units .................................................................... 1–10
Two-Node Configurations with UltraSCSI BA356 Storage
Units and Dual SCSI Buses ......................................... 1–12
Cluster Configuration with HSZ70 Controllers in Transparent
Failover Mode ......................................................... 1–13
NSPOF Cluster Using HSZ70s in Multiple-Bus Failover
Mode .................................................................... 1–15
1–8
2–1
3–1
3–2
3–3
3–4
3–5
3–6
3–7
3–8
4–1
5–1
5–2
5–3
5–4
5–5
5–6
5–7
5–8
6–1
6–2
6–3
6–4
6–5
6–6
6–7
6–8
7–1
7–2
7–3
NSPOF Fibre Channel Cluster Using HSG80s in Multiple-Bus
Failover Mode .........................................................
PCI Backplane Slot Layout ..............
............................
VHDCI Trilink Connector (H8861-AA) ............................
DS-DWZZH-03 Front View ..........................................
DS-DWZZH-05 Rear View ...............
............................
1–16
2–2
3–8
3–10
3–14
DS-DWZZH-05 Front View .......................................... 3–15
Shared SCSI Bus with HSZ70 Configured for Transparent
Failover .................................................................
3–20
Shared SCSI Bus with HSZ80 Configured for Transparent
Failover .................................................................
3–21
TruCluster Server Configuration with HSZ70 in Multiple-Bus
Failover Mode .........................................................
3–24
TruCluster Server Configuration with HSZ80 in Multiple-Bus
Failover Mode .........................................................
3–25
KZPBA-CB Termination Resistors ................................. 4–18
Connecting Memory Channel Adapters to Hubs ................. 5–9
MC1 to MC2 Virtual Hub Rolling Upgrade ....................... 5–20
MC1 to MC2 Standard Hub Rolling Upgrade: Initial
Configuration ..........................................................
5–21
MC1 to MC2 Standard Hub Rolling Upgrade: First MC1
Module Replaced ......................................................
5–22
MC1 to MC2 Standard Hub Rolling Upgrade: Replace First
MC1 Adapter in Second System .................................... 5–23
MC1 to MC2 Standard Hub Rolling Upgrade: Replace Third
System Memory Channel Adapters ................................ 5–24
MC1 to MC2 Standard Hub Rolling Upgrade: Replace Second
MC1 in Second System ............................................... 5–25
MC1 to MC2 Standard Hub Rolling Upgrade: Final
Configuration .......................................................... 5–26
Point-to-Point Topology .............................................. 6–7
Fabric Topology ........................................................ 6–8
Arbitrated Loop Topology ............................................ 6–9
Fibre Channel Single Switch Transparent Failover
Configuration ......................................................... 6–10
Multiple-Bus NSPOF Configuration Number 1 .................. 6–12
Multiple-Bus NSPOF Configuration Number 2 .................. 6–13
Multiple-Bus NSPOF Configuration Number 3 .................. 6–14
A Simple Zoned Configuration ...................................... 6–15
Portion of QBB Showing I/O Riser Modules ...................... 7–3
Front View of Expansion and Primary PCI Drawers ............ 7–6
Rear View of Expansion and Primary PCI Drawers ............. 7–7
Contents xiii
8–1
8–2
8–3
8–4
8–5
8–6
8–7
8–8
8–9
8–10
8–11
8–12
8–13
8–14
8–15
8–16
8–17
8–18
8–19
8–20
9–1
9–2
9–3
9–4
9–5
9–6
9–7
9–8
9–9
9–10
9–11
9–12
9–13
9–14
TZ88N-VA SCSI ID Switches .......................................
8–2
Shared SCSI Buses with SBB Tape Drives ....................... 8–4
DS-TZ89N-VW SCSI ID Switches ..................................
8–7
Compaq 20/40 GB DLT Tape Drive Rear Panel .................. 8–11
Cabling a Shared SCSI Bus with a Compaq 20/40 GB DLT
Tape Drive .............................................................
8–13
Cabling a Shared SCSI Bus with a Compaq 40/80-GB DLT
Drive ....................................................................
8–16
Cabling a Shared SCSI Bus with a TZ885 ........................ 8–19
TZ887 DLT MiniLibrary Rear Panel ............................... 8–21
Cabling a Shared SCSI Bus with a TZ887 ........................ 8–22
TruCluster Server Cluster with a TL892 on Two Shared SCSI
Buses ....................................................................
8–28
TL890 and TL892 DLT MiniLibraries on Shared SCSI Buses . 8–32
TL894 Tape Library Four-Bus Configuration .................... 8–40
Shared SCSI Buses with TL894 in Two-Bus Mode .............. 8–42
TL895 Tape Library Internal Cabling ............................. 8–47
TL893 Three-Bus Configuration .................................... 8–53
TL896 Six-Bus Configuration ....................................... 8–54
Shared SCSI Buses with TL896 in Three-Bus Mode ............ 8–56
TL891 Standalone Cluster Configuration ......................... 8–66
TL891 DLT MiniLibrary Rackmount Configuration ............ 8–69
ESL9326D Internal Cabling ........................................ 8–76
Standalone SCSI Signal Converter ................................ 9–4
SBB SCSI Signal Converter ......................................... 9–4
DS-BA35X-DA Personality Module Switches .................... 9–5
BN21W-0B Y Cable ................................................... 9–7
HD68 Trilink Connector (H885-AA) ............................... 9–8
BA350 Internal SCSI Bus ........................................... 9–10
BA356 Internal SCSI Bus ........................................... 9–12
BA356 Jumper and Terminator Module Identification Pins ... 9–13
BA350 and BA356 Cabled for Shared SCSI Bus Usage ......... 9–19
Two BA356s Cabled for Shared SCSI Bus Usage ................ 9–21
Two UltraSCSI BA356s Cabled for Shared SCSI Bus Usage .. 9–23
Externally Terminated Shared SCSI Bus with Mid-Bus HSZ50
RAID Array Controllers ............................................. 9–26
Externally Terminated Shared SCSI Bus with HSZ50 RAID
Array Controllers at Bus End ....................................... 9–27
TruCluster Server Cluster Using DS-DWZZH-03, SCSI
Adapter with Terminators Installed, and HSZ50 ................ 9–30
xiv Contents
9–15
10–1
Tables
2–1
2–2
2–3
2–4
3–1
3–2
3–3
3–4
4–1
4–2
4–3
5–1
5–2
5–3
5–4
6–1
6–2
8–1
8–2
8–3
8–4
8–5
8–6
8–7
8–8
TruCluster Server Cluster Using KZPSA-BB SCSI Adapters,
a DS-DWZZH-05 UltraSCSI Hub, and an HSZ50 RAID Array
Controller ..............................................................
KZPSA-BB Termination Resistors .................................
RAID Controller Minimum Required Array Controller
Software ................................................................
RAID Controller SCSI IDs .......................................... 2–10
Supported SCSI Cables ..............................................
Supported SCSI Terminators and Trilink Connectors .......... 2–14
SCSI Bus Speeds ......................................................
SCSI Bus Segment Length .......................................... 3–6
DS-DWZZH UltraSCSI Hub Maximum Configurations ........ 3–11
Hardware Components Used in Configuration Shown in
Figure 3–5 Through Figure 3–8 ....................................
Planning Your Configuration ....................................... 4–3
Configuring TruCluster Server Hardware ........................ 4–6
Installing the KZPBA-CB for Radial Connection to a DWZZH
UltraSCSI Hub ........................................................
MC1 and MC1.5 Jumper Configuration ........................... 5–2
MC2 Jumper Configuration ......................................... 5–4
MC2 Linecard Jumper Configurations ............................ 5–5
Adding a Memory Channel Interconnect or Rolling from a
Dual, Redundant MC1 Interconnect to MC2 Interconnects .... 5–15
Telnet Session Default User Names for Fibre Channel
Switches ................................................................ 6–22
Converting Storageset Unit Numbers to Disk Names .......... 6–43
TZ88N-VA Switch Settings .......................................... 8–3
Hardware Components Used to Create the Configuration
Shown in Figure 8–2 ................................................. 8–4
DS-TZ89N-VW Switch Settings .................................... 8–7
Hardware Components Used to Create the Configuration
Shown in Figure 8–5 ................................................. 8–13
Hardware Components in the Configuration in Figure 8–6.... 8–16
Hardware Components Used to Create the Configuration
Shown in Figure 8–6 ................................................. 8–19
Hardware Components Used to Create the Configuration
Shown in Figure 8–9 ................................................. 8–28
Hardware Components Used to Create the Configuration
Shown in Figure 8–10 ................................................ 8–33
9–31
10–18
2–10
2–13
3–5
3–21
4–9
Contents xv
8–9
8–10
8–11
8–12
8–13
8–14
8–15
8–16
8–17
8–18
8–19
8–20
8–21
TL894 Default SCSI ID Settings ...................................
8–37
Hardware Components Used to Create the Configuration
Shown in Figure 8–12 ................................................
TL895 Default SCSI ID Settings ...................................
MUC Switch Functions ..............................................
MUC SCSI ID Selection .............................................
TL893 Default SCSI IDs .............................................
TL896 Default SCSI IDs .............................................
8–42
8–44
8–50
8–51
8–51
8–52
Hardware Components Used to Create the Configuration
Shown in Figure 8–16 ................................................
8–56
TL881 and TL891 MiniLibrary Performance and Capacity
Comparison ............................................................
8–60
DLT MiniLibrary Part Numbers ................................... 8–60
Hardware Components Used to Create the Configuration
Shown in Figure 8–17 ................................................
8–66
Hardware Components Used to Create the Configuration
Shown in Figure 8–18 ................................................
8–70
Shared SCSI Bus Cable and Terminator Connections for the
ESL9326D Enterprise Library ...................................... 8–77
9–1
9–2
9–3
9–4
Hardware Components Used for Configuration Shown in
Figure 8–9 and Figure 8–10 .........................................
Hardware Components Used for Configuration Shown in
Figure 9–11 ............................................................
Hardware Components Used for Configuration Shown in
Figure 8–12 and Figure 8–13 .......................................
Hardware Components Used in Configuration Shown in
9–20
9–24
9–27
Figure 9–14 ............................................................ 9–30
10–1
Configuring TruCluster Server Hardware for Use with a PCI
SCSI Adapter .......................................................... 10–2
10–2
Installing the KZPSA-BB or KZPBA-CB for Radial Connection
to a DWZZH UltraSCSI Hub ........................................ 10–4
10–3
Installing a KZPSA-BB or KZPBA-CB for Use with External
Termination ............................................................ 10–7
A–1 Converting Storageset Unit Numbers to Disk Names .......... A–1
xvi Contents
This manual describes how to set up and maintain the hardware
configuration for a TruCluster™ Server cluster.
Audience
This manual is for system administrators who will set up and configure the
hardware before installing the TruCluster Server software. The manual
assumes that you are familiar with the tools and methods needed to
maintain your hardware, operating system, and network.
Organization
This manual contains 10 chapters, an appendix, and an index. The
organization of this manual has been restructured to provide a more
streamlined manual. The chapters that contained information on SCSI bus
requirements and configuration, and configuring hardware, in previous
versions of this manual have been split up into two sets of two chapters each.
One set (Chapters 3 and 4) covers the UltraSCSI hardware and is geared
towards radial configurations. The other set (Chapters 9 and 10) covers
configurations using either external termination or radial connection to
non-UltraSCSI devices. A brief description of the contents follows:
About This Manual
Chapter 1 Introduces the TruCluster Server product and provides an overview
Chapter 2
Chapter 3 Contains information about setting up a shared SCSI bus, SCSI
Chapter 4 Describes how to prepare systems for a TruCluster Server
Chapter 5 Describes how to set up the Memory Channel cluster interconnect,
Chapter 6
of setting up TruCluster Server hardware.
Describes hardware requirements and restrictions.
bus requirements, and how to connect storage to a shared SCSI bus
using the latest UltraSCSI products (DS-DWZZH UltraSCSI hubs,
and HSZ70 and HSZ80 RAID array controllers).
configuration, and how to connect host bus adapters to shared
storage using the DS-DWZZH UltraSCSI hubs and the newest
RAID array controllers (HSZ70 and HSZ80).
and how to upgrade Memory Channel interconnects.
Provides an overview of Fibre Channel and describes how
to set up Fibre Channel hardware.
About This Manual xvii
Chapter 7 Describes the use of AlphaServer™ GS80, GS160, or GS320 hardware
partitions in a TruCluster Server configuration.
Chapter 8 Describes how to configure a shared SCSI bus for tape drive,
tape loader, or tape library usage.
Chapter 9 Describes the requirements for a shared SCSI bus using externally
terminated configurations and radial configurations using
non-UltraSCSI RAID array controllers.
Chapter 10 Describes how to prepare systems and host bus adapters for the
TruCluster Server configurations described in Chapter 9.
Appendix A Provides a blank table to use to convert from the HSG80 unit
numbers to
Fibre Channel TruCluster Server configuration.
Related Documents
Consult the following manuals for assistance in TruCluster Server
installation, administration, and programming tasks:
• TruCluster Server Software Product Description (SPD) — The
comprehensive description of the TruCluster Server Version 5.1 product.
You can find the latest version of the SPD and other TruCluster Server
documentation at the following URL:
http://www.tru64unix.compaq.com/faqs/publications/pub_page/cluster_list.html
/dev/disk/dskn and device names for an actual
• Release Notes — Provides important information about TruCluster
Server Version 5.1.
• Technical Overview — Provides an overview of the TruCluster Server
technology.
• Software Installation — Describes how to install the TruCluster Server
product.
• Cluster Administration — Describes cluster-specific administration
tasks.
• Highly Available Applications — Describes how to deploy applications on
a TruCluster Server cluster.
Consult the following AlphaServer GS80/160/320 documentation to assist
you in configuring an AlphaServer GS80, GS160, or GS320 system in a
TruCluster Server configuration:
• Installation Guide
• System Management Console Installation and User’s Guide
• User’s Guide
• Firmware Reference Manual
xviii About This Manual
The StorageWorks™ UltraSCSI Configuration Guidelines document
provides guidelines regarding UltraSCSI configurations.
For information about setting up a RAID subsystem, see the following
manuls as appropriate for your configuration:
• DEC RAID Subsystem User’s Guide
• HS Family of Array Controllers User’s Guide
• RAID Array 310 Configuration and Maintenance Guide User’s Guide
• Configuring Your StorageWorks Subsystem HSZ40 Array Controllers
HSOF Version 3.0
• Getting Started RAID Array 450 V5.4 for Compaq Tru64™ UNIX
Installation Guide
• HSZ70 Array Controller HSOF Version 7.0 Configuration Manual
• HSZ80 Array Controller ACS Version 8.2
• Compaq StorageWorks HSG80 Array Controller ACS Version 8.5
Configuration Guide
• Compaq StorageWorks HSG80 Array Controller ACS Version 8.5 CLI
Reference Guide
• MA6000 HSG60 Array Controller ACS Version 8.5 Solution Software for
Compaq Tru64 UNIX Installation and Configuration Guide
• Compaq StorageWorks HSG60/HSG80 Array Controller ACS Version 8.5
Maintenance and Service Guide
• MA6000 HSG60 Array Controller ACS Version 8.5 Solution Software for
Compaq Tru64 UNIX Installation and Configuration Guide
• Compaq StorageWorks Release Notes RA8000/ESA12000 and
MA8000/EMA12000 Solution Software V8.5b for Tru64 UNIX
• Compaq StorageWorks Modular Array Configuration Guide
• Model 2100 and 2200 Ultra SCSI Controller Enclosures User Guide
• Compaq StorageWorks Enclosure 4200 Family LVD Disk Enclosure
User Guide
• Wwidmgr User’s Manual
Consult the following documentation for other Fibre Channel storage
information:
• KGPSA-BC PCI-to-Optical Fibre Channel Host Adapter User Guide
• 64-Bit PCI-to-Fibre Channel Host Bus Adapter User Guide
• Compaq StorageWorks Fibre Channel Storage Switch User’s Guide
About This Manual xix
• Compaq StorageWorks SAN Switch 8 Installation and Hardware Guide
• Compaq StorageWorks SAN Switch 16 Installation and Hardware Guide
• Compaq StorageWorks Fibre Channel SAN Switch 8-EL Installation
and Hardware Guide
• Compaq StorageWorks Fibre Channel SAN Switch Management Guide
• Compaq StorageWorks SAN Switch Fabric Operating System
Management Guide
• Fibre Channel Storage Hub 7 Installation Guide
• Fibre Channel Storage Hub 7 Rack Mounting Installation Card
For information about the tape devices, see the following manuals:
• TZ88 DLT Series Tape Drive Owner’s Manual
• TZ89 DLT Series Tape Drive User’s Guide
• TZ885 Model 100/200 GB DLT5-Cartridge MiniLibrary Owner’s Manual
• TZ887 Model 140/280 GB DLT7-Cartridge MiniLibrary Owner’s Manual
• TL881 MiniLibrary System User’s Guide
• TL881 MiniLibrary Drive Upgrade Procedure
• Pass-Through Expansion Kit Installation Instructions
• TL891 MiniLibrary System User’s Guide
• TL81X/TL894 Automated Tape Library for DLT Cartridges Facilities
Planning and Installation Guide
• TL81X/TL894 Automated Tape Library for DLT Cartridges Diagnostic
Software User’s Manual
• TL895 DLT Tape Library Facilities Planning and Installation Guide
• TL895 DLT Library Operator’s Guide
• TL895 DLT Tape Library Diagnostic Software User’s Manual
• TL895 Drive Upgrade Instructions
• TL82X/TL893/TL896 Automated Tape Library for DLT Cartridges
Facilities Planning and Installation Guide
• TL82X/TL893/TL896 Automated Tape Library for DLT Cartridges
Operator’s Guide
• TL82X/TL893/TL896 Automated Tape Library for DLT Cartridges
Diagnostic Software User’s Manual
• TL82X Cabinet-to-Cabinet Mounting Instructions
• TL82X/TL89X MUML to MUSL Upgrade Instructions
xx About This Manual
• For more information on the ESL9326D Enterprise Library, see
the following Compaq StorageWorks ESL9000 Series Tape Library
documentation:
Unpacking and Installation Guide
–
– Reference Guide
– Maintenance and Service Guide
– Tape Drive Upgrade Guide
The Golden Eggs Visual Configuration Guide provides configuration
diagrams of workstations, servers, storage components, and clustered
systems. It is available on line in PostScript and Portable Document Format
(PDF) formats at:
http://www.compaq.com/info/golden-eggs
At this URL you will find links to individual system, storage, or cluster
configurations. You can order the document through the Compaq Literature
Order System (LOS) as order number EC-R026B-36.
In addition, have available the following manuals from the Tru64 UNIX
documentation set:
• Installation Guide
• Release Notes
• System Administration
• Network Administration
Also have available the hardware documentation for the systems, SCSI
controllers, disk storage shelves or RAID controllers, and any other
hardware you plan to install.
Documentation for the following optional software products will be useful if
you intend to use these products with TruCluster Server:
• Compaq Analyze
• DECevent
• Logical Storage Manager (LSM)
• NetWorker
• Advanced File System (AdvFS) Utilities
• Performance Manager
About This Manual xxi
Reader’s Comments
Compaq welcomes any comments and suggestions you have on this and
other Tru64 UNIX manuals.
You can send your comments in the following ways:
• Fax: 603-884-0120 Attn: UBPG Publications, ZKO3-3/Y32
• Internet electronic mail:
readers_comment@zk3.dec.com
A Reader’s Comment form is located on your system in the following
location:
/usr/doc/readers_comment.txt
• Mail:
Compaq Computer Corporation
UBPG Publications Manager
ZKO3-3/Y32
110 Spit Brook Road
Nashua, NH 03062-2698
A Reader’s Comment form is located in the back of each printed manual.
The form is postage paid if you mail it in the United States.
Please include the following information along with your comments:
• The full title of the book and the order number. (The order number is
printed on the title page of this book and on its back cover.)
• The section numbers and page numbers of the information on which
you are commenting.
• The version of Tru64 UNIX that you are using.
• If known, the type of processor that is running the Tru64 UNIX software.
The Tru64 UNIX Publications group cannot respond to system problems
or technical support inquiries. Please address technical questions to your
local system vendor or to the appropriate Compaq technical support office.
Information provided with the software media explains how to send problem
reports to Compaq.
Conventions
The following typographical conventions are used in this manual:
#
% cat
xxii About This Manual
A number sign represents the superuser prompt.
Boldface type in interactive examples indicates
typed user input.
file
Italic (slanted) type indicates variable values,
placeholders, and function argument names.
.
.
.
A vertical ellipsis indicates that a portion of an
example that would normally be present is not
shown.
cat
(1)
A cross-reference to a reference page includes
the appropriate section number in parentheses.
For example, cat
(1) indicates that you can find
information on the cat command in Section 1 of
the reference pages.
cluster Bold text indicates a term that is defined in the
glossary.
About This Manual xxiii
This chapter introduces the TruCluster Server product and some basic
cluster hardware configuration concepts.
Subsequent chapters describe how to set up and maintain TruCluster Server
hardware configurations. See the TruCluster Server Software Installation
manual for information about software installation; see the TruCluster
Server Cluster Administration manual for detailed information about setting
up member systems and highly available applications.
1.1 The TruCluster Server Product
TruCluster Server, the newest addition to the Compaq Tru64 UNIX
TruCluster Software products family, extends single-system management
capabilities to clusters. It provides a clusterwide namespace for files and
directories, including a single root file system that all cluster members
share. It also offers a cluster alias for the Internet protocol suite (TCP/IP) so
that a cluster appears as a single system to its network clients.
1
Introduction
TruCluster Server preserves the availability and performance features found
in the earlier TruCluster products:
• Like the TruCluster Available Server Software and TruCluster
Production Server products, TruCluster Server lets you deploy highly
available applications that have no embedded knowledge that they are
executing in a cluster. They can access their disk data from any member
in the cluster.
• Like the TruCluster Production Server Software product, TruCluster
Server lets you run components of distributed applications in parallel,
providing high availability while taking advantage of cluster-specific
synchronization mechanisms and performance optimizations.
TruCluster Server augments the feature set of its predecessors by allowing
all cluster members access to all file systems and all storage in the cluster,
regardless of where they reside. From the viewpoint of clients, a TruCluster
Server cluster appears to be a single system; from the viewpoint of a system
administrator, a TruCluster Server cluster is managed as if it were a single
system. Because TruCluster Server has no built-in dependencies on the
architectures or protocols of its private cluster interconnect or shared storage
Introduction 1–1
interconnect, you can more easily alter or expand your cluster’s hardware
configuration as newer and faster technologies become available.
1.2 Memory Requirements
Cluster members require a minimum of 128 MB of memory.
1.3 Minimum Disk Requirements
This section provides an overview of the minimum file system or disk
requirements for a two-node cluster. For more information on the amount
of space required for each required cluster file system, see the TruCluster
Server
1.3.1 Disks Needed for Installation
You need to allocate disks for the following uses:
• One or more disks to hold the Tru64 UNIX operating system. The disk(s)
• One or more disks on a shared SCSI bus to hold the clusterwide root (
• One disk per member, normally on a shared SCSI bus, to hold member
Software Installation manual.
are either private disk(s) on the system that will become the first cluster
member, or disk(s) on a shared bus that the system can access.
/usr, and /var AdvFS file systems.
boot partitions.
/),
• Optionally, one disk on a shared SCSI bus to act as the quorum disk. See
Section 1.3.1.4, and for a more detailed discussion of the quorum disk,
see the TruCluster Server Cluster Administration manual.
The following sections provide more information about these disks.
Figure 1–1 shows a generic two-member cluster with the required file
systems.
1.3.1.1 Tru64 UNIX Operating System Disk
The Tru64 UNIX operating system is installed using AdvFS file systems on
one or more disks on the system that will become the first cluster member.
For example:
dsk0a root_domain#root
dsk0g usr_domain#usr
dsk0h var_domain#var
The operating system disk (Tru64 UNIX disk) cannot be used as a
clusterwide disk, a member boot disk, or as the quorum disk.
1–2 Introduction
Because the Tru64 UNIX operating system will be available on the first
cluster member, in an emergency, after shutting down the cluster, you have
the option of booting the Tru64 UNIX operating system and attempting to
fix the problem. See the TruCluster Server
for more information.
1.3.1.2 Clusterwide Disk(s)
When you create a cluster, the installation scripts copy the Tru64 UNIX
root (/), /usr, and /var file systems from the Tru64 UNIX disk to the disk
or disks you specify.
We recommend that the disk or disks used for the clusterwide file systems
be placed on a shared SCSI bus so that all cluster members have access to
these disks.
During the installation, you supply the disk device names and partitions
that will contain the clusterwide root (/), /usr, and /var file systems. For
example, dsk3b, dsk4c, and dsk3g:
dsk3b cluster_root#root
dsk4c cluster_usr#usr
dsk3g cluster_var#var
The /var fileset cannot share the cluster_usr domain, but must be a
separate domain, cluster_var. Each AdvFS file system must be a separate
partition; the partitions do not have to be on the same disk.
Cluster Administration manual
If any partition on a disk is used by a clusterwide file system, only
clusterwide file systems can be on that disk. A disk containing a clusterwide
file system cannot also be used as the member boot disk or as the quorum
disk.
1.3.1.3 Member Boot Disk
Each member has a boot disk. A boot disk contains that member’s boot,
swap, and cluster-status partitions. For example, dsk1 is the boot disk for
the first member and dsk2 is the boot disk for the second member:
dsk1 first member’s boot disk [pepicelli]
dsk2 second member’s boot disk [polishham]
The installation scripts reformat each member’s boot disk to contain three
partitions: an a partition for that member’s root (/) file system, a b partition
for swap, and an h partition for cluster status information. (There are no
/usr or /var file systems on a member’s boot disk.)
A member boot disk cannot contain one of the clusterwide root (/), /usr,
and /var file systems. Also, a member boot disk cannot be used as the
quorum disk. A member disk can contain more than the three required
Introduction 1–3
partitions. You can move the swap partition off the member boot disk. See
the TruCluster Server Cluster Administration manual for more information.
1.3.1.4 Quorum Disk
The quorum disk allows greater availability for clusters consisting of two
members. Its h partition contains cluster status and quorum information.
See the TruCluster Server Cluster Administration manual for a discussion of
how and when to use a quorum disk.
The following restrictions apply to the use of a quorum disk:
• A cluster can have only one quorum disk.
• The quorum disk should be on a shared bus to which all cluster members
are directly connected. If it is not, members that do not have a direct
connection to the quorum disk may lose quorum before members that
do have a direct connection to it.
• The quorum disk must not contain any data. The clu_quorum command
will overwrite existing data when initializing the quorum disk. The
integrity of data (or file system metadata) placed on the quorum disk
from a running cluster is not guaranteed across member failures.
This means that the member boot disks and the disk holding the
clusterwide root (/) cannot be used as quorum disks.
• The quorum disk can be small. The cluster subsystems use only 1 MB
of the disk.
• A quorum disk can have either 1 vote or no votes. In general, a quorum
disk should always be assigned a vote. You might assign an existing
quorum disk no votes in certain testing or transitory configurations,
such as a one-member cluster (in which a voting quorum disk introduces
a second point of failure).
• You cannot use the Logical Storage Manager (LSM) on the quorum disk.
1.4 Generic Two-Node Cluster
This section describes a generic two-node cluster with the minimum disk
layout of four disks. Note that additional disks may be needed for highly
available applications. In this section, and the following sections, the type
of PCI SCSI bus adapter is not significant. Also, although an important
consideration, SCSI bus cabling, including Y cables or trilink connectors,
termination, and the use of UltraSCSI hubs are not considered at this time.
1–4 Introduction
Figure 1–1 shows a generic two-node cluster with the minimum number
of disks.
• Tru64 UNIX disk
• Clusterwide root (
/), /usr, and /var
• Member 1 boot disk
• Member 2 boot disk
A minimum configuration cluster may have reduced availability due to the
lack of a quorum disk. As shown, with only two-member systems, both
systems must be operational to achieve quorum and form a cluster. If only
one system is operational, it will loop, waiting for the second system to boot
before a cluster can be formed. If one system crashes, you lose the cluster.
Figure 1–1: Two-Node Cluster with Minimum Disk Configuration and No
Quorum Disk
Network
Member
System
1
PCI SCSI
Adapter
Memory Channel
Tru64
UNIX
Disk
Member
System
2
PCI SCSI
Adapter
Shared SCSI Bus
Cluster File
System
root (/)
/usr
/var
Member 1
root (/)
swap
Member 2
root (/)
swap
ZK-1587U-AI
Figure 1–2 shows the same generic two-node cluster as shown in Figure 1–1,
but with the addition of a quorum disk. By adding a quorum disk, a cluster
may be formed if both systems are operational, or if either of the systems
and the quorum disk is operational. This cluster has a higher availability
than the cluster shown in Figure 1–1. See the TruCluster Server Cluster
Introduction 1–5
Administration manual for a discussion of how and when to use a quorum
disk.
Figure 1–2: Generic Two-Node Cluster with Minimum Disk Configuration
and Quorum Disk
Network
Member
System
1
PCI SCSI
Adapter
Memory Channel
Tru64
UNIX
Disk
Member
System
2
PCI SCSI
Adapter
Shared SCSI Bus
Cluster File
System
root (/)
/usr
/var
Member 1
root (/)
swap
Member 1
root (/)
swap
Quorum
ZK-1588U-AI
1.5 Growing a Cluster from Minimum Storage to a NSPOF
Cluster
The following sections take a progression of clusters from a cluster with
minimum storage to a no-single-point-of-failure (NSPOF) cluster; a cluster
where one hardware failure will not interrupt the cluster operation:
• A cluster with minimum storage for highly available applications
(Section 1.5.1).
• A cluster with more storage, but the single SCSI bus is a single point
of failure (Section 1.5.2).
• Adding a second SCSI bus allows the use of LSM to mirror the /usr and
/var file systems and data disks. However, as LSM cannot mirror the
root (/), member system boot, swap, or quorum disks, so full redundancy
is not achieved (Section 1.5.3).
1–6 Introduction
Loading...