Sun Microsystems 4500-3 User Manual

Cluster Platform 4500/3

User’s Guide

A product from the SunTone™Platforms portfolio

Sun Microsystems, Inc. 901 San Antonio Road Palo Alto, CA 94303-4900 U.S.A. 650-960-1300

Part No. 816-0445-11 July 2001, Revision A

Send comments about this document to: docfeedback@sun.com

Copyright 2001Sun Microsystems, Inc.,901 SanAntonio Road,Palo Alto,CA 94303-4900U.S.A. Allrights reserved. This product ordocument isdistributed underlicenses restrictingits use,copying, distribution,and decompilation.No partof thisproduct or

document may be reproduced inany formby anymeans withoutprior writtenauthorization ofSun andits licensors,if any.Third-party software,including fonttechnology,is copyrighted and licensed fromSun suppliers.

Parts of the product maybe derivedfrom BerkeleyBSD systems,licensed fromthe University of California. UNIX is a registered trademarkin the U.S. and other countries, exclusively licensed through X/OpenCompany,Ltd.

Sun, Sun Microsystems,the Sunlogo, AnswerBook2,docs.sun.com, SolsticeDiskSuite, OpenBoot,Sun Enterprise,BluePrints, JumpStart,Sun StorEdge, Netra, Solaris, and SunTone are trademarks,registered trademarks,or service marks of Sun Microsystems, Inc. in theU.S. andother countries. All SPARC trademarks areused underlicense andare trademarksor registeredtrademarksof SPARC International, Inc. in theU.S. and other countries. Products bearingSPARCtrademarks arebased upon an architecture developedby SunMicrosystems, Inc.Netscape Navigator is a trademark or registered trademarkof NetscapeCommunications Corporation.Oracle isa registeredtrademarkof Oracle Corporation.

The OPEN LOOK and Sun™ Graphical User Interface was developed bySun Microsystems,Inc. forits usersand licensees.Sun acknowledges the pioneering effortsof Xeroxin researchingand developing the concept of visual orgraphical userinterfaces forthe computerindustry.Sun holds a non-exclusive license fromXerox tothe XeroxGraphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements.

Federal Acquisitions: CommercialSoftware—Government UsersSubject toStandard License TermsandConditions. DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,

INCLUDING ANY IMPLIED WARRANTYOF MERCHANTABILITY,FITNESS FOR A PARTICULARPURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Copyright 2001 Sun Microsystems, Inc.,901 SanAntonio Road,Palo Alto,CA 94303-4900Etats-Unis. Tousdroits réservés. Ce produit oudocument estdistribué avecdes licencesqui enrestreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune

partie de ce produit oudocument nepeut êtrereproduitesous aucuneforme, parquelque moyenque cesoit, sansl’autorisation préalableet écrite de Sun et de ses bailleurs de licence, s’il y en a.Le logicieldétenu pardes tiers,et quicomprend latechnologie relativeaux policesde caractères,est protégépar uncopyright etlicencié pardes fournisseursde Sun.

Des parties de ce produitpourront êtredérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque déposée aux Etats-Unis et dans d’autres payset licenciéeexclusivement parX/Open Company,Ltd.

Sun, Sun Microsystems,le logoSun, AnswerBook2,docs.sun.com, SolsticeDiskSuite, OpenBoot,Sun Enterprise,BluePrints, JumpStart,Sun StorEdge, Netra, Solaris, et SunTone sont des marques defabrique oudes marquesdéposées, oumarques de service, de Sun Microsystems, Inc. aux Etats-Unis et dans d’autrespays. Toutesles marquesSPARCsont utilisées sous licence et sont des marques defabrique oudes marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autres pays.Les produitsportant lesmarques SPARCsont baséssur une architecturedéveloppée parSun Microsystems,Inc. Netscape Navigator est une marque deNetscape CommunicationsCorporation auxEtatsUnis et dans d’autres pays.in theUnited Statesand othercountries.

L’interfaced’utilisation graphique OPEN LOOK et Sun™ a été développéepar SunMicrosystems, Inc.pour sesutilisateurs etlicenciés. Sun reconnaîtles effortsde pionniersde Xeroxpour la rechercheet ledéveloppement duconcept desinterfaces d’utilisationvisuelle ougraphique pour l’industrie de l’informatique. Sun détient une licence non exclusive deXerox surl’interface d’utilisationgraphique Xerox,cette licence couvrant également les licenciés de Sun qui mettent en place l’interfaced’utilisation graphiqueOPEN LOOKet quien outrese conformentaux licences écrites de Sun.

LA DOCUMENTATIONEST FOURNIE “EN L’ETAT” ET TOUTESAUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENTEXCLUES, DANSLA MESUREAUTORISEE PARLA LOIAPPLICABLE, YCOMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

Please

Recycle

Preface ix

Related Documentation ix Typographic Conventions x Shell Prompts xi Accessing Sun Documentation Online xi Ordering Sun Documentation xi Sun Welcomes Your Comments xii

1. Introduction 1

Tools for Reading Online Documentation 1

Netscape Navigator Browser 2 AnswerBook2 Server 2 Adobe Acrobat Reader 2 The man Command 2

Solaris Documentation 3

2. The Cluster Platform 4500/3 System 5

Your Integrated System 6

Task Overview 6 Hardware Components 8

iii

Connectivity 9 Network Administration 9 Miscellaneous Hardware 9

Software Components 10

Cluster Platform Component Location 13

Power and Heating Requirements 15

Cabling the System 16 Customizing the Cluster

Platform 4500/3 18

▼ Customizing the Terminal Concentrator 18 ▼ Starting the Cluster Console 38 ▼ Installing the Software Stack on Both Cluster Nodes 41

Cluster Platform 4500/3 Recovery 42

Before You Begin Recovery 42 Recovery CD-ROMs 43

▼ Installing the Recovery CD 43

Completing the Recovery Process 46

A. Laptop Settings to Access Monitor Mode 47

▼ To Access the Terminal Concentrator from a Laptop 47

B. Console Output for the Automatic Software Install on the Cluster Nodes 51

C. Connections Within the Cluster

Platform 4500/3 65

D. Troubleshooting the Cluster Platform 4500/3 Installation 69

iv Cluster Platform 4500/3 User’s Guide • July 2001

Figures

FIGURE 2-1 Ethernet Address on the Disk Array Pull Tab 7 FIGURE 2-2 Cluster Platforms I/O Board Placement 10 FIGURE 2-3 Cluster Platform Interconnections and NAFO 12 FIGURE 2-4 Cluster Platform Rack Placement 14 FIGURE 2-5 Cluster Platform Internal Cabling 17 FIGURE 2-6 Cluster Control Panel Window 39 FIGURE 2-7 Cluster Nodes Console Windows 40 FIGURE 2-8 Cluster Console Window 41

vi Cluster Platform 4500/3 User’s Guide • July 2001

Tables

TABLE 2-1 Ethernet IP Address Worksheet 8 TABLE 2-2 Cluster Platform Rack Components 15 TABLE 2-3 Power and Heat Requirements for Cluster Platforms 15 TABLE 2-4 Cluster Platform Cables 16 TABLE C-1 Boot Disk to Server Connections 65

TABLE C-2 Disk Array to Hub Connections 66 TABLE C-3 FC-AL Hub to Server Connections 66 TABLE C-4 Management Server Connections 66 TABLE C-5 Terminal Concentrator to Management Server and Cluster Nodes Connections 67 TABLE C-6 Node to Node Connections 67 TABLE C-7 Ethernet Cables to Production Network Connections 67 TABLE C-8 Ethernet Hub Connections 68

vii

viii Cluster Platform 4500/3 User’s Guide • July 2001

Preface

Cluster Platform 4500/3 provides self-sustained building blocks integrated through Sun Cluster technology to support highly available applications.

Typographic Conventions

Typeface or Symbol

AaBbCc123 The names of commands, files,

AaBbCc123 What you type, when

AaBbCc123 Book titles, new words or terms,

x Cluster Platform 4500/3 User’s Guide • July 2001

Meaning Examples

and directories; on-screen computer output

contrasted with on-screen computer output

words to be emphasized

Command-line variable; replace with a real name or value

Edit your .login file. Use ls -a to list all files.

% You have mail. % su

Password:

Read Chapter 6 in the User’s Guide. These are called class options. You must be superuser to do this.

To delete a file, type rm filename.

Shell Prompts

Shell Prompt

C shell machine_name% C shell superuser machine_name# Bourne shell and Korn shell $ Bourne shell and Korn shell superuser #

Accessing Sun Documentation Online

A broad selection of Sun system documentation is located at:

http://www.sun.com/products-n-solutions/hardware/docs

A complete set of Solaris documentation and many other titles are located at:

http://docs.sun.com

Ordering Sun Documentation

Fatbrain.com, an Internet professional bookstore, stocks select product documentation from Sun Microsystems, Inc.

For a list of documents and how to order them, visit the Sun Documentation Center on Fatbrain.com at:

http://www.fatbrain.com/documentation/sun

Sun Welcomes Your Comments

Sun is interested in improving its documentation and welcomes your comments and suggestions. You can email your comments to Sun at:

docfeedback@sun.com

Please include the part number (816-0445-10) of your document in the subject line of your email.

xii Cluster Platform 4500/3 User’s Guide • July 2001

CHAPTER

Introduction

This manual describes how to customize your Cluster Platform 4500/3 system. It is intended for the Solaris System Administrator who is familiar with the Solaris™ operating environment, Solstice DiskSuite™ software, and Sun Cluster software. For specific information about configuring disksets, disk volumes, file systems, and data services, refer to the Sun Cluster 3.0 documentation.

This system is shipped with hardcopy documentation for the Sun Enterprise™ 4500 system and the Sun StorEdge™ T3 array. The documentation for the Solaris software and the Sun Cluster 3.0 software are provided in online format.

Tools for Reading Online Documentation

The following tools for reading online documentation are preinstalled on the system:

■ Netscape Navigator™

■ AnswerBook2™

■ Acrobat Reader

■ man command

Netscape Navigator Browser

Use the Netscape browser to read documentation provided as HTML files, view the output from an AnswerBook2 server, or read Sun product documentation at:

http://docs.sun.com.

The Netscape browser can be downloaded from the following location:

http://www.netscape.com

AnswerBook2 Server

The AnswerBook2 server software processes sets of online manuals into content that you can access, search, and view through the Netscape Navigator browser.

After the AnswerBook2 software is installed on the management server, it can be accessed at:

http://name_of_management_server:8888

Note – Review “Tools for Reading Online Documentation” on page 1 for additional

details about AnswerBook2 server installation.

Adobe Acrobat Reader

The Adobe Acrobat Reader (acroread 4.0) software enables you to view documentation in PDF format. You can download it from the following location:

http://www.adobe.com

The man Command

The man command displays documentation about individual commands and files in the Solaris operating environment and other software products in the Cluster Platform 4500/3. If the directory that contains man page files for a product is not automatically accessed by the man command, you can add that location to the MANPATH environmental variable included in the .profile file (when using Bourne Shell, or Korn Shell environment), or the .cshrc file (when using the C Shell environment).

2 Cluster Platform 4500/3 User’s Guide • July 2001

Solaris Documentation

AnswerBook™ documentation about the Solaris 8 10/00 Operating Environment is included on the Solaris™ 8 Documentation CD. The following document collections are included:

The /opt/answerbooks/english/solaris_8 directory contains the following document collections:

■ SUNWaadm—Solaris 8 System Administrator Collection

■ SUNWabsdk—Solaris 8 Software Developer Collection

■ SUNWabe—Solaris 8 User Collection

■ SUNWaman—Solaris 8 Reference Manual Collection

■ SUNWdtad—Solaris 8 Common Desktop Environment Developer Collection

■ SUNWinab—Solaris 8 Installation Collection

■ SUNWolrn—Solaris 8 Release Documents Collection

■ SUNWopen—OpenBoot Collection

■ SUNWakcs—KCMS Collection

Chapter 1 Introduction 3

4 Cluster Platform 4500/3 User’s Guide • July 2001

CHAPTER

The Cluster Platform 4500/3 System

The Cluster Platform 4500/3 system provides self-sustained platforms, integrated through the Sun Cluster technology, to support highly available applications. This two-node cluster system with shared, mirrored FC-AL storage can be used to implement a highly available file server, web server, mail server, or Oracle server.

Sun Cluster 3.0 provides global file systems, global devices, and scalable services. These features allow independent cluster nodes, running independent Solaris operating environment instances, to run distributed applications while providing client access through a single IP address.

Note – This Cluster platform only provides a basic cluster environment. Data

services must be installed and configured by the customer.

database

Your Integrated System

Your system includes a two-node cluster with shared, mirrored storage, a terminal concentrator, and a management server. The Sun StorEdge T3 arrays are connected to two FC-AL hubs. An Ethernet hub provides connection to the management server and Sun StorEdge T3 arrays. These components are cabled to provide redundant cluster interconnect between nodes, and to provide access to shared storage and production networks.

For more information on the T3 arrays, refer to “Related Documentation” on page ix. The management server is the repository for software and patches that are loaded on

the system. The management server provides access to the cluster console, and it functions as a JumpStart™ server (installation server) for the cluster nodes.

Note – The management server has sufficient CPU power and memory to

implement a Sun™ Management Center server to monitor the cluster nodes, if advanced system monitoring is required.

Task Overview

To integrate your Cluster Platform 4500/3 into a production environment, you must:

1. Provide a name, IP address, and root password for the management server.

2. Provide a name and IP address for the terminal concentrator.

3. Provide a name for the cluster environment and a default router (gateway).

4. Provide names and IP addresses for individual cluster nodes.

5. Provide names and IP addresses for the Sun StorEdge T3 arrays.

6. Configure shared disk storage under Solstice DiskSuite or VERITAS volume manager. Configuration includes the creation of disksets (or disk groups), disk volumes, and file systems.

7. Select a quorum device from the shared disk storage.

8. Install and configure the required highly available applications.

9. Install and configure data services to support the highly available applications.

10. Configure network adapter failover (NAFO) for automatic failover.

6 Cluster Platform 4500/3 User’s Guide • July 2001

Note – This document does not provide information to support items 6 through 9.

For specific implementation details, refer to the Sun Cluster 3.0 documentation.

The Ethernet address for each cluster node is located on the Customer Information, System Record sheet. Use the serial number on the information sheet and on the back of each node to correctly identify the Ethernet address. (See Figure 2-4 for the placement of each cluster node.)

TABLE 2-1 shows the location of the Ethernet address for the disk array.

00:12:e2:00:4c:4b

SN: XXXX XXXX PN: XXX-XXXX-XX

ASSEMBLED IN USA

00:12:e2:00:4c:4b

FIGURE 2-1 Ethernet Address on the Disk Array Pull Tab

Chapter 2 The Cluster Platform 4500/3 System 7

TABLE 2-1 provides a worksheet to assist with networking information. You will be

referred back to the information you place in this table when you customize the cluster configuration.

TABLE2-1 Ethernet IP Address Worksheet

Network Device

Ethernet Address IP Address Node Name

Netra™ T1 AC200 Terminal concentrator Sun Enterprise 4500

system No. 1 (node 1) Sun Enterprise 4500

system No. 2 (node 2) Sun StorEdge T3 array

No. 1 (node 1) Sun StorEdge T3 array

No. 2 (node 2)

Hardware Components

Your Cluster Platform 4500/3 shipment includes the following hardware components (see

■ Consolidation rack: One Sun StorEdge Expansion Cabinet which includes power

sequencers, front and rear door options, and a fan assembly

■ Management server: One Sun Netra T1 AC200 system configured with one 500

MHz UltraSPARC II CPU, 512 Mbytes of memory and two 18.2-Gbyte boot disks with mirroring

■ Terminal concentrator: One 8-port terminal concentrator preinstalled in a Sun

StorEdge expansion cabinet

■ Two Sun Enterprise 4500 system servers, each configured with 400 MHz

UltraSPARC II CPUs

■ Four I/O boards with on-board DWIS, four on-board hme network interfaces,

two qfe SBus network interface cards, two on-board FC-ALs with GBICs, internal DVD and internal 20 Gbyte 4mm DDS4 tape drive

■ Four Netra st D130 boot disks

■ Two Sun StorEdge T3 arrays

FIGURE 2-5 for details on cable connectivity between components):

8 Cluster Platform 4500/3 User’s Guide • July 2001

■ Two FC-100 FC-AL hubs, with three installed GBICs in each hub

Connectivity

■ Cluster interconnect: The cluster connectivity uses Ethernet patch cables (no

Ethernet switch required), with redundant qfe 100BASE-T ports (qfe0 and qfe4) on two separate SBus controllers to avoid a controller single point of failure.

■ Public networks: The cluster nodes main network connection is implemented

using the on-board hme0 (100BASE-T) primary port with hme1 (100BASE-T) as the failover port. The main network connection has a failover interface on separate controllers. Two hme ports, hme2 and hme3, are available to implement additional production networks.

■ Storage access: Access to disk arrays is achieved through proper configuration of

the FC-AL from each node.

■ Administration networks: Sun StorEdge™ Component Manager 2.1, included

with your Cluster Platform 4500/3, is used to administer the disk arrays.

Network Administration

The administration network provides access to the Ethernet hub.

Miscellaneous Hardware

The following miscellaneous hardware is used:

■ Two one-meter RS-232C (DB-25/RJ-45) serial cables (Part No. 2151A) for the

cluster nodes console (Part No. 530-2151)

■ One one-meter RS-232C (RJ-45/RJ-45) serial cable for the management server

(Part No. 530-9524) to connect the terminal concentrator to the management server

■ Power cables provided with the power sequencers in the expansion cabinet

■ Terminal concentrator mounting bracket

■ Ethernet hub mounting brackets

By default, the I/O board slots in the servers are labeled 1, 3, 5, and 7 (

Chapter 2 The Cluster Platform 4500/3 System 9

FIGURE 2-2).

FIGURE 2-2 Cluster Platforms I/O Board Placement

Software Components

The Cluster Platform 4500/3 software packages include the following:

■ Solaris 8 10/00 operating environment

■ Sun Cluster 3.0 software with patches, and Sun Cluster 3.0 installation CDs

■ Solstice DiskSuite 4.2.1

■ UNIX

■ Sun StorEdge Component Manager 2.1

File System (UFS)

Note – A collection of software patches is applied to each cluster node. The patches

are located in the /SOFTWARE/Solaris8_10-00/Patches directory of the management server. Detailed information on how the software solution is integrated is provided at http://www.sun.com/blueprints.

FIGURE 2-3 shows cluster node interface hme1 providing network automatic failover

(NAFO) for hme0 (production network). Interfaces hme2 and hme3 are available to expand application network services. Interfaces qfe0 and qfe4 are used for the cluster interconnect.

10 Cluster Platform 4500/3 User’s Guide • July 2001

Note – See “Ethernet IP Address Worksheet” on page 8 to specify the appropriate

information for your network environment.

Chapter 2 The Cluster Platform 4500/3 System 11

Production Network

sc3sconﬂ1-ms

Ser. A eri0

Management

Server

8:0:20:c2:1b:3c

129.153.47.38

sc3sconf1-tc

Terminal

Concentrator

0:50:bd:bb:a:4:0

10BT

129.153.47.62

sc3sconf1-n1

Cluster

node 2

8:0:20:d1:e1:4

129.153.47.82

hme0

hme1

c1 c3

qfe4qfe0

sc3sconf1-n0

Cluster

node 1

8:0:20:d1:e6:f6

129.153.47.81

hme0 hme1

T3 No. 1 data T3 No. 2 mirror

sc3sconf1-T3 1

FC-AL hub 0

sc3sconf1-T3 2

FC-AL hub 1

0 1 2 3 4 5 6 0 1 2 3 4 5 6

Administration Network

FIGURE 2-3 Cluster Platform Interconnections and NAFO

12 Cluster Platform 4500/3 User’s Guide • July 2001

3 4 5 6

1 2

Ethernet hub

7 8 9 10 1112

NAFO

Cluster Platform Component Location

FIGURE 2-4 shows how the Cluster Platform 4500/3 is arranged in the expansion

cabinet.

Note – The rack arrangement complies with weight distribution, cooling, EMI, and

power requirements.

FIGURE 2-2 lists the rack components and the quantities required.

Chapter 2 The Cluster Platform 4500/3 System 13

+ 57 hidden pages