Symantec Veritas 5 User Manual

Page 1

Veritas Cluster Server Installation Guide

Linux for IBM Power
5.0 Release Update 3
Page 2
Veritas Cluster Server Installation Guide
The softwaredescribed inthis bookis furnishedunder alicense agreementand maybe used only in accordance with the terms of the agreement.
Product version: 5.0 RU3
Legal Notice
Copyright © 2009 Symantec Corporation. All rights reserved.
Symantec, the Symantec Logo, Veritas and Veritas Storage Foundation are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.
The product described in this document is distributed under licenses restricting its use, copying, distribution, and decompilation/reverse engineering. No part of this document may be reproduced in any form by any means without prior written authorization of Symantec Corporation and its licensors, if any.
THE DOCUMENTATIONISPROVIDED "ASIS" AND ALLEXPRESS ORIMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLYINVALID. SYMANTEC CORPORATIONSHALL NOT BELIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE.
The LicensedSoftware andDocumentation aredeemed tobe commercialcomputer software as definedin FAR 12.212 and subject to restricted rights asdefined inFAR Section52.227-19 "Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights in Commercial Computer Software or Commercial Computer Software Documentation", as applicable, and any successor regulations. Any use, modification, reproduction release, performance, displayor disclosure of the Licensed Software and Documentation bythe U.S. Government shall be solely in accordance with the terms of this Agreement.
Page 3
Symantec Corporation 350 Ellis Street Mountain View, CA 94043
http://www.symantec.com
Page 4

Technical Support

Symantec Technical Support maintains support centers globally. Technical Supports primary role is to respond to specific queries about product features and functionality.The Technical Support group also creates contentfor our online Knowledge Base. The Technical Support group works collaboratively with the other functional areas within Symantec to answer your questions in a timely fashion. Forexample, theTechnical Supportgroup works with Product Engineering and SymantecSecurity Response to provide alerting services andvirus definition updates.
Symantecs maintenance offerings include the following:
A range of support options that give you the flexibility to select the right
amount of service for any size organization
Telephone and Web-based support that provides rapid response and
up-to-the-minute information
Upgrade assurance that delivers automatic software upgrade protection
Global support that is available 24 hours a day, 7 days a week
Advanced features, including Account Management Services
For information about Symantecs Maintenance Programs, you can visitour Web site at the following URL:
www.symantec.com/techsupp/
Contacting Technical Support
Customers with a current maintenanceagreement may access Technical Support information at the following URL:
www.symantec.com/business/support/assistance_care.jsp
Before contacting Technical Support, make sure you have satisfied the system requirements that are listed in your product documentation. Also, you should be at thecomputer on which the problem occurred, incase itis necessaryto replicate the problem.
When you contact Technical Support, please have the following information available:
Product release level
Hardware information
Available memory, disk space, and NIC information
Operating system
Page 5
Version and patch level
Network topology
Router, gateway, and IP address information
Problem description:
Error messages and log files
Troubleshooting that was performed before contacting Symantec
Recent software configuration changes and network changes
Licensing and registration
If yourSymantec product requires registration or a licensekey, accessour technical support Web page at the following URL:
www.symantec.com/techsupp/
Customer service
Customer service information is available at the following URL:
www.symantec.com/techsupp/
Customer Service is available to assist with the following types of issues:
Questions regarding product licensing or serialization
Product registration updates, such as address or name changes
General product information (features, language availability, local dealers)
Latest information about product updates and upgrades
Information about upgrade assurance and maintenance contracts
Information about the Symantec Buying Programs
Advice about Symantec's technical support options
Nontechnical presales questions
Issues that are related to CD-ROMs or manuals
Documentation feedback
Your feedback on product documentation is important to us. Send suggestions for improvements and reports on errors or omissions to
clustering_docs@symantec.com. Include the title and document version (located
on the second page), and chapter and section titles of the text on which you are reporting.
Page 6
Maintenance agreement resources
If you want to contact Symantec regarding an existing maintenance agreement, please contact the maintenance agreement administration team for your region as follows:
Additional enterprise services
Symantec offers a comprehensive setof services that allow you to maximize your investment in Symantec products and to develop your knowledge, expertise, and global insight, which enable you to manage your business risks proactively.
Enterprise services that are available include the following:
customercare_apac@symantec.comAsia-Pacific and Japan
semea@symantec.comEurope, Middle-East, and Africa
supportsolutions@symantec.comNorth America and Latin America
Symantec EarlyWarning Solutions
Managed Security Services
Consulting Services
Educational Services
To access more information about Enterprise services, please visit our Web site at the following URL:
www.symantec.com
Select your country or language from the site index.
These solutions provide early warning of cyber attacks, comprehensive threat analysis, and countermeasures to prevent attacks before they occur.
These servicesremove theburden ofmanaging andmonitoring securitydevices and events, ensuring rapid response to real threats.
Symantec Consulting Services provide on-site technical expertise from Symantec andits trustedpartners. SymantecConsulting Servicesoffer avariety of prepackaged and customizable options that include assessment, design, implementation, monitoring,and managementcapabilities. Each is focused on establishing andmaintaining the integrity and availability of your IT resources.
Educational Services provide a full array of technical training, security education, security certification, and awareness communication programs.
Page 7

Contents

Technical Support ..... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ... 4
Chapter 1 Introducing Veritas Cluster Server ..... . .... ..... ..... .... . .... . .... 13
About Veritas Cluster Server ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... 13
About VCS basics .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... . 13
About multiple nodes . .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... .... . ... 14
About shared storage .... .... . .... . .... . .... ..... ..... .... . .... . .... ..... ..... .... . 14
About LLT and GAB . .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... .... . .... . 15
About network channels for heartbeating .... ..... .... . .... . .... . .... ..... .. 15
About preexisting network partitions ... . .... . .... ..... ..... .... . .... . .... . .. 16
About VCS seeding ... . .... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... .... 16
About VCS features . .... . .... . .... ..... ..... .... . .... . .... . .... ..... .... . .... . .... . .... .. 17
Veritas Installation Assessment Service ... .... . .... . .... . .... ..... ..... .... . 17
About VCS notifications .... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... . 17
About global clusters ... . .... . .... ..... ..... .... . .... . .... . .... ..... .... . .... . .... . 17
About I/O fencing .... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . 18
About VCS optional components ... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... 18
About Symantec Product Authentication Service (AT) .... ..... .... . .... . 19
About Cluster Manager (Java Console) . .... . .... . .... ..... ..... .... . .... . .... . 20
About Veritas Cluster Server Management Console .... . .... . .... ..... ... 20
About VCS Simulator ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... .... 20
Chapter 2 Planning to install VCS .. .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... 23
About planning to install VCS .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... 23
Hardware requirements .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... .. 23
Required disk space .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... ..... .. 24
Supported operating systems ... . .... ..... ..... .... . .... . .... ..... ..... .... . .... . .... . 25
Required Linux RPMs for VCS ... . .... ..... ..... .... . .... . .... . .... ..... ..... ... 26
Supported software . ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... . 27
Chapter 3 Preparing to install VCS .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... 29
About preparing to install VCS ... .... . .... . .... . .... ..... ..... .... . .... . .... . .... .... 29
Preparing to configure the clusters in secure mode . ..... .... . .... . .... . .... ... 29
Installing the root broker for the security infrastructure . ..... .... . .... 33
Page 8
Contents8
Creating authentication broker accounts on root broker
system ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .. 34
Creating encrypted files for the security infrastructure ... .... . .... . .... 35
Preparing the installation system for the security
infrastructure ... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... ..... .... 37
Performing preinstallation tasks .... . .... . .... ..... .... . .... . .... . .... ..... ..... .... . 38
Obtaining VCS license keys . ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... 39
Setting up the private network . . .... . .... ..... ..... .... . .... . .... . .... ..... .... 40
Configuring SuSE network interfaces .... ..... ..... .... . .... . .... . .... ..... ... 42
Setting up inter-system communication .... ..... ..... .... . .... . .... . .... .... 44
Setting up shared storage ... . .... . .... . .... ..... ..... .... . .... . .... ..... ..... .... 46
Setting the PATH variable . . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... 47
Setting the MANPATH variable ... . .... ..... ..... .... . .... . .... ..... ..... .... . . 47
Setting the kernel.panic tunable . ..... ..... .... . .... . .... . .... ..... ..... .... . .. 48
Optimizing LLT media speed settings on private NICs .... . .... . .... . .... 48
Guidelines for setting the media speed of the LLT
interconnects .... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... 48
Mounting the product disc .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .. 49
Performing automated pre-installation check .... ..... .... . .... . .... ..... .. 49
Chapter 4 Installing and configuring VCS .... ..... .... . .... . .... . .... ..... ..... ... 51
About installing and configuring VCS . .... ..... ..... .... . .... . .... . .... ..... ..... .. 51
Getting your VCS installation and configuration information
ready .... . .... ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . . 52
Optional VCS RPMs ... ..... .... . .... . .... . .... ..... .... . .... . .... . .... ..... ..... ... 54
About the VCS installation program . . .... . .... ..... ..... .... . .... . .... ..... ..... ... 55
Optional features of the installvcs program .... . .... . .... ..... ..... .... . ... 55
Interacting with the installvcs program ..... .... . .... . .... ..... ..... .... . ... 56
About installvcs program command options .... . .... . .... ..... ..... .... . .. 56
Installing VCS using installonly option .... ..... ..... .... . .... . .... ..... ..... 59
Configuring VCS using configure option ... .... . .... . .... . .... ..... ..... .... . 59
Installing and configuring VCS 5.0 RU3 .... ..... ..... .... . .... . .... . .... ..... ..... . 60
Overview of tasks ... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... . 60
Starting the software installation ..... ..... .... . .... . .... . .... ..... ..... .... . .. 61
Specifying systems for installation . . .... . .... ..... ..... .... . .... . .... . .... .... 62
Licensing VCS . .... . .... ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ... 63
Choosing VCS RPMs for installation . ..... .... . .... . .... . .... ..... ..... .... . .. 64
Choosing to install VCS RPMs or configure VCS ..... .... . .... . .... . .... ... 65
Starting the software configuration . .... . .... . .... . .... ..... ..... .... . .... . ... 66
Specifying systems for configuration ... . .... ..... ..... .... . .... . .... ..... .... 67
Configuring the basic cluster . ..... .... . .... . .... . .... ..... ..... .... . .... . .... ... 67
Configuring the cluster in secure mode . .... . .... . .... . .... ..... ..... .... . ... 69
Page 9
Adding VCS users ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... 71
Configuring SMTP email notification ... . .... . .... . .... ..... ..... .... . .... . ... 71
Configuring SNMP trap notification . . .... . .... . .... ..... ..... .... . .... . .... . . 73
Configuring global clusters ... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ... 74
Installing VCS RPMs ... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... .. 75
Creating VCS configuration files . . .... . .... . .... ..... ..... .... . .... . .... . .... .. 76
Starting VCS ... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... .. 77
Completing the installation ..... ..... .... . .... . .... . .... ..... .... . .... . .... . .... . 77
About enabling LDAP authentication for clusters that run in
secure mode ... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .. 78
Installing the Java Console . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... . 80
Installing VCS Simulator ... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... . 82
Verifying the cluster after installation .... ..... ..... .... . .... . .... . .... ..... . 84
Verifying and updating licenses on the system ... .... . .... . .... . .... ..... ..... .. 84
Checking licensing information on the system ..... .... . .... . .... . .... ..... 84
Updating product licenses using vxlicinst . . .... . .... . .... ..... ..... .... . .... 85
Accessing the VCS documentation ... . .... . .... . .... ..... ..... .... . .... . .... . .... .... 86
Chapter 5 Configuring VCS clusters for data integrity . ..... .... . .... . ... 87
About configuring VCS clusters for data integrity . ..... .... . .... . .... . .... ..... 87
About I/O fencing components ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . . 88
About data disks .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . 88
About coordination points .... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ... 88
About setting up disk-based I/O fencing .... . .... . .... . .... ..... ..... .... . .... . .... 89
Preparing to configure disk-based I/O fencing ... ..... .... . .... . .... . .... ..... ... 92
Initializing disks as VxVM disks .... . .... . .... . .... ..... ..... .... . .... . .... . .... 93
Identifying disks to use as coordinator disks . . .... ..... ..... .... . .... . .... . 95
Checking shared disks for I/O fencing . .... . .... . .... ..... ..... .... . .... . .... . 95
Setting up disk-based I/O fencing manually . . .... . .... ..... ..... .... . .... . .... ... 99
Setting up coordinator disk groups . . .... . .... ..... ..... .... . .... . .... . .... .... 99
Creating I/O fencing configuration files ... ..... .... . .... . .... . .... ..... .... 100
Modifying VCS configuration to use I/O fencing .... . .... ..... ..... .... . . 101
Verifying I/O fencing configuration . . .... ..... ..... .... . .... . .... . .... ..... . 102
Removing permissions for communication .... . .... ..... ..... .... . .... . ... 103
9Contents
Chapter 6 Verifying the VCS installation ... ..... .... . .... . .... . .... ..... ..... .... 105
About verifying the VCS installation . . .... . .... ..... ..... .... . .... . .... . .... ..... . 105
About the LLT and GAB configuration files . .... . .... ..... ..... .... . .... . .... . .. 105
About the VCS configuration file main.cf .... ..... ..... .... . .... . .... ..... ..... .. 107
Sample main.cf file for VCS clusters .... .... . .... . .... . .... ..... ..... .... . ... 108
Sample main.cf file for global clusters . .... . .... ..... ..... .... . .... . .... . ... 110
Verifying the LLT, GAB, and VCS configuration files . . .... . .... . .... ..... .... 113
Page 10
Contents10
Verifying LLT, GAB, and cluster operation . . .... . .... . .... ..... ..... .... . .... . .. 113
Verifying LLT ... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . ... 114
Verifying GAB ... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... ..... ..... .... 116
Verifying the cluster . . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . . 117
Verifying the cluster nodes ... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . 118
Chapter 7 Adding and removing cluster nodes ... ..... ..... .... . .... . .... ... 121
About adding and removing nodes ... ..... ..... .... . .... . .... . .... ..... ..... .... . .. 121
Adding a node to a cluster . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . . 121
Setting up the hardware . . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... . 122
Preparing for a manual installation when adding a node ... . .... ..... . 123
Installing VCS RPMs for a manual installation . .... . .... ..... ..... .... . .. 124
Adding a license key .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . ... 125
Setting up the node to run in secure mode .... . .... . .... . .... ..... ..... .... 126
Configuring LLT and GAB . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... 128
Adding the node to the existing cluster . .... . .... . .... ..... ..... .... . .... . .. 130
Starting VCS and verifying the cluster .... ..... .... . .... . .... . .... ..... ..... 131
Removing a node from a cluster .... . .... . .... . .... ..... ..... .... . .... . .... . .... .... 131
Verifying the status of nodes and service groups . ..... .... . .... . .... . ... 132
Deleting the departing node from VCS configuration ... ..... .... . .... . 133
Modifying configuration files on each remaining node . . .... . .... ..... 136
Removing security credentials from the leaving node .... . .... . .... . .. 136
Unloading LLT and GAB and removing VCS on the departing
node ... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... 137
Chapter 8 Installing VCS on a single node . .... ..... ..... .... . .... . .... ..... .... 139
About installing VCS on a single node .... . .... ..... ..... .... . .... . .... . .... ..... . 139
Creating a single-node cluster using the installer program ..... .... . .... . .. 139
Preparing for a single node installation . .... ..... ..... .... . .... . .... ..... .. 140
Starting the installer for the single node cluster . ..... .... . .... . .... ..... 140
Creating a single-node cluster manually . .... . .... ..... ..... .... . .... . .... . .... .. 141
Setting the path variable for a manual single node
installation ... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... . 141
Installing the VCS software manually on a single node .... ..... ..... .. 141
Renaming the LLT and GAB startup files . .... . .... . .... . .... ..... ..... .... 142
Modifying the startup files .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... 142
Verifying single-node operation ..... .... . .... . .... . .... ..... .... . .... . .... . .. 142
Adding a node to a single-node cluster . .... . .... . .... . .... ..... ..... .... . .... . ... 143
Setting up a node to join the single-node cluster .... ..... .... . .... . .... . . 144
Installing and configuring Ethernet cards for private
network . ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . ... 144
Configuring the shared storage . . .... ..... ..... .... . .... . .... ..... ..... .... . .. 145
Page 11
Bringing up the existing node .... ..... ..... .... . .... . .... . .... ..... ..... .... . . 145
Installing the VCS software manually when adding a node to a
single node cluster .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... 146
Configuring LLT . . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .. 146
Configuring GAB when adding a node to a single node
cluster ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... ..... ..... .... . .... . . 149
Starting LLT and GAB ... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .. 149
Reconfiguring VCS on the existing node . . .... . .... ..... ..... .... . .... . .... 149
Verifying configuration on both nodes . . .... ..... ..... .... . .... . .... . .... .. 150
Chapter 9 Uninstalling VCS .... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .. 153
About the uninstallvcs program . . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .. 153
Preparing to uninstall VCS . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . 153
Uninstalling VCS 5.0 RU3 . . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . 154
Removing VCS 5.0 RU3 RPMs .... ..... .... . .... . .... ..... ..... .... . .... . .... . . 154
Running uninstallvcs from the VCS 5.0 RU3 disc ... . .... ..... ..... .... . . 155
Appendix A Advanced VCS installation topics .... ..... .... . .... . .... . .... ..... . 157
Using the UDP layer for LLT ... . .... ..... ..... .... . .... . .... . .... ..... .... . .... . .... . 157
When to use LLT over UDP . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... 157
Configuring LLT over UDP .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . 157
Performing automated VCS installations . .... . .... . .... ..... ..... .... . .... . .... . 164
Syntax in the response file .... .... . .... . .... . .... ..... ..... .... . .... . .... . .... . 165
Example response file ... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ... 165
Response file variable definitions . . .... . .... ..... ..... .... . .... . .... . .... .... 166
Installing VCS with a response file where ssh or rsh are disabled ..... .... 171
11Contents
Index . . .... ..... ..... .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .... ..... ..... .... . .... . .... . .. 173
Page 12
Contents12
Page 13
Chapter

Introducing Veritas Cluster Server

This chapter includes the following topics:
About Veritas Cluster Server
About VCS basics
About VCS features
About VCS optional components
1

About Veritas Cluster Server

VeritasCluster Server by Symantec is a high-availability solution for cluster configurations. Veritas Cluster Server (VCS) monitors systems and application services, and restarts services when hardware or software fails.

About VCS basics

A single VCS cluster consists of multiple systems that are connected in various combinations to shared storage devices. When a system is part of a VCS cluster, it is a node. VCS monitors and controls applications running in the cluster on nodes, and restarts applications in response to a variety of hardware or software faults.
Applications can continue to operate with little or no downtime. In some cases, such asNFS, this continuation is transparentto high-level applications and users. In other cases, a user might have to retry an operation, such as a Web server reloading a page.
Page 14
Client workstation Client workstation
Public network
Shared storage
VCS private network
VCS nodes
Introducing Veritas Cluster Server
14
About VCS basics
Figure 1-1illustrates a typical VCS configurationof four nodes that are connected
to shared storage.
Figure 1-1
Client workstations receive service over the public network from applications running on VCS nodes. VCS monitors the nodes and their services. VCS nodes in the cluster communicate over a private network.

About multiple nodes

VCS runs in a replicated state on each node in the cluster. A private network enables the nodes to share identical state information about all resources. The private network also recognizes active nodes, the nodes that join or leave the cluster, and failed nodes. The private network requires two communication channels to guard against network partitions.
Example of a four-node VCS cluster

About shared storage

A VCS hardware configuration typically consists of multiple nodes that are connected to shared storage through I/O channels. Shared storage provides multiple systems with an access path to the same data. It also enables VCS to restart applications on alternate nodes when a node fails, which ensures high availability.
VCS nodes can only access physically-attached storage.
Figure 1-2 illustrates the flexibility of VCS shared storage configurations.
Page 15
Fully shared storage Distributed shared storage
About VCS basics
15Introducing Veritas Cluster Server
Figure 1-2

About LLT and GAB

VCS uses two components, LLT and GAB, to share data over private networks among systems. These components provide the performance and reliability that VCS requires.
LLT (Low Latency Transport) provides fast, kernel-to-kernel communications, and monitors network connections.
LLT configuration files are as follows:
/etc/llthostslists all the nodes in the cluster
Two examples of shared storage configurations
/etc/llttab filedescribes the local systems private network links to the other
nodes in the cluster
GAB (Group Membership and Atomic Broadcast) provides the global message order that is required to maintain a synchronized state among the nodes. It monitors diskcommunications suchas theVCS heartbeat utility. The /etc/gabtab file is the GAB configuration file.
See “About the LLT and GAB configuration files” on page 105.

About network channels for heartbeating

For the VCS private network, two network channels must be available to carry heartbeat information.These networkconnections alsotransmit other VCS-related information.
Each Linux for IBM Power cluster configuration requires at least two network channels between the systems. The requirement for two channels protects your cluster against network partitioning. For more information on network partitioning, refer to the Veritas Cluster Server User's Guide.
Page 16
VCS private network: two ethernet connections
Shared disks
Public network
galaxy nebula
Introducing Veritas Cluster Server
16
About VCS basics
Figure 1-3 illustrates a two-node VCS cluster where the nodes galaxy and nebula
have two private network connections.
Figure 1-3
Two Ethernet connections connecting two nodes

About preexisting network partitions

A preexistingnetwork partition refers to a failure inthe communicationchannels that occurswhile thesystems aredown andVCS cannot respond. When the systems start, VCS is vulnerable to network partitioning, regardless of the cause of the failure.

About VCS seeding

To protect your cluster from a preexisting network partition, VCS uses a seed. A seed is a function of GAB that determines whether or not all nodes have joined a cluster. For this determination, GAB requires that you declare the number of nodes in the cluster. Note that only seeded nodes can run VCS.
GAB automatically seeds nodes under the following conditions:
An unseeded node communicates with a seeded node
All nodes in the cluster are unseeded but can communicate with each other
When thelast system starts and joins the cluster, the clusterseeds and starts VCS on all nodes. You can then bring down and restart nodes in any combination. Seeding remains in effect as long as at least one instance of VCS is running somewhere in the cluster.
Perform a manual seed to run VCS from a cold start when one or more systems of the cluster are unavailable. VCS does not start service groupson a systemuntil it has a seed.
Page 17

About VCS features

You can use the Veritas Installation Assessment Service to assess your setup for VCS installation.
See “Veritas Installation Assessment Service” on page 17.
VCS offersthe following features that you can configureduring VCSconfiguration:
See “About VCS notifications” on page 17.VCS notifications
See “About global clusters” on page 17.VCS global clusters
See “About I/O fencing” on page 18.I/O fencing

Veritas Installation Assessment Service

The Veritas Installation Assessment Service (VIAS) utility assists you in getting ready fora Veritas Storage Foundation and High AvailabilitySolutions installation or upgrade. The VIAS utility allows the preinstallation evaluation of a configuration, to validate it prior to starting an installation or upgrade.
https://vias.symantec.com/
About VCS features
17Introducing Veritas Cluster Server

About VCS notifications

You can configure both SNMP and SMTP notifications for VCS. Symantec recommends you to configure one of these notifications. You have the following options:
Configure SNMP trap notification of VCS events using the VCS Notifier
component
Configure SMTP email notification of VCS events using the VCS Notifier
component.
See the Veritas Cluster Server Users Guide.

About global clusters

Global clustersprovide the ability to failover applications between geographically distributed clusters when disaster occurs. You require a separate license to configure global clusters. You must add this license during the installation. The installer only asks about configuring global clusters if you have used the global cluster license.
See the Veritas Cluster Server User's Guide.
Page 18
Introducing Veritas Cluster Server
18

About VCS optional components

About I/O fencing

I/O fencing protects the data on shared disks when nodes in a cluster detect a change in the cluster membership that indicates a split brain condition.
See the Veritas Cluster Server User's Guide.
The fencing operation determines the following:
The nodes that must retain access to the shared storage
The nodes that must be ejected from the cluster
This decision prevents possible data corruption. The installvcs program installs the VCSI/O fencingdriver, VRTSvxfen. To protect data on shared disks, you must configure I/O fencing after you install and configure VCS.
I/O fencing technology uses coordination points for arbitration in the event of a network partition.
Note: Symantec recommends that you use I/O fencing to protect your cluster against split-brain situations.
About VCS optional components
You can add the following optional components to VCS:
Symantec Product Authentication Service
Veritas Cluster Server Management Console
To configure the optional components, make sure to install all RPMs when the installation program prompts you.
Figure 1-4 illustrates a sample VCS deployment with the optional components
configured.
See “About Symantec Product Authentication Service
(AT)on page 19.
See “AboutVeritas ClusterServer ManagementConsole” on page 20.
See “AboutCluster Manager (JavaConsole)” on page20.Cluster Manager (Java console)
See “About VCS Simulator” on page 20.VCS Simulator
Page 19
Symantec Product Authentication Service root broker
VCS cluster 1 VCS cluster 2
VCS Management Console management server
Optional
About VCS optional components
19Introducing Veritas Cluster Server

About Symantec Product Authentication Service (AT)

Figure 1-4
Typical VCS setup with optional components
VCS uses Symantec Product Authentication Service (AT) to provide secure communication between cluster nodes and clients. It uses digital certificates for authentication and SSL to encrypt communication over the public network to secure communications.
AT uses the following brokers to establish trust relationship between the cluster components:
Root broker
A root broker serves as the main registration and certification authority; it has a self-signed certificate and can authenticate other brokers. The root broker is only used during initial creation of an authentication broker.
A root broker can serve multiple clusters. Symantec recommends that you install a single root broker on a utility system. The utility system, such as an email server or domain controller, can be highly available.
Authentication brokers
Authentication brokers serve as intermediate registration and certification authorities. Authentication brokers have root-signed certificates. Each node in VCS serves as an authentication broker.
See Symantec Product Authentication Service documentation for more information.
Page 20
Introducing Veritas Cluster Server
20
About VCS optional components
See “Preparing to configure the clusters in secure mode” on page 29.

About Cluster Manager (Java Console)

Cluster Manager (Java Console) offers complete administration capabilities for your cluster. Use the different views in the Java Console to monitor clusters and VCS objects, including service groups, systems, resources, and resource types.
You canperform manyadministrative operations using the Java Console. You can also perform these operations using the command line interface or using the Veritas Cluster Server Management Console.
See “Installing the Java Console” on page 80.
See Veritas Cluster Server User's Guide.

About Veritas Cluster Server Management Console

Veritas Cluster Server Management Console is a high availability management solution that enables monitoring and administering clusters from a single Web console.
You canconfigure VeritasCluster Server Management Console to manage multiple clusters.
Refer to the Veritas Cluster Server Management Console Implementation Guide for installation, upgrade, and configuration instructions.
For information on updates and patches for VCS Management Console, see
http://seer.entsupport.symantec.com/docs/308405.htm.
To download the most current version of VCS Management Console, go to
www.symantec.com/business/cluster-server and click Utilities.

About VCS Simulator

VCS Simulator enables you to simulate and test cluster configurations. Use VCS Simulator to view and modify service group and resource configurations and test failover behavior. VCS Simulator can be run on a stand-alone system and does not require any additional hardware.
VCS Simulator runs an identical version of the VCS High Availability Daemon (HAD) as in a cluster, ensuring that failover decisions are identical to those in an actual cluster.
You cantest configurationsfrom different operating systems using VCS Simulator. For example, you can run VCS Simulator on a Windows system and test VCS
Page 21
About VCS optional components
configurations for Windows, Linux, and Solaris clusters. VCS Simulator also enables creating and testing global clusters.
You can administer VCS Simulator from the Java Console or from the command line.
21Introducing Veritas Cluster Server
Page 22
Introducing Veritas Cluster Server
22
About VCS optional components
Page 23
Chapter

Planning to install VCS

This chapter includes the following topics:
About planning to install VCS
Hardware requirements
Supported operating systems
Supported software

About planning to install VCS

2
Every node where you want to install VCS must meet the hardware and software requirements.
For the latest information on updates, patches, and software issues, read the following Veritas Technical Support TechNote:
http://entsupport.symantec.com/docs/285834
To find information on supported hardware, see the hardware compatibility list (HCL) in the following TechNote:
http://entsupport.symantec.com/docs/286819

Hardware requirements

Table 2-1 lists the hardware requirements for a VCS cluster.
Page 24
Planning to install VCS
24
Hardware requirements
Table 2-1
VCS nodes
DVD drive
Disks
Disk space
Network Interface Cards (NICs)
Fibre Channel or SCSI host bus adapters
Hardware requirements for a VCS cluster
DescriptionItem
From 1 to 32 Linux PPC systems running the supported Linux PPC operating system version.
One drive in a system that can communicate to all the nodes in the cluster.
Typical VCS configurations require that shared disks support the applications that migrate between systems in the cluster.
The VCS I/O fencing feature requires that all data and coordinator disks support SCSI-3 Persistent Reservations (PR).
See “About setting up disk-based I/O fencing” on page 89.
Note: VCSmay requiremore temporarydisk spaceduring installation
than the specified disk space.
In addition to the built-in public NIC, VCS requires at least one more NIC per system. Symantec recommends two additional NICs.
You can also configure aggregated interfaces.
Typical VCSconfiguration requiresat leastone SCSIor FibreChannel Host Bus Adapter per system for shared data disks.

Required disk space

Confirm that your system has enough free disk space to install VCS.
Table 2-2 shows the approximate disk space usage by directory for the Veritas
Cluster Server RPMs.
Table 2-2
optional total
Each VCS node requires at least 256 megabytes.RAM
Disk space requirements and totals
Totals/var/usr/opt/Packages
283 MB1 MB8 MB271 MB3 MBRequired
60 MB7 MB0 MB52 MB1 MBOptional
343 MB8 MB8 MB323 MB4 MBRequired and
Page 25
Note: If you do not have enough free space in /var, then use the installvcs command with tmppath option. Make sure that the specified tmppath file system has the required free space.

Supported operating systems

VCS operates on the Linux operating systems and kernels distributed by Red Hat and SUSE.
Table 2-3 lists the supported operating system versions for Red Hat Enterprise
Linux (RHEL) and SUSE Linux Enterprise Server (SLES). The table also lists the supported kernel versions and the architecture.
Supported operating systems
25Planning to install VCS
Table 2-3
Supported Linux operating system and kernel versions
ArchitectureKernelOperating System
ppc642.6.18-53.el5RHEL 5 Update 1
ppc642.6.18-92.el5RHEL 5 Update 2
SLES 10 with SP1
2.6.16.46-0.12-smp
SLES 10 with SP2
2.6.16.60-0.21-smp
ppc642.6.16.46-0.12-default
ppc642.6.16.60-0.21-default
Note: If your system runs an older version of either Red Hat Enterprise Linux or SUSE Linux Enterprise Server, you must upgrade the operating system before you attempt to install the VCS software. Refer to the Red Hat or SUSE documentation for more information on upgrading your system.
Symantec supports only Red Hat and SUSE distributed kernel binaries.
Symantec products operate on subsequent kernel and patch releases provided the operating systems maintain kernel ABI (application binary interface) compatibility.
Information about the latest supported Red Hat erratas and updates and SUSE service packs is available in the following TechNote. The TechNote also includes any updatesto the supported operating systems and software. Read thisTechNote before you install Symantec products.
http://entsupport.symantec.com/docs/285834
Page 26
Planning to install VCS
26
Supported operating systems

Required Linux RPMs for VCS

Make sure you installed the following operating system-specific RPMs on the systems where you want to install or upgrade VCS. VCS will support any updates made to the following RPMs, provided the RPMs maintain the ABI compatibility.
Table 2-4 lists the RPMs that VCS requires for a given Linux operating system.
Table 2-4
RHEL 5
SLES 10
Required RPMs
Required RPMsOperating system
glibc-2.5-34.ppc.rpm
glibc-2.5-34.ppc64.rpm
glibc-common-2.5-34.ppc.rpm
libgcc-4.1.2-44.el5.ppc.rpm
libgcc-4.1.2-44.el5.ppc64.rpm
compat-libgcc-296-2.96-138.ppc.rpm
libstdc++-4.1.2-44.el5.ppc.rpm
libstdc++-4.1.2-44.el5.ppc64.rpm
compat-libstdc++-296-2.96-138.ppc.rpm
compat-libstdc++-33-3.2.3-61.ppc.rpm
compat-libstdc++-33-3.2.3-61.ppc64.rpm
java-1.4.2-gcj-compat-1.4.2.0-40jpp.115.ppc.rpm
glibc-2.4-31.54.ppc.rpm
glibc-64bit-2.4-31.54.ppc.rpm
compat-libstdc++-64bit-5.0.7-22.2.ppc.rpm
compat-libstdc++-5.0.7-22.2.ppc.rpm
compat-2006.1.25-11.2.ppc.rpm
libgcc-4.1.2_20070115-0.21.ppc.rpm
libgcc-64bit-4.1.2_20070115-0.21.ppc.rpm
libstdc++-4.1.2_20070115-0.21.ppc.rpm
libstdc++-64bit-4.1.2_20070115-0.21.ppc.rpm
Page 27

Supported software

Veritas Cluster Server supports the previous and next versions of Storage Foundation to facilitate product upgrades, when available.
VCS supports the following volume managers and files systems:
ext2, ext3, reiserfs, NFS, NFSv4, and bind on LVM2, Veritas Volume Manager
(VxVM) 5.0, and raw disks.
Veritas Volume Manager (VxVM) with Veritas File System (VxFS)
VxVM
VRTSvxvm-common-5.0.33.00-RU3_SLES10 VRTSvxvm-platform-5.0.33.00-RU3_SLES10 VRTSvxvm-common-5.0.33.00-RU3_RHEL5 VRTSvxvm-platform-5.0.33.00-RU3_RHEL5
VxFS
VRTSvxfs-common-5.0.33.00-RU3_SLES10 VRTSvxfs-platform-5.0.33.00-RU3_SLES10 VRTSvxfs-common-5.0.33.00-RU3_RHEL5 VRTSvxfs-platform-5.0.33.00-RU3_RHEL5
Supported software
27Planning to install VCS
Page 28
Planning to install VCS
28
Supported software
Page 29
Chapter

Preparing to install VCS

This chapter includes the following topics:
About preparing to install VCS
Preparing to configure the clusters in secure mode
Performing preinstallation tasks

About preparing to install VCS

Before you perform the preinstallation tasks, make sure you reviewed the installation requirements,set up the basic hardware, and plannedyour VCS setup.
3

Preparing to configure the clusters in secure mode

You can set up Symantec Product Authentication Service (AT) for the cluster during the VCS installation or after the installation.
If you want to enable AT in a cluster at a later time, refer to the Veritas Cluster Server User's Guide for instructions.
The prerequisites to configure a cluster in secure mode are as follows:
A system in your enterprise is configured as root broker (RB).
If a root broker system does not exist, install and configure root broker on a system.
See “Installing the root broker for the security infrastructure” on page 33.
An authentication broker (AB) account for each node in the cluster is set up
on the root broker system. See “Creating authentication broker accounts on root broker system”
on page 34.
Page 30
Preparing to install VCS
30
Preparing to configure the clusters in secure mode
The system clocks of the root broker and authentication brokers must be in
sync.
The installvcs program provides the following configuration modes:
Automatic mode
Semi-automatic mode
Manual mode
The root broker system must allow rsh or ssh passwordless login to use this mode.
This mode requires encrypted files (BLOB files) from the AT administrator to configure a cluster in secure mode.
The nodes in the cluster must allow rsh or ssh passwordless login.
See “Setting up inter-system communication” on page 44.
This mode requires root_hash file and the root broker information from the AT administrator to configure a cluster in secure mode.
The nodes in the cluster must allow rsh or ssh passwordless login.
See “Setting up inter-system communication” on page 44.
Figure 3-1 depicts the flow of configuring VCS cluster in secure mode.
Page 31
Configure the cluster in secure mode
Review AT concepts and gather required information
Install root broker on a stable system
Select a mode to
configure the
cluster in secure
mode
Does the root
broker allow you
to login without
password
Automatic mode
On the root broker system, create encrypted file (BLOB) for each node
Copy encrypted files to the installation system
Set up passwordless communication between nodes
No action required
Copy root_hash file from the root broker system to the installation system
Gather information to answer prompts
Set up passwordless communication between nodes
Manual mode
Semiautomatic mode
Enable LDAP authentication plugin if VCS users belong to LDAP domain
On the root broker system, create authentication broker identities for each node
No No
Yes
Automatic mode
Preparing to configure the clusters in secure mode
31Preparing to install VCS
Figure 3-1
Workflow to configure VCS cluster in secure mode
Table 3-1 lists the preparatory tasks in the order which the AT and VCS
administrators must perform.
Page 32
Preparing to install VCS
32
Preparing to configure the clusters in secure mode
Table 3-1
Preparatory tasks to configure a cluster in secure mode
Tasks
secure mode:
Automatic mode
Semi-automatic mode
Manual mode
See “Installing the root broker for the security infrastructure” on page 33.
each node in the cluster.
See “Creating authentication broker accountson root broker system” on page 34.
AT administrator requires the following information from the VCS administrator:
Node namesthat aredesignated toserve as authentication brokers
Password for each authentication broker
files) for each node and provide the files to the VCS administrator.
See “Creating encrypted files for the security infrastructure” on page 35.
AT administrator requires the following additional information from the VCS administrator:
Administrator password for each authentication broker
Typically, the password is the same for all nodes.
Who performs this task
VCS administratorDecide one of the followingconfiguration modes to set up a cluster in
AT administratorInstall the root broker on a stable system in the enterprise.
AT administratorOn the root broker system, create authentication broker accounts for
AT administratorTo use the semi-automatic mode, create the encrypted files (BLOB
(/opt/VRTSat/bin/root_hash) from the root brokersystem to the VCS administrator.
to the system from where you plan to install and configure VCS.
See “Preparingthe installation system for the security infrastructure” on page 37.
AT administratorTo use the manual mode, provide the root_hash file
VCS administratorCopy the files that are required to configure a cluster in secure mode
Page 33
Preparing to configure the clusters in secure mode

Installing the root broker for the security infrastructure

Install the root broker only if you plan to useAT to configure the cluster in secure mode. The root broker administrator must install and configure the root broker before you configure the Authentication Service for VCS. Symantec recommends that you install the root broker on a stable system that is outside the cluster.
You can install the root broker on an AIX, HP-UX, Linux, or Solaris system.
See Symantec Product Authentication Service documentation for more information.
See “About Symantec Product Authentication Service (AT)” on page 19.
To install the root broker
Change to the directory where you can start the Veritas product installer:
1
# ./installer
From the opening Selection Menu, choose: I for "Install/Upgrade a Product."
2
From the displayed list of products to install, choose: Symantec Product
3
Authentication Service.
To install the root broker, select the mode of AT installation as root mode
4
from the three choices that the installer presents:
33Preparing to install VCS
1)Root+AB Mode
2)Root Mode
3)AB Mode
Enter the mode which you would like AT installed? [1-3,q] 2
Enter the name of the system where you want to install the root broker.
5
Enter the system name on which to install AT: venus
Review the output as the installer does the following:
6
Checks to make sure that VCS supports the operating system
Checks if the system is already configured for security
Review theoutput as the installer checks for theinstalled RPMson the system.
7
The installerlists the RPMs that theprogram is about to install on thesystem. Press Enter to continue.
Review the output as the installer installs the root broker on the system.
8
Page 34
Preparing to install VCS
34
Preparing to configure the clusters in secure mode
Enter y when the installer prompts you to configure the Symantec Product
9
Authentication Service.
Press the Enter key to start the Authentication Server processes.
10
Do you want to start Symantec Product Authentication Service
processes now? [y,n,q] y
Enter an encryption key. Make sure that you enter a minimum of five
11
characters.
You must use this encrypted key with the -enckeyfile option when you use the -responsefile option for installation.
Press Enter to continue and review the output as the installer displays the
12
location of the installation log files, summary file, and the response file.

Creating authentication broker accounts on root broker system

On theroot broker system, the administrator must create an authenticationbroker (AB) account for each node in the cluster.
To create authentication broker accounts on root broker system
Determine the root broker domain name. Enter the following command on
1
the root broker system:
venus> # vssat showalltrustedcreds
For example, the domain name resembles "Domain Name: root@venus.symantecexample.com" in the output.
For each node in the cluster, verify whether an account exists on the root
2
broker system.
For example, to verify that an account exists for node galaxy:
venus> # vssat showprpl --pdrtype root \
--domain root@venus.symantecexample.com --prplname galaxy
If the output displays the principal account on root broker for the
authentication broker on the node, then delete the existing principal accounts. For example:
venus> # vssat deleteprpl --pdrtype root \
--domain root@venus.symantecexample.com \
--prplname galaxy --silent
Page 35
Preparing to configure the clusters in secure mode
If the output displays the following error, then the account for the given
authentication broker is not created on this root broker:
"Failed To Get Attributes For Principal"
Proceed to step 3.
Create a principal account for each authentication broker in the cluster. For
3
example:
venus> # vssat addprpl --pdrtype root --domain \
root@venus.symantecexample.com --prplname galaxy \
--password password --prpltype service
You mustuse thispassword that you create in the input file for the encrypted file.

Creating encrypted files for the security infrastructure

Create encrypted files (BLOB files) only if you plan to choose the semiautomatic mode that uses an encrypted file to configure the Authentication Service. The administrator must create the encrypted files on the root broker node. The administrator must create encrypted files for each node that is going to be a part of the cluster before you configure the Authentication Service for VCS.
To create encrypted files
35Preparing to install VCS
Make a note of the following root broker information. This information is
1
required for the input file for the encrypted file:
hash
root_domain
Make anote of the following authentication broker informationfor eachnode.
2
The value of the root hash string, which consists of 40 characters. Execute the following command to find this value:
venus> # vssat showbrokerhash
The value for the domain name of the root broker system. Execute the following command to find this value:
venus> # vssat showalltrustedcreds
This information is required for the input file for the encrypted file:
Page 36
Preparing to install VCS
36
Preparing to configure the clusters in secure mode
identity
password
broker_admin_password
For each node in the cluster, create the input file for the encrypted file.
3
The valuefor theauthentication brokeridentity, which you providedto createauthentication broker principal on the root broker system.
This is the value for the --prplname option of the addprpl command.
See “Creating authentication broker accounts on root
broker systemon page 34.
The value for the authentication broker password, which you provided to create authentication broker principal on the root broker system.
This is the value for the --password option of the addprpl command.
See “Creating authentication broker accounts on root
broker systemon page 34.
The value for the authentication broker password for Administrator account on the node. This password must be at least five characters.
The installer presents the formatof the input file for the encrypted filewhen you proceed to configure the Authentication Service using encrypted file. For example, the input file for authentication broker on galaxy resembles:
[setuptrust]
broker=venus.symantecexample.com
hash=758a33dbd6fae751630058ace3dedb54e562fe98
securitylevel=high
[configab]
identity=galaxy
password=password
root_domain=vx:root@venus.symantecexample.com
root_broker=venus.symantecexample.com:2821
broker_admin_password=ab_admin_password
start_broker=false
enable_pbx=false
Back up these input files that you created for the authentication broker on
4
each node in the cluster.
Page 37
Preparing to configure the clusters in secure mode
Note that for security purposes, the command to create the output file for the encrypted file deletes the input file.
For each node in thecluster, create the output file for the encryptedfile from
5
the root broker system using the following command.
RootBroker> # vssat createpkg \
--in /path/to/blob/input/file.txt \
--out /path/to/encrypted/blob/file.txt \
--host_ctx AB-hostname
For example:
venus> # vssat createpkg --in /tmp/galaxy.blob.in \
--out /tmp/galaxy.blob.out --host_ctx galaxy
Note that this command creates an encrypted file even if you provide wrong password for "password=" entry. But such an encrypted file with wrong password fails to install on authentication broker node.
After you complete creating the output files for the encrypted file, you must
6
copy these files to the installer node.
37Preparing to install VCS

Preparing the installation system for the security infrastructure

The VCS administrator must gather the required information and prepare the installation system to configure a cluster in secure mode.
To prepare the installation system for the security infrastructure
Depending on the configuration mode you decided to use, do one of the
following:
Automatic mode
Semi-automatic mode
Do the following:
Gather the root broker system name from the AT
administrator.
During VCS configuration, choose the configuration option
1 when the installvcs program prompts.
Do the following:
Copy theencrypted files(BLOB files)to thesystem fromwhere
you plan to install VCS. Note the path of these files that you copiedto theinstallation
system.
During VCS configuration, choose the configuration option
2 when the installvcs program prompts.
Page 38
Preparing to install VCS
38

Performing preinstallation tasks

Manual mode
Do the following:
Copy the root_hash file that you fetched to the system from
where you plan to install VCS. Note the path of the root hash file that you copied to the
installation system.
Gather the root broker information such as name, fully
qualified domain name, domain, and port from the AT administrator.
Note the principal name and password information for each
authentication broker that you provided to the AT administrator to create the authentication broker accounts.
During VCS configuration, choose the configuration option
3 when the installvcs program prompts.
Performing preinstallation tasks
Table 3-2 lists the tasks you must perform before proceeding to install VCS.
Table 3-2
network.
Preinstallation tasks
ReferenceTask
See “Obtaining VCS license keys” on page 39.Obtain license keys.
See “Setting up the private network” on page 40.Set up the private
interfaces
between systems.
systems.
I/O fencing (optional)
Set the PATH and the MANPATH variables.
tunable
See “Configuring SuSE network interfaces” on page 42.Configure SuSE network
See “Setting up inter-system communication” on page 44.Enable communication
See “Setting up ssh on cluster systems” on page 45.Set up ssh on cluster
See “Setting up shared storage” on page 46.Set upshared storagefor
See “Setting the PATH variable” on page 47.
See “Setting the MANPATH variable” on page 47.
See “Setting the kernel.panic tunable” on page 48.Set the kerne.panic
Page 39
Performing preinstallation tasks
39Preparing to install VCS
Table 3-2
Review basic instructions to optimize LLT media speeds.
Review guidelinesto help you set the LLT interconnects.
before installation

Obtaining VCS license keys

This productincludes a License Key certificate. The certificatespecifies the product keys and the number of product licenses purchased. A single key lets you install the product on the number and type of systems for which you purchased the license. A key may enable the operation of more products than are specified on the certificate.However, you are legally limited to thenumber ofproduct licenses purchased. The product installation procedure describes how to activate the key.
To registerand receive a software licensekey, goto the Symantec Licensing Portal at the following location:
https://licensing.symantec.com
Make sure you have your Software Product License document. You need information in this document to retrieve and manage license keys for your Symantec product. After you receive the license key, you can install the product.
Click the Help link at this site to access the License Portal User Guide and FAQ.
The VRTSvlic package enables product licensing. After the VRTSvlic is installed, the following commands and their manual pages are available on the system:
Preinstallation tasks (continued)
ReferenceTask
See “Optimizing LLT media speed settings on private NICs” on page 48.
See “Guidelines for setting the media speed of the LLT
interconnectson page 48.
See “Mounting the product disc” on page 49.Mount the product disc
See “Performing automated pre-installation check” onpage 49.Verify the systems
vxlictest
Installs a license key for a Symantec productvxlicinst
Displays currently installed licensesvxlicrep
Retrieves the features and their descriptions that are encoded in a license key
Page 40
Public network Public network
Private network
Private network switches or hubs
Preparing to install VCS
40
Performing preinstallation tasks
You canonly installthe Symantec software products for which you have purchased a license.The enclosedsoftware discsmight includeother products for which you have not purchased a license.

Setting up the private network

VCS requires you to set up a private network between the systems that form a cluster. Youcan use either NICs oraggregated interfacesto set up private network.
You can use network switches instead of hubs.
Refer to the Veritas Cluster Server Administrator's Guide to review VCS performance considerations.
Figure 3-2 shows two private networks for use with VCS.
Figure 3-2
Symantec recommendsconfiguring twoindependent networksbetween thecluster nodes with a network switch for each network. You can also connect the two switches at layer 2 for advanced failure protection. Such connections for LLT at layer 2 are called cross-links.
Figure 3-3 shows a private network configuration with crossed links between the
network switches.
Private network setups: two-node and four-node clusters
Page 41
Crossed link
Public network
Private networks
Performing preinstallation tasks
41Preparing to install VCS
Figure 3-3
Private network setup with crossed links
To set up the private network
Install the required network interface cards (NICs).
1
Create aggregatedinterfaces if you want to use theseto setup private network.
Connect the VCS private NICs on each system.
2
Use crossover Ethernet cables, switches, or independent hubs for each VCS
3
communication network. Note that the crossover Ethernet cables are supported only on two systems.
Ensure that you meet the following requirements:
The power to the switches or hubs must come from separate sources.
On each system, you must use two independent network cards to provide
redundancy.
The network interface card to set up private interface is not part of any
aggregated interface.
During theprocess of setting up heartbeat connections, considera case where a failure removes all communications between the systems.
Note that a chance for data corruption exists under the following conditions:
The systems still run, and
Page 42
Preparing to install VCS
42
Performing preinstallation tasks
The systems can access the shared storage.
Test the network connections. Temporarily assign network addresses and
4
use telnet or ping to verify communications.
LLT uses its own protocol, and does not use TCP/IP. So, youmust ensure that the private network connections are used only for LLT communication and not for TCP/IP traffic. To verify this requirement, unplumb and unconfigure any temporary IP addresses that are configured on the network interfaces.
The installvcs program configures the private network in the cluster during installation.
See “About installing and configuring VCS” on page 51.

Configuring SuSE network interfaces

You mustperform additionalnetwork configurationon SuSE.You neednot perform this procedure for the systems that run SLES 10 or later. By default, SLES 10 uses udev to achieve persistent interface names. Refer to the OS documentation for information on configuring persistent interfaces on SLES 10.
In rare cases where RedHat does not automatically configure the network interfaces, RedHat users may also have to perform the network configuration.
Review the following tasks that allow VCS to function properly:
VCS must be able to find the same network interface names across reboots.
VCS must have network interfaces up before LLT starts to run.
Symantec suggests the following steps for configuring network interfaces on SUSE.
Note: You must not reboot the system between configuringthe persistentinterface names and configuring the interfaces to be up before starting LLT.
Note: The MAC address in the ifcfg-eth-id-mac file can be in uppercase or lowercase. SUSE,and therefore the Veritas product installer, ignoresthe filewith lowercase MAC address if the file with uppercase MAC address is present.
Page 43
Performing preinstallation tasks
To configure persistent interface names for network devices
Navigate to the hotplug file in the /etc/sysconfig directory:
1
# cd /etc/sysconfig
Open the hotplug file in an editor.
2
Set HOTPLUG_PCI_QUEUE_NIC_EVENTS to yes:
3
HOTPLUG_PCI_QUEUE_NIC_EVENTS=yes
Run the command:
4
ifconfig -a
Make sure that the interface name to MAC address mapping remains same
5
across the reboots.
Symantec recommends adding the PERSISTENT_NAME entries to the configuration files for all the network interfaces (including the network interfaces that are not used).
For each ethernet interface displayed, do the following:
If afile named /etc/sysconfig/network/ifcfg-eth-id-mac, where mac is the
hardware address of that interface, does not exist, then do the following: Create the file. If a file exists for the same network interface with the name
/etc/sysconfig/network/ifcfg-ethX, thencopy the contents of thatfile into the newly created file. The variable ethX represents the interface name.
Add the following line at the end of the file
/etc/sysconfig/network/ifcfg-eth-id-mac.
43Preparing to install VCS
PERSISTENT_NAME=ethX
where ethX is the interface name.
For example:
# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:02:B3:DB:38:FE
inet addr:10.212.99.30 Bcast:10.212.99.255
Mask:255.255.254.0
inet6 addr: fe80::202:b3ff:fedb:38fe/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:453500 errors:0 dropped:0 overruns:0 frame:0
TX packets:8131 errors:0 dropped:0 overruns:0 carrier:0
Page 44
Preparing to install VCS
44
Performing preinstallation tasks
collisions:0 txqueuelen:1000
RX bytes:35401016 (33.7 Mb) TX bytes:999899 (976.4 Kb)
Base address:0xdce0 Memory:fcf20000-fcf40000
If a file named etc/sysconfig/network/ifcfg-eth-id-00:02:B3:DB:38:FE does not exist, do the following task:
Create the file.
If thefile /etc/sysconfig/network/ifcfg-eth0 exists, then copythe contents
of this file into etc/sysconfig/network/ifcfg-eth-id-00:02:B3:DB:38:FE.
Add the following to the end of the file named etc/sysconfig/network/ifcfg-eth-id-00:02:B3:DB:38:FE,
PERSISTENT_NAME=eth0
Perform the procedure for all the interfaces that the ifconfig -a command displays.
To configure interfaces to be up before starting LLT
For each network interface that you want LLT to use, find its MAC address
1
by running the ifconfig command:
# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:0C:0D:08:C4:32
Where eth0 is the sample network interface name. The output displays 00:0C:0D:08:C4:32 as the interfaces MAC address.
Navigate to the config file in the /etc/sysconfig/network directory:
2
# cd /etc/sysconfig/network
Open the config file in an editor.
3
Append the string eth-id-macaddress to the MANDATORY_DEVICES list in
4
the config file. Separate each address with a space, for example:
MANDATORY_DEVICES="eth-id-00:0C:0D:08:C4:31
eth-id-00:0C:0D:08:C4:32"

Setting up inter-system communication

When you install VCS using the installvcs program, to install and configure the entire clusterat one time, make sure that communicationbetween systems exists. By default the installer uses ssh. You must grant root privileges for the system
Page 45
Performing preinstallation tasks
where you run installvcs program. This privilege facilitates to issue ssh or rsh commands on all systems in the cluster. If ssh is used to communicate between systems, it must be configured in a way suchthat it operates without requests for passwords or passphrases. Similarly, rsh must be configured in such a way to not prompt for passwords.
If system communication is not possible between systems using ssh or rsh, you have recourse.
Warning: The rsh and ssh commands to the remote systems, where VCS is to be installed, must not print any extraneous characters.
Setting up ssh on cluster systems
Use the Secure Shell (ssh) to install VCS on all systems in a cluster from a system outside of the cluster. Before you start the installation process, verify that ssh is configured correctly.
Use Secure Shell (ssh) to do the following:
Log on to another system over a network
Execute commands on a remote system
45Preparing to install VCS
Copy files from one system to another
The ssh shell provides strong authentication and secure communications over channels. It is intended to replace rlogin, rsh, and rcp.
The Remote Shell (rsh) is disabled by default to provide better security. Use ssh for remote command execution.
Configuring ssh
The procedureto configuressh uses OpenSSH example file names and commands.
Note: You can configure ssh in other ways. Regardless of how ssh is configured, complete the last step in the example to verify the configuration.
To configure ssh
Log on to the system from which you want to install VCS.
1
Generate a DSA key pair on this system by running the following command:
2
# ssh-keygen -t dsa
Accept the default location of ~/.ssh/id_dsa.
3
Page 46
Preparing to install VCS
46
Performing preinstallation tasks
When the command prompts, enter a passphrase and confirm it.
4
Change the permissions of the .ssh directory by typing:
5
# chmod 755 ~/.ssh
The file ~/.ssh/id_dsa.pub contains a line that begins with ssh_dss and ends
6
with the name of the system on which it was created. Copy this line to the /root/.ssh/authorized_keys2 fileon all systems where youplan toinstall VCS.
If the local system is part of the cluster, make sure to edit the authorized_keys2 file on that system.
Run the following commands on the system where you are installing:
7
# exec /usr/bin/ssh-agent $SHELL
# ssh-add
This step is shell-specific and is valid for the duration the shell is alive.
When the command prompts, enter your DSA passphrase.
8
You are ready to install VCS on several systems in one of the following ways:
Run the installvcs program on any one of the systems
Run the installvcs program on an independent system outside the cluster
To avoid running the ssh-agent on each shell, run the X-Window system and configure it so that you are not prompted for the passphrase. Refer to the Red Hat documentation for more information.
To verify that you can connect to the systems where you plan to install VCS,
9
type:
# ssh -x -l root north ls
# ssh -x -l root south ifconfig
# ssh-copy-id -i ~/.ssh/id_dsa.pub root@north
The commandsshould executeon theremote systemwithout havingto enter a passphrase or password.

Setting up shared storage

For VCS I/O fencing, the data disks must support SCSI-3 persistent reservations. You needto configure a coordinator disk group thatsupports SCSI-3 PR and verify that it works.
See “About setting up disk-based I/O fencing” on page 89.
Page 47
See also the Veritas Cluster Server User's Guide for a description of I/O fencing.

Setting the PATH variable

Installation commands as well as other commands reside in the /sbin, /usr/sbin, /opt/VRTS/bin, and /opt/VRTSvcs/bin directories. Add these directories to your PATH environment variable.
To set the PATH variable
Do one of the following:
For the Bourne Shell (sh or ksh), type:
$ PATH=/usr/sbin:/sbin:/opt/VRTS/bin:/opt/VRTSvcs/bin:\
$PATH; export PATH
For the C Shell (csh or tcsh), type:
% setenv PATH /usr/sbin:/sbin:/opt/VRTS/bin:\
/opt/VRTSvcs/bin:$PATH
Performing preinstallation tasks
47Preparing to install VCS

Setting the MANPATH variable

Set the MANPATH variable to view the manual pages.
To set the MANPATH variable
Do one of the following:
For the Bourne Shell (sh or ksh), type:
$ MANPATH=/usr/share/man:/opt/VRTS/man; export MANPATH
For the C Shell (csh or tcsh), type:
% setenv MANPATH /usr/share/man:/opt/VRTS/man
If you use the man command to access manual pages, set LC_ALL to "C" in your shell for correct page display.
# export LC_ALL=C
See incident 82099 on the Red Hat support web site for more information.
Page 48
Preparing to install VCS
48
Performing preinstallation tasks

Setting the kernel.panic tunable

By default, the kernel.panic tunable is set to zero. Therefore the kernel does not reboot automatically if a node panics. To ensure that the node reboots automatically after it panics, this tunable must be set to a non zero value.
To set the kernel.panic tunable
Set the kernel.panic tunable to a desired value in the /etc/sysctl.conf file.
1
For example, kernel.panic = 10, will assign a value 10 seconds to the kernel.panic tunable. This step makes the change persistent across reboots.
Run the command:
2
sysctl -w kernel.panic=10
In case of a panic, the node will reboot after 10 seconds.

Optimizing LLT media speed settings on private NICs

For optimal LLT communication among the cluster nodes, the interface cards on each node must use the same media speed settings. Also, the settings for the switches or the hubs that are used for the LLT interconnections must match that of the interface cards. Incorrect settings can cause poor network performance or even network failure.

Guidelines for setting the media speed of the LLT interconnects

Review the following guidelines for setting the media speed of the LLT interconnects:
Symantec recommends that you manually set the same media speed setting
on each Ethernet card on each node.
If you have hubs or switches for LLT interconnects, then set the hub or switch
port to the same setting as used on the cards on each node.
If you use directly connected Ethernet links (using crossover cables), set the
media speed to the highest value common to both cards, typically 100_Full_Duplex.
Symantec does not recommend using dissimilar network cards for private
links.
Details for setting the media speeds for specific devices are outside of the scope of this manual. Consult the devices documentation for more information.
Page 49

Mounting the product disc

You must have superuser (root) privileges to load the VCS software.
To mount the product disc
Log in as superuser on a system where you want to install VCS.
1
The system from which you install VCS need not be part of the cluster. The systems must be in the same subnet.
Insert the product disc with the VCS software into a drive that is connected
2
to the system.
The disc is automatically mounted.
If the disc does not automatically mount, then enter:
3
# mount -o ro /dev/cdrom /mnt/cdrom
Navigate to the location of the RPMs.
4
Depending on the OS distribution, type the following appropriate command:
Performing preinstallation tasks
49Preparing to install VCS
# cd /mnt/cdrom/rhel5_ppc64/cluster_serverRHEL 5
# cd /mnt/cdrom/sles10_ppc64/cluster_serverSLES 10

Performing automated pre-installation check

Before you begin the installation of VCS software, you can check the readiness of the systems where you plan to install VCS. The command to start the pre-installation check is:
installvcs -precheck system1 system2 ...
You can also use the Veritas Installation Assessment Service utility for a detailed assessment of your setup.
See “Veritas Installation Assessment Service” on page 17.
Page 50
Preparing to install VCS
50
Performing preinstallation tasks
To check the systems
Navigate to the folder that contains the installvcs program.
1
See “Mounting the product disc” on page 49.
Start the pre-installation check:
2
# ./installvcs -precheck galaxy nebula
The program proceeds in a noninteractive mode to examine the systems for licenses, RPMs, disk space, and system-to-system communications.
Review the output as the program displays the results of the checkand saves
3
the results of the check in a log file.
Page 51
Chapter

Installing and configuring VCS

This chapter includes the following topics:
About installing and configuring VCS
Getting your VCS installation and configuration information ready
About the VCS installation program
Installing and configuring VCS 5.0 RU3
4
Verifying and updating licenses on the system
Accessing the VCS documentation

About installing and configuring VCS

You can install Veritas Cluster Server on clusters of up to 32 systems. You can install VCS using one of the following:
Use theproduct installerto installmultiple Veritasproducts.Veritas product installer
Use this to install just VCS.installvcs program
The Veritas product installer and the installvcs program use ssh to install by default. Refer to the Getting Started Guide for more information.
Page 52
Installing and configuring VCS
52

Getting your VCS installation and configuration information ready

Getting your VCS installation and configuration information ready
The VCS installation and configuration program prompts you for information about certain VCS components.
When you perform the installation, prepare the following information:
To install VCS RPMs you need:
Example: galaxy, nebulaThe systemnames whereyou
plan to install VCS
The required license keys
To decide whether to install:
the required VCS RPMs
all the VCS RPMs
To configure Veritas Cluster Server you need:
A name for the cluster
A unique ID number for the cluster
Depending on the type of installation, keys include:
A valid site license key
A valid demo license key
A valid license key for VCS global clusters
See “Obtaining VCS license keys” on page 39.
Install only the required RPMs if you do not want to configure any optional components or features.
The default option is to install all RPMs.
See “Optional VCS RPMs” on page 54.
The cluster name must begin with a letter of the alphabet. The cluster name can contain only the characters "a"through "z","A" through"Z", thenumbers "0" through "9", the hyphen "-", and the underscore "_".
Example: vcs_cluster27
A number in the range of 0-65535. Within the site that contains the cluster, each cluster must have a unique ID.
Example: 7
The devicenames ofthe NICs that theprivate networksuse among systems
A network interface card that is not part of any aggregated interface, or an aggregated interface.
Do not use the network interface card that is used for the public network, which is typically eth0.
Example: eth1, eth2
Page 53
Getting your VCS installation and configuration information ready
To configure VCS clusters in secure mode (optional), you need:
53Installing and configuring VCS
For automaticmode (default)
For semiautomatic mode using encrypted files
For semiautomatic mode without usingencrypted files
The name of the Root Broker system
Example: east
See “AboutSymantec ProductAuthentication Service
(AT)on page 19.
Access to the Root Broker system without use of a
password.
The path for the encrypted files that you get from the Root Broker administrator.
See “Creating encrypted files for the security
infrastructureon page 35.
The fully-qualified hostname (FQDN) of the Root
Broker . (e.g. east.symantecexample.com) The given example puts a system in the (DNS)
domain symantecexample.comwith theunqualified hostname east, which is designated as the Root Broker.
The root brokers security domain (e.g.
root@east.symantecexample.com)
The root brokers port (e.g. 2821)
The path to the local root hash (e.g.
/var/tmp/privatedir/root_hash)
The authentication brokers principal nameon each
cluster node(e.g. galaxy.symantecexample.com and nebula.symantecexample.com)
To add VCS users, which is notrequired if you configure yourcluster in secure
mode, you need:
User names
User passwords
To decide user privileges
To configure SMTP email notification (optional), you need:
VCS usernames are restricted to 1024 characters.
Example: smith
VCS passwords are restricted to 255 characters.
Enter the password at the prompt.
Users have three levels of privileges: A=Administrator, O=Operator, or G=Guest.
Example: A
Page 54
Installing and configuring VCS
54
Getting your VCS installation and configuration information ready
The domain-basedaddress of the SMTP server
SMTP recipientto benotified
To decide the minimum severity of events for SMTP email notification
To configure SNMP trap notification (optional), you need:
SNMP trap daemon
SNMP console
To decide the minimum severity of events for SNMP trap notification
To configure global clusters (optional), you need:
The SMTP server sends notification emails about the events within the cluster.
Example: smtp.symantecexample.com
Example: john@symantecexample.comThe email address of each
Events have four levels of severity: I=Information, W=Warning, E=Error, and S=SevereError.
Example: E
The default port number is 162.The port number for the
Example: saturnThe system name for each
Events have four levels of severity: I=Information, W=Warning, E=Error, and S=SevereError.
Example: E
The name of the public NIC
The virtual IP address of the NIC
The netmask for the virtual IP address

Optional VCS RPMs

The optional VCS RPMs include the following packages:
VRTScssim VCS Simulator
VRTScscm Veritas Cluster Server Cluster Manager
You must specify appropriate values for the NIC.
Example: eth0
You must specify appropriate values for the virtual IP address.
Example: 192.168.1.16
You must specify appropriate values for the netmask.
Example: 255.255.240.0
Page 55
VRTSvcsmn Manual pages for VCS commands

About the VCS installation program

You can access the installvcs program from the command line or through the Veritas product installer.
The VCS installation program is interactive and manages the following tasks:
Licensing VCS
Installing VCS RPMs on multiple cluster systems
Configuring VCS, by creating several detailed configuration files on each
system
Starting VCS processes
You can choose to configure different optional features, such as the following:
SNMP and SMTP notification
The Symantec Product Authentication Services feature
The wide area Global Cluster feature
About the VCS installation program
55Installing and configuring VCS
Review the highlights of the information for which installvcs program prompts you as you proceed to configure.
See “About preparing to install VCS” on page 29.
The uninstallvcs program, a companion to installvcs program, uninstalls VCS RPMs.
See “About the uninstallvcs program” on page 153.

Optional features of the installvcs program

Table 4-1 specifies the optional actions that the installvcs program can perform.
Table 4-1
Check the systems to verify that they meet the requirements to install VCS.
Install VCS RPMs without configuring VCS.
Configure or reconfigure VCS when VCS RPMs are already installed.
installvcs optional features
ReferenceOptional action
See “Performingautomated pre-installation
checkon page 49.
See “InstallingVCS usinginstallonly option” on page 59.
See “Configuring VCS using configure
optionon page 59.
Page 56
Installing and configuring VCS
56
About the VCS installation program
Table 4-1
Perform secureinstallations usingthe values that are stored in a configuration file.
Perform automated installations using the values that are stored in a configurationfile.
installvcs optional features (continued)

Interacting with the installvcs program

As you run the program, you are prompted to answer yes or no questions. A set of responses that resemble [y, n, q, ?] (y) typically follow these questions. The response within parentheses is the default, which you can select by pressing the Enter key. Enter the ? character to get help to answer the prompt. Enter q to quit the installation.
Installation of VCS RPMs takes place only after you have confirmed the information. However, you must remove the partially installed VCS files before you run the installvcs program again.
See “About the uninstallvcs program” on page 153.
During theinstallation, the installer prompts you to typeinformation. Theinstaller expects your responses to be within a certain range or in a specific format. The installer providesexamples. If you are promptedto enter an item from a list, enter your selection exactly as it is shown in the list.
The installer also prompts you to answer a series of questions that are related to a configuration activity. For such questions, you can enter the b character to return to the first prompt in the series. When the installer displays a set of information itemsyou have entered, you are prompted toconfirm it. If you answer n, the program lets you reenter all of the information for the set.
You can install the VCS Java Console on a single system, which is not required to be part of the cluster. Note that the installvcs program does not install the VCS Java Console.
See “Installing the Java Console” on page 80.
ReferenceOptional action
See “Installing VCS with a response file
where ssh or rsh are disabledon page 171.
See “Performing automated VCS
installationson page 164.

About installvcs program command options

In addition to the -precheck, -responsefile, -installonly, and -configure options, the installvcs program has other useful options.
The installvcs command usage takes the following form:
Page 57
About the VCS installation program
installvcs [ system1 system2... ] [ options ]
Table 4-2 lists the installvcs command options.
57Installing and configuring VCS
Table 4-2
-configure
encryption_key_file
-encrypt password
-hostfile
-installonly
-installpkgs
-keyfile
ssh_key_file
installvcs options
DescriptionOption and Syntax
Configure VCS after using -installonly option to install VCS.
See “Configuring VCS using configure option” on page 59.
See the -responsefile and the -encrypt options.-enckeyfile
Encrypt password using the encryption key that is provided with the-enckeyfile option so thatthe encryptedpassword can be stored in response files.
Specifies thelocation ofa filethat containsthe systemnames for the installer.
Install product RPMs on systems without configuring VCS.
See “Installing VCS using installonly option” on page 59.
Display VCS packages in correct installation order. Output can be used to create scripts for command line installs, or for installations over a network. See the requiredpkgs option.
Specifies a key file for SSH. The option passes -i ssh_key_file with each SSH invocation.
-license
-logpath log_path
-noextrapkgs
-nolic
Register or update product licenses on the specified systems. Useful for replacing demo license.
Specifies that log_path, not /opt/VRTS/install/logs, is the location whereinstallvcs log files, summary file, and response file are saved.
Specifies that additional product RPMs such as VxVM and VxFS need not be installed.
Note: VCS product upgrades in the future can be simplified
if you do not install additional product RPMs.
Install product RPMs on systems without licensing or configuration. License-based features or variants are not installed when using this option.
Page 58
Installing and configuring VCS
58
About the VCS installation program
Table 4-2
-nooptionalpkgs
-nostart
-pkgpath pkg_path
-precheck
-requiredpkgs
-responsefile
response_file
[-enckeyfile
encryption_key_file]
installvcs options (continued)
DescriptionOption and Syntax
Specifies that the optional product RPMs such as man pages and documentation need not be installed.
Bypass starting VCS after completing installation and configuration.
Specifies that pkg_path contains all RPMs that the installvcs program is about to install on all systems. The pkg_path is the complete path of a directory, usually NFS mounted.
Verify thatsystems meetthe installationrequirements before proceeding with VCS installation.
Symantec recommends doing a precheck before installing VCS.
See “Performing automated pre-installation check” on page 49.
Displays all required VCS packages in correct installation order. Optional packages are not listed. Output can be used to create scripts for command line installs, or forinstallations over a network. See installpkgs option.
Perform automatedVCS installationusing thesystem andthe configuration information that is stored in a specified file instead of prompting for information.
The response_file must be a full path name. If not specified, the response file is automatically generated as installerernumber.response where number is random. You must edit the response file to use it for subsequent installations. Variablefield definitions are defined within the file.
The -enckeyfile option and encryption_key_file name are required withthe -responsefileoption whenthe response file contains encrypted passwords.
See “Installing VCS with a response file where ssh or rsh are
disabledon page 171.
See “Performing automated VCS installations” on page 164.
Page 59
About the VCS installation program
59Installing and configuring VCS
Table 4-2
-rsh
-security
-serial
-timeout
-tmppath tmp_path
installvcs options (continued)
DescriptionOption and Syntax
Specifies that rsh and rcp are to be used for communication between systemsinstead ofssh andscp. Thisoption requires that systems be preconfigured such that rsh commands between systems execute without prompting for passwords or confirmations
Enable or disable Symantec Product Authentication Service in a VCS cluster that is running. Install and configure Root Broker for Symantec Product Authentication Service.
See “About Symantec Product Authentication Service (AT)” on page 19.
Performs the installation, uninstallation, start, and stop operations on the systems in a serial fashion. By default, the installer performsthese operations simultaneously on all the systems.
Specifies the timeout value (in seconds) for each command that the installer issues during the installation. The default timeout value is set to 600 seconds.
Specifies thattmp_path is the working directory for installvcs program. This path is different from the /var/tmp path. This destination is where initial logging is performed and where RPMs are copied on remote systems before installation.
-verbose
Displays the details when the installer installs the RPMs. By default, the installer displays only a progress bar during the RPMs installation.

Installing VCS using installonly option

In certainsituations, users may choose to install theVCS RPMs on a system before they are ready for cluster configuration. During such situations, the installvcs
-installonly option can be used. The installation program licenses and installs
VCS RPMs on the systems that you enter without creating any VCS configuration files.

Configuring VCS using configure option

If you installed VCS and did not choose to configure VCS immediately, use the
installvcs -configure option. You can configure VCS when you are ready for
Page 60
Node: galaxy
Cluster name: vcs_cluster2 Cluster id: 7
VCS private network
Public network
eth1 eth1
eth2eth2
eth0
eth0
Node: nebula
Installing and configuring VCS
60

Installing and configuring VCS 5.0 RU3

cluster configuration. The installvcs program prompts for cluster information, and creates VCS configuration files without performing installation.
See “Configuring the basic cluster” on page 67.
The -configure option can be used to reconfigure a VCS cluster. VCS must not be running on systems when this reconfiguration is performed.
If you manually edited the main.cf file, you need to reformat the main.cf file.
Installing and configuring VCS 5.0 RU3
The exampleinstallation demonstrateshow toinstall VCSon twosystems: galaxy and nebula. The example installation chooses to install all VCS RPMs and configures alloptional features. For this example, the clustersname isvcs_cluster2 and the clusters ID is 7.
Figure 4-1 illustrates the systems on which you would install and run VCS.

Overview of tasks

Figure 4-1
Table 4-3 lists the installation and the configuration tasks.
An example of a VCS installation on a two-node cluster
Page 61
Installing and configuring VCS 5.0 RU3
61Installing and configuring VCS
Table 4-3
License andinstall VCS
Configure the cluster and its features
files
Start VCS and its components
Installation and configuration tasks
ReferenceTask
See Starting the software installationon page 61.
See Specifying systems for installationon page 62.
See Licensing VCSon page 63.
See Choosing VCS RPMs for installationon page 64.
See Choosing to install VCS RPMs or configure VCS
on page 65.
See Installing VCS RPMson page 75.
See Starting the software configurationon page 66.
See Specifying systems for configurationon page 67.
See Configuring the basic clusteron page 67.
See Adding VCS userson page 71. (optional)
See Configuring SMTP email notificationon page 71.
(optional)
See “Configuring SNMP trap notificationon page 73.
(optional)
See “Configuring global clusterson page 74. (optional)
See “Creating VCS configuration files” on page 76.Create configuration
See Starting VCSon page 77.
See Completing the installationon page 77.
For clustersthat runin
See “Aboutenabling LDAPauthentication for clusters that run
secure mode, enable LDAP authentication plug-in if VCS users belong to LDAP domain.
Perform the
See “About configuring VCS clusters for data integrity
post-installation tasks
See “Installing the Java Consoleon page 80.
See “Verifying the cluster after installation” on page 84.Verify the cluster

Starting the software installation

You can install VCS using the Veritas product installer or the installvcs program.
in secure modeon page 78.
on page 87.
Page 62
Installing and configuring VCS
62
Installing and configuring VCS 5.0 RU3
Note: The system from where you install VCS must run thesame Linuxdistribution
as the target systems.
To install VCS using the product installer
1
2
3
4
To install VCS using the installvcs program
1
2
Confirm that you are logged in as the superuser and mounted the product disc.
Start the installer.
# ./installer
The installerstarts the product installation program with acopyright message and specifies the directory where the logs are created.
From the opening Selection Menu, choose: I for "Install/Upgrade a Product."
From the displayed list of products to install, choose: Veritas Cluster Server.
Confirm that you are logged in as the superuser and mounted the product disc.
Navigate to the folder that contains the installvcs program.
# cd /cluster_server
Start the installvcs program.
3
# ./installvcs [-rsh]
The installer begins with a copyright message and specifies the directory where the logs are created.

Specifying systems for installation

The installerprompts forthe systemnames onwhich youwant toinstall andthen performs an initial system check.
Page 63

Licensing VCS

Installing and configuring VCS 5.0 RU3
To specify system names for installation
Enter the names of the systems where you want to install VCS.
1
Enter the system names separated by spaces on which to install
VCS: galaxy nebula
For a single node installation, enter one name for the system.
See “Creatinga single-node cluster using the installer program” on page139.
Review the output as the installer verifies the systems you specify.
2
The installer does the following:
Checks that the local node running the installer can communicate with
remote nodes If the installer finds ssh binaries, it confirms thatssh can operate without
requests for passwords or passphrases.
Makes sure the systems use the proper operating system
Checks whether VCS is installed
63Installing and configuring VCS
The installer checks whether VCS license keys are currently in place on each system. If license keys are not installed, the installer prompts you for the license keys.
See “Checking licensing information on the system” on page 84.
To license VCS
Review the output as the utility checks system licensing and installs the
1
licensing RPM.
Enter the license key for Veritas Cluster Server as the installer prompts for
2
each node.
Enter a VCS license key for galaxy: [?] XXXX-XXXX-XXXX-XXXX-XXX
XXXX-XXXX-XXXX-XXXX-XXX successfully registered on galaxy
VCS license registered on galaxy
Page 64
Installing and configuring VCS
64
Installing and configuring VCS 5.0 RU3
3
4
Enter keys for additional product features.
Do you want to enter another license key for galaxy? [y,n,q,?]
(n) y
Enter a VCS license key for galaxy: [?] XXXX-XXXX-XXXX-XXXX-XXX
XXXX-XXXX-XXXX-XXXX-XXX successfully registered on galaxy
Do you want to enter another license key for galaxy? [y,n,q,?]
(n)
Review theoutput as the installer registersthe license key on the other nodes. Enter keys for additional product features on the other nodes when the installer prompts you.
XXXX-XXXX-XXXX-XXXX-XXX successfully registered on nebula
VCS license registered on nebula
Do you want to enter another license key for nebula? [y,n,q,?]
(n)

Choosing VCS RPMs for installation

The installer verifies for any previously installed RPMs and then based on your choice installs all the VCS RPMs or only the required RPMs.
Page 65
Installing and configuring VCS 5.0 RU3
To install VCS RPMs
Review theoutput asthe installerchecks theRPMs thatare alreadyinstalled.
1
Choose the VCS RPMs that you want to install.
2
Select the RPMs to be installed on all systems? [1-3,q,?]
(3) 2
Based on what RPMs you want to install, enter one of the following:
Installs only the required VCS RPMs.1
Installs all the VCS RPMs.
2
You must choose this option to configure any optional VCS feature.
Note that this option is the default if you already installed the SF HA RPMs.
Installs all the VCS and the SF HA RPMs. (default option)
3
If you already installed the SF HA RPMs, theinstaller doesnot listthis option.
View the list of RPMs that the installer would install on each node.
3
If the current version of a RPM is on a system, the installer removes it from the RPM installation list for the system.
65Installing and configuring VCS

Choosing to install VCS RPMs or configure VCS

While you must configure VCS before you can use VCS, you can do one of the following:
Choose to install and configure VCS now.
See “Configuring the basic cluster” on page 67.
Install packages on the systems and leave the cluster configuration steps for
later.
Page 66
Installing and configuring VCS
66
Installing and configuring VCS 5.0 RU3
To install VCS packages now and configure VCS later
If you do not want to configure VCS now, enter n at the prompt.
1
Are you ready to configure VCS? [y,n,q] (y) n
The utility checks for the required file system space and makes sure that any processes that are running do not conflict with the installation. If requirements for installation are not met, the utility stops and indicates the actions required to proceed with the process.
Review theoutput as the installer uninstalls any previousversions andinstalls
2
the VCS 5.0 RU3 packages.
Configure the cluster later.
3
See “Configuring VCS using configure option” on page 59.

Starting the software configuration

You can configure VCS using the Veritas product installer or the installvcs program.
To configure VCS using the product installer
Confirm that you are logged in as the superuser and mounted the product
1
disc.
Start the installer.
2
# ./installer
The installerstarts the product installation program with acopyright message and specifies the directory where the logs are created.
From the opening Selection Menu, choose: C for "Configure an Installed
3
Product."
From the displayed list of products to configure, choose: Veritas Cluster
4
Server.
Page 67
To configure VCS using the installvcs program
Confirm that you are logged in as the superuser and mounted the product
1
disc.
Navigate to the folder that contains the installvcs program.
2
# cd /dvdrom/cluster_server
Start the installvcs program.
3
# ./installvcs -configure
The installer begins with a copyright message and specifies the directory where the logs are created.

Specifying systems for configuration

The installer prompts for the system names on whichyou want to configure VCS. The installer performs an initial check on the systems that you specify.
To specify system names for installation
Enter the names of the systems where you want to configure VCS.
1
Installing and configuring VCS 5.0 RU3
67Installing and configuring VCS
Enter the system names separated by spaces on which to configure
VCS: galaxy nebula
Review the output as the installer verifies the systems you specify.
2
The installer does the following tasks:
Checks that the local node running the installer can communicate with
remote nodes If theinstaller finds ssh binaries, it confirms thatssh can operate without
requests for passwords or passphrases.
Makes sure the systems use the proper operating system
Checks whether VCS is installed
Exits if VCS 5.0 RU3 is not installed

Configuring the basic cluster

Enter the cluster information when the installer prompts you.
Page 68
Installing and configuring VCS
68
Installing and configuring VCS 5.0 RU3
To configure the cluster
1
2
3
4
Review the configuration instructions that the installer presents.
Enter the unique cluster name and cluster ID.
Enter the unique cluster name: [?] clus1
Enter the unique Cluster ID number between 0-65535: [b,?] 7
Review the NICs available on the first system as the installer discovers and reports them.
The private heartbeats can either use NIC or aggregated interfaces. To use aggregated interfacesfor private heartbeat, enter the name ofthe aggregated interface. To use a NIC for private heartbeat, enter a NIC which is not part of an aggregated interface.
Enter the network interface card details for the private heartbeat links.
You canchoose the network interface cards or theaggregated interfaces that the installer discovers.
See “Starting VCS” on page 77.
You must not enter the network interface card that is used for the public network (typically eth0.)
Enter the NIC for the first private heartbeat NIC on galaxy:
[b,?] eth1
Would you like to configure a second private heartbeat link?
[y,n,q,b,?] (y)
Enter the NIC for the second private heartbeat NIC on galaxy:
[b,?] eth2
Would you like to configure a third private heartbeat link?
[y,n,q,b,?](n)
Do you want to configure an additional low priority heartbeat
link? [y,n,q,b,?] (n)
Page 69
Choose whether to use the same NIC details to configure private heartbeat
5
links on other systems.
Are you using the same NICs for private heartbeat links on all
systems? [y,n,q,b,?] (y)
If you want to use the NIC details that you entered for galaxy, make sure the same NICs are available on each system. Then, enter y at the prompt.
If theNIC devicenames aredifferent onsome of the systems, enter n. Provide the NIC details for each system as the program prompts.
Verify and confirm the information that the installer summarizes.
6

Configuring the cluster in secure mode

If you want to configure the cluster in secure mode, make sure that you meet the prerequisites for secure cluster configuration.
The installvcs program provides different configuration modes to configure a secure cluster. Make sure that you completed the pre-configuration tasks for the configuration mode that you want to choose.
See “Preparing to configure the clusters in secure mode” on page 29.
To configure the cluster in secure mode
Installing and configuring VCS 5.0 RU3
69Installing and configuring VCS
Choose whether to configure VCS to use Symantec Product Authentication
1
Service.
Would you like to configure VCS to use Symantec Security
Services? [y,n,q] (n) y
If you want to configure the cluster in secure mode, make sure you meet
the prerequisites and enter y.
If you do not want to configure the cluster in secure mode, enter n.
You must add VCS users when the configuration program prompts. See “Adding VCS users” on page 71.
Select one of the options to enable security.
2
Select the Security option you would like to perform [1-3,q,?]
Review the following configuration modes. Based on the configuration that you want to use, enter one of the following values:
Page 70
Installing and configuring VCS
70
Installing and configuring VCS 5.0 RU3
Option 1. Automatic configuration
Option 2. Semiautomatic configuration
Option 3. Manual configuration
Enter the name of the Root Broker system when prompted.
Requires a remote access to the Root Broker.
Review theoutput asthe installerverifies communication with the Root Broker system, checks vxatd process and version, and checks security domain.
Enter the path of the encrypted file (BLOB file) for each node when prompted.
Enter the following Root Broker information as the installer prompts you:
Enter root Broker name:
east.symantecexample.com
Enter root broker FQDN: [b]
(symantecexample.com)
symantecexample.com
Enter root broker domain: [b]
(root@east.symantecexample.com)
root@east.symantecexample.com
Enter root broker port: [b] (2821) 2821
Enter path to the locally accessible
root hash [b] (/var/tmp/
installvcs-1Lcljr/root_hash)
/root/root_hash
Enter the following Authentication Broker information as the installer prompts you for each node:
Enter authentication broker principal name on
galaxy [b]
(galaxy.symantecexample.com)
galaxy.symantecexample.com
Enter authentication broker password on galaxy:
Enter authentication broker principal name on
nebula [b]
(nebula.symantecexample.com)
nebula.symantecexample.com
Enter authentication broker password on nebula:
After you provide the requiredinformation to configure the cluster in secure
3
mode, the program prompts you to configure SMTP email notification.
Note thatthe installer does not prompt you toadd VCS users if you configured the cluster in secure mode. However, you must add VCS users later.
Page 71

Adding VCS users

If you have enabled Symantec Product Authentication Service, you do not need to add VCS users now. Otherwise, on systems operating under an English locale, you can add VCS users at this time.
To add VCS users
1
2
3
Installing and configuring VCS 5.0 RU3
71Installing and configuring VCS
See Veritas Cluster Server User's Guide for more information.
Review the required information to add VCS users.
Reset the password for the Admin user, if necessary.
Do you want to set the password for the Admin user
(default password=’password’)? [y,n,q] (n) y
Enter New Password:******
Enter Again:******
To add a user, enter y at the prompt.
Do you want to add another user to the cluster? [y,n,q] (y)
Enter the users name, password, and level of privileges.
4
Enter the user name: [?] smith
Enter New Password:*******
Enter Again:*******
Enter the privilege for user smith (A=Administrator, O=Operator,
G=Guest): [?] a
Enter n at the prompt if you have finished adding users.
5
Would you like to add another user? [y,n,q] (n)
Review the summary of the newly added users and confirm the information.
6

Configuring SMTP email notification

You can choose to configure VCS to send event notifications to SMTP email services. Youneed to provide the SMTP server nameand emailaddresses of people to be notified. Note that you can also configure the notification after installation.
Page 72
Installing and configuring VCS
72
Installing and configuring VCS 5.0 RU3
Refer to the Veritas Cluster Server Users Guide for more information.
To configure SMTP email notification
1
2
3
Review the required information to configure the SMTP email notification.
Specify whether you want to configure the SMTP notification.
Do you want to configure SMTP notification? [y,n,q] (y) y
If you do not want to configure the SMTP notification, you can skip to the next configuration option.
See “Configuring SNMP trap notification” on page 73.
Provide information to configure SMTP notification.
Provide the following information:
Enter the SMTP servers host name.
Enter the domain-based hostname of the SMTP server
(example: smtp.yourcompany.com): [b,?] smtp.example.com
Enter the email address of each recipient.
Enter the full email address of the SMTP recipient
(example: user@yourcompany.com): [b,?] ozzie@example.com
Enter theminimum security level of messagesto besent to each recipient.
Enter the minimum severity of events for which mail should be
sent to ozzie@example.com [I=Information, W=Warning,
E=Error, S=SevereError]: [b,?] w
Add more SMTP recipients, if necessary.
4
If you want to add another SMTP recipient, enter y and provide the
required information at the prompt.
Would you like to add another SMTP recipient? [y,n,q,b] (n) y
Enter the full email address of the SMTP recipient
(example: user@yourcompany.com): [b,?] harriet@example.com
Enter the minimum severity of events for which mail should be
sent to harriet@example.com [I=Information, W=Warning,
E=Error, S=SevereError]: [b,?] E
Page 73
If you do not want to add, answer n.
Would you like to add another SMTP recipient? [y,n,q,b] (n)
Verify and confirm the SMTP notification information.
5
SMTP Address: smtp.example.com
Recipient: ozzie@example.com receives email for Warning or
higher events
Recipient: harriet@example.com receives email for Error or
higher events
Is this information correct? [y,n,q] (y)

Configuring SNMP trap notification

You canchoose toconfigure VCS to send event notifications to SNMP management consoles. Youneed toprovide theSNMP managementconsole nameto benotified and message severity levels.
Note that you can also configure the notification after installation.
Refer to the Veritas Cluster Server User's Guide for more information.
To configure the SNMP trap notification
Installing and configuring VCS 5.0 RU3
73Installing and configuring VCS
Review the required information to configure the SNMP notification feature
1
of VCS.
Specify whether you want to configure the SNMP notification.
2
Do you want to configure SNMP notification? [y,n,q] (y)
If you skip this option and if you had installed a valid HA/DR license, the installer presentsyou with an option toconfigure this cluster as global cluster. If you did not install an HA/DR license, the installer proceeds to configure VCS based on the configuration details you provided.
See “Configuring global clusters” on page 74.
Provide information to configure SNMP trap notification.
3
Provide the following information:
Enter the SNMP trap daemon port.
Enter the SNMP trap daemon port: [b,?] (162)
Enter the SNMP console system name.
Page 74
Installing and configuring VCS
74
Installing and configuring VCS 5.0 RU3
4
Enter the SNMP console system name: [b,?] saturn
Enter the minimum security level of messages to be sent to each console.
Enter the minimum severity of events for which SNMP traps
should be sent to saturn [I=Information, W=Warning, E=Error,
S=SevereError]: [b,?] E
Add more SNMP consoles, if necessary.
If youwant to add another SNMP console, entery and provide the required
information at the prompt.
Would you like to add another SNMP console? [y,n,q,b] (n) y
Enter the SNMP console system name: [b,?] jupiter
Enter the minimum severity of events for which SNMP traps
should be sent to jupiter [I=Information, W=Warning,
E=Error, S=SevereError]: [b,?] S
If you do not want to add, answer n.
Would you like to add another SNMP console? [y,n,q,b] (n)
Verify and confirm the SNMP notification information.
5
SNMP Port: 162
Console: saturn receives SNMP traps for Error or
higher events
Console: jupiter receives SNMP traps for SevereError or
higher events
Is this information correct? [y,n,q] (y)

Configuring global clusters

If you had installed a valid HA/DR license, the installer provides you an option to configure this cluster as global cluster. If not, the installer proceeds to configure VCS based on the configuration details you provided. You can also run the gcoconfig utility in each cluster later to update the VCS configuration file for global cluster.
You can configure global clustersto link clusters at separate locations and enable wide-area failover and disaster recovery. The installer adds basic global cluster
Page 75
Installing and configuring VCS 5.0 RU3
information to the VCS configuration file. You must perform additional configuration tasks to set up a global cluster.
See Veritas Cluster Server User's Guide for instructions to set up VCS global clusters.
Note: If you installed a HA/DR license to set up replicated data cluster or campus cluster, skip this installer option.
To configure the global cluster option
Review the required information to configure the global cluster option.
1
Specify whether you want to configure the global cluster option.
2
Do you want to configure the Global Cluster Option? [y,n,q] (y)
If you skip this option, the installer proceeds to configure VCS based on the configuration details you provided.
Provide information to configure this cluster as global cluster.
3
The installer prompts you for a NIC, a virtual IP address, and value for the netmask.
You can also enter an IPv6 address as a virtual IP address.
Verify and confirm the configuration of the global cluster.
4
75Installing and configuring VCS

Installing VCS RPMs

After the installer gathers allthe configuration information, the installer installs the RPMs on the cluster systems. If you already installed the RPMs and chose to configure or reconfigure the cluster, the installer proceeds to create the configuration files.
See “Creating VCS configuration files” on page 76.
The utility checks for the required file system space and makes sure that any processes that are running do not conflict with the installation. If requirements
Global Cluster Option configuration verification:
NIC: eth0
IP: 192.168.1.16
Netmask: 255.255.240.0
Is this information correct? [y,n,q] (y)
Page 76
Installing and configuring VCS
76
Installing and configuring VCS 5.0 RU3
for installation are not met, the utility stops and indicates the actions that are required toproceed with the process. Review the output as the installer uninstalls any previous versions and installs the VCS 5.0 RU3 RPMs.

Creating VCS configuration files

After youinstall the RPMs and provide the configurationinformation, theinstaller continues to create configuration files and copies them to each system.
Creating Cluster Server configuration files ............ Done
Copying configuration files to galaxy.................... Done
Copying configuration files to nebula.................... Done
Cluster Server configured successfully.
If you chose to configure the cluster in secure mode, the installer also configures the Symantec Product Authentication Service.
Depending on the mode you chose to set up Authentication Service, the installer does one of the following:
Creates the security principal
Executes the encrypted file to create security principal on each node in the
cluster
The installer then does the following before the installer starts VCS in secure mode:
Creates the VxSS service group
Creates the Authentication Server credentials on each node in the cluster
Creates the Web credentials for VCS users
Sets up trust with the root broker
Verifying the NIC configuration
The installer verifies on all the nodes if all NICs have PERSISTENT_NAME set correctly.
If the persistent interface names are not configured correctly for the network devices, the installer displays the following warnings:
Verifying that all NICs have PERSISTENT_NAME set correctly on
galaxy:
For VCS to run correctly, the names of the NIC cards must be
boot persistent.
Page 77

Starting VCS

Installing and configuring VCS 5.0 RU3
77Installing and configuring VCS
CPI WARNING V-9-122-1021
No PERSISTENT_NAME set for NIC with MAC address
00:11:43:33:17:28 (present name eth0), though config file exists!
CPI WARNING V-9-122-1022
No config file for NIC with MAC address 00:11:43:33:17:29
(present name eth1) found!
CPI WARNING V-9-122-1022
No config file for NIC with MAC address 00:04:23:ac:25:1f
(present name eth3) found!
PERSISTENT_NAME is not set for all the NICs.
You need to set them manually before the next reboot.
Set the PERSISTENT_NAME for all the NICs.
Warning: If the installer finds the network interface name to be different from the name in the configuration file, then the installer exits.
You can now start VCS and its components on each system. If you chose to configure the cluster in secure mode, the installer also starts the Authentication Service processes on each node in the cluster.
To start VCS
Confirm to start VCS and its components on each node.
Enter y if you want to start VCS.
Do you want to start Veritas Cluster Server processes now?
[y,n,q] (y) n

Completing the installation

After VCS 5.0 RU3 installation completes successfully, the installer creates summary, log, and response files. The files provide the useful information that can assist you with the installation and can also assist future installations.
Review the location of the installation log files, summary file, and response file that the installer displays.
Table 4-4 specifies the files that are created at the end of the installation.
Page 78
Installing and configuring VCS
78
Installing and configuring VCS 5.0 RU3
Table 4-4
File description
DescriptionFile
summary file
response file
Lists the RPMs that are installed on each system.
Describes the cluster and its configured resources.
Provides the information for managing the cluster.
Details the entire installation.log file
Contains the configuration information that can be used to perform secure or unattended installations on other systems.
See “Example response file” on page 165.

About enabling LDAP authentication for clusters that run in secure mode

Symantec Product Authentication Service (AT) supports LDAP (Lightweight Directory Access Protocol) user authentication through a plug-in for the authentication broker. AT supports all common LDAP distributions such as Sun Directory Server, Netscape, OpenLDAP, and Windows Active Directory.
For a cluster that runs in secure mode, you must enable the LDAP authentication plug-in ifthe VCS users belong to an LDAP domain. Toenable LDAPauthentication plug-in, you must verify theLDAP environment, add the LDAP domain in AT, and then verifyLDAP authentication.The ATcomponent packagedwith VCSrequires you to manually edit the VRTSatlocal.conf file to enable LDAP authentication.
Refer to the Symantec Product Authentication Service Administrators Guide for instructions.
If youhave not already added VCS users during installation, you can add the users later.
See the Veritas Cluster Server User's Guide for instructions to add VCS users.
Figure 4-2 depicts the VCS cluster communication with the LDAP servers when
clusters run in secure mode.
Page 79
VCS node (authentication broker)
VCS client
1. When a user runs HA commands, AT initiates user authentication with the authentication broker.
4. AT issues the credentials to the user to proceed with the command.
2. Authentication broker on VCS node performs an LDAP bind operation with the LDAP directory.
3. Upon a successful LDAP bind, AT retrieves group information from the LDAP direcory.
LDAP server (such as OpenLDAP or Windows Active Directory)
Installing and configuring VCS 5.0 RU3
79Installing and configuring VCS
Figure 4-2
Client communication with LDAP servers
See the Symantec Product Authentication Service Administrators Guide.
The LDAPschema and syntax for LDAP commands (such as, ldapadd, ldapmodify, and ldapsearch) vary based on your LDAP implementation.
Before adding the LDAP domain in Symantec Product Authentication Service, note the following information about your LDAP environment:
The type of LDAP schema used (the default is RFC 2307)
UserObjectClass (the default is posixAccount)
UserObject Attribute (the default is uid)
User Group Attribute (the default is gidNumber)
Group Object Class (the default is posixGroup)
GroupObject Attribute (the default is cn)
Group GID Attribute (the default is gidNumber)
Group Membership Attribute (the default is memberUid)
URL to the LDAP Directory
Page 80
Installing and configuring VCS
80
Installing and configuring VCS 5.0 RU3
Distinguished name for the user container (for example,
UserBaseDN=ou=people,dc=comp,dc=com)
Distinguished name for the group container (for example,
GroupBaseDN=ou=group,dc=comp,dc=com)

Installing the Java Console

You can administer VCS using the VCS Java-based graphical user interface, Java Console. After VCS has been installed, install the Java Console on a Windows system or Linux system. Review the software requirements for Java Console.
See “Software requirements for the Java Console” on page 80.
The system from which you run the Java Console can be a system in the cluster or aremote workstation.A remoteworkstation enables each system in the cluster to be administered remotely.
Review the information about using the Cluster Manager and the Configuration Editor componentsof theJava Console. For more information, refer to the Veritas Cluster Server User's Guide.
Software requirements for the Java Console
Cluster Manager (Java Console) is supported on:
RHEL 4 Update 3, RHEL 5, SLES 9 SP3, SLES 10, and SLES 11
Windows XP, Windows 2003 Server Edition
Note: Make sure that you are using an operating system version that supports JRE 1.5.
Hardware requirements for the Java Console
The minimum hardware requirements for the Java Console follow:
Pentium II 300 megahertz
256 megabytes of RAM
800x600 display resolution
8-bit color depth of the monitor
A graphics card that is capable of 2D images
Page 81
Installing and configuring VCS 5.0 RU3
Note: Symantec recommends using Pentium III, 400MHz, 256MB RAM, and
800x600 display resolution.
The version of the Java2 Runtime Environment (JRE) requires 32 megabytes of RAM. Thisversion issupported onthe IntelPentium platformsthat runthe Linux kernel v 2.2.12 and glibc v2.1.2-11 (or later).
Symantec recommends using the following hardware:
48 megabytes of RAM
16-bit color mode
The KDE and the KWM window managers that are used with displays set to
local hosts
Installing the Java Console on Linux for IBM Power
Review the procedure to install the Java console.
To install Java console on Linux
Insert the VCS software disc into a drive on the system.
1
The software automatically mounts the disc on /mnt/cdrom.
If the disc does not get automatically mounted, then enter:
2
81Installing and configuring VCS
# mount -o ro /dev/cdrom /mnt/cdrom
Navigate to the folder that contains the RPMs.
3
# cd /mnt/cdrom/dist_arch/cluster_server/rpms
Where dist is the Linux distribution, rhel5 or sles10 and arch is the architecture, ppc64.
Install the RPM using rpm -i command.
4
# rpm -i VRTScscm-5.0.33.00-RU3_GENERIC.noarch.rpm
Installing the Java Console on a Windows system
Review the procedure to install the Java console on a Windows system.
To install the Java Console on a Windows system
Insert the software disc with the VCS software into a drive on your Windows
1
system.
Using Windows Explorer, select the disc drive.
2
Page 82
Installing and configuring VCS
82
Installing and configuring VCS 5.0 RU3
Go to \windows\VCSWindowsInstallers\ClusterManager.
3
Open the language folder of your choice, for example EN.
4
Double-click setup.exe.
5
The Veritas Cluster Manager Install Wizard guides you through the
6
installation process.

Installing VCS Simulator

You can administer VCS Simulator from the Java Console or from the command line. Review the software requirements for VCS Simulator.
Software requirements for VCS Simulator
VCS Simulator is supported on:
RHEL 4 Update 3, RHEL 5, SLES 9 SP3, SLES 10, and SLES 11
Windows XP, Windows 2003
Note: Make sure that you are using an operating system version that supports JRE 1.5.
Installing VCS Simulator on UNIX systems
This section describes the procedure to install VCS Simulator on UNIX systems.
To install VCS Simulator on UNIX systems
Insert the VCS installation disc into a drive.
1
Navigate to the following directory and locate VRTScssim.
2
Linuxrpms
Install the VRTScssim package.
3
To use Cluster Manager with Simulator, you must also install the VRTScscm package.
Installing VCS Simulator on Windows systems
This sectiondescribes theprocedure to install VCS Simulator on Windowssystems.
Page 83
Installing and configuring VCS 5.0 RU3
To install VCS Simulator on Windows systems
Insert the VCS installation disc into a drive.
1
Navigate to the path of the Simulator installer file:
2
\your_platform_architecture\cluster_server\windows\ VCSWindowsInstallers\Simulator
Double-click the installer file.
3
Read the information in the Welcome screen and click Next.
4
In the Destination Folders dialog box, click Next to accepted the suggested
5
installation path or click Change to choose a different location.
In the Ready to Install the Program dialog box, click Back to make changes
6
to your selections or click Install to proceed with the installation.
In the Installshield Wizard Completed dialog box, click Finish.
7
Reviewing the installation
VCS Simulator installs Cluster Manager (Java Console) and Simulator binaries on the system. The Simulator installation creates the following directories:
83Installing and configuring VCS
ContentDirectory
Information about attributes associated with VCS objectsattrpool
VCS Simulator binariesbin
Files for the default cluster configurationdefault_clus
sample_clus
conf
A samplecluster configuration, whichserves asa templatefor each new cluster configuration
Various templates that are used by the Java Consoletemplates
The types.cf files for all supported platformstypes
Contains another directory called types. This directory contains assorted resourcetype definitions that are useful for the Simulator. The type definition files are present in platform-specific sub directories.
Additionally, VCSSimulator installs directories for variouscluster configurations.
VCS Simulator creates a directory for every new simulated cluster and copies the contents ofthe sample_clus directory. Simulator also creates alog directory within each cluster directory for logs that are associated with the cluster.
Page 84
Installing and configuring VCS
84

Verifying and updating licenses on the system

Verifying the cluster after installation

When you have used installvcs program and chosen to configure and start VCS, VCS andall components are properly configured and canstart correctly.You must verify that your cluster operates properly after the installation.
See “About verifying the VCS installation” on page 105.
Verifying and updating licenses on the system
After you install VCS, you can verify the licensing information using the vxlicrep program. You can replace the demo licenses with a permanent license.

Checking licensing information on the system

You can use the vxlicrep program to display information about the licenses on a system.
To check licensing information
Navigate to the folder containing the vxlicrep program and enter:
1
# cd /opt/VRTS/bin
# ./vxlicrep
Review the following output to determine the following information:
2
The license key
The type of license
The product for which it applies
Its expiration date, if any. Demo keys have expiration dates. Permanent
keys and site keys do not have expiration dates.
License Key = xxx-xxx-xxx-xxx-xxx
Product Name = Veritas Cluster Server
Serial Number = 1249
License Type = PERMANENT
OEM ID = 478
Features :=
Platform = Linux for IBM Power
Version = 5.0
Tier = 0
Page 85
Reserved = 0
Mode = VCS

Updating product licenses using vxlicinst

You can use the vxlicinst command to add the VCS license key on each node. If you have VCS already installed and configured and you use a demo license, you can replace the demo license.
See “Replacing a VCS demo license with a permanent license” on page 85.
To update product licenses
On each node, enter the license key using the command:
# cd /opt/VRTS/bin
# ./vxlicinst -k XXXX-XXXX-XXXX-XXXX-XXXX-XXX
Replacing a VCS demo license with a permanent license
When a VCS demonstration key license expires, you can replace it with a permanent license using the vxlicinst(1) program.
Verifying and updating licenses on the system
85Installing and configuring VCS
To replace a demo key
Make sure you have permissions to log in as root on each of the nodes in the
1
cluster.
Shut down VCS on all nodes in the cluster:
2
# hastop -all -force
This command does not shut down any running applications.
Enter the permanent license key using thefollowing command on each node:
3
# cd /opt/VRTS/bin
# ./vxlicinst -k XXXX-XXXX-XXXX-XXXX-XXXX-XXX
Page 86
Installing and configuring VCS
86

Accessing the VCS documentation

Make sure demo licenses are replaced on all cluster nodes before starting
4
VCS.
# cd /opt/VRTS/bin
# ./vxlicrep
Start VCS on each node:
5
# hastart
Accessing the VCS documentation
The software disc contains the documentation for VCS in Portable Document Format (PDF)in the cluster_server/docs directory. After you installVCS, Symantec recommends that you copy the PDF version of the documents to the /opt/VRTS/docs directory on each node to make it available for reference.
To access the VCS documentation
Copy the PDF from the software disc (cluster_server/docs/) to the directory
/opt/VRTS/docs.
Page 87
Chapter

Configuring VCS clusters for data integrity

This chapter includes the following topics:
About configuring VCS clusters for data integrity
About I/O fencing components
About setting up disk-based I/O fencing
Preparing to configure disk-based I/O fencing
5
Setting up disk-based I/O fencing manually

About configuring VCS clusters for data integrity

When a node fails, VCS takes corrective action and configures its components to reflect the altered membership. If an actual node failure did not occur and if the symptoms were identical to those of a failed node, then such corrective action would cause a split-brain situation.
Some example scenarios that can causesuch split-brain situations are as follows:
Broken set of private networks
If a system in a two-node cluster fails, the system stops sending heartbeats over the private interconnects. The remaining node then takes corrective action. The failure of the private interconnects, instead of the actual nodes, presents identical symptoms and causes each node to determine its peer has departed. Thissituation typically results in data corruption becauseboth nodes try to take control of data storage in an uncoordinated manner
System that appears to have a system-hang
Page 88
Configuring VCS clusters for data integrity
88

About I/O fencing components

If a system is so busy that it appears to stop responding, the othernodes could declare it as dead. This declaration may also occur for the nodes that use the hardware that supports a "break" and "resume" function. When a node drops to PROM level with a break and subsequently resumes operations, the other nodes maydeclare the system dead. They can declareit dead even if the system later returns and begins write operations.
I/O fencing is a feature that prevents data corruption in the event of a communication breakdown in a cluster. VCS uses I/O fencing to remove the risk that is associated with split brain. I/O fencing allows write access for members of the active cluster. It blocks access to storage from non-members so that even a node that is alive is unable to cause damage.
After you install and configure VCS, you must configure I/O fencing in VCS to ensure data integrity.
About I/O fencing components
The sharedstorage for VCS must support SCSI-3 persistentreservations to enable I/O fencing. VCS involves two types of shared storage:
Data disksStore shared data
See “About data disks” on page 88.
Coordination pointsAct as a global lock during membership changes
See “About coordination points” on page 88.

About data disks

Data disksare standard disk devices for data storageand are either physical disks or RAID Logical Units (LUNs). These disks must support SCSI-3 PR and are part of standard VxVM or CVM disk groups.
CVM is responsible for fencing data disks on a disk group basis. Disks that are added to a disk group and new paths that are discovered for a device are automatically fenced.

About coordination points

Coordination points provide a lock mechanism to determine which nodes get to fence off data drives from other nodes. A node must eject a peer from the coordination points before it can fence the peer from the data drives. Racing for control of the coordination points to fence data disks is the key to understand how fencing prevents split brain.
Page 89

About setting up disk-based I/O fencing

Disks that act as coordination points are called coordinator disks. Coordinator disks are three standard disks or LUNs set aside for I/O fencing during cluster reconfiguration. Coordinator disks do not serve any other storage purpose in the VCS configuration.
You can configure coordinator disks to use Veritas Volume Manager Dynamic Multipathing (DMP) feature. Dynamic Multipathing (DMP) allows coordinator disks to take advantage of the path failover and the dynamic adding and removal capabilities of DMP. So, you can configure I/O fencing to use either DMP devices or the underlying raw character devices. I/O fencing uses SCSI-3 disk policy that is either raw or dmp based on the disk device that you use. The disk policy is raw by default. Symantec recommends using the DMP disk policy.
See the Veritas Volume Manager Administrators Guide.
About setting up disk-based I/O fencing
Figure 5-1 illustrates the tasks involved to configure I/O fencing.
89Configuring VCS clusters for data integrity
Page 90
Initialize disks as VxVM disks
Identify disks to use as coordinator disks
Set up coordinator disk group
Create I/O fencing configuration files
Modify VCS configuration to use I/O fencing
Verify I/O fencing configuration
Check shared disks for I/O fencing compliance
Preparing to set up I/O fencing
Setting up I/O fencing
Configuring VCS clusters for data integrity
90
About setting up disk-based I/O fencing
Figure 5-1
Workflow to configure disk-based I/O fencing
See “Preparing to configure disk-based I/O fencing” on page 92.
See “Setting up disk-based I/O fencing manually” on page 99.
I/O fencing requires the coordinator disks be configured in a disk group. The coordinator disksmust be accessible to each node inthe cluster. These disks enable the vxfen driver to resolve potential split-brain conditions and prevent data corruption.
Review the following requirements for coordinator disks:
You must have three coordinator disks.
The coordinator disks can be raw devices, DMP devices, or iSCSI devices. You must use DMP disk policy for iSCSI-based coordinator disks.
Page 91
About setting up disk-based I/O fencing
For the latest information on supported hardware visit the following URL:
http://entsupport.symantec.com/docs/283161
Each of the coordinator disks must use a physically separate disk or LUN.
Symantec recommendsusing thesmallest possible LUNs for coordinator disks.
Each ofthe coordinatordisks should exist on a different disk array, if possible.
The coordinator disks must support SCSI-3 persistent reservations.
Symantec recommendsusing hardware-basedmirroring for coordinator disks.
Coordinator disks must not be used to store data or must not be included in
disk groups that store user data.
Coordinator disks cannot be the special devices that array vendors use. For
example, you cannot use EMC gatekeeper devices as coordinator disks.
The I/O fencing configuration files include:
91Configuring VCS clusters for data integrity
/etc/vxfendg
/etc/vxfenmode
You must create this file to include the coordinator disk group information.
You must set the I/O fencing mode to SCSI-3.
You can configure the vxfen module to use either DMP devices or the underlying raw character devices. Note that you must use the same SCSI-3 disk policy on all the nodes. The SCSI-3 disk policy can either be raw or dmp. The policy is raw by default. If you use iSCSI devices, you must set the disk policy as dmp.
Page 92
Configuring VCS clusters for data integrity
92

Preparing to configure disk-based I/O fencing

/etc/vxfentab
When you run the vxfen startup file to start I/O fencing, the script creates this /etc/vxfentab file on each node with a list of all paths to eachcoordinator disk.The startupscript uses the contents of the /etc/vxfendg and /etc/vxfenmode files.
Thus any time a system is rebooted, the fencing driver reinitializes the vxfentab file with the current list of all paths to the coordinator disks.
Note: The /etc/vxfentab file is a generated file; do not
modify this file.
An example of the /etc/vxfentab file on one noderesembles as follows:
Raw disk:
/dev/sdx
/dev/sdy
/dev/sdz
DMP disk:
/dev/vx/rdmp/sdx
/dev/vx/rdmp/sdy
/dev/vx/rdmp/sdz
In some cases you must remove disksfrom or add disks toan existing coordinator disk group.
Warning: If you remove disks from an existing coordinator disk group, then be sure to remove the registration and reservation keys from these disks before you add the disks to another disk group.
Preparing to configure disk-based I/O fencing
Make sure you performed the following tasks before configuring I/O fencing for VCS:
Install the correct operating system.
Install the VRTSvxfen RPM when you installed VCS.
Install a version of Veritas Volume Manager (VxVM) that supports SCSI-3
persistent reservations (SCSI-3 PR).
Page 93
Preparing to configure disk-based I/O fencing
Refer tothe installation guide that comes with theStorage Foundation product that you use.
Perform the following preparatory tasks to configure I/O fencing:
93Configuring VCS clusters for data integrity
Initialize disks as VxVM disks
Identify disks to use as coordinator disks
Check shared disks for I/O fencing
The tasks involved in checking the shared disks for I/O fencing are as follows:
Verify that the nodes have access to the
same disk
Test the disks using the vxfentsthdw
utility

Initializing disks as VxVM disks

Perform the following procedure to initialize disks as VxVM disks.
To initialize disks as VxVM disks
Make the new disks recognizable. On each node, enter:
1
# fdisk -l
If the Array Support Library (ASL) for the array that you add is not installed,
2
obtain and install it on each node before proceeding.
The ASL for the supported storage device that you add is available from the disk array vendor or Symantec technical support.
See “Initializing disks as VxVM disks” on page 93.
See “Identifying disks to use as coordinator
diskson page 95.
See “Checking shared disks for I/O fencing” on page 95.
Page 94
Configuring VCS clusters for data integrity
94
Preparing to configure disk-based I/O fencing
Verify that the ASL for the disk array is installed on each of the nodes. Run
3
the following command on each node and examine the output to verify the installation of ASL.
The following output is a sample:
# vxddladm listsupport all
LIBNAME VID PID
=================================================================================
libvx3par.so 3PARdata VV
libvxCLARiiON.so DGC All
libvxcscovrts.so CSCOVRTS MDS9
libvxemc.so EMC SYMMETRIX
libvxhds.so HITACHI All
libvxhds9980.so HITACHI All
libvxhdsalua.so HITACHI DF600, DF600-V, DF600F, DF600F-V
libvxhdsusp.so HITACHI All
libvxhitachi.so HITACHI DF350, DF400, DF400F, DF500, DF500F
libvxhpalua.so HP, COMPAQ HSV101, HSV111 (C)COMPAQ, HSV111, HSV200,
libvxhpmsa.so HP MSA VOLUME
libvxibmds4k.so IBM 1722, 1724, 3552, 3542, 1742-900, 1742,
libvxibmds6k.so IBM 1750
libvxibmds8k.so IBM 2107
libvxpp.so EMC, DGC All
libvxpurple.so SUN T300
libvxshark.so IBM 2105
libvxveritas.so VERITAS All
libvxxiv.so XIV, IBM NEXTRA, 2810XIV
libvxxp1281024.so HP All
libvxxp12k.so HP All
libvxxp256.so HP All
HSV210
3526, 1815, 1814
Page 95
Preparing to configure disk-based I/O fencing
Scan all disk drives and their attributes, update the VxVM device list, and
4
reconfigure DMP with the new devices. Type:
# vxdisk scandisks
See the Veritas Volume Manager documentation for details on how to add and configure disks.
To initialize the disks as VxVM disks, use one of the following methods:
5
Use theinteractive vxdiskadm utility to initializethe disks as VxVM disks.
For more information see the Veritas Volume Managers Administrators Guide.
Use the vxdisksetup command to initialize a disk as a VxVM disk.
vxdisksetup -i device_name
The example specifies the CDS format:
# vxdisksetup -i sdr
Repeat this command for each disk you intend to use as a coordinator disk.
95Configuring VCS clusters for data integrity

Identifying disks to use as coordinator disks

After you add and initialize disks, identify disks to use as coordinator disks.
To identify the coordinator disks
List the disks on each node.
1
For example, execute the following commands to list the disks:
# vxdisk list
Pick three SCSI-3 PR compliant shared disks as coordinator disks.
2

Checking shared disks for I/O fencing

Make sure that the shared storage you set up while preparing to configure VCS meets the I/O fencing requirements. You can test the shared disks using the vxfentsthdw utility.The two nodes must have ssh (default) or rsh communication. To confirm whether a disk (or LUN) supports SCSI-3 persistent reservations, two nodes must simultaneously have access to the same disks. Because a shared disk is likely to have a different name on each node, check the serial number to verify the identity of the disk. Use the vxfenadm command with the -i option. This
Page 96
Configuring VCS clusters for data integrity
96
Preparing to configure disk-based I/O fencing
command option verifies that the same serial number for the LUN is returned on all paths to the LUN.
Make sure to test the disks that serve as coordinator disks.
The vxfentsthdw utility has additional options suitable for testing many disks. Review the options for testing the disk groups (-g) and the disks that are listed in a file (-f). You can also test disks without destroying data using the -r option.
See Veritas Cluster Server User's Guide.
Checking that disks support SCSI-3 involves the following tasks:
Verifying that nodes have access to the same disk
See “Verifying that the nodes have access to the same disk” on page 96.
Testing the shared disks for SCSI-3
See “Testing the disks using vxfentsthdw utility” on page 97.
Verifying that the nodes have access to the same disk
Before you test the disks that you plan to use as shared data storage or as coordinator disks using the vxfentsthdw utility, you must verify that the systems see the same disk.
To verify that the nodes have access to the same disk
Verify the connection of the shared storage for data to two of the nodes on
1
which you installed VCS.
Ensure that both nodes are connected to the same disk during the testing.
2
Use the vxfenadm command to verify the disk serial number.
/sbin/vxfenadm -i diskpath
Refer to the vxfenadm (1M) manual page.
For example, an EMC disk is accessible by the /dev/sdx path on node A and the /dev/sdy path on node B.
From node A, enter:
# /sbin/vxfenadm -i /dev/sdx
SCSI ID=>Host: 2 Channel: 0 Id: 0 Lun: E
Vendor id : EMC
Product id : SYMMETRIX
Revision : 5567
Serial Number : 42031000a
Page 97
Preparing to configure disk-based I/O fencing
The same serial number information should appear when you enter the equivalent command on node B using the /dev/sdy path.
On a disk from another manufacturer, Hitachi Data Systems, the output is different and may resemble:
# /sbin/vxfenadm -i /dev/sdz
SCSI ID=>Host: 2 Channel: 0 Id: 0 Lun: E
Vendor id : HITACHI
Product id : OPEN-3
Revision : 0117
Serial Number : 0401EB6F0002
Testing the disks using vxfentsthdw utility
This procedure uses the /dev/sdx disk in the steps.
If the utility does not show a message that states a disk is ready, the verification has failed. Failure of verification can be the result of an improperly configured disk array. The failure can also be due to a bad disk.
If the failure is due to a bad disk, remove and replace it. The vxfentsthdw utility indicates a disk can be used for I/O fencing with a message resembling:
97Configuring VCS clusters for data integrity
The disk /dev/sdx is ready to be configured for I/O Fencing on
node galaxy
For more information on how to replace coordinator disks, refer to the Veritas Cluster Server User's Guide.
To test the disks using vxfentsthdw utility
Make sure system-to-system communication functions properly.
1
See “Setting up inter-system communication” on page 44.
After you complete the testing process, remove permissions for communication and restore public network connections.
See “Removing permissions for communication” on page 103.
From one node, start the utility.
2
Do one of the following:
If you use ssh for communication:
# /opt/VRTSvcs/vxfen/bin/vxfentsthdw
Page 98
Configuring VCS clusters for data integrity
98
Preparing to configure disk-based I/O fencing
If you use rsh for communication:
The script warns that the tests overwrite data on the disks. After you review
3
the overview and the warning, confirm to continue the process and enter the node names.
Warning: The tests overwrite and destroy data on the disks unless you use the -r option.
******** WARNING!!!!!!!! ********
THIS UTILITY WILL DESTROY THE DATA ON THE DISK!!
Do you still want to continue : [y/n] (default: n) y
Enter the first node of the cluster: galaxy
Enter the second node of the cluster: nebula
Enter the names of the disks that you want to check. Each node may know
4
the same disk by a different name:
# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n
Enter the disk name to be checked for SCSI-3 PGR on node
galaxy in the format: /dev/sdx
/dev/sdr
Enter the disk name to be checked for SCSI-3 PGR on node
nebula in the format: /dev/sdx
Make sure it’s the same disk as seen by nodes galaxy and nebula
/dev/sdr
If the serial numbers of the disks are not identical. then the test terminates.
Review the output as the utility performs the checks and report its activities.
5
If a disk is ready for I/O fencing on each node, the utility reports success:
6
The disk is now ready to be configured for I/O Fencing on node
galaxy
ALL tests on the disk /dev/sdx have PASSED
The disk is now ready to be configured for I/O Fencing on node
galaxy
Run the vxfentsthdw utility for each disk you intend to verify.
7
Page 99

Setting up disk-based I/O fencing manually

Setting up disk-based I/O fencing manually
Make sure you completed the preparatory tasks before you set up I/O fencing.
Tasks that are involved in setting up I/O fencing include:
99Configuring VCS clusters for data integrity
Table 5-1
groups
configuration files
Modifying VCS configuration to use I/O fencing
configuration
Tasks to set up I/O fencing manually

Setting up coordinator disk groups

From onenode, create a disk group named vxfencoorddg. This groupmust contain three disksor LUNs. If you useVxVM 5.0or later, you must also set thecoordinator attribute for the coordinator disk group. VxVM uses this attribute to prevent the reassignment of coordinator disks to other disk groups.
Note that if you create a coordinator disk group as a regular disk group, you can turn on the coordinator attribute in Volume Manager.
Refer to the Veritas Volume Manager Administrators Guide for details on how to create disk groups.
The following example procedure assumes that the disks have the device names sdx, sdy, and sdz.
To create the vxfencoorddg disk group
DescriptionAction
See “Setting up coordinator disk groups” on page 99.Setting up coordinator disk
See “CreatingI/O fencingconfiguration files”on page100.Creating I/O fencing
See “Modifying VCS configuration to use I/O fencing” on page 101.
See “Verifying I/O fencing configuration” on page 102.Verifying I/O fencing
On any node, create the disk group by specifying the device names:
1
# vxdg init vxfencoorddg sdx sdy sdz
If you use VxVM 5.0 or later, set the coordinator attribute value as "on" for
2
the coordinator disk group.
# vxdg -g vxfencoorddg set coordinator=on
Page 100
Configuring VCS clusters for data integrity
100
Setting up disk-based I/O fencing manually
Deport the coordinator disk group:
3
# vxdg deport vxfencoorddg
Import the disk group with the -t option to avoid automatically importing it
4
when the nodes restart:
# vxdg -t import vxfencoorddg
Deport the disk group. Deporting the disk group prevents the coordinator
5
disks from serving other purposes:
# vxdg deport vxfencoorddg

Creating I/O fencing configuration files

After youset up the coordinator disk group, youmust do the following to configure I/O fencing:
Create the I/O fencing configuration file /etc/vxfendg
Update the I/O fencing configuration file /etc/vxfenmode
To update the I/O fencing files and start I/O fencing
On each nodes, type:
1
# echo "vxfencoorddg" > /etc/vxfendg
Do not use spaces between the quotes in the "vxfencoorddg" text.
This command creates the /etc/vxfendg file, which includes the name of the coordinator disk group.
On all cluster nodes depending on the SCSI-3 mechanism, type one of the
2
following selections:
For DMP configuration:
# cp /etc/vxfen.d/vxfenmode_scsi3_dmp /etc/vxfenmode
For raw device configuration:
Loading...