Redhat Oracle9i RAC with GFS User Manual

Red Hat GFS

Installing and Conﬁguring

Oracle9i RAC with GFS 6.0

Red Hat GFS: Installing and Conﬁguring Oracle9i RAC with GFS 6.0

Red Hat, Inc.

1801 Varsity Drive Raleigh NC 27606-2072 USA Phone: +1 919 754 3700 Phone: 888 733 4281 Fax: +1 919 754 3701 PO Box 13588 Research Triangle Park NC 27709 USA

rh-gfsog(EN)-6.0-Print-RHI (2005-10-13T14:56) Copyright © 2005 by Red Hat, Inc. This material may be distributedonly subject to the terms and conditions set forth in the Open Publication License, V1.0 or later (the latest version is presentlyavailableat http://www.opencontent.org/openpub/). Distribution of substantively modiﬁed versions of this document is prohibitedwithout the explicit permission of the copyright holder. Distribution of the work or derivative of the work in any standard (paper)book form for commercial purposes is prohibited unless prior permission is obtained from the copyright holder. Red Hat and the Red Hat "Shadow Man" logo are registered trademarks of Red Hat, Inc. in the United States and other

countries. All other trademarks referencedherein are the property of their respective owners. The GPG ﬁngerprint of the security@redhat.comkey is: CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E

Introduction.......................................................................................................................................... i

1. Audience ................................................................................................................................ i

2. Related Documentation .......................................................................................................... i

3. Document Conventions .......................................................................................................... i

4. More to Come ...................................................................................................................... iv

4.1. Send in Your Feedback ......................................................................................... iv

5. Activate Your Subscription .................................................................................................. iv

1. Overview ...........................................................................................................................................1

1.1. Sample Cluster ....................................................................................................................1

1.2. Requirements ......................................................................................................................5

1.3. Prerequisites ........................................................................................................................5

2. Installing and Conﬁguring GFS .....................................................................................................7

2.1. Installation and Conﬁguration Procedure ...........................................................................7

2.2. Installing GFS .....................................................................................................................7

2.3. Conﬁguring Pools ...............................................................................................................8

2.4. Setting Up the GFS Conﬁguration ....................................................................................10

2.5. Starting the Lock Servers ..................................................................................................15

2.6. Creating and Mounting GFS File Systems .......................................................................15

3. Installing and Conﬁguring Oracle9i RAC ...................................................................................17

3.1. Installation and Conﬁguration Procedure .........................................................................17

3.2. Setting Up a Compatibility Environment..........................................................................17

3.2.1. Installing Libraries .............................................................................................17

3.2.2. Setting Up GCC .................................................................................................17

3.3. Preparing Oracle Nodes ....................................................................................................18

3.4. Installing and Conﬁguring Oracle Components ...............................................................21

3.4.1. Installing Oracle Cluster Manager .....................................................................22

3.4.2. Patching the Cluster Manager ............................................................................23

3.4.3. Modifying the $ORACLE_HOME/oracm/admin/cmcfg.ora File ...................23

3.4.4. Enabling oracm CDPN......................................................................................24

3.4.5. Conﬁguring Cluster Manager ............................................................................24

3.4.6. Starting Oracle Cluster Manager .......................................................................25

3.4.7. Installing Oracle RDBMS ..................................................................................25

3.4.8. Patching the RDBMS Server .............................................................................26

3.5. Creating an Oracle Database.............................................................................................26

3.5.1. Creating Network Conﬁguration Files...............................................................26

3.5.2. Enabling network CDPN .................................................................................28

3.5.3. Setting Up the Cluster Conﬁguration File .........................................................29

3.5.4. Setting Up the Database .....................................................................................29

Index....................................................................................................................................................33

Colophon............................................................................................................................................. 35

Introduction

Welcome to Red Hat GFS: Installing and Conﬁguring Oracle9i RAC with GFS 6.0.

This manual provides information about installing and conﬁguring Oracle9i Real Application Clusters (RAC) for use with Red Hat GFS 6.0. The document contains procedures for a sample cluster that can be used as a reference for your cluster.

HTML and PDF versions of all the Red Hat Enterprise Linux manuals and Release Notes are available online at http://www.redhat.com/docs/.

1. Audience

This manual is intended primarily for Oracle database administrators and Linux system administrators who are familiar with the following activities:

• Oracle database administration procedures

• Linux system administration procedures, including kernel conﬁguration

• Installing and conﬁguring shared-storage networks, such as Fibre Channel SANs

2. Related Documentation

For additional information, refer to the following documentation:

• Red Hat GFS Administrator’s Guide for Red Hat GFS 6.0

• Red Hat GFS product release notes for Red Hat GFS 6.0

• Relevant Oracle9i RAC documentation

3. Document Conventions

In this manual, certain words are represented in different fonts, typefaces, sizes, and weights. This highlighting is systematic; different words are represented in the same style to indicate their inclusion in a speciﬁc category. The types of words that are represented this way include the following:

command

Linux commands (and other operating system commands, when used) are represented this way. This style should indicate to you that you can type the word or phrase on the command line and press [Enter] to invoke a command. Sometimes a command contains words that would be displayed in a different style on their own (such as ﬁle names). In these cases, they are considered to be part of the command, so the entire phrase is displayed as a command. For example:

Use the cat testfile command to view the contents of a ﬁle, named testfile, in the current working directory.

file name

File names, directory names, paths, and RPM package names are represented this way. This style indicates that a particular ﬁle or directory exists with that name on your system. Examples:

The .bashrc ﬁle in your home directory contains bash shell deﬁnitions and aliases for your own use.

ii Introduction

The /etc/fstab ﬁle contains information about different system devices and ﬁle systems.

Install the webalizer RPM if you want to use a Web server log ﬁle analysis program.

application

This style indicates that the program is an end-user application (as opposed to system software). For example:

Use Mozilla to browse the Web.

[key]

A key on the keyboard is shown in this style. For example:

To use [Tab] completion, type in a character and then press the [Tab] key. Your terminal displays the list of ﬁles in the directory that start with that letter.

[key]-[combination]

A combination of keystrokes is represented in this way. For example:

The [Ctrl]-[Alt]-[Backspace] key combination exits your graphical session and returns you to the graphical login screen or the console.

text found on a GUI interface

A title, word, or phrase found on a GUI interface screen or window is shown in this style. Text shown in this style indicates that a particular GUI screen or an element on a GUI screen (such as text associated with a checkbox or ﬁeld). Example:

Select the Require Password checkbox if you would like your screensaver to require a password before stopping.

top level of a menu on a GUI screen or window

A word in this style indicates that the word is the top level of a pulldown menu. If you click on the word on the GUI screen, the rest of the menu should appear. For example:

Under File on a GNOME terminal, the New Tab option allows you to open multiple shell prompts in the same window.

Instructions to type in a sequence of commands from a GUI menu look like the following example:

Go to Applications (the main menu on the panel) => Programming => Emacs Text Editor to start the Emacs text editor.

button on a GUI screen or window

This style indicates that the text can be found on a clickable button on a GUI screen. For example:

Click on the Back button to return to the webpage you last viewed.

computer output

Text in this style indicates text displayed to a shell prompt such as error messages and responses to commands. For example:

The ls command displays the contents of a directory. For example:

Desktop about.html logs paulwesterberg.png Mail backupfiles mail reports

The output returned in response to the command (in this case, the contents of the directory) is shown in this style.

Introduction iii

prompt

A prompt, which is a computer’s way of signifying that it is ready for you to input something, is shown in this style. Examples:

[stephen@maturin stephen]$

leopard login:

user input

Text that the user types, either on the command line or into a text box on a GUI screen, is displayed in this style. In the following example, text is displayed in this style:

To boot your system into the text based installation program, you must type in the text command at the boot: prompt.

Text used in examples that is meant to be replaced with data provided by the user is displayed in this style. In the following example, <version-number> is displayed in this style:

The directory for the kernel source is /usr/src/kernels/<version-number>/, where <version-number> is the version and type of kernel installed on this system.

Additionally, we use several different strategies to draw your attention to certain pieces of information. In order of urgency, these items are marked as a note, tip, important, caution, or warning. For example:

Note

Remember that Linux is case sensitive. In other words, a rose is not a ROSE is not a rOsE.

Tip

The directory /usr/share/doc/ contains additional documentation for packages installed on your system.

Important

If you modify the DHCP conﬁguration ﬁle, the changes do not take effect until you restart the DHCP daemon.

iv Introduction

Caution

Do not perform routine tasks as root — use a regular user account unless you need to use the root account for system administration tasks.

Warning

Be careful to remove only the necessary par titions. Removing other partitions could result in data loss or a corrupted system environment.

4. More to Come

Red Hat GFS: Installing and Conﬁguring Oracle9i RAC with GFS 6.0 is part of Red Hat’s growing commitment to provide useful and timely support to users of Red Hat products. As new releases are made available, this guide will continue to expand.

4.1. Send in Your Feedback

If you spot a typo in Red Hat GFS: Installing and Conﬁguring Oracle9i RAC with GFS 6.0, or if you have thought of a way to make this manual better, we would love to hear from you! Please submit a report in Bugzilla (http://www.redhat.com/bugzilla) against the component rh-gfsog.

Be sure to mention the manual’s identiﬁer:

rh-gfsog(EN)-6.0-Print-RHI (2005-10-13T14:56)

If you mention this manual’s identiﬁer, we will know exactly which version of the guide you have.

If you have a suggestion for improving the documentation, try to be as speciﬁc as possible. If you have found an error, please include the section number and some of the surrounding text so we can ﬁnd it easily.

5. Activate Your Subscription

Before you can access service and software maintenance information, and the support documentation included in your subscription, you must activate your subscription by registering with Red Hat. Registration includes these simple steps:

• Provide a Red Hat login

• Provide a subscription number

• Connect your system

You can complete the Red Hat registration process online at:

http://www.redhat.com/register/

A Red Hat login enables your access to:

• Software updates, errata and maintenance via Red Hat Network

Introduction v

• Red Hat technical support resources, documentation, and Knowledgebase

If you have forgotten your Red Hat login, you can search for your Red Hat login online at:

https://rhn.redhat.com/help/forgot_password.pxt

Good luck, and thank you for choosing Red Hat GFS!

The Red Hat Documentation Team

vi Introduction

Chapter 1. Overview

This manual provides information about installing and conﬁguring Oracle9i Real Application Clusters (RAC) Release 2 on a Linux cluster that uses Red Hat GFS 6.0. It contains procedures for a sample cluster that can be used as a reference for your cluster. You should use this manual in conjunction with the appropriate guides for installing and maintaining Oracle9i RAC on a Linux cluster, and the Red Hat GFS Administrator’s Guide for Red Hat GFS 6.0.

This chapter provides an introduction to the sample cluster used in the manual, requirements based on the sample cluster, and prerequisites for installing and conﬁguring Oracle9i RAC on a Linux cluster with Red Hat GFS 6.0. The chapter consists of the following sections:

• Section 1.1 Sample Cluster

• Section 1.2 Requirements

• Section 1.3 Prerequisites

1.1. Sample Cluster

The sample cluster used in this manual consists of the following components (refer to Figure 1-1 and Figure 1-2):

• A Red Hat GFS 6.0 cluster with four GFS nodes and three external lock servers

• Three WTI network power switches (NPS)

• A QLogic switch

• An HP StorageWorks 1000 Modular Smart Array (MSA1000)

• A private network (via a NETGEAR switch)

2 Chapter 1. Overview

Private network,

Netgear switch

QLogic switch

Shared Files

Red Hat GFS

Oracle9i RAC

HP StorageWorks MSA1000:

- sda

- sdb

- sdc

- sdd

WTI NPS wti1

Four-node Red Hat GFS cluster with Oracle9i RAC

rac1/ racpvt1

rac4/ racpvt4

rac2/ racpvt2

rac3/ racpvt3

External lock servers: lock1, lock2, and lock3

WTI NPS: wti2, wti3

Figure 1-1. Sample Cluster: Red Hat GFS Nodes

Chapter 1. Overview 3

Private network,

Netgear switch

WTI NPS wti1

Four-node Red Hat GFS cluster with Oracle9i RAC

WTI NPS wti2

WTI NPS wti3

Lock server lock1

Lock server lock3

Lock server lock2

Figure 1-2. Sample Cluster: External Lock Servers

The GFS nodes are attached to the MSA1000 array via the QLogic switch. Also, the GFS nodes and the external lock-server nodes are connected to the WTI network power switches (NPS) for fencing. One power switch is connected to the GFS nodes, another power switch is connected to lock servers lock1 and lock2, and another power switch is connected to lock server lock3. The MSA1000 array is conﬁgured as follows:

• Four 120-gigabyte LUNs (RAID 0)

• Seven disks, each at 15,000 RPM

• Twenty-eight spindles, total (over 2 shelves)

Table 1-1 lists the GFS node names and IP addresses for the public network and the private network. Table 1-2 lists the lock-server node names and IP addresses. Table 1-3 lists the network power switch node names, IP addresses, and nodes controlled. Table 1-4 lists the GFS pools required, showing pool name, size, function, disk, and location.

Public Network: Node Name

rac1 10.10.10.101 racpvt1 192.168.100.101

rac2 10.10.10.102 racpvt2 192.168.100.102

rac3 10.10.10.103 racpvt3 192.168.100.103

Public Network: IP Address

Private Network: Node Name

Private Network: IP Address

rac4 10.10.10.104 racpvt4 192.168.100.104

Table 1-1. Sample Cluster: GFS Node Names and IP Addresses

4 Chapter 1. Overview

Note

The procedures in this manual are based on the assumption that you are using a private network for your Oracle cluster software, and that the Oracle management node is rac1/racpvt1.

Node Name IP Address

lock1 192.168.100.201

lock2 192.168.100.202

lock3 192.168.100.203

Table 1-2. Sample Cluster: Lock-Server Node Names and IP Addresses

Node Name IP Address Node Controlled

wti1 192.168.100.211 rac1/racpvt1

rac2/racpvt2 rac3/racpvt3

rac4/racpvt4

wti2 192.168.100.212 lock1

lock2

wti3 192.168.100.213 lock3

Table 1-3. Sample Cluster: Network Power Switch Node Names, IP Addresses, and Nodes Controlled

Pool Name Size Function Disk Location

oraraw1.pool 100 MB Oracle quorum disk sda1 /dev/pool/raw1

oraraw2.pool 100 MB Oracle cluster

sda2 /dev/pool/raw2

conﬁguration

ccs_archive.pool 10 MB GFS cluster archive sda3

oracle_base.pool 20 GB Oracle binaries, logs, and

sda4 /dev/pool/oracle_base

related ﬁles

oraundo.pool 98 GB Oracle undo tablespace sdb1 /dev/pool/oraundo

oraindex.pool 98 GB Oracle index ﬁles sdc1 /dev/pool/oraindex

oradata.pool 60 GB Oracle data ﬁles sdd1 /dev/pool/oradata

Table 1-4. Sample Cluster: Storage Conﬁguration

+ 32 hidden pages