Red Hat and the Red Hat "Shadow Man" logo are registered trademarks of Red Hat, Inc. in the United States
and other countries.
All other trademarks referencedherein are the property of their respective owners.
The GPG fingerprint of the security@redhat.comkey is:
CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E
Table of Contents
Introduction........................................................................................................................ i
1. Audience .............................................................................................................. i
2. Document Conventions........................................................................................ i
3. More to Come .................................................................................................... iv
3.1. Send in Your Feedback ....................................................................... iv
4. Activate Your Subscription ..................................................................................v
4.1. Provide a Red Hat Login.......................................................................v
4.2. Provide Your Subscription Number .................................................... vi
4.3. Connect Your System.......................................................................... vi
5. Recommended References................................................................................. vi
Welcome to the Red Hat GFS Administrator’s Guide. This book provides information
about installing, configuring, and maintaining Red Hat GFS (Red Hat Global File System). Red Hat GFS depends on the cluster infrastructure of Red Hat Cluster Suite. For
information about Red Hat Cluster Suite refer to Red Hat Cluster Suite Configuring andManaging a Cluster.
HTML and PDF versions of all the official Red Hat Enterprise Linux manuals and release
notes are available online at http://www.redhat.com/docs/.
1. Audience
This book is intended primarily for Linux system administrators who are familiar with the
following activities:
• Linux system administration procedures, including kernel configuration
• Installation and configuration of shared storage networks, such as Fibre Channel SANs
2. Document Conventions
In this manual, certain words are represented in different fonts, typefaces, sizes, and
weights. This highlighting is systematic; different words are represented in the same style
to indicate their inclusion in a specific category. The types of words that are represented
this way include the following:
command
Linux commands (and other operating system commands, when used) are represented
this way. This style should indicate to you that you can type the word or phrase on
the command line and press [Enter] to invoke a command. Sometimes a command
contains words that would be displayed in a different style on their own (such as file
names). In these cases, they are considered to be part of the command, so the entire
phrase is displayed as a command. For example:
Use the cat testfile command to view the contents of a file, named testfile,
in the current working directory.
file name
File names, directory names, paths, and RPM package names are represented this
way. This style indicates that a particular file or directory exists with that name on
your system. Examples:
iiIntroduction
The .bashrc file in your home directory contains bash shell definitions and aliases
for your own use.
The /etc/fstab file contains information about different system devices and file
systems.
Install the webalizer RPM if you want to use a Web server log file analysis program.
application
This style indicates that the program is an end-user application (as opposed to system
software). For example:
Use Mozilla to browse the Web.
[key]
A key on the keyboard is shown in this style. For example:
To use [Tab] completion, type in a character and then press the [Tab] key. Your terminal displays the list of files in the directory that start with that letter.
[key]-[combination]
A combination of keystrokes is represented in this way. For example:
The [Ctrl]-[Alt]-[Backspace] key combination exits your graphical session and returns
you to the graphical login screen or the console.
text found on a GUI interface
A title, word, or phrase found on a GUI interface screen or window is shown in this
style. Text shown in this style indicates that a particular GUI screen or an element on
a GUI screen (such as text associated with a checkbox or field). Example:
Select the Require Password checkbox if you would like your screensaver to require
a password before stopping.
top level of a menu on a GUI screen or window
A word in this style indicates that the word is the top level of a pulldown menu. If you
click on the word on the GUI screen, the rest of the menu should appear. For example:
Under File on a GNOME terminal, the New Tab option allows you to open multiple
shell prompts in the same window.
Instructions to type in a sequence of commands from a GUI menu look like the following example:
Go to Applications (the main menu on the panel) => Programming => Emacs Text
Editor to start the Emacs text editor.
Introductioniii
button on a GUI screen or window
This style indicates that the text can be found on a clickable button on a GUI screen.
For example:
Click on the Back button to return to the webpage you last viewed.
computer output
Text in this style indicates text displayed to a shell prompt such as error messages and
responses to commands. For example:
The ls command displays the contents of a directory. For example:
The output returned in response to the command (in this case, the contents of the
directory) is shown in this style.
prompt
A prompt, which is a computer’s way of signifying that it is ready for you to input
something, is shown in this style. Examples:
$
#
[stephen@maturin stephen]$
leopard login:
user input
Text that the user types, either on the command line or into a text box on a GUI screen,
is displayed in this style. In the following example, text is displayed in this style:
To boot your system into the text based installation program, you must type in the
text command at the boot: prompt.
replaceable
Text used in examples that is meant to be replaced with data provided by the user
is displayed in this style. In the following example,
version-numberis dis-
played in this style:
The directory for the kernel source is /usr/src/kernels/
version-numberis the version and type of kernel installed on this
where
version-number/,
system.
Additionally, we use several different strategies to draw your attention to certain pieces of
information. In order of urgency, these items are marked as a note, tip, important, caution,
or warning. For example:
ivIntroduction
Note
Remember that Linux is case sensitive. In other words, a rose is not a ROSE is not a
rOsE.
Tip
The directory /usr/share/doc/ contains additional documentation for packages installed
on your system.
Important
If you modify the DHCP configuration file, the changes do not take effect until you restart
the DHCP daemon.
Caution
Do not perform routine tasks as root — use a regular user account unless you need to
use the root account for system administration tasks.
Warning
Be careful to remove only the necessary partitions. Removing other partitions could result
in data loss or a corrupted system environment.
3. More to Come
The Red Hat GFS Administrator’s Guide is part of Red Hat’s growing commitment to
provide useful and timely support to Red Hat Enterprise Linux users.
Introductionv
3.1. Send in Your Feedback
If you spot a typo in the Red Hat GFS Administrator’s Guide, or if you have thought of a
way to make this manual better, we would love to hear from you! Please submit a report in
Bugzilla (http://www.redhat.com/bugzilla) against the component rh-gfsg.
Be sure to mention the manual’s identifier:
rh-gfsg(EN)-6.1-Print-RHI (2007-01-05T17:28)
If you mention this manual’s identifier, we will know exactly which version of the guide
you have.
If you have a suggestion for improving the documentation, try to be as specific as possible.
If you have found an error, please include the section number and some of the surrounding
text so we can find it easily.
4. Activate Your Subscription
Before you can access service and software maintenance information, and the support documentation included in your subscription, you must activate your subscription by registering with Red Hat. Registration includes these simple steps:
• Provide a Red Hat login
• Provide a subscription number
• Connect your system
The first time you boot your installation of Red Hat Enterprise Linux, you are prompted to
register with Red Hat using the Setup Agent. If you follow the prompts during the SetupAgent, you can complete the registration steps and activate your subscription.
If you can not complete registration during the Setup Agent (which requires network
access), you can alternatively complete the Red Hat registration process online at
http://www.redhat.com/register/.
4.1. Provide a Red Hat Login
If you do not have an existing Red Hat login, you can create one when prompted during
the Setup Agent or online at:
• Software updates, errata and maintenance via Red Hat Network
viIntroduction
• Red Hat technical support resources, documentation, and Knowledgebase
If you have forgotten your Red Hat login, you can search for your Red Hat login online at:
https://rhn.redhat.com/help/forgot_password.pxt
4.2. Provide Your Subscription Number
Your subscription number is located in the package that came with your order. If your
package did not include a subscription number, your subscription was activated for you
and you can skip this step.
You can provide your subscription number when prompted during the Setup Agent or by
visiting http://www.redhat.com/register/.
4.3. Connect Your System
The Red Hat Network Registration Client helps you connect your system so that you can
begin to get updates and perform systems management. There are three ways to connect:
1. During the Setup Agent — Check the Send hardware information and Send sys-tem package list options when prompted.
2. After the Setup Agent has been completed — From Applications (the main menu
on the panel), go to System Tools, then select Red Hat Network.
3. After the Setup Agent has been completed — Enter the following command from
the command line as the root user:
• /usr/bin/up2date --register
5. Recommended References
For additional references about related topics, refer to the following table:
TopicReferenceComment
Shared Data Clustering and
File Systems
Shared Data Clusters by
Dilip M. Ranade. Wiley,
2002.
Provides detailed technical
information on cluster file
system and cluster
volume-manager design.
Introductionvii
TopicReferenceComment
Storage Area Networks
(SANs)
Applications and High
Availability
Table 1. Recommended References Table
Designing Storage Area
Networks: A Practical
Reference for Implementing
Fibre Channel and IP
SANs, Second Edition by
Tom Clark.
Addison-Wesley, 2003.
Building SANs with
Brocade Fabric Switches by
C. Beauchamp, J. Judd, and
B. Keo. Syngress, 2001.
Building Storage Networks,
Second Edition by Marc
Farley.
Osborne/McGraw-Hill,
2001.
Blueprints for High
Availability: Designing
Resilient Distributed
Systems by E. Marcus and
H. Stern. Wiley, 2000.
Provides a concise
summary of Fibre Channel
and IP SAN Technology.
Best practices for building
Fibre Channel SANs based
on the Brocade family of
switches, including
core-edge topology for
large SAN fabrics.
Provides a comprehensive
overview reference on
storage networking
technologies.
Provides a summary of
best practices in high
availability.
viiiIntroduction
Chapter 1.
GFS Overview
Red Hat GFS is a cluster file system that is available with Red Hat Cluster Suite. Red
Hat GFS nodes are configured and managed with Red Hat Cluster Suite configuration and
management tools. Red Hat GFS provides data sharing among GFS nodes in a Red Hat
cluster. GFS provides a single, consistent view of the file-system name space across the
GFS nodes in a Red Hat cluster. GFS allows applications to install and run without much
knowledge of the underlying storage infrastructure. GFS is fully compliant with the IEEE
POSIX interface, allowing applications to perform file operations as if they were running
on a local file system. Also, GFS provides features that are typically required in enterprise
environments, such as quotas, multiple journals, and multipath support.
GFS provides a versatile method of networking your storage according to the performance,
scalability, and economic needs of your storage environment. This chapter provides some
very basic, abbreviated information as background to help you understand GFS. It contains
the following sections:
• Section 1.1 New and Changed Features
• Section 1.2 Performance, Scalability, and Economy
• Section 1.3 GFS Functions
• Section 1.4 GFS Software Subsystems
• Section 1.5 Before Setting Up GFS
1.1. New and Changed Features
This section lists new and changed features included with the initial release of Red Hat
Red Hat GFS 6.1 and Red Hat GFS 6.1 for Red Hat Enterprise Linux 4 Update 2.
For information about upgrading from GFS 6.0 to GFS 6.1, refer to Appendix A UpgradingGFS.
Note
Multipath GNBD is not available with Red Hat GFS 6.1. That is, device mapper multipath
(dm-multipath) cannot use GNBD. GNBD without multipath is available.
New and Changed Features with the Initial Release of Red Hat GFS 6.1
2Chapter 1. GFS Overview
• Cluster infrastructure provided by Red Hat Cluster Suite — This release of Red Hat
GFS uses the cluster infrastructure of Red Hat Cluster Suite, taking advantage of the
Red Hat Cluster Suite configuration file and cluster graphical user interface (GUI),
system-config-cluster. For information about configuring and managing Red Hat
Cluster Suite, refer to Red Hat Cluster Suite Configuring and Managing a Cluster. Previous versions of Red Hat GFS provided a cluster infrastructure that was exclusive to
GFS (even though used with Red Hat Cluster Suite). Additionally, configuration files in
earlier versions of Red Hat GFS were created and maintained via text editors only (that
is, no GUI was available).
• Red Hat Cluster Suite lock architectures — Via Red Hat Cluster Suite, GFS can use the
following lock architectures:
• DLM (Distributed Lock Manager), new for Red Hat GFS 6.1 — DLM provides lock
management throughout a Red Hat cluster, requiring no nodes to be configured as
lock management nodes (contrasted to GULM, which does require certain nodes to
be configured as lock management nodes).
• GULM (Grand Unified Lock Manager) — A client/server lock architecture that is
compatible with Red Hat GFS 6.0.
• Nolock — For single node operation only.
• New volume manager, LVM2 — The pool volume manager in earlier releases of Red
Hat GFS is replaced with LVM2 for this release. LVM2 is used in conjunction with
CLVM (Cluster Logical Volume Manager). This release provides a tool to convert GFS
6.0 pool volumes to the LVM2 format. For information about converting pool volumes
to the LVM2 format, refer to Appendix A Upgrading GFS.
• Enhanced gfs_fsck performance and changes to the gfs_fsck command — The
gfs_fsck function performs 10 times as fast as gfs_fsck in earlier GFS releases.
(This enhancement has been included in a recent update to Red Hat GFS 6.0, also.) In
addition, the enhanced gfs_fsck function includes changes to certain command options. For more information about changes to the command options, refer to Section
5.12 Repairing a File System.
• Withdraw individual mount points — Allows individual GFS mount points to gracefully
discontinue operations on a node without causing that node to panic. This feature provides the ability to continue operations with unaffected file systems on that node. The
feature can be overridden to allow a node to panic, thereby providing more information for troubleshooting. For more information, refer to the mount command option,
oopses_ok, in Table 5-2
• Increased storage supported — Red Hat GFS supports 8 terabytes of storage per GFS
file system. For more information about Red Hat GFS requirements, refer to Chapter 2
System Requirements
Chapter 1. GFS Overview3
New and Changed Features with Red Hat GFS 6.1 for Red Hat Enterprise Linux 4
Update 2
Red Hat GFS 6.1 for Red Hat Enterprise Linux 4 Update 2 supports iSCSI and multipath
iSCSI. That is, device mapper multipath (dm-multipath) can use iSCSI.
1.2. Performance, Scalability, and Economy
You can deploy GFS in a variety of configurations to suit your needs for performance,
scalability, and economy. For superior performance and scalability, you can deploy GFS in
a cluster that is connected directly to a SAN. For more economical needs, you can deploy
GFS in a cluster that is connected to a LAN with servers that use GNBD (Global Network
Block Device). (For more information about GNBD, refer to Chapter 6 Using GNBD withRed Hat GFS.)
The following sections provide examples of how GFS can be deployed to suit your needs
for performance, scalability, and economy:
• Section 1.2.1 Superior Performance and Scalability
The deployment examples in this chapter reflect basic configurations; your needs might
require a combination of configurations shown in the examples.
1.2.1. Superior Performance and Scalability
You can obtain the highest shared-file performance when applications access storage directly. The GFS SAN configuration in Figure 1-1 provides superior file performance for
shared files and file systems. Linux applications run directly on GFS nodes. Without file
protocols or storage servers to slow data access, performance is similar to individual Linux
servers with directly connected storage; yet, each GFS application node has equal access
to all data files. GFS supports over 300 GFS nodes.
4Chapter 1. GFS Overview
SAN
Fabric
GFS
Applications
Shared Files
Figure 1-1. GFS with a SAN
1.2.2. Performance, Scalability, Moderate Price
Multiple Linux client applications on a LAN can share the same SAN-based data as shown
in Figure 1-2. SAN block storage is presented to network clients as block storage devices
by GNBD servers. From the perspective of a client application, storage is accessed as if
it were directly attached to the server in which the application is running. Stored data is
actually on the SAN. Storage devices and data can be equally shared by network client
applications. File locking and sharing functions are handled by GFS for each network
client.
Note
Clients implementing ext2 and ext3 file systems can be configured to access their own
dedicated slice of SAN storage.
Chapter 1. GFS Overview5
LAN
Clients
GNBD
servers
SAN
Fabric
GFS
Applications
Shared Files
Figure 1-2. GFS and GNBD with a SAN
1.2.3. Economy and Performance
Figure 1-3 shows how Linux client applications can take advantage of an existing Ethernet
topology to gain shared access to all block storage devices. Client data files and file systems
can be shared with GFS on each client. Application failover can be fully automated with
Red Hat Cluster Suite.
6Chapter 1. GFS Overview
LAN
Clients
GNBD
servers
Disk
A
GFS
Applications
Disk
B
Disk
D
Disk
C
Disk
E
Disk
F
Shared Files
Figure 1-3. GFS and GNBD with Directly Connected Storage
1.3. GFS Functions
GFS is a native file system that interfaces directly with the VFS layer of the Linux
kernel file-system interface. GFS is a cluster file system that employs distributed
metadata and multiple journals for optimal operation in a cluster. Cluster management
of GFS nodes is managed through Red Hat Cluster Suite. Volume management
is managed through CLVM (Cluster Logical Volume Manager). For information
about Red Hat Cluster Suite refer to Red Hat Cluster Suite Configuring andManaging a Cluster. For information about using CLVM, refer to the LVM HOWTO
(http://www.tldp.org/HOWTO/LVM-HOWTO/index.html).
Note
CLVM is a cluster-wide implementation of LVM, enabled by the CLVM daemon, clvmd
running in a Red Hat Cluster Suite cluster. The daemon makes it possible to use LVM2
to manage logical volumes across a cluster, allowing all nodes in the cluster to share the
logical volumes.
Chapter 1. GFS Overview7
GFS provides the following main functions:
• Making a File System
• Mounting a File System
• Unmounting a File System
• GFS Quota Management
• Growing a File System
• Adding Journals to a File System
• Direct I/O
• Data Journaling
• Configuring atime Updates
• Suspending Activity on a File System
• Displaying Extended GFS Information and Statistics
• Repairing a File System
• Context-Dependent Path Names (CDPN)
1.4. GFS Software Subsystems
GFS consists of the following subsystems: GFS and GNBD.
Table 1-1 summarizes the GFS Software subsystems and their components.
Software
ComponentsDescription
Subsystem
GFSgfs.koKernel module that implements the
GFS file system and is loaded on GFS
cluster nodes.
gfs_fsckCommand that repairs an unmounted
GFS file system.
gfs_growCommand that grows a mounted GFS
file system.
8Chapter 1. GFS Overview
Software
ComponentsDescription
Subsystem
gfs_jaddCommand that adds journals to a
mounted GFS file system.
gfs_mkfsCommand that creates a GFS file
system on a storage device.
gfs_quotaCommand that manages quotas on a
mounted GFS file system.
gfs_toolCommand that configures or tunes a
GFS file system. This command can
also gather a variety of information
about the file system.
lock_harness.koImplements a pluggable lock module
interface for GFS that allows for a
variety of locking mechanisms to be
used (for example, the DLM lock
module, lock_dlm.ko).
lock_dlm.koA lock module that implements DLM
locking for GFS. It plugs into the lock
harness, lock_harness.ko and
communicates with the DLM lock
manager in Red Hat Cluster Suite.
lock_gulm.koA lock module that implements
GULM locking for GFS. It plugs into
the lock harness, lock_harness.ko
and communicates with the GULM
lock manager in Red Hat Cluster
Suite.
lock_nolock.koA lock module for use when GFS is
used as a local file system only. It
plugs into the lock harness,
lock_harness.ko and provides
local locking.
GNBDgnbd.koKernel module that implements the
GNBD device driver on clients.
gnbd_exportCommand to create, export and
manage GNBDs on a GNBD server.
gnbd_importCommand to import and manage
GNBDs on a GNBD client.
Chapter 1. GFS Overview9
Software
Subsystem
Table 1-1. GFS Software Subsystem Components
ComponentsDescription
gnbd_servA server daemon that allows a node to
export local storage over the network.
1.5. Before Setting Up GFS
Before you install and set up GFS, note the following key characteristics of your GFS file
systems:
Number of file systems
Determine how many GFS file systems to create initially. (More file systems can be
added later.)
File-system name
Determine a unique name for each file system. Each file-system name is required in
the form of a parameter variable. For example, this book uses file-system names gfs1
and gfs2 in some example procedures.
Journals
Determine the number of journals for your GFS file systems. One journal is required
for each node that mounts a GFS file system. Make sure to account for additional
journals needed for future expansion.
GFS nodes
Determine which nodes in the Red Hat Cluster Suite will mount the GFS file systems.
GNBD server nodes
If you are using GNBD, determine how many GNBD server nodes are needed. Note
the hostname and IP address of each GNBD server node for setting up GNBD clients
later.
Storage devices and partitions
Determine the storage devices and partitions to be used for creating logical volumes
(via CLVM) in the file systems.
10Chapter 1. GFS Overview
Chapter 2.
System Requirements
This chapter describes the system requirements for Red Hat GFS Release 6.1 and consists
of the following sections:
• Section 2.1 Platform Requirements
• Section 2.2 Red Hat Cluster Suite
• Section 2.3 Fencing
• Section 2.4 Fibre Channel Storage Network
• Section 2.5 Fibre Channel Storage Devices
• Section 2.6 Network Power Switches
• Section 2.7 Console Access
2.1. Platform Requirements
Table 2-1 shows the platform requirements for GFS.
Operating SystemHardware
Red Hat Enterprise Linux AS, ES,
or WS, Version 4 or later
Table 2-1. Platform Requirements
Architecture
ia64, x86-64, x86
SMP supported
RAM
256 MB, minimum
2.2. Red Hat Cluster Suite
Red Hat GFS runs with Red Hat Cluster Suite 4.0 or later. The Red Hat Cluster Suite
software must be installed on the cluster nodes before you can install and run Red Hat
GFS.
Note
Red Hat Cluster Suite 4.0 provides the infrastructure for application failover in the cluster
and network communication among GFS nodes (and other Red Hat Cluster Suite nodes).
12Chapter 2. System Requirements
2.3. Fencing
You must configure each GFS node in your Red Hat cluster for at least one form of fencing.
Fencing is configured and managed in Red Hat Cluster Suite. For more information about
fencing options, refer to Red Hat Cluster Suite Configuring and Managing a Cluster.
2.4. Fibre Channel Storage Network
Table 2-2 shows requirements for GFS nodes that are to be connected to a Fibre Channel
SAN.
RequirementDescription
HBA (Host Bus Adapter)One HBA minimum per GFS node
Connection methodFibre Channel switch
Note: If an FC switch is used for fencing, you may
want to consider using Brocade, McData, or Vixel
FC switches, for which Red Hat Cluster Suite
fencing agents exist. Refer to Red Hat Cluster SuiteConfiguring and Managing a Cluster for more
information about supported fencing agents.
Note: When a small number of nodes is used, it may
be possible to connect the nodes directly to ports on
the storage device.
Note: FC drivers may not work reliably with FC hubs.
Table 2-2. Fibre Channel Network Requirements
2.5. Fibre Channel Storage Devices
Table 2-3 shows requirements for Fibre Channel devices that are to be connected to a GFS
cluster.
RequirementDescription
Loading...
+ 60 hidden pages
You need points to download manuals.
1 point = 1 manual.
You can buy points or you can get point for every manual you upload.