Red Hat ENTERPRISE LINUX 5 DM MULTIPATH Configuration And Administration Manual

Red Hat Enterprise Linux 5

DM Multipath

DM Multipath Configuration and Administration

DM Multipath

Red Hat Enterprise Linux 5 DM Multipath
DM Multipath Configuration and Administration
Edition 3

Copyright © 2009 Red Hat, Inc. This material may only be distributed subject to the terms and
conditions set forth in the Open Publication License, V1.0 or later (the latest version of the OPL is
presently available at http://www.opencontent.org/openpub/).

Red Hat and the Red Hat "Shadow Man" logo are registered trademarks of Red Hat, Inc. in the United
States and other countries.

All other trademarks referenced herein are the property of their respective owners.

1801 Varsity Drive
Raleigh, NC 27606-2072 USA
Phone: [phone]
Phone: [phone]
Fax: [phone]
PO Box 13588 Research Triangle Park, NC 27709 USA

This book provides information on using the Device-Mapper Multipath feature of Red Hat Enterprise
Linux 5

iii

Preface v

1. Audience ........................................................................................................................ v

2. Related Documentation ................................................................................................... v

3. Feedback ....................................................................................................................... vi

4. Document Conventions ................................................................................................... vi

4.1. Typographic Conventions ..................................................................................... vi

4.2. Pull-quote Conventions ....................................................................................... viii

4.3. Notes and Warnings ........................................................................................... viii

1. Device Mapper Multipathing 1

1.1. Overview of DM-Multipath ............................................................................................. 1

1.2. Storage Array Support .................................................................................................. 3

1.3. DM-Multipath Components ............................................................................................ 3

1.4. DM-Multipath Setup Overview ....................................................................................... 4

2. Multipath Devices 5

2.1. Multipath Device Identifiers ........................................................................................... 5

2.2. Consistent Multipath Device Names in a Cluster ............................................................ 5

2.3. Multipath Device Attributes ........................................................................................... 6

2.4. Multipath Devices in Logical Volumes ............................................................................ 6

3. Setting Up DM-Multipath 7

3.1. Setting Up DM-Multipath ............................................................................................... 7

3.2. Ignoring Local Disks when Generating Multipath Devices ................................................ 8

3.3. Adding Devices to the Multipathing Database ............................................................. 10

4. The DM-Multipath Configuration File 11

4.1. Configuration File Overview ........................................................................................ 11

4.2. Configuration File Blacklist .......................................................................................... 12

4.2.1. Blacklisting by WWID ....................................................................................... 12

4.2.2. Blacklisting By Device Name ............................................................................ 12

4.2.3. Blacklisting By Device Type ............................................................................. 13

4.2.4. Blacklist Exceptions ......................................................................................... 13

4.3. Configuration File Defaults .......................................................................................... 14

4.4. Multipaths Device Configuration Attributes ................................................................... 17

4.5. Configuration File Devices .......................................................................................... 18

5. DM-Multipath Administration and Troubleshooting 23

5.1. Resizing an Online Multipath Device ........................................................................... 23

5.2. The Multipath Daemon ............................................................................................... 24

5.3. Issues with Large Number of LUNs ............................................................................. 24

5.4. Issues with queue_if_no_path feature .......................................................................... 24

5.5. Multipath Command Output ........................................................................................ 25

5.6. Multipath Queries with multipath Command ................................................................. 25

5.7. Multipath Command Options ....................................................................................... 26

5.8. Determining Device Mapper Entries with the dmsetup Command ................................... 26

5.9. Troubleshooting with the multipathd Interactive Console ................................................ 27

A. Revision History 29

Index 31

iv

v

Preface

This book describes the Device Mapper Multipath (DM-Multipath) feature of Red Hat Enterprise Linux
for the RHEL 5 release.

1. Audience

This book is intended to be used by system administrators managing systems running the Linux
operating system. It requires familiarity with Red Hat Enterprise Linux.

2. Related Documentation

For more information about using Red Hat Enterprise Linux, refer to the following resources:

• Red Hat Enterprise Linux Installation Guide — Provides information regarding installation of Red
Hat Enterprise Linux 5.

• Red Hat Enterprise Linux Deployment Guide — Provides information regarding the deployment,
configuration and administration of Red Hat Enterprise Linux 5.

For more information about Red Hat Cluster Suite for Red Hat Enterprise Linux 5, refer to the following
resources:

• Red Hat Cluster Suite Overview — Provides a high level overview of the Red Hat Cluster Suite.

• Configuring and Managing a Red Hat Cluster — Provides information about installing, configuring
and managing Red Hat Cluster components.

• Logical Volume Manager Administration — Provides a description of the Logical Volume Manager
(LVM), including information on running LVM in a clustered environment.

• Global File System: Configuration and Administration — Provides information about installing,
configuring, and maintaining Red Hat GFS (Red Hat Global File System).

• Global File System 2: Configuration and Administration — Provides information about installing,
configuring, and maintaining Red Hat GFS2 (Red Hat Global File System 2).

• Using GNBD with Global File System — Provides an overview on using Global Network Block
Device (GNBD) with Red Hat GFS.

• Linux Virtual Server Administration — Provides information on configuring high-performance
systems and services with the Linux Virtual Server (LVS).

• Red Hat Cluster Suite Release Notes — Provides information about the current release of Red Hat
Cluster Suite.

Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML,
PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at http://

www.redhat.com/docs/.

Preface

vi

3. Feedback

If you spot a typo, or if you have thought of a way to make this manual better, we would love to
hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against the
component rh-cs.

Be sure to mention the manual's identifier:

Bugzilla component: Documentation-cluster
Book identifier: DM_Multipath(EN)-5 (2009-08-18T15:35)

By mentioning this manual's identifier, we know exactly which version of the guide you have.

If you have a suggestion for improving the documentation, try to be as specific as possible. If you have
found an error, please include the section number and some of the surrounding text so we can find it
easily.

4. Document Conventions

This manual uses several conventions to highlight certain words and phrases and draw attention to
specific pieces of information.

In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts1 set. The
Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not,
alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later includes
the Liberation Fonts set by default.

4.1. Typographic Conventions

Four typographic conventions are used to call attention to specific words and phrases. These
conventions, and the circumstances they apply to, are as follows.

Mono-spaced Bold

Used to highlight system input, including shell commands, file names and paths. Also used to highlight
key caps and key-combinations. For example:

To see the contents of the file my_next_bestselling_novel in your current
working directory, enter the cat my_next_bestselling_novel command at the
shell prompt and press Enter to execute the command.

The above includes a file name, a shell command and a key cap, all presented in Mono-spaced Bold
and all distinguishable thanks to context.

Key-combinations can be distinguished from key caps by the hyphen connecting each part of a key­combination. For example:

Press Enter to execute the command.

Press Ctrl-Alt-F1 to switch to the first virtual terminal. Press Ctrl-Alt-F7 to return
to your X-Windows session.

1

https://fedorahosted.org/liberation-fonts/

Typographic Conventions

vii

The first sentence highlights the particular key cap to press. The second highlights two sets of three
key caps, each set pressed simultaneously.

If source code is discussed, class names, methods, functions, variable names and returned values
mentioned within a paragraph will be presented as above, in Mono-spaced Bold. For example:

File-related classes include filesystem for file systems, file for files, and dir for
directories. Each class has its own associated set of permissions.

Proportional Bold

This denotes words or phrases encountered on a system, including application names; dialogue
box text; labelled buttons; check-box and radio button labels; menu titles and sub-menu titles. For
example:

Choose System > Preferences > Mouse from the main menu bar to launch Mouse

Preferences. In the Buttons tab, click the Left-handed mouse check box and click
Close to switch the primary mouse button from the left to the right (making the mouse

suitable for use in the left hand).

To insert a special character into a gedit file, choose Applications > Accessories

> Character Map from the main menu bar. Next, choose Search > Find… from the
Character Map menu bar, type the name of the character in the Search field and click
Next. The character you sought will be highlighted in the Character Table. Double-
click this highlighted character to place it in the Text to copy field and then click the
Copy button. Now switch back to your document and choose Edit > Paste from the
gedit menu bar.

The above text includes application names; system-wide menu names and items; application-specific
menu names; and buttons and text found within a GUI interface, all presented in Proportional Bold and
all distinguishable by context.

Note the > shorthand used to indicate traversal through a menu and its sub-menus. This is to avoid
the difficult-to-follow 'Select Mouse from the Preferences sub-menu in the System menu of the main
menu bar' approach.

Mono-spaced Bold Italic or Proportional Bold Italic

Whether Mono-spaced Bold or Proportional Bold, the addition of Italics indicates replaceable or
variable text. Italics denotes text you do not input literally or displayed text that changes depending on
circumstance. For example:

To connect to a remote machine using ssh, type ssh [e-mail] at
a shell prompt. If the remote machine is example.com and your username on that
machine is john, type ssh [e-mail].

The mount -o remount file-system command remounts the named file
system. For example, to remount the /home file system, the command is mount -o
remount /home.

To see the version of a currently installed package, use the rpm -q package
command. It will return a result as follows: package-version-release.

Note the words in bold italics above — username, domain.name, file-system, package, version and
release. Each word is a placeholder, either for text you enter when issuing a command or for text
displayed by the system.

Preface

viii

Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and
important term. For example:

When the Apache HTTP Server accepts requests, it dispatches child processes
or threads to handle them. This group of child processes or threads is known as
a server-pool. Under Apache HTTP Server 2.0, the responsibility for creating and
maintaining these server-pools has been abstracted to a group of modules called
Multi-Processing Modules (MPMs). Unlike other modules, only one module from the
MPM group can be loaded by the Apache HTTP Server.

4.2. Pull-quote Conventions

Two, commonly multi-line, data types are set off visually from the surrounding text.

Output sent to a terminal is set in Mono-spaced Roman and presented thus:

books Desktop documentation drafts mss photos stuff svn
books_tests Desktop1 downloads images notes scripts svgs

Source-code listings are also set in Mono-spaced Roman but are presented and highlighted as
follows:

package org.jboss.book.jca.ex1;

import javax.naming.InitialContext;

public class ExClient

{
public static void main(String args[])
throws Exception
{
InitialContext iniCtx = new InitialContext();
Object ref = iniCtx.lookup("EchoBean");
EchoHome home = (EchoHome) ref;
Echo echo = home.create();

System.out.println("Created Echo");

System.out.println("Echo.echo('Hello') = " + echo.echo("Hello"));
}

}

4.3. Notes and Warnings

Finally, we use three visual styles to draw attention to information that might otherwise be overlooked.

Notes and Warnings

ix

Note

A Note is a tip or shortcut or alternative approach to the task at hand. Ignoring a note
should have no negative consequences, but you might miss out on a trick that makes your
life easier.

Important

Important boxes detail things that are easily missed: configuration changes that only
apply to the current session, or services that need restarting before an update will apply.
Ignoring Important boxes won't cause data loss but may cause irritation and frustration.

Warning

A Warning should not be ignored. Ignoring warnings will most likely cause data loss.

x

Chapter 1.

1

Device Mapper Multipathing

Device Mapper Multipathing (DM-Multipath) allows you to configure multiple I/O paths between server
nodes and storage arrays into a single device. These I/O paths are physical SAN connections that can
include separate cables, switches, and controllers. Multipathing aggregates the I/O paths, creating a
new device that consists of the aggregated paths.

1.1. Overview of DM-Multipath

DM-Multipath can be used to provide:

• Redundancy

DM-Multipath can provide failover in an active/passive configuration. In an active/passive
configuration, only half the paths are used at any time for I/O. If any element of an I/O path (the
cable, switch, or controller) fails, DM-Multipath switches to an alternate path.

• Improved Performance

DM-Multipath can be configured in active/active mode, where I/O is spread over the paths in a
round-robin fashion. In some configurations, DM-Multipath can detect loading on the I/O paths and
dynamically re-balance the load.

Figure 1.1, “Active/Passive Multipath Configuration with One RAID Device” shows an active/passive

configuration with two I/O paths from the server to a RAID device. There are 2 HBAs on the server, 2
SAN switches, and 2 RAID controllers.

Figure 1.1. Active/Passive Multipath Configuration with One RAID Device

In this configuration, there is one I/O path that goes through hba1, SAN1, and controller 1 and a
second I/O path that goes through hba2, SAN2, and controller2. There are many points of possible
failure in this configuration:

Chapter 1. Device Mapper Multipathing

2

• HBA failure

• FC cable failure

• SAN switch failure

• Array controller port failure

With DM-Multipath configured, a failure at any of these points will cause DM-Multipath to switch to the
alternate I/O path.

Figure 1.2, “Active/Passive Multipath Configuration with Two RAID Devices” shows a more complex

active/passive configuration with 2 HBAs on the server, 2 SAN switches, and 2 RAID devices with 2
RAID controllers each.

Figure 1.2. Active/Passive Multipath Configuration with Two RAID Devices

In the example shown in Figure 1.2, “Active/Passive Multipath Configuration with Two RAID Devices”,
there are two I/O paths to each RAID device (just as there are in the example shown in Figure 1.1,

“Active/Passive Multipath Configuration with One RAID Device”). With DM-Multipath configured, a

failure at any of the points of the I/O path to either of the RAID devices will cause DM-Multipath to
switch to the alternate I/O path for that device.

Figure 1.3, “Active/Active Multipath Configuration with One RAID Device” shows an active/active

configuration with 2 HBAs on the server, 1 SAN switch, and 2 RAID controllers. There are four I/O
paths from the server to a storage device:

• hba1 to controller1

• hba1 to controller2

• hba2 to controller1

• hba2 to controller2

Storage Array Support

3

In this configuration, I/O can be spread among those four paths.

Figure 1.3. Active/Active Multipath Configuration with One RAID Device

1.2. Storage Array Support

By default, DM-Multipath includes support for the most common storage arrays that support DM­Multipath. The supported devices can be found in the multipath.conf.defaults file. If your
storage array supports DM-Multipath and is not configured by default in this file, you may need to add
them to the DM-Multipath configuration file, multipath.conf. For information on the DM-Multipath
configuration file, see Chapter 4, The DM-Multipath Configuration File.

Some storage arrays require special handling of I/O errors and path switching. These require separate
hardware handler kernel modules.

1.3. DM-Multipath Components

Table 1.1, “DM-Multipath Components”. describes the components of DM-Multipath.

Component Description

dm-multipath kernel module Reroutes I/O and supports failover for paths and path groups.

multipath command Lists and configures multipath devices. Normally started up

with /etc/rc.sysinit, it can also be started up by a udev
program whenever a block device is added or it can be run by the

initramfs file system.

multipathd daemon Monitors paths; as paths fail and come back, it may initiate path

group switches. Provides for interactive changes to multipath
devices. This must be restarted for any changes to the /etc/
multipath.conf file.