Novell SUSE Linux Enterprise 10 Storage Administration Guide

Download

Storage Administration Guide

Novell®

SUSE® Linux Enterprise Server

novdocx (en) 13 May 2009

AUTHORIZED DOCUMENTATION

10 SP2

May 15, 2009

www.novell.com

SLES 10 SP2: Storage Administration Guide

Legal Notices

Novell, Inc. makes no representations or warranties with respect to the contents or use of this documentation, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Novell, Inc. reserves the right to revise this publication and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes.

Further, Novell, Inc. makes no representations or warranties with respect to any software, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Novell, Inc. reserves the right to make changes to any and all parts of Novell software, at any time, without any obligation to notify any person or entity of such changes.

Any products or technical information provided under this Agreement may be subject to U.S. export controls and the trade laws of other countries. You agree to comply with all export control regulations and to obtain any required licenses or classification to export, re-export or import deliverables. You agree not to export or re-export to entities on the current U.S. export exclusion lists or to any embargoed or terrorist countries as specified in the U.S. export laws. You agree to not use deliverables for prohibited nuclear, missile, or chemical biological weaponry end uses. See the

Novell International Trade Services Web page (http://www.novell.com/info/exports/) for more information on

exporting Novell software. Novell assumes no responsibility for your failure to obtain any necessary export approvals.

novdocx (en) 13 May 2009

Copyright © 2006-2008 Novell, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the publisher.

Novell, Inc. has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed on theNovell Legal Patents Web page (http://www.novell.com/company/legal/patents/) and one or more additional patents or pending patent applications in the U.S. and in other countries.

Novell, Inc. 404 Wyman Street, Suite 500 Waltham, MA 02451 U.S.A. www.novell.com

Online Documentation: To access the latest online documentation for this and other Novell products, see

the Novell Documentation Web page (http://www.novell.com/documentation).

Novell Trademarks

For Novell trademarks, see the Novell Trademark and Service Mark list (http://www.novell.com/company/legal/

trademarks/tmlist.html).

Third-Party Materials

All third-party trademarks and copyrights are the property of their respective owners.

Some content in this document is copied, distributed, and/or modified from the following document under the terms

specified in the document’s license.

EVMS User Guide, January 18, 2005

License Information

This document may be reproduced or distributed in any form without prior permission provided the copyright

notice is retained on all copies. Modified versions of this document may be freely distributed provided that they are clearly identified as such, and this copyright is included intact.

novdocx (en) 13 May 2009

4 SLES 10 SP2: Storage Administration Guide

Contents

About This Guide 9

1 Overview of EVMS 11

1.1 Benefits of EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.2 Plug-In Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3 Supported File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.5 Location of Device Nodes for EVMS Storage Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Using EVMS to Manage Devices 15

2.1 Configuring the System Device at Install to Use EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1.1 Before the Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1.2 During the Server Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.3 After the Server Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2 Configuring an Existing System Device to Use EVMS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.2.1 Disable the boot.lvm and boot.md Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.2.2 Enable the boot.evms Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.2.3 Edit the /etc/fstab File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.2.4 Edit the Boot Loader File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.2.5 Force the RAM Disk to Recognize the Root Partition . . . . . . . . . . . . . . . . . . . . . . . . 25

2.2.6 Restart the Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.2.7 Verify that EVMS Manages the Boot, Swap, and Root Partitions . . . . . . . . . . . . . . . 26

2.3 Configuring LVM Devices to Use EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.4 Using EVMS with iSCSI Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.5 Using the ELILO Loader Files (IA-64) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.6 Starting EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.7 Starting the EVMS Management Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

novdocx (en) 13 May 2009

3 Using UUIDs to Mount Devices 31

3.1 Naming Devices with udev. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2 Understanding UUIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2.1 Using UUIDs to Assemble or Activate File System Devices . . . . . . . . . . . . . . . . . . . 32

3.2.2 Finding the UUID for a File System Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3 Using UUIDs in the Boot Loader and /etc/fstab File (x86) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.4 Using UUIDs in the Boot Loader and /etc/fstab File (IA64) . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5 Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4 Managing EVMS Devices 35

4.1 Understanding Disk Segmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.1.1 Segment Managers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.1.2 Disk Segments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2 Initializing Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.1 Before You Begin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.2 Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2.3 Adding a Segment Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3 Removing the Segment Manager from a Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Contents 5

4.4 Creating Disk Segments (or Partitions) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.5 Configuring Mount Options for Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.6 What’s Next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5 Managing Multipath I/O for Devices 43

5.1 Understanding Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1.1 What Is Multipathing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1.2 Benefits of Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.2 Planning for Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.2.1 Guidelines for Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.2.2 Using Multipathed Devices Directly or in EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.2.3 Using LVM2 on Multipath Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.2.4 Using mdadm with Multipath Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.2.5 Partitioning Multipath Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.2.6 Supported Architectures for Multipath I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.2.7 Supported Storage Arrays for Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.3 Multipath Management Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.3.1 Device Mapper Multipath Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.3.2 Multipath I/O Management Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.3.3 Using mdadm for Multipathed Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.3.4 The Linux multipath(8) Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.4 Configuring the System for Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.4.1 Preparing SAN Devices for Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.4.2 Partitioning Multipathed Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.4.3 Configuring the Server for Multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.4.4 Adding multipathd to the Boot Sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

5.4.5 Creating and Configuring the /etc/multipath.conf File . . . . . . . . . . . . . . . . . . . . . . . . 55

5.5 Enabling and Starting Multipath I/O Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.6 Configuring Path Failover Policies and Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.6.1 Configuring the Path Failover Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.6.2 Configuring Failover Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.6.3 Using a Script to Set Path Priorities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.6.4 Configuring ALUA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.6.5 Reporting Target Path Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.7 Tuning the Failover for Specific Host Bus Adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.8 Configuring Multipath I/O for the Root Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.9 Configuring Multipath I/O for an Existing Software RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5.10 Scanning for New Devices without Rebooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5.11 Scanning for New Partitioned Devices without Rebooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

5.12 Viewing Multipath I/O Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.13 Managing I/O in Error Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.14 Resolving Stalled I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.15 Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.16 What’s Next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

novdocx (en) 13 May 2009

6 Managing Software RAIDs with EVMS 75

6.1 Understanding Software RAIDs on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

6.1.1 What Is a Software RAID?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

6.1.2 Overview of RAID Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

6.1.3 Comparison of RAID Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

6.1.4 Comparison of Disk Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

6.1.5 Configuration Options for RAIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

6.1.6 Interoperability Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

6 SLES 10 SP2: Storage Administration Guide

6.1.7 Guidelines for Component Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6.1.8 RAID 5 Algorithms for Distributing Stripes and Parity . . . . . . . . . . . . . . . . . . . . . . . . 80

6.1.9 Multi-Disk Plug-In for EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.1.10 Device Mapper Plug-In for EVMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.2 Creating and Configuring a Software RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.3 Expanding a RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6.3.1 Adding Mirrors to a RAID 1 Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6.3.2 Adding Segments to a RAID 4 or 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6.4 Adding or Removing a Spare Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6.4.1 Do You Need a Spare Disk? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6.4.2 Adding a Spare Disk When You Create the RAID. . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.4.3 Adding a Spare Disk to an Existing RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.4.4 Removing a Spare Disk from a RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.5 Managing Disk Failure and RAID Recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.5.1 Understanding the Disk Failure and RAID Recovery . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.5.2 Identifying the Failed Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.5.3 Replacing a Failed Device with a Spare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

6.5.4 Removing the Failed Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

6.6 Monitoring Status for a RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

6.6.1 Monitoring Status with EVMSGUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

6.6.2 Monitoring Status with /proc/mdstat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

6.6.3 Monitoring Status with mdadm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

6.6.4 Monitoring a Remirror or Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

6.6.5 Configuring mdadm to Send an E-Mail Alert for RAID Events. . . . . . . . . . . . . . . . . . 93

6.7 Deleting a Software RAID and Its Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

novdocx (en) 13 May 2009

7 Managing Software RAIDs 6 and 10 with mdadm 97

7.1 Creating a RAID 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

7.1.1 Understanding RAID 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

7.1.2 Creating a RAID 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

7.2 Creating Nested RAID 10 Devices with mdadm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

7.2.1 Understanding Nested RAID Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

7.2.2 Creating Nested RAID 10 (1+0) with mdadm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

7.2.3 Creating Nested RAID 10 (0+1) with mdadm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

7.3 Creating a Complex RAID 10 with mdadm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

7.3.1 Understanding the mdadm RAID10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

7.3.2 Creating a RAID10 with mdadm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

7.4 Creating a Degraded RAID Array. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

8 Resizing Software RAID Arrays with mdadm 107

8.1 Understanding the Resizing Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

8.1.1 Guidelines for Resizing a Software RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

8.1.2 Overview of Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

8.2 Increasing the Size of a Software RAID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

8.2.1 Increasing the Size of Component Partitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

8.2.2 Increasing the Size of the RAID Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

8.2.3 Increasing the Size of the File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

8.3 Decreasing the Size of a Software RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

8.3.1 Decreasing the Size of the File System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

8.3.2 Decreasing the Size of Component Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

8.3.3 Decreasing the Size of the RAID Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Contents 7

9 Installing and Managing DRBD Services 117

9.1 Understanding DRBD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

9.2 Installing DRBD Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

9.3 Configuring the DRBD Service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

9.4 Testing the DRBD Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

9.5 Troubleshooting DRBD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

9.5.1 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

9.5.2 Host Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

9.5.3 TCP Port 7788 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

9.5.4 The --do-what-i-say Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

9.6 Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

10 Troubleshooting Storage Issues 123

10.1 Is DM-MP Available for the Boot Partition? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

10.2 Rescue System Cannot Find Devices That Are Managed by EVMS . . . . . . . . . . . . . . . . . . . 123

10.3 Volumes on EVMS Devices Do Not Appear After Reboot . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

10.4 Volumes on EVMS Devices Do Not Appear When Using iSCSI . . . . . . . . . . . . . . . . . . . . . . 124

10.5 Device Nodes Are Not Automatically Re-Created on Restart . . . . . . . . . . . . . . . . . . . . . . . . 124

novdocx (en) 13 May 2009

A Documentation Updates 125

A.1 May 15, 2009 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

A.1.1 Managing Multipath I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

A.2 November 24, 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

A.2.1 Managing Multipath I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

A.2.2 Using UUIDs to Mount Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

A.3 June 10, 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

A.3.1 Managing Multipath I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

A.4 March 20, 2008 (SLES 10 SP2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

A.4.1 Managing Multipath I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

8 SLES 10 SP2: Storage Administration Guide

About This Guide

This guide provides information about how to manage storage devices on a SUSE® Linux Enterprise Server 10 Support Pack 2 server with an emphasis on using the Enterprise Volume Management System (EVMS) 2.5.5 or later to manage devices. Related storage administration issues are also covered as noted below.

 Chapter 1, “Overview of EVMS,” on page 11

 Chapter 2, “Using EVMS to Manage Devices,” on page 15

 Chapter 3, “Using UUIDs to Mount Devices,” on page 31

 Chapter 4, “Managing EVMS Devices,” on page 35

 Chapter 5, “Managing Multipath I/O for Devices,” on page 43

 Chapter 6, “Managing Software RAIDs with EVMS,” on page 75

 Chapter 7, “Managing Software RAIDs 6 and 10 with mdadm,” on page 97

 Chapter 8, “Resizing Software RAID Arrays with mdadm,” on page 107

novdocx (en) 13 May 2009

 Chapter 9, “Installing and Managing DRBD Services,” on page 117

 Chapter 10, “Troubleshooting Storage Issues,” on page 123

Audience

This guide is intended for system administrators.

Feedback

We want to hear your comments and suggestions about this manual and the other documentation included with this product. Please use the User Comments feature at the bottom of each page of the online documentation, or go to www.novell.com/documentation/feedback.html and enter your comments there.

Documentation Updates

For the most recent version of the SUSE Linux Enterprise Server 10 Storage Administration Guide for EVMS, visit the Novell Documentation Web site for SUSE Linux Enterprise Server 10 (http://

www.novell.com/documentation/sles10).

Additional Documentation

For information about managing storage with the Linux Volume Manager (LVM), see the SUSE

Linux Enterprise Server 10 Installation and Administration Guide (http://www.novell.com/ documentation/sles10).

For information about iSNS (Internet Storage Name Service), see “iSNS” (http://www.novell.com/

documentation/sles10/sles_admin/index.html?page=/documentation/sles10/sles_admin/data/ cha_isns.html) in the SUSE Linux Enterprise Server 10 Installation and Administration Guide (http:/ /www.novell.com/documentation/sles10).

About This Guide 9

Documentation Conventions

In Novell documentation, a greater-than symbol (>) is used to separate actions within a step and items in a cross-reference path.

A trademark symbol (

, TM, etc.) denotes a Novell trademark. An asterisk (*) denotes a third-party

trademark.

novdocx (en) 13 May 2009

10 SLES 10 SP2: Storage Administration Guide

Overview of EVMS

The Enterprise Volume Management System (EVMS) 2.5.5 management tool for Linux* is an extensible storage management tool that integrates all aspects of volume management, such as disk partitioning, the Logical Volume Manager (LVM), the Multiple-Disk (MD) manager for software RAIDs, the Device Mapper (DM) for multipath I/O configuration, and file system operations.

 Section 1.1, “Benefits of EVMS,” on page 11

 Section 1.2, “Plug-In Layers,” on page 11

 Section 1.3, “Supported File Systems,” on page 12

 Section 1.4, “Terminology,” on page 12

 Section 1.5, “Location of Device Nodes for EVMS Storage Objects,” on page 13

1.1 Benefits of EVMS

novdocx (en) 13 May 2009

EVMS provides the following benefits:

 An open source volume manager

 A plug-in framework for flexible extensibility and customization

 Plug-ins to extend functionality for new or evolving storage managers

 Support for foreign partition formats

 Cluster-aware

1.2 Plug-In Layers

EVMS abstracts the storage objects in functional layers to make storage management more userfriendly. The following table describes the current EVMS plug-in layers for managing storage devices and file systems:

Table 1-1 EVMS Plug-In Layers

Storage Managers Description Plug-Ins

Device Manages the physical and logical

devices

Segment Manages the partitioning of

physical and logical devices into smaller segments of free space.

Segment managers can be stacked. For example, a cluster segment can contain other storage objects or volumes.

Device Mapper (DM)

Uses Device Mapper (DM)

Segment managers include DOS, GPT, System/390* (S/390), Cluster, BSD, Mac, and BBR

For more information, see

Section 4.1, “Understanding Disk Segmentation,” on page 35.

Overview of EVMS

Storage Managers Description Plug-Ins

novdocx (en) 13 May 2009

Regions Manages the combination of

multiple storage objects

EVMS Features Manages EVMS features Drive linking (linear

File System Interface Modules (FSIM)

Cluster Manager Interface Modules

Manages the interface between the file system managers and the segment managers

Manages the interface between the cluster manager and the file systems and devices

LVM/LVM2 for containers and region, MD for RAIDs, and DM for multipath I/O

concatenation), Bad Block Relocation (BBR), and Snapshot

For information, see Section 1.3,

“Supported File Systems,” on page 12.

HeartBeat 2

1.3 Supported File Systems

EVMS supports the following Linux file systems:

 EXT3

 ReiserFS

 XFS

 OCFS2

 JFS

 EXT2

 Swap

 NTFS (read only)

 FAT (read only)

For more information about file systems supported in SUSE® Linux Enterprise Server 10, see the

SUSE Linux Enterprise Server 10 Installation and Administration Guide. (http://www.novell.com/ documentation/sles10).

The File System Primer (http://wiki.novell.com/index.php/File_System_Primer) describes the variety of file systems available on Linux and which ones are the best to use for which workloads and data.

1.4 Terminology

EVMS uses the following terminology in the EVMS user interface:

Table 1-2 EVMS Terms

Term Description

Sector The lowest level that can be addressed on a block device.

Disk A physical disk or a logical device.

12 SLES 10 SP2: Storage Administration Guide

Term Description

Segment An ordered set of physically contiguous sectors on a single device. It is similar

to traditional disk partitions.

Region An ordered set of logically contiguous sectors that might or might not be

physically contiguous. The underlying mapping can be to logical disks, disk segments, or other storage regions.

novdocx (en) 13 May 2009

Feature

(Feature Object, EVMS Feature, EVMS Object)

Storage Object Any storage structure in EVMS that is capable of being a block device. Disks,

Container

(Storage Container)

A logically contiguous address space created from one or more disks, segments, regions, or other feature objects through the use of an EVMS feature.

segments, regions, and feature objects are all storage objects.

A collection of devices that is managed as a single pool of storage.

Private Storage Container: A storage container that is exclusively owned and accessed by only one server.

Cluster Storage Container: A storage container managed by the Cluster Resource Manager. It is accessible to all nodes of a cluster. An administrator can configure the storage objects in the cluster container from any node in the cluster. Cluster containers can be private, shared, or deported.

 Private: The cluster container is exclusively owned and accessed by only

one particular node of a cluster at any given time. The ownership can be reassigned by failover policies or the administrator.

 Shared: The cluster container is concurrently owned and accessed by all

nodes of a cluster. Shared containers are preferred for distributed databases, clustered file systems, and cluster-aware applications that can coordinate safe access to shared volumes.

 Deported: The cluster container is not owned or accessed by any node of

the cluster.

Vol ume

(Logical Volume)

A mountable storage object. Logical volumes can be EVMS volumes or compatibility volumes.

 EVMS Volume: Volumes that contain EVMS metadata and support all

EVMS features. Device nodes for EVMS volumes are stored in the

directory. For example:

evms

/dev/evms/my_volume

/dev/

 Compatibility Volume: Volumes that are backward-compatible to other

volume managers. They do not contain EVMS metadata and cannot support EVMS features.

1.5 Location of Device Nodes for EVMS Storage Objects

EVMS creates a unified namespace for the logical volumes on your system in the directory. It detects the storage objects actually present on a system, and creates an appropriate device node for each one, such as those shown in the following table.

/dev/evms

Overview of EVMS 13

Table 1-3 Device Node Location

Storage Object Standard Location the Device Node EVMS Location of the Device Node

novdocx (en) 13 May 2009

A disk segment of disk

A software RAID device

An LVM volume

/dev/sda5

/dev/md1

/dev/lvm_group/lvm_volume

/dev/evms/sda5

/dev/evms/md/md1

/dev/evms/lvm/lvm_group/ lvm_volume

14 SLES 10 SP2: Storage Administration Guide

Using EVMS to Manage Devices

This section describes how to configure EVMS as the volume manager of your devices.

 Section 2.1, “Configuring the System Device at Install to Use EVMS,” on page 15

 Section 2.2, “Configuring an Existing System Device to Use EVMS,” on page 22

 Section 2.3, “Configuring LVM Devices to Use EVMS,” on page 27

 Section 2.4, “Using EVMS with iSCSI Volumes,” on page 27

 Section 2.5, “Using the ELILO Loader Files (IA-64),” on page 28

 Section 2.6, “Starting EVMS,” on page 28

 Section 2.7, “Starting the EVMS Management Tools,” on page 28

2.1 Configuring the System Device at Install to Use EVMS

novdocx (en) 13 May 2009

This section describes how to configure the system device during the Linux install to use EVMS as the volume manager instead of the current default of Linux Volume Manager (LVM).

 Section 2.1.1, “Before the Install,” on page 15

 Section 2.1.2, “During the Server Install,” on page 17

 Section 2.1.3, “After the Server Install,” on page 20

2.1.1 Before the Install

 “System Device” on page 15

 “Device Size Limits” on page 16

 “Data Loss Considerations for the System Device” on page 16

 “Storage Deployment Considerations for the System Device” on page 16

System Device

For the purposes of this install documentation, a system device is any device that contains the Linux

/boot, swap

The install instructions assume the following:

 All three system partitions are on the same physical disk.

, or root (/) partitions for your Linux computer.

If you use different disks for any of the system partitions, make sure to modify the install instructions for your deployment scenario so that all of the system partitions are managed by EVMS.

 You must configure the boot partition within the BIOS-addressable space (such as 2 GB for x86

or 8 GB for x86-64) of the first disk recognized by the system.

If this restriction is not required for your hardware, you can modify the location of the partition as desired.

Using EVMS to Manage Devices

/boot

 Your system uses the Grub or LILO boot loaders.

If you have an IA64 system, you must modify these install instructions to use the ELILO boot loader (

/boot/efi/elilo.conf

) instead.

novdocx (en) 13 May 2009

WARNING: Whenever you manually alter the kernel or

/sbin/elilo

run

before shutting down the computer. If you leave out this step, your system

initrd

on your system, make sure to

might not be bootable.

Device Size Limits

Version 2.3 and later of

mdadm

supports component devices up to 4 TB in size each. Earlier versions

support component devices up to 2 TB in size.

IMPORTANT: If you have a local disk, external disk arrays, or SAN devices that are larger than the supported device size, use a third-party disk partitioner to carve the devices into smaller logical devices.

You can combine up to 28 component devices to create the RAID array. The

RAID device you

create can be up to the maximum device size supported by the file system you plan to use. For

information about file system limits for SUSE

Linux Enterprise Server 10, see “Large File System Support” in the SUSE Linux Enterprise Server 10 Installation and Administration Guide. (http://

www.novell.com/documentation/sles10).

Data Loss Considerations for the System Device

This install requires that you delete the default partitioning settings created by the install, and create new partitions to use EVMS instead. This destroys all data on the disk.

WARNING: To avoid data loss, it is best to use the EVMS install option only on a new device.

If you have data volumes on the system device, take one or more of the following precautionary measures:

 Move the data volumes from the system device to another device.

 If you cannot move the volumes, make a backup copy of the data, so you can restore the data

volumes later from a backup copy.

Storage Deployment Considerations for the System Device

By default, the YaST install for SUSE Linux Enterprise Server uses the Linux Volume Manager to manage the system device. The install procedures in this section describe how to install SUSE Linux Enterprise Server with EVMS as the volume manager of the system device. The instructions assume the following:

 You want to use EVMS to manage the system device.

 Only the system device is to be configured during the install.

 Other devices on the system are not configured during the install, or are attached to the server

later. These additional devices are configured only after the system is operating and performing as expected.

16 SLES 10 SP2: Storage Administration Guide

2.1.2 During the Server Install

To install Linux with EVMS as the volume manager for your boot and system partitions, you must modify the Partitioning configuration in the Installation Settings.

WARNING: The following procedure destroys all data on the system device.

1 Begin the install, according to the instructions provided in Deployment (http://

www.novell.com/documentation/sles10/sles_admin/data/part_setup.html) in the SUSE Linux Enterprise 10 Installation and Administration Guide (http://www.novell.com/documentation/ sles10/sles_admin/data/bookinfo_book_sles_admin.html).

2 When the installation reaches the Installations Settings screen, delete the proposed LVM-based

partioning solution. This deletes the proposed partitions and the partition table on the system device so that the device can be marked to use EVMS as the volume manager instead of LVM.

2a In the list of Installation Settings, select Partitioning.

2b In the Partitioning menu, select Create Custom Partition Setup, then click Next.

2c Select Custom Partition - for Experts, then click Next to open the Expert Partitioner

dialog box.

novdocx (en) 13 May 2009

2d Select Expert > Delete Partition Table and Disk Label, then click Yes twice to continue

through the Warning advisories.

This deletes the recommended partitions and the partition table on the system disk.

3 Create a primary partition on the system disk to use as the boot partition:

3a Click Create.

3b From the list of devices, select the device you want to use for the boot partition, such as

dev/hda

, then click OK.

If you have a single system disk, only one device is available, and you are not prompted to choose the device.

3c Select Primary Partition, then click OK.

3d Select Format, then select the native Linux file system you want to use, such as Ext3.

IMPORTANT: In a paravirtualized environment, use Ext2 as the file system for the boot device.

3e In Size (End Value) field, specify 200 MB or larger.

For example, to set the size at 300 MB, type

300M

3f Set the mount point to /boot.

3g Click OK.

The partition appears as a logical device in the devices list, such as

/dev/hda1

4 Create a second primary partition on the system disk to use for both the swap and system

volumes:

4a Click Create.

4b From the list of devices, select the device you want to use for the second primary partition,

such as

/dev/hda

, then click OK.

If you have a single system disk, only one device is available and you are not prompted to choose the device.

Using EVMS to Manage Devices 17

4c Select Primary Partition, then click OK.

4d Select Do Not Format, then select Linux LVM (0x8E) from the list of file system IDs.

4e In Size (End Value field), set the cylinder End value to 5 GB or larger, depending on the

combined partition size you need to contain your system and swap volumes.

IMPORTANT: Do not make the system partition larger than necessary. The remaining space on the system disk can be used to create NSS volumes or native Linux volumes that are managed by EVMS.

To determing how much space to use, consider the following recommendations:

 For your system volume, allow 2 GB (minimum) to 10 GB (recommended),

depending on the OES services that you intend to install.

 If you intend to create additional NSS volumes on the same physical disk, you must

leave unpartitioned space available.

 Set aside 128 MB or larger for the swap volume.

Swap management is different for Linux kernel 2.4.10 and later. How much swap to add depends on the RAM size, the tasks that are planned for the system, and whether you want to make more virtual memory available than the RAM provides.

Some swap (at least 128 MB) is good to have to minimize the risk of losing data when active processes run out of RAM space. Swap is not required for systems with more than 1 GB of RAM. You must have at least 1 GB of virtual memory (RAM plus swap) during the install, but if the swap is more than 2 GB, you might not be able to install on some machines.

novdocx (en) 13 May 2009

 The total size should be the size you need for your system volume plus the size you

need for your swap volume.

For example, if you have a 20 GB hard drive with 2 GB of RAM and plan to install all of the OES services on the system volume, your system partition should be at least 11 GB. The remaining 9 GB should remain as free unpartitioned space that can be used for NSS volumes or other Linux partitions that you might want to create later.

4f Click OK.

The partition appears as a logical device in the devices list, such as

/dev/hda2

5 Modify the volume management type from LVM to EVMS for the second primary partition

you created in Step 4:

5a At the bottom of the page, click EVMS.

Available partitions for EVMS appear as devices under

hda2

/dev/evms

, such as

/dev/evms/

5b In the EVMS Configurator, select the LVM partition created in Step 4, then click Create

Container.

5c In the Create EVMS Container dialog box, select the partition, specify the container name

(such as

system

is the container name.

), then click Add Volume to create the

lvm/system

container, where

5d Click OK.

The EVMS Configurator displays the

lvm/system

container you just created, its size, and

free space.

18 SLES 10 SP2: Storage Administration Guide

novdocx (en) 13 May 2009

6 Create the swap volume in the

6a Select

lvm/system

, then click Add.

container:

6b In the Create Logical Volume dialog box, select Format, then select Swap from the File

System drop-down menu.

swap

6c Specify

as the volume name.

6d Specify 1 GB (recommended) for the swap volume.

The swap size should be 128 MB or larger, with a recommended size of 1 GB. For an explanation of this recommendation, see Step 4e.

6e Specify the mount point as swap.

6f Click OK.

7 Create the system volume in the

7a Select

lvm/system

, then click Add.

lvm/system

container:

7b In the Create Logical Volume dialog box, select Format, then select the file system to use

from the File System drop-down menu, such as Reiser or Ext3.

7c In the Volume Name field, specify a volume name, such as

sys_lx

7d In the Size field, click Max to set the size of the system volume as the remaining space

available in the

lvm/system

7e Specify the mount point as

partition.

(root volume).

7f Click OK.

8 Click Next to return to the list of devices.

Below is an example of the physical and logical devices that should be configured on your system. Your setup depends on the number of devices in the server and the sizes you choose for your partitions.

Device Size F Type Mount Start End Used By

/dev/hda

/dev/hda1

/dev/hda2

/dev/hdb

/dev/evms/lvm/system/ sys_lx

/dev/evms/lvm/system/ swap

149.0 GB 6Y160p0 0 19456

305.9 MB F Linux Native (Reiser)

20.0 GB Linux LVM 39 2649 EVMS

111.8 GB SP1203N 0 14595

10.0 GB F EVMS

1.0 GB F EVMS

/boot

swap

038

9 Click Next to return to the Installation Settings page.

You can dismiss the message warning that you should not mix EVMS and non-EVMS partitions on the same device.

10 Continue with the SUSE Linux Enterprise Server installation.

lvm/ system

Using EVMS to Manage Devices 19

IMPORTANT: After the install is complete, make sure to perform the mandatory post-install configuration of the related system settings to ensure that the system device functions properly under EVMS. Otherwise, the system fails to boot properly.

For information, see “After the Server Install” on page 20.

2.1.3 After the Server Install

After the SUSE Linux Enterprise Server 10 install is complete, you must perform the following tasks to ensure that the system device functions properly under EVMS:

 “Edit the /etc/fstab File” on page 20

 “Make a New initrd” on page 21

 “Disable the boot.lvm and boot.md Services” on page 21

 “Enable the boot.evms Service” on page 21

 “Restart the Server” on page 22

 “Verify the System Services” on page 22

novdocx (en) 13 May 2009

Edit the /etc/fstab File

When you boot the system, the kernel reads the

/etc/fstab

file to identify which file systems

should be mounted and then mounts them. This file contains a table of file system information about

/boot

, and

swap

the root (

The

/boot

partition is separate from the EVMS container where you placed the root (/) and

partitions plus other partitions and file systems you want to mount.

swap

partitions and is not managed by EVMS at this time. However, in the following steps, you disable

boot.lvm

partitions at boot time, including the directory. Therefore, this makes requires that the device be listed under with so it is under the

fstab /dev/sda1

SServeRA_Drive_1_600BC00000-part1

changed to include

The procedure in this section shows how to change with the device name of the device you used for your

IMPORTANT: When working in the

and

boot.md

boot.evms

. You must edit the

, then enable

/dev/evms

/boot

/etc/fstab

directory.

boot.evms

/boot

. In effect, this forces EVMS to scan all the

partition, and it activates

/boot

under the

/dev/evms

a partition that is discovered by EVMS at startup, and

/dev/evms

in the

fstab

file so it can be found when booting

file to modify the location of the

/boot

partition

, the entry for the boot device might present the boot device by the device node name (such

) or by the UUID pathname (such as

/dev/disk/by-id/scsi-

). In ether case, that name for the boot device must be

evms

in the path, such as

/dev/evms/sda1

/etc/fstab

/dev/sda1

/boot

/dev/evms/sda1

partition.

. Replace

file, do not leave any stray characters or spaces

sda1

in the file. This is a configuration file, and it is highly sensitive to such mistakes.

To modify the path of the boot device in the

1 Open the

/etc/fstab

file in a text editor.

2 Locate the line that contains the

For example, if your

/boot

partition uses device

line similar to this:

20 SLES 10 SP2: Storage Administration Guide

/boot

/etc/fstab

partition.

sda1

file, complete the following procedure:

and the Reiser file system, look for a

/dev/sda1 /boot reiser defaults 1 1

3 In the Device Name column, modify the location of the

so it can be managed by EVMS. Modify only the device name by adding

evms

/boot

partition from

path:

/dev/evms/sda1 /boot reiser defaults 1 1

4 Save the file.

The changes do not take effect until the server is restarted. Do not restart at this time.

5 Continue with “Make a New initrd” on page 21.

Make a New initrd

/dev

/evms

novdocx (en) 13 May 2009

/dev/

to the

1 Open a terminal console, and log in as the

root

user.

2 At the console prompt, enter

mkinitrd

This creates a new

initrd

file with the correct settings for the boot device. The changes do not

take effect until the server is restarted. Do not restart at this time.

3 Continue with “Disable the boot.lvm and boot.md Services” on page 21.

Disable the boot.lvm and boot.md Services

Disable the

boot.lvm

and

boot.md

services so they do not run at boot time (runlevel B). EVMS

now handles the boot.

1 In YaST, click System > System Services (Runlevel) > Expert Mode.

2 Select boot.lvm.

3 Click Set/Reset > Disable the Service.

4 Select boot.md.

5 Click Set/Reset > Disable the Service.

6 Click Finish, then click Ye s.

The changes do not take effect until the server is restarted. Do not restart at this time.

7 Continue with “Enable the boot.evms Service” on page 21.

Enable the boot.evms Service

The

boot.evms

service should be enabled automatically after the install, but you should verify that

it is enabled.

1 In YaST, click System > System Services (Runlevel) > Expert Mode.

2 Select boot.evms.

3 Click Set/Reset > Enable the Service.

The B runlevel option is automatically selected.

4 Click Finish, then click Ye s.

The changes do not take effect until the server is restarted.

Using EVMS to Manage Devices 21

novdocx (en) 13 May 2009

NOTE: Effective in SUSE Linux Enterprise 10, the nodes are automatically re-created on boot. It is no longer necessary to modify the

init.d/boot.evms

script to delete the device nodes on system restart, as was required for

/dev

directory is on

tmpfs

, and the device

/etc/

previous versions of SUSE Linux.

5 Continue with “Restart the Server” on page 22.

Restart the Server

1 Restart the server to apply the post-install configuration settings.

2 On restart, if the system device does not appear, it might be because EVMS has not been

activated. At the prompt, enter

evms_activate

Verify the System Services

After the post-install configuration is complete and you have restarted the server, make sure the server is operating as expected.

2.2 Configuring an Existing System Device to Use EVMS

If you have already installed Linux with a different volume manager for the system device (that is, the devices where you installed the

/boot, swap

configure the device for EVMS at any time after the install.

, or root (/) partitions), you can optionally

If you do not configure the device to use EVMS, you must manage the device and all of its volumes with its current volume manager (the default is LVM), and free space on the device cannot be used for volumes you want to create using EVMS. Beginning with the Linux 2.6 kernel, a given device cannot be managed by multiple volume managers. However, you can have different volume managers for different devices.

The following procedures assume that you installed Linux with three partitions on a single SCSI

sda

device named

/dev/sda1 reiserfs /boot

/dev/sda2 swap swap

/dev/sda3 reiserfs /

IMPORTANT: Make sure to modify the following procedures as necessary for your specific setup.

 Section 2.2.1, “Disable the boot.lvm and boot.md Services,” on page 23

 Section 2.2.2, “Enable the boot.evms Service,” on page 23

 Section 2.2.3, “Edit the /etc/fstab File,” on page 23

 Section 2.2.4, “Edit the Boot Loader File,” on page 24

 Section 2.2.5, “Force the RAM Disk to Recognize the Root Partition,” on page 25

 Section 2.2.6, “Restart the Server,” on page 26

 Section 2.2.7, “Verify that EVMS Manages the Boot, Swap, and Root Partitions,” on page 26

22 SLES 10 SP2: Storage Administration Guide

2.2.1 Disable the boot.lvm and boot.md Services

novdocx (en) 13 May 2009

You need to disable

boot.lvm

(handles devices for Linux Volume Manager) and

boot.md

(handles

multiple devices in software RAIDs) so they do not run at boot time. In the future, you want

boot.evms

to run at boot time instead.

1 In YaST, click System > Runlevel Editor > Expert Mode.

2 Select boot.lvm.

3 Click Set/Reset > Disable the Service.

4 Select boot.md.

5 Click Set/Reset > Disable the Service.

6 Click Finish, then click Ye s.

The changes do not take effect until the server is restarted. Do not restart at this time.

7 Continue with Section 2.2.2, “Enable the boot.evms Service,” on page 23.

2.2.2 Enable the boot.evms Service

You need to enable the

1 In YaST, click System > Runlevel Editor > Expert Mode.

2 Select boot.evms.

3 Click Set/Reset > Enable the Service.

boot.evms

service so that it boots devices when you restart the server.

The B runlevel option is automatically selected.

4 Click Finish, then click Ye s.

The changes do not take affect until the server is restarted. Do not restart at this time.

NOTE: Effective in SUSE Linux Enterprise 10, the nodes are automatically re-created on boot. It is no longer necessary to modify the

init.d/boot.evms

script to delete the device nodes on system restart as was required for

/dev

directory is on

tmpfs

and the device

/etc/

previous versions of SUSE Linux.

5 Continue with “Edit the /etc/fstab File” on page 23.

2.2.3 Edit the /etc/fstab File

When you boot the system, the kernel reads the should be mounted and then mounts them. This file contains a table of file system information about

/boot, swap

the

You must edit the mounted under the

Although the

, and root (/) partitions plus other partitions and file systems you want to mount.

/etc/fstab

/dev/evms

/boot

partition is not managed by EVMS, the

file to modify the mount location of these three partitions so they are directory. For example, change

all the partitions at boot time, including the

directory. Therefore, this makes

evms

/boot

requires that the device’s path be listed under booting with

boot.evms

/etc/fstab

/boot

partition, and it activates

file to identify which file systems

/dev/sda1

boot.evms

/dev/evms/sda1

script forces EVMS to scan

/boot

under the

/dev/

a partition that is discovered by EVMS at startup, and

/dev/evms

in the

fstab

file so it can be found when

Make sure to replace sda1, sda2, and sda3 with the device names you used for your partitions.

Using EVMS to Manage Devices 23

novdocx (en) 13 May 2009

IMPORTANT: When working in the

/etc/fstab

file, do not leave any stray characters or spaces

in the file. This is a configuration file, and it is highly sensitive to such mistakes.

1 Open the

2 Locate the line that contains the

For example, if your

/etc/fstab

/boot

file in a text editor.

/boot

partition.

partition uses device sda1 and the Reiser file system, look for a line

similar to this:

/dev/sda1 /boot reiser defaults 1 1

3 In the Device Name column, modify the mount location of the

dev/evms

so it can be managed by EVMS. Modify only the device name by adding

/boot

partition from

the path:

/dev/evms/sda1 /boot reiser defaults 1 1

4 Repeat Step 2 and Step 3 to edit the Device Name entry in the lines for the

partitions.

For example, change

sda3

/dev/sda2

/dev/evms/sda2

, and change

/dev/sda3

5 Save the file.

The changes do not take effect until the server is restarted. Do not restart at this time.

6 Continue with Section 2.2.4, “Edit the Boot Loader File,” on page 24.

swap

and root (/)

/dev

/evms

/dev/evms/

2.2.4 Edit the Boot Loader File

When you boot the system, the kernel reads the boot loader file for information about your system. For Grub, this is the

/boot/grub/menu.1st

You must edit the boot loader file to modify the mount location of partitions so they are mounted under the

/dev/evms

directory. For example, change /dev/sda1 to /dev/evms/sda1. Make sure to replace the path for all lines that contain device paths in the files. You can modify the boot loader file by editing fields in YaST, or use a text editor to modify the file directly.

IMPORTANT: When working in the boot loader file, do not leave any stray characters or spaces in the file. This is a configuration file, and it is highly sensitive to such mistakes.

Using YaST

To modify the boot loader file in the YaST Control Center:

1 Log in as the

root

user or equivalent.

2 In Yast, select System > Boot Loader.

3 Modify the boot loader image so that the root file system is mounted as

/dev/

3a Select the boot loader image file, then click Edit.

3b Edit the device path in the Root Device field.

For example, change the Root Device value from

/dev/sda2

file. For LILO, this is the

/etc/lilo.conf

/dev/evms/

file.

instead of

24 SLES 10 SP2: Storage Administration Guide

/dev/evms/sda2

Replace

sda2

with the actual device on your machine.

3c Edit any device paths in the Other Kernel Parameters field.

3d Click OK to save the changes and return to the Boot Loader page.

4 Modify the failsafe image so that the failsafe root file system is mounted as

/dev/

instead of

/dev/evms/

4a Select the failsafe image file, then click Edit.

4b Edit the device path in the Root Device field.

4c Check the Other Kernel Parameters field and make changes if needed.

4d Click OK to save the change and return to the Boot Loader page.

5 Click Finish.

6 Continue with Section 2.2.5, “Force the RAM Disk to Recognize the Root Partition,” on

page 25.

novdocx (en) 13 May 2009

Using a Text Editor

To edit the boot loader file in a text editor:

1 Log in as the

root

user or equivalent.

2 Open the boot loader file in a text editor.

For Grub, this is the

3 Locate the line that contains the

/boot/grub/menu.1st

root=

parameter.

file. For LILO, this is the

/etc/lilo.conf

For example, if your root file system uses device sda1, look for a line similar to this:

kernel (sd0,0)/vmlinuz root=/dev/sda1 vga=0x31a splash=silent showopts

4 Modify the mount location from

/dev

/dev/evms

so it can be managed by EVMS.

For example, after the change, the line looks like this:

kernel (sd0,0)/vmlinuz root=/dev/evms/sda1 vga=0x31a splash=silent showopts

5 Repeat Step 3 and Step 4 to locate other lines in the file that need to be similarly modified.

6 Save the file.

The changes do not take effect until the server is restarted. Do not restart at this time.

7 Continue with Section 2.2.5, “Force the RAM Disk to Recognize the Root Partition,” on

page 25.

file.

2.2.5 Force the RAM Disk to Recognize the Root Partition

The mkinitrd(8)

These RAM disk images are often used to preload the block device modules (SCSI or RAID) needed to access the root file system.

You might need to force the RAM to update its device node information so that it loads the root ( partition from the

command creates file system images for use as initial RAM disk (initrd) images.

/dev/evms

path.

Using EVMS to Manage Devices 25

)

novdocx (en) 13 May 2009

NOTE: Recent patches to

mkinitrd

might resolve the need to do this task. For the latest version of mkinitrd, see Recommended Updates for mkinitrd (http://support.novell.com/techcenter/psdb/

24c7dfbc3e0c183970b70c1c0b3a6d7d.html) at the Novell Technical Support Center.

root

1 At a terminal console prompt, enter the EVMS Ncurses command as the

user or

equivalent:

evmsn

2 Review the output to verify that EVMS shows only the

/boot

and

swap

partitions as active in

EVMS.

You should see the following devices mounted (with your own partition names, of course) for these two partitions:

/dev/evms/sda1

/dev/evms/sda2

3 At a terminal console prompt, enter the following to update the

path information for the root (/) partition:

evms

/sbin/mkinitrd -f evms

initrd

image with the

This does not take effect until you restart the server.

4 Continue with Section 2.2.6, “Restart the Server,” on page 26.

2.2.6 Restart the Server

/dev/

1 Restart the server to apply the post-install configuration settings.

When your system restarts, the kernel loads the

init-ramdisk

, which runs the EVMS tools to activate your volumes and mount your root file system. Then your boot scripts run the EVMS tools once more to make sure your

/dev/evms/

directory correctly reflects the current state of

your volumes. Finally, the remaining EVMS volumes are mounted as specified in your

file. Everything else on your system should start up as you would normally expect.

fstab

2 Continue with Section 2.2.7, “Verify that EVMS Manages the Boot, Swap, and Root

Partitions,” on page 26.

2.2.7 Verify that EVMS Manages the Boot, Swap, and Root Partitions

1 At a terminal prompt, enter the EVMS Ncurses command as the

evmsn

2 Review the output to verify that EVMS shows the

/boot, swap

in EVMS.

You should see the following devices mounted (with your own partition names, of course) under the

/dev/evms/sda1

/dev/evms/sda2

/dev/evms/sda3

/dev/evms

directory:

root

user or equivalent:

, and root (/) partitions as active

/etc/

26 SLES 10 SP2: Storage Administration Guide

2.3 Configuring LVM Devices to Use EVMS

Use the following post-installation procedure to configure data devices (not system devices) to be managed by EVMS. If you need to configure an existing system device for EVMS, see Section 2.2,

“Configuring an Existing System Device to Use EVMS,” on page 22.

root

1 In a terminal console, run the EVMSGUI by entering the following as the

equivalent:

evmsgui

2 In the Vo l u me s panel, review the names that EVMS reports as compatibility volumes, find the

devices that represent the devices you want to manage using EVMS, then write down the names for future reference.

For example,

/dev/sdb1

user or

novdocx (en) 13 May 2009

3 In a text editor, edit the

For example, change the following entry for an LVM2 volume from this

/dev/sdb1 / reiserfs defaults 1 2

to this

/dev/evms/lvm2/sdb1 / reiserfs defaults 1 2

IMPORTANT: Make sure not to leave any stray characters or spaces in the line.

With these changes, each time your system boots, your file system is mounted using EVMS as the volume manager.

4 Update the boot scripts as follows:

 The command

your volumes so they can be mounted.

 If you run software-RAID (

and if you are moving all devices to EVMS, remove or disable those commands.

5 If you have not already done so, enable the

For information, see “Enable the boot.evms Service” on page 21.

6 Restart your system.

/etc/fstab

evms_activate

file to use the EVMS volume names.

must be run from your boot scripts in order to activate

boot.md

) or LVM (

boot.evms

boot.lvm

service.

) boot files in your boot scripts,

2.4 Using EVMS with iSCSI Volumes

If your EVMS devices, RAIDs, and volumes use storage devices from an iSCSI SAN, make sure that your system starts iSCSI before EVMS so that the SAN and its disks are available to EVMS on system startup. iSCSI must be started and running before any disks or volumes on the iSCSI SAN can be accessed. If EVMS starts before iSCSI, EVMS cannot see or access the devices in the iSCSI SAN to mount the storage objects they contain, so the EVMS devices, RAIDs, and volumes might not be visible or accessible.

If EVMS starts before iSCSI on your system so that your EVMS devices, RAIDs, and volumes are not visible or accessible, you must correct the order in which iSCSI and EVMS are started. Enter the

chkconfig

1 At a terminal console prompt, enter either

command at the Linux server console of every server that is part of your iSCSI SAN.

chkconfig evms on

Using EVMS to Manage Devices 27

chkconfig boot.evms on

This ensures that EVMS and iSCSI start in the proper order each time your servers restart.

2.5 Using the ELILO Loader Files (IA-64)

On a SUSE Linux Enterprise Server boot device EFI System Partition, the full paths to the loader and configuration files are:

/boot/efi/SuSE/elilo.efi

/boot/efi/SuSE/elilo.conf

novdocx (en) 13 May 2009

When configuring partitioning during the install on IA64 systems, set the file system type for the

partition to

boot

vfat

, then choose Fstab Options and set the Arbitrary option value to

umask=077

to ensure that the partition is accessible only to administrators.

WARNING: Whenever you manually alter the kernel or

sbin/elilo

before shutting down the computer. If you leave out this step, your system might not be

initrd

on your system, make sure to run

bootable.

2.6 Starting EVMS

If EVMS does not start during the system boot, you must activate it manually.

1 Open a terminal console, then log in as the

root

user or equivalent.

2 At the terminal console prompt, enter

evms_activate

2.7 Starting the EVMS Management Tools

Use the following procedure to start the EVMS management tools.

IMPORTANT: When you are done, make sure to exit the EVMS UI tool. When it is running, the EVMS UI tool locks the EVMS engine, potentially blocking other EVMS actions from taking place.

1 Open a terminal console, then log in as the

2 Enter one of the following commands to open the desired EVMS UI:

Command Description

evmsgui

evmsn

28 SLES 10 SP2: Storage Administration Guide

Starts the graphical interface for EVMS GUI. For information about features in this interface, see ”EVMS GUI” (http://evms.sourceforge.net/user_guide/#GUI) in the EVMS User Guide at the EVMS project on SourceForge.net.

Starts the text-mode interface for EVMS Ncurses. For information about features in this interface, see the “EVMS Ncurses Interface” (http://

evms.sourceforge.net/user_guide/#NCURSES) in the EVMS User Guide at the

EVMS project on SourceForge.net.

root

user or equivalent.

Command Description

novdocx (en) 13 May 2009

evms

To stop

1 Close

evmsgui

Starts the EVMS commandline interpreter (CLI) interface. For information about command options, see “EVMS Command Line Interpreter” (http://

evms.sourceforge.net/user_guide/#COMMANDLINE) in the EVMS User Guide

at the EVMS project on SourceForge.net.

from running automatically on restart:

2 Do a clean shutdown (not a restart).

3 Start the server.

When the server comes back up,

evmsgui

is not automatically loaded on restart.

Using EVMS to Manage Devices 29

novdocx (en) 13 May 2009

30 SLES 10 SP2: Storage Administration Guide

Using UUIDs to Mount Devices

This section describes the optional use of UUIDs instead of device names to identify file system devices in the boot loader file and the

 Section 3.1, “Naming Devices with udev,” on page 31

 Section 3.2, “Understanding UUIDs,” on page 31

 Section 3.3, “Using UUIDs in the Boot Loader and /etc/fstab File (x86),” on page 32

 Section 3.4, “Using UUIDs in the Boot Loader and /etc/fstab File (IA64),” on page 33

 Section 3.5, “Additional Information,” on page 34

/etc/fstab

file.

3.1 Naming Devices with udev

novdocx (en) 13 May 2009

In the Linux 2.6 and later kernel, with persistent device naming. As part of the hotplug system, or removed from the system.

A list of rules is used to match against specific device attributes. The (defined in the regardless of their order of recognition or the connection used for the device. The examine every appropriate block device that the kernel creates to apply naming rules based on certain buses, drive types, or file systems. For information about how to define your own rules for

udev

, see Writing udev Rules (http://reactivated.net/writing_udev_rules.html).

Along with the dynamic kernel-provided device node name, symbolic links pointing to the device in the

by-id, by-label, by-path

the

NOTE: Other programs besides not listed in

/etc/udev/rules.d

/dev/disk

udev

provides a userspace solution for the dynamic

udev

is executed if a device is added

udev

rules infrastructure

directory) provides stable names for all disk devices,

udev

maintains classes of persistent

/dev/disk

, and

by-uuid

udev

, such as LVM or md, might also generate UUIDs, but they are

subdirectories.

directory, which is further categorized by

/dev

udev

directory,

tools

3.2 Understanding UUIDs

A UUID (Universally Unique Identifier) is a 128-bit number for a file system that is unique on both the local system and across other systems. It is a randomly generated with system hardware information and time stamps as part of its seed. UUIDs are commonly used to uniquely tag devices.

 Section 3.2.1, “Using UUIDs to Assemble or Activate File System Devices,” on page 32

 Section 3.2.2, “Finding the UUID for a File System Device,” on page 32

Using UUIDs to Mount Devices

3.2.1 Using UUIDs to Assemble or Activate File System Devices

The UUID is always unique to the partition and does not depend on the order in which it appears or where it is mounted. With certain SAN devices attached to the server, the system partitions are

renamed and moved to be the last device. For example, if root ( the install, it might be assigned to problem is to use the UUID in the boot loader and

The device ID assigned by the manufactuer for a drive never changes, no matter where the device is mounted, so it can always be found at boot. The UUID is a property of the filesystem and can change if you reformat the drive. In a boot loader file, you typically specify the location of the device (such as also mount devices by their UUIDs and administrator-specified volume labels. However, if you use a label and file location, you cannot change the label name when the partition is mounted.

You can use the UUID as criterion for assembling and activating software RAID devices. When a RAID is created, the superblock.

/dev/sda1

driver generates a UUID for the device, and stores the value in the md

/dev/sdg1

/dev/evms/sda1

after the SAN is connected. One way to avoid this

/etc/fstab

) to mount it at system boot. The boot loader can

) is assigned to

files for the boot device.

/dev/sda1

during

novdocx (en) 13 May 2009

3.2.2 Finding the UUID for a File System Device

You can find the UUID for any block device in the UUID looks like this:

e014e482-1c2d-4d09-84ec-61b3aefde77a

/dev/disk/by-uuid

directory. For example, a

3.3 Using UUIDs in the Boot Loader and /etc/ fstab File (x86)

After the install, you can optionally use the following procedure to configure the UUID for the system device in the boot loader and

1 Install the SUSE

2 After the install, boot the system.

3 Open a terminal console as the

4 Navigate to the

installed

4a At the terminal console prompt, enter

4b List all partitions by entering

4c Find the UUID, such as

5 Edit

/boot, /root

cd /dev/disk/by-uuid

e014e482-1c2d-4d09-84ec-61b3aefde77a —> /dev/sda1

/boot/grub/menu.1st

Linux Enterprise Server for x86 with no SAN devices connected.

/dev/disk/by-uuid

, and

swap

file, using the Boot Loader option in YaST2 or using a text editor.

/etc/fstab

root

user or equivalent.

directory to find the UUID for the device where you

files for your x86 system.

For example, change

kernel /boot/vmlinuz root=/dev/sda1

32 SLES 10 SP2: Storage Administration Guide

kernel /boot/vmlinuz root=/dev/disk/by-uuid/e014e482-1c2d-4d09-84ec61b3aefde77a

IMPORTANT: Make a copy of the original boot entry, then modify the copy. If you make a mistake, you can boot the server without the SAN connected, and fix the error.

If you use the Boot Loader option in YaST, there is a defect where it adds some duplicate lines to the boot loader file when you change a value. Use an editor to remove the following duplicate lines:

color white/blue black/light-gray

default 0

timeout 8

gfxmenu (sd0,1)/boot/message

When you use YaST to change the way that the root (/) device is mounted (such as by UUID or by label), the boot loader configuration needs to be saved again to make the change effective for the boot loader.

6 As the

user or equivalent, do one of the following to place the UUID in the

/etc/fstab

root

file:

 Open YaST to System > Partitioner, select the device of interest, then modify Fstab

Options.

 Edit the

For example, if the root (

e014e482-1c2d-4d09-84ec-61b3aefde77a

/dev/sda1 / reiserfs acl,user_xattr 1 1

/etc/fstab

file to modify the system device from the location to the UUID.

) volume has a device path of

/dev/sda1

and its UUID is

, change line entry from

UUID=e014e482-1c2d-4d09-84ec-61b3aefde77a / reiserfs acl,user_xattr 1 1

novdocx (en) 13 May 2009

IMPORTANT: Make sure to make a backup copy of the

fstab

file before you begin, and

do not leave stray characters or spaces in the file.

3.4 Using UUIDs in the Boot Loader and /etc/ fstab File (IA64)

After the install, use the following procedure to configure the UUID for the system device in the boot loader and configuration file is

/etc/fstab

/boot/efi/SuSE/elilo.conf

1 Install the SUSE Linux Enterprise Server for IA64 with no SAN devices connected.

2 After the install, boot the system.

3 Open a terminal console as the

4 Navigate to the

installed

/dev/disk/by-uuid

/boot, /root

4a At the terminal console prompt, enter

cd /dev/disk/by-uuid

files for your IA64 system. IA64 uses the EFI BIOS. Its file system

root

user or equivalent.

instead of

/etc/fstab

directory to find the UUID for the device where you

, and

swap

Using UUIDs to Mount Devices 33

4b List all partitions by entering

4c Find the UUID, such as

e014e482-1c2d-4d09-84ec-61b3aefde77a —> /dev/sda1

5 Edit the boot loader file, using the Boot Loader option in YaST2.

For example, change

root=/dev/sda1

root=/dev/disk/by-uuid/e014e482-1c2d-4d09-84ec-61b3aefde77a

6 Edit the

/boot/efi/SuSE/elilo.conf

file to modify the system device from the location to

the UUID.

For example, change

/dev/sda1 / reiserfs acl,user_xattr 1 1

UUID=e014e482-1c2d-4d09-84ec-61b3aefde77a / reiserfs acl,user_xattr 1 1

novdocx (en) 13 May 2009

IMPORTANT: Make sure to make a backup copy of the

/boot/efi/SuSE/elilo.conf

before you begin, and do not leave stray characters or spaces in the file.

3.5 Additional Information

For more information about using

Management with udev” (http://www.novell.com/documentation/sles10/sles_admin/data/ cha_udev.html) in the SUSE

For more information about console prompt:

man 8 udev

udev(8)

Linux Enterprise Server 10 Installation and Administration Guide.

udev(8)

for managing devices, see “Dynamic Kernel Device

commands, see its man page. Enter the following at a terminal

file

34 SLES 10 SP2: Storage Administration Guide

Managing EVMS Devices

This section describes how to initialize a disk for EVMS management by adding a segment management container to manage the partitions that you later add to the disk.

 Section 4.1, “Understanding Disk Segmentation,” on page 35

 Section 4.2, “Initializing Disks,” on page 36

 Section 4.3, “Removing the Segment Manager from a Device,” on page 38

 Section 4.4, “Creating Disk Segments (or Partitions),” on page 38

 Section 4.5, “Configuring Mount Options for Devices,” on page 39

 Section 4.6, “What’s Next,” on page 41

4.1 Understanding Disk Segmentation

In EVMS, you initialize a disk by assigning a segment manager to it. The segment manager creates metadata for the disk and exposes its free space so you can subdivide it into one or multiple segments (also called partitions).

novdocx (en) 13 May 2009

 Section 4.1.1, “Segment Managers,” on page 35

 Section 4.1.2, “Disk Segments,” on page 36

4.1.1 Segment Managers

The most commonly used segment manager is the DOS Segment Manager. The following table describes the segment managers available in EVMS.

Table 4-1 EVMS Segment Managers

Segment Manager Description

DOS The standard MS-DOS* disk partitioning scheme. It is the most commonly used

partitioning scheme for Linux, NetWare and UnixWare*.

GPT (Globally Unique Identifier (GUID) Partitioning Table)

S/390 A partitioning scheme used exclusively for System/390 mainframes.

A partitioning scheme used for IA-64 platforms, as defined in the Intel* Extensible Firmware Interface (EIF) Specification. It is not compatible with DOS, Windows, or OS/2 systems.

The GUID is also known as Universally Unique Identifier (UUID). The GPT combines time and space descriptors to create this unique 128-bit tag for the disk and its segments.

, Windows*, OS/2*, BSD, Solaris* X86,

Cluster A partitioning scheme for high-availability clusters. It provides a GUID for the

disk, creates an EVMS container for the shared cluster devices, and specifies a node ID for the node that owns the device and the cluster ID.

BSD A partitioning scheme for BSD UNIX*.

Managing EVMS Devices

Segment Manager Description

MAC A partitioning scheme for Mac-OS partitions.

4.1.2 Disk Segments

After you initialize the disk by adding a segment manager, you see metadata and free space segments on the disk. You can then create one or multiple data segments in a disk segment.

Table 4-2 Disk Segment Types

Segment Type Description

Metadata A set of contiguous sectors that contain information needed by the segment

manager.

Free Space A set of contiguous sectors that are unallocated or not in use. Free space can

be used to create a segment.

novdocx (en) 13 May 2009

Data A set of contiguous sectors that has been allocated from a disk. The segment

might be in use for a volume or a software RAID.

4.2 Initializing Disks

You must initialize new disks and disks that you want to reformat. After the disk is initialized, you can subdivide, or carve, the device into one or more disk segments for your file systems.

 Section 4.2.1, “Before You Begin,” on page 36

 Section 4.2.2, “Guidelines,” on page 37

 Section 4.2.3, “Adding a Segment Manager,” on page 37

4.2.1 Before You Begin

If you use large disks or disk arrays, use the vendor’s tools to carve them into the sizes that are

usable for the management tools you plan to use. For example, the

up to 2 TB in size, so the limit also applies to the

plug-in for EVMS. Software RAID devices you create with EVMS can be larger than 2 TB, of course, because the disks underneath that storage structure.

When you boot the server, EVMS scans and recognizes all devices it manages. If you add a new

mkfs

device to the server or create a device using

dev/evms

as a compatibility volume, such as

, EVMS automatically mounts it on reboot under

/dev/evms/sdb

driver recognizes disks only

driver plug-in manages the

IMPORTANT: If you cannot find a new disk, device, or volume, look under browser, or look for compatibility volumes in the Volumes Manager in the EVMS GUI (

36 SLES 10 SP2: Storage Administration Guide

/dev/evms

in a file

evmsgui

4.2.2 Guidelines

Consider the following guidelines when initializing a disk:

 EVMS might allow you to create segments without first adding a segment manager for the

disk, but it is best to explicitly add a segment manager to avoid problems later.

IMPORTANT: You must add a Cluster segment manager if you plan to use the devices for volumes that you want to share as cluster resources.

 When you initialize an existing disk that is already formatted, the process of adding a segment

manager destroys all data on the disk. If you want to keep the data on the disk, make sure to back up the data before you begin this process.

 For existing disks on the system or disks that you move from another system, you must delete

any existing volume management structures, and remove any segment managers. This removes the device’s metadata and data, and destroys all data on the disk.

WARNING: Do not initialize the device that contains your current system disk or any device that contains the

/boot, swap

, or root (/) volumes.

novdocx (en) 13 May 2009

 If a new disk does not show up in the list of Available Objects, look for it in the Vo lu m e s list to

sdb

see if the disk shows up as a compatibility volume. For example, a new disk

/dev/evms/sdb

up as Objects, then create segments as desired.

. Delete it from the Volumes list to force the disk to show up in Available

would show

4.2.3 Adding a Segment Manager

Use the following procedure to assign a segment manager to device for servers using x86, x64, and IA64 controllers. This option is not available for S390 platforms, so simply continue with configuring software RAIDs or file system partitions, as desired.

WARNING: Adding a segment manager initializes the disk, completely removing all the segments it contains. All the data stored on the device is lost.

1 If the disk has any existing volume management structures or an existing segment manager,

remove them.

1a Select Actions > Delete > Volume to view the Volumes list.

1b Select any existing volume management structures on the device, then click Delete.

1c Select Actions > Remove > Segment Manager from Storage Object.

1d Select the type of Segment Manager in use, then click Next.

1e Select the device, then click Remove.

2 If the disk is a new one that is listed as a compatibility volume in the Volumes list, delete it as a

compatibility volume.

2a Select Actions > Delete > Volume to view the Volumes list.

2b Select the device, then click Delete.

3 Add the Segment Manager.

3a In the list of Availability Objects, select the device, then click Actions > Add > Segment

Manager to Storage Object.

Managing EVMS Devices 37

3b From the list, select one of the following types of segment manager, then click Next.

 DOS Segment Manager (the most common choice)

 GPT Segment Manager (for IA-64 platforms)

 Cluster Segment Manager (available only if it is a viable option for the selected disk)

3c Select the device from the list of Plugin Acceptable Objects, then click Next.

3d If required, specify the disk type as Linux.

3e Click Add to create the segment management container for the disk, then click OK to

dismiss the confirmation message.

4.3 Removing the Segment Manager from a Device

1 If the disk has any existing volume management structures, remove them.

1a Select Actions > Delete > Volume to view the Volumes list.

1b Select any existing volume management structures on the device, then click Delete.

2 Select Actions > Remove > Segment Manager from Storage Object.

3 Select the type of Segment Manager in use, then click Next.

novdocx (en) 13 May 2009

4 Select the device, then click Remove.

4.4 Creating Disk Segments (or Partitions)

1 In EVMS, select Actions > Create > Segment to see a list of segment managers.

2 From the list, select the segment manager for the device you want to manage, then click Next.

 DOS Segment Manager (the most common choice)

 GPT Segment Manager (for IA-64 platforms)

 Cluster Segment Manager (available only if it is a viable option for the selected disk)

For information about these and other segment managers available, see “Segment Managers”

on page 35.

3 Select the storage object that you want to segment, then click Next.

4 Complete the required configuration options for the segment, and modify default values as

desired.

 Size (MB): Specify the amount of space (in MB) that you want to use. Use the arrows or

type a value. The interface corrects the value to the lower or upper size limit if you specify a size that is too small or that exceeds the amount of free space available.

 Offset (Sectors): Specify the number of sectors to skip before beginning this partition if

you want to leave free space in front of it.

 Partition Type: From the drop-down list, select Linux (default), Linux Swap, Linux LVM,

NTFS, HPFS, FA T 1 6 , or Other Partition Type.

 Partition Type ID: This value changes automatically based on the Partition Type value,

except for the Other Partition Type option, where you must manually enter a value.

 Bootable: Click Ye s to make a primary partition active so that you can boot from it, or

click No to make it unbootable. No is the only option if you are creating a logical partition.

38 SLES 10 SP2: Storage Administration Guide

 Primary Partition: Click Ye s for a primary partition, or click No for a logical partition.

Required settings are denoted in the page by an asterisk (*). All required fields must be completed to make the Create button active.

5 Click Create to create the segment.

6 Verify that the new segment appears in the Segment list.

4.5 Configuring Mount Options for Devices

The following table describes the Fstab Options that are configurable in YaST. The values are written to the

Table 4-3 Fstab Options in YaST

Fstab Option Description

Mount by  Device name (K) (default, such as /dev/sda2)

/etc/fstab

file and are applied upon reboot.

 Volume label (L)

Make sure to also specify a value in Volume Label.

 UUID (U)

For information about why you might want to discover partitions and devices by UUID, see Section 3.2.1, “Using UUIDs to Assemble or

Activate File System Devices,” on page 32.

 Device ID (I)

 Device Path (P)

novdocx (en) 13 May 2009

Volume label A useful name to help you easily identify the volume on the server. By default,

this field is empty.

Mount read-only Select the check box to enable this option. It is deselected (disabled) by default.

If this option is enabled, files and directories cannot be modified or saved on the volume.

No access time Select the check box to enable this option. It is deselected (disabled) by default.

By default, the Linux file is opened. The No Access Time option disables the updating of access time, so that reading a file does not update its access time. Enabling the No Access Time option allows you to back up a volume without modifying the access times of its files.

Mountable by user Select the check box to enable this option. It is deselected (disabled) by default.

If this option is enabled, the volume can be mounted by any user; privileges are not required.

Do Not Mount at System Start-up

Select the check box to enable this option. It is deselected (disabled) by default.

The system volumes such as at system start. For other volumes, enable this option for a volume if you want to mount it manually later using the mount command at a terminal console prompt.

open(2)

command updates the access time whenever a

root

/boot, swap

, and root (/) should all be mounted

Managing EVMS Devices 39

Fstab Option Description

Data journaling mode For journaling file systems, select the preferred journaling mode:

 Ordered: Writes data to the file system, then enters the metadata in the

journal. This is the default.

 Journal: Writes data twice; once to the journal, then to the file system.

 Writeback: Writes data to the file system and writes metadata in the

journal, but the writes are performed in any order.

novdocx (en) 13 May 2009

Access Control LIsts (ACL)

Extended user attributes

Arbitrary option value Specify any mount option that is legal for the Mount Options column for a

You can modify these values for each entry by editing the procedure to modify the mount options for a volume in the

Select this option to enable access control lists on the file system. It is enabled by default.

Select this option to enable extended user attributes on the file system. It is enabled by default.

device entry in the multiple options.

/etc/fstab

file. Use a comma with no spaces to separate

/etc/fstab

file, or use the following

file from YaST.

1 Open YaST, then click System > Partitioning.

2 Select the device you want to modify, then click Fstab Options.

3 Modify the settings as desired, then click OK to accept your changes.

40 SLES 10 SP2: Storage Administration Guide

4.6 What’s Next

If multiple paths exist between your host bus adapters (HBAs) and the storage devices, configure multipathing for the devices before creating software RAIDs or file system volumes on the devices. For information, see Chapter 5, “Managing Multipath I/O for Devices,” on page 43.

If you want to configure software RAIDs, do it before you create file systems on the devices. For information, see Chapter 6, “Managing Software RAIDs with EVMS,” on page 75.

novdocx (en) 13 May 2009

Managing EVMS Devices 41

novdocx (en) 13 May 2009

42 SLES 10 SP2: Storage Administration Guide

Managing Multipath I/O for

novdocx (en) 13 May 2009

Devices

This section describes how to manage failover and path load balancing for multiple paths between the servers and block storage devices.

 Section 5.1, “Understanding Multipathing,” on page 43

 Section 5.2, “Planning for Multipathing,” on page 44

 Section 5.3, “Multipath Management Tools,” on page 49

 Section 5.4, “Configuring the System for Multipathing,” on page 53

 Section 5.5, “Enabling and Starting Multipath I/O Services,” on page 59

 Section 5.6, “Configuring Path Failover Policies and Priorities,” on page 59

 Section 5.7, “Tuning the Failover for Specific Host Bus Adapters,” on page 67

 Section 5.8, “Configuring Multipath I/O for the Root Device,” on page 67

 Section 5.9, “Configuring Multipath I/O for an Existing Software RAID,” on page 68

 Section 5.10, “Scanning for New Devices without Rebooting,” on page 69

 Section 5.11, “Scanning for New Partitioned Devices without Rebooting,” on page 70

 Section 5.12, “Viewing Multipath I/O Status,” on page 71

 Section 5.13, “Managing I/O in Error Situations,” on page 72

 Section 5.14, “Resolving Stalled I/O,” on page 73

 Section 5.15, “Additional Information,” on page 73

 Section 5.16, “What’s Next,” on page 74

5.1 Understanding Multipathing

 Section 5.1.1, “What Is Multipathing?,” on page 43

 Section 5.1.2, “Benefits of Multipathing,” on page 43

5.1.1 What Is Multipathing?

Multipathing is the ability of a server to communicate with the same physical or logical block storage device across multiple physical paths between the host bus adapters in the server and the storage controllers for the device, typically in Fibre Channel (FC) or iSCSI SAN environments. You can also achieve multiple connections with direct attached storage when multiple channels are available.

5.1.2 Benefits of Multipathing

Linux multipathing provides connection fault tolerance and can provide load balancing across the active connections. When multipathing is configured and running, it automatically isolates and identifies device connection failures, and reroutes I/O to alternate connections.

Managing Multipath I/O for Devices

Typical connection problems involve faulty adapters, cables, or controllers. When you configure multipath I/O for a device, the multipath driver monitors the active connection between devices. When the multipath driver detects I/O errors for an active path, it fails over the traffic to the device’s designated secondary path. When the preferred path becomes healthy again, control can be returned to the preferred path.

5.2 Planning for Multipathing

 Section 5.2.1, “Guidelines for Multipathing,” on page 44

 Section 5.2.2, “Using Multipathed Devices Directly or in EVMS,” on page 45

 Section 5.2.3, “Using LVM2 on Multipath Devices,” on page 46

 Section 5.2.4, “Using mdadm with Multipath Devices,” on page 46

 Section 5.2.5, “Partitioning Multipath Devices,” on page 46

 Section 5.2.6, “Supported Architectures for Multipath I/O,” on page 47

 Section 5.2.7, “Supported Storage Arrays for Multipathing,” on page 47

novdocx (en) 13 May 2009

5.2.1 Guidelines for Multipathing

Use the guidelines in this section when planning your multipath I/O solution.

 “Prerequisites” on page 44

 “Vendor-Provided Multipath Solutions” on page 44

 “Disk Management Tasks” on page 45

 “Software RAIDs” on page 45

 “High-Availability Solutions” on page 45

 “Volume Managers” on page 45

 “Virtualization Environments” on page 45

Prerequisites

 Multipathing is managed at the device level.

 The storage array you use for the multipathed device must support multipathing. For more

information, see Section 5.2.7, “Supported Storage Arrays for Multipathing,” on page 47.

 You need to configure multipathing only if multiple physical paths exist between host bus

adapters in the server and host bus controllers for the block storage device. You configure multipath for the logical device as seen by the server.

Vendor-Provided Multipath Solutions

For some storage arrays, the vendor will provide its own multipathing software to manage multipathing for the array’s physical and logical devices. In this case, you should follow the vendor’s instructions for configuring multipathing for those devices.

44 SLES 10 SP2: Storage Administration Guide

Disk Management Tasks

Perform the following disk management tasks before you attempt to configure multipathing for a physical or logical device that has multiple paths:

 Use third-party tools to carve physical disks into smaller logical disks.

 Use third-party tools to partition physical or logical disks. If you change the partitioning in the

running system, the Device Mapper Multipath (DM-MP) module does not automatically detect and reflect these changes. DM-MP must be reinitialized, which usually requires a reboot.

 Use third-party SAN array management tools to create and configure hardware RAID devices.

 Use third-party SAN array management tools to create logical devices such as LUNs. Logical

device types that are supported for a given array depend on the array vendor.

Software RAIDs

The Linux software RAID management software runs on top of multipathing. For each device that has multiple I/O paths and that you plan to use in a software RAID, you must configure the device for multipathing before you attempt to create the software RAID device. Automatic discovery of multipathed devices is not available. The software RAID is not aware of the multipathing management running underneath.

novdocx (en) 13 May 2009

High-Availability Solutions

High-availability solutions for clustering typically run on top of the multipathing server. For example, the Distributed Replicated Block Device (DRBD) high-availability solution for mirroring devices across a LAN runs on top of multipathing. For each device that has multiple I/O paths and that you plan to use in a DRDB solution, you must configure the device for multipathing before you configure DRBD.

Volume Managers

Volume managers such as LVM2 and EVMS run on top of multipathing. You must configure multipathing for a device before you use LVM2 or EVMS to create segment managers and file systems on it.

Virtualization Environments

When using multipathing in a virtualization environment, the multipathing is controlled in the host server environment. Configure multipathing for the device before you assign it to a virtual guest machine.

5.2.2 Using Multipathed Devices Directly or in EVMS

If you want to use the entire LUN directly (for example, if you are using the SAN features to partition your storage), you can simply use the

fstab

, your application, etc.

/dev/disk/by-id/xxx

names directly for

mkfs

If the user-friendly names option is enabled in the use the

/dev/mapper/mpathn

device name because this name is aliased to the devices ID. For

/etc/multipath.conf

file, you can optionally

information, see “Configuring User-Friendly Names or Alias Names in /etc/multipath.conf” on

page 57.

Managing Multipath I/O for Devices 45

5.2.3 Using LVM2 on Multipath Devices

By default, LVM2 does not recognize multipathed devices. To make LVM2 recognize the multipathed devices as possible physical volumes, you must modify

/etc/lvm/lvm.conf

important to modify it in a way that it does not scan and use the physical paths, but only accesses the multipath I/O storage through the multipath I/O layer.

. It is

novdocx (en) 13 May 2009

To modi fy

1 Open the

2 Change the

/etc/lvm/lvm.conf

filter

filter = [ "a|/dev/disk/by-id/.*|", "r|.*|" ]

and

for multipath use:

file in a text editor.

types

entry in

/etc/lvm/lvm.conf

as follows:

This allows LVM2 to scan only the by-id paths and reject everything else.

3 If you are also using LVM2 on non-multipathed devices, make the necessary adjustments to

suit your setup.

4 Save the file.

5 Add dm-multipath to

6 Make a new

initrd

/etc/sysconfig/kernel:INITRD_MODULES

to ensure that the Device Mapper Multipath services are loaded with the

changed settings. Enter

mkinitrd -f mpath

7 Reboot the server to apply the changes.

5.2.4 Using mdadm with Multipath Devices

The

mdadm

tool requires that the devices be accessed by the ID rather than by the device node path.

Therefore, the

DEVICE /dev/disk/by-id/*

DEVICE

entry in

/etc/mdadm.conf

should be set as follows:

This is the default handling in SUSE Linux Enterprise Server 10 and later.

5.2.5 Partitioning Multipath Devices

SUSE Linux Enterprise Server 10

In SUSE Linux Enterprise Server 10, the

boot.multipath

to add symlinks to the

kpartx

/dev/dm-*

for any newly created partitions without requiring a reboot. This triggers

disk/by-*

symlinks. The main benefit is that you can call

having to reboot the server.

SUSE Linux Enterprise Server 9

In SUSE Linux Enterprise Server 9, it is not possible to partition multipath I/O devices themselves. If the underlying physical device is already partitioned, the multipath I/O device reflects those partitions and the layer provides

/dev/disk/by-id/<name>p1 ... pN

the partitions through the multipath I/O layer. As a consequence, the devices need to be partitioned

software is used in the

line in the

multipath.conf

kpartx

with the new parameters without

/etc/init.d/

configuration file

udevd

to fill in the

/dev/

devices so you can access

46 SLES 10 SP2: Storage Administration Guide

prior to enabling multipath I/O. If you change the partitioning in the running system, DM-MP does not automatically detect and reflect these changes. The device must be reinitialized, which usually requires a reboot.

5.2.6 Supported Architectures for Multipath I/O

The multipathing drivers and tools in SUSE® Linux Enterprise Server 10 support all seven of the supported processor architectures: IA32, AMD64/EM64T, IPF/IA64, p-Series (32-bit/64-bit), zSeries (31-bit and 64-bit).

5.2.7 Supported Storage Arrays for Multipathing

The multipathing drivers and tools in SUSE Linux Enterprise Server 10 support most storage arrays. The storage array that houses the multipathed device must support multipathing in order to use the multipathing drivers and tools. Some storage array vendors provide their own multipathing management tools. Consult the vendor’s hardware documentation to determine what settings are required.

 “Storage ArraysThat Are Automatically Detected for Multipathing” on page 47

novdocx (en) 13 May 2009

 “Tested Storage Arrays for Multipathing Support” on page 48

 “Storage Arrays that Require Specific Hardware Handlers” on page 48

Storage ArraysThat Are Automatically Detected for Multipathing

The

multipath-tools

3PARdata VV Compaq* HSV110 Compaq MSA1000 DDN SAN MultiDirector DEC* HSG80 EMC* CLARiiON* CX EMC Symmetrix FSC CentricStor* Hewlett Packard* (HP*) A6189A HP HSV110 HP HSV210 HP OpenHitachi* DF400 Hitachi DF500 Hitachi DF600 IBM* 3542 IBM ProFibre 4000R NetApp* SGI* TP9100 SGI TP9300 SGI TP9400 SGI TP9500

package automatically detects the following storage arrays:

Managing Multipath I/O for Devices 47

STK OPENstorage DS280 Sun* StorEdge 3510 Sun T4

In general, most other storage arrays should work. When storage arrays are automatically detected, the default settings for multipathing apply. If you want non-default settings, you must manually create and configure the

/etc/multipath.conf

file. For information, see Section 5.4.5, “Creating

and Configuring the /etc/multipath.conf File,” on page 55.

Hardware that is not automatically detected requires an appropriate entry for configuration in the

DEVICES

section of the

/etc/multipath.conf

file. In this case, you must manually create and configure the configuration file. For information, see Section 5.4.5, “Creating and Configuring the /

etc/multipath.conf File,” on page 55.

Consider the following caveats:

 Not all of the storage arrays that are automatically detected have been tested on SUSE Linux

Enterprise Server. For information, see “Tested Storage Arrays for Multipathing Support” on

page 48.

 Some storage arrays might require specific hardware handlers. A hardware handler is a kernel

module that performs hardware-specific actions when switching path groups and dealing with I/O errors. For information, see “Storage Arrays that Require Specific Hardware Handlers” on

page 48.

 After you modify the

/etc/multipath.conf

file, you must run

mkinitrd

to re-create the

INITRD on your system, then reboot in order for the changes to take effect.

novdocx (en) 13 May 2009

Tested Storage Arrays for Multipathing Support

The following storage arrays have been tested with SUSE Linux Enterprise Server:

EMC Hitachi Hewlett-Packard/Compaq IBM NetApp SGI

Most other vendors’ storage arrays should also work. Consult your vendor’s documentation for guidance. For a list of the default storage arrays recognized by the

multipath-tools

package, see

“Storage ArraysThat Are Automatically Detected for Multipathing” on page 47.

Storage Arrays that Require Specific Hardware Handlers

Storage arrays that require special commands on failover from one path to the other or that require special nonstandard error handling might require more extensive support. Therefore, the Device Mapper Multipath service has hooks for hardware handlers. For example, one such handler for the EMC CLARiiON CX family of arrays is already provided.

IMPORTANT: Consult the hardware vendor’s documentation to determine if its hardware handler must be installed for Device Mapper Multipath.

48 SLES 10 SP2: Storage Administration Guide

The

multipath -t

command shows an internal table of storage arrays that require special handling with specific hardware handlers. The displayed list is not an exhaustive list of supported storage arrays. It lists only those arrays that require special handling and that the

multipath-tools

developers had access to during the tool development.

IMPORTANT: Arrays with true active/active multipath support do not require special handling, so they are not listed for the

multipath -t

command.

novdocx (en) 13 May 2009

A listing in the

multipath -t

table does not necessarily mean that SUSE Linux Enterprise Server was tested on that specific hardware. For a list of tested storage arrays, see “Tested Storage Arrays

for Multipathing Support” on page 48.

5.3 Multipath Management Tools

The multipathing support in SUSE Linux Enterprise Server 10 is based on the Device Mapper Multipath module of the Linux 2.6 kernel and the

mdadm

use

 Section 5.3.1, “Device Mapper Multipath Module,” on page 49

 Section 5.3.2, “Multipath I/O Management Tools,” on page 51

 Section 5.3.3, “Using mdadm for Multipathed Devices,” on page 51

 Section 5.3.4, “The Linux multipath(8) Command,” on page 52

to view the status of multipathed devices.

multipath-tools

5.3.1 Device Mapper Multipath Module

The Device Mapper Multipath (DM-MP) module provides the multipathing capability for Linux. DM-MP is the preferred solution for multipathing on SUSE Linux Enterprise Server 10. It is the only multipathing option shipped with the product that is completely supported by Novell SUSE.

DM-MP features automatic configuration of the multipathing subsystem for a large variety of setups. Configurations of up to 8 paths to each device are supported. Configurations are supported for active/passive (one path active, others passive) or active/active (all paths active with round-robin load balancing).

userspace package. You can

and

The DM-MP framework is extensible in two ways:

 Using specific hardware handlers. For information, see “Storage Arrays that Require Specific

Hardware Handlers” on page 48.

 Using more sophisticated load-balancing algorithms than round-robin

The user-space component of DM-MP takes care of automatic path discovery and grouping, as well as automated path retesting, so that a previously failed path is automatically reinstated when it becomes healthy again. This minimizes the need for administrator attention in a production environment.

DM-MP protects against failures in the paths to the device, and not failures in the device itself. If one of the active paths is lost (for example, a network adapter breaks or a fiber-optic cable is removed), I/O is redirected to the remaining paths. If the configuration is active/passive, then the path fails over to one of the passive paths. If you are using the round-robin load-balancing

Managing Multipath I/O for Devices 49

configuration, the traffic is balanced across the remaining healthy paths. If all active paths fail, inactive secondary paths must be waked up, so failover occurs with a delay of approximately 30 seconds.

If a disk array has more than one storage processor, make sure that the SAN switch has a connection to the storage processor that owns the LUNs you want to access. On most disk arrays, all LUNs belong to both storage processors, so both connections are active.

NOTE: On some disk arrays, the storage array manages the traffic through storage processors so that it presents only one storage processor at a time. One processor is active and the other one is passive until there is a failure. If you are connected to the wrong storage processor (the one with the passive path) you might not see the expected LUNs, or you might see the LUNs but get errors when trying to access them.

Table 5-1 Multipath I/O Features of Storage Arrays

Features of Storage Arrays Description

Active/passive controllers One controller is active and serves all LUNs. The second controller acts as

a standby. The second controller also presents the LUNs to the multipath component so that the operating system knows about redundant paths. If the primary controller fails, the second controller takes over, and it serves all LUNs.

novdocx (en) 13 May 2009

In some arrays, the LUNs can be assigned to different controllers. A given LUN is assigned to one controller to be its active controller. One controller does the disk I/O for any given LUN at a time, and the second controller is the standby for that LUN. The second controller also presents the paths, but disk I/O is not possible. Servers that use that LUN are connected to the LUN’s assigned controller. If the primary controller for a set of LUNs fails, the second controller takes over, and it serves all LUNs.

Active/active controllers Both controllers share the load for all LUNs, and can process disk I/O for

any given LUN. If one controller fails, the second controller automatically handles all traffic.

Load balancing The Device Mapper Multipath driver automatically load balances traffic

across all active paths.

Controller failover When the active controller fails over to the passive, or standby, controller,

the Device Mapper Multipath driver automatically activates the paths between the host and the standby, making them the primary paths.

Boot/Root device support Multipathing is supported for the root (

Server 10 and later. The host server must be connected to the currently active controller and storage processor for the boot device. The partition must be on a separate, non-multipathed partition. Otherwise, no boot loader is written. For information, see Section 5.8, “Configuring

Multipath I/O for the Root Device,” on page 67.

) device in SUSE Linux Enterprise

/boot

Device Mapper Multipath detects every path for a multipathed device as a separate SCSI device. The SCSI device names take the form

/dev/sdN

beginning with a and issued sequentially as the devices are created, such as

, where N is an autogenerated letter for the device,

/dev/sda, /dev/sdb

and so on. If the number of devices exceeds 26, the letters are duplicated so that the next device after

/dev/sdz

will be named

/dev/sdaa, /dev/sdab

, and so on.

50 SLES 10 SP2: Storage Administration Guide

novdocx (en) 13 May 2009

If multiple paths are not automatically detected, you can configure them manually in the

multipath.conf

file. The

multipath.conf

file does not exist until you create and configure it.

/etc/

For information, see Section 5.4.5, “Creating and Configuring the /etc/multipath.conf File,” on

page 55.

5.3.2 Multipath I/O Management Tools

The

multipath-tools

automatically tests the path periodically, so that a previously failed path is automatically reinstated when it becomes healthy again. This minimizes the need for administrator attention in a production environment.

Table 5-2 Tools in the multipath-tools Package

Tool Description

multipath Scans the system for multipathed devices and assembles them.

multipathd Waits for maps events, then executes

devmap-name Provides a meaningful device name to

kpartx Maps linear devmaps to partitions on the multipathed device, which makes

user-space package takes care of automatic path discovery and grouping. It

multipath

udev

it possible to create multipath monitoring for partitions on the device.

for device maps (devmaps).

For a list of files included in this package, see the multipath-tools Package Description (http://

www.novell.com/products/linuxpackages/suselinux/multipath-tools.html).

1 Ensure that the

multipath-tools

package is installed by entering the following at a terminal

console prompt:

rpm -q multipath-tools

If it is installed, the response repeats the package name and provides the version information, such as:

multipath-tools-04.7-34.23

If it is not installed, the response reads:

package multipath-tools is not installed

5.3.3 Using mdadm for Multipathed Devices

In SUSE Linux Enterprise Server 10, Udev is the default device handler, and devices are automatically known to the system by the Worldwide ID instead of by the device node name. This resolves problems in previous releases where recognize multipathed devices.

mdadm

Just as for LVM2, node path. Therefore, the

DEVICE /dev/disk/by-id/*

requires that the devices be accessed by the ID rather than by the device

DEVICE

entry in

/etc/mdadm.conf

mdadm.conf

and

lvm.conf

did not properly

should be set as follows:

This is the default handling in SUSE Linux Enterprise Server 10 and later, as noted above.

Managing Multipath I/O for Devices 51

novdocx (en) 13 May 2009

To verify that

1 Ensure that the

mdadm

is installed:

mdadm

package is installed by entering the following at a terminal console

prompt:

rpm -q mdadm

If it is installed, the response repeats the package name and provides the version information. For example:

mdadm-2.6-0.11

If it is not installed, the response reads:

package mdadm is not installed

For information about modifying the

/etc/lvm/lvm.conf

file, see Section 5.2.3, “Using LVM2 on

Multipath Devices,” on page 46.

5.3.4 The Linux multipath(8) Command

Use the Linux

General syntax for the

multipath(8)

command to configure and manage multipathed devices.

command:

General Examples

Configure multipath devices:

multipath

Configure a specific multipath device:

multipath devicename

Replace devicename with the device node name such as

/dev/sdb

(as shown by udev in the

$DEVNAME variable), or in the major:minor format.

Selectively suppress a multipath map, and its device-mapped partitions:

multipath -f

Display potential multipath devices, but do not create any devices and do not update device maps (dry run):

multipath -d

Configure multipath devices and display multipath map information:

multipath -v2 <device>

multipath -v3

The -v2 option in multipath -v2 -d shows only local disks. Use the -v3 option to show the full path list. For example:

multipath -v3 -d

Display the status of all multipath devices, or a specified multipath device:

52 SLES 10 SP2: Storage Administration Guide

multipath -ll

multipath -ll <device>

Flush all unused multipath device maps (unresolves the multiple paths; it does not delete the device):

multipath -F

multipath -F <device>

Set the group policy:

multipath -p [failover|multibus|group_by_serial|group_by_prio|group_by_node_name]

Specify one of the group policy options that are described in Table 5-3 on page 53:

Table 5-3 Group Policy Options for the multipath -p Command

Policy Option Description

novdocx (en) 13 May 2009

failover One path per priority group. You can use only one path at a time.

multibus All paths in one priority group.

group_by_serial One priority group per detected SCSI serial number (the controller node

worldwide number).

group_by_prio One priority group per path priority value. Paths with the same priority are in the

same priority group. Priorities are determined by callout programs specified as a global, per-controller, or per-multipath option in the configuration file.

group_by_node_name One priority group per target node name. Target node names are fetched in the

/sys/class/fc_transport/target*/node_name

/etc/multipath.conf

location.

5.4 Configuring the System for Multipathing

 Section 5.4.1, “Preparing SAN Devices for Multipathing,” on page 53

 Section 5.4.2, “Partitioning Multipathed Devices,” on page 54

 Section 5.4.3, “Configuring the Server for Multipathing,” on page 54

 Section 5.4.4, “Adding multipathd to the Boot Sequence,” on page 55

 Section 5.4.5, “Creating and Configuring the /etc/multipath.conf File,” on page 55

5.4.1 Preparing SAN Devices for Multipathing

Before configuring multipath I/O for your SAN devices, prepare the SAN devices, as necessary, by doing the following:

 Configure and zone the SAN with the vendor’s tools.

 Configure permissions for host LUNs on the storage arrays with the vendor’s tools.

Managing Multipath I/O for Devices 53

 Install the Linux HBA driver module. Upon module installation, the driver automatically scans

the HBA to discover any SAN devices that have permissions for the host. It presents them to the host for further configuration.

NOTE: Ensure that the HBA driver you are using does not have native multipathing enabled.

See the vendor’s specific instructions for more details.

 After the driver module is loaded, discover the device nodes assigned to specific array LUNs or

partitions.

novdocx (en) 13 May 2009

If the LUNs are not seen by the HBA driver,

lsscsi

can be used to check whether the SCSI devices are seen correctly by the operating system. When the LUNs are not seen by the HBA driver, check the zoning setup of the SAN. In particular, check whether LUN masking is active and whether the LUNs are correctly assigned to the server.

If the LUNs are seen by the HBA driver, but there are no corresponding block devices, additional kernel parameters are needed to change the SCSI device scanning behavior, such as to indicate that LUNs are not numbered consecutively. For information, see Options for SCSI Device Scanning

(http://support.novell.com/techcenter/sdb/en/2005/06/drahn_scsi_scanning.html) in the Novell

Support Knowledgebase.

5.4.2 Partitioning Multipathed Devices

Partitioning devices that have multiple paths is not recommended, but it is supported.

SUSE Linux Enterprise Server 10

In SUSE Linux Enterprise Server 10, you can use the multipathed devices without rebooting. You can also partition the device before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool.

SUSE Linux Enterprise Server 9

kpartx

tool to create partitions on

In SUSE Linux Enterprise Server 9, if you want to partition the device, you should configure its partitions before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool. This is necessary because partitioning an existing multipathed device is not supported. Partitioning operations on multipathed devices fail if attempted.

If you configure partitions for a device, DM-MP automatically recognizes the partitions and indicates them by appending p1 to pn to the device’s ID, such as

/dev/disk/by-id/26353900f02796769p1

To partition multipathed devices, you must disable the DM-MP service, partition the normal device node (such as

/dev/sdc

), then reboot to allow the DM-MP service to see the new partitions.

5.4.3 Configuring the Server for Multipathing

The system must be manually configured to automatically load the device drivers for the controllers to which the multipath I/O devices are connected within the driver module to the variable INITRD_MODULES in the file

54 SLES 10 SP2: Storage Administration Guide

INITRD

. Therefore add the needed

/etc/sysconfig/kernel

novdocx (en) 13 May 2009

For example, if your system contains a RAID controller accessed by the

cciss

driver and multipathed devices connected to a QLogic* controller accessed by the driver qla2xxx, this entry would look like:

INITRD_MODULES="cciss"

Because the QLogic driver is not automatically loaded on start-up, add it here:

INITRD_MODULES="cciss qla23xx"

After having changed with the command

When you are using LILO as a boot manager, reinstall it with the command

/etc/sysconfig/kernel

mkinitrd

, then reboot in order for the changes to take effect.

, you must re-create the

INITRD

on your system

/sbin/lilo

. No further

action is required if you are using GRUB.

5.4.4 Adding multipathd to the Boot Sequence

Use either of the methods in this section to add multipath I/O services ( sequence.

 “YaST” on page 55

 “Command Line” on page 55

YaST

multipathd

) to the boot

1 In YaST, click System > System Services (Runlevel) > Simple Mode.

2 Select multipathd, then click Enable.

3 Click OK to acknowledge the service startup message.

4 Click Finish, then click Ye s.

The changes do not take affect until the server is restarted.

Command Line

1 Open a terminal console, then log in as the

root

user or equivalent.

2 At the terminal console prompt, enter

insserv multipathd

5.4.5 Creating and Configuring the /etc/multipath.conf File

The

/etc/multipath.conf packages/multipath-tools/multipath.conf.synthetic multipath.conf packages/multipath-tools/multipath.conf.annotated

file that you can use as a guide for multipath settings. See

comments for each of the attributes and their options.

 “Creating the multipath.conf File” on page 56

file does not exist unless you create it. The

file contains a sample

for a template with extensive

/usr/share/doc/

/etc/

/usr/share/doc/

 “Verifying the Setup in the etc/multipath.conf File” on page 56

 “Configuring User-Friendly Names or Alias Names in /etc/multipath.conf” on page 57

 “Blacklisting Non-Multipathed Devices in /etc/multipath.conf” on page 57

Managing Multipath I/O for Devices 55

 “Configuring Default Multipath Behavior in /etc/multipath.conf” on page 58

 “Applying Changes Made to the /etc/multipath.conf File” on page 58

Creating the multipath.conf File

If the

/etc/multipath.conf

file does not exist, copy the example to create the file:

novdocx (en) 13 May 2009

1 In a terminal console, log in as the

root

user.

2 Enter the following command (all on one line, of course) to copy the template:

cp /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic /etc/ multipath.conf

3 Use the

/usr/share/doc/packages/multipath-tools/multipath.conf.annotated

file

as a reference to determine how to configure multipathing for your system.

4 Make sure there is an appropriate

documentation on the proper setup of the

/etc/multipath.conf

The

device

entry for your SAN. Most vendors provide

device

section.

file requires a different

device

section for different SANs. If you are using a storage subsystem that is automatically detected (see “Tested Storage Arrays for

Multipathing Support” on page 48), the default entry for that device can be used; no further

configuration of the

/etc/multipath.conf

file is required.

5 Save the file.

Verifying the Setup in the etc/multipath.conf File

After having set up the configuration, you can perform a “dry run” by entering

multipath -v3 -d

This command scans the devices, then displays what the setup would look like. The output is similar to the following:

26353900f02796769 [size=127 GB] [features="0"] [hwhandler="1 emc"]

\_ round-robin 0 [first] \_ 1:0:1:2 sdav 66:240 [ready ] \_ 0:0:1:2 sdr 65:16 [ready ]

\_ round-robin 0 \_ 1:0:0:2 sdag 66:0 [ready ] \_ 0:0:0:2 sdc 8:32 [ready ]

Paths are grouped into priority groups. Only one priority group is ever in active use. To model an active/active configuration, all paths end up in the same group. To model active/passive configuration, the paths that should not be active in parallel are placed in several distinct priority groups. This normally happens automatically on device discovery.

The output shows the order, the scheduling policy used to balance I/O within the group, and the paths for each priority group. For each path, its physical address (host:bus:target:lun), device node name, major:minor number, and state is shown.

56 SLES 10 SP2: Storage Administration Guide

Configuring User-Friendly Names or Alias Names in /etc/multipath.conf

A multipath device can be identified by either its WWID or an alias that you assign for it. The WWID (World Wide Identifier) is an identifier for the multipath device that is guaranteed to be globally unique and unchanging. The default name used in multipathing is the ID of the logical unit as found in the

/dev/dm-n

and

/dev/disk/by-id

can change on reboot, referring to multipath devices by their ID is preferred.

directory. Because device node names in the form of

/dev/sdn

novdocx (en) 13 May 2009

The multipath device names in the always consistent because they use the association. These device names are user-friendly names such as

You can specify your own device names to use via the ALIAS directive in the

multipath.conf

file. Alias names override the use of ID and

/dev/mapper

/var/lib/multipath/bindings

directory reference the ID of the LUN and are

/dev/mapper/mpathN

mpath0

file to track the

/etc/

names.

IMPORTANT: We recommend that you do not use aliases for the root device, because the ability to seamlessly switch off multipathing via the kernel command line is lost because the device name differs.

For an example of

tools/multipath.conf.synthetic

1 In a terminal console, log in as the

2 Open the

3 Uncomment the

4 Uncomment the

multipath.conf

settings, see the

file.

root

/etc/multipath.conf

Defaults

user_friendly_names option

file in a text editor.

directive and its ending bracket.

/usr/share/doc/packages/multipath-

user.

, then change its value from No to Yes.

For example:

## Use user friendly names, instead of using WWIDs as names. defaults { user_friendly_names yes }

5 Optionally specify your own user-friendly names for devices using the

multipath

section.

alias

directive in the

For example:

multipath { wwid 26353900f02796769 alias sdd4l0 }

6 Save your changes, then close the file.

Blacklisting Non-Multipathed Devices in /etc/multipath.conf

The

/etc/multipath.conf

file should contain a

blacklist

section where all non-multipathed devices should be listed. For example, local IDE hard drives and floppy drives are not normally multipathed. If you have single-path devices that multipath is trying to manage and you want

multipath

For example, to blacklist local devices and all arrays from the multipath, the

to ignore them, put them in the

blacklist

section looks like this:

blacklist

section to resolve the problem.

cciss

driver from being managed by

Managing Multipath I/O for Devices 57

devnode_blacklist { wwid 26353900f02796769 devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sda)[0-9]*" devnode "^hd[a-z][0-9]*" devnode "^cciss!c[0-9]d[0-9].*" }

You can also blacklist only the partitions from a driver instead of the entire array. For example, using the following regular expression would blacklist only partitions from the cciss driver and not the entire array:

^cciss!c[0-9]d[0-9]*[p[0-9]*]

novdocx (en) 13 May 2009

After you modify the

/etc/multipath.conf

file, you must run

mkinitrd

to re-create the INITRD

on your system, then reboot in order for the changes to take effect.

Afterwards, the local devices should no longer be listed in the multipath maps when you issue the

multipath -ll

command.

Configuring Default Multipath Behavior in /etc/multipath.conf

The

/etc/multipath.conf

behaviors. If the field is not otherwise specified in a

file should contain a

defaults

device

section where you can specify default

section, the default setting is applied for

that SAN configuration.

The following defaults section specifies a simple failover policy:

defaults { multipath_tool "/sbin/multipath -v0" udev_dir /dev polling_interval 10 default_selector "round-robin 0" default_path_grouping_policy failover default_getuid_callout "/sbin/scsi_id -g -u -s /block/%n" default_prio_callout "/bin/true" default_features "0" rr_min_io 100 failback immediate

Applying Changes Made to the /etc/multipath.conf File

Changes to the

/etc/multipath.conf

you make changes, save and close the file, then do the following to apply the changes:

1 Stop the

multipathd

service.

2 Clear old multipath bindings by entering

/sbin/multipath -F

3 Create new multipath bindings by entering

/sbin/multipath -v2 -l

4 Start the

5 Run

multipathd

mkinitrd

service.

to re-create the INITRD on your system, then reboot in order for the changes to

take effect.

58 SLES 10 SP2: Storage Administration Guide

file cannot take effect when

multipathd

is running. After

5.5 Enabling and Starting Multipath I/O Services

To start multipath services and enable them to start at reboot:

novdocx (en) 13 May 2009

1 Open a terminal console, then log in as the

2 At the terminal console prompt, enter

chkconfig multipathd on

chkconfig boot.multipath on

If the boot.multipath service does not start automatically on system boot, do the following:

1 Open a terminal console, then log in as the

2 Enter

/etc/init.d/boot.multipath start

/etc/init.d/multipathd start

root

user or equivalent.

root

user or equivalent.

5.6 Configuring Path Failover Policies and Priorities

In a Linux host, when there are multiple paths to a storage controller, each path appears as a separate block device, and results in multiple block devices for single LUN. The Device Mapper Multipath service detects multiple paths with the same LUN ID, and creates a new multipath device with that ID. For example, a host with two HBAs attached to a storage controller with two ports via a single unzoned Fibre Channel switch sees four block devices:

dev/sdd mpath1

. The Device Mapper Multipath service creates a single block device,

that reroutes I/O through those four underlying block devices.

/dev/sda, /dev/sdb, /dev/sdc

/dev/mpath/

, and

This section describes how to specify policies for failover and configure priorities for the paths.

 Section 5.6.1, “Configuring the Path Failover Policies,” on page 59

 Section 5.6.2, “Configuring Failover Priorities,” on page 60

 Section 5.6.3, “Using a Script to Set Path Priorities,” on page 65

 Section 5.6.4, “Configuring ALUA,” on page 65

 Section 5.6.5, “Reporting Target Path Groups,” on page 67

5.6.1 Configuring the Path Failover Policies

Use the

multipath devicename -p policy

Replace policy with one of the following policy options:

Table 5-4 Group Policy Options for the multipath -p Command

Policy Option Description

failover One path per priority group.

multipath

command with the -p option to set the path failover policy:

Managing Multipath I/O for Devices 59

Policy Option Description

multibus All paths in one priority group.

group_by_serial One priority group per detected serial number.

group_by_prio One priority group per path priority value. Priorities are determined by callout

programs specified as a global, per-controller, or per-multipath option in the

etc/multipath.conf

configuration file.

novdocx (en) 13 May 2009

group_by_node_name One priority group per target node name. Target node names are fetched in the

/sys/class/fc_transport/target*/node_name

location.

5.6.2 Configuring Failover Priorities

You must manually enter the failover priorities for the device in the Examples for all settings and options can be found in the

multipath-tools/multipath.conf.annotated

/usr/share/doc/packages/

file.

 “Understanding Priority Groups and Attributes” on page 60

 “Configuring for Round-Robin Load Balancing” on page 64

 “Configuring for Single Path Failover” on page 64

 “Grouping I/O Paths for Round-Robin Load Balancing” on page 64

Understanding Priority Groups and Attributes

A priority group is a collection of paths that go to the same physical LUN. By default, I/O is distributed in a round-robin fashion across all paths in the group. The automatically creates priority groups for each LUN in the SAN based on the path_grouping_policy setting for that SAN. The

multipath

command multiplies the number of paths in a group by the group’s priority to determine which group is the primary. The group with the highest calculated value is the primary. When all paths in the primary group are failed, the priority group with the next highest value becomes active.

/etc/multipath.conf

multipath

command

file.

A path priority is an integer value assigned to a path. The higher the value, the higher is the priority. An external program is used to assign priorities for each path. For a given device, its paths with the same priorities belong to the same priority group.

Table 5-5 Multipath Attributes

Multipath Attribute Description Values

user_friendly_names Specifies whether to use IDs

or to use the

multipath/bindings

to assign a persistent and unique alias to the multipath devices in the form of

mapper/mpathN

/var/lib/

60 SLES 10 SP2: Storage Administration Guide

file

/dev/

yes Autogenerate user-friendly names as aliases for the multipath devices instead of the actual ID.

no Default. Use the WWIDs shown in the

disk/by-id/

location.

/dev/

Multipath Attribute Description Values

novdocx (en) 13 May 2009

blacklist Specifies the list of device

names to ignore as nonmultipathed devices, such as cciss, fd, hd, md, dm, sr, scd, st, ram, raw, loop.

blacklist_exceptions Specifies the list of device

names to treat as multipath devices even if they are included in the blacklist.

getuid_callout The default program and

argumentss to callout to obtain a unique path identifier. Should be specified with an absolute path.

path_grouping_policy Specifies the path grouping

policy for a multipath device hosted by a given controller.

For an example, see “Blacklisting Non-

Multipathed Devices in /etc/multipath.conf” on page 57.

For an example, see the

/usr/share/doc/ packages/multipath-tools/ multipath.conf.annotated

file.

/lib/udev/scsi_id -g -u -s

This is the default location and arguments.

failover One path is assigned per priority group so that only one path at a time is used.

multibus (Default) All valid paths are in one priority group. Traffic is load-balanced across all active paths in the group.

group_by_prio One priority group exists for each path priority value. Paths with the same priority are in the same priority group. Priorities are assigned by an external program.

group_by_serial Paths are grouped by the SCSI target serial number (controller node WWN).

group_by_node_name One priority group is assigned per target node name. Target node names are fetched in

fc_transport/target*/node_name

/sys/class/

Managing Multipath I/O for Devices 61

Multipath Attribute Description Values

novdocx (en) 13 May 2009

path_checker Determines the state of the

path.

directio (Default in

0.4.8 and later) Reads the first sector that has direct I/O. This is useful for DASD devices. Logs failure messages in

readsector0 (Default in version 0.4.7 and earlier) Reads the first sector of the device. Logs failure messages in

log/messages

tur Issues a SCSI test unit ready command to the device. This is the preferred setting if the LUN supports it. The command does not fill up

var/log/messages

Some SAN vendors provide custom

path_checker



emc_clariion

EVPD page 0xC0 to determine the path state.



hp_sw

Ghost) for HP storage arrays with Active/ Standby firmware.



rdac

Checks the path state for the LSI/

Engenio RDAC storage controller.

multipath-tools

/var/log/messages

multipath-tools

version

/var/

on failure with messages.

options:

Queries the EMC Clariion

Checks the path state (Up, Down, or

path_selector Specifies the path-selector

algorithm to use for loadbalancing.

pg_timeout Specifies path group timeout

handling.

round-robin 0 (Default) The load-balancing algorithm used to balance traffic across all active paths in a priority group.

This is currently the only algorithm available.

NONE (internal default)

62 SLES 10 SP2: Storage Administration Guide

Multipath Attribute Description Values

novdocx (en) 13 May 2009

prio_callout Specifies the program and

arguments to use to determine the layout of the multipath map.

When queried by the

multipath

specified mpath_prio_* callout program returns the priority for a given path in relation to the entire multipath layout.

When it is used with the path_grouping_policy of group_by_prio, all paths with the same priority are grouped into one multipath group. The group with the highest aggregate priority becomes the active group.

When all paths in a group fail, the group with the next highest aggregate priority becomes active. Additionally, a failover command (as determined by the hardware handler) might be send to the target.

The mpath_prio_* program can also be a custom script created by a vendor or administrator for a specified setup.

A %n in the command line expands to the device name in the

A %b expands to the device number in major:minor format in the

command, the

/dev

directory.

/dev

directory.

If no

prio_callout

are equal. This is the default.

/bin/true Use this value when the group_by_priority is not being used.

prioritizer programs generate path priorities

The when queried by the program names must begin with and are named by the device type or balancing method used. Current prioritizer programs include the following:

/sbin/mpath_prio_alua %n Generates path priorities based on the SCSI-3 ALUA settings.

/sbin/mpath_prio_balance_units Generates the same priority for all paths.

/sbin/mpath_prio_emc %n Generates the path priority for EMC arrays.

/sbin/mpath_prio_hds_modular %b

Generates the path priority for Hitachi HDS Modular storage arrays.

/sbin/mpath_prio_hp_sw %n Generates the path priority for Compaq/HP controller in active/ standby mode.

/sbin/mpath_prio_netapp %n Generates the path priority for NetApp arrays.

/sbin/mpath_prio_random %n Generates a random priority for each path.

/sbin/mpath_prio_rdac %n Generates the path priority for LSI/Engenio RDAC controller.

/sbin/mpath_prio_tpc %n You can optionally use a script created by a vendor or administrator that gets the priorities from a file where you specify priorities to use for each path.

/usr/local/sbin/mpath_prio_spec.sh %n

attribute is used, all paths

multipath

command. The

mpath_prio_

A %d expands to the device ID in the directory.

If devices are hot-pluggable, use the %d flag instead of %n. This addresses the short time that elapses between the time when devices are available and when creates the device nodes.

/dev/disk/by-id

udev

Provides the path of a user-created script that generates the priorities for multipathing based on information contained in a second data file. (This path and filename are provided as an example. Specify the location of your script instead.) The script can be created by a vendor or administrator. The script’s target file identifies each path for all multipathed devices and specifies a priority for each path. For an example, see Section 5.6.3, “Using a Script to

Set Path Priorities,” on page 65.

Managing Multipath I/O for Devices 63

Multipath Attribute Description Values

novdocx (en) 13 May 2009

rr_min_io Specifies the number of I/O

transactions to route to a path before switching to the next path in the same path group, as determined by the specified algorithm in the

path_selector

rr_weight Specifies the weighting

method to use for paths.

no_path_retry Specifies the behaviors to use

on path failure.

failback Specifies whether to monitor

the failed path recovery, and indicates the timing for group failback after failed paths return to service.

When the failed path recovers, the path is added back into the multipath enabled path list based on this setting. Multipath evaluates the priority groups, and changes the active priority group when the priority of the primary path exceeds the secondary group.

setting.

n (>0) Specify an integer value greater than 0.

1000 Default.

uniform Default. All paths have the same

round-robin weightings.

priorities Each path’s weighting is determined by the path’s priority times the rr_min_io setting.

n (> 0) Specifies the number of retries until

multipath

Specify an integer value greater than 0.

fail Specified immediate failure (no queuing).

queue Never stop queuing (queue forever until

the path comes alive).

immediate When a path recovers, enable the path immediately.

n (> 0) When the path recovers, wait n seconds before enabling the path. Specify an integer value greater than 0.

manual (Default) The failed path is not monitored for recovery. The administrator runs

multipath

the paths and priority groups.

stops the queuing and fails the path.

command to update enabled

Configuring for Round-Robin Load Balancing

All paths are active. I/O is configured for some number of seconds or some number of I/O transactions before moving to the next open path in the sequence.

Configuring for Single Path Failover

A single path with the highest priority (lowest value setting) is active for traffic. Other paths are available for failover, but are not used unless failover occurs.

Grouping I/O Paths for Round-Robin Load Balancing

Multiple paths with the same priority fall into the active group. When all paths in that group fail, the device fails over to the next highest priority group. All paths in the group share the traffic load in a round-robin load balancing fashion.

64 SLES 10 SP2: Storage Administration Guide

5.6.3 Using a Script to Set Path Priorities

You can create a script that interacts with DM-MP to provide priorities for paths to the LUN when set as a resource for the

prio_callout

First, set up a text file that lists information about each device and the priority values you want to assign to each path. For example, name the file for each path in the following format:

host_wwpn target_wwpn scsi_id priority_value

Return a priority value for each path on the device. Make sure that the variable FILE_PRIMARY_PATHS resolves to a real file with appropriate data (host wwpn, target wwpn, scsi_id and priority value) for each device.

setting.

/usr/local/etc/primary-paths

. Enter one line

novdocx (en) 13 May 2009

The contents of the

0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:0 sdb 3600a0b8000122c6d00000000453174fc 50

0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:1 sdc 3600a0b80000fd6320000000045317563 2

0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:2 sdd 3600a0b8000122c6d0000000345317524 50

0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:3 sde 3600a0b80000fd6320000000245317593 2

0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:0 sdi 3600a0b8000122c6d00000000453174fc 5

0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:1 sdj 3600a0b80000fd6320000000045317563 51

0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:2 sdk 3600a0b8000122c6d0000000345317524 5

0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:3 sdl 3600a0b80000fd6320000000245317593 51

primary-paths

file for a single LUN with eight paths each might look like this:

To continue the example mentioned in Table 5-5 on page 60, create a script named

sbin/path_prio.sh

. You can use any path and filename. The script does the following:

/usr/local/

 On query from multipath, grep the device and its path from the

file.

paths

 Return to multipath the priority value in the last column for that entry in the file.

/usr/local/etc/primary-

5.6.4 Configuring ALUA

The

mpath_prio_alua(8)

command. It returns a number that is used by DM-MP to group SCSI devices with the same priority together. This path priority tool is based on ALUA (Asynchronous Logical Unit Access).

 “Syntax” on page 66

 “Prerequisite” on page 66

command is used as a priority callout for the Linux

Managing Multipath I/O for Devices 65

multipath(8)

 “Options” on page 66

 “Return Values” on page 66

Syntax

mpath_prio_alua [-d directory] [-h] [-v] [-V] device [device...]

Prerequisite

SCSI devices

Options

-d directory

Specifying the Linux directory path where the listed device node names can be found. The default directory is

/dev

. When used, specify the device node name only (such as

device or devices you want to manage.

-h

Displays help for this command, then exits.

sda

novdocx (en) 13 May 2009

) for the

-v

Turns on verbose output to display status in human-readable format. Output includes information about which port group the specified device is in and its current state.

-V

Displays the version number of this tool, then exits.

device

Specifies the SCSI device you want to manage. The device must be a SCSI device that supports the Report Target Port Groups (

sg_rtpg(8)

) command. Use one of the following formats for

the device node name:

 The full Linux directory path, such as

 The device node name only, such as

 The major and minor number of the device separated by a colon (:) with no spaces, such as

8:0

. This creates a temporary device node in the

tmpdev-<major>:<minor>-<pid>

/dev/sda

sda

. Specify the directory path using the -d option.

. For example,

. Do not use with the -d option.

/dev

directory with a name in the format

/dev/tmpdev-8:0-<pid>

Return Values

On success, returns a value of 0 and the priority value for the group. Tab le 5 -6 shows the priority values returned by the

mpath_prio_alua

command.

Table 5-6 ALUA Priorities for Device Mapper Multipath

Priority Value Description

50 The device is in the active, optimized group.

10 The device is in an active but non-optimized group.

66 SLES 10 SP2: Storage Administration Guide

Priority Value Description

1 The device is in the standby group.

0 All other groups.

novdocx (en) 13 May 2009

Values are widely spaced because of the way the the number of paths in a group with the priority value for the group, then selects the group with the highest result. For example, if a non-optimized path group has six paths (6 x 10 = 60) and the optimized path group has a single path (1 x 50 = 50), the non-optimized group has the highest score, so multipath chooses the non-optimized group. Traffic to the device uses all six paths in the group in a round-robin fashion.

On failure, returns a value of 1 to 5 indicating the cause for the command’s failure. For information, see the man page for

mpath_prio_alua

multipath

command handles them. It multiplies

5.6.5 Reporting Target Path Groups

Use the SCSI Report Target Port Groups ( page for

sg_rtpg(8)

) command. For information, see the man

5.7 Tuning the Failover for Specific Host Bus Adapters

When using multipath I/O, you want any host bus adapter (HBA) failure or cable failures to be reported faster than when multipathing is not in use. Configure time-out settings for your HBA to disable failover at the HBA level to allow the failure to propogate up to the multipath I/O layer as fast as possible where the I/O can be redirected to another, healthy path.

To disable the HBA handling of failover, modify the driver’s options in the

modprobe.conf.local

to disable failover settings for your driver.

For example, for the QLogic qla2xxx family of host bus adapters, the following setting is recommended:

options qla2xxx qlport_down_retry=1

file. Refer to the HBA vendor’s documentation for information about how

/etc/

5.8 Configuring Multipath I/O for the Root Device

In the SUSE Linux Enterprise Server 10, the root partition (/) on multipath is supported only if the

partition is on a separate, non-multipathed partition. Otherwise, no boot loader is written.

boot

IMPORTANT: If you apply all online updates, the DM-MP is available but is not supported for

and

/root

boot mkinitrd

packages and set up the configuration, you might run into update issues later.

Full multipath support is available in SUSE Linux Enterprise Server 11.

in SUSE Linux Enterprise Server 10 SP1 and later. More specifically, you need

1.2-106.61 and

multipath-tools

0.4.7-34.23 or later. However, if you install the

Managing Multipath I/O for Devices 67

To enable multipathing on the existing root device:

novdocx (en) 13 May 2009

1 Install Linux with only a single path active, preferably one where the

by-id

symlinks are listed

in the partitioner.

2 Mount the devices by using the

3 After installation, add

4 Re-run

/sbin/mkinitrd

dm-multipath

/dev/disk/by-id

to update the

path used during the install.

/etc/sysconfig/kernel:INITRD_MODULES

initrd

image.

5 Reboot the server.

To disable multipathing on the root device:

1 Add

multipath=off

to the kernel command line.

This affects only the root device. All other devices are not affected.

5.9 Configuring Multipath I/O for an Existing Software RAID

Ideally, you should configure multipathing for devices before you use them as components of a software RAID device. If you add multipathing after creating any software RAID devices, the DMMP service might be starting after the

multipath

appear not to be available for RAIDs. You can use the procedure in this section to get multipathing running for a previously existing software RAID.

For example, you might need to configure multipathing for devices in a software RAID under the following circumstances:

service on reboot, which makes multipathing

 If you create a new software RAID as part of the Partitioning settings during a new install or

upgrade.

 If you did not configure the devices for multipathing before using them in the software RAID

as a member device or spare.

 If you grow your system by adding new HBA adapters to the server or expanding the storage

subsystem in your SAN.

NOTE: The following instructions assume the software RAID device is

/dev/mapper/mpath0

, which is its device name as recognized by the kernel. Make sure to modify the instructions for the device name of your software RAID.

1 Open a terminal console, then log in as the

root

user or equivalent.

Except where otherwise directed, use this console to enter the commands in the following steps.

2 If any software RAID devices are currently mounted or running, enter the following commands

for each device to dismount the device and stop it.

umount /dev/mapper/mpath0

mdadm --misc --stop /dev/mapper/mpath0

3 Stop the

/etc/init.d/boot.md stop

4 Start the

boot.md

boot.multipath

service by entering

and

multipathd

services by entering the following commands:

68 SLES 10 SP2: Storage Administration Guide

/etc/init.d/boot.multipath start

/etc/init.s/multipathd start

5 After the multipathing services are started, verify that the software RAID’s component devices

are listed in the

 Devices Are Listed: The device names should now have symbolic links to their Device

/dev/disk/by-id

Mapper Multipath device names, such as

 Devices Are Not Listed: Force the multipath service to recognize them by flushing and

directory. Do one of the following:

/dev/dm-1

rediscovering the devices.

To do this, enter the following commands:

multipath -F

multipath -v0

The devices should now be listed in

/dev/disk/by-id

, and have symbolic links to their

Device Mapper Multipath device names. For example:

lrwxrwxrwx 1 root root 10 Jun 15 09:36 scsi-mpath1 -> ../../dm-1

6 Restart the

/etc/init.d/boot.md start

boot.md

service and the RAID device by entering

7 Check the status of the software RAID by entering

mdadm --detail /dev/mapper/mpath0

The RAID’s component devices should match their Device Mapper Multipath device names that are listed as the symbolic links of devices in the

8 Make a new

initrd

to ensure that the Device Mapper Multipath services are loaded before the

/dev/disk/by-id

directory.

RAID services on reboot. Enter

mkinitrd -f mpath

9 Reboot the server to apply these post-install configuration settings.

novdocx (en) 13 May 2009

10 Verify that the software RAID array comes up properly on top of the multipathed devices by

checking the RAID status. Enter

mdadm --detail /dev/mapper/mpath0

For example:

Number Major Minor RaidDevice State

0 253 0 0 active sync /dev/dm-0

1 253 1 1 active sync /dev/dm-1

2 253 2 2 active sync /dev/dm-2

5.10 Scanning for New Devices without Rebooting

If your system has already been configured for multipathing and you later need to add more storage to the SAN, use the following procedure to scan the devices and make them available to multipathing without rebooting the system.

1 On the storage subsystem, use the vendor’s tools to allocate the device and update its access

control settings to allow the Linux system access to the new storage. Refer to the vendor’s documentation for details.

Managing Multipath I/O for Devices 69

2 On the Linux system, scan the SAN at a low level to discover the new devices. At a terminal

console prompt, enter

echo 1 > /sys/class/fc_host/host<number>/issue_lip

For example, to probe the HBA on host1, enter

echo 1 > /sys/class/fc_host/host1/issue_lip

At this point, the newly added device is not known to the higher layers of the Linux kernel's SCSI subsystem and is not yet usable.

3 Scan all targets for a host to make its new device known to the middle layer of the Linux

kernel's SCSI subsystem. At a terminal console prompt, enter

echo "- - -" > /sys/class/scsi_host/host<number>/scan

For example, to probe the HBA on host1, enter

echo "- - -" > /sys/class/scsi_host/host1/scan

4 Check for scanning progress in the system log (the

/var/log/messages

file). At a terminal

console prompt, enter

tail -30 /var/log/messages

This command displays the last 30 lines of the log. For example:

# tail -30 /var/log/messages . . . Feb 14 01:03 kernel: SCSI device sde: 81920000 Feb 14 01:03 kernel: SCSI device sdf: 81920000 Feb 14 01:03 multipathd: sde: path checker registered Feb 14 01:03 multipathd: sdf: path checker registered Feb 14 01:03 multipathd: mpath4: event checker started Feb 14 01:03 multipathd: mpath5: event checker started Feb 14 01:03:multipathd: mpath4: remaining active paths: 1 Feb 14 01:03 multipathd: mpath5: remaining active paths: 1

5 Repeat Step 2 through Step 4 to add paths through other HBA adapters on the Linux system

that are connected to the new device.

6 Run the

multipath

command to recognize the devices for DM-MP configuration. At a

terminal console prompt, enter

multipath

You can now configure the new device for multipathing.

novdocx (en) 13 May 2009

5.11 Scanning for New Partitioned Devices without Rebooting

Use the example in this section to detect a newly added multipathed LUN without rebooting.

1 Open a terminal console, then log in as the

2 Detect the newly added multipathed LUN by entering:

echo '- - -' >>/sys/class/fc_host/host1/device/scsi_host\:host1/scan

echo '- - -' >>/sys/class/fc_host/host2/device/scsi_host\:host2/scan

3 Verify that the device is seen (link has a fresh timestamp) by entering

ls -lrt /dev/dm-*

4 Verify the new WWN of the device appears in the log by entering

70 SLES 10 SP2: Storage Administration Guide

root

user.

tail -33 /var/log/messages

5 Use a text editor to add a new alias definition for the device in the

such as

oradata3

/etc/multipath.conf

6 Create a partition table for the device by entering

fdisk /dev/dm-8

7 Add a

/dev/dm-*

/sbin/kpartx -a -p -part /dev/dm-8

link for the new partition by entering

8 Verify that the link was created by entering

ls -lrt /dev/dm-*

9 Create a filesystem and label for the new partition by entering

mke2fs -j /dev/dm-9

tune2fs -L oradata3 /dev/dm-9

10 Restart DM-Multipath to let it read the aliases by entering

/etc/init.d/multipathd restart

11 Verify that the device is recognized by

multipath -ll

12 Use a text editor to add a mount entry in the

At this point, the alias you created in Step 5 is not yet in the Add the mount entry the

/dev/dm-9

multipathd

/etc/fstab

by entering

file.

/dev/disk/by-label

path, then change the entry before the next time you

reboot to

LABEL=oradata3

13 Create a directory to use as the mount point, then mount the device by entering

md /oradata3

mount /oradata3

novdocx (en) 13 May 2009

file,

directory.

5.12 Viewing Multipath I/O Status

Querying the multipath I/O status outputs the current status of the multipath maps.

The

multipath -l

was run. It does not run the path checker.

multipath -ll

The current status information. This option always the displays the latest information about the path status.

1 At a terminal console prompt, enter

multipath -ll

This displays information for each multipathed device. For example:

3600601607cf30e00184589a37a31d911 [size=127 GB][features="0"][hwhandler="1 emc"]

option displays the current path status as of the last time that the path checker

option runs the path checker, updates the path information, then displays the

Managing Multipath I/O for Devices 71

\_ round-robin 0 [active][first] \_ 1:0:1:2 sdav 66:240 [ready ][active] \_ 0:0:1:2 sdr 65:16 [ready ][active]

\_ round-robin 0 [enabled] \_ 1:0:0:2 sdag 66:0 [ready ][active] \_ 0:0:0:2 sdc 8:32 [ready ][active]

For each device, it shows the device’s ID, size, features, and hardware handlers.

Paths to the device are automatically grouped into priority groups on device discovery. Only one priority group is active at a time. For an active/active configuration, all paths are in the same group. For an active/passive configuration, the passive paths are placed in separate priority groups.

The following information is displayed for each group:

 Scheduling policy used to balance I/O within the group, such as round-robin

 Whether the group is active, disabled, or enabled

 Whether the group is the first (highest priority) group

 Paths contained within the group

novdocx (en) 13 May 2009

The following information is displayed for each path:

 The physical address as host:bus:target:lun, such as 1:0:1:2

 Device node name, such as

 Major:minor numbers

 Status of the device

sda

5.13 Managing I/O in Error Situations

You might need to configure multipathing to queue I/O if all paths fail concurrently. In certain scenarios, where the driver, the HBA, or the fabric experiences spurious errors, it is advisable that DM-MP be configured to queue all I/O where those errors lead to a loss of all paths, and never propagate errors upwards. Because this leads to I/O being queued indefinitely unless a path is reinstated, make sure that

multipathd

might be stalled indefinitely on the affected multipathed device, until reboot or until you manually return to failover instead of queuing.

To test the scenario:

1 In a terminal console, log in as the

2 Activate queuing instead of failover for the device I/O by entering:

dmsetup message device_ID 0 queue_if_no_path

Replace the device_ID with the ID for your device. For example, enter:

dmsetup message 3600601607cf30e00184589a37a31d911 0 queue_if_no_path

3 Return to failover for the device I/O by entering:

dmsetup message device_ID 0 fail_if_no_path

This command immediately causes all queued I/O to fail.

Replace the device_ID with the ID for your device. For example, enter:

dmsetup message 3600601607cf30e00184589a37a31d911 0 fail_if_no_path

is running and works for your scenario. Otherwise, I/O

root

user.

72 SLES 10 SP2: Storage Administration Guide

To set up queuing I/O for scenarios where all paths fail:

novdocx (en) 13 May 2009

1 In a terminal console, log in as the

2 Open the

/etc/multipath.conf

3 Uncomment the defaults section and its ending bracket, then add the

root

user.

file in a text editor.

default_features

setting, as follows:

defaults { default_features "1 queue_if_no_path" }

4 After you modify the

/etc/multipath.conf

file, you must run

mkinitrd

to re-create the

INITRD on your system, then reboot in order for the changes to take effect.

5 When you are ready to return over to failover for the device I/O, enter:

dmsetup message mapname 0 fail_if_no_path

Replace the mapname with the mapped alias name or the device ID for the device.

This command immediately causes all queued I/O to fail and propagates the error to the calling application.

5.14 Resolving Stalled I/O

If all paths fail concurrently and I/O is queued and stalled, do the following:

1 Enter the following command at a terminal console prompt:

dmsetup message mapname 0 fail_if_no_path

Replace all queued I/O to fail and propagates the error to the calling application.

2 Reactivate queueing by entering the following command at a terminal console prompt:

dmsetup message mapname 0 queue_if_no_path

mapname

with the correct device ID or mapped alias name for the device. This causes

5.15 Additional Information

For more information about configuring and using multipath I/O on SUSE Linux Enterprise Server, see the following additional resources in the Novell Support Knowledgebase:

 How to Setup/Use Multipathing on SLES (http://support.novell.com/techcenter/sdb/en/2005/

04/sles_multipathing.html)

 Troubleshooting SLES Multipathing (MPIO) Problems (Technical Information Document

3231766) (http://www.novell.com/support/ search.do?cmd=displayKC&docType=kc&externalId=3231766&sliceId=SAL_Public)

 Dynamically Adding Storage for Use with Multipath I/O (Technical Information Document

3000817) (https://secure-support.novell.com/KanisaPlatform/Publishing/911/ 3000817_f.SAL_Public.html)

 DM MPIO Device Blacklisting Not Honored in multipath.conf (Technical Information

Document 3029706) (http://www.novell.com/support/ search.do?cmd=displayKC&docType=kc&externalId=3029706&sliceId=SAL_Public&dialogI D=57872426&stateId=0%200%2057878058)

Managing Multipath I/O for Devices 73

 Static Load Balancing in Device-Mapper Multipathing (DM-MP) (Technical Information

Document 3858277) (http://www.novell.com/support/ search.do?cmd=displayKC&docType=kc&externalId=3858277&sliceId=SAL_Public&dialogI D=57872426&stateId=0%200%2057878058)

 Troubleshooting SCSI (LUN) Scanning Issues (Technical Information Document 3955167)

(http://www.novell.com/support/ search.do?cmd=displayKC&docType=kc&externalId=3955167&sliceId=SAL_Public&dialogI D=57868704&stateId=0%200%2057878206)

5.16 What’s Next

If you want to use software RAIDs, create and configure them before you create file systems on the devices. For information, see the following:

 Chapter 6, “Managing Software RAIDs with EVMS,” on page 75

 Chapter 7, “Managing Software RAIDs 6 and 10 with mdadm,” on page 97

novdocx (en) 13 May 2009

74 SLES 10 SP2: Storage Administration Guide

Managing Software RAIDs with

novdocx (en) 13 May 2009

EVMS

This section describes how to create and manage software RAIDs with the Enterprise Volume Management System (EVMS). EVMS supports only RAIDs 0, 1, 4, and 5 at this time. For RAID 6 and 10 solutions, see Chapter 7, “Managing Software RAIDs 6 and 10 with mdadm,” on page 97.

 Section 6.1, “Understanding Software RAIDs on Linux,” on page 75

 Section 6.2, “Creating and Configuring a Software RAID,” on page 81

 Section 6.3, “Expanding a RAID,” on page 85

 Section 6.4, “Adding or Removing a Spare Disk,” on page 86

 Section 6.5, “Managing Disk Failure and RAID Recovery,” on page 87

 Section 6.6, “Monitoring Status for a RAID,” on page 90

 Section 6.7, “Deleting a Software RAID and Its Data,” on page 95

6.1 Understanding Software RAIDs on Linux

 Section 6.1.1, “What Is a Software RAID?,” on page 75

 Section 6.1.2, “Overview of RAID Levels,” on page 76

 Section 6.1.3, “Comparison of RAID Performance,” on page 77

 Section 6.1.4, “Comparison of Disk Fault Tolerance,” on page 77

 Section 6.1.5, “Configuration Options for RAIDs,” on page 78

 Section 6.1.6, “Interoperability Issues,” on page 78

 Section 6.1.7, “Guidelines for Component Devices,” on page 79

 Section 6.1.8, “RAID 5 Algorithms for Distributing Stripes and Parity,” on page 80

 Section 6.1.9, “Multi-Disk Plug-In for EVMS,” on page 81

 Section 6.1.10, “Device Mapper Plug-In for EVMS,” on page 81

6.1.1 What Is a Software RAID?

A RAID combines multiple devices into a multi-disk array to provide resiliency in the storage device and to improve storage capacity and I/O performance. If a disk fails, some RAID levels keep data available in a degraded mode until the failed disk can be replaced and its content reconstructed.

A software RAID provides the same high availability that you find in a hardware RAID. The key operational differences are described in the following table:

Table 6-1 Comparison of Software RAIDs and Hardware RAIDs

Feature Linux Software RAID Hardware RAID

RAID function Multi-disk (md) driver or

mdadm

RAID controller on the disk array

Managing Software RAIDs with EVMS

Feature Linux Software RAID Hardware RAID

RAID processing In the host server’s processor RAID controller on the disk array

novdocx (en) 13 May 2009

RAID levels 0, 1, 4, 5, and 10 plus the

raid10

Component devices Disks from same or different disk

array

mdadm

Varies by vendor

Same disk array

6.1.2 Overview of RAID Levels

The following table describes the advantages and disadvantages of the RAID levels supported by EVMS. The description assumes that the component devices reside on different disks and that each disk has its own dedicated I/O capability.

IMPORTANT: For information about creating complex or nested RAID devices with mdadm, see

Chapter 7, “Managing Software RAIDs 6 and 10 with mdadm,” on page 97.

Table 6-2 RAID Levels Supported by EVMS

RAID Level Description Performance and Fault Tolerance

0 Stripes data using a round-

robin method to distribute data over the RAID’s component devices.

Improves disk I/O performance for both reads and writes. Actual performance depends on the stripe size, the actual data, and the application.

Does not provide disk fault tolerance and data redundancy. Any disk failure causes all data in the RAID to be lost.

1 Mirrors data by copying

blocks of one disk to another and keeping them in continuous synchronization. If disks are different sizes, the smallest disk determines the size of the RAID.

4 Stripes data and records

parity to a dedicated disk. If disks are different sizes, the smallest disk determines the size of the RAID.

Improves disk reads by making multiple copies of data available via different I/O paths. The write performance is about the same as for a single disk because a copy of the data must be written to each of the disks in the mirror.

Provides 100% data redundancy. If one disk fails then the data remains available on its mirror, and processing continues.

Improves disk I/O performance for both reads and writes. Write performance is considerably slower than for RAID 0, because parity must be calculated and written. Write performance is slightly slower than RAID 5. Read performance is slower than for a RAID 1 array with the same number of component devices. The dedicated parity disk can become a bottleneck for writing parity.

Provides disk fault tolerance. If a disk fails, performance is degraded while the RAID uses the parity to reconstruct data for the replacement disk.

76 SLES 10 SP2: Storage Administration Guide

RAID Level Description Performance and Fault Tolerance

novdocx (en) 13 May 2009

5 Stripes data and distributes

parity in a round-robin fashion across all disks. If disks are different sizes, the smallest disk determines the size of the RAID.

Improves disk I/O performance for reads and writes. Write performance is considerably less than for RAID 0, because parity must be calculated and written. Write performance is faster than RAID 4. Read performance is slower than for a RAID 1 array with the same number of component disks. Actual performance depends on the number of component disks, the stripe size, the actual data, and the application.

Provides disk fault tolerance. If a disk fails, performance is degraded while the RAID uses the parity to reconstruct data for the replacement disk. Provides slightly less data redundancy than mirroring because it uses parity to reconstruct the data.

6.1.3 Comparison of RAID Performance

The following table compares the read and write performance for RAID devices.

Table 6-3 Read and Write Performance for RAIDs

Raid Level Read Performance Write Performance

0 Faster than for a single disk Faster than for a single disk and other

RAIDs.

1 Faster than for a single disk, increasing as

more mirrors are added

4 Faster than for a single disk. Slower than a

RAID 0 because one disk is used for parity.

5 Faster than for a single disk; comparable to a

RAID 0.

Slower than for a single disk, declining as more mirrors are added.

Faster than for a single disk. Slower than a RAID 0 because of writes for parity. Slower than a RAID 5 because of possible bottlenecks for writes of parity to the dedicated parity disk.

Faster than a single disk. Slower than a RAID 0 because of writes for parity.

6.1.4 Comparison of Disk Fault Tolerance

The following table compares the disk fault tolerance for RAID devices.

Table 6-4 Fault Tolerance for RAIDs

Raid Level Number of Disk Failures Tolerated Data Redundancy

0None No

1 Number of disks minus 1 100% redundancy for each mirror

4 1 Dedicated parity disk to reconstruct data. If

the parity disk fails, all parity must be recalculated.

Managing Software RAIDs with EVMS 77

Raid Level Number of Disk Failures Tolerated Data Redundancy

5 1 Distributed parity to reconstruct data and

parity on the failed disk.

6.1.5 Configuration Options for RAIDs

In EVMS management tools, the following RAID configuration options are provided:

Table 6-5 Configuration Options in EVMS

Option Description

Spare Disk For RAIDs 1, 4, and 5, you can optionally specify a device, segment, or

region to use as the replacement for a failed disk (the member device, segment, or region). On failure, the spare disk automatically replaces the failed disk, then reconstructs the data.

However, if the parity disk fails on a RAID 5, parity cannot be reconstructed.

novdocx (en) 13 May 2009

Chunk Size (KB) For RAIDs 0, 4, or 5, specify the stripe size in KB.

Consider the intended use of the RAID, such as the file system block size, the applications used, and the actual data (file sizes and typical reads and writes). A typical write size for large files is 128 KB.

Default: 32 KB

Range: 4 KB to 4096 KB, in powers of 2.

RAID Level If you selected MD RAID 4/5 Region Manager, specify RAID 4 or RAID 5

(default).

RAID Algorithm For RAID 5, specify one of the following algorithms to use for striping and

distributing parity on the disk.

 Left Asymmetric

 Left Symmetric (Default, fastest performance for large reads)

 Right Asymmetric

 Right Symmetric

6.1.6 Interoperability Issues

Linux software RAID cannot be used underneath clustered file systems because it does not support concurrent activation. If you want RAID and OCFS2, you need the RAID to be handled by the storage subsystem.

WARNING: Activating Linux software RAID devices concurrently on multiple servers can result in data corruption or inconsistencies.

78 SLES 10 SP2: Storage Administration Guide

6.1.7 Guidelines for Component Devices

For efficient use of space and performance, the disks you use to create the RAID should have the same storage capacity. Typically, if component devices are not of identical storage capacity, then each member of the RAID uses only an amount of space equal to the capacity of the smallest member disk.

mdadm

Version 2.3 and later of support component devices up to 2 TB in size.

You can combine up to 28 component devices to create the RAID array. The md RAID device you create can be up to the maximum device size supported by the file system you plan to use. For information about file system limits for SUSE Support” in the SUSE Linux Enterprise Server 10 Installation and Administration Guide. (http://

www.novell.com/documentation/sles10).

supports component devices up to 4 TB in size each. Earlier versions

Linux Enterprise Server 10, see “Large File System

novdocx (en) 13 May 2009

In general, each storage object included in the RAID should be from a different physical disk to maximize I/O performance and to achieve disk fault tolerance where supported by the RAID level you use. In addition, they should be of the same type (disks, segments, or regions).

Using component devices of differing speeds might introduce a bottleneck during periods of demanding I/O. The best performance can be achieved by using the same brand and models of disks and controllers in your hardware solution. If they are different, you should try to match disks and controllers with similar technologies, performance, and capacity. Use a low number of drives on each controller to maximize throughput.

IMPORTANT: As with any hardware solution, using the same brand and model introduces the risk of concurrent failures over the life of the product, so plan maintenance accordingly.

The following table provides recommendations for the minimum and maximum number of storage objects to use when creating a software RAID:

Table 6-6 Recommended Number of Storage Objects to Use in the Software RAID

RAID Type

RAID 0 (striping) 2 8

Minimum Number of Storage Objects

Recommended Maximum Number of Storage Objects

RAID 1 (mirroring) 2 4

RAID 4 (striping with dedicated parity) 3 8

RAID 5 (striping with distributed parity) 3 8

Connection fault tolerance can be achieved by having multiple connection paths to each storage object in the RAID. For more information about configuring multipath I/O support before configuring a software RAID, see Chapter 5, “Managing Multipath I/O for Devices,” on page 43.

Managing Software RAIDs with EVMS 79

6.1.8 RAID 5 Algorithms for Distributing Stripes and Parity

RAID 5 uses an algorithm to determine the layout of stripes and parity. The following table describes the algorithms.

Table 6-7 RAID 5 Algorithms

Algorithm EVMS Type Description

Left Asymmetric 1 Stripes are written in a round-robin fashion from the first to last

member segment. The parity’s position in the striping sequence moves in a round-robin fashion from last to first. For example:

sda1 sdb1 sdc1 sde1

0 1 2 p

3 4 p 5

6 p 7 8

p 9 10 11

12 13 14 p

novdocx (en) 13 May 2009

Left Symmetric 2 This is the default setting and is considered the fastest method for

large reads.

Stripes wrap to follow the parity. The parity’s position in the striping sequence moves in a round-robin fashion from last to first. For example:

sda1 sdb1 sdc1 sde1

0 1 2 p

4 5 p 3

8 p 6 7

p 9 10 11

12 13 14 p

Right Asymmetric 3 Stripes are written in a round-robin fashion from the first to last

member segment. The parity’s position in the striping sequence moves in a round-robin fashion from first to last. For example:

sda1 sdb1 sdc1 sde1

p 0 1 2

3 p 4 5

6 7 p 8

9 10 11 p

p 12 13 14

Right Symmetric 4 Stripes wrap to follow the parity. The parity’s position in the striping

sequence moves in a round-robin fashion from first to last. For example:

sda1 sdb1 sdc1 sde1

p 0 1 2

5 p 3 4

7 8 p 6

9 10 11 p

p 12 13 14

80 SLES 10 SP2: Storage Administration Guide

For information about the layout of stripes and parity with each of these algorithms, see Linux

RAID-5 Algorithms (http://www.accs.com/p_and_p/RAID/LinuxRAID.html).

6.1.9 Multi-Disk Plug-In for EVMS

The Multi-Disk (MD) plug-in supports creating software RAIDs 0 (striping), 1 (mirror), 4 (striping with dedicated parity), and 5 (striping with distributed parity). The MD plug-in to EVMS allows you to manage all of these MD features as “regions” with the Regions Manager.

6.1.10 Device Mapper Plug-In for EVMS

The Device Mapper plug-in supports the following features in the EVMS MD Region Manager:

 Multipath I/O: Connection fault tolerance and load balancing for connections between the

server and disks where multiple paths are available. If you plan to use multipathing, you should configure MPIO for the devices that you plan to use in the RAID before configuring the RAID itself. For information, see Chapter 5, “Managing Multipath I/O for Devices,” on page 43.

IMPORTANT: The EVMS interface manages multipathing under the MD Region Manager, which originally supported the the interface and in naming of device nodes, but implements the storage objects with Device Mapper.

md multipath

functions. It uses the legacy md terminology in

novdocx (en) 13 May 2009

 Linear RAID: A linear concatenation of discontinuous areas of free space from the same or

multiple storage devices. Areas can be of different sizes.

 Snapshots: Snapshots of a file system at a particular point in time, even while the system is

active, thereby allowing a consistent backup.

The Device Mapper driver is not started by default in the rescue system.

root

1 Open a terminal console, then log in as the

2 Start the Device Mapper by entering the following at the terminal console prompt:

/etc/init.d/boot.device-mapper start

user or equivalent.

6.2 Creating and Configuring a Software RAID

1 Open a terminal console, then log in as the

2 Start the EVMS GUI by entering the following at the terminal console prompt:

evmsgui

3 If the disks have not been initialized, initialize them by adding the DOS Segment Manager

now.

The following instructions assume you are initializing new disks. For information about initializing an existing disk or a disk moved from another system, see Section 4.2, “Initializing

Disks,” on page 36.

root

user or equivalent.

Repeat the following steps for each disk that you want to initialize:

3a Select Actions > Add > Segment Manager to Storage Object.

3b From the list, select the DOS Segment Manager, then click Next.

3c Select the device, then click Add to initialize it.

Managing Software RAIDs with EVMS 81

4 If segments have not been created on the disks, create a segment on each disk that you plan to

use in the RAID.

For x86 platforms, this step is optional if you treat the entire disk as one segment.

For IA-64 platforms, this step is necessary to make the RAID 4/5 option available in the Regions Manager.

For information about creating segments, see Section 4.4, “Creating Disk Segments (or

Partitions),” on page 38.

4a Select Action > Create > Segment to open the DOS Segment Manager.

4b Select the free space segment you want to use.

4c Specify the amount of space to use for the segment.

4d Specify the segment options, then click Create.

5 Create and configure a software RAID Device.

5a Select Action > Create > Region to open the Create Storage Region dialog box.

novdocx (en) 13 May 2009

5b Specify the type of software RAID you want to create by selecting one of the following

Region Managers, then click Next.

 MD RAID 0 Region Manager

 MD RAID 1 Region Manager

 MD RAID 4/5 Region Manager

5c From the Storage Objects listed, select the ones to use for the RAID device.

IMPORTANT: The order of the objects in the RAID is implied by their order in the list.

82 SLES 10 SP2: Storage Administration Guide

novdocx (en) 13 May 2009

5d Specify values for Configuration Options by changing the following default settings as

desired.

 For RAIDs 1, 4, or 5, optionally specify a device to use as the spare disk for the

RAID. The default is none.

 For RAIDs 0, 4, or 5, specify the chunk (stripe) size in KB. The default is 32 KB.

 For RAIDs 4/5, specify RAID 4 or RAID 5 (default).

 For RAID 5, specify the algorithm to use for striping and parity. The default is Left

Symmetric.

For information about these settings, see “Configuration Options for RAIDs” on page 78.

Managing Software RAIDs with EVMS 83

novdocx (en) 13 May 2009

5e Click Create to create the RAID device under the

md0

The device is given a name such as

md0

, so its EVMS mount location is

/dev/evms/md

6 Specify a human-readable label for the device.

6a Select Action > Create > EVMS Volume or Compatible Volume.

6b Select the device that you created in Step 5.

6c Specify a name for the device.

Use standard ASCII characters and naming conventions. Spaces are allowed.

6d Click Done.

7 Create a file system on the RAID device you created.

7a Select Action > File System > Make to view a list of file system modules.

7b Select the type of file system you want to create, such as the following:

 ReiserFS File System Module

 Ext2/3FS File System Module

7c Select the RAID device you created in Step 5, such as

/dev/evms/md/md0

7d Specify a name to use as the Vol u m e L a bel, then click Make.

The name must not contain space or it will fail to mount later.

7e Click Save to create the file system.

8 Mount the RAID device.

directory.

/dev/evms/md/

8a Select Action > File System > Mount.

84 SLES 10 SP2: Storage Administration Guide

novdocx (en) 13 May 2009

8b Select the RAID device you created in Step 5, such as

8c Specify the location where you want to mount the device, such as

8d Click Mount.

9 Enable

10 Edit the

mount the device manually from

boot.evms

9a In YaST, select System > System Services (Run Level).

9b Select Expert Mode.

9c Select Boot.evms.

9d Select Set/Reset.

9e Select Enable the Service.

/etc/fstab

to activate EVMS automatically at reboot.

file to automount the RAID mount point created in Step 8c, or you can

evmsgui

/dev/evms/md/md0

/home

6.3 Expanding a RAID

This section explains how to expand a RAID by adding segments to it.

IMPORTANT: Before you can expand the size of a RAID device, you must deactivate it.

 Section 6.3.1, “Adding Mirrors to a RAID 1 Device,” on page 85

 Section 6.3.2, “Adding Segments to a RAID 4 or 5,” on page 86

6.3.1 Adding Mirrors to a RAID 1 Device

In a RAID 1 device, each member segment contains its own copy of all of the data stored in the RAID. You can add a mirror to the RAID to increase redundancy. The segment must be at least the same size as the smallest member segment in the existing RAID 1 device. Any excess space in the segment is not used. Ideally, all member segments of a RAID 1 device are the same size.

Adding an Available Segment as the New Mirror

1 Deactivate the RAID 1 device.

2 Use the Add Active (

3 From the list of available segments, select one that is the same size or larger than the smallest

existing member of the RAID device.

4 Reactivate the RAID device.

Activating A Spare Disk as the New Mirror

1 If you have not set up a spare disk, do it now.

For information, see Section 6.4, “Adding or Removing a Spare Disk,” on page 86.

2 Use the Activate Spare (

new mirror.

addactive

activatespare

plug-in) function.

plug-in) function to add it to the RAID 1 device as a

Managing Software RAIDs with EVMS 85

6.3.2 Adding Segments to a RAID 4 or 5

If the RAID region is clean and operating normally, the kernel driver adds the new object as a regular spare, and it acts as a hot standby for future failures. If the RAID region is currently degraded, the kernel driver immediately activates the new spare object and begins synchronizing the data and parity information.

6.4 Adding or Removing a Spare Disk

The MD driver allows you to optionally designate a spare disk (device, segment, or region) for RAID 1, 4, and 5 devices. You can assign a spare disk when you create the RAID or at any time thereafter. The RAID can be active and in use when you add or remove the spare. The spare is activated for the RAID only on disk failure.

 Section 6.4.1, “Do You Need a Spare Disk?,” on page 86

 Section 6.4.2, “Adding a Spare Disk When You Create the RAID,” on page 87

 Section 6.4.3, “Adding a Spare Disk to an Existing RAID,” on page 87

 Section 6.4.4, “Removing a Spare Disk from a RAID,” on page 87

novdocx (en) 13 May 2009

6.4.1 Do You Need a Spare Disk?

The advantage of specifying a spare disk for a RAID is that the system monitors the failure and begins recovery without human interaction. The disadvantage is that the space on the spare disk is not available until it is activated by a failed RAID.

As noted in “Overview of RAID Levels” on page 76, RAIDs 1, 4, and 5 can tolerate at least one disk failure. Any given RAID can have one spare disk designated for it, but the spare itself can serve as the designated spare for one RAID, for multiple RAIDs, or for all arrays. The spare disk is a hot standby until it is needed. It is not an active member of any RAIDs where it is assigned as the spare disk until it is activated for that purpose.

If a spare disk is defined for the RAID, the RAID automatically deactivates the failed disk and activates the spare disk on disk failure. The MD driver then begins synchronizing mirrored data for a RAID 1 or reconstructing the missing data and parity information for RAIDs 4 and 5. The I/O performance remains in a degraded state until the failed disk’s data is fully remirrored or reconstructed.

Creating a spare-group name allows a single hot spare to service multiple RAID arrays. The spare-

mdadm

group name can be any character string, but must be uniquely named for the server. For move spares from one array to another, the different arrays must be labelled with the same sparegroup name in the configuration file.

mdadm

For example, when if the array has a spare device. If no spare is available, mdadm looks in the array’s assigned sparegroup for another array that has a full complement of working drives and a spare. It attempts to remove the spare from the working array and add it to the degraded array. If the removal succeeds but the adding fails, then the spare is added back to its source array.

detects that an array is missing a component device, it first checks to see

86 SLES 10 SP2: Storage Administration Guide

6.4.2 Adding a Spare Disk When You Create the RAID

When you create a RAID 1, 4, or 5 in EVMS, specify the Spare Disk in the Configuration Options dialog box. You can browse to select the available device, segment, or region that you want to be the RAID’s spare disk. For information, see Step 5d in Section 6.2, “Creating and Configuring a

Software RAID,” on page 81.

6.4.3 Adding a Spare Disk to an Existing RAID

The RAID 1, 4, or 5 device can be active and in use when you add a spare disk to it. If the RAID is operating normally, the specified disk is added as a spare and it acts as a hot standby for future failures. If the RAID is currently degraded because of a failed disk, the specified disk is added as a spare disk, then it is automatically activated as a replacement disk for the failed disk, and it begins synchronizing the data and parity information.

1 Prepare a disk, segment, or region to use as the replacement disk, just as you did for the

component devices of the RAID device.

novdocx (en) 13 May 2009

2 In EVMS, select the Actions > Add > Spare Disk to a Region (the

EVMS GUI).

3 Select the RAID device you want to manage from the list of Regions, then click Next.

4 Select the device to use as the spare disk.

5 Click Add.

addspare

plug-in for the

6.4.4 Removing a Spare Disk from a RAID

The RAID 1, 4, or 5 device can be active and in use when you remove its spare disk.

1 In EVMS, select the Actions > Remove > Spare Disk from a Region (the

the EVMS GUI).

2 Select the RAID device you want to manage from the list of Regions, then click Next.

3 Select the spare disk.

4 Click Remove.

remspare

plug-in for

6.5 Managing Disk Failure and RAID Recovery

 Section 6.5.1, “Understanding the Disk Failure and RAID Recovery,” on page 87

 Section 6.5.2, “Identifying the Failed Drive,” on page 88

 Section 6.5.3, “Replacing a Failed Device with a Spare,” on page 89

 Section 6.5.4, “Removing the Failed Disk,” on page 90

6.5.1 Understanding the Disk Failure and RAID Recovery

RAIDs 1, 4, and 5 can survive a disk failure. A RAID 1 device survives if all but one mirrored array fails. Its read performance is degraded without the multiple data sources available, but its write performance might actually improve when it does not write to the failed mirrors. During the

Managing Software RAIDs with EVMS 87

synchronization of the replacement disk, write and read performance are both degraded. A RAID 5 can survive a single disk failure at a time. A RAID 4 can survive a single disk failure at a time if the disk is not the parity disk.

Disks can fail for many reasons such as the following:

 Disk crash

 Disk pulled from the system

 Drive cable removed or loose

 I/O errors

When a disk fails, the RAID removes the failed disk from membership in the RAID, and operates in a degraded mode until the failed disk is replaced by a spare. Degraded mode is resolved for a single disk failure in one of the following ways:

 Spare Exists: If the RAID has been assigned a spare disk, the MD driver automatically

activates the spare disk as a member of the RAID, then the RAID begins synchronizing (RAID

1) or reconstructing (RAID 4 or 5) the missing data.

 No Spare Exists: If the RAID does not have a spare disk, the RAID operates in degraded mode

until you configure and add a spare. When you add the spare, the MD driver detects the RAID’s degraded mode, automatically activates the spare as a member of the RAID, then begins synchronizing (RAID 1) or reconstructing (RAID 4 or 5) the missing data.

novdocx (en) 13 May 2009

6.5.2 Identifying the Failed Drive

On failure, md automatically removes the failed drive as a component device in the RAID array. To

mdadm

determine which device is a problem, use “removed”.

1 Enter the following a a terminal console prompt

mdadm -D /dev/md1

Replace

For example, an

/dev/md1

mdadm

with the actual path for your RAID.

report for a RAID 1 device consisting of

look like this:

blue6:~ # mdadm -D /dev/md1

/dev/md1:

Version : 00.90.03

Creation Time : Sun Jul 2 01:14:07 2006

Raid Level : raid1

Array Size : 180201024 (171.85 GiB 184.53 GB)

Device Size : 180201024 (171.85 GiB 184.53 GB)

and look for the device that has been reported as

/dev/sda2

and

/dev/sdb2

might

Raid Devices : 2

Total Devices : 1

Preferred Minor : 1

88 SLES 10 SP2: Storage Administration Guide

Persistence : Superblock is persistent

Update Time : Tue Aug 15 18:31:09 2006

State : clean, degraded

Active Devices : 1

Working Devices : 1 Failed Devices : 0

Spare Devices : 0

UUID : 8a9f3d46:3ec09d23:86e1ffbc:ee2d0dd8

Events : 0.174164

Number Major Minor RaidDevice State

0 0 0 0 removed

1 8 18 1 active sync /dev/sdb2

The “Total Devices : 1”, “Active Devices : 1”, and “Working Devices : 1” indicate that only one of the two devices is currently active. The RAID is operating in a “degraded” state.

novdocx (en) 13 May 2009

The “Failed Devices : 0” might be confusing. This setting has a non-zero number only for that brief

period where the

driver finds a problem on the drive and prepares to remove it from the RAID.

When the failed drive is removed, it reads “0” again.

In the devices list at the end of the report, the device with the “removed” state for Device 0 indicates that the device has been removed from the software RAID definition, not that the device has been physically removed from the system. It does not specifically identify the failed device. However, the working device (or devices) are listed. Hopefully, you have a record of which devices were members of the RAID. By the process of elimination, the failed device is

/dev/sda2

The “Spare Devices : 0” indicates that you do not have a spare assigned to the RAID. You must assign a spare device to the RAID so that it can be automatically added to the array and replace the failed device.

6.5.3 Replacing a Failed Device with a Spare

When a component device fails, the md driver replaces the failed device with a spare device assigned to the RAID. You can either keep a spare device assigned to the RAID as a hot standby to use as an automatic replacement, or assign a spare device to the RAID as needed.

IMPORTANT: Even if you correct the problem that caused the problem disk to fail, the RAID does not automatically accept it back into the array because it is a “faulty object” in the RAID and is no longer synchronized with the RAID.

If a spare is available, md automatically removes the failed disk, replaces it with the spare disk, then begins to synchronize the data (for RAID 1) or reconstruct the data from parity (for RAIDs 4 or 5).

If a spare is not available, the RAID operates in degraded mode until you assign spare device to the RAID.

Managing Software RAIDs with EVMS 89

To assign a spare device to the RAID:

1 Prepare the disk as needed to match the other members of the RAID.

novdocx (en) 13 May 2009

2 In EVMS, select the Actions > Add > Spare Disk to a Region (the

EVMS GUI).

3 Select the RAID device you want to manage from the list of Regions, then click Next.

4 Select the device to use as the spare disk.

5 Click Add.

driver automatically begins the replacement and reconstruction or synchronization

The process.

6 Monitor the status of the RAID to verify the process has begun.

For information about how monitor RAID status, see Section 6.6, “Monitoring Status for a

RAID,” on page 90.

7 Continue with Section 6.5.4, “Removing the Failed Disk,” on page 90.

addspare

plug-in for the

6.5.4 Removing the Failed Disk

You can remove the failed disk at any time after it has been replaced with the spare disk. EVMS does not make the device available for other use until you remove it from the RAID. After you remove it, the disk appears in the Available-Objects list in the EVMS GUI, where it can be used for any purpose.

NOTE: If you pull a disk or if it is totally unusable, EVMS no longer recognizes the failed disk as part of the RAID.

The RAID device can be active and in use when you remove its faulty object.

1 In EVMS, select the Actions > Remove > Faulty Object from a Region (the

the EVMS GUI).

2 Select the RAID device you want to manage from the list of Regions, then click Next.

3 Select the failed disk.

4 Click Remove.

remfaulty

6.6 Monitoring Status for a RAID

 Section 6.6.1, “Monitoring Status with EVMSGUI,” on page 90

 Section 6.6.2, “Monitoring Status with /proc/mdstat,” on page 91

 Section 6.6.3, “Monitoring Status with mdadm,” on page 91

 Section 6.6.4, “Monitoring a Remirror or Reconstruction,” on page 93

 Section 6.6.5, “Configuring mdadm to Send an E-Mail Alert for RAID Events,” on page 93

6.6.1 Monitoring Status with EVMSGUI

The Regions tab in EVMS GUI ( whether they are currently active.

evmsgui

) reports any software RAID devices that are defined and

plug-in

90 SLES 10 SP2: Storage Administration Guide

6.6.2 Monitoring Status with /proc/mdstat

novdocx (en) 13 May 2009

A summary of RAID and status information (active/not active) is available in the file.

root

1 Open a terminal console, then log in as the

2 View the

/proc/mdstat

file by entering the following at the console prompt:

user or equivalent.

cat /proc/mdstat

3 Evaluate the information.

The following table shows an example output and how to interpret the information.

Status Information Description Interpretation

Personalities : [raid5]

[raid4]

md0 : active

raid5

sdg1[0] sdk1[4] sdj1[3]

sdi1[2]

List of the RAIDs on the server by RAID label.

<device> : <active | not active> <RAID label you specified> < storage object> [RAID order]

You have two RAIDs defined with labels of

raid4

The RAID is active and

mounted at

md/md0

The RAID label is The active segments are

sdi1, sdj1

ordered in the RAID.

The RAID numbering of 0 to 4

indicates that the RAID has 5 segments, and the second segment the list. Based on the segment names, the missing segment is

[1]

/proc/mdstat

raid5

and

/dev/evms/

raid5

sdg1

, and

sdk1

, as

is missing from

sdh1

35535360 blocks level 5, 128k chunk, algorithm 2 [5/4] [U_UUU]

unused devices: <none>

<number of blocks> blocks level < 0 | 1 | 4 | 5 > <stripe size in KB> chunk algorithm <1 | 2 | 3 | 4 > [number of devices/number of

working devices]

[U-UUU]

All segments in the RAID are in use.

6.6.3 Monitoring Status with mdadm

To view the RAID status with the

mdadm -D /dev/mdx

mdadm

command, enter the following at a terminal prompt:

If the block size on the server is

4 KB, the total size of the RAID (including parity) is 142 GB, with a data capacity

of 113.7 GB. The stripe size is 128 KB. The RAID is using left

symmetric. algorithm <1 | 2 | 3 | 4 > [number of devices/number of

working devices] [U-UUU]

There are no spare devices available on the server.

Managing Software RAIDs with EVMS 91

Replace mdx with the RAID device number.

Example 1: A Disk Fails

novdocx (en) 13 May 2009

In the following example, only four of the five devices in the RAID are active (

Total Devices : 4

). When it was created, the component devices in the device were numbered 0

Raid Devices : 5

to 5 and are ordered according to their alphabetic appearance in the list where they were chosen, such as

/dev/sdg1, /dev/sdh1, /dev/sdi1, /dev/sdj1

filenames of the other devices, you determine that the device that was removed was named

sdh1

/dev/md0:

Version : 00.90.03

Creation Time : Sun Apr 16 11:37:05 2006

Raid Level : raid5

Array Size : 35535360 (33.89 GiB 36.39 GB)

Device Size : 8883840 (8.47 GiB 9.10 GB)

Raid Devices : 5

Total Devices : 4

Preferred Minor : 0

Persistence : Superblock is persistent

Update Time : Mon Apr 17 05:50:44 2006

State : clean, degraded

Active Devices : 4

Working Devices : 4

Failed Devices : 0

Spare Devices : 0

Layout : left-symmetric

Chunk Size : 128K

UUID : 2e686e87:1eb36d02:d3914df8:db197afe

Events : 0.189

Number Major Minor RaidDevice State

0 8 97 0 active sync /dev/sdg1

1 8 0 1 removed

2 8 129 2 active sync /dev/sdi1

3 8 45 3 active sync /dev/sdj1

4 8 161 4 active sync /dev/sdk1

, and

/dev/sdk1

. From the pattern of

/dev/

Example 2: Spare Disk Replaces the Failed Disk

In the following

Devices : 4

from the RAID (

mdadm

report, only 4 of the 5 disks are active and in good condition (

Working Devices : 5

Failed Devices: 0

assumed the diskname of the failed disk ( removed from the RAID) is not identified in the report. The RAID is running in degraded mode

State : clean, degraded, recovering

(

dev/sdh1

), and the process is 3% complete (

92 SLES 10 SP2: Storage Administration Guide

Active

). The failed disk was automatically detected and removed

). The spare was activated as the replacement disk, and has

/dev/sdh1

Rebuild Status : 3% complete

). The faulty object (the failed disk that was

). The data is being rebuilt (

spare rebuilding /

mdadm -D /dev/md0

/dev/md0:

Version : 00.90.03

Creation Time : Sun Apr 16 11:37:05 2006

Raid Level : raid5

Raid Devices : 5

Total Devices : 5

Preferred Minor : 0

Persistence : Superblock is persistent

Update Time : Mon Apr 17 05:50:44 2006

State : clean, degraded, recovering

Active Devices : 4

Working Devices : 5

Failed Devices : 0

Spare Devices : 1

Rebuild Status : 3% complete

UUID : 2e686e87:1eb36d02:d3914df8:db197afe

Events : 0.189

Number Major Minor RaidDevice State

0 8 97 0 active sync /dev/sdg1

1 8 113 1 spare rebuilding /dev/sdh1

2 8 129 2 active sync /dev/sdi1

3 8 145 3 active sync /dev/sdj1

4 8 161 4 active sync /dev/sdk1

Array Size : 35535360 (33.89 GiB 36.39 GB)

Device Size : 8883840 (8.47 GiB 9.10 GB)

Layout : left-symmetric

Chunk Size : 128K

novdocx (en) 13 May 2009

6.6.4 Monitoring a Remirror or Reconstruction

You can follow the progress of the synchronization or reconstruction process by examining the

proc/mdstat

You can control the speed of synchronization by setting parameters in the

speed_limit_min

echo a larger number into the

file.

and

/proc/sys/dev/raid/speed_limit_max

speed_limit_min

file.

/proc/sys/dev/raid/

files. To speed up the process,

6.6.5 Configuring mdadm to Send an E-Mail Alert for RAID Events

You might want to configure the Monitoring is only meaningful for RAIDs 1, 4, 5, 6, 10 or multipath arrays because only these have missing, spare, or failed drives to monitor. RAID 0 and Linear RAIDs do not provide fault tolerance so they have no interesting states to monitor.

mdadm

service to send an e-mail alert for software RAID events.

Managing Software RAIDs with EVMS 93

The following table identifies RAID events and indicates which events trigger e-mail alerts. All events cause the program to run. The program is run with two or three arguments: the event name, the array device (such as

/dev/md1

), and possibly a second device. For Fail, Fail Spare, and Spare Active, the second device is the relevant component device. For MoveSpare, the second device is the array that the spare was moved from.

Table 6-8 RAID Events in mdadm

novdocx (en) 13 May 2009

RAID Event

Device Disappeared No An md array that was previously configured appears to no longer be

Rebuild Started No An

Rebuild NN No Where NN is 20, 40, 60, or 80. This indicates the percent completed

Rebuild Finished No An

Fail Yes An active component device of an array has been marked as faulty.

Fail Spare Yes A spare component device that was being rebuilt to replace a faulty

Spare Active No A spare component device that was being rebuilt to replace a faulty

Trigger E-Mail Alert

Description

configured. (syslog priority: Critical)

If mdadm was told to monitor an array which is RAID0 or Linear, then it reports DeviceDisappeared with the extra information Wrong-Level. This is because RAID0 and Linear do not support the device-failed, hot-spare, and resynchronize operations that are monitored.

array started reconstruction. (syslog priority: Warning)

for the rebuild. (syslog priority: Warning)

array that was rebuilding is no longer rebuilding, either because it finished normally or was aborted. (syslog priority: Warning)

(syslog priority: Critical)

device has failed. (syslog priority: Critical)

device has been successfully rebuilt and has been made active. (syslog priority: Info)

New Array No A new

(syslog priority: Info)

Degraded Array Yes A newly noticed array appears to be degraded. This message is not

generated when degradation. It is generated only when degraded when it first sees the array. (syslog priority: Critical)

Move Spare No A spare drive has been moved from one array in a spare group to

another to allow a failed drive to be replaced. (syslog priority: Info)

Spares Missing Yes The

number of spare devices, but than this number when it first sees the array. (syslog priority: Warning)

Test Message Yes An array was found at startup, and the

(syslog priority: Info)

94 SLES 10 SP2: Storage Administration Guide

array has been detected in the

mdadm

notices a drive failure that causes

mdadm.conf

file indicates that an array should have a certain

mdadm

/proc/mdstat

mdadm

notices that an array is

detects that the array has fewer

--test

flag was given.

file.

To configure an e-mail alert:

novdocx (en) 13 May 2009

1 At a terminal console, log in as the

2 Edit the

/etc/mdadm/mdadm.conf

root

user.

file to add your e-mail address for receiving alerts. For

example, specify the MAILADDR value (using your own e-mail address, of course):

DEVICE partitions

ARRAY /dev/md0 level=raid1 num-devices=2

UUID=1c661ae4:818165c3:3f7a4661:af475fda

devices=/dev/sdb3,/dev/sdc3

MAILADDR yourname@example.com

The MAILADDR line gives an e-mail address that alerts should be sent to when mdadm is running in line in

3 Start

mdadm --monitor --mail=yourname@example.com --delay=1800 /dev/md0

The

--monitor

mdadm.conf

mdadm

monitoring by entering the following at the terminal console prompt:

--monitor

on any events noticed.

mode with the

--scan

option. There should be only one MAILADDR

, and it should have only one address.

option causes

mdadm

to periodically poll a number of md arrays and to report

never exits once it decides that there are arrays to be checked, so

it should normally be run in the background.

In addition to reporting events in this mode,

mdadm

might move a spare drive from one array to another if they are in the same spare-group and if the destination array has a failed drive but no spares.

Listing the devices to monitor is optional. If any devices are listed on the command line, monitors only those devices. Otherwise, all arrays listed in the configuration file are monitored. Further, if

proc/mdstat

--scan

For more information about using

option is added in the command, then any other md devices that appear in

are also monitored.

mdadm

, see the

mdadm(8)

and

mdadm.conf(5)

man pages.

mdadm

4 To configure the

suse:~ # egrep 'MAIL|RAIDDEVICE' /etc/sysconfig/mdadm

MDADM_MAIL="yourname@example.com"

MDADM_RAIDDEVICES="/dev/md0"

MDADM_SEND_MAIL_ON_START=no

suse:~ # chkconfig mdadmd --list

mdadmd 0:off 1:off 2:off 3:on 4:off 5:on 6:off

/etc/init.d/mdadmd

service as a script:

6.7 Deleting a Software RAID and Its Data

If you want to remove the prior multipath settings, deactivate the RAID, delete the data on the RAID, and release all resources used by the RAID, do the following:

1 If you want to keep the data stored on the software RAID device, make sure to back up the data

to alternate media, using your normal backup procedures. Make sure the backup is good before proceeding.

root

2 Open a terminal console prompt as the

commands described in the remaining steps.

3 Dismount the software RAID device by entering

user or equivalent. Use this console to enter the

Managing Software RAIDs with EVMS 95

umount <raid-device>

4 Stop the RAID device and its component devices by entering

mdadm --stop <raid-device>

mdadm --stop <member-devices>

For more information about using

mdadm

, please see the

mdadm(8)

man page.

5 Delete all data on the disk by literally overwriting the entire device with zeroes. Enter

mdadm --misc --zero-superblock <member-devices>

6 You must now reinitialize the disks for other uses, just as you would when adding a new disk to

your system.

novdocx (en) 13 May 2009

96 SLES 10 SP2: Storage Administration Guide

Managing Software RAIDs 6 and

novdocx (en) 13 May 2009

10 with mdadm

This section describes how to create software RAID 6 and 10 devices, using the Multiple Devices Administration ( tool provides the functionality of legacy programs

 Section 7.1, “Creating a RAID 6,” on page 97

 Section 7.2, “Creating Nested RAID 10 Devices with mdadm,” on page 98

 Section 7.3, “Creating a Complex RAID 10 with mdadm,” on page 101

 Section 7.4, “Creating a Degraded RAID Array,” on page 104

mdadm(8)

7.1 Creating a RAID 6

 Section 7.1.1, “Understanding RAID 6,” on page 97

 Section 7.1.2, “Creating a RAID 6,” on page 98

7.1.1 Understanding RAID 6

RAID 6 is essentially an extension of RAID 5 that allows for additional fault tolerance by using a second independent distributed parity scheme (dual parity). Even if one of the hard disk drives fails during the data recovery process, the system continues to be operational, with no data loss.

) tool. You can also use

mdadm

to create RAIDs 0, 1, 4, and 5. The

mdtools

and

raidtools

mdadm

RAID6 provides for extremely high data fault tolerance by sustaining multiple simultaneous drive failures. It handles the loss of any two devices without data loss. Accordingly, it requires N+2 drives to store N drives worth of data. It requires a minimum of 4 devices.

The performance for RAID 6 is slightly lower but comparable to RAID 5 in normal mode and single disk failure mode. It is very slow in dual disk failure mode.

Table 7-1 Comparison of RAID 5 and RAID 6

Feature RAID 5 RAID 6

Number of devices N+1, minimum of 3 N+2, minimum of 4

Parity Distributed, single Distributed, dual

Performance Medium impact on write and

rebuild

Fault-tolerance Failure of one component device Failure of two component devices

More impact on sequential write than RAID 5

Managing Software RAIDs 6 and 10 with mdadm

7.1.2 Creating a RAID 6

novdocx (en) 13 May 2009

The procedure in this section creates a RAID 6 device

dev/sdb1

device nodes.

1 Open a terminal console, then log in as the

2 Create a RAID 6 device. At the command prompt, enter

3 Create a file system on the RAID 6 device

4 Edit the

5 Edit the

6 Reboot the server.

7 (Optional) Add a hot spare to service the RAID array. For example, at the command prompt

/dev/sdc1

mdadm --create /dev/md0 --run --level=raid6 --chunk=128 --raid-devices=4 / dev/sdb1 /dev/sdc1 /dev/sdc1 /dev/sdd1

The default chunk size is 64 (KB).

For example, at the command prompt, enter

mkfs.reiserfs /dev/md0

Modify the command if you want to use a different file system.

/etc/mdadm.conf

/dev/md0

The RAID 6 device is mounted to

enter:

mdadm /dev/md0 -a /dev/sde1

/etc/fstab

, and

/dev/sdd1

file to add entries for the component devices and the RAID device

file to add an entry for the RAID 6 device

. Make sure to modify the procedure to use your actual

/local

/dev/md0

root

user or equivalent.

/dev/md0

, such as a Reiser file system (reiserfs).

with four devices:

/dev/md0

/dev/sda1, /

7.2 Creating Nested RAID 10 Devices with mdadm

 Section 7.2.1, “Understanding Nested RAID Devices,” on page 98

 Section 7.2.2, “Creating Nested RAID 10 (1+0) with mdadm,” on page 99

 Section 7.2.3, “Creating Nested RAID 10 (0+1) with mdadm,” on page 100

7.2.1 Understanding Nested RAID Devices

A nested RAID device consists of a RAID array that uses another RAID array as its basic element, instead of using physical disks. The goal of this configuration is to improve the performance and fault tolerance of the RAID.

Linux supports nesting of RAID 1 (mirroring) and RAID 0 (striping) arrays. Generally, this combination is referred to as RAID 10. To distinguish the order of the nesting, this document uses the following terminology:

 RAID 1+0: RAID 1 (mirror) arrays are built first, then combined to form a RAID 0 (stripe)

array.

 RAID 0+1: RAID 0 (stripe) arrays are built first, then combined to form a RAID 1 (mirror)

array.

98 SLES 10 SP2: Storage Administration Guide

The following table describes the advantages and disadvantages of RAID 10 nesting as 1+0 versus 0+1. It assumes that the storage objects you use reside on different disks, each with a dedicated I/O capability.

Table 7-2 RAID Levels Supported in EVMS

RAID Level Description Performance and Fault Tolerance

novdocx (en) 13 May 2009

10 (1+0) RAID 0 (stripe)

built with RAID 1 (mirror) arrays

10 (0+1) RAID 1 (mirror)

built with RAID 0 (stripe) arrays

RAID 1+0 provides high levels of I/O performance, data redundancy, and disk fault tolerance. Because each member device in the RAID 0 is mirrored individually, multiple disk failures can be tolerated and data remains available as long as the disks that fail are in different mirrors.

You can optionally configure a spare for each underlying mirrored array, or configure a spare to serve a spare group that serves all mirrors.

RAID 0+1 provides high levels of I/O performance and data redundancy, but slightly less fault tolerance than a 1+0. If multiple disks fail on one side of the mirror, then the other mirror is available. However, if disks are lost concurrently on both sides of the mirror, all data is lost.

This solution offers less disk fault tolerance than a 1+0 solution, but if you need to perform maintenance or maintain the mirror on a different site, you can take an entire side of the mirror offline and still have a fully functional storage device. Also, if you lose the connection between the two sites, either site operates independently of the other. That is not true if you stripe the mirrored segments, because the mirrors are managed at a lower level.

If a device fails, the mirror on that side fails because RAID 1 is not faulttolerant. Create a new RAID 0 to replace the failed side, then resynchronize the mirrors.

7.2.2 Creating Nested RAID 10 (1+0) with mdadm

A nested RAID 1+0 is built by creating two or more RAID 1 (mirror) devices, then using them as component devices in a RAID 0.

IMPORTANT: If you need to manage multiple connections to the devices, you must configure multipath I/O before configuring the RAID devices. For information, see Chapter 5, “Managing

Multipath I/O for Devices,” on page 43.

The procedure in this section uses the device names shown in the following table. Make sure to modify the device names with the names of your own devices.

Managing Software RAIDs 6 and 10 with mdadm 99

Table 7-3 Scenario for Creating a RAID 10 (1+0) by Nesting

Raw Devices RAID 1 (mirror) RAID 1+0 (striped mirrors)

/dev/sdb1 /dev/md0 /dev/md2

/dev/sdc1

/dev/sdd1 /dev/md1

/dev/sde1

1 Open a terminal console, then log in as the root user or equivalent.

2 Create 2 software RAID 1 devices, using two different devices for each RAID 1 device. At the

command prompt, enter these two commands:

mdadm --create /dev/md0 --run --level=1 --raid-devices=2 /dev/sdb1 /dev/ sdc1

mdadm --create /dev/md1 --run --level=1 --raid-devices=2 /dev/sdd1 /dev/ sde1

3 Create the nested RAID 1+0 device. At the command prompt, enter the following command

using the software RAID 1 devices you created in Step 2:

mdadm --create /dev/md2 --run --level=0 --chunk=64 --raid-devices=2 /dev/ md0 /dev/md1

The default chunk size is 64 KB.

4 Create a file system on the RAID 1+0 device

/dev/md2

, such as a Reiser file system (reiserfs).

For example, at the command prompt, enter

mkfs.reiserfs /dev/md2

Modify the command if you want to use a different file system.

novdocx (en) 13 May 2009

5 Edit the

/dev/md2

6 Edit the

/etc/mdadm.conf

/etc/fstab

file to add entries for the component devices and the RAID device

file to add an entry for the RAID 1+0 device

/dev/md2

7 Reboot the server.

The RAID 1+0 device is mounted to

/local

8 (Optional) Add hot spares to service the underlying RAID 1 mirrors.

For information, see Section 6.4, “Adding or Removing a Spare Disk,” on page 86.

7.2.3 Creating Nested RAID 10 (0+1) with mdadm

A nested RAID 0+1 is built by creating two to four RAID 0 (striping) devices, then mirroring them as component devices in a RAID 1.

IMPORTANT: If you need to manage multiple connections to the devices, you must configure multipath I/O before configuring the RAID devices. For information, see Chapter 5, “Managing

Multipath I/O for Devices,” on page 43.

In this configuration, spare devices cannot be specified for the underlying RAID 0 devices because RAID 0 cannot tolerate a device loss. If a device fails on one side of the mirror, you must create a replacement RAID 0 device, than add it into the mirror.

100 SLES 10 SP2: Storage Administration Guide

Novell SUSE Linux Enterprise 10 Storage Administration Guide

Specifications and Main Features

Frequently Asked Questions

User Manual

SLES 10 SP2: Storage Administration Guide

About This Guide

Overview of EVMS

1.1 Benefits of EVMS

1.2 Plug-In Layers

1.3 Supported File Systems

1.4 Terminology

1.5 Location of Device Nodes for EVMS Storage Objects

Using EVMS to Manage Devices

2.1 Configuring the System Device at Install to Use EVMS

2.1.1 Before the Install

2.1.2 During the Server Install

2.1.3 After the Server Install

2.2 Configuring an Existing System Device to Use EVMS

2.2.1 Disable the boot.lvm and boot.md Services

2.2.2 Enable the boot.evms Service

2.2.3 Edit the /etc/fstab File

2.2.4 Edit the Boot Loader File

2.2.5 Force the RAM Disk to Recognize the Root Partition

2.2.6 Restart the Server

2.2.7 Verify that EVMS Manages the Boot, Swap, and Root Partitions

2.3 Configuring LVM Devices to Use EVMS

2.4 Using EVMS with iSCSI Volumes

2.5 Using the ELILO Loader Files (IA-64)

2.6 Starting EVMS

2.7 Starting the EVMS Management Tools

Using UUIDs to Mount Devices

3.1 Naming Devices with udev

3.2 Understanding UUIDs

3.2.1 Using UUIDs to Assemble or Activate File System Devices

3.2.2 Finding the UUID for a File System Device

3.3 Using UUIDs in the Boot Loader and /etc/ fstab File (x86)

3.4 Using UUIDs in the Boot Loader and /etc/ fstab File (IA64)

3.5 Additional Information

Managing EVMS Devices

4.1 Understanding Disk Segmentation

4.1.1 Segment Managers

4.1.2 Disk Segments

4.2 Initializing Disks

4.2.1 Before You Begin

4.2.2 Guidelines

4.2.3 Adding a Segment Manager

4.3 Removing the Segment Manager from a Device

4.4 Creating Disk Segments (or Partitions)

4.5 Configuring Mount Options for Devices

4.6 What’s Next

5.1 Understanding Multipathing

5.1.1 What Is Multipathing?

5.1.2 Benefits of Multipathing

Managing Multipath I/O for Devices

5.2 Planning for Multipathing

5.2.1 Guidelines for Multipathing

5.2.2 Using Multipathed Devices Directly or in EVMS

5.2.3 Using LVM2 on Multipath Devices

5.2.4 Using mdadm with Multipath Devices

5.2.5 Partitioning Multipath Devices

5.2.6 Supported Architectures for Multipath I/O

5.2.7 Supported Storage Arrays for Multipathing

5.3 Multipath Management Tools

5.3.1 Device Mapper Multipath Module

5.3.2 Multipath I/O Management Tools

5.3.3 Using mdadm for Multipathed Devices

5.3.4 The Linux multipath(8) Command

5.4 Configuring the System for Multipathing

5.4.1 Preparing SAN Devices for Multipathing

5.4.2 Partitioning Multipathed Devices

5.4.3 Configuring the Server for Multipathing

5.4.4 Adding multipathd to the Boot Sequence

5.4.5 Creating and Configuring the /etc/multipath.conf File

5.5 Enabling and Starting Multipath I/O Services

5.6 Configuring Path Failover Policies and Priorities

5.6.1 Configuring the Path Failover Policies

5.6.2 Configuring Failover Priorities

5.6.3 Using a Script to Set Path Priorities

5.6.4 Configuring ALUA

5.6.5 Reporting Target Path Groups