HP StorageWorks Scalable File Share User Manual
HP Scalable File Share User's Guide
G3.1-0
HP Part Number: SFSUGG31-E
Published: June 2009
Edition: 5
© Copyright 2009 Hewlett-Packard Development Company, L.P.
Confidential computersoftware. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211and 12.212,Commercial
Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under
vendor's standardcommercial license.The informationcontained hereinis subject to change without notice. The only warranties forHP products
and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as
constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
Intel, Intel Xeon, and Itanium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the U.S. and other countries.
InfiniBand is a registered trademark and service mark of the InfiniBand Trade Association.
Lustre and the Lustre Logo are trademarks of Sun Microsystems.
Myrinet and Myricom are registered trademarks of Myricom Inc.
Quadrics and QsNetII are trademarks or registered trademarks of Quadrics, Ltd.
UNIX is a registered trademark of The Open Group.
Voltaire, ISR 9024, Voltaire HCA 400, and VoltaireVision are all registered trademarks of Voltaire, Inc.
Red Hat is a registered trademark of Red Hat, Inc.
Fedora is a trademark of Red Hat, Inc.
SUSE is a registered trademark of SUSE AG, a Novell business.
AMD Opteron is a trademark of Advanced Micro Devices, Inc.
Sun and Solaris are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.
Table of Contents
About This Document.........................................................................................................9
Intended Audience.................................................................................................................................9
New and Changed Information in This Edition.....................................................................................9
Typographic Conventions......................................................................................................................9
Related Information..............................................................................................................................10
Structure of This Document..................................................................................................................11
Documentation Updates.......................................................................................................................11
HP Encourages Your Comments..........................................................................................................11
1 What's In This Version.................................................................................................13
1.1 About This Product.........................................................................................................................13
1.2 Benefits and Features......................................................................................................................13
1.3 Supported Configurations ..............................................................................................................13
1.3.1 Hardware Configuration.........................................................................................................14
1.3.1.1 Fibre Channel Switch Zoning..........................................................................................16
1.4 Server Security Policy......................................................................................................................16
2 Installing and Configuring MSA Arrays.....................................................................19
2.1 Installation.......................................................................................................................................19
2.2 Accessing the MSA2000fc CLI.........................................................................................................19
2.3 Using the CLI to Configure Multiple MSA Arrays.........................................................................19
2.3.1 Configuring New Volumes.....................................................................................................19
2.3.2 Creating New Volumes...........................................................................................................20
3 Installing and Configuring HP SFS Software on Server Nodes..............................23
3.1 Supported Firmware ......................................................................................................................24
3.2 Installation Requirements...............................................................................................................25
3.2.1 Kickstart Template Editing......................................................................................................25
3.3 Installation Phase 1..........................................................................................................................26
3.3.1 DVD/NFS Kickstart Procedure................................................................................................26
3.3.2 DVD/USB Drive Kickstart Procedure.....................................................................................27
3.3.3 Network Installation Procedure..............................................................................................28
3.4 Installation Phase 2..........................................................................................................................28
3.4.1 Patch Download and Installation Procedure..........................................................................29
3.4.2 Run the install2.sh Script.................................................................................................29
3.4.3 10 GigE Installation.................................................................................................................29
3.5 Configuration Instructions..............................................................................................................30
3.5.1 Configuring Ethernet and InfiniBand or 10 GigE Interfaces..................................................30
3.5.2 Creating the /etc/hosts file................................................................................................30
3.5.3 Configuring pdsh...................................................................................................................31
3.5.4 Configuring ntp......................................................................................................................31
3.5.5 Configuring User Credentials.................................................................................................31
3.5.6 Verifying Digital Signatures (optional)...................................................................................32
3.5.6.1 Verifying the HP Public Key (optional)..........................................................................32
3.5.6.2 Verifying the Signed RPMs (optional)............................................................................32
3.6 Upgrade Installation........................................................................................................................32
3.6.1 Rolling Upgrades.....................................................................................................................33
3.6.2 Client Upgrades.......................................................................................................................35
Table of Contents 3
4 Installing and Configuring HP SFS Software on Client Nodes...............................37
4.1 Installation Requirements...............................................................................................................37
4.1.1 Client Operating System and Interconnect Software Requirements......................................37
4.1.2 InfiniBand Clients....................................................................................................................37
4.1.3 10 GigE Clients........................................................................................................................37
4.2 Installation Instructions...................................................................................................................38
4.3 Custom Client Build Procedures.....................................................................................................39
4.3.1 CentOS 5.2/RHEL5U2 Custom Client Build Procedure..........................................................39
4.3.2 SLES10 SP2 Custom Client Build Procedure...........................................................................39
5 Using HP SFS Software................................................................................................41
5.1 Creating a Lustre File System..........................................................................................................41
5.1.1 Creating the Lustre Configuration CSV File...........................................................................41
5.1.1.1 Multiple File Systems......................................................................................................43
5.1.2 Creating and Testing the Lustre File System...........................................................................43
5.2 Configuring Heartbeat....................................................................................................................44
5.2.1 Preparing Heartbeat................................................................................................................45
5.2.2 Generating Heartbeat Configuration Files Automatically......................................................45
5.2.3 Configuration Files..................................................................................................................45
5.2.3.1 Generating the cib.xml File..........................................................................................47
5.2.3.2 Editing cib.xml.............................................................................................................47
5.2.4 Copying Files...........................................................................................................................47
5.2.5 Starting Heartbeat...................................................................................................................48
5.2.6 Monitoring Failover Pairs........................................................................................................48
5.2.7 Moving and Starting Lustre Servers Using Heartbeat............................................................48
5.2.8 Things to Double-Check..........................................................................................................49
5.2.9 Things to Note.........................................................................................................................49
5.3 Starting the File System...................................................................................................................49
5.4 Stopping the File System.................................................................................................................50
5.5 Testing Your Configuration.............................................................................................................50
5.5.1 Examining and Troubleshooting.............................................................................................50
5.5.1.1 On the Server...................................................................................................................50
5.5.1.2 The writeconf Procedure.................................................................................................52
5.5.1.3 On the Client...................................................................................................................53
5.6 Lustre Performance Monitoring......................................................................................................54
6 Licensing........................................................................................................................55
6.1 Checking for a Valid License...........................................................................................................55
6.2 Obtaining a New License................................................................................................................55
6.3 Installing a New License.................................................................................................................55
7 Known Issues and Workarounds................................................................................57
7.1 Server Reboot...................................................................................................................................57
7.2 Errors from install2....................................................................................................................57
7.3 Application File Locking.................................................................................................................57
7.4 MDS Is Unresponsive......................................................................................................................57
7.5 Changing group_upcall Value to Disable Group Validation.....................................................57
7.6 Configuring the mlocate Package on Client Nodes......................................................................58
7.7 System Behavior After LBUG..........................................................................................................58
4 Table of Contents
A HP SFS G3 Performance.............................................................................................59
A.1 Benchmark Platform.......................................................................................................................59
A.2 Single Client Performance..............................................................................................................60
A.3 Throughput Scaling........................................................................................................................62
A.4 One Shared File..............................................................................................................................64
A.5 Stragglers and Stonewalling...........................................................................................................64
A.6 Random Reads................................................................................................................................65
Index.................................................................................................................................67
Table of Contents 5
List of Figures
1-1 Platform Overview........................................................................................................................15
1-2 Server Pairs....................................................................................................................................16
A-1 Benchmark Platform......................................................................................................................59
A-2 Storage Configuration...................................................................................................................60
A-3 Single Stream Throughput............................................................................................................61
A-4 Single Client, Multi-Stream Write Throughput.............................................................................61
A-5 Writes Slow When Cache Fills.......................................................................................................62
A-6 Multi-Client Throughput Scaling..................................................................................................63
A-7 Multi-Client Throughput and File Stripe Count...........................................................................63
A-8 Stonewalling..................................................................................................................................64
A-9 Random Read Rate........................................................................................................................65
6 List of Figures
List of Tables
1-1 Supported Configurations ............................................................................................................13
3-1 Minimum Firmware Versions.......................................................................................................24
7
8
About This Document
This document provides installation and configuration information for HP Scalable File Share
(SFS) G3.1-0. Overviews of installing and configuring the Lustre® File System and MSA2000
Storage Arrays are also included in this document.
Pointers to existing documents are provided where possible. Refer to those documents for related
information.
Intended Audience
This document is intended for anyone who installs and uses HP SFS. The information in this
guide assumes that you have experience with the following:
• The Linux operating system and its user commands and tools
• The Lustre File System
• Smart Array storage administration
• HP rack-mounted servers and associated rack hardware
• Basic networking concepts, network switch technology, and network cables
New and Changed Information in This Edition
• CentOS 5.2 support
• Lustre 1.6.7 support
• 10 GigE support
• License checking
• Upgrade path
Typographic Conventions
This document uses the following typographical conventions:
%, $, or #
audit(5) A manpage. The manpage name is audit, and it is located in
Command
Computer output
Ctrl+x A key sequence. A sequence such as Ctrl+x indicates that you
ENVIRONMENT VARIABLE The name of an environment variable, for example, PATH.
[ERROR NAME]
Key The name of a keyboard key. Return and Enter both refer to the
Term The defined use of an important word or phrase.
User input
Variable
[] The contents are optional in syntax. If the contents are a list
A percent sign represents the C shell system prompt. A dollar
sign represents the system prompt for the Bourne, Korn, and
POSIX shells. A number sign represents the superuser prompt.
Section 5.
A command name or qualified command phrase.
Text displayed by the computer.
must hold down the key labeled Ctrl while you press another
key or mouse button.
The name of an error, usually returned in the errno variable.
same key.
Commands and other text that you type.
The name of a placeholder in a command, function, or other
syntax display that you replace with an actual value.
separated by |, you must choose one of the items.
Intended Audience 9
{} The contents are required in syntax. If the contents are a list
... The preceding element can be repeated an arbitrary number of
\ Indicates the continuation of a code example.
| Separates items in a list of choices.
WARNING A warning calls attention to important information that if not
CAUTION A caution calls attention to important information that if not
IMPORTANT This alert provides essential information to explain a concept or
NOTE A note contains additional information to emphasize or
Related Information
Pointers to existing documents are provided where possible. Refer to those documents for related
information.
For Sun Lustre documentation, see:
http://manual.lustre.org
separated by |, you must choose one of the items.
times.
understood or followed will result in personal injury or
nonrecoverable system problems.
understood or followed will result in data loss, data corruption,
or damage to hardware or software.
to complete a task.
supplement important points of the main text.
The Lustre 1.6 Operations Manual is installed on the system in /opt/hp/sfs/doc/
LustreManual_v1_15.pdf. Or refer to the Lustre website:
http://manual.lustre.org/images/8/86/820-3681_v15.pdf
For HP XC Software documentation, see:
http://docs.hp.com/en/linuxhpc.html
For MSA2000 products, see:
http://www.hp.com/go/msa2000
For HP servers, see:
http://www.hp.com/go/servers
For InfiniBand information, see:
http://www.hp.com/products1/serverconnectivity/adapters/infiniband/specifications.html
For Fibre Channel networking, see:
http://www.hp.com/go/san
For HP support, see:
http://www.hp.com/support
For product documentation, see:
http://www.hp.com/support/manuals
For collectl documentation, see:
http://collectl.sourceforge.net/Documentation.html
For Heartbeat information, see:
http://www.linux-ha.org/Heartbeat
For HP StorageWorks Smart Array documentation, see:
HP StorageWorks Smart Array Manuals
10
For SFS Gen 3 Cabling Tables, see: http://docs.hp.com/en/storage.html and click the Scalable File
Share (SFS) link.
For SFS V2.3 Release Notes, see:
HP StorageWorks Scalable File Share Release Notes Version 2.3
For documentation of previous versions of HP SFS, see:
• HP StorageWorks Scalable File Share Client Installation and User Guide Version 2.2 at:
http://docs.hp.com/en/8957/HP_StorageWorks_SFS_Client_V2_2-0.pdf
Structure of This Document
This document is organized as follows:
Chapter 1 Provides information about what is included in this product.
Chapter 2 Provides information about installing and configuring MSA2000fc arrays.
Chapter 3 Provides information about installing and configuring the HP SFS Software on the
server nodes.
Chapter 4 Provides information about installing and configuring the HP SFS Software on the
client nodes.
Chapter 5 Provides information about using the HP SFS Software.
Chapter 6 Provides information about licensing.
Chapter 7 Provides information about known issues and workarounds.
Appendix A Provides performance data.
Documentation Updates
Documentation updates (if applicable) are provided on docs.hp.com. Use the release date of a
document to determine that you have the latest version.
HP Encourages Your Comments
HP encourages your comments concerning this document. We are committed to providing
documentation that meets your needs. Send any errors found, suggestions for improvement, or
compliments to:
http://docs.hp.com/en/feedback.html
Include the document title, manufacturing part number, and any comment, error found, or
suggestion for improvement you have concerning this document.
Structure of This Document 11
12
1 What's In This Version
1.1 About This Product
HP SFS G3.1-0 uses the Lustre File System on MSA2000fc hardware to provide a storage system
for standalone servers or compute clusters.
Starting with this release, HP SFS servers can be upgraded. If you are upgrading from one version
of HP SFS G3 to a more recent version, see the instructions in “Upgrade Installation” (page 32).
IMPORTANT: If you are upgrading from HP SFS version 2.3 or older, you must contact your
HP SFS 2.3 support representative to obtain the extra documentation and tools necessary for
completing that upgrade. The upgrade from HP SFS version 2.x to HP SFS G3 cannot be done
successfully with just the HP SFS G3 CD and the user's guide.
HP SFS 2.3 to HP SFS G3 upgrade documentation and tools change regularly and independently
of the HP SFS G3 releases. Verify that you have the latest available versions.
If you are upgrading from one version of HP SFS G3, on a system that was previously upgraded
from HP SFS version 2.3 or older, you must get the latest upgrade documentation and tools from
HP SFS 2.3 support.
1.2 Benefits and Features
HP SFS G3.1-0 consists of a software set required to providehigh performance and highly available
Lustre File System service over InfiniBand or 10 Gigabit Ethernet (GigE) for HP MSA2000fc
storage hardware. The software stack includes:
• Lustre Software 1.6.7
• Open Fabrics Enterprise Distribution (OFED) 1.3.1
• Mellanox 10 GigE driver
• Heartbeat V2.1.3
• HP multipath drivers
• collectl (for system performance monitoring)
• pdsh for running file system server-wide commands
• Other scripts, tests, and utilities
1.3 Supported Configurations
HP SFS G3.1-0 supports the following configurations:
Table 1-1 Supported Configurations
Server Operating System
SupportedComponent
CentOS 5.2, RHEL5U2, SLES10 SP2, XCV4Client Operating Systems
Opteron, XeonClient Platform
V1.6.7Lustre Software
CentOS 5.2
ProLiant DL380 G5Server Nodes
MSA2000fcStorage Array
OFED 1.3.1 InfiniBand or 10 GigEInterconnect
1
1.1 About This Product 13
Table 1-1 Supported Configurations (continued)
1 CentOS 5.2 is available for download from the HP Software Depot at:
http://www.hp.com/go/softwaredepot
1.3.1 Hardware Configuration
A typical HP SFS system configuration consists of the base rack only that contains:
• ProLiant DL380 MetaData Servers (MDS), administration servers, and Object Storage Servers
(OSS)
• HP MSA2000fc enclosures
• Management network ProCurve Switch
• SAN switches
• InfiniBand or 10 GigE switches
• Keyboard, video, and mouse (KVM) switch
• TFT console display
All DL380 G5 file system servers must have their eth0 Ethernet interfaces connected to the
ProCurve Switch making up an internal Ethernet network. The iLOs for the DL380 G5 servers
should also be connected to the ProCurve Switch, to enable Heartbeat failover power control
operations. HP recommends at least two nodes with Ethernet interfaces be connected to an
external network.
DL380 G5 file system servers using HP SFS G3.1-0 must be configured with mirrored system
disks to protect against a server disk failure.Use the ROM-based HP ORCA ArrayConfiguration
utility to configure mirrored system disks (RAID 1) for each server by pressing F8 during system
boot. More information is available at:
http://h18004.www1.hp.com/products/servers/proliantstorage/software-management/acumatrix/
index.html
The MDS server, administration server, and each pair of OSS servers have associated HP
MSA2000fc enclosures. Figure 1-1 provides a high-level platform diagram. For detailed diagrams
of the MSA2000 controller and the drive enclosure connections, see the HP StorageWorks 2012fc
Modular Smart Array User Guide at:
http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01394283/c01394283.pdf
SupportedComponent
SAS, SATAStorage Array Drives
8.10 and laterProLiant Support Pack (PSP)
14 What's In This Version
Figure 1-1 Platform Overview
1.3 Supported Configurations 15
Figure 1-2 Server Pairs
Figure 1-2 shows typical wiring for server pairs.
1.3.1.1 Fibre Channel Switch Zoning
If your Fibre Channel is configured with a single Fibre Channel switch connected to more than
one server node failover pair and its associated MSA2000 storage devices, you must set up zoning
on the Fibre Channel switch. Most configurations are expected to require this zoning. The zoning
should be set up such that each server node failover pair only can see the MSA2000 storage
devices that are defined for it, similar to the logical view shown in Figure 1-1 (page 15). The
Fibre Channel ports for each server node pair, and its associated MSA2000 storage devices should
be put into the same switch zone.
For the commands used to set up Fibre Channel switch zoning, see the documentation for your
specific Fibre Channel B-series switch available from:
http://www.hp.com/go/san
1.4 Server Security Policy
The HP Scalable File Share G3 servers run a generic Linux operating system. Security
considerations associated with the servers are the responsibility of the customer. HP strongly
recommends that access to the SFS G3 servers be restricted to administrative users only. Doing
16 What's In This Version
so will limit or eliminate user access to the servers, thereby reducing potential security threats
and the need to apply security updates. For information on how to modify validation of user
credentials, see “Configuring User Credentials” (page 31).
HP provides security updates for all non-operating-system components delivered by HP as part
of the HP SFS G3 product distribution. This includes all rpm's delivered in /opt/hp/
sfs. Additionally, HP SFS G3 servers run a customized kernel which is modified to provide
Lustre support. Generic kernels cannot be used on the HP SFS G3 servers. For this reason, HP
also provides kernel security updates for critical vulnerabilities as defined by CentOS kernel
releases which are based on RedHat errata kernels. These kernel security patches are delivered
via ITRC along with installation instructions.
It is the customer's responsibility to monitor, download, and install user space security updates
for the Linux operating system installed on the SFS G3 servers, as deemed necessary, using
standard methodsavailable for CentOS. CentOS security updatescan be monitored bysubscribing
to the CentOS Announce mailing list.
1.4 Server Security Policy 17
18
2 Installing and Configuring MSA Arrays
This chapter summarizes the installation and configuration steps for MSA2000fc arrays usee in
HP SFS G3.1-0 systems.
2.1 Installation
For detailed instructions of how to set up and install the MSA2000fc, see Chapter 4 of the HP
StorageWorks 2012fc Modular Smart Array User Guide on the HP website at:
http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01394283/c01394283.pdf
2.2 Accessing the MSA2000fc CLI
You can use the CLI software, embedded in the controller modules, to configure, monitor, and
manage a storage system. CLI can be accessed using telnet over Ethernet. Alternatively, you can
use a terminal emulator if the management network is down. For information on setting up the
terminal emulator, see the HP StorageWorks 2000 Family Modular Smart Array CLI Reference Guide
on the HP website at:
http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01505833/c01505833.pdf
NOTE: The MSA2000s must be connected to a server with HP SFS G3.1-0 software installed as
described in Chapter 3 (page 23) to use scripts to perform operations on multiple MSA2000
arrays.
2.3 Using the CLI to Configure Multiple MSA Arrays
The CLI is used for managing a number of arrays in a large HP SFS configuration because it
enables scripted automation of tasks that must be performed on each array. CLI commands are
executed on an array by opening a telnet session from the management server to the array. The
provided script, /opt/hp/sfs/msa2000/msa2000cmd.pl, handles the details of opening a
telnet session on an array, executing a command, and closing the session. This operation is quick
enough to be practical in a script that repeats the command on each array. For a detailed
description of CLI commands, see the HP StorageWorks 2000 Family Modular Smart Array CLI
Reference Guide.
2.3.1 Configuring New Volumes
Only a subset of commands is needed to configure the arrays for use with HP SFS. To configure
new volumes on the storage arrays, follow these steps:
1. Power on all the enclosures.
2. Use the rescan command on the array controllers to discover all the attached enclosures
and drives.
3. Use the create vdisk command to create one vdisk from the disks of each storage
enclosure. For MGS and MDS storage, HP SFS uses RAID10 with 10 data drives and 2 spare
drives. For OST storage, HP SFS uses RAID6 with 9 data drives, 2 parity drives, and 1 hot
spare. The command is executed for each enclosure.
4. Use the create volume command to create a single volume occupying the full extent of
each vdisk. In HP SFS, one enclosure contains one vdisk, which contains one volume, which
becomes one Lustre Object Storage Target (OST).
To examine the configuration and status of all the arrays, use the show commands. For more
information about show commands, see the HP StorageWorks 2000 Family Modular Smart Array
CLI Reference Guide.
2.1 Installation 19
2.3.2 Creating New Volumes
To create new volumes on a set of MSA2000 arrays, follow these steps:
1. Power on all the MSA2000 shelves.
2. Define an alias.
One way to execute commands on a set of arrays is to define a shell alias that calls
/opt/hp/sfs/msa2000/msa2000cmd.pl for each array. The alias defines a shell for-loop
which is terminated with ; done. For example:
# alias forallmsas='for NN in `seq 101 2 119` ; do \
./msa2000cmd.pl 192.168.16.$NN'
In the above example, controller A of the first array has an IP address of 192.168.16.101,
controller B has the next IP address, and the rest of the arrays have consecutive IP addresses
up through 192.168.16.[119,120] on the last array. This command is only executed on one
controller of the pair.
For the command examples in this section, the MGS and MDS use the MSA2000 A controllers
assigned to IP addresses 192.168.16.101–103. The OSTs use the A controllers assigned to the
IP addresses 192.168.16.105–119. The vdisks and volumes created for MGS and MDS are not
the same as vdisks and volumes created for OSTs. So, for convenience, define an alias for
each set of MDS (MGS and MDS) and OST controllers.
# alias formdsmsas='for NN in `seq 101 2 103` ; do ./msa2000cmd.pl 192.168.16.$NN'
# alias forostmsas='for NN in `seq 105 2 119` ; do ./msa2000cmd.pl 192.168.16.$NN'
NOTE: You may receive the following error if a controller is down:
# alias forallmsas='for NN in `seq 109 2 115` ; do ./msa2000cmd.pl 192.168.16.$NN'
# forallmsas show disk 3 ; done
----------------------------------------------------
On MSA2000 at 192.168.16.109 execute < show disk 3 >
ID Serial# Vendor Rev. State Type Size(GB) Rate(Gb/s) SP
------------------------------------------------------------------------------
3 3LN4CJD700009836M9QQ SEAGATE 0002 AVAIL SAS 146 3.0
------------------------------------------------------------------------------
On MSA2000 at 192.168.16.111 execute < show disk 3 >
ID Serial# Vendor Rev. State Type Size(GB) Rate(Gb/s) SP
------------------------------------------------------------------------------
3 3LN4DX5W00009835TQX9 SEAGATE 0002 AVAIL SAS 146 3.0
------------------------------------------------------------------------------
On MSA2000 at 192.168.16.113 execute < show disk 3 >
problem connecting to "192.168.16.113", port 23: No route to host at ./msa2000cmd.pl line 12
----------------------------------------------------
On MSA2000 at 192.168.16.115 execute < show disk 3 >
problem connecting to "192.168.16.115", port 23: No route to host at ./msa2000cmd.pl line 12
3. Storage arrays consist of a controller enclosure with two controllers and up to three connected
disk drive enclosures. Each enclosure can contain up to 12 disks.
Use the rescan command to find all the enclosures and disks. For example:
# forallmsas rescan ; done
# forallmsas show disks ; done
The CLI syntax for specifying disks in enclosures differs based on the controller type used
in the array. The following vdisk and volume creation steps are organized by controller
types MSA2212fc and MSA2312fc, and provide examples of command-line syntax for
specifying drives. This assumes that all arrays in the system are using the same controller
type.
20 Installing and Configuring MSA Arrays
• MSA2212fc Controller
Disks are identified by SCSI ID. The first enclosure has disk IDs 0-11, the second has
16-27, the third has 32-43, and the fourth has 48-59.
• MSA2312fc Controller
Disks are specified by enclosure ID and slot number. Enclosure IDs increment from 1.
Disk IDs increment from 1 in each enclosure. The first enclosure has disk IDs 1.1-12,
the second has 2.1-12, the third has 3.1-12, and the fourth has 4.1-12.
Depending on the order in which the controllers powered on, youmight see different ranges
of disk numbers. If this occurs, run the rescan command again.
4. If you have MSA2212fc controllers in your arrays, use the following commands to create
vdisks and volumes for each enclosure in all of the arrays. When creating volumes, all
volumes attached to a given MSA must be assigned sequential LUN numbers to ensure
correct assignment of multipath priorities.
a. Create vdisks in the MGS and MDS array. The following example assumes the MGS
and MDS do not have attached disk enclosures and creates one vdisk for the controller
enclosure. The disks 0-4 are mirrored by disks 5-9 in this configuration:
# formdsmsas create vdisk level raid10 disks 0-4:5-9 assigned-to a spare 10,11 mode offline vdisk1;
done
Creating vdisks using offline mode is faster, but in offline mode the vdisk must be
created before you can create the volume. Use the show vdisks command to check
the status. When the status changes from OFFL, you can create the volume.
# formdsmsas show vdisks; done
Make a note of the size of the vdisks and use that number <size> to create the volume
in the next step.
b. Create volumes on the MDS and MDS vdisk.
# formdsmsas create volume vdisk vdisk1 size <size> mapping 0-1.11 volume1; done
c. Create vdisks ineach OST array. For OST arrays with one attached disk drive enclosure,
create two vdisks, one for the controller enclosure and one for the attached disk
enclosure. For example:
# forostmsas create vdisk level raid6 disks 0-10 assigned-to a spare 11 mode offline vdisk1; done
# forostmsas create vdisk level raid6 disks 16-26 assigned-to b spare 27 mode offline vdisk2; done
Use the show vdisks command to check the status. When the status changes from
OFFL, you can create the volume.
# forostmsas show vdisks; done
Make a note of the size of the vdisks and use that number <size> to create the volume
in the next step.
d. Create volumes on all OST vdisks. In the following example, LUN numbers are 21 and
22.
# forostmsas create volume vdisk vdisk1 size <size> mapping 0-1.21 volume1; done
# forostmsas create volume vdisk vdisk2 size <size> mapping 0-1.22 volume2; done
5. If you have MSA2312fc controllers in your arrays, use the following commands to create
vdisks and volumes for each enclosure in all of the arrays. When creating volumes, all
volumes attached to a given MSA must be assigned sequential LUN numbers to ensure
correct assignmentof multipath priorities. HP recommends mapping all ports to each volume
to facilitate proper hardware failover.
2.3 Using the CLI to Configure Multiple MSA Arrays 21
Loading...