Apple XSAN 1.4 Administrator Guide
Xsan
Administrator’s Guide for Xsan 1.4
K
Apple Computer, Inc.
2006 Apple Computer, Inc. All rights reserved.
©
Under the copyright laws, this manual may not be
copied, in whole or in part, without the written consent
of Apple.
The Apple logo is a trademark of Apple Computer, Inc.,
registered in the U.S. and other countries. Use of the
“keyboard” Apple logo (Option-Shift-K) for commercial
purposes without the prior written consent of Apple
may constitute trademark infringement and unfair
competition in violation of federal and state laws.
Every effort has been made to ensure that the
information in this manual is accurate. Apple is not
responsible for printing or clerical errors.
Apple
1 Infinite Loop
Cupertino, CA 95014-2084
408-996-1010
www.apple.com
Apple, the Apple logo, Final Cut Pro, Mac, Macintosh, the
Mac logo, Mac OS, Panther, Xsan, and Xserve are
trademarks of Apple Computer, Inc., registered in the
U.S. and other countries.
Finder and Tiger are trademarks of Apple Computer, Inc.
ADIC and StorNext are registered trademarks of
Advanced Digital Information Corporation.
Intel and Intel Core are trademarks of Intel Corp. in the
U.S. and other countries.
PowerPC and the PowerPC logo are trademarks of
International Business Machines Corporation, under
license therefrom.
Other company and product names mentioned herein
are trademarks of their respective companies. Mention
of third-party products is for informational purposes
only and constitutes neither an endorsement nor a
recommendation. Apple assumes no responsibility with
regard to the performance or use of these products.
The product described in this manual incorporates
copyright protection technology that is protected by
method claims of certain U.S. patents and other
intellectual property rights owned by Macrovision
Corporation and other rights owners. Use of this
copyright protection technology must be authorized by
Macrovision Corporation and is intended for home and
other limited viewing uses only unless otherwise
authorized by Macrovision Corporation. Reverse
engineering or disassembly is prohibited.
Apparatus Claims of U.S. Patent Nos. 4,631,603, 4,577,216,
4,819,098 and 4,907,093 licensed for limited viewing
uses only.
Simultaneously published in the United States and
Canada.
034-3767-A/07-21-06
Contents
11 About This Book
12
What’s New in Xsan 1.4
12
Version Compatibility
14
Upgrading from an Earlier Version of Xsan
14
For More Information
15
Notation Conventions
17 Chapter 1: Overview of Xsan
17
What Is Xsan?
19
Xsan Storage Area Networks
21
Shared SAN Volumes
21
Controllers and Clients
22
SAN Connections
23
How Xsan Storage Is Organized
24
LUNs
25
Storage Pools
26
Volumes
28
Folders with Affinities
28
How Xsan Utilizes Available Storage
3
28
Metadata and Journal Data
29
Striping at a Higher Level
30
Security
30
Expanding Storage
31
Xsan Capacities
33 Chapter 2: Setting Up a Storage Area Network
33
Hardware and Software Requirements
34
Supported Computers
34
Supported Storage Devices
35
Fibre Channel Fabric
36
Ethernet TCP/IP Network
37
Directory Services
38
Outgoing Mail Service
39
Planning Your SAN
41
Planning Considerations and Guidelines
50
Connecting Computers and Storage Devices
51
Preparing LUNs
51
Using the Xsan Admin Application
52
Installing Just the Xsan Admin Application
52
Connecting Through a Firewall
52
Xsan Admin Preferences
52
Getting Help
53
Using the Command Line
4
Contents
53
SAN and Volume Setup Summary
54
Setting Up an Xsan Volume on a Storage Area Network
74
Renaming a SAN
75
Removing a SAN
75
Setting Up Additional SANs
77 Chapter 3: Managing SAN Storage
78
Adding Storage
79
Restoring the Previous Volume Configuration After Changes
79
About Rearranging Fibre Channel Connections
80
Adding LUNs to a Storage Pool
82
Adding a Storage Pool to a Volume
84
Adding a Volume to a SAN
86
Assigning a Storage Pool Affinity to a Folder
87
Assigning an Affinity to a Folder Within a Folder
89
Removing an Affinity
90
Changing Storage Pool Settings
91
Renaming a Storage Pool
92
Choosing the Types of Files Stored on a Storage Pool
92
Setting Storage Pool Stripe Breadth
94
Setting the Selection Method for Multiple Connections
95
Changing Volume Settings
96
Renaming a Volume
97
Setting the Block Allocation Size
Contents
5
99
Setting the Volume Allocation Strategy
10 0
Enabling or Disabling Access Control Lists
101
Choosing the Windows ID Mapping Method
101
Checking Volume Fragmentation
10 2
Defragmenting a Volume
10 3
Checking the Integrity of a Volume
10 4
Repairing a Volume
10 5 Chapter 4: Managing Clients and Users
10 6
Adding a Client
10 7
Adding a Client to a StorNext SAN
10 8
Mounting a Volume on a Client
10 9
Controlling Client and User Access
10 9
Controlling Access to Folders on Volumes
110
Setting Up Access Control Lists
110
Unmounting a Volume on a Client
110
Restricting a Client to Read-Only Access
111
Removing a Client from a SAN
111
Removing Xsan Software from a Computer
112
Mapping Windows User and Group IDs
11 4
Setting User and Group Quotas
115
About Xsan Quotas
117
Checking User Quota Use
119
Helping Clients Check Their Own Quotas
6
Contents
12 0
Creating Local Home Folders for Network Accounts
121 Chapter 5: Managing Metadata Controllers
12 2
Adding a Controller
12 3
Setting Controller Failover Priority
12 4
Switching to a Standby Controller
12 5
Finding Out Which Controller Is Hosting a Volume
12 7
Listing the Volumes Hosted by a Controller
12 8
Changing a Controller’s IP Address
12 9
Upgrading Controller Software
13 0
Monitoring Controller Status
131 Chapter 6: Monitoring SAN Status
13 2
Locking Xsan Admin Views for Secure Monitoring
13 3
Checking Overall SAN Status
13 4
Checking Overall Volume Status
13 5
Checking Free Space on a Volume
13 6
Checking Free Space on a Storage Pool
13 6
Checking Quota Use
13 6
Viewing a Controller’s CPU and Network Utilization
13 7
Viewing File System CPU and Memory Utilization
13 8
Setting Up Status Notifications
13 9
Checking the Status of File System Processes
13 9
Viewing Xsan Logs
Contents
7
14 0
Checking Volume Clients
141
Checking for Fibre Channel Connection Failures
141
Checking the State of Xserve RAID Systems
14 3 Chapter 7: Solving SAN Problems
14 3
A Dialog on a Client Reports That a LUN Is Unreadable
14 3
You’re Unable to Connect to a Computer Using Xsan Admin
14 4
You’re Unable to Install the Xsan Software
14 4
Some Computers Aren’t Listed in Xsan Admin
14 4
You’re Unable to Mount a Volume on a Client
14 5
Xserve RAID LUNs Aren’t Accessible over Fibre Channel
14 5
You’re Unable to Restart a Volume After Adding LUNs or Storage Pools
14 6
Some LUNs Aren’t Listed in Xsan Admin
14 7
Some LUNs Are Listed Twice in Xsan Admin
14 7
Files and Folders Created by Mac OS 9 Computers Show the Wrong Creation Date
14 8
Problems Using Command-Line Tools
14 8
A LUN Doesn’t Have as Much Space as Expected
14 9
You’re Unable to Rename an Xsan Volume in the Finder
14 9
Fibre Channel Performance Is Poorer Than Expected
14 9
A Client User Sees Error Code –1425
14 9
File Copy Doesn’t Finish
15 0
A Client is Unable to Use a Volume After a Fibre Channel Interruption
15 0
SAN Performance Declines Periodically and Predictably
151
You’re Unable to Add a Storage Pool
8
Contents
152
You’re Unable to Add LUNs to a Storage Pool
153
The Capacity of a Larger LUN is Listed as 2 Terabytes
155 Appendix A: Combining Xsan and StorNext Clients and Controllers
155
Terminology
15 6
Compatible Software Versions
15 6
Licensing
157
Adding Macintosh Clients to a StorNext SAN
159
Using Xsan Controllers with StorNext Clients
161 Appendix B: Using the Command Line
161
Using the Shell Commands
162
Sending Commands to Remote Computers
162
Viewing the Man Pages
163
Notation Conventions
164
The Commands
164
Viewing or Changing Volume and Storage Pool Settings (cvadmin)
168
Copying Files or Folders (cvcp)
171
Checking or Repairing a Volume (cvfsck)
17 2 Labeling, Listing, and Unlabeling LUNs (cvlabel)
173 Creating a Folder with an Affinity (cvmkdir)
174 Creating and Preallocating a File (cvmkfile)
175 Initializing a Volume (cvmkfs)
17 6 Applying Volume Configuration Changes (cvupdatefs)
Contents
9
17 6 Starting a Volume Controller (fsm)
17 7 Starting a Port Mapper Process (fsmpm)
17 7 Defragmenting a File, Folder, or Volume (snfsdefrag)
17 9 Mounting an Xsan Volume
18 0 Unmounting an Xsan Volume
181 Viewing Logs
181 The Configuration Files
181 Examples
183 Glossary
187 Index
10 Contents
About This Book
Use this guide to learn how to set up and manage Xsan
volumes on a storage area network.
This guide shows how to use Xsan to combine Xserve RAID arrays and slices into large,
easy-to-expand volumes of storage that clients use like local disks but are actually
shared over a high-speed Fibre Channel fabric.
Chapter 1 provides an overview of Xsan and how you can use it to organize RAID arrays
into shared volumes of storage.
Chapter 2 includes hardware and software requirements, SAN planning guidelines, and
basic steps for setting up an Xsan volume.
Chapter 3 contains instructions for expanding storage, creating folders with affinities,
changing volume and storage pool settings, and checking, defragmenting, and
repairing SAN volumes.
Chapter 4 shows how to add client computers to a SAN, mount volumes on clients,
control client and user access to SAN files, and control user space using quotas.
Chapter 5 contains information about managing volume metadata controllers.
Chapter 6 includes instructions for monitoring and automatically reporting the
condition of a SAN.
Preface
11
Chapter 7 lists solutions to common problems you might encounter.
Appendix A contains information to help you combine Xsan controllers or clients with
ADIC StorNext controllers or clients on the same SAN.
Appendix B describes command-line utilities and configuration files you can use to
manage an Xsan SAN using Terminal.
What’s New in Xsan 1.4
Xsan 1.4 offers these new features and capabilities:
 You can use access control lists (ACLs) to manage user access to Xsan volumes.
 Xsan runs on Intel-based Macintosh computers.
Version Compatibility
The following table shows which versions of Xsan and StorNext controllers and clients
can be used on the same SAN.
Important: If you set up a LUN that is larger than 2 terabytes (TB), be aware that older
clients (those running Xsan on Mac OS X 10.3 Panther or Xsan 1.2 or earlier on
Mac OS X 10.4 Tiger) cannot read these large LUNs. A dialog appears on these older
clients that says the LUN is unreadable. Be sure to click Ignore to dismiss this dialog.
Do not click Format, or information on the corresponding Xsan volume will be lost.
If you use LUNs greater than 2 TB, you should upgrade all SAN clients to Xsan 1.3 or
later on Mac OS X or Mac OS X Server v10.4.
12 Preface About This Book
Controller Client Compatible
Xsan 1.4 Xsan 1.4 Yes
Xsan 1.3 (Mac OS X v10.4) Yes
Xsan 1.3 (Mac OS X v10.3) Yes, for LUNs < 2 TB
Xsan 1.2 or earlier No
StorNext 2.8 No
StorNext FX 1.3 or 1.4 Yes
StorNext 2.6 or 2.7 Yes
StorNext 2.5 or earlier No
Xsan 1.3 (Mac OS X v10.4) Xsan 1.4 No
Xsan 1.3 (Mac OS X v10.4) Yes
Xsan 1.3 (Mac OS X v10.3) Yes
StorNext FX 1.3 or 1.4 Yes
StorNext 2.6 or 2.7 Yes
StorNext 2.5 or earlier No
StorNext 2.8 Xsan 1.4 or 1.3 Yes
StorNext 2.7 Xsan 1.4 or 1.3 Yes
StorNext 2.6 or earlier Xsan 1.4 or 1.3 No
Preface About This Book 13
Upgrading from an Earlier Version of Xsan
For more information about migrating your SAN storage to Xsan 1.4 from an earlier
version of Xsan, including tips for upgrading with the least impact on existing storage,
see the Xsan Migration Guide at www.apple.com/server/documentation/.
For More Information
For more information about Xsan, consult these resources:
Read Me documents—important updates and special information. Look for them on the
Xsan installation disc.
Xsan website (www.apple.com/xsan/)—gateway to extensive product and technology
information.
Xsan Support website (www.apple.com/support/xsan/)—access to articles about Xsan
from Apple’s support organization.
Apple customer training (train.apple.com/)—instructor-led and self-paced courses for
honing your Xsan administration skills.
Apple discussion groups (discussions.info.apple.com/)—a way to share questions,
knowledge, and advice with other Xsan administrators.
Apple mailing list directory (www.lists.apple.com/)—subscribe to mailing lists so you can
communicate with other Xsan administrators using email.
14 Preface About This Book
Notation Conventions
The following conventions are used in this book wherever shell commands or other
command-line items are described.
Notation Indicates
fixed-width font A command or other terminal text
$ A shell prompt
[text_in_brackets] An optional parameter
(one|other) Alternative parameters (type one or the other)
underlined A parameter you must replace with a value
[...] A parameter that may be repeated
<angle_brackets> A displayed value that depends on your SAN configuration
Preface About This Book 15
1 Overview of Xsan
This chapter gives you an overview of Xsan and storage area
networks.
Read this chapter for an overview of Xsan and how you can use it to set up a storage
area network (SAN) to provide fast, shared storage.
What Is Xsan?
Xsan is a storage area network file system and a management application (Xsan
Admin) that you can use to provide users or applications on client computers with
shared high-speed access to expandable storage.
1
17
Xsan lets you
combine RAID arrays
into volumes clients
use like local disks.
Storage
pools
RAID
arrays (LUNs)
18 Chapter 1 Overview of Xsan
Mac OS X
San Volume
File data moves
over Fibre Channel
Xsan Storage Area Networks
A storage area network is a way of connecting computers to storage devices that gives
users very fast access to files and gives administrators the ability to expand storage
capacity as needed without interrupting users.
An Xsan SAN consists of:
 Volumes of shared storage, stored on Xserve RAID systems, available to clients as
mounted volumes that they can use like local disks
 At least one computer acting as a metadata controller that coordinates access to the
shared volumes
 Client computers that access storage in accordance with established permissions and
quotas
 Underlying Fibre Channel and Ethernet networks
The following illustration shows the physical components of a typical Xsan SAN.
Chapter 1 Overview of Xsan 19
Metadata
controller
Clients
Ethernet - TCP/IP (Public)
Ethernet - TCP/IP (Private)
20 Chapter 1 Overview of Xsan
Standby
controller
Fibre
Channel switch
Xserve RAID
storage
Intranet/
Internet
Shared SAN Volumes
Users and applications see shared SAN storage as local volumes. Xsan volumes are
logical disks made up of groups of RAID arrays. The elements you combine to create an
Xsan volume are described under “How Xsan Storage Is Organized” on page 23.
Controllers and Clients
When you add a computer to an Xsan SAN, you specify whether it will play the role of
client, controller, or both.
Controllers
When you set up an Xsan SAN, you assign at least one computer to act as the
controller. The controller manages the SAN volume metadata, maintains a file system
journal, and controls concurrent access to files. Metadata includes such information as
where files are actually stored and what portions of available storage are allocated to
new files.
For high availability, you should add more than one controller to a SAN, as shown in
the illustration on page 20. If the primary controller fails, the standby controller takes
over. Though not recommended for best performance, controllers can also act as
clients, so you can use a standby controller as a working client while the primary
controller is operational.
Clients
The computers that users or applications use to access SAN volumes are called clients.
Clients communicate with controllers over the Ethernet network but use Fibre Channel
to send and retrieve file data to and from the RAID systems that provide storage for the
volumes.
Chapter 1 Overview of Xsan 21
SAN Connections
Xsan uses independent networks to connect storage devices, metadata controllers, and
client computers: a Fibre Channel network and one or two Ethernet networks.
User Data Over Fibre Channel
User data is transferred over high-speed Fibre Channel connections. Controllers also
use a Fibre Channel connection to move metadata to and from the volume.
Metadata Over Ethernet
To eliminate unnecessary traffic on the Fibre Channel connections, controllers and
clients use an Ethernet network to exchange file system metadata. (When a controller
reads or writes metadata on a volume, it uses Fibre Channel.) The Xsan Admin
application also uses the Ethernet connection to let you manage the SAN.
To prevent Internet or intranet traffic from interfering with metadata communications,
you should set up separate Ethernet networks as shown in the illustration on page 20.
Fibre Channel Multipathing
Xsan can take advantage of multiple Fibre Channel connections between clients and
storage. Xsan can alternate between connections for each read and write, or assign
each LUN in a volume to one of the connections when the volume is mounted.
22 Chapter 1 Overview of Xsan
How Xsan Storage Is Organized
Users use an Xsan volume the same way they use a local disk. What they don’t see is
that the SAN volume actually consists of numerous physical disks combined on several
levels using RAID techniques.
The following illustration shows an example of how disk space provided by the
individual drive modules in several Xserve RAID systems is combined into a volume
that users see as a large local disk.
Faster
Affinity Affinity
Storage pool
(Striping)
LUN LUN
RAID 0
RAID 0
array
array
Chapter 1 Overview of Xsan 23
LUN LUN
RAID 0
array
Safer
RAID 0
array
Shared
SAN volume
LUN LUN
RAID 5
array
Storage pool
(Striping)
RAID 5
array
LUN LUN
RAID 5
RAID 5
array
array
The following paragraphs describe these storage elements and how you organize them
to create shared Xsan volumes.
LUNs
The smallest storage element you work with in Xsan is a logical storage device called a
LUN (a SCSI logical unit number). In most storage area networks a LUN represents a
group of drives such as a RAID array or a JBOD (just a bunch of disks) device. In Xsan,
LUNs are Xserve RAID arrays or slices.
You create a LUN when you use the RAID Admin application to create an Xserve RAID
array. The controller hardware and software in the Xserve RAID system combine
individual drive modules into an array based on the RAID scheme you choose. Each
array appears on the network as a separate LUN. If you slice an array, each slice appears
as a LUN.
One of your first tasks when you set up a SAN volume is to prepare LUNs. If the two
RAID 5 arrays on a new Xserve RAID are not right for your application, you can use
RAID Admin to create arrays based on other RAID schemes. For help choosing schemes
for your LUNs, see “Choosing RAID Schemes for LUNs” on page 44.
The illustration on page 23 shows four Xserve RAID systems that each host two arrays.
Half of the arrays use a RAID 0 scheme (striping only) for speed while the others use
RAID 5 (distributed parity) to ensure against data loss. Xsan sees the arrays as LUNs that
can be combined to create a volume.
After your Xserve RAID LUNs are set up, you label and initialize them for use with the
Xsan file system using Xsan Admin.
24 Chapter 1 Overview of Xsan
Storage Pools
LUNs are combined to form storage pools. A storage pool in a small volume might
consist of a single RAID array, but storage pools in many volumes consist of multiple
arrays.
Xsan distributes file data in parallel across the LUNs in a storage pool using a RAID 0
(striping) scheme. So, you can improve a client’s access speed by distributing available
storage over several LUNs in a storage pool.
You can set up storage pools that have different performance or recoverability
characteristics and assign folders to them using affinities. Users can then select where
to store files based on their need for speed or safety. See “Folders with Affinities” on
page 28.
The illustration on page 23 shows eight LUNs combined into two storage pools, one
pool consisting of RAID 0 (fast, but not recoverable) arrays and the other made up of
RAID 5 (not as fast, but recoverable) arrays. Xsan stripes data across the four LUNs in
each storage pool.
You use Xsan Admin to add available LUNs to specific storage pools.
Chapter 1 Overview of Xsan 25
Volumes
Storage pools are combined to create the volumes that users see. From the user’s
perspective, the SAN volume looks and behaves just like a large local disk, except that:
 The size of the volume can grow as you add underlying arrays or storage pools
 Other users on the SAN can access files on the volume at the same time
In the illustration on page 23, two storage pools are combined to create a single shared
volume.
You create volumes and mount them on client computers using the Xsan Admin
application.
26 Chapter 1 Overview of Xsan
The following screen image shows how LUNs, storage pools, and volumes appear as
you organize them in Xsan Admin. This example shows a SAN named “Editing SAN”
with a single shared volume named “SanVol.” Storage for the volume is provided by
two storage pools, “Meta” and “Data,” the first based on a single LUN and the second on
two. Each of the LUNs is a 3-disk RAID 5 array on an Xserve RAID using 115 GB drive
modules.
SAN
Volume
LUN
Storage pool
Chapter 1 Overview of Xsan 27
Folders with Affinities
To control which storage pool is used to store specific files (for example, to provide
different levels of service for different users or applications), you can associate a folder
on an Xsan volume with one of the storage pools that make up the volume.
If, for example, you set up storage pools with different balances of performance and
data redundancy, users can choose between faster and safer storage by putting files in
the appropriate folder.
In the illustration on page 23, a predefined folder has an affinity for the faster storage
pool that is based on RAID 0 arrays. Any file that a user copies into this folder is
automatically stored on the faster arrays. A second folder is associated with the more
secure RAID 5 storage.
How Xsan Utilizes Available Storage
Xsan stores both user files and file system data on SAN volumes, and stripes data
across the LUNs in a volume for better performance.
Metadata and Journal Data
Xsan records information about the files in an Xsan volume using metadata files and
file system journals. File system metadata includes information such as which specific
parts of which disks are used to store a particular file and whether the file is being
accessed. The journal data includes a record of file system transactions that can help
ensure the integrity of files in the event of a failure.
28 Chapter 1 Overview of Xsan
These files are managed by the Xsan metadata controller, but are stored on SAN
volumes, not on the controller itself. By default, metadata and journal data are stored
on the first storage pool you add to a volume. You can use Xsan Admin to choose
where these files are stored when you add storage pools to a new volume.
Striping at a Higher Level
When you write a file to a RAID array using RAID 0 (striping), the file is broken into
segments that are spread across the individual disk drives in the array. This improves
performance by writing pieces of the file in parallel (instead of one piece at a time) to
the individual disks in the array. Xsan applies this same technique at a second, higher
level in the storage hierarchy. Within each storage pool in a volume, Xsan stripes file
data across the individual LUNs that make up the storage pool. Once again,
performance is improved because data is written in parallel.
You can tune SAN performance by adjusting the amount of data written to each LUN in
a storage pool (the “stripe breadth”) to suit a critical application.
Chapter 1 Overview of Xsan 29
Security
As the SAN administrator, you can control access to shared volumes in several ways.
First, users cannot browse or mount SAN volumes. Only a SAN administrator can
specify which volumes are mounted on which client computers. One way you can
control access to data is to mount a volume only on appropriate client computers.
To prevent users from modifying data on a volume, you can mount the volume with
read-only access.
You can control user access to folders on a volume by setting up access control lists
(ACLs) in Workgroup Manager or by specifying owner, group, and general access
permissions in the Finder.
You can also set up zones in the underlying Fibre Channel network to segregate users
and volumes.
Expanding Storage
There are two ways you can add free space to an Xsan volume:
 Add Xserve RAID systems (new LUNs) to existing storage pools
 Add entire new storage pools to volumes
Both methods require you to unmount and remount the volume on clients.
You can also add new volumes to a SAN at any time.
For information about expanding Xsan storage, see “Adding Storage” on page 78.
30 Chapter 1 Overview of Xsan
Loading...