Apple XSAN 1.0 Administrator Guide
Xsan
Administrator’s Guide
Includes information for managing Xsan
volumes in a storage area network using
Xsan Admin or the command line
Apple Computer, Inc.
© 2004 Apple Computer, Inc. All rights reserved.
The owner or authorized user of a valid copy of Xsan
software may reproduce this publication for the purpose
of learning to use such software. No part of this
publication may be reproduced or transmitted for
commercial purposes, such as selling copies of this
publication or for providing paid for support services.
The Apple logo is a trademark of Apple Computer, Inc.,
registered in the U.S. and other countries. Use of the
“keyboard” Apple logo (Option-Shift-K) for commercial
purposes without the prior written consent of Apple
may constitute trademark infringement and unfair
competition in violation of federal and state laws.
Apple, the Apple logo, Mac, Macintosh, Mac OS, Power
Mac, and Xserve are trademarks of Apple Computer, Inc.,
registered in the U.S. and other countries. Finder and
Xsan are trademarks of Apple Computer, Inc.
UNIX is a registered trademark in the United States and
other countries, licensed exclusively through
X/Open Company, Ltd.
StorNext and ADIC are registered trademarks of
Advanced Digital Information Corporation.
019-0192/08-27-04
3
Contents
Preface 7 About This Book
8
Notation Conventions
Chapter 1 9 Overview of Xsan
9
What Is Xsan?
10
Xsan Storage Area Networks
11
11
11
12
12
13
13
14
15
15
15
15
16
16
Shared SAN Volumes
Controllers and Clients
SAN Connections
How Xsan Storage Is Organized
LUNs (RAID Arrays)
Storage Pools
Volumes
Folders With Affinities
How Xsan Utilizes Available Storage
Metadata and Journal Data
Striping at a Higher Level
Security
Expanding Storage
Xsan Capacities
Chapter 2 17 Setting Up a Storage Area Network
18
Hardware and Software Requirements
18
18
19
19
20
20
21
22
27
27
Supported Computers
Supported Storage Devices
Fibre Channel Fabric
Ethernet TCP/IP Network
Directory Services
Outgoing Mail Service
Planning Your SAN
Planning Considerations and Guidelines
Connecting Computers and Storage Devices
Preparing LUNs (RAID Arrays and Slices)
3
28
Using the Xsan Admin Application
28
28
28
28
28
29
30
38
38
39
Installing Xsan Admin Separately
Connecting Through a Firewall
Xsan Admin Preferences
Getting Help
Using the Command Line
SAN Setup Summary
Setting Up an Xsan Storage Area Network
Renaming a SAN
Deleting a SAN
Setting Up Additional SANs
Chapter 3 41 Managing SAN Storage
42
Adding Storage
43
44
45
46
47
47
48
48
49
49
50
51
52
52
53
54
55
56
56
Adding LUNs to a Storage Pool
Adding a Storage Pool to a Volume
Adding a Volume to a SAN
Assigning a Storage Pool Affinity to a Folder
Assigning an Affinity to a Folder Within a Folder
Removing an Affinity
Changing Storage Pool Settings
Renaming a Storage Pool
Setting Storage Pool Access Permissions
Choosing the Types of Files Stored on a Storage Pool
Setting Storage Pool Stripe Breadth
Setting the Selection Method for Multiple Connections
Changing Volume Settings
Renaming a Volume
Setting the Volume Allocation Strategy
Setting the Block Allocation Size
Defragmenting a Volume
Checking the Integrity of a Volume
Repairing a Volume
Chapter 4 57 Managing Clients and Users
58
Adding a Client
58
Adding a Client to a StorNext SAN
59
Mounting a Volume on a Client
59
Controlling Client and User Access
59
60
60
60
4
Controlling Access to Folders on Volumes
Unmounting a Volume From a Client
Restricting a Client to Read-Only Access
Removing a Client From a SAN
Contents
60
61
61
62
Removing Xsan Software From a Computer
Setting User and Group Quotas
About Xsan Quotas
Checking Quota Use
Chapter 5 63 Managing Metadata Controllers
64
Adding a Controller
64
Setting Controller Failover Priority
65
Switching to a Standby Controller
66
Finding Out Which Controller Is Hosting a Volume
66
Listing the Volumes Hosted by a Controller
67
Changing a Controller’s IP Address
68
Upgrading Controller Software
68
Monitoring Controller Status
Chapter 6 69 Monitoring SAN Status
70
Checking Free Space on a SAN Volume
70
Checking Free Space on a Storage Pool
71
Checking User Quota Use
71
Viewing Controller CPU and Network Utilization Graphs
72
Setting Up Status Notifications
72
Checking Status of File System Processes
73
Viewing Logs
73
Checking Volume Clients
74
Checking for Fibre Channel Connection Failures
74
Checking the State of Xserve RAID Systems
Chapter 7 75 Solving SAN Problems
75
75
75
75
75
76
77
77
77
77
You Can’t Install the Xsan Software
Some Computers Aren’t Listed During Setup
You Can’t Connect to a SAN Computer From Xsan Admin
Xserve RAID Systems Aren’t Accessible Over Fibre Channel
You Can’t Mount a Volume on a Client
You Can’t Add a Storage Pool
After Slicing, Some LUNs Aren’t Listed in Xsan Admin
Problems Using Command-Line Tools
SAN User Sees Error Code –1425
LUN Doesn’t Have as Much Space as Expected
Appendix A 79 Combining Xsan and StorNext Clients and Controllers
79
Compatible Software Versions
79
Licensing
79
Terminology
80
Adding Macintosh Clients to a StorNext SAN
Contents
5
81
Using Xsan Controllers With StorNext Clients
Appendix B 83 Using the Command Line
83
Using the Shell Commands
83
83
84
84
85
88
Working on Remote Computers
Viewing the Man Pages
Notation Conventions
The Commands
Viewing or Changing Volume and Storage Pool Settings (
Copying Files or Folders (
89 Checking or Repairing a Volume (cvfsck)
90 Labeling LUNs (cvlabel)
91 Creating a Folder With an Affinity (cvmkdir)
91 Creating and Pre-Allocating a File (cvmkfile)
92 Initializing a Volume (cvmkfs)
92 Viewing Storage Pool Information (cvsginfo)
92 Applying Volume Configuration Changes (cvupdatefs)
93 Starting a Volume Controller (fsm)
93 Starting a Port Mapper Process (fsmpm)
93 Defragmenting a File, Directory, or Volume (snfsdefrag)
95 Mounting an Xsan Volume
95 Unmounting an Xsan Volume
95 Viewing Logs
96 The Configuration Files
96 The Volume Configuration File
10 2 The Volume Auto-Start List
10 4 The Controller List
cvcp)
cvadmin
)
Glossary 10 7
Index 111
6
Contents
About This Book
Use this guide to learn how to set up and manage Xsan
volumes on a storage area network.
This guide shows how to use Xsan to combine Xserve RAID arrays and slices into large,
easy-to-expand volumes of storage that clients use like local disks but are actually
shared over a high-speed Fibre Channel fabric.
Chapter 1 provides an overview of Xsan and how you can use it to organize RAID arrays
and storage pools into shared volumes of storage.
Chapter 2 includes hardware and software requirements, SAN planning guidelines, and
basic steps for setting up an Xsan SAN.
Chapter 3 contains instructions for adding storage to a SAN, creating folders with
affinities, changing volume and storage pool settings, and checking, defragmenting,
and repairing SAN volumes.
Preface
Chapter 4 shows how to add client computers to a SAN, mount volumes on clients,
control client and user access to SAN files, and control user space using quotas.
Chapter 5 contains information on managing metadata controllers.
Chapter 6 includes instructions for monitoring and automatically reporting the
condition of a SAN.
Chapter 7 lists solutions to common problems you might encounter.
Appendix A contains information to help you join Xsan controllers or clients together
with ADIC StorNext controllers or clients in the same SAN.
Appendix B describes available command-line utilities and configuration files to help
you manage an Xsan SAN using Terminal.
7
Notation Conventions
The following conventions are used in this book wherever shell commands or other
command-line items are described.
Notation Indicates
monospaced font A command or other terminal text
$ A shell prompt
[text_in_brackets] An optional parameter
(one|other) Alternative parameters (type one or the other)
underlined
[...] A parameter that may be repeated
<anglebrackets> A displayed value that depends on your SAN configuration
A parameter you must replace with a value
8 Preface About This Book
1 Overview of Xsan
1
This chapter gives you an overview of Xsan and storage
area networks.
Read this chapter for an overview of Xsan and how you can use it to set up a storage
area network (SAN) to provide fast, shared storage.
What Is Xsan?
Xsan is a storage area network file system and a management application (Xsan Admin)
you can use to provide users or applications on client computers with shared highspeed access to expandable storage.
Volumes
SAN vol
Xsan lets you
Fibre
Channel
9
combine RAID arrays
into volumes clients
use like local disks.
RAID
arrays (LUNs)
Storage
pools
Xsan Storage Area Networks
A storage area network is a way of connecting computers to storage devices that gives
users very fast access to files and gives administrators the ability to expand storage
capacity as needed without interrupting users.
An Xsan SAN consists of:
• Volumes of shared storage, stored on Xserve RAID systems, available to clients as
mounted volumes that they can use like local disks
• At least one computer acting as a metadata controller that coordinates access to the
shared volumes
• Client computers that access storage in accordance with established permissions and
quotas
• Underlying Fibre Channel and Ethernet networks
The following illustration shows the physical components of a typical Xsan SAN.
Clients
Ethernet - TCP/IP
(Private)
Ethernet - TCP/IP
(Public)
Metadata
controller
Fibre
Channel switch
Standby
controller
Xserve RAID
storage
Intranet/
Internet
10 Chapter 1 Overview of Xsan
Shared SAN Volumes
Users and applications see shared SAN storage as local volumes. Xsan volumes are
logical disks made up of groups of RAID arrays. The elements you combine to create an
Xsan volume are described under “How Xsan Storage Is Organized” on page 12.
Controllers and Clients
When you add a computer to an Xsan SAN, you specify whether it will play the role of
client, controller, or both.
Controllers
When you set up an Xsan SAN, you assign at least one computer to act as the
controller. The controller manages the SAN volume metadata, maintains a file system
journal, and controls concurrent access to files. Metadata includes such information as
where files are actually stored and what portions of available storage are allocated to
new files.
For high availability, you can add more than one controller to a SAN, as shown in the
illustration on page 10. If the primary controller fails, the standby controller takes over.
Controllers can also act as clients, so you can use a standby controller as a working
client while the primary controller is operational.
Clients
The computers that users or applications use to access SAN volumes are called clients.
Clients communicate with controllers over the Ethernet network but use Fibre Channel
to send and retrieve file data to and from the RAID systems that provide storage for the
volumes.
SAN Connections
Xsan uses independent networks to connect storage devices, metadata controllers, and
client computers: a Fibre Channel network and one or two Ethernet networks.
User Data Over Fibre Channel
User data is transferred over high-speed Fibre Channel connections.
Metadata Over Ethernet
To eliminate unnecessary traffic on the Fibre Channel connections, controllers and
clients use an Ethernet network to exchange file system metadata. The Xsan Admin
application also uses the Ethernet connection to let you manage the SAN.
Fibre Channel Multipathing
Xsan can take advantage of multiple Fibre Channel connections between clients and
storage. Xsan can alternate between connections for each read and write, or assign
each LUN in a volume to one of the connections when the volume is mounted.
Chapter 1 Overview of Xsan 11
How Xsan Storage Is Organized
Users use an Xsan volume the same way they use a local disk. What they don’t see is
that the SAN volume actually consists of numerous physical disks combined on several
levels using RAID techniques.
The following illustration shows an example of how disk space provided by the
individual drive modules in Xserve RAID systems is combined into a volume that users
see as a large local disk.
Shared SAN
volume
Faster
RAID 0
array
Safer
LUN LUN
RAID 5
RAID 5
array
array
LUN LUN
RAID 5
RAID 5
array
array
LUN LUN
RAID 0
RAID 0
array
Affinity Affinity
Storage pool Storage pool
(Striping) (Striping)
LUN LUN
RAID 0
array
array
The following paragraphs describe these storage elements and how you organize them
to create shared Xsan volumes.
LUNs (RAID Arrays)
The smallest storage element you work with in Xsan is a logical storage device called a
LUN (a SCSI logical unit number). In most storage area networks a LUN represents a
group of drives such as a RAID array or a JBOD (just a bunch of disks) device. In Xsan,
LUNs are Xserve RAID arrays or slices.
You create a LUN when you use RAID Admin to create an Xserve RAID array. The
controller hardware and software in the Xserve RAID system combine individual drive
modules into an array based on the RAID scheme you choose. Each array appears on
the network as a separate LUN. If you slice an array, each slice appears as a LUN.
12 Chapter 1 Overview of Xsan
One of your first tasks when you set up a SAN volume is to prepare LUNs. If the two
RAID 5 arrays on a new Xserve RAID are not right for your application, you can use RAID
Admin to create arrays based on other RAID schemes. For help choosing schemes for
your LUNs, see “Choosing RAID Schemes for LUNs” on page 23.
The illustration on page 12 shows four Xserve RAID systems hosting two arrays each.
Half of the arrays use a RAID 0 scheme (striping only) for speed while the others use
RAID 5 (distributed parity) to ensure against data loss. Xsan sees the arrays as LUNs that
can be combined to create a volume.
After your Xserve RAID LUNs are set up, you label and initialize them for use with the
Xsan file system using Xsan Admin.
Storage Pools
LUNs are combined to form storage pools. A storage pool in a small volume might
consist of a single RAID array, but storage pools in many volumes consist of multiple
arrays.
Xsan distributes file data in parallel across the LUNs in a storage pool using a RAID 0
(striping) scheme. So, you can improve access speed by distributing available storage
over several LUNs in a storage pool.
You can set up storage pools that have different performance or recoverability
characteristics and assign folders to them using affinities. Users can then select where
to store files based on their need for speed or safety. See “Folders With Affinities” on
page 14.
As an example, the illustration on page 12 shows eight LUNs combined into two
storage pools, one pool consisting of RAID 0 (fast but not recoverable) arrays and the
other made up of RAID 5 (not as fast, but recoverable) arrays. Xsan stripes data across
the four LUNs in each storage pool.
You use Xsan Admin to add available LUNs to specific storage pools.
Volumes
Storage pools are combined to create the volumes that users see. From the user’s
perspective, the SAN volume looks and behaves just like a large local disk, except that:
• The size of the volume can grow as you add underlying arrays or storage pools
• Other users on the SAN can access files on the volume at the same time
In the example illustrated on page 12, two storage pools are combined to create a
single shared volume.
You create volumes and mount them on client computers using the Xsan Admin
application.
Chapter 1 Overview of Xsan 13
The following screen image shows how LUNs, storage pools, and volumes appear as
you organize them in the Xsan Admin application. This example shows a SAN named
“Test SAN” with a single shared volume named “SanVol.” Storage for the volume is
provided by two storage pools, “Pool1” and “Pool2,” each based on a single LUN. Each
of the LUNs is a 3-disk RAID 5 array on an Xserve RAID using 115 GB drive modules.
SAN
Volume
Storage pool
LUN
Folders With Affinities
To control which storage pool is used to store specific files (for example, to provide
different levels of service for different users or applications), you can associate a folder
on an Xsan volume with one of the storage pools that make up the volume.
If, for example, you set up storage pools with different balances of performance and
data redundancy, users can choose between faster and safer storage by putting files in
the appropriate folder.
In the illustration on page 12, a predefined folder has an affinity for the faster storage
pool that is based on RAID 0 arrays. Any file that a user copies into this folder is
automatically stored on the faster arrays. A second folder is associated with the more
secure RAID 5 storage.
14 Chapter 1 Overview of Xsan
How Xsan Utilizes Available Storage
Xsan stores both user files and file system data on SAN volumes, and stripes data across
the LUNs in a volume for better performance.
Metadata and Journal Data
Xsan records information about the files in an Xsan volume using metadata files and
file system journals. File system metadata includes information such as which specific
parts of which disks are used to store a particular file and whether the file is being
accessed. The journal data includes a record of file system transactions that can help
ensure the integrity of files in the event of a failure.
These files are managed by the Xsan metadata controller, but are stored on SAN
volumes, not on the controller itself. By default, metadata and journal data are stored
on the first storage pool you add to a volume. You can use Xsan Admin to choose
where these files are stored when you add storage pools to a new volume.
Striping at a Higher Level
When you write a file to a RAID array using RAID 0 (striping), the file is broken into
segments that are spread across the individual disk drives in the array. This improves
performance by writing pieces of the file in parallel (instead of one piece at a time) to
the individual disks in the array. Xsan applies this same technique at a second, higher
level in the storage hierarchy. Within each storage pool in a volume, Xsan stripes file
data across the individual LUNs that make up the storage pool. Once again,
performance is improved because data is written in parallel.
You can tune SAN performance by adjusting the amount of data written to each LUN in
a storage pool (the “stripe breadth”) to suit a critical application.
Security
As SAN administrator, you can control access to shared volumes in several ways.
First, users cannot browse or mount SAN volumes. Only a SAN administrator can
specify which volumes are mounted on which client computers. One way you can
control access to data is to mount a volume only on appropriate client computers.
To prevent users from modifying data on a volume, you can mount the volume with
read-only access.
You can also control user access to folders on a volume by specifying owner, group,
and general access permissions as you would in the Finder.
You can also set up zones in the underlying Fibre Channel network to segregate users
and volumes.
Chapter 1 Overview of Xsan 15
Expanding Storage
You can add free space to an Xsan volume without interrupting the users or
applications that rely on the volume for access to files. There are two ways to add
storage to a volume:
• Add Xserve RAID systems (new LUNs) to existing storage pools
• Add entire new storage pools to volumes
The first method requires you to unmount and remount the volume on clients. The
second method lets you add space without users noticing any change other than the
availability of more space in the volumes they use.
You can also add new volumes to a SAN at any time.
For information on expanding Xsan storage, see “Adding Storage” on page 42.
Xsan Capacities
The following table lists limits and capacities for Xsan volumes.
Parameter Maximum
Number of computers in a SAN (controllers and clients) 64
Number of storage pools in a volume 512
Number of LUNs in a storage pool 32
Number of LUNs in a volume 512
Number of files in a volume 4,294,967,296
Volume size 16 TB
File size 16 TB
Volume name length 70 characters
File or folder name length 251 characters
SAN name length 255 characters
Storage pool name length 255 characters
16 Chapter 1 Overview of Xsan
2 Setting Up a Storage Area
Network
This chapter lists requirements, suggests planning tips,
and gives instructions for setting up an Xsan SAN.
This chapter contains:
• Xsan hardware and software requirements (page 18)
• Planning guidelines (page 21)
• SAN setup instructions (page 29)
2
17
Hardware and Software Requirements
Your SAN environment needs to satisfy requirements in these areas:
• Supported computers
• Supported storage devices
• Fibre Channel fabric, adapters, and switches
• Ethernet network
• Directory services (optional)
• Outgoing mail service (optional)
Supported Computers
Xsan controller and client computers must meet these minimum requirements:
Systems
• Xserve
• Xserve G5
• Power Mac G4 dual 800 MHz or faster
• Power Mac G5
Memory
• Clients should have a minimum of 256 MB of RAM.
• Controllers should have a minimum of 512 MB of RAM. (For optimum performance,
add an additional 512 MB of RAM for each SAN volume hosted by the controller.)
Supported Operating Systems
• Mac OS X v10.3; version 10.3.5 or later
• Mac OS X Server v10.3; version 10.3.5 or later
Windows, AIX, IRIX, Linux, and Solaris clients must be running ADIC’s StorNext File
System version 2.4.
Supported Storage Devices
This guide assumes you are using Xserve RAID systems for your storage devices.
Important: Be sure to install the latest firmware update on your Xserve RAID systems
before you use them with Xsan.
For demanding applications such as video editing, use Xserve RAID systems that have:
• A full set of 14 Apple Drive Modules
• 512 MB of cache in each controller (1 GB total)
18 Chapter 2 Setting Up a Storage Area Network
Fibre Channel Fabric
Unlike file system metadata, which is transferred over Ethernet, actual file content in an
Xsan SAN is transferred over Fibre Channel connections. To set up the connections, you
need:
• Apple Fibre Channel PCI cards for each client and controller computer
• One or more supported Fibre Channel switches
• Fibre Channel cables connecting computers and storage devices to the switches to
form a Fibre Channel fabric
Fibre Channel PCI Cards
Install Apple Fibre Channel PCI cards in all Macintosh computers that will connect to
the SAN.
Fibre Channel Switches
The following Fibre Channel switches have been tested with Xsan, Xserve RAID
systems, and the Apple Fibre Channel PCI card:
• Brocade Silkworm 3200, 3800, 3900, and 12000 series
• QLogic SANbox 2–8, SANbox 2–16, SANbox 2–64, and SANbox 5200
• Emulex SAN Switch 355, 375, and 9200
For the latest additions to this list of qualified switches, see the Xsan webpages at
www.apple.com/xsan.
Fabric Configuration
You must connect the computers, storage devices, and switches in your Fibre Channel
network to form a Fibre Channel “fabric.” In a fabric, Fibre Channel cables connect node
ports (F or N_Port). See the documentation that came with your Fibre Channel switches
for more information.
Note: If you are using a Vixel 355 switch, you must connect Xserve RAID systems to an
FL (arbitrated loop) port on the switch.
You cannot use Fibre Channel Arbitrated Loop (FL ports) with an Xsan SAN, with the
exception noted above for Vixel 355 switches.
Ethernet TCP/IP Network
Computers in the SAN must also be connected to an Ethernet network. Xsan uses this
network instead of the Fibre Channel network to transfer file system metadata,
reserving the Fibre Channel connections for actual file contents.
If the computers in your SAN need to communicate with directory servers, a corporate
or campus intranet, or the Internet, you should connect each SAN client and controller
to two separate Ethernet networks: one private subnet for the SAN and a separate
connection for directory, intranet, or Internet traffic. This is important if you plan to use
the SAN for high-performance applications such as video editing.
Chapter 2 Setting Up a Storage Area Network 19
IP Addresses and Domain Names
For best results, assign fixed, non-routed IP addresses to all clients, controllers, and
storage devices connected to the SAN Ethernet network. You can use the following
ranges of IP addresses in your private (non-routed) subnet:
Private address range Associated subnet mask Comments
10.0.0.0 – 10.255.255.255 255.0.0.0 10/8
172.16.0.0 – 172.31.255.255 255.240.0.0 172.16/12
192.168.0.0 – 192.168.255.255 255.255.0.0 192.168/16
Directory Services
If you plan to use user and group privileges to control access to files and folders on the
SAN, you can simplify management by setting up or joining a user and group directory.
Although not required, a central directory lets you manage users and groups on one
computer instead of having to visit all SAN clients and controllers.
If you already have a directory, you can use the Directory Access application on each
controller and client to access the directory for user and group information.
If you don’t use a central directory service, you need to set up users and groups on
each SAN computer.
Important: If you create users and groups on each SAN computer, be sure that 1) each
user or group has a numeric user ID (UID) or group ID (GID) that is unique throughout
the SAN and 2) each user or group defined on more than one computer has the same
UID or GID on each computer.
If you don’t have access to a directory, you can use the directory services in Mac OS X
Server to set up an LDAP directory of SAN users and groups.
Outgoing Mail Service
Xsan can send SAN status notifications via email on your local subnet or corporate
network without using a separate mail server. However, to send notifications outside
your local network, you need an SMTP server to act as a mail gateway. If you don’t have
access to an outgoing mail server, you can use the mail service in Mac OS X Server to
set one up.
20 Chapter 2 Setting Up a Storage Area Network
Planning Your SAN
It’s easy to add storage to an existing Xsan SAN without interrupting users, but
reorganizing a SAN after you set it up is not so simple. So, it’s important to plan the
layout and organization of your SAN and its storage before you set it up.
An Xsan SAN is composed of:
• Storage devices (usually Xserve RAID systems)
• LUNs (logical unit numbers, usually RAID arrays)
• Storage pools (groups of LUNs)
• Volumes (groups of storage pools visible to users)
• Clients (computers that use volumes)
• Controllers (computers that manage volume metadata)
• Underlying Fibre Channel and Ethernet networks
Before you use Xsan Admin to set up a SAN, decide how you want to organize these
components. Take the time to create a drawing or a table that organizes available
hardware into RAID arrays, storage pools, volumes, client computers, and controllers in
a way that meets both your users’ needs and your needs as SAN administrator.
First, consider these questions:
• How much storage do you need?
• How do you want to present available storage to users?
• What storage organization makes the most sense for user workflow?
• What levels of performance do your users require?
• How important is constant availability?
• What are your requirements for security?
Your answers to the above questions will help you decide the following:
• What RAID schemes should you use for your RAID arrays?
• How many SAN volumes do you need?
• How should individual volumes be organized?
• Which LUNs go in each storage pool?
• Which storage pools make up each volume?
• Which clients, users, and groups should have access to each volume?
• Which computers will act as controllers?
• Do you need standby controllers?
• Do you want to use controllers as clients also?
• Where do you want to store file system metadata and journal data?
• What allocation strategy should you use?
Review the considerations and guidelines on the following pages for help translating
your answers into a suitable SAN design.
Chapter 2 Setting Up a Storage Area Network 21
Planning Considerations and Guidelines
The following paragraphs might help you make some of your SAN design decisions.
How Much Storage?
Because it’s easy to add storage to an Xsan SAN without interrupting user work, you
only need to decide on an adequate starting point. You can then add storage as
needed.
Note that the number of Xserve RAID systems you use affects not only available space
but also SAN performance. See “Performance Considerations,” below.
How Should Users See Available Storage?
If you want the users working on a particular project to see a volume dedicated to their
work, create a separate volume for each project. If it’s acceptable for a user to see a
folder for his or her work on a volume with other peoples’ folders, you can create a
single volume and organize it into project folders.
Workflow Considerations
How much file sharing is required by your users’ workflow? If, for example, different
users or groups work on the same files, either simultaneously or in sequence, it makes
sense to store those files on a single volume to avoid having to maintain or hand off
copies. Xsan uses file locking to manage shared access to a single copy of the files.
Performance Considerations
If your SAN supports an application (such as high resolution video capture and
playback) that requires the fastest possible sustained data transfers, design your SAN
with these performance considerations in mind:
• Set up the LUNs (RAID arrays) using a RAID scheme that offers high performance. See
“Choosing RAID Schemes for LUNs” on page 23.
• Group your fastest LUNs in storage pools reserved for the application. Reserve slower
devices for a volume dedicated to less demanding or supporting applications.
• To increase parallelism, spread LUNs across different Xserve RAID controllers. For
example, instead of creating a single 4-disk LUN on one side of an Xserve RAID,
create two 2-disk LUNs, one on each side, and add these LUNs to a storage pool.
Xsan then stripes data across the two LUNs and benefits from simultaneous transfers
through two controllers.
• To increase parallelism in a relatively small storage pool (the size of one or a few
drive modules), create a slice of similar size across all the drives on a controller
instead of creating the storage pool from just one or two drive modules.
• Spread file transfers across as many drives and RAID controllers as possible. Try
creating slices across the drives in RAID systems, then combine these slices into a
storage pool.
• To increase throughput, connect both ports on client Fibre Channel cards to the
fabric and set the multipathing method for the storage pool to Rotate.
22 Chapter 2 Setting Up a Storage Area Network
• Store user files, file system metadata, and journal data on separate storage pools, and
create these storage pools using LUNs from different RAID controllers.
• Use a router to isolate the Ethernet network used by the SAN from a company
intranet or the Internet, or better, use a second Ethernet network (including a second
Ethernet card in each SAN computer) for the SAN.
• If your SAN uses directory services, mail services, or other services on a separate
server, use a second, separate Ethernet network to connect SAN computers to that
server.
• As a rule of thumb, consider that a single Xserve RAID controller, after file system
overhead, can transfer roughly 80 MB of user data per second (160 MB per Xserve
RAID system). If your SAN must support an application running on multiple clients
that requires specific throughput on each client, you can use this number to estimate
the number of Xserve RAID systems necessary to support the aggregate transfer rate.
Availability Considerations
If high availability is important for your data, set up at least one standby controller in
addition to your primary controller. Also, consider setting up dual Fibre Channel
connections between each client, controller, and storage device using redundant Fibre
Channel switches.
Important: Losing a metadata controller without a standby can result in the loss of all
data on a volume. A standby controller is recommended.
Also, if you have a standby controller, you can upgrade the Xsan software without
interrupting the SAN. For more information, see “Upgrading Controller Software” on
page 68.
Security Considerations
If your SAN will support projects that need to be completely secure and isolated from
each other, you can create separate volumes for each project to eliminate any
possibility of the wrong client or user accessing files stored on a volume.
As SAN administrator, you control which client computers can use a volume. Clients
can’t browse for or mount SAN volumes on their own. You use Xsan Admin to specify
which clients a volume is mounted on.
You can also assign user and group permissions to folders you create on a volume or
use standard file access permissions to control access to other items.
Choosing RAID Schemes for LUNs
Much of the reliability and recoverability of data in a SAN is not provided by Xsan itself
but by the RAID arrays you combine to create your storage pools and volumes. Before
you set up a SAN, you use RAID Admin to prepare LUNs based on specific RAID
schemes.
Chapter 2 Setting Up a Storage Area Network 23
Xserve RAID supports all popular RAID levels. Each RAID scheme offers a different
balance of performance, data protection, and storage efficiency, as summarized in the
following table.
RAID level Storage efficiency Read performance Write performance Data redundancy
RAID 0 Highest Very High Highest No
RAID 1 Low High Medium Yes
RAID 3 High to very high Medium Medium Yes
RAID 5 High to very high High High Yes
RAID 0+1 Low High High Yes
RAID 10, 30, and 50 schemes assume the use of AppleRAID software striping and aren’t
appropriate for use with Xsan, which performs its own striping. For more help choosing
RAID schemes for your arrays, see the Xserve RAID User’s Guide or the Xserve RAID
Technology Overview (at www.apple.com).
Deciding on the Number of Volumes
A volume is the largest unit of shared storage in the SAN. If your users need shared
access to files, you should store those files on the same volume. This makes it
unnecessary for them to pass copies of the files among themselves.
On the other hand, if security is critical, one way to control client access is to create
separate volumes and mount only the authorized volume on each client.
For a more typical balance of security and shared access, a flexible compromise is to
create a single volume and use folder access privileges to control access.
Note: The maximum size of any volume is 16 TB. If you need more than 16 TB of
storage, you need to create more than one volume.
Deciding How to Organize a Volume
You can help users organize data on a volume or restrict users to specific areas of the
volume by creating predefined folders. You can control access to these folders by
assigning access permissions using Xsan Admin.
You can assign folders to specific storage pools using affinities. You can, for example,
create a folder for data that requires fast access and assign that folder to your fastest
storage pool.
Assigning LUNs to Storage Pools
You should set up a storage pool using LUNs that have similar capacity and
performance characteristics.
24 Chapter 2 Setting Up a Storage Area Network
To provide high performance, Xsan uses the RAID 0 scheme to stripe data across the
LUNs in a storage pool. This requires that the LUNs in the pool be the same size. If you
set up a storage pool using LUNs of different sizes, Xsan uses available space on each
LUN equal to the capacity of the smallest LUN. If the LUNs vary in size, this can result in
wasted capacity. For example, if you assign 240 GB and 360 GB RAID arrays to a storage
pool, 120 GB of the larger array will not be used. By combining LUNs with similar
capacities, you avoid wasting available storage.
If you want to set up a storage pool for use by a high performance application, assign
similarly high speed LUNs. Assign slower LUNs to a storage pool where you keep data
that doesn’t have critical performance requirements.
Creating storage pools from LUNs that are hosted on different drive modules and
different RAID controllers increases performance by increasing the parallelism of data
transfers. For example, a storage pool consisting of two LUNs, each a single drive
module on the left side of an Xserve RAID, will not be as fast as a similarly sized storage
pool made up of two LUNs that are single slices across all seven drives, one slice on
each controller. In the first case, all transfers go through a single RAID controller to just
two drives; in the second case the same transfer is spread across two RAID controllers
and fourteen drives.
Assigning Storage Pools to Volumes
After you decide how to combine available LUNs into storage pools, assign the storage
pools to the volumes you want to create.
For best performance, create separate storage pools for file system metadata and
journal data.
Note: No storage pool or volume can be larger than 16 TB.
Deciding Which Clients to Mount a Volume On
If you create multiple volumes, decide which volumes should be mounted on which
clients.
Choosing Controllers
You must choose at least one computer to be the SAN controller, the computer that is
responsible for managing file system metadata.
Note: File system metadata and journal data are stored on selected SAN volumes, not
on the controller itself. For more information, see “Choosing Where to Store Metadata
and Journal Data” on page 26.
If you have a small number of clients or if performance is not critical you can use a
single computer as both controller and client. You can even set up a SAN consisting of
a single storage device and a single computer that acts as both controller and client (to
provide network attached storage, for example).
Chapter 2 Setting Up a Storage Area Network 25
If high availability is important, you should use at least two controllers, one as the
primary controller and one as a standby. You can specify additional controllers as
needed, and set their failover priorities to determine the order in which they are tried if
the primary controller stops responding.
Choosing Standby Controllers
To be sure that SAN volumes are always available, set up at least one standby controller
that can take over if your primary metadata controller fails. A standby controller also
makes it possible for you to upgrade software on the controllers without interrupting
user access to SAN volumes.
Combining Clients and Controllers
The same computer can function as both a metadata controller and a client. It’s
possible, for example, to set up a SAN consisting of a single Xserve RAID and one
computer that acts as both controller and client. Any computer you specify as a
controller can also act as a client.
If, for example, you don’t have a computer to dedicate as a standby controller, you can
assign a computer that is normally used as a client to take over controller duties if the
primary controller fails.
To keep clients and controllers separate, you can set up client-only computers for your
users.
Choosing Where to Store Metadata and Journal Data
The metadata and journal data that describe a volume are not stored on the volume’s
metadata controller but on the volume itself. By default, they are stored on the first
storage pool in the volume. If the volume consists of more than one storage pool, you
can choose which storage pool is used to store metadata and journal data.
In most cases, storing metadata and journal data on the same storage pool as user data
results in good performance. However, for the best possible performance, consider
storing metadata and journal data on separate storage pools within the volume.
Choosing an Allocation Strategy
The allocation strategy you choose for a volume determines the order in which its
storage pools are filled with data. You can choose round robin, fill, and balance.
If you choose round robin, Xsan writes new data in turn to each storage pool in the
volume.
If you choose fill, Xsan writes all new data to the first storage pool in the volume until
that storage pool is full, then moves to the next storage pool. This is a good choice if
you want to keep a particular storage pool unused as long as possible.
If you choose balance, Xsan writes new data to the storage pool with the most free
space.
26 Chapter 2 Setting Up a Storage Area Network
Connecting Computers and Storage Devices
Before you set up your Xsan SAN, connect client computers, controller computers, and
storage devices to the SAN’s Fibre Channel and Ethernet networks. Make sure your
networks meet the requirements summarized under “Fibre Channel Fabric” on page 19
and “Ethernet TCP/IP Network” on page 19.
Preparing LUNs (RAID Arrays and Slices)
Xserve RAID systems usually come preconfigured with two RAID 5 arrays, one on each
side (on each controller). So, out of the box, each Xserve RAID provides two LUNs. If this
suits your needs, no other preparation is needed.
If you want to set up some other combination of RAID arrays or slices, you need to do
so using the RAID Admin utility before you can add the resulting LUNs to your SAN’s
storage pools. For help using RAID Admin, see Using RAID Admin 1.2 and Disk Utility
(available at www.apple.com/server/documentation). For information on choosing a
RAID scheme, see “Choosing RAID Schemes for LUNs” on page 23.
Note: You don’t need to use Disk Utility to format arrays or slices for use with Xsan. The
LUNs are labeled and initialized when you add them to a storage pool using Xsan
Admin. After they are labeled, the LUNs can’t be viewed or modified using Disk Utility.
Be sure to create arrays of similar size if you plan to combine them into the same
storage pool. For more information, see “Assigning LUNs to Storage Pools” on page 24.
Chapter 2 Setting Up a Storage Area Network 27
Using the Xsan Admin Application
You use the Xsan Admin application to set up and manage SANs. You can use Xsan
Admin to manage a SAN from any computer that has access to the SAN’s TCP/IP
subnet.
Xsan Admin is installed in /Applications/Server.
Installing Xsan Admin Separately
Xsan Admin is included when you install the Xsan software on SAN controller and
client computers. You can also install just Xsan Admin on any other computer you want
to use to manage the SAN. For help, see page 31.
Connecting Through a Firewall
If there is a firewall between the SAN and the computer you’re using to run Xsan
Admin, you need to open port 311 in the firewall so Xsan Admin can communicate with
the SAN computers.
Xsan Admin Preferences
Open Xsan Admin and choose Xsan Admin > Preferences to adjust these settings:
• The number of states reported by the colored status indicator next to SANs and
volumes in the SAN Components list:
• The use of SSL or digital signatures to secure communications
• The use of DNS to display names instead of IP addresses
• Smoothing of SAN utilization graphs
• Connection alerts
• SAN status refresh interval
• The amount of log information displayed
Getting Help
Xsan Admin includes online help. Choose Help > Xsan Admin Help or click the help
button in any dialog or pane where it appears.
Using the Command Line
If necessary, you can perform many Xsan setup and management tasks from a shell
command prompt. For more information, see Appendix B.
28 Chapter 2 Setting Up a Storage Area Network
SAN Setup Summary
You’ll perform the following tasks to set up your first Xsan storage area network. Details
for each task are on the indicated pages.
1 Set up the Fibre Channel network (page 30)
2 Set up the Ethernet network (page 30)
3 Set up SAN users and groups (page 30)
4 Set up RAID arrays (page 30)
5 Install Xsan software on SAN computers (page 31)
6 Log in to the SAN (page 31)
7 Choose a controller and add clients (page 32)
8 Label and initialize available LUNs (page 33)
9 Create volumes (page 34)
10 Add storage pools to volumes (page 35)
11 Add LUNs to storage pools (page 35)
12 (Optional) Set up status notifications (page 36)
13 (Optional) Assign folders to storage pools (page 36)
14 (Optional) Set user and group quotas (page 37)
15 Start the volumes and mount them on clients (page 37)
Chapter 2 Setting Up a Storage Area Network 29
Setting Up an Xsan Storage Area Network
Step 1: Set Up the Fibre Channel Network
m
Connect controller computers, client computers, and Xserve RAID storage systems to a
Fibre Channel network. Be sure to configure the switch and make the connections so
that you create a Fibre Channel fabric.
For more information, see the guidelines and requirements under “Fibre Channel
Fabric” on page 19.
Step 2: Set Up the Ethernet Network
m
Connect controller computers, client computers, and Xserve RAID systems to a private
TCP/IP subnet, or to the same subnet of an intranet.
Follow the guidelines summarized under “Ethernet TCP/IP Network” on page 19.
Step 3: Set Up SAN Users and Groups
m
If you already have a centralized directory of users and groups, use the Directory Access
application on each SAN computer to choose that directory for authentication. If you
don’t have a central directory, you can set one up using Workgroup Manager and the
Open Directory service in Mac OS X Server. Otherwise, you need to recreate the same
set of users and groups on each SAN computer.
Important: If you create users and groups individually on each SAN computer, be sure
that each user or group name is assigned the same numeric user ID (UID) or group ID
(GID) on all SAN computers. One way to do this is to create an identical list of users and
groups in the same order on each computer.
Step 4: Create RAID Arrays (Prepare LUNs)
m
New Xserve RAID systems are usually preconfigured as two RAID 5 arrays that are ready
to use as LUNs.To set up some other configuration of LUNs, use the RAID Admin
application to create RAID arrays or slices on your Xserve RAID systems. For help
choosing other RAID schemes, see “Choosing RAID Schemes for LUNs” on page 23.
30 Chapter 2 Setting Up a Storage Area Network
Loading...