Apple XSAN 1.X User Manual

0 (0)
Apple XSAN 1.X User Manual

Xsan

Deployment and

Tuning Guide

Guidelines and examples for configuring Xsan storage area network volumes

K Apple Computer, Inc.

© 2005 Apple Computer, Inc. All rights reserved.

Under the copyright laws, this manual may not be copied, in whole or in part, without the written consent of Apple.

The Apple logo is a trademark of Apple Computer, Inc., registered in the U.S. and other countries. Use of the “keyboard” Apple logo (Option-Shift-K) for commercial purposes without the prior written consent of Apple may constitute trademark infringement and unfair competition in violation of federal and state laws.

Every effort has been made to ensure that the information in this manual is accurate. Apple is not responsible for printing or clerical errors.

Apple

1 Infinite Loop

Cupertino, CA 95014-2084 408-996-1010 www.apple.com

Apple, the Apple logo, Final Cut Pro, Mac, Macintosh, the Mac logo, Mac OS, and Xserve are trademarks of Apple Computer, Inc., registered in the U.S. and other countries.

Finder and Xsan are trademarks of Apple Computer, Inc.

StorNext and ADIC are registered trademarks of Advanced Digital Information Corporation.

Other company and product names mentioned herein are trademarks of their respective companies. Mention of third-party products is for informational purposes only and constitutes neither an endorsement nor a recommendation. Apple assumes no responsibility with regard to the performance or use of these products.

The product described in this manual incorporates copyright protection technology that is protected by method claims of certain U.S. patents and other intellectual property rights owned by Macrovision Corporation and other rights owners. Use of this copyright protection technology must be authorized by Macrovision Corporation and is intended for home and other limited viewing uses only unless otherwise authorized by Macrovision Corporation. Reverse engineering or disassembly is prohibited.

Apparatus Claims of U.S. Patent Nos. 4,631,603, 4,577,216, 4,819,098 and 4,907,093 licensed for limited viewing uses only.

Simultaneously published in the United States and Canada.

019-0242/09-02-05

Contents

Preface

5

About This Guide

 

5

Using This Guide

 

6

For More Information

 

6

Notation Conventions

Chapter 1

7

Setup and Tuning Guidelines

 

7

Setting Up the Ethernet TCP/IP Network

 

8

Set Up a Private Metadata Network

 

8

Use Switches Instead of Hubs

 

8

Setting Up the Fibre Channel Network

 

8

Verify Base Fibre Channel Performance

 

8

If Your Fibre Channel Fabric Is Running Slower Than Expected

 

9

Configuring Xserve RAID Systems

 

9

Install the Latest Firmware

 

9

Connecting Xserve RAID Systems to an Ethernet Network

 

10

Choosing RAID Levels for LUNs

 

10

7-Drive LUN vs 6-Drive LUN With Hot Spare

 

10

Create One LUN per Xserve RAID Controller

 

11

Adjusting Xserve RAID Fibre Channel Settings

 

11

Adjusting Xserve RAID Performance Settings

 

12

Estimating Base Xserve RAID Throughput

 

13

Configuring the Xsan File System

 

13

Organizing LUNs, Storage Pools, and Volumes

 

13

Volume and Storage Pool Settings

 

15

About SAN Write Latency

Chapter 2

17

Using the Xsan Tuner Application

 

17

Where to Get Xsan Tuner

 

17

Installing Xsan Tuner

 

18

Starting Xsan Tuner

 

18

About the Tests

 

18

The UNIX Read and Write Tests

 

18

The Final Cut Pro Read and Write Tests

3

 

19

About the Test Files

 

20

Performing a Test

 

23

Interpreting Test Results

 

23

Understanding UNIX Test Results

 

24

Understanding Final Cut Pro Test Results

Chapter 3

27

Deployment Examples

 

27

Which Example Should You Follow?

 

28

Example 1: Network Attached Storage

 

29

Objectives of This Configuration

 

29

Deployment Decisions

 

32

Example 2: Storage for Computational Clusters

 

32

Requirements for This Example

 

32

Small Computational Cluster Setup

 

34

A Larger Computational Cluster

 

35

Volume Configuration

 

36

Example 3: Storage for Video or Film Production Group

 

37

Deployment Decisions

4

Contents

 

 

About This Guide

Preface

This guide shows you how to choose the best Xsan setup and configuration options for your users and applications.

The performance and availability of an Xsan volume depend on a variety of parameters, including how you organize available storage, how you configure the SAN’s Fibre Channel and Ethernet networks, and how you set basic file system parameters such as block size and stripe breadth. This guide goes beyond the basic instructions in the Xsan Administrator’s Guide to help you choose options that result in the best performance from your Xsan volumes. The guide includes:

ÂGuidelines for configuration choices

ÂInstructions for using the Xsan Tuner application to measure SAN and volume performance

ÂSample configurations for network attached storage, high-performance computing, and video production

Using This Guide

ÂFor general tips and guidelines, see Chapter 1.

ÂFor help using the Xsan Tuner application, see Chapter 2.

ÂFor sample deployments, see Chapter 3.

5

For More Information

The Xsan Administrator’s Guide contains basic instructions for setting up Xsan volumes along with information about managing Xsan volumes, including problem-solving tips and command-line alternatives for common tasks.

You can find the guide:

ÂOn the Xsan Installer disc

ÂIn the folder /Library/Documentation/Xsan on any computer where Xsan is installed

ÂAt www.apple.com/server/documentation

You can also check the Xsan web pages at www.apple.com/xsan.

Notation Conventions

The following conventions are used in this book wherever shell commands or other command-line items are described.

Notation

Indicates

monospaced font

A command or other terminal text

 

 

$

A shell prompt

 

 

[text_in_brackets]

An optional parameter

 

 

(one|other)

Alternative parameters (type one or the other)

 

 

underlined

A parameter you must replace with a value

 

 

[...]

A parameter that may be repeated

 

 

<anglebrackets>

A displayed value that depends on your SAN configuration

 

 

6

Preface About This Guide

 

 

Setup and Tuning Guidelines

1

 

 

 

This chapter offers guidelines for making configuration choices that can affect SAN performance.

How quickly SAN clients can transfer data to and from Xsan volumes depends on a variety of factors, including:

ÂThe configuration of and load on the SAN’s Ethernet network

ÂThe layout and performance of the SAN’s Fibre Channel network

ÂSettings for the Xserve RAID systems that provide LUNs

ÂThe organization of Xsan volumes and storage pools

ÂXsan file system settings

Setting Up the Ethernet TCP/IP Network

Ethernet connections are used in several ways in an Xsan storage area network:

ÂXsan clients and controllers use Ethernet to exchange volume metadata.

ÂXsan clients can use Ethernet for access to networks outside the SAN (campus or corporate intranet or the Internet).

ÂXsan controllers can use Ethernet connections for remote management.

ÂXserve RAID systems can use Ethernet connections for system management.

ÂFibre Channel switches can use Ethernet connections for switch management.

You have two basic options:

ÂUse one Ethernet network for all traffic. This is the less expensive option, but is also less secure and might not provide the best possible performance.

ÂUse two separate networks; one for metadata and another for all other IP traffic. This configuration is slightly more expensive (requiring two Ethernet adapters for each computer) but offers greater security and better performance because routine network traffic doesn’t interfere with SAN volume metadata traffic.

7

Set Up a Private Metadata Network

Non-SAN-related Ethernet traffic can interfere with the exchange of metadata among Xsan controllers and clients. For example, using the same connection for both Xsan metadata exchange and Internet access can slow file system performance. Similarly, using the same Ethernet network to connect client computers to directory services and SAN metadata can affect SAN performance.

If SAN performance is critical for your users or applications, keep all extraneous traffic off the network that clients and controllers use to exchange metadata. For best SAN performance, set up a private Ethernet TCP/IP network for the exclusive use of Xsan clients and controllers. For other types of network traffic, including Internet access, Xserve RAID and Fibre Channel switch management, remote SAN management, or directory services, connect each client or controller to a second, private SAN Ethernet network using a second network adapter.

Use Switches Instead of Hubs

Ethernet switches generally offer better performance than hubs. Use switches, not hubs, in the SAN Ethernet network.

Setting Up the Fibre Channel Network

Xsan uses Fibre Channel connections to:

ÂTransfer user data directly between clients and data storage pools

ÂTransfer metadata between controllers and metadata storage pools

Verify Base Fibre Channel Performance

Because the devices connected to a Fibre Channel network automatically adjust their speed to match the slowest device on the fabric, it is important to check that all connections in the fabric are operating at 2 GB/s.

To check Fibre Channel connection performance:

mUse the management software provided with your Fibre Channel switches to test the performance of your Fibre Channel fabric.

If Your Fibre Channel Fabric Is Running Slower Than Expected

The following paragraphs list things you can check if your Fibre Channel fabric is not running at the expected 2 GB/s.

Check Cables

One faulty cable in a fabric can slow the entire network. Check all cables to make sure they are capable of full transmission speed. Use your switch management software to isolate the faulty cable by checking the performance of specific connections.

8

Chapter 1 Setup and Tuning Guidelines

 

 

Use Qualified Transceivers in Matching Pairs

Check with the manufacturers of the devices you are connecting to your fabric to be sure that the transceivers (GBICs) you are using are qualified for use with their devices.

Also, use identical transceivers (same manufacturer and model number) on both ends of each cable. Mismatched optical transceivers (even if they are both separately qualified for use with your devices) can cause Fibre Channel communication errors and degrade SAN performance.

Check Fibre Channel Switch Port Configuration

The Request for State Change Notifications (RSCN) that is generated when a client on the SAN restarts can cause dropped frames in video streams to other clients.

To avoid interrupting SAN traffic to other clients if one client restarts, check your

Fibre Channel switch documentation to see if you can configure the switch to suppress RSCNs on initiator ports. (On Qlogic switches, for example, this feature is called I/O StreamGuard.)

Connect Devices to Specific Blades

If your Fibre Channel switch is based on a blade architecture, you might be able to improve performance by:

ÂConnecting pairs of devices that routinely exchange large volumes of data to the same blade in the switch

ÂDistributing loads across multiple blades instead of concentrating all of the load on one or two blades

Configuring Xserve RAID Systems

Follow these guidelines when you set up your Xserve RAID systems for use as Xsan LUNs.

Install the Latest Firmware

To be sure you get the best performance and reliability from your Xserve RAID systems, be sure to install the latest available firmware.

To check for firmware updates:

m Visit www.apple.com/support/xserve/raid/

Connecting Xserve RAID Systems to an Ethernet Network

For best performance, don’t connect Xserve RAID controller Ethernet management ports to the SAN’s metadata network. Connect the ports to a separate Ethernet network.

Chapter 1 Setup and Tuning Guidelines

9

 

 

Choosing RAID Levels for LUNs

Use RAID 1 for metadata LUNs and RAID 5 for data LUNs.

Use RAID 1 for Metadata LUNs

RAID 1 (mirroring) can give slightly better performance than the default RAID 5 scheme for the small, two-drive metadata LUNs that Xsan uses to store volume information.

A single drive is almost always adequate for storing the primary volume metadata (10 GB of metadata space is enough for approximately 10 million files). The second, mirror drive protects you against metadata loss.

Use RAID 5 for Data LUNs

Xserve RAID systems are optimized for excellent performance and data redundancy using a RAID 5 scheme. (RAID 5 stripes data across the available drives and also distributes parity data across the drives.) Xserve RAID systems ship already configured as RAID 5 LUNs. RAID 0 (striping with no parity) might give slightly better write performance but provides no data recovery protection, so RAID 5 is always a better choice for LUNs used to store user data.

7-Drive LUN vs 6-Drive LUN With Hot Spare

For best performance, use full 7-drive RAID 5 LUNs. Even if a drive fails, the degraded 6-drive array will continue to provide excellent performance. Keep in mind, however, that the array is unprotected against the loss (however unlikely) of a second drive until someone replaces the original faulty drive.

If you can’t afford to have the LUN operating in a degraded state until someone replaces the faulty drive, you can configure your Xserve RAID systems as 6-drive RAID 5 arrays and use the seventh drive as a hot spare. Data on the faulty drive is reconstructed automatically without human intervention.

Create One LUN per Xserve RAID Controller

For high performance data sets, create only one LUN on each Xserve RAID controller (one array on each side of the system). Xserve RAID systems ship with one RAID 5 array on each controller.

Working With LUNs Larger Than 2 Terabytes

The capacity of an Xserve RAID array can exceed 2 terabytes (TB) if the system contains large drive modules. However, Xsan can’t use a LUN that is larger than 2 TB. If you set up your Xserve RAID systems as one array per controller, as suggested above, you can’t take advantage of the array capacity beyond 2 TB. To use as much available space as possible, you can move drive modules to other controllers or slice a large array into two smaller (less than 2 TB) LUNs. Slicing an array might, however, slow SAN performance.

Note: For the best possible SAN performance, don’t slice an array to create multiple

LUNs on a single controller.

10

Chapter 1 Setup and Tuning Guidelines

 

 

Adjusting Xserve RAID Fibre Channel Settings

There are several Xserve RAID settings that can affect the Fibre Channel performance of the device and the SAN as a whole.

Fibre Channel Speed

Be sure the Fibre Channel connection is set to operate at 2 GB/s.

Fibre Channel Topology

To add an Xserve RAID system to a Fibre Channel fabric, set the topology to Automatic.

Disable Hard Loop ID

Don’t enable hard loop IDs for Xserve RAID systems in a Fibre Channel fabric.

To adjust Xserve RAID Fibre Channel settings:

mOpen RAID Admin, choose a system, click Settings, and enter the management password for the system. Then click Fibre Channel.

Adjusting Xserve RAID Performance Settings

Xserve RAID performance settings, affecting parameters such as drive caching, controller caching, and read prefetching, can have a significant effect on Xsan volume performance. Follow these guidelines.

Enable Drive Cache

In addition to the caching performed by the Xserve RAID controller, each drive in an array can perform its own caching at the drive level to improve performance.

Important: If you enable drive cache for an Xserve RAID set, be sure that the system is connected to a UPS. Otherwise, you could lose cached data if the power fails.

To enable drive cache for an Xserve RAID array:

mOpen the RAID Admin application, select the RAID system, and click Settings. Then click Performance and enable Drive Cache for the array.

Enable Controller Write Cache

Without RAID controller write caching, a request to write data to the associated LUN is not considered finished until the data has been completely written to the physical disks that make up the array. Only then can the next write request be processed. (This is sometimes called “write-through caching.”)

When the RAID controller write cache is enabled, a request to write data is considered finished as soon as the data is in the cache. This is sometimes called “write-back caching.”Write requests are processed more quickly because the file system only needs to write to the fast cache memory and doesn’t need to wait for the slower disk drives.

Always be sure to enable write caching on controllers that support metadata storage pools.

Chapter 1 Setup and Tuning Guidelines

11

 

 

Although some large write requests might benefit from caching, often they do not. By placing a volume’s metadata storage pool on a controller separate from the data storage pools, you can configure the metadata controller to use caching and the data controller to run without caching.

When the file system is relying on caching in this way, you must guarantee that data in the cache isn’t lost before it is actually written to disk. Data that has been written to disk is safe if the power fails, but data in a cache is not. So, to be sure that a power failure can’t cause the loss of cached data, protect your Xserve RAID systems with controller backup batteries or an uninterruptable power supply (UPS).

Important: If you enable Controller Write Cache on an Xserve RAID system, be sure that the system includes controller backup batteries and, preferably, is connected to a UPS.

To enable Xserve RAID write cache:

mOpen the RAID Admin application, select the RAID system, and click Settings. Then click Performance and enable Write Cache for each controller.

Set Read Prefetch to 8 Stripes

Read prefetch is a technique that improves file system read performance in cases where data is being read sequentially, as in the case of audio or video streaming, for example. When read prefetch is enabled, the controller assumes that a read request for a particular block of data will be followed by requests for subsequent, adjacent data blocks. To prepare for these requests, the controller reads not only the requested data, but also the following data, and stores it in cache memory. Then, if the data is actually requested, it is retrieved from the fast cache instead of from the slow disk drives.

Read prefetch is always enabled on Xserve RAID systems, though you can adjust the amount of data that is read. If you’re using other RAID systems, check the documentation to find out how to enable read prefetch.

To adjust the Xserve RAID read prefetch size:

mOpen the RAID Admin application, select the RAID system, and click Settings. Then click Performance and select a Read Prefetch size for each controller.

The default of 8 stripes is best for most applications.

Estimating Base Xserve RAID Throughput

To estimate how many Xserve RAID systems you need to support specific throughput requirements, you can assume that one Xserve RAID with 14 drives set up as two RAID 5 arrays can handle a minimum of 160 MB of data per second (80 MB/s per RAID controller). This value is applicable to video streaming applications; other applications might achieve higher data rates.

Overall performance is also affected by SAN latency; see “About SAN Write Latency” on page 15.

12

Chapter 1 Setup and Tuning Guidelines

 

 

Loading...
+ 27 hidden pages