HP Data Protector V5.5 White Paper

HP OpenView Storage Data Protector 5.5

Advanced Backup to Disk

Performance White Paper

Executive Summary .............................................................................................................................. 3

Objective and Target Audience............................................................................................................. 4

Introducing HP OpenView Storage Data Protector 5.5 ............................................................................. 5

File Library ...................................................................................................................................... 5

Object Copy ................................................................................................................................... 5

Tape Drives......................................................................................................................................... 6

Basics of Backup Performance............................................................................................................... 8

Disk to Disk to Tape (D2D2T) Data Protection Architecture ...................................................................... 10

Test Data .......................................................................................................................................... 12

HPCreateData ............................................................................................................................... 12

Typical Files .................................................................................................................................. 12

Small Files..................................................................................................................................... 13

HPReadData.................................................................................................................................. 14

Configuration.................................................................................................................................... 15

HP ProLiant DL380 G4 Server.......................................................................................................... 15

HP OpenView Storage Data Protector 5.5 ........................................................................................ 15

Internal Database (IDB) Logging Level ........................................................................................... 15

File Library ................................................................................................................................16

File System Tree Walk................................................................................................................. 18

HP StorageWorks SDLT 320 Tape Drive ........................................................................................... 18

HP StorageWorks Ultrium 460 Tape Drive ........................................................................................ 19

HP StorageWorks Ultrium 960 Tape Drive ........................................................................................ 20

HP StorageWorks Modular Smart Array 1000 (MSA1000) ................................................................ 21

HP StorageWorks Modular Smart Array 1500 (MSA1500) ................................................................ 22

RAID 0 and 1 Logical Array ........................................................................................................ 22

RAID 0 and 4 Logical Arrays ....................................................................................................... 23

RAID 1 and 1 Logical Array ........................................................................................................ 24

RAID 1+0 and 3 Logical Arrays ................................................................................................... 25

RAID 5 and 2 Logical Arrays ....................................................................................................... 26

Test Data....................................................................................................................................... 27

Library and Tape Tools ................................................................................................................... 27

Results .............................................................................................................................................. 29

Tape Drive Write Test..................................................................................................................... 29

MSA1000 Write Test ..................................................................................................................... 29

MSA1500 Write Test ..................................................................................................................... 29

MSA1000 Read Test ...................................................................................................................... 30

MSA1500 Read Test ...................................................................................................................... 30

Backup to NULL Device................................................................................................................... 31

Backup to Tape.............................................................................................................................. 31

Advanced Backup to Disk ............................................................................................................... 33

MSA1000 to File Library (MSA1500)........................................................................................... 33

File Library (MSA1500) to Tape................................................................................................... 33

Restore ......................................................................................................................................... 34

File Library (MSA1500) to MSA1000 ........................................................................................... 34

Tape to MSA1000 ..................................................................................................................... 34

Observations..................................................................................................................................... 35

HP ProLiant DL380 G4 Server.......................................................................................................... 35

Tape Drives and LTT ....................................................................................................................... 35

MSA Read and Write Tests ............................................................................................................. 36

MSA1000................................................................................................................................. 36

MSA1500................................................................................................................................. 36

Backup to NULL Device................................................................................................................... 37

Backup to Tape.............................................................................................................................. 37

Advanced Backup to Disk ............................................................................................................... 38

Restore ......................................................................................................................................... 40

Tuning Recommendations ................................................................................................................... 41

General ........................................................................................................................................ 41

Fragmentation of File Library File Systems ......................................................................................... 42

Summary and Conclusions.................................................................................................................. 45

For more information.......................................................................................................................... 46

Executive Summary

This white paper provides performance-related information for HP OpenView Storage Data Protector

5.5 and the Advanced Backup to Disk feature.

This white paper covers HP ProLiant Windows 2003 server connected to HP StorageWorks Modular Smart Array (MSA) disk arrays and different tape drive technologies (LTO and SDLT).

The proof points are all Windows-based for simplification of equipment needs, but the lessons will still hold good for heterogeneous environments.

As a result of these tests, several recommendations and rules of thumb have emerged:

• HP OpenView Storage Data Protector tuning can help to improve the performance, e.g. by

modifying tape block and file depot sizes. Please check chapter

• HP StorageWorks Ultrium 960 tape drives are best utilized with a block size of 256 KB as

described in “Getting the most performance from your HP StorageWorks Ultrium 960 tape drive white paper” (downloadable from

• A single high-performance tape drive (Ultrium 960) causes less CPU load than multiple slower tape

drives (e.g. 2 x SDLT 320) at a comparable or better transfer rate.

• For entry-level JBOD and low-end disk array usage, it is important to understand what your disk

subsystem is capable of delivering. This can be done using HP performance assessment tools (downloadable from within the HP industry-leading Library and Tape Tools diagnostics (downloadable from

http://www.hp.com/support/tapetools).

• The configuration of disk arrays can have a remarkable impact on the backup and restore

performance. Important parameters are the configured number of logical arrays, logical volumes and type of RAID levels. The configuration of logical arrays and volumes should reflect the internal disk array layout. E.g., disk arrays with 2 SCSI buses could perform best by creating 2 logical arrays with one logical volume each. RAID levels must be chosen carefully. RAID 0 provides the best performance but should not be considered due to missing fault tolerance. Therefore, it is recommended to configure RAID 5, which fits best for staging areas. It is space and cost efficient and provides a good read performance.

• Serial ATA (SATA) disks used as secondary disk storage arrays can have slower rotational speeds

than their SCSI counterparts (for example, the HP StorageWorks Modular Smart Array SATA drives are only 7.2K rpm) and although they are high capacity (250 GB), their performance reflects their pricing. Therefore, backing up to tape from a staged (secondary) disk array can be slower than backing up directly from the primary storage to tape in some circumstances. Ironically, tape is now faster than disk. Disk staging is useful, however, for gathering several small files into a single object, or backing up slow networked hosts before the data is sent to tape. Both small files and slow hosts can cause very slow backups.

• The restore of many small files (22 million in this setup) could cause serious file system bottlenecks.

Data Protector must wait until Windows and the belonging NTFS responds.

• Finally, the test environment with the HP ProLiant Server DL380 G4 and MSA1000/1500 disk

arrays is capable to manage data with low CPU utilization. The typical file backup (from MSA100) directly to the Ultrium 960 tape drive showed a high transfer rate (91,97 MB/s or 323,33 GB/h) together with a low average CPU load (11%). The comparable disk backup (from MSA1500) to tape still showed an acceptable transfer rate (43,37 MB/s or 152,42 GB/h) together with a low average CPU load (9%).

http://www.hp.com/support/pat). The performance tools are also embedded

http://h18006.www1.hp.com/storage/tapewhitepapers.html).

Tuning Recommendations.

Objective and Target Audience

The main objective of this white paper is to educate and inform users of the HP OpenView Storage Data Protector 5.5 Advanced Backup to Disk feature about what levels of performance are achievable in different backup scenarios.

The emphasis is in showing what is typical and not what best-case scenarios are. This white paper highlights where the current performance bottlenecks are and how these might be overcome.

The target audience for this white paper are system integrators and solution architects and indeed anyone involved in getting the best backup performance out of their infrastructure investments. Throughout the white paper the HP StorageWorks Ultrium may also be referred to as HP LTO.

Introducing HP OpenView Storage Data Protector 5.5

HP OpenView Storage Data Protector (DP) is software that manages backup and recovery from both disks and tapes, delivering maximum data protection while providing continuous 24x7 business operations. The software is designed to simplify and to centralize backup and recovery operations by integrating a variety of techniques to eliminate backup windows. These range from on-line backup, open file backup, and instant recovery or zero-downtime backups.

HP OpenView Storage Data Protector simplifies the use of complex backup and recovery procedures with the fastest installation, automated routine tasks, and easy-to-use features. The ideal solution to reduce IT costs and complexity while remaining reliable and scalable to grow from single server environments to the largest distributed enterprise infrastructures, providing broad compatibility of operating systems, applications, drives, libraries, and disk arrays.

The Advanced Backup to Disk feature improves the backup process with continuous backup of transaction log files, backup of slow clients without multiplexing, easy resource access and sharing, plus backup in tape-less branch offices, while offering fast and easy configuration and licensing. Furthermore, it allows single file restore directly from disk or tape.

The following subchapters required for Advanced Backup to Disk.

For further information on Advanced Backup to Disk, please refer to the Disk-Assisted Backup White Paper (downloadable from

File Library and Object Copy give a short overview about the features

http://www.openview.hp.com/sso/searchdocs?prod=DATAPRO&ct=TWP).

File Library

DP provides a device type called “File Library”. It focuses on low cost disk arrays, especially the current HP SATA disk array, which is positioned mainly as a backup device. In case the file library is running out of free space, new backup capacity can be assigned automatically.

The file library is configured by defining mount points where DP will create its “media” and optionally, the number of simultaneous “writers” that will be used. Then, DP can utilize this just like any other physical backup device and will auto-create the media files on the fly as required. This capability is called “Advanced Backup to Disk”.

Object Copy

Object Copy provides the capability to copy from one type of media to another. This means that you can perform a backup to the file library, and then automatically (immediately after backup or at a scheduled time) copy the file-based “media” to a physical media such as LTO.

Tape Drives

This white paper covers the Ultrium 960, Ultrium 460 and SDLT 320 tape drives.

The HP StorageWorks Ultrium 960 tape drive performance is best utilized for high-performance backup in a disk and tape SAN environment, backing up directly from a high-performance disk subsystem (from primary disk storage, split mirrors, snapshots, or clones). The HP StorageWorks Ultrium 960 is not ideal (performance wise) for network backups (e.g. 100BASE-T) or backup from lower performance disk arrays. HP recommends matching appropriate LTO technology generations to your server capabilities. However, if you have capacity needs met by a higher generation product, you can utilize it in the knowledge that it will give you performance headroom in future.

Figure 1. HP StorageWorks Ultrium 960 Tape Drive

The HP StorageWorks Ultrium 460 tape drive fits very well in most of all server environments by delivering a considerable capacity and performance.

Figure 2. HP StorageWorks Ultrium 460 Tape Drive

The HP StorageWorks SDLT 320 tape drive is very useful for existing DLT and SDLT-based environments, particularly for backward-read compatibility.

Figure 3. HP StorageWorks SDLT 320 Tape Drive

The following table shows the major differences:

Table 1. Tape Drive Characteristics

Specification HP StorageWorks

Ultrium 960

Capacity (native) 400 GB 200 GB 160 GB

Transfer rate (native) Up to 80 MB/s Up to 30 MB/s Up to 16 MB/s

Data rate matching 27 – 80 MB/s 10 – 30 MB/s N/A

Host interfaces

Head channels 16 8 16

Number of tracks 704 512 448

Media length 680 m 609 m 560 m

WORM media support Yes No No

Ultra320, 2/4 Gigabit FC

HP StorageWorks

Ultrium 460

Ultra160, 2 Gigabit FC Ultra2

HP StorageWorks SDLT 320

Media compatibility

Read

Write

Ultrium Gen. 1, 2, 3

Ultrium Gen. 2, 3

Ultrium Gen. 1, 2

DLTtape IV

Super DLTtape , SDLT 220

Basics of Backup Performance

Backup performance will always be limited by one or more bottlenecks in the system, of which the tape drive is just one part. The goal is to make the tape drive the bottleneck. That way the system will achieve the performance figures as advertised on the drive's specification sheet.

Please note that backup jobs can stress hardware resources up to their highest limit, which would never happen during normal application load. This puts the emphasis on the rest of the system and causes failures, which are based in the involvement of many components and their time-critical handshake of data.

The flow of data throughout the system must be fast enough to provide the tape drive with data at its desired rates. High-speed tape drives, such as the Ultrium 960, are so fast that this can be a difficult goal to achieve. If the backup performance is not matching the data sheet of the tape drive, then there is a bottleneck somewhere else in the system.

One single component, like the 100BASE-T network, can decrease a SDLT or LTO tape drive performance to a very low transfer rate (this would be a very good use case for first staging the data on disk and then backing it up to tape.)

All components must be considered for getting the theoretical backup performance. Practical performance data can only be obtained from benchmarks.

Factors, which critically affect the backup speed:

• Multiplexing

This allows better bandwidth utilization of the tape drive during backup but can slow down restore performance because all the data is interleaved all the way down the tape. Therefore, the time spent to perform a single stream restore is higher due to other streams having to be read (and potentially ignored).

• Disk and Tape Buffers

DP offers a set of advanced options for backup devices and disk agents. The default settings are device-based and match most of all backup environments. Ultrium 960 is an exception and requires a modification as described in chapter

• Data File Size

The larger the number of smaller files there are, the larger the overhead there is associated with backing them up. The worst-case scenario for backup is large numbers of small files due to system overhead of file access.

• Data compressibility

Incompressible data will back up slower than higher compressible data. JPEG files, for example, are not very compressible, whereas database files can be highly compressible. The accepted standard for quoting tape backup specifications revolves around an arbitrary figure of 2:1 compressible data.

• Disk Array Performance

It is often overlooked that you cannot put data on tape any faster than you can read it from disk. Backup is more sequential in nature than random (from a disk array access perspective). Disk array performance depends on the number of disks, RAID configuration, the number of Fibre Channel ports to access the array, queue depth available, and so on. HP has written several utilities to “read data” from disk arrays and deliver a performance value. This enables users to determine the throughput of their disk arrays when operating in backup mode and performing file system traversals typical of this activity. These Performance Assessment Tools (PAT utilities) can be downloaded from

http://www.hp.com/support/pat. The performance tools are also embedded within the HP industry-

leading Library and Tape Tools diagnostics, which can be downloaded from

http://www.hp.com/support/tapetools.

Tuning Recommendations.

• Transfer Size

Transfer size is the overall total size of the SCSI transfer within a single SCSI command. In some operating systems, there is limit set on this. For example, in Microsoft Windows the default transfer size is 64 KB and to increase the overall transfer size above 64 KB in Windows, a registry entry called “MaximumSGList” (associated with the HBA) must be changed. Many modern HBAs already install their drivers with this registry value set appropriately. Check the registry entries.

• Fragmentation

The more fragmented the files are on disk, the more random will be the disk access method, hence the backup will take longer. If your system does have a de-fragmentation utility, it is advisable to run it before full backups or on a regular scheduled basis to ensure files are largely contiguously arranged on disk.

• SAN Inter Switch Links

In the switched fabric, SAN Inter Switch Links (ISLs) ensure that SAN connections have sufficient bandwidth to support the backup traffic going through them. Trunk multiple ISLs together where possible.

2 Gigabit Fibre Channel

•

Expect no more than 180 MB/s maximum from a single 2 GB/s Fibre Channel connection.

• SCSI Burst Rate

Beware of disk drives quoted as Ultra320; Ultra 320 refers to the burst rate not the sustained rate. The typical sustained rate from a 15K rpm SCSI Ultra320 disk drive is around 80 MB/s for raw sequential I/Os (that is, without file system read overhead).

• SATA Disks

These types of disks are lower cost, lower performance, and lower reliability than the SCSI disk previously listed but are useful for staging backups because they offer high capacity. A typical 7200K rpm SATA disk drive has lower seek times than an equivalent SCSI disk drive, a burst rate of around 150 MB/s, and a sustained transfer rate of around 50 MB/s.

Disk to Disk to Tape (D2D2T) Data Protection Architecture

With the high capacity and lower cost offered by SATA and Fibre Channel ATA (FATA) disk technologies, many customers are now considering implementing backup to low cost disk arrays before backup to tape.

The use of secondary disk arrays for backup is best suited to environments where:

• The business dictates rapid single file restore capabilities (seconds to minutes). It is generally

quicker to restore from a secondary disk subsystem than it is from tape (minutes to hours). However, the secondary storage array is not infinite in capacity and only the most recent backups may still reside on the disk array. In addition, disk-to-disk backup is no substitute for offsite media.

• The hosts can only supply data at a relatively modest rate (10–20 MB/s). The backup image can

be gradually produced on the secondary array without having to interleave multiple streams to tape and without the need for a large powerful dedicated backup server.

• Small file backup to tape has always been a performance limiter. With backup to disk, a complete

backup image of small files can be constructed and then passed to tape at much higher transfer rates than if the small files were transferred directly to tape.

Table 2. Comparing SATA and SCSI Disks

Feature SATA Disks SCSI Disks

Mean time to failure (MTTF) 500,000 hrs @ 20% duty cycle 1,200,000 hrs @ 100% duty cycle

Burst transfer rate 150 MB/s 320 MB/s

RPM 7.2K 10K or 15K

Queuing

Capacity 250 GB 146 GB

Warranty 1 year 3 years

Non-Tagged Serial execution

Tagged Optimized seeks Better performance

While the SCSI disk MTTF of 1,200,000 looks high, this figure decreases when there are many disk drives bound together in an array. By comparison, Ultrium 460 tape drives have a MTTF of 250,000 at 100% duty cycle but are used in smaller volumes within an automated tape library. Therefore, do not assume that disk is automatically more reliable than tape.

The key point is D2D2T has its place and with proper administration can improve the data protection process but it is not a replacement for tape. Tape is still the foundation of a robust data protection strategy.

HP OpenView Storage Data Protector 5.5 provides a comprehensive implementation of D2D2T called “Advanced Backup to Disk.” The following example shows a scenario with a single central backup server and multiple network connected clients:

• The first step is to backup clients over the network to a central backup server and its staging area.

• The second step is to copy the consolidated backups from the staging area to tape. Typically the

central backup and restore environment is designed to provide high speed backup from disk to tape.

• If a restore is required, data can be accessed either from disk (if still available) or from tape over

the network.

Figure 4. DP 5.5 Advanced Backup to Disk Example

Data Protector 5.5 - Backup to Disk Disk Staging Example

Client

Use cases

1. Backup client to disk (File Library)

Backup server

2. Backup disk to tape (object copy)

Staging Area

Tape

3b. Direct restore from tape

3a. Fast restore from disk (if data still available)

• Continuous backup of transaction log files (avoids tape drives being in start/stop mode)

• Backup of slow clients without multiplexing

• Tape-less branch office backup

• Faster small file backup

• Resource sharing

On the other hand, database backups are typically not first candidates for a D2D2T backup solution. The direct backup of large database files to tape can usually be done very efficiently, utilizing the full performance of the tape drive.

Test Data

In the following proof points, two different file systems were created to cover large file servers and typical clients, so that the results shown are realistically achievable in similar situations:

1. File server data with millions of small files

2. Typical client data with fewer files and a broad range of size (KB/MB)

HPCreateData

The datasets were developed using the HPCreateData PAT utility, which is downloadable from

http://www.hp.com/support/pat. It generates different file sizes with different data contents (fixed,

random, up to 4:1 compression ratio) and different distribution (file-based, MB-based).

Typical Files

The “typical” file system is created with file sizes between 64 KB and 64 MB and the compressibility of the data 2:1. The utility creates an equal distribution of files in each directory. Finally, the file system contains 49.8 GB with 4,389 files in 20 folders.

Figure 5. HPCreateData for Typical Files

Small Files

The “small” file system is created with file sizes between 4 KB and16 KB and the compressibility of the data 2:1. The utility creates an equal distribution of files in each directory. Finally, the file system contains 49.2 GB with 5,535,750 files in 7,380 folders. The created files have a name with maximal 16 characters for avoiding corner cases.

Figure 6. HPCreateData for Small Files

HPReadData

The HPReadData PAT utility is useful in assessing the rate at which your disk subsystem can supply data, and this is ultimately what will limit the backup performance. It simulates the way DP read files. A single instance of HPReadData can read eight streams simultaneously from your array. To read more than eight streams, initiate multiple instances of HPReadData. HPReadData is available for Windows, HP-UX, Solaris, and Linux. It can be downloaded free from The performance tools are also embedded within the HP industry-leading Library and Tape Tools diagnostics which are downloadable from

Figure 7. HPReadData for Determination of Read Performance

http://www.hp.com/support/tapetools.

http://www.hp.com/support/pat.

The above screenshot shows HPReadData reading one single LUN in a manner similar to the way a backup application will read files. We can see that the maximum read rate from this configuration is 57 MB/s, so we cannot expect any higher back-up transfer performance to tape than this figure.

+ 32 hidden pages