HP IO Accelerator User Manual

HP IO Accelerator

This guide is designed to help verify and improve HP IO Accelerator performance.

Performance Tuning Guide

Abstract

Part Number: 652147-001
August 2011
Edition: 1

© Copyright 2011 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express

warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall
not be liable for technical or editorial errors or omissions contained herein.

Microsoft, Windows, and Windows Server are U.S. registered trademarks of Microsoft Corporation. Intel, Pentium, and Itanium are trademarks of
Intel Corporation in the United States and other countries.

Contents

Introduction .................................................................................................................................. 5

About the Performance and Tuning Guide ...................................................................................................... 5

System performance ...................................................................................................................... 6

Verifying Linux system performance ................................................................................................................ 6

Running fio tests ................................................................................................................................. 6

Write bandwidth test .......................................................................................................................... 7

Verifying Windows system performance with Iometer ....................................................................................... 9

Debugging performance issues ..................................................................................................... 10

Improperly configured benchmark ................................................................................................................ 10

Oversubscribed bus .................................................................................................................................... 10

Handling PCIe errors .................................................................................................................................. 11

PCIe link width improperly negotiated .......................................................................................................... 12

CPU thermal throttling or auto-idling ............................................................................................................. 13

Benchmarking through a filesystem .............................................................................................................. 14

Slow performance using RAID5 on Linux ...................................................................................................... 14

Using CP and other system utilities ............................................................................................................... 14

ext4 in Kernel.org 2.6.33 or earlier might silently corrupt data when discard (trim) is enabled ........................... 14

General tuning techniques ........................................................................................................... 16

Using direct I/O, unbuffered, or zero copy 10 .............................................................................................. 16

Multiple outstanding IOs ............................................................................................................................. 16

Pre-conditioning ......................................................................................................................................... 17

Increasing outstanding requests allowed by the kernel (Linux only) .................................................................. 18

Pre-allocating memory ................................................................................................................................ 18

preallocate_memory ......................................................................................................................... 18

preallocate_mb ................................................................................................................................ 19

expected_io_size ............................................................................................................................. 19

Tuning techniques for writes ......................................................................................................... 20

Increased steady-state write performance with fio-format ................................................................................ 20

Linux filesystem tuning ................................................................................................................. 21

ext2-3-4 tuning ........................................................................................................................................... 21

Setting stride size and stripe width for ext2/3 (extN) when using RAID .................................................. 21

Using the IO Accelerator as swap space ...................................................................................................... 22

fio benchmark ............................................................................................................................ 23

Compiling the fio benchmark ....................................................................................................................... 23

Verifying IO Accelerator performance on Windows operating systems ............................................. 24

Using Iometer to verify IO Accelerator performance on Windows operating systems ......................................... 24

Programming using direct I/O ...................................................................................................... 25

Using direct I/O on Linux ............................................................................................................................ 25

Using direct I/O on Windows ..................................................................................................................... 26

C++ code sample ...................................................................................................................................... 26

Windows driver affinity ............................................................................................................... 29

Contents 3

Setting Windows driver affinity .................................................................................................................... 29

Acronyms and abbreviations ........................................................................................................ 31

Index ......................................................................................................................................... 32

Contents 4

Introduction

About the Performance and Tuning Guide

Welcome to the Performance and Tuning Guide for the HP IO Accelerator. This guide is designed to help you
achieve the following objectives:

• Verify IO Accelerator performance on Linux, including using sample benchmarks and solving common

performance issues.

• Verify IO Accelerator performance on Windows® operating system, including using sample

benchmarks and solving common performance issues.

• Understand and use effective benchmarking techniques to study and enhance IO Accelerator

performance.

• Tune your system by describing common tuning techniques applicable to many systems and

applications.

• Improve your application I/O capabilities by describing programming techniques to maximize

performance.

Introduction 5

System performance

Verifying Linux system performance

To verify Linux system performance with an IO Accelerator, HP recommends using the fio benchmark. This
benchmark was developed by Jens Axboe, a Linux kernel developer. Fio is included in many distributions,
or can be compiled from source. The latest source distribution (http://freshmeat.net/projects/fio) requires
having the libaio development headers in place. For step-by-step instructions on compiling fio from
source, see "Compiling the fio benchmark (on page 23)."

To test the raw performance of the IO Accelerator, HP recommends using raw block access. The best way to
verify system performance is to run the fio tests in "Running fio tests (on page 6)."

CAUTION: The write tests destroy all data that currently resides on the IO Accelerator.

NOTE: For best performance on the IO Accelerator, use driver stack 2.1.0 or later.

Running fio tests

The sample jobs include the following four fio tests:

• One-card write bandwidth

• One-card read IOPS

• One-card read bandwidth

• One-card write IOPS

The following benchmarks are designed for maximally stressing the system and detecting performance
issues, not to highlight IO Accelerator performance.

Benchmark tests

# Write Bandwidth test
$ fio --filename=/dev/fioa --direct=1 --rw=randwrite --bs=1m --size=5G

--numjobs=4 --runtime=10 --group_reporting --name=file1
# Read IOPS test
$ fio --filename=/dev/fioa --direct=1 --rw=randread --bs=4k --size=5G

--numjobs=64 --runtime=10 --group_reporting --name=file1
# Read Bandwidth test
$ fio --filename=/dev/fioa --direct=1 --rw=randread --bs=1m --size=5G

--numjobs=4 --runtime=10 --group_reporting --name=file1
# Write IOPS test

System performance 6