IBM System Storage DS3500 Introduction And Implementation Manual

Front cover

Draft Document for Review March 28, 2011 12:24 pm SG24-7914-00

IBM System Storage
DS3500: Introduction and
Implementation Guide

Sample configurations with step by
step instructions

Configuration and administration
with Storage Manager

Toubleshooting and
Maintenance

Sangam Racherla

Reza Fanaei Aghdam

Hartmut Lonzer

L G (Len) O’Neill

Mario Rodriguez

Vaclav Sindelar

Alexander Watson

ibm.com/redbooks

Draft Document for Review March 28, 2011 12:24 pm 7914edno.fm

International Technical Support Organization

IBM System Storage DS3500: Introduction and
Implementation Guide

September 2010

SG24-7914-00

7914edno.fm Draft Document for Review March 28, 2011 12:24 pm

Note: Before using this information and the product it supports, read the information in “Notices” on

page xv.

First Edition (September 2010)

This edition applies to IBM System Storage DS3500 running:

򐂰 Firmware version 7.70
򐂰 IBM System Storage DS Storage Manager version 10.70

This document created or updated on March 28, 2011.

© Copyright International Business Machines Corporation 2010. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.

Draft Document for Review March 28, 2011 12:24 pm 7914edno.fm

iii

7914edno.fm Draft Document for Review March 28, 2011 12:24 pm

iv IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914TOC.fm

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

The team who wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xx

Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx

Chapter 1. Disk attachment technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Fibre Channel disk attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Serial Attached SCSI (SAS) disk attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 iSCSI disk attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.1 iSCSI initiators and targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.3.2 iSCSI discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3.3 iSCSI security considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Chapter 2. Introduction to IBM System Storage DS3500 . . . . . . . . . . . . . . . . . . . . . . . 13

2.1 IBM System Storage Portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 DS3500 product models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2.1 DS3512 and DS3524 Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3 EXP3512 and EXP3524 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4 Premium Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5 DS3500 and DS3950 Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.6 IBM System Storage DS Storage Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Chapter 3. IBM System Storage DS3500 Storage System planning tasks. . . . . . . . . . 29

3.1 Planning your SAN and storage server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.1.1 SAN zoning for the DS3500 Storage System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.1.2 Zoning considerations for Enhanced Remote Mirroring . . . . . . . . . . . . . . . . . . . . 33

3.2 Planning for physical components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.2.1 Rack considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.2.2 SAS cables and connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.2.3 Ethernet cable and connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2.4 Fiber Channel cables and connectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.2.5 Fibre Channel adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.2.6 Disk expansion enclosures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.3 Planning your storage structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.3.1 Selecting drives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.3.2 Understanding RAID types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.3.3 Array configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3.3.4 Hot Spare drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

3.3.5 Enclosure loss protection planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.3.6 Logical drives and controller ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.3.7 Storage partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.3.8 Segment size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.3.9 Media scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.3.10 Cache parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.4 Planning for premium features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

© Copyright IBM Corp. 2010. All rights reserved. v

7914TOC.fm Draft Document for Review March 28, 2011 12:24 pm

3.4.1 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.4.2 VolumeCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.4.3 Enhanced Remote Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.4.4 Drive Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.4.5 Obtaining premium features key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.5 Additional planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.5.1 Planning for systems with LVM: AIX example. . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.5.2 Planning for systems without LVM: Windows example. . . . . . . . . . . . . . . . . . . . . 77

3.5.3 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.5.4 IBM System Storage SAN Volume Controller overview . . . . . . . . . . . . . . . . . . . . 79

3.6 Host support and multipathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.6.1 Supported server platforms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.6.2 Supported operating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.6.3 Clustering support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.6.4 Multipathing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.6.5 Microsoft Windows MPIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.6.6 AIX MPIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.6.7 AIX Subsystem Device Driver Path Control Module . . . . . . . . . . . . . . . . . . . . . . . 82

3.6.8 HP-UX IBM Subsystem Device Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.6.9 Linux: RHEL/SLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.6.10 Function of Auto-Logical Drive Transfer feature . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.7 Operating system restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

3.7.1 Maximum capacity for a logical drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.7.2 Maximum number of LUNs per host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Chapter 4. IBM System Storage DS3500 and EXP3500 Cabling . . . . . . . . . . . . . . . . . . 89

4.1 DS3500 controller connectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

4.1.1 DS3500 controller with standard port configuration . . . . . . . . . . . . . . . . . . . . . . . 90

4.1.2 DS3500 controller with optional SAS host port adapter . . . . . . . . . . . . . . . . . . . . 90

4.1.3 DS3500 controller with optional Fibre Channel host port adapter. . . . . . . . . . . . . 91

4.1.4 DS3500 controller with optional iSCSI host port adapter . . . . . . . . . . . . . . . . . . . 91

4.1.5 EXP3500 ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4.2 Enclosure ID settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

4.3 SAS cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4.4 Fibre Channel cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.4.1 SFP transceiver modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.4.2 Fibre Channel cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4.4.3 Interoperability of 2 Gbps, 4 Gbps and 8 Gbps devices . . . . . . . . . . . . . . . . . . . . 99

4.5 iSCSI Ethernet cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4.6 EXP3500 attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.6.1 Redundant drive channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.6.2 Drive channel cabling rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.6.3 Single controller DS3500 with one or more EXP3500 enclosures . . . . . . . . . . . 102

4.6.4 Dual controller DS3500 with one EXP3500 enclosure . . . . . . . . . . . . . . . . . . . . 102

4.6.5 Dual Controller DS3500 with two or more EXP3500 enclosures . . . . . . . . . . . . 103

4.6.6 Adding an EXP3500 enclosure to a running dual-controller configuration . . . . . 105

4.6.7 SAS Drive Channel Miswires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.7 Management connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.7.1 Out-of-band management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4.7.2 In-band management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.8 Host attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

4.8.1 SAS attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.8.2 iSCSI attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

vi IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914TOC.fm

4.8.3 Direct attached Fibre Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

4.8.4 SAN fabric-attached DS3500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

4.9 Power Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4.9.1 The DS3500 power supply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4.9.2 Powering on and off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Chapter 5. Installing IBM System Storage DS Storage Manager . . . . . . . . . . . . . . . . 133

5.1 Installing DS Storage Manager on Microsoft Windows 2008 . . . . . . . . . . . . . . . . . . . 134

5.1.1 Installation preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

5.1.2 Installing or upgrading the Storage Manager Client on Microsoft Windows 2008 134

5.2 Installing DS Storage Manager on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.2.1 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.2.2 Installing Storage Manager using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

5.2.3 Installing DS Storage Manager using a text console . . . . . . . . . . . . . . . . . . . . . 148

5.2.4 Uninstalling DS Storage Manager on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

5.3 Installing DS Storage Manager on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

5.3.1 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

5.4 Completing the DS Storage Manager installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.4.1 Performing an automatic discovery of storage subsystems . . . . . . . . . . . . . . . . 154

5.4.2 Performing a manual discovery of storage subsystems . . . . . . . . . . . . . . . . . . . 155

5.4.3 Add Storage Subsystem verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Chapter 6. Administration - Enterprise Management . . . . . . . . . . . . . . . . . . . . . . . . . 161

6.1 Enterprise Management window overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

6.1.1 Initial Setup Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

6.1.2 Enterprise Management window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6.2 Functions in the Enterprise Management window . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.2.1 Subsystem context menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.2.2 The Enterprise Management window menu bar . . . . . . . . . . . . . . . . . . . . . . . . . 175

6.2.3 The Quick Access buttons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Chapter 7. Administration - Summary Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

7.1 Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

7.1.1 Storage Subsystem Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

7.1.2 Storage subsystem status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

7.1.3 Operations in Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

7.1.4 Connection lost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

7.2 Hardware Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

7.3 Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

7.4 Hosts & Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

7.4.1 Configured Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

7.4.2 Host-to-Logical Drive Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

7.4.3 Storage partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

7.5 Arrays & Logical Drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

7.6 Information Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Chapter 8. Administration - Subsystem Management. . . . . . . . . . . . . . . . . . . . . . . . . 193

8.1 DS Storage Manager - Subsystem Manger window . . . . . . . . . . . . . . . . . . . . . . . . . . 194

8.2 Pull-Down Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

8.2.1 Storage Subsystem Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

8.2.2 View Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

8.2.3 Mappings Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

8.2.4 Array Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

8.2.5 Logical Drive Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Contents vii

7914TOC.fm Draft Document for Review March 28, 2011 12:24 pm

8.2.6 Controller Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

8.2.7 Drive Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

8.2.8 Advanced Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

8.2.9 Help Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

8.3 Toolbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

8.3.1 Create new logical drives and arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

8.3.2 View diagnostic event log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

8.3.3 Monitor Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

8.3.4 Recover from failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

8.3.5 Manage enclosure alarm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

8.3.6 Find in tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

8.3.7 Launch copy manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

8.4 Status bar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

8.5 Tabs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Chapter 9. Administration - Logical Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

9.1 Logical Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

9.2 Working with Unconfigured Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

9.2.1 View Associated Physical Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

9.2.2 Create Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

9.3 Working with Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

9.3.1 Locate and View Associated Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

9.3.2 Change Ownership and RAID level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

9.3.3 Add Free Capacity (Drive). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

9.3.4 Secure Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

9.3.5 Delete and Rename . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

9.3.6 Replace Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

9.4 Working with Free Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

9.4.1 Create Logical Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

9.5 Working with Logical Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

9.5.1 Change Modification Priority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

9.5.2 Change Cache Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

9.5.3 Change Media Scan Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

9.5.4 Change Pre-Read Redundancy Check. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

9.5.5 Change Ownership/Preferred Path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

9.5.6 Change Segment Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

9.5.7 Increase Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

9.5.8 Copy Services operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

9.5.9 Delete and Rename . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

Chapter 10. Administration - Physical Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

10.1 Physical tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

10.2 Discover component properties and location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

10.2.1 Show disks type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

10.2.2 View Enclosure Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

10.2.3 Disk Drive menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

10.2.4 Controller menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

10.3 Set hot spare drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

10.4 Failed disk drive replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

10.5 Set preferred loop ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

10.6 Set remote access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

10.7 Set Ethernet management ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

10.7.1 Initial settings for default IP addresses in DS Storage Manager. . . . . . . . . . . . 277

viii IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914TOC.fm

10.7.2 Configure Ethernet Management Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

10.8 Configure iSCSI Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Chapter 11. Administration - Mappings Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

11.1 Mappings tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

11.2 Defining Host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

11.2.1 Adding a new Host to existing Host Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

11.3 Defining Storage Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

11.4 Defining Host Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

11.5 Manage Host Port Identifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

11.6 Define Additional Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

11.7 View Unassociated Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

11.8 Move, Remove and Rename Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

11.9 Change Host Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

11.10 Change and Remove Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Chapter 12. Administration - Setup tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

12.1 Setup tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

12.2 Locate Storage Subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

12.3 Rename Storage Subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

12.4 Set a Storage Subsystem Password. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

12.5 Configure iSCSI Host Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

12.6 Configure Storage Subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

12.6.1 Automatic configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

12.6.2 Configure hot spare drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

12.6.3 Create arrays and logical drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

12.7 Map Logical Drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

12.8 Save Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

12.9 Manually Define Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

12.10 Configure Ethernet Management Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

12.11 View/Enable Premium Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

12.12 Manage iSCSI Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

Chapter 13. Administration - iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

13.1 Planning for iSCSI attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

13.2 iSCSI Configuration summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

13.3 Manage iSCSI protocol settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

13.3.1 Target Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

13.3.2 Mutual Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322

13.3.3 Target Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

13.3.4 Target Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

13.4 Configure iSCSI Host Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

13.5 View/End iSCSI Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

13.6 View iSCSI Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331

13.7 Defining iSCSI hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

13.7.1 View Unassociated iSCSI initiators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

13.7.2 Defining new iSCSI host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

13.7.3 Manage iSCSI host ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

Chapter 14. Administration - Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

14.1 The Subsystem Management support tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

14.2 Gather Support Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

14.2.1 Saving the Support Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

14.2.2 Support Data Automatic Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

Contents ix

7914TOC.fm Draft Document for Review March 28, 2011 12:24 pm

14.2.3 Collect drive data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

14.3 View Storage Subsystem Profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346

14.4 Storage Manager Support Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

14.4.1 Support Monitor installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

14.4.2 Support Monitor overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

14.4.3 The Support Monitor Profiler console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

14.4.4 Support Monitor functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

14.4.5 Support Monitor - View Module Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

14.5 Download firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

14.5.1 Before you upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

14.5.2 Updating the host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

14.5.3 Upgrading the DS3500 controller firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

14.5.4 Using the Enterprise Management upgrade tool. . . . . . . . . . . . . . . . . . . . . . . . 363

14.5.5 Using the DS3500 Storage Manager (Subsystem Management). . . . . . . . . . . 376

14.6 View Event Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

14.7 Performance Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

14.8 Import/Export Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

14.8.1 Export Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

14.8.2 Import Array procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

14.9 Maintenance - Persistent reservations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

14.10 Troubleshooting - Drive channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

14.11 Troubleshooting - Run Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

14.12 Troubleshooting - Prepare for removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418

14.13 Recovery Guru - Recover from Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

14.14 Common Recovery Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421

14.14.1 Initialize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422

14.14.2 Revive drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425

14.14.3 Recovery - Clear Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

14.14.4 Recovery - Place controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

14.14.5 Recovery - Reset controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432

14.14.6 Recovery - Enable controller data transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . 433

14.14.7 Recovery - Place Logical drives online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

14.14.8 Recovery - Redistribute Logical Drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

14.14.9 Recovery - Fail drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

14.14.10 Recovery - reconstruct drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438

14.14.11 Recovery - Defragment Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

14.14.12 Recovery - check array redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442

14.14.13 Recovery - Unreadable sectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

14.15 View Online Help. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446

14.16 About IBM System Storage DS Storage Manager . . . . . . . . . . . . . . . . . . . . . . . . . 446

Chapter 15. Disk Security with Full Disk Encryption drives . . . . . . . . . . . . . . . . . . . . 449

15.1 The need for encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

15.1.1 Encryption method used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

15.2 Disk Security components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

15.2.1 DS3500 Disk Encryption Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

15.2.2 Full Data Encryption (FDE) disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

15.2.3 Premium feature license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

15.2.4 Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

15.2.5 Security key identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

15.2.6 Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

15.3 Setting up and enabling a secure disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

15.3.1 FDE and premium feature check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

x IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914TOC.fm

15.3.2 Secure key creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458

15.3.3 Enable Disk Security on array. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

15.4 Additional secure disk functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

15.4.1 Changing the security key. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

15.4.2 Save security key file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469

15.4.3 Secure erase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

15.4.4 FDE drive status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471

15.4.5 Hot spare drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

15.5 Migrating secure disk arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

15.5.1 Planning checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

15.5.2 Export the array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473

15.6 Import secure drive array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

15.6.1 Unlock drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

15.6.2 Import array. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

Chapter 16. IBM Remote Support Manager for Storage . . . . . . . . . . . . . . . . . . . . . . . 485

16.1 IBM Remote Support Manager for Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

16.1.1 Hardware and software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

16.1.2 DS-RSM Model RS3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489

16.1.3 Installation choices for RSM for Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490

16.1.4 How RSM for Storage works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491

16.1.5 Notification e-mail and events filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492

16.1.6 Remote access methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498

16.1.7 RSM management interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499

16.1.8 RSM security considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500

16.2 Installing and setting up RSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502

16.2.1 Installing the host OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

16.2.2 Installing RSM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

16.2.3 Setting up RSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

16.2.4 Configuring SNMP traps in Storage Manager. . . . . . . . . . . . . . . . . . . . . . . . . . 520

16.2.5 Activating RSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

16.2.6 Remote access security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524

16.2.7 Managing alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529

Chapter 17. Command-Line Interface(CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533

17.1 How to Use the Command Line Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535

17.1.1 Usage Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535

17.2 Running the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536

17.2.1 Script Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536

17.3 General SMcli syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538

17.4 Adding a storage subsystem to the Storage Manager configuration . . . . . . . . . . . . 542

17.5 Showing defined subsystems in the Storage Manager configuration . . . . . . . . . . . . 543

17.6 Configuring alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544

17.6.1 Defining the mail server and e-mail address to send out the e-mail alerts . . . . 544

17.6.2 Defining email alert recipients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544

17.6.3 Deleting e-mail alert recipients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545

17.6.4 SNMP alert recipients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546

17.7 Issuing commands to the storage subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547

17.7.1 Sample command: Save configuration script file . . . . . . . . . . . . . . . . . . . . . . . 549

17.8 More Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551

Chapter 18. Windows SAS configuration guide for IBM BladeCenter . . . . . . . . . . . . 553

18.1 Equipment required . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554

18.2 IBM BladeCenter setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554

Contents xi

7914TOC.fm Draft Document for Review March 28, 2011 12:24 pm

18.2.1 Installing Windows Server 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

18.2.2 HS21 SAS Expansion Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555

18.2.3 Recording the SAS Expansion Card WWPN . . . . . . . . . . . . . . . . . . . . . . . . . . 555

18.2.4 HS21 SAS Expansion Card device driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558

18.2.5 SAS Connectivity Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558

18.2.6 SAS Connectivity Module firmware update. . . . . . . . . . . . . . . . . . . . . . . . . . . . 559

18.2.7 Configuring the SAS connectivity module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562

18.2.8 SAS Connectivity Module zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563

18.3 Installing DS Storage Manager host software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565

18.4 Configure the disk space in Windows Server 2008. . . . . . . . . . . . . . . . . . . . . . . . . . 566

Chapter 19. Microsoft Cluster configuration with DS3500 . . . . . . . . . . . . . . . . . . . . . 571

19.1 Overview of a failover cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572

19.1.1 Hardware requirements for a two-node failover cluster . . . . . . . . . . . . . . . . . . 572

19.2 Preparing the environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573

19.2.1 SAN Zoning configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573

19.2.2 DS3500 Storage configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573

19.3 Installing DS Storage Manager host software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578

19.3.1 Installing the multipath driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578

19.4 Windows Server 2008 Failover Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580

19.4.1 Installing the Failover Clustering Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580

19.4.2 Validate a Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583

19.4.3 Create a Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587

19.4.4 Quorum configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589

19.4.5 Steps for configuring a two-node file server cluster . . . . . . . . . . . . . . . . . . . . . 595

Chapter 20. SuSE Linux configuration guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603

20.1 DS3500 SAS storage configuration on SLES 11 using RDAC . . . . . . . . . . . . . . . . . 604

20.1.1 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604

20.1.2 Installing the RDAC Multipath Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610

20.1.3 Setting up the DS3500 logical drives and host mapping. . . . . . . . . . . . . . . . . . 611

20.1.4 Scan and verify the storage logical drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612

20.1.5 Configuring RDAC (MPP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615

20.2 DS3500 iSCSI storage configuration on SLES 11 using RDAC . . . . . . . . . . . . . . . . 616

20.2.1 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616

20.2.2 Configuring iSCSI software initiator with YaST . . . . . . . . . . . . . . . . . . . . . . . . . 618

20.2.3 Configuring iSCSI software initiator manually . . . . . . . . . . . . . . . . . . . . . . . . . . 624

20.3 DS3500 FC SAN boot configuration for SLES 11 server using RDAC . . . . . . . . . . . 627

20.3.1 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627

20.3.2 SuSE Linux Enterprise 11 installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633

20.3.3 SuSE Linux final zoning topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634

20.4 DS3500 FC storage configuration on SLES 11 using DMM . . . . . . . . . . . . . . . . . . . 635

20.4.1 DMM Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635

20.4.2 Comparing RDAC(MPP) to DMM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636

20.4.3 Planning for the installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636

20.4.4 Installing the DMM multipath driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637

20.5 Scan and manage the storage logical drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639

Chapter 21. AIX 6.1 configuration guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643

21.1 Planning for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644

21.1.1 Zoning considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645

21.1.2 SAN Boot implementation possibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645

21.1.3 Planning MPIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646

21.2 Configuring MPIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650

xii IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914TOC.fm

21.3 Setting up the DS3500 logical drives and host mapping. . . . . . . . . . . . . . . . . . . . . . 652

21.4 Scan and manage the storage logical drive from AIX . . . . . . . . . . . . . . . . . . . . . . . . 652

21.4.1 Ways to manage the paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654

21.5 AIX SAN Boot with the IBM System Storage DS3500 . . . . . . . . . . . . . . . . . . . . . . . 655

21.5.1 Creating a boot disk with alt_disk_install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655

21.5.2 AIX SAN installation with NIM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656

21.5.3 AIX SAN installation with CD-ROM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660

21.5.4 AIX Operating System Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660

Chapter 22. VMware ESX Server and DS3500 Storage Configuration . . . . . . . . . . . . 665

22.1 Introduction to IBM VMware Storage Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666

22.1.1 VMware installation prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666

22.2 SAN Zoning configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667

22.3 DS3500 Storage configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668

22.3.1 Mapping LUNs to a storage partition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669

22.3.2 Steps for verifying the storage configuration for VMware . . . . . . . . . . . . . . . . . 669

22.4 Installing the VMware ESX Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671

22.4.1 Configuring the hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671

22.4.2 Configuring the software on the VMware ESX Server host . . . . . . . . . . . . . . . 675

22.4.3 Connecting to the VMware vSphere Server . . . . . . . . . . . . . . . . . . . . . . . . . . . 698

22.4.4 Post-Install Server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706

22.4.5 Configuring VMware ESX Server Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708

22.4.6 Creating additional virtual switches for guests’ connectivity . . . . . . . . . . . . . . . 718

22.4.7 Creating virtual machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722

22.4.8 Additional VMware ESX Server Storage configuration . . . . . . . . . . . . . . . . . . . 736

Appendix A. IBM Support Portal web site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737

Sample navigation procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

Download code updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741

My notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751

Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751

Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751

How to get Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752

Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755

Contents xiii

7914TOC.fm Draft Document for Review March 28, 2011 12:24 pm

xiv IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914spec.fm

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION

PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs.

© Copyright IBM Corp. 2010. All rights reserved. xv

7914spec.fm Draft Document for Review March 28, 2011 12:24 pm

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. These and other IBM trademarked terms are
marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US
registered or common law trademarks owned by IBM at the time this information was published. Such
trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:

AIX 5L™
AIX®
Application Advantage™
BladeCenter®
DB2®
DS4000®
DS6000™
DS8000®
Express Storage™

FlashCopy®
GPFS™
IBM®
Netfinity®
Power Systems™
POWER6®
POWER®
Redbooks®
Redbooks (logo) ®

System i®
System p®
System Storage DS®
System Storage®
System x®
Tivoli®
TotalStorage®

The following terms are trademarks of other companies:

Java, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other
countries, or both.

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.

Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

xvi IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914pref.fm

Preface

This IBM® Redbooks® publication introduces the IBM System Storage® DS3500, providing
an overview of its design and specifications, and describing in detail how to set up, configure,
and administer it. This edition covers updates and functions available with the DS3500
Storage Manager Version 10.70 (firmware level 7.70).

IBM has combined best-of-breed development with leading 6 Gbps host interface and drive
technology in the IBM System Storage DS3500 Express. With its simple, efficient and flexible
approach to storage, the DS3500 is a cost-effective, fully integrated complement to System
x® servers, BladeCenter® and Power Systems™. Offering substantial improvements at a
price that will fit most budgets, the DS3500 delivers superior price to performance ratios,
functionality, scalability and ease of use for the entry-level storage user.

The DS3500 supports intermixing four 1Gbps iSCSI or four 8Gbps FC host ports with its
native 6Gbps SAS interfaces. This flexible and multi-purpose dual protocol approach allows
organizations to implement a single storage system to support all of their shared storage
requirements, there by maximizing productivity, reliability, and cost.

Delivering solid input/output per second (IOPS) and throughput, the DS3500 controllers offer
balanced and sustainable performance. The DS3500 can effectively double the performance
of the previous DS3000 series of storage systems in both throughput and IOPS.

The DS3500 DS Storage Manager is the same management software offered with the
DS5000 and DS4000® series. Now, any of these storage systems can be viewed and
managed from a single interface. This allows for consolidated management of these various
storage systems as well as a reduced learning curve. The DS3500 also supports enhanced
remote mirroring over FC host ports, which is also compatible with the DS5000 and DS4000
series. This allows for low-cost backup and recovery with a DS5000 and DS4000 at a
production site and a DS3500 at the secondary site.

This book is intended for customers, IBM Business Partners, and IBM technical professionals
who want to learn more about the capabilities and advanced functions of the IBM System
Storage DS3500 with Storage Manager Software. It also targets those who have a DS3500
storage system and need detailed advice on how to configure and manage it.

The team who wrote this book

This book was produced by a team of specialists from around the world working at the
International Technical Support Organization, Raleigh Center.

was produced by a team of specialists from around the world working at the International
Technical Support Organization, Raleigh Center.

© Copyright IBM Corp. 2010. All rights reserved. xvii

7914pref.fm Draft Document for Review March 28, 2011 12:24 pm

Sangam Racherla is an IT Specialist and Project Leader

working at the International Technical Support
Organization (ITSO), San Jose Center. He holds a degree
in electronics and communication engineering and has ten
years of experience in the IT field. He has been with the
ITSO for the past seven years and has extensive
experience installing and supporting the ITSO lab
equipment for various Redbooks publication projects. His
areas of expertise include Microsoft® Windows®, Linux®,
AIX®, System x, and System p® servers, and various SAN
and storage products.

Reza Fanaei Aghdam is a Senior IT Specialist working in
Zurich, Switzerland. He has 17 years of professional
experience with x86-based hardware, storage technologies
and systems management more than 12 of them at IBM.
He instructs Business Partners and customers on how to
configure and install the System x, BladeCenter, Systems
Director, Storage, VMware® and Hyper-V. He is an IBM
Certified Systems Expert - System x BladeCenter, IBM
Certified Specialist - Midrange Storage Technical Support
and VMware Certified Professional.

Hartmut Lonzer is a Technical Consultant in the
Partnership Solution Center Southwest / Germany. As
former Storage FTSS member his main Focus is on
Storage and System x. Today, he is responsible to educate
and support the Business Partner and Customers in his
Area in technical matters. His experience regarding the DS
Storage goes back to the beginning of this Product. He has
been with IBM 33 years and all the time in various technical
roles.

L G (Len) O’Neill is a Product Field Engineer (PFE) for
IBM System x hardware support based at IBM Greenock in
the UK. The PFE team in IBM Greenock provides
post-sales technical support for all IBM System x and IBM
BladeCenter products for the EMEA (Europe, Middle-East
and Africa) region. He has been with IBM for 12 years and
in his current role for 11 years. He specializes in providing
post-sales technical support for the IBM DS3000 storage
products and has previously specialised in supporting IBM
SCSI, ServeRAID and Microsoft Windows clustering
products within the System x product range. He holds a
degree in Physics from Trinity College Dublin.

xviii IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914pref.fm

Mario Rodriguez is an IT Specialist in IBM Uruguay since

2001. He holds MCSE, AIX, LPI and other Comptia
certifications. His areas of expertise include SAN switches
(Brocade, Cisco MDS), SAN Storage (DS3000, DS4000,
DS6000™, and DS8000®), Linux, AIX, TSM and VMware.
His role in IBM Uruguay is to provide technical support
services for virtualization and storage products.

Vaclav Sindelar is a Field Technical Support Specialist
(FTSS) for IBM System Storage at the IBM Czech Republic
headquarters in Prague. His daily support activities include
pre-sales support for IBM Storage products. He has 7
years of FTSS Storage experience with a focus on IBM disk
arrays and SAN. He has been with IBM since 2001 and
worked as storage specialist before he came to IBM. He
holds a Master’s degree in computer science from the
Technical University of Brno in the Czech Republic.

Alexander (Al) Watson is an ATS Specialist for Storage
Advanced Technical Skills (ATS) Americas in the United
States. He is a Subject Matter Expert on SAN switches and
the IBM Midrange system storage products. He has over
fifteen years of experience in planning, managing,
designing, implementing, problem analysis, and tuning of
SAN environments and storage systems. He has worked at
IBM for eleven years. His areas of expertise include SAN
fabric networking, Open System Storage IO and the IBM
Midrange Storage solutions.

Thanks to the following people for their contributions to this project:

Tam ik ia B ar row
Margaret Ticknor
David Watts
International Technical Support Organization, Raleigh Center

Doris Konieczny
Harold Pike
Tony Iles
Pete Urbisci
John Fasano
Roger Bullard
Danh T Le
Raul A Gallardo
Paul Goetz
Gene Cullum
James l (Jim) Kennish
David Bennin
Richard Conway

Preface xix

7914pref.fm Draft Document for Review March 28, 2011 12:24 pm

Donald Brennan
IBM

David Worley
Stacey Dershem
Jamal Boudi
LSI Corporation

Brian Steffler
Yon g Ch oi
Alan Hicks
Brocade Communications Systems, Inc.

Now you can become a published author, too!

Here's an opportunity to spotlight your skills, grow your career, and become a published
author—all at the same time! Join an ITSO residency project and help write a book in your
area of expertise, while honing your experience using leading-edge technologies. Your efforts
will help to increase product acceptance and customer satisfaction, as you expand your
network of technical contacts and relationships. Residencies run from two to six weeks in
length, and you can participate either in person or as a remote resident working from your
home base.

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Comments welcome

Your comments are important to us!

We want our books to be as helpful as possible. Send us your comments about this book or
other IBM Redbooks publications in one of the following ways:

򐂰 Use the online Contact us review Redbooks form found at:

ibm.com/redbooks

򐂰 Send your comments in an email to:

[e-mail]

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400

Stay connected to IBM Redbooks

򐂰 Find us on Facebook:

http://www.facebook.com/IBMRedbooks

򐂰 Follow us on Twitter:

xx IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914pref.fm

http://twitter.com/ibmredbooks

򐂰 Look for us on LinkedIn:

http://www.linkedin.com/groups?home=&gid=2130806

򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks

weekly newsletter:

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

򐂰 Stay current on recent Redbooks publications with RSS Feeds:

http://www.redbooks.ibm.com/rss.html

Preface xxi

7914pref.fm Draft Document for Review March 28, 2011 12:24 pm

xxii IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DiskAttach0908.fm

1

Chapter 1. Disk attachment technology

In this chapter, we describe basic disk attachment methods in the context of the IBM System
Storage DS3500. We discuss the following technologies:

򐂰 Fibre Channel (FC)
򐂰 Serial-Attached SCSI (SAS)
򐂰 Internet SCSI (iSCSI)

Fibre Channel has traditionally been used to attach storage subsystems in midrange and
large scale environments. However, as the DS3500 products are geared towards Small and
Medium Business (SMB) and departmental environments, SAS and iSCSI attachment
technologies are supported as well.

© Copyright IBM Corp. 2010. All rights reserved. 1

7914DiskAttach0908.fm Draft Document for Review March 28, 2011 12:24 pm

1.1 Fibre Channel disk attachment

Fibre Channel (FC) is a high-speed disk attachment technology primarily used for storage
networking. It is designed to connect a large number of storage devices to a number of host
servers across a Storage Area Network (SAN). Fibre Channel is a transport Protocol (FCP)
which transfers SCSI commands and data over Fibre Channel networks.

FC supports a much higher number of devices and much longer cable lengths than SCSI. It
has become the preferred disk attachment technology in midrange and large scale datacenter
solutions.

At the time of writing, DS3500 Storage maximum FC throughput is 8 Gbps. In fact, 10 Gbps
links can be used today, but only for SAN switch interconnection.

Host servers contain one or more FC Host Bus Adapters (HBA). The HBAs provide
connectivity to the storage devices using FC cabling and SAN Switch.

For more information about Fibre Channel and SANs, see Introduction to Storage Area
Networks, SG24-5470.

FC topologies

There are three major Fibre Channel topologies, describing how a number of ports are
connected together. A port in Fibre Channel terminology is any entity that actively
communicates over the network, not necessarily a hardware port. This port is usually
implemented in a device such as disk storage, an HBA on a server or a Fibre Channel switch.

򐂰 Point-to-point

Two devices are connected directly to each other. This is the simplest topology and
provides a direct link between an FC HBA inside a host server and a storage device, with
limited connectivity.

򐂰 Arbitrated loop

This topology can be used to interconnect several FC devices. A typical example would be
to attach a certain number of host servers to an FC storage subsystem. A loop can consist
of up to 127 devices.

A minimal loop containing only two ports, while appearing to be similar to FC-P2P, differs
considerably in terms of the protocol. Only one pair of ports can communicate
concurrently on a loop. This means the devices share bandwidth, so the arbitrated loop
topology is not suitable for high performance requirements.

Arbitrated loops were commonly implemented with the use of an FC hub. Even though this
is physically a star topology, logically it will be a loop. Alternatively, devices can be
connected in a daisy chain manner.

Arbitrated loops are rarely seen these days, as switched fabrics have become the norm.

򐂰 Switched fabric

The most commonly used topology in a typical SAN today is switched fabric. SAN
switches are used to provide FC connectivity between the host servers and storage
devices. Switched fabrics can become very complex in large scenarios, connecting
hundreds of host servers to a very large number of storage subsystems.

SAN switches provide optimized traffic flow and increased performance by allowing
concurrent data transfers between many connected hosts and storage devices. Switched
fabrics can provide dedicated bandwidth, as opposed to arbitrated loop technology, where
the bandwidth is shared among all the devices in the loop.

2 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DiskAttach0908.fm

All devices or loops of devices are connected to Fibre Channel switches, similar
conceptually to modern Ethernet implementations. Advantages of this topology over
FC-P2P or FC-AL include:

– The switches manage the state of the fabric, providing optimized interconnections.
– The traffic between two ports flows through the switches only, it is not transmitted to

any other port.
– Failure of a port is isolated and should not affect operation of other ports.
– Multiple pairs of ports may communicate simultaneously in a fabric.

FC protocol layers

Fibre Channel does not follow the OSI model layering, but is split similarly into 5 layers as
shown in Figure 1-1. These layers include:

򐂰 FC0 is the physical layer, which describes cabling, connectors, signalling, and so on. This

layer defines the physical media implementation.

򐂰 Data link layer, which implements line coding of signals. This layer contains the 8b/10b

encoding and decoding of signals for transmission across the physical media.

򐂰 FC2 is the network layer and defines the main FC protocols. This layer defines how the

frames are transferred.

򐂰 FC3 is the common services layer. This layer provides services such as multi-casting and

striping.

򐂰 FC4 is the application protocol mapping layer. In storage connectivity applications, FCP

protocol is used to encapsulate SCSI data into FC frames.

Layers FC0 through FC2 are also known as FC-PH, the physical layers of Fibre Channel.

Fibre Channel routers operate up to FC4 level (i.e. they may operate as SCSI routers),
switches up to FC2, and hubs on FC0 only.

Fibre Channel products are available at 1, 2, 4, 8, 10 and 20 Gbit/s. Products based on the 2,
4 and 8 Gbit/s standards should be interoperable and backward compatible. The 10 Gbit/s
standard and its 20 Gbit/s derivative, however, are not backward compatible with any of the
slower speed devices, as they differ considerably on FC1 level in using 64b/66b encoding
instead of 8b/10b encoding, and are primarily used as inter-switch links.

Figure 1-1 Fibre Channel Layers

FC cable types

FC implementations can utilize either single-mode or multi-mode FC cables.

Chapter 1. Disk attachment technology 3

7914DiskAttach0908.fm Draft Document for Review March 28, 2011 12:24 pm

The name multi-mode fiber indicates that multiple modes, or rays of light, can travel through
the cable core simultaneously. The multi-mode fiber cable uses a larger diameter core, which
makes it easier to couple than the single-mode fibre cable. With a throughput of 8 Gbps, the
length of the cable can be up to 300 m.

Single-mode fibre transfers a single ray of light. The core diameter is much smaller than the

core of multi-mode cable. Therefore, coupling is much more demanding and tolerances for
single-mode connectors and splices are very low. However, single-mode fiber cables can be
much longer. Cable length can exceed 50 km.

Multi-mode cabling is much more common, as it is easier to work with and meets the
requirements of most customer scenarios. However, in situations where very long cable
lengths are needed, single-mode cabling will be required.

Despite its name, Fibre Channel signaling can run on both copper wire and fiber-optic cables
as shown in Figure 1-2.

Figure 1-2 FC Cable Types

Small form-factor pluggable (SFP) transceiver

The small form-factor pluggable (SFP) or Mini-GBIC is a compact, hot-pluggable transceiver
used for both telecommunication and data communications applications. It interfaces a
network device mother board (for a switch, router, media converter or similar device) to a fiber
optic or copper networking cable. SFP transceivers are designed to support SONET, Gigabit
Ethernet, Fibre Channel, and other communications standards.

SFP transceivers are available with a variety of different transmitter and receiver types,
allowing users to select the appropriate transceiver for each link to provide the required
optical reach over the available optical fiber type (e.g. multi-mode fiber or single-mode fiber).

Optical SFP modules are commonly available in several different categories:

򐂰 850 nm 550m MMF (SX)
򐂰 1310 nm 10 km SMF (LX)

4 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DiskAttach0908.fm

򐂰 1550 nm [40 km (XD)
򐂰 80 km (ZX)
򐂰 120 km (EX or EZX)], and DWDM.

SFP transceivers are also available with a copper cable interface, allowing a host device
designed primarily for optical fiber communications to also communicate over unshielded
twisted pair networking cable.

SFP transceivers are commercially available with capability for data rates up to 4.25 Gbit/s.
The standard is expanding to SFP+ which supports data rates up to 10.0 Gbit/s (that will
include the data rates for 8 gigabit Fibre Channel, 10GbE, and OTU2).

FC World Wide Names (WWN)

A World Wide Name (WWN) or World Wide Identifier (WWID) is a unique identifier which
identifies a particular Fibre Channel, Advanced Technology Attachment (ATA) or Serial
Attached SCSI (SAS) target. Each WWN is an 8 byte number derived from an IEEE OUI and
vendor-supplied information.

There are two formats of WWN defined by the IEEE:
򐂰 Original format: addresses are assigned to manufacturers by the IEEE standards

committee, and are built into the device at build time, similar to an Ethernet MAC address.
First 2 bytes are either hex 10:00 or 2x:xx (where the x's are vendor-specified) followed by
the 3-byte vendor identifier and 3 bytes for a vendor-specified serial number

򐂰 New addressing schema: first nibble is either hex 5 or 6 followed by a 3-byte vendor

identifier and 36 bits for a vendor-specified serial number

1.2 Serial Attached SCSI (SAS) disk attachment

SAS is a computer bus used to move data to and from computer storage devices such as
hard drives and tape drives. SAS depends on a point-to-point serial protocol that replaces the
parallel SCSI bus technology and it uses the standard SCSI command set.

At the time of writing, typical SAS throughput is 6 Gbps full duplex. SAS has the capability to
reach 24 Gbps if the host can drive it at that speed. When the first 6 Gbps connection is full,
the next 6 Gbps connection is used, and so on, up to four connections.

Figure 1-3 shows the SAS technical specifications.

Figure 1-3 SAS Technical Specifications

Chapter 1. Disk attachment technology 5

7914DiskAttach0908.fm Draft Document for Review March 28, 2011 12:24 pm

A SAS Domain, an I/O system, consists of a set of SAS devices that communicate with one
another by means of a service delivery subsystem. Each SAS device in a SAS domain has a
globally unique identifier called a World Wide Name (WWN or SAS address). The WWN
uniquely identifies the device in the SAS domain just as a SCSI ID identifies a device in a
parallel SCSI bus. A SAS domain may contain up to a total of 65,535 devices.

Basically, SAS uses point-to-point serial links. Point-to-point topology essentially dictates that
only two devices can be connected; however, with the use of SAS expanders, the number of
devices in a SAS domain can greatly increase. There are two types of expanders:

򐂰 Fan-out expanders

A fanout expander can connect up to 255 sets of edge expanders, known as an edge
expander device set, allowing for even more SAS devices to be addressed. A fanout
expander cannot do subtractive routing, it can only forward subtractive routing requests to
the connected edge expanders.

򐂰 Edge expanders

An edge expander allows for communication with up to 255 SAS addresses, allowing the
SAS initiator to communicate with these additional devices. Edge expanders can do direct
table routing and subtractive routing.

In the current DS3500 implementation, up to 96 drives can be configured in a single DS3500
using three EXP3500 expansion units.

SAS protocol layers

The SAS protocol consists of four layers:
򐂰 The physical (or

This layer represents the hardware components, such as transceivers, which send and
receive electrical signals on the wire.

򐂰 The link layer

The link layer manages connections across phy interfaces.

򐂰 The port layer

The port layer passes the SAS frames to the link layer. It also selects the most appropriate
physical layer for data transmission (when multiple layers are available).

򐂰 The transport layer

Serial Attached SCSI comprises three transport protocols:

򐂰 Serial SCSI Protocol (SSP) — supporting SAS disk drives.
򐂰 Serial ATA Tunneling Protocol (STP) — supporting SATA disks.
򐂰 Serial Management Protocol (SMP) — for managing SAS Expanders.

At the physical layer, the SAS standard defines connectors and voltage levels. The physical
characteristics of the SAS wiring and signaling are compatible with and have loosely tracked
that of SATA up to the present 6 Gbit/s rate, although SAS defines more rigorous physical
signaling specifications as well as a wider allowable differential voltage swing intended to
support longer cabling. While SAS-1.0/SAS-1.1 adopted the physical signaling characteristics
of SATA at the 1.5 Gbit/s and 3 Gbit/s rates, SAS-2.0 development of a 6 Gbit/s physical rate
led the development of an equivalent SATA speed. According to the SCSI Trade Association,
12 Gbit/s is slated to follow 6 Gbit/s in a future SAS-3.0 specification.

phy) layer

6 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DiskAttach0908.fm

SAS wide ports

Each SAS port includes four full duplex links or lanes within a single connector, as shown in
Figure 1-4, with each lane running a speed of 6 Gbps. A single lane is used as the path to the
drives; the second, third, and fourth lanes are used as overflow when concurrent I/Os
overload the channel. For example, suppose the first link is transmitting data at 6 gigabits per
second. If another block of data then needs to be written to disk, while the link one is still busy,
then link two will manage the overflow of data that cannot be transmitted by link one. If link
one finishes its transmission of data, then the next block of data will be transmitted on link one
again; otherwise, another link will be used. In this way, for heavy I/O workloads, it is possible
that all links are being used at certain times, providing a simultaneous data speed of 24 Gbps.

Figure 1-4 SAS wide ports

SAS drive technology

Figure 1-5 on page 8 shows how SAS drives are attached to the controllers. The
point-to-point topology used in SAS configurations, as shown below, means that there is a
direct path to each drive from each controller, so communication can take place directly, with
no effects caused by an individual drive failure.

Chapter 1. Disk attachment technology 7

7914DiskAttach0908.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 1-5 Point to Point SAS Topology

1.3 iSCSI disk attachment

iSCSI stands for Internet Small Computer System Interface, an Internet Protocol (IP)-based
storage networking standard for linking data storage facilities. By carrying SCSI commands
over IP networks, iSCSI is used to facilitate data transfers over intranets and to manage
storage over long distances. iSCSI can be used to transmit data over local area networks
(LANs), wide area networks (WANs), or the Internet and can enable location-independent
data storage and retrieval.

iSCSI uses TCP/IP (typically TCP ports 860 and 3260). In essence, iSCSI simply allows two
hosts to negotiate and then exchange SCSI commands using IP networks. By doing this
iSCSI takes a popular high-performance local storage bus and emulates it over wide-area
networks, creating a storage area network (SAN).

Unlike some SAN protocols, iSCSI requires no dedicated cabling; it can be run over existing
switching and IP infrastructure. However, the performance of an iSCSI SAN deployment can
be severely degraded if not operated on a dedicated network or subnet (LAN or VLAN). As a
result, iSCSI is often seen as a low-cost alternative to Fibre Channel, which requires
dedicated infrastructure except in its Fibre Channel over Ethernet (FCoE) form.

IP SANs are a cheaper alternative to FC SANs; however the lower cost of iSCSI also implies
lower performance and scalability. Encapsulation has an impact and the transfer rate is lower.
A typical Ethernet network operates at 1 Gbps, while an FC SAN can run up to 8 Gbps.
However, there are ways to address iSCSI performance:

򐂰 While the host servers can use almost any Ethernet network interface card for iSCSI

traffic, this does mean that the CPUs in the host server have to run the iSCSI stack (to
perform encapsulation of SCSI commands and data). This causes CPU and memory
overhead, which can impact performance.

For increased performance, it is better to use dedicated iSCSI HBAs to process the
TCP/IP stack. This technology is known as
relieves the CPUs in the host server from having to process the SCSI encapsulation,
which can lead to better performance.

򐂰 Ethernet transfer rate is growing. 10 Gbps Ethernet is coming and it gains wider

commercial acceptance. Migrating to 10 GbE can significantly increase the performance
of an iSCSI infrastructure.

TCP Offload Engine (TOE). TOE technology

8 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DiskAttach0908.fm

1.3.1 iSCSI initiators and targets

iSCSI uses the concept of initiators and targets, as shown in Figure 1-6.

Figure 1-6 iSCSI components

The protocol allows clients (called initiators) to send SCSI commands (CDBs) to SCSI
storage devices (targets) on remote servers.

An initiator functions as an iSCSI client. An initiator typically serves the same purpose to a
computer as a SCSI bus adapter would, except that instead of physically cabling SCSI
devices (like hard drives and tape changers), an iSCSI initiator sends SCSI commands over
an IP network. An initiator falls into two broad types:

򐂰 Software initiator

A software initiator uses code to implement iSCSI. Typically, this happens in a
kernel-resident device driver that uses the existing network card (NIC) and network stack
to emulate SCSI devices by speaking the iSCSI protocol. Software initiators are available
for most operating systems, and this type is the most common mode of deploying iSCSI.

An example of an iSCSI stack is the Microsoft iSCSI Software Initiator, which runs on
Windows Server 2003, Windows Server 2008 and VMware ESX4. At the time of writing,
the current version is V2.08 and is available for download from the Microsoft Web site. For
Windows Sever 2008 and ESX4 the iSCSI initiator is included in the box.

Note: Refer to System Storage Operation Center (SSIC) for the complete list of the
supported operating systems. The SSIC can be found at:

http://www-03.ibm.com/systems/support/storage/config/ssic/displayesssearchwi
thoutjs.wss?start_over=yes

For the IBM AIX operating system, refer to the “iSCSI software initiator and software
target” topic at http://publib.boulder.ibm.com/infocenter/systems/index.jsp.

Chapter 1. Disk attachment technology 9

7914DiskAttach0908.fm Draft Document for Review March 28, 2011 12:24 pm

򐂰 Hardware initiator

A hardware initiator uses dedicated hardware, typically in combination with software
(firmware) running on that hardware, to implement iSCSI. A hardware initiator mitigates
the overhead of iSCSI and TCP processing and Ethernet interrupts, and therefore may
improve the performance of servers that use iSCSI.

An iSCSI host bus adapter (more commonly, HBA) implements a hardware initiator. A
typical HBA is packaged as a combination of a Gigabit (or 10 Gigabit) Ethernet NIC, some
kind of TCP/IP offload engine (TOE) technology and a SCSI bus adapter, which is how it
appears to the operating system. Inside the operating system, the iSCSI HBAs are
classified as storage adapters.

An iSCSI HBA can include PCI option ROM to allow booting from an iSCSI target.
A TCP Offload Engine, or “TOE Card”, offers an alternative to a full iSCSI HBA. A TOE

“offloads” the TCP/IP operations for this particular network interface from the host
processor, freeing up CPU cycles for the main host applications. When a TOE is used
rather than an HBA, the host processor still has to perform the processing of the iSCSI
protocol layer itself, but the CPU overhead for that task is low.

iSCSI HBAs or TOEs are used when the additional performance enhancement justifies the
additional expense of using an HBA for iSCSI, rather than using a software-based iSCSI
client (initiator).

An iSCSI target usually represents hard disk storage that works over the IP or Ethernet
networks. DS3500. Other types of peripheral devices, like tape drives and medium changers,
can act as iSCSI targets as well.

iSCSI naming

The iSCSI initiators and targets on a SAN are known by their respective iSCSI names, which
must be unique. The iSCSI name is used as part of an ISCSI address, as part of all sessions
established between initiators and targets. Next, we describe two types of iSCSI names:

򐂰 IQN
򐂰 EUI
򐂰 NASA

IQN

A commonly used format of iSCSI names is the iSCSI Qualified Name (IQN). The format of
an IQN is:

iqn.yyyy-mm.{reversed domain name}

For example, an iSCSI HBA inside a host server named Rhine in the domain rivers.local
would be assigned the following IQN:

iqn.2010-08.local.rivers.rhine

The DS3500 uses IQN names.

EUI

An alternative type of iSCSI name is the Enterprise Unique Identifier (EUI). The format of an
EUI is

eui plus 16 hex digits. For example:

eui.acdc15882005bdef

NASA

NAA name formats were added to iSCSI to provide compatibility with naming conventions
used in Fibre Channel and Serial Attached SCSI (SAS) storage technologies.

10 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DiskAttach0908.fm

The format of a NASA is naa plus NASA 64 or 128 bit identifier. For example:

naa.52004567BA64678D

iSCSI addressing

Usually an iSCSI participant can be defined by three or four fields:

1. Hostname or IP Address (e.g., “iscsi.example.com”)

2. Port Number (e.g., 3260)

3. iSCSI Name (e.g., the IQN "iqn.2003-01.com.ibm:00.fcd0ab21.shark128")

4. An optional CHAP Secret (e.g., "secrets")

The iSCSI address can have the following format.

<IP address>[:<port>]/<iSCSI name>

The IP address can be either IPv4, IPv6, or the fully qualified domain name. The <port> is
optional; it specifies the TCP port that the target is listening for connections on. If it is not
used, the well-known iSCSI port (3260) is assumed. The <iSCSI name> is the IQN or EUI
name of the device. It is optional.

The iSCSI address specifies a single path to an iSCSI target. The iSCSI address is primarily
used during discovery.

1.3.2 iSCSI discovery

iSCSI discovery allows an initiator to find the target(s) to which it has access. This requires a
minimum of user configuration. Several methods of discovery may be used:

A list of targets at the initiator

An administrator can statically define the iSCSI targets to the host system initiator. This
process allows the administrator to specify the iSCSI target node name and IP address:port
to the host system initiator or its host bus adapter (HBA). iSCSI HBAs should support an
administrator defining this information. This type of discovery is useful in small installations
and is known as

Queries to known iSCSI servers

An iSCSI initiator can probe its environment and, when a possible iSCSI target is found, start
a

discovery session with the target by issuing a SendTargets command. The target can reply

to a SendTargets command by returning a list of all iSCSI target nodes it knows about.

Queries to an Internet Storage Name Server (iSNS)

The Internet Storage Name Server permits iSCSI targets to register with a central point. The
administrator can set up discovery domains so that when a host iSCSI initiator queries the
central control point for the locations of iSCSI storage controllers, only the authorized
controllers are reported. The iSNS server can be located by one of the following techniques:

򐂰 iSCSI initiators multicasting to the iSNS server

static discovery.

򐂰 Setting the iSNS server IP address in the DHCP server
򐂰 Setting the iSNS server IP address in the iSCSI initiator or target
򐂰 Setting the iSNS server IP address in the SLP server (see “Service Location Protocol

(SLP)” on page 12)

Chapter 1. Disk attachment technology 11

7914DiskAttach0908.fm Draft Document for Review March 28, 2011 12:24 pm

Service Location Protocol (SLP)

The Service Location Protocol can be used to locate iSCSI target devices. SLP operates with
three agents:

򐂰 User agent (UA): Works on the client (iSCSI initiator) to help establish contact with a

service (iSCSI target). It does this by retrieving information from service agents (SA) or
directory agents (DA).

򐂰 Service agent (SA): Runs on the iSCSI target device to advertise the service and its

capabilities.

򐂰 Directory agent (DA): Collects service advertisements from the iSCSI targets.

1.3.3 iSCSI security considerations

FC disk attachment uses a separate FC SAN, not accessible to Ethernet network users.
iSCSI, on the other hand, is a SAN technology that uses the Ethernet network, which is a lot
more vulnerable to intrusion. Therefore, iSCSI security is very important.

iSCSI connection authentication

iSCSI initiators and targets prove their identity to each other using the Challenge Handshake
Authentication Protocol (CHAP), which includes a mechanism to prevent cleartext passwords
from appearing on the wire. When enabled, the iSCSI target will authenticate the initiator.
Optionally, the initiator can authenticate the target as well. Each connection within a session
has to be authenticated. In addition to CHAP, several authentication methods can be used:

򐂰 Secure Remote Password (SRP)
򐂰 Kerberos V5 (KRB5)
򐂰 Simple Public-Key generic security service API Mechanism (SPKM1)
򐂰 Simple Public-Key generic security service API Mechanism (SPKM2)

In our sample configurations, we used CHAP.

IP Security (IPSec)

As iSCSI relies on TCP/IP communication, IP Security (IPSec) can be used to achieve
increased security. IPSec authenticates and encrypts each packet in the IP data stream.
There are two IPSec modes:

򐂰 Transport mode

With transport mode, only the payload in each packet is encrypted. The IP header is left
unencrypted, so the routing works just the same as without IPSec.

򐂰 Tunnel mode

With tunnel mode, the entire packet is encrypted, including the IP header. This means that
the whole encrypted packet must be encapsulated in a new IP packet, so that routing will
function properly.

IPsec is commonly used to set up Virtual Private Networks (VPN)

12 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

2

Chapter 2. Introduction to IBM System

Storage DS3500

In this chapter, we introduce the new IBM System Storage DS3500 Storage Subsystem
offerings and functionality. These products consists of different models of storage subsystems
which provide for a variety of different environments to meet various user needs. We describe
the EXP3512 and EXP3524 SAS disk drive enclosures as well.

We also explain the Premium Features philosophy and how the Storage Manager utility works
with these new products.

© Copyright IBM Corp. 2010. All rights reserved. 13

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

2.1 IBM System Storage Portfolio

IBM has brought together into one family, known as the DS family, a broad range of disk
systems to help small to large size enterprises select the right solutions for their needs. The
DS family combines the high-performance IBM System Storage DS8000 Series of enterprise
servers, with the DS5000 series of mid-range systems, and the DS3000 entry level systems.

The DS3000 series consist of two new major products; the DS3500 and the DS3950. Both of
these products are a good fit for the entry to mid-range SAN and direct attach market space.
With the common Storage Manager being shared by these new DS3000 storage systems and
the DS5000 storage systems there is a smooth link into the DS5000 series systems, with
remote mirroring and copy services features being shared by these two platforms. The
DS3500 and the DS3950 offer robust functionality, exceptional reliability and availability with
the common ease of storage management being shared by all. The overall positioning of
these new DS3000 series products within the IBM System Storage DS® family is shown in
Figure 2-1.

Figure 2-1 IBM System Storage family

2.2 DS3500 product models

The new IBM System Storage DS3500 series storage subsystems support up to two
redundant RAID controllers in either a twelve or twenty four drive configuration. The models
for the storage servers are: DS3512 and DS3524. There are also two models of drive
expansion chassis (a twelve and a twenty four drive) that can be attached to either of the
storage subsystems. The models for these are: EXP3512 and EXP3524. The new DS3500
models provides a number of new capabilities from the previous generations. The
enhancements are:

򐂰 Allows for one storage subsystem to be able to perform in the environments of the three

legacy DS3000 family members, with support options for SAS, iSCSI and fiber channel
host connections.

򐂰 With this new generation we have the marriage of the DS3000 and the DS5000 Storage

Manager and firmware releases, allowing for a common management console to support
the entry and midrange DS families.

򐂰 Adds the Enhanced Remote Mirroring (ERM) Premium Feature to the DS3000 line
򐂰 New 6Gbps SAS technology for host and drive attachments.

1

Only available in DS3500 and DS3950

1

.

14 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

򐂰 Support for greater capacity with new larger capacity SAS drive offerings.

Figure 2-2 and Figure 2-3 show the front view of both chassis models of these subsystems.

Figure 2-2 DS3512 or EXP3512 subsystem assembly from the front view

Figure 2-3

DS3524 or EXP3524 servers assembly from the front view

2.2.1 DS3512 and DS3524 Components

The DS3500 storage server is a 2U rack mountable enclosure, containing either one or two
RAID controller modules, two power supplies and up to twelve or twenty four disks modules.
See Figure 2-4 for the component layouts.

Figure 2-4 DS 3500 components

RAID controller

RAID controllers support RAID levels 0, 1, 3, 5, 6, and 10. Each controller has 1GB
(upgradeable to 2GB) of user data cache with battery backup. The battery provides power to
allow for the cache to be destaged to the SD flash card if power is disrupted.

Chapter 2. Introduction to IBM System Storage DS3500 15

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

In dual controller configurations, the controller on the left is A and the right is B, when viewed
from the rear of the subsystem. Dual controller configurations offer redundant access to disk
storage. In case of controller or I/O path failure, the other controller will continue to provide
access to disk drives.

All DS3500 RAID controllers have connectors for the following connections built into them:

򐂰 Two 6Gbps SAS host server attachment ports
򐂰 Drive side 6Gbps SAS expansion port
򐂰 Ethernet management port
򐂰 Serial management port

The RAID controllers and two redundant power supply modules are installed in the rear of the
subsystem as shown in Figure 2-5 below.

Figure 2-5 DS3500 controller subsystem rear view

In Figure 2-5, the controller modules are in the upper half of the subsystem and the power
supply modules are in the lower half.

Power Supply

The DS3500 power supply module is a 585 Watt DC power supply. It is auto ranging
100-240VAC input capable. As shown in Figure 2-5 the power supply provides LED indicators
for the following states (starting from left):

򐂰 Standby power LED (green) - Currently this LED is not used.
򐂰 DC power LED (green) - When this LED is lit, it indicates that the DS3500 is turned on and

is supplying both 5-volt and 12-volt dc power.

򐂰 OK to remove LED (blue) - When this blue LED is lit, it indicates that it is safe to remove

the power supply.

򐂰 Fault LED (amber) - When this amber LED is lit, it indicates that a power supply or fan has

failed or that a redundant power supply is not turned on.

򐂰 AC power LED (green) - When this green LED is lit, it indicates that the storage subsystem

is receiving ac power

Host interface cards

As mentioned earlier the DS3500 comes with two SAS host attachment ports built into the
controller modules. Additional host server connectivity is supported through the use of an
optional daughter card (shown in Figure 2-6 on page 17). This interface card can provide for
one of the following to be added to the DS3500:

򐂰 Additional four SAS ports
򐂰 Eight 1Gbit iSCSI ports (four per controller)
򐂰 Eight FC ports (four per controller)

16 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

Figure 2-6 Example host interface daughter card module

Both the single and the dual controller models of the DS3500 storage servers can be
upgraded to include an optional host interface daughter card. When dual controllers are
installed both controllers must be equipped with the same daughter card option to enable the
support of the controller failover functions.

Figure 2-7 shows the SAS optional daughter card installed in the controller. With this option
the subsystem will have up to eight 6Gbps SAS connections for host attachments. For details
on the cabling and use of with the bladecenter and standalone environments see 4.8, “Host
attachment” on page 110.

Figure 2-7 Controller module with optional SAS host interface daughter card

Figure 2-8 shows the iSCSI optional daughter card installed in the controller. With this option
the subsystem will have up to eight iSCSI connections for host attachments. For details on the
cabling and use of with the bladecenter and standalone environments see 4.8, “Host
attachment” on page 110

Figure 2-8 Controller module with optional iSCSI host interface daughter card

Figure 2-9 on page 18 shows the fiber channel optional daughter card installed in the
controller. With this option the subsystem will have up to eight 8Gbps fiber channel
connections for host attachments. For details on the cabling and use of with the bladecenter
and standalone environments see 4.8, “Host attachment” on page 110.

Chapter 2. Introduction to IBM System Storage DS3500 17

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 2-9 Controller module with optional FC host interface daughter card

Note: Only one type of optional interface can be added to any one DS3500 storage server.
Mixing interface daughter cards between controllers in the same DS3500 is not supported.

Disk drives

The most important difference between the DS3512 and the DS3524 product models and
their equivalent expansion models are the hard disk that are supported with them. The
difference starts with the physical drive size and extends to their speeds and storage
capacities. The DS3512 and EXP3512 support twelve drive in the 3.5 inch format; the
DS3524 and EXP3524 supports twenty four drive in the 2.5 inch format. The disk drives are
installed at the front, as shown above in Figure 2-2 on page 15 and Figure 2-3 on page 15.
Available drive types for each of these subsystems are shown below in Table 2-1.

Table 2-1 DS3500 families HDD support

Drives Supported DS3512/EXP3512 DS3524/EXP3524

SAS 15KRPM 300GB, 450GB,

600GB

SAS 15KRPM SED 600GB None

SAS 10KRPM None 300GB

SAS 10KRPM SED None 300GB

Nearline SAS

7.5KRPM

Maximum drives 12/96 24/96

Storage system
capacity (max)

1TB, 2TB 500GB

450 GB SAS / 1 TB
SATA

Note: In DS3500 family, you can add a mix of EXP3512 or EXP3524 expansion units to
attain a maximum capacity of 190TB per subsystem.

2.3 EXP3512 and EXP3524

73GB, 146GB

450 GB SAS / 1 TB
SATA

The EXP3512 and EXP3524 expansion subsystems allow for the growth of the DS3500
storage subsystem up to the 96 drive maximum, by adding either the twelve or twenty four
drive chassis to the storage server’s SAS drive expansion port. Any mix of the expansion
models can be added up to the maximum allowed drive count. The EXP3512 and EXP3524
differ from the DS3512 and DS3524 in that in place of the controller module they are
equipped with an Environmental Services Module (ESM). As with the DS3500 controllers the

18 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

expansions can be optionally upgraded with a second ESM module for redundant paths.
Each ESM has a 6Gbps SAS connection providing 600MB/sec throughput. Figure 2-10
shows the rear view of the EXP35xx with its port connections for cabling.

Figure 2-10 EXP3512 and EXP3524 rear view with SAS port connections...

With the EXP 3512 and EXP3524 only one of the two IN ports are used on each ESM to
connect expansions together into a cascaded loop configuration. As shown in Figure 2-11 on
page 20, the cabling scheme used for connecting these expansions follows what is known as
a “top down, bottom up” method. This provides the expansion loops with redundant paths to
the enclosures; and in the event of one expansion encountering a catastrophic failure, the
others are still able to continue to run. With a proper RAID layout this can provide for
uninterrupted operations.

Chapter 2. Introduction to IBM System Storage DS3500 19

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 2-11 EXP35XX expansion cascaded loop

2.4 Premium Features

Standard configurations of DS3500 storage subsystems come with the following Premium
Features.

DS3000 Partition Expansion License

As part of the standard configuration, four storage partitions are enabled on the DS3500 with
Windows and Linux on Intel® host attach license (this can be increased to 8, 16, 32 or 64).
The maximum number of storage partitions is 64 on all DS3500 products. Use the DS3500
Partition Expansion License to enable all 64 storage partitions.

DS3000 FlashCopy Expansion License

This feature enables FlashCopy®. FlashCopy is a point-in-time copy of a source logical drive.
The FlashCopy logical drive becomes available almost instantaneously.

FlashCopy requires the use of a defined
content of the data that has since been altered. FlashCopy logical drives are often used as a

20 IBM System Storage DS3500: Introduction and Implementation Guide

FlashCopy repository, which will contain the original

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

source for a backup operation. They can also be used to simply and quickly roll back to an
original data state, thus providing a restore point. However, if the source logical drive is lost,
the point-in-time FlashCopy will be lost as well. For more information about FlashCopy, see
IBM Midrange System Storage Copy Services Guide, SG24-7822.

Note: Flashcopy does not provide a permanent full image copy for recovery use. For this
functionality you must use the VolumeCopy feature.

As part of the standard configuration, two FlashCopies are enabled on every DS3500 storage
subsystem and this Premium Feature enables up to 64 FlashCopies. A maximum of eight
flashcopies can be created on a single logical drive.

DS3000 VolumeCopy License

VolumeCopy is a way to provide a complete point-in-time copy of a source logical drive. As
opposed to FlashCopy (where only the original values of changed data are copied to the
repository), the whole source logical drive is copied to target. You can use this functionality for
data replication, relocation, backup, or to restore snapshot data to the original logical drive.
The time required to establish a copy will depend on the size of the source data and the
operation priority settings. While establishing the copy, the source logical drive will be in
read-only state.

Once all the data is copied to the target, the target will remain available if the source logical
drive is lost. For more information about VolumeCopy, see IBM Midrange System Storage
Copy Services Guide, SG24-7822.

The VolumeCopy Premium Feature allows for up to 128 VolumeCopies. Be aware that
FlashCopy is a prerequisite for VolumeCopy.

DS3000 FlashCopy VolumeCopy License

As stated above, the FlashCopy Premium Feature must be enabled before you can enable
and use VolumeCopy. For this reason, IBM provides the FlashCopy VolumeCopy license; this
is actually a bundle of both Premium Features.

DS3000 Enhanced Remote Mirroring ERM

With the DS3500 and DS3950 ERM provides a way to create a remote image copy of a
source logical drive. This capability is frequently used for the creation of a disaster recovery
site located in a different location some distance from the primary location. ERM provides
three different mirroring modes to chose from:

򐂰 synchronous - Provides mirroring capability in the metro (campus) environments,

generally within 10 mile radius. Requires a low latency network as mirroring must
complete before the primary completes the acknowledgement to the host.

򐂰 asynchronous with write order consistency group - Provides mirroring capability in the

global environments, generally over 10 mile distances, where latency may exceed
acceptable response times for the primary host IO operation. IO operations are handled in
the order of their original request across the selected logical drives who are grouped
together through the use of a consistency group.

򐂰 asynchronous copy - Provides copying capability in the global environments, generally

over 10 mile distances, where latency is high and bandwidth is limited causing a high
backup of IO which may impact the principle operation site. IO’s in this mode are not
guaranteed to be processed in order received across a number of logical drives. exceed
acceptable response times for the primary host IO operation.

Chapter 2. Introduction to IBM System Storage DS3500 21

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

This capability is shared across the DS3500 and DS3950 and the DS5000 midrange products
which allows for the remote location to be a consolidated collection site.

Up to eight mirrored pairs can be supported on the DS3500 and 64 mirrored pairs are
supported by the DS3950. For more information about Enhanced Remote Mirroring, see IBM
Midrange System Storage Copy Services Guide, SG24-7822.

2.5 DS3500 and DS3950 Comparisons

Table 2-2 shows the different capabilities of the new DS3000 product offerings for ease of
comparison.

Table 2-2 DS3500 and DS3950 Specification comparisons

Fully configured systems DS3500 DS3950

Host interfaces Eight 8 Gb/s FC and four 6 Gb/s

SAS
Eight 1 Gb/s iSCSI and four 6
Gb/s SAS
Four or eight 6 Gb/s SAS

Redundant drive channels Two 6 Gb/s SAS Four 4 Gb/s FC

Drive enclosures 12-drive (3.5-in) 2U

24-drive (2.5-in) 2U

Max drives 48 standard

96 premium feature

Drives supported SAS, NL SAS, FDE FC,SATA, FDE

XOR engine Integrated XOR / P+Q Integrated XOR / P+Q

Dual Ethernet Yes Yes

Data cache / memory 2 or 4 GB 2 or 4 GB

Max logical drives 256 1,024

Max LUN size > 2TB > 2TB

RAID levels 1, 3, 5, 6, 10 1, 3, 5, 6, 10

Max RAID 10 / 0 size 96 drives 112 drives

Four 8 Gbps FC,
or Four 8 Gbps FC and four 1
Gbps iSCSI

16-drive (3.5-in) 3U

112

Max RAID 3 / 5 / 6 size 30 drives 30 drives

Max partitions 64 128

Max snapshots per base
volume

Maximum remote mirrors
(across FC)

22 IBM System Storage DS3500: Introduction and Implementation Guide

88

864

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

2.6 IBM System Storage DS Storage Manager

The new DS3500 and DS3950 storage subsystem uses the same Storage Manager as the
DS5000 product line and the legacy DS3000 models. At the time of this writing the current
version of the software is 10.70.G5.08. When you receive your DS3500 storage subsystem,
you may also receive with it a copy of the IBM System Storage DS Storage Manager Software
and Host Kit CDs. If you do not receive this software, or the version you receive is not
compatible with your DS3500’s firmware release, you can download it from the IBM support
Web site.

Using IBM System Storage DS Storage Manager software, you can perform tasks such as
creating arrays and logical drives, assigning logical drives to the host servers, setting up
FlashCopy and VolumeCopy, capturing logs for troubleshooting, and so on. We discuss the
IBM System Storage DS Storage Manager software and it’s use in great detail later in this
book, so we will just briefly describe some of the main points here.

When discussing the IBM System Storage DS Storage Manager, it is important to differentiate
between the following two terms:

򐂰 Host server

This is a server attached to the DS3500 storage subsystem through the I/O path (SAS,
iSCSI, or Fibre Channel). The host server has access to the logical drives which are
defined on the DS3500 storage server for its storage use.

򐂰 Management station

The management station is the system that is responsible for managing all, or a portion of,
a storage network. The IBM System Storage DS Storage Manager provides a GUI which
runs on the management station. This management station can be based on Windows,
Linux, AIX or Solaris operating systems. There may be slight screen shading differences
between the different operating system version of the displays, but the fit and functions are
the same for all. You need to establish a management connection between the
management station and the DS3500 storage subsystem. This can be done in two ways:

– Out-of-band

When using out-of-band management, the management station is connected to the

Ethernet management port in each DS3500 RAID controller. All management

communication flows across the TCP/IP connection between the management station

and the DS3500. We also call this method

management station in this case only requires an Ethernet connection to the DS3500.
– In-band

This method utilizes the I/O path between a host server and the DS3500. In this case,

the management station does not have direct TCP/IP connection to the DS3500, but

rather communicates with the DS3500 through an HBA, which acts as a gateway to the

DS3500 storage subsystem, that is, communication between the management station

and the host server is across the fiber channel, SAS or iSCSI I/O path.

We also call this method

host-attached management.

direct-attached management. The

Because each method has some associated advantages as well as disadvantages we will
discuss them both to help you select which is more appropriate for your environment. Both
methods offer identical functionality; you can perform any management task with either of
these methods.

Chapter 2. Introduction to IBM System Storage DS3500 23

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

In-band management

The in-band management method uses the I/O path between the host server and the DS3500
to transfer management commands and information.

This method does not use the management Ethernet ports on DS3500 RAID controllers and
does not require a management TCP/IP network. However, it does require a special

logical drive

maximum number of logical drives, because one of them is reserved for the access logical
drive. But this is usually not a problem, because virtually all customers will find the maximum
number of logical drives more than sufficient.

An example of in-band management is shown in Figure 2-12. Two host servers are attached
to the DS3500 subsystem with FC cables. They both run SMagent code. The management
workstation runs SMclient code. SMclient communicates with SMagent through Ethernet, and
SMagent communicates with the DS3500 across the FC I/O path.

to manage the DS3500 controllers. This means that you cannot configure the

access

Figure 2-12 In-band management

24 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

Access logical drive

The access logical drive exists in each storage partition by default (no manual configuration is
required). This is not actually a real logical drive, although it is presented to the operating
system as a drive on LUN 31. In order to allow for in-band management communication
across the I/O path, we need a target device for SCSI commands. The SCSI commands to
this target are used as a vehicle for Storage Management communication along the I/O path.
The access logical drive is that target device. The access logical drive is also sometimes
referred to as the

Universal Transport Mechanism (UTM) device or the Universal XPort

device.

Out-of-band management

Out-of-band management requires that the management IP addresses are configured on
both controllers and that the controllers’ management ports are connected to the
management network. This should be a separate LAN or a VLAN, as we do not recommend
using the production LAN or VLAN for the management network traffic.

A separate management workstation is another requirement; typically, the system
administrator uses their own workstation for this purpose. Figure 2-13 shows the
management workstation and the DS3500 subsystem, connected on the Ethernet
management network.

Figure 2-13 Out-of-band management

Chapter 2. Introduction to IBM System Storage DS3500 25

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

Out-of-band management offers the following advantages:
򐂰 There is no need for an access logical drive (unlike for in-band management). Therefore,

you can use the maximum number of logical drives supported by the host servers’
operating system.

򐂰 If the I/O paths fail, you can still access the DS3500 storage subsystem out-of-band, check

the status, and capture logs for effective troubleshooting. Access to both controllers is
required for almost all in-band management functions.

򐂰 If in-band management cannot be used (for example, when the SMagent is not available

for the host server operating system), you can effectively use out-of-band management.

We recommend setting up and using both methods, if at all possible. This will introduce some
redundancy to your management setup and provide management access to the DS3500
subsystem even in the case of I/O path or Ethernet management network failure.

The IIBM System Storage DS Storage Manager package consists of the following
components:

򐂰 Storage Manager Client (SMclient/SMcli)
򐂰 Storage Manager Agent (SMagent)
򐂰 Storage Manager Utility (SMutil)
򐂰 Storage Manager multipath support
򐂰 Java™ access bridge (for Windows only)

Storage Manager Client (SMclient)

This is the actual graphical user interface (GUI) that you use to manage the DS3500
subsystems. You install SMclient on the management station. This is usually not one of the
host servers, but rather a workstation that belongs to a system administrator. However, it is
also possible to install SMclient on a host server. There are two high level management
screens that are used for these functions:

򐂰 Enterprise Management Window
򐂰 Subsystem Management Window

Enterprise Management Window

This window opens when the DS3500 Storage Manager is launched. It lists the storage
subsystems it knows about. You can add new storage subsystems and perform various tasks
on the enterprise level. Below we show a sample of the Linux based Enterprise Management
Window in Figure 2-14.

Figure 2-14 Enterprise Management Window

26 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KIntro_081410alw.fm

Subsystem Management Window

This window allows you to manage a particular DS3500 Storage Subsystem. Management
tasks such as creating arrays, logical drives, storage partitions, FlashCopy, and VolumeCopy
are all performed from within the Subsystem Management Window. See Figure 2-15 for an
example of this window again based on the Linux based version.

Figure 2-15 DS3500 IBM System Storage DS Subsystem Management screen

You install SMclient on the management station. This is usually not one of the host servers,
but rather a workstation that belongs to a system administrator. However, it is also possible to
install SMclient on a host server.

SMclient is available for Windows and Linux operating systems.

SMcli command-line interface

Besides providing a GUI for the management tasks, SMclient also includes a component
called SMcli. SMcli provides a powerful command-line interface (CLI). All the tasks available
in the Storage Manager GUI can also be run using the CLI. In addition, there are certain tasks
that are only available in the CLI, but not in the GUI. There are two ways to use the CLI:

򐂰 The SMcli executable, which runs the CLI commands from the operating system

command prompt.

򐂰 The Script Editor in the DS3500 Storage Manager, launched from the Enterprise

Management Window.

You can either run a single command at a time or execute pre-written scripts. See Chapter 17,

“Command-Line Interface(CLI)” on page 533 for more information about the CLI and Script

Editor.

Chapter 2. Introduction to IBM System Storage DS3500 27

7914DS3KIntro_081410alw.fm Draft Document for Review March 28, 2011 12:24 pm

Storage Manager Agent (SMagent)

SMagent is an optional component that is only required for in-band management. SMagent is
installed in the host server and allows the SMclient to communicate with DS3500 across the
I/O path. At the time of writing, only the FC I/O path is supported for management
communication.

Storage Manager Utility (SMutil)

The Storage Manager Utility package contains the following components:
򐂰 Hot-add utility

You can use this utility to dynamically add newly created logical drives to the operating
system running in the host without having to reboot the host server. This utility is available
for Windows only.

򐂰 SMdevices utility

This utility can be used to associate logical drives to device names in the operating
system. It is installed with both the Windows and Linux package.

򐂰 SMrepassist utility

SMrepassist is a Windows utility used with FlashCopy and VolumeCopy. The utility allows
you to flush cached data prior to creating a FlashCopy/VolumeCopy image. In other
operating systems, you need to unmount the file system. SMrepassist does not exist in the
Linux package.

SMutil is a required package in the host server for ease of troubleshooting and problem
analysis. With these utilities there are may quick changes that can be implemented as well as
a great deal of detail on the current configuration settings and their impacts.

Multipath driver support

We recommend installing two HBAs in the host servers and using the dual controller DS3500
subsystems. This provides the ability to create completely redundant paths to the host
server’s storage therefore improving the availability through multiple I/O paths to the
controllers and full storage redundancy out the backend to the disks. However, host dual path
configurations can only work properly if you install the appropriate multipath driver for your
particular operating system into the host server.

The IBM System Storage DS Storage Manager Software and Host Kit for Windows includes
multipath support, based on the Microsoft MPIO framework. The IBM System Storage DS
Storage Manager Software and Host Kit for Linux does not include any multipath support.
RDAC for Linux is available as a separate open source package called MPP. For Solaris there
is a version of DMP which can be implemented for multi-path support. And in AIX there is an
MPIO driver that is available to support the DS family of storage subsystems. All of these
drivers require special configurations and setups to be followed when implementing them and
desiring them to function in the expected manner. A simple parameter setting can be the
difference between a successful failover to an alternate path, and an application outage.

Java Access Bridge

This is included in the Windows Storage Manager package only. Java Access Bridge for
Microsoft Windows makes it possible for Windows based assistive technology to access and
interact with the application.

28 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

3

Chapter 3. IBM System Storage DS3500

Storage System planning tasks

Careful planning is essential to any new storage installation. This chapter provides guidelines
to help you with the planning process.

Choosing the right equipment and software, and also knowing what the right settings are for a
particular installation, can be challenging. Every installation has to answer these questions
and accommodate specific requirements, and there can be many variations in the solution.

Having a well thought-out design and plan prior to the implementation will help you get the
most out of your investment for the present and protect it for the future.

During the planning process, you need to answer numerous questions about your
environment:

򐂰 What are my SAN requirements?
򐂰 What hardware do I need to buy?
򐂰 What reliability do I require?
򐂰 What redundancy do I need? (For example, do I need off-site mirroring?)
򐂰 What compatibility issues do I need to address?
򐂰 Will I use any storage virtualization product such as IBM SAN Volume controller?
򐂰 Will I use any unified storage product such as the IBM System Storage N series?
򐂰 What operating system am I going to use (existing or new installation)?
򐂰 What applications will access the storage subsystem?
򐂰 What are the hardware and software requirements of these applications?
򐂰 What will be the physical layout of the installation? Only local site, or remote sites as well?
򐂰 What level of performance do I need?
򐂰 How much does it cost?

This list of questions is not exhaustive, and as you can see, certain questions go beyond
simply configuring the storage system. But the focus in this chapter is to help with the creation
of a successful solution and that frequently extends beyond a single subsystem.

Use this chapter as a reference to help you gather the information for the statements.

© Copyright IBM Corp. 2010. All rights reserved. 29

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

3.1 Planning your SAN and storage server

When planning the setup of a Storage Area Network (SAN), you want the solution to answer
your current requirements and fulfill your future needs.

First, the SAN fabric must be able to accommodate a growing demand in storage (it is
estimated that storage needs double every two years). Second, the SAN must be able to keep
up with the constant evolution of technology and resulting hardware upgrades and
improvements. It is estimated that a storage installation needs to be upgraded every 2 to 3
years.

Ensuring compatibility among various pieces of equipment is crucial when planning the
installation. The important question is what device works with what, and also who has tested
and certified that equipment.

When designing a SAN storage solution, it is a best practice to complete the following steps:

1. Produce a statement outlining the solution requirements that can be used to determine the
type of configuration you need. Then use this statement to cross-check that the solution
design delivers the basic requirements. The statement must have easily defined bullet
points covering the requirements, for example:

– New installation or upgrade of existing infrastructure
– Infrastructure type(s) to be used: SAS, iSCSI, fiber channel (direct or fabric)
– Host Bus Adapter (HBA) selection
– HBA driver type selection: SCSIPort or StorPort
– Multipath Driver selection: RDAC, DMMP, MPIO, or SDDPCM
– Types of applications accessing the SAN (whether transaction or throughput intensive)
– Required capacity
– Required redundancy levels
– Type of data protection needed
– Current data growth patterns for your environment
– Whether current data is more read or write based
– Backup strategies in use: Network, LAN-free, or Server-less
– Premium features required: Partitioning, FlashCopy, Volume Copy, or Enhanced

Remote Mirroring
– Number of host connections required
– Types of hosts and operating systems that will connect to the SAN
– Zoning required
– Distances between equipment and sites (if there is there more than one site)

2. Produce a hardware checklist. It must cover such items that require you to:
– Make an inventory of existing hardware infrastructure. Ensure that any existing

hardware meets the minimum hardware requirements and is supported with the

DS3500 Storage System.
– Make a complete list of the planned hardware requirements.
– Ensure that you have enough rack space for future capacity expansion.
– Ensure that the power and environmental requirements are met.

30 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

– Ensure that your existing network of SAS, ethernet or fibre channel switches and

cables are properly configured.

3. Produce a software checklist to cover all the required items that need to be certified and
checked. It must include such items that require you to:

– Ensure that the existing versions of firmware and storage management software are up

to date.

– Ensure that host operating systems are supported with the storage system. Check the

IBM System Storage interoperation Center (SSIC) for your specific storage system
available from this Web site for more information:

http://www.ibm.com/servers/storage/disk

These lists are not exhaustive, but the creation of the statements is an exercise in information
gathering and planning; it gives you a greater understanding of what your needs are in your
current environment and creates a clear picture of your future requirements. The goal ought
to be quality rather than quantity of information.

Use this chapter as a reference to help you gather the information for the statements.

Understanding the applications is another important consideration in planning for your
DS3500 Storage System setup. Applications can typically either be I/O intensive, such as
high number of I/O per second (IOPS); or characterized by large I/O requests, that is, high
throughput or MBps.

򐂰 Typical examples of high IOPS environments are Online Transaction Processing (OLTP),

databases, and Microsoft Exchange servers. These have random writes and fewer reads.

򐂰 Typical examples of high throughput applications are data mining, imaging, and backup

storage pools. These have large sequential reads and writes.

By understanding your data and applications, you can also better understand growth
patterns. Being able to estimate an expected growth is vital for the capacity planning of your
DS3500 Storage System installation. Clearly indicate the expected growth in the planning
documents: The actual patterns might differ from the plan according to the dynamics of your
environment.

Selecting the right DS Storage System model for your current and perceived future needs is
one of the most crucial decisions you will make. The good side, however, is that the DS3500
platforms offer scalability and expansion flexibility. Premium features can be purchased and
installed at a later time to add more functionality to the storage server as well.

In any case, it is perhaps better to purchase a higher model than one strictly dictated by your
current requirements and expectations, which will allow for greater performance and
scalability as your needs and data grow. Starting out with a maximum configured storage
solution to meet your current needs with no room to expand may save initially, but could
quickly be too small for your business needs and near term growth.

3.1.1 SAN zoning for the DS3500 Storage System

Zoning is an important part of integrating a DS3500 Storage System in a SAN. When done
correctly, it can eliminate many common problems. Zoning also helps with creating paths that
can be used by the multi-path drivers for better failover protection. Understanding the
capabilities of the multi-path driver is important when designing the paths it will use.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 31

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Important: Disk and tape must be on separate HBAs, following the best practice for

zoning; then the disk and tape access will also be in separate zones. With certain UNIX®
systems, this arrangement is supported by the DS3500 Storage System, but might have
hardware limitations; therefore,

do not share HBAs between disk storage and tape.

Zoning with MPIO

MPIO is a multipath driver which provides for a host to be able to recognize multiple paths to
the attached storage device. This is done by utilizing multiple HBA ports or devices within the
host server connected to SAN fabric switches, which are also connected to the multiple ports
on the storage devices. For the storage products referred to as DS3000, DS4000, or DS5000,
these devices have two controllers within the subsystem that manage and control the disk
drives. These controllers behave in an active/passive fashion. Ownership and control of a
particular LUN is done by one controller. The other controller is in a passive mode until a
failure occurs, at which time the LUN ownership is transferred to that controller. Each
controller may have more than one fabric port for connectivity to the SAN fabric. This helps to
provide faster recovery for path failures that can be resolved through the use of an alternate
connection to the same controller verses requiring the full failover process be ran.

The DS3500 models only support drivers that by design follow the rules for this type of path
management. See Figure 3-1 for an example of how the configuration should be
implemented.

Figure 3-1 Host HBA to storage subsystem controller multipath sample configuration

With MPIO, it is better to create a zone for each HBA port to be able to see both controllers to
help decrease failover operations for network related failures. In the above example to do this
you would create zones for the three host servers (two zones each) with one being from HBA
port 1 to both controller A port 1 and controller B port 1 of the DS3500 storage server, and the

32 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

second zone from HBA port 2 of the host to controller A and Controller B through port 2 of the
controllers.

With this configuration if a single path has a fault another path can be used to access the
devices without the need of a controller failover function being done.

Best practice: For MPIO driver solutions it is best to create separate zones for each HBA
port connection from the host to both controllers (one zone for HBA port 1 and one zone for
HBA port 2), which isolates each initiator (HBA port) from the other.

3.1.2 Zoning considerations for Enhanced Remote Mirroring

The Enhanced Remote Mirroring (ERM) can only be connected through a Fibre Channel (FC)
connection which must be dedicated for data replication between the subsystems. The ports
used for ERM cannot be used to send or receive I/Os from any host. These requirements are
addressed by defining SAN zoning. There must be two zones defined for the ERM network,
one for controller A and one for controller B. Zones defined must separate the host ports from
the storage system mirroring ports and also separate the mirroring ports between the
controllers.

When using ERM, you must create two additional zones:
򐂰 The first zone contains the ERM source DS3500 controller A and ERM target DS3500

controller A.

򐂰 The second zone contains the ERM source DS3500 controller B and ERM target DS3500

controller B.

ERM is detailed further in IBM Midrange System Storage Copy Services Guide, SG24-7822.

3.2 Planning for physical components

In this section, we review elements related to physical characteristics of an installation, such
as rack considerations, fiber cables, Fibre Channel adapters, and other elements related to
the structure of the storage system and disks, including enclosures, arrays, controller
ownership, segment size, storage partitioning, caching, hot spare drives, and Enhanced
Remote Mirroring.

3.2.1 Rack considerations

The DS3500 Storage System and possible expansions are mounted in rack enclosures.

General planning

Consider the following general planning guidelines. Determine:
򐂰 The size of the floor area required by the equipment:

– Floor-load capacity
– Space needed for expansion
– Location of columns

򐂰 The power and environmental requirements

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 33

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Create a floor plan to check for clearance problems. Be sure to include the following
considerations in the layout plan:

򐂰 Determine service clearances required for each rack or suite of racks.
򐂰 If the equipment is on a raised floor, determine:

– The height of the raised floor
– Things that might obstruct cable routing

򐂰 If the equipment is not on a raised floor, determine:

– The placement of cables to minimize obstruction
– If the cable routing is indirectly between racks (such as along walls or suspended), the

amount of additional cable needed

– Cleanliness of floors, so that the fan units will not attract foreign material such as dust

or carpet fibers

򐂰 Determine the location of:

– Power receptacles
– Air conditioning equipment, placement of grilles, and controls
– File cabinets, desks, and other office equipment
– Room emergency power-off controls
– All entrances, exits, windows, columns, and pillars
– Fire control systems

򐂰 Check access routes for potential clearance problems through doorways and passage

ways, around corners, and in elevators for racks and additional hardware that will require
installation.

򐂰 Store all spare materials that can burn in properly designed and protected areas.

Rack layout

To be sure that you have enough space for the racks, create a floor plan before installing the
racks. You might need to prepare and analyze several layouts before choosing the final plan.

If you are installing the racks in two or more stages, prepare a separate layout for each stage.

The following considerations apply when you make a layout:

򐂰 The flow of work and personnel within the area
򐂰 Operator access to units, as required
򐂰 If the rack is on a raised floor, determine:

– The need for adequate cooling and ventilation

򐂰 If the rack is not on a raised floor, determine:

– The maximum cable lengths
– The need for cable guards, ramps, and so on to protect equipment and personnel

򐂰 Location of any planned safety equipment
򐂰 Future expansion

Review the final layout to ensure that cable lengths are not too long and that the racks have
enough clearance.

You need at least 152 cm (60 in.) of clearance at the front and at least 76 cm (30 in.) at the
rear of the 42U rack suites. This space is necessary for opening the front and rear doors and
for installing and servicing the rack. It also allows air circulation for cooling the equipment in
the rack. All vertical rack measurements are given in rack units (U). One U is equal to 4.45 cm

34 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

(1.75 in.). The U levels are marked on labels on one front mounting rail and one rear
mounting rail.

Important: All DS3500 storage systems are in chassis that are 2U in height.

Figure 3-2 shows an example of the required service clearances for a 9306-900 42U rack.
Check the documentation for the specific rack model that you will use for a statement on the
required clearances.

Figure 3-2 9306 enterprise rack space requirements

3.2.2 SAS cables and connectors

All DS3500 storage subsystems, regardless of the optional host attachment feature added
will have SAS host connections available. Also with the addition of any expansion subsystems
they use SAS cabling to attach expansion enclosures. Therefore, let us have a look at the
SAS cables and connectors used.

The SAS ports on the DS3500 controller and EXP3500 ESM all support mini-SAS 4x
multilane connectors. SAS cables with mini-SAS connectors that will fit in these ports are
required, as shown in Figure 3-3 on page 36. IBM provides SAS cables in two cable lengths:
1 and 3 meters.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 35

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 3-3 SAS cable

Careful planning should be done to avoid damage to the SAS cables, consider the following
precautions:

򐂰 When you route the cable along a folding cable-management arm, leave enough slack in

the cable.

򐂰 Route the cable away from places where it can be damaged by other devices in the rack

cabinet.

򐂰 Do not put excess weight on the cable at the connection point. Make sure that the cable is

well supported.

To connect a mini-SAS cable, insert the mini-SAS connector into a mini-SAS port. Make sure
that it locks into place.

To remove a mini-SAS cable, complete the following steps:

1. Put one finger into the hole on the blue plastic tab on the mini-SAS connector and gently
pull on the tab to release the locking mechanism.

2. As you pull on the tab, pull out the connector to remove it from the port

Attention: Care should be taken to not use the cable for leverage when removing the
cable from the mini-SAS port.

In Figure 3-4 on page 37 we show an example of SAS connections being made to direct
attached hosts and a bladecenter through it internal SAS switch module. The DS3500 comes
with two standard built-in SAS ports per controller; and this figure show the system with the
additional option daughter card for the additional two ports per controller.

36 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Figure 3-4 DS3500 with optional added SAS host connections

3.2.3 Ethernet cable and connections

Each DS3500 RAID controller contains an Ethernet management port, which you can use for
out-of-band management. If you have a dual controller DS3500 subsystem, make sure the
management workstation can access the management port on each controller. If only one
controller is assessable by the management machine, the DS3500 Storage Manager will not
be able to manage the enclosure.

It is a best practice to not use your public LAN for DS3500 out-of-band management. Instead,
set up a dedicated LAN or VLAN just for management purposes. This will provide increased
security of your DS3500 storage system. If the DS3500 RAID controllers are on a public LAN,
a knowledgeable user could install the DS3500 Storage Manager on a separate workstation,
or use the CLI to run potentially destructive tasks. For an additional layer of security, we also
recommend that you enable password protection on the DS3500 storage system. Refer to
“Set Password” on page 210 for more details.

The DS3500 supports iSCSI SAN, which utilizes the standard Ethernet infrastructure, using
regular Ethernet cables on the host side. The simplest case is a direct connection to a single
host server. It is more typical to attach multiple host servers through an Ethernet switch. See
Figure 3-5 on page 38 for an example.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 37

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 3-5 Four host servers attached to a DS3500 subsystem with iSCSI

3.2.4 Fiber Channel cables and connectors

In this section, we discuss various essential characteristics of fiber cables and connectors.
This information can help you understand the options you have for connecting and cabling the
DS3500 Storage System.

Cable types (shortwave or longwave)

Fiber cables are basically available in multi-mode fiber (MMF) or single-mode fiber (SMF).

Multi-mode fiber allows light to disperse in the fiber so that it takes many paths, bouncing off
the edge of the fiber repeatedly to finally get to the other end (multi-mode means multiple
paths for the light). The light taking these various paths gets to the other end of the cable at
slightly separate times (separate paths, separate distances, and separate times). The
receiver has to determine which incoming signals go together.

The maximum distance is limited by how “blurry” the original signal has become. The thinner
the glass, the less the signals “spread out,” and the further you can go and still determine
what is what on the receiving end. This dispersion (called modal dispersion) is the critical
factor in determining the maximum distance a high-speed signal can travel. It is more relevant
than the attenuation of the signal (from an engineering standpoint, it is easy enough to
increase the power level of the transmitter or the sensitivity of your receiver, or both, but too
much dispersion cannot be decoded no matter how strong the incoming signals are).

38 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

There are two core sizes of multi-mode cabling available: 50 micron and 62.5 micron. The
intermixing of the two core sizes can produce unpredictable and unreliable operation.
Therefore, core size mixing is not supported by IBM. Users with an existing optical fiber
infrastructure are advised to ensure that it meets Fibre Channel specifications and is a
consistent size between pairs of FC transceivers.

Single-mode fiber (SMF) is so thin (9 microns) that the light can barely “squeeze” through and
it tunnels through the center of the fiber using only one path (or mode). This behavior can be
explained (although not simply) through the laws of optics and physics. The result is that
because there is only one path that the light takes to the receiver, there is no “dispersion
confusion” at the receiver. However, the concern with single mode fiber is attenuation of the
signal. Table 3-1 lists the supported distances.

Table 3-1 Cable type overview

Fiber type Speed Maximum distance

9 micron SMF (longwave) 1 Gbps 10 km

9 micron SMF (longwave) 2 Gbps 2 km

50 micron MMF (shortwave) 1 Gbps 500 m

50 micron MMF (shortwave) 2 Gbps 300 m

50 micron MMF (shortwave) 4 Gbps 150 m

50 micron MMF (shortwave) 8 Gbps 50 m

62.5 micron MMF (shortwave) 1 Gbps 300 m

62.5 micron MMF (shortwave) 2 Gbps 150 m

62.5 micron MMF (shortwave) 4 Gbps 70 m

62.5 micron MMF (shortwave) 8 Gbps 21 m

Note that the “maximum distance” shown in Table 3-1 is just that, a maximum. Low quality
fiber, poor terminations, excessive numbers of patch panels, and so on, can cause these
maximums to be far shorter.

All IBM fiber feature codes that are orderable with the DS3500 Storage System will meet the
standards.

Interfaces, connectors, and adapters

In Fibre Channel technology, frames are moved from source to destination using gigabit
transport, which is a requirement to achieve fast transfer rates. To communicate with gigabit
transport, both sides have to support this type of communication, which is accomplished by
using specially designed interfaces that can convert other types of communication transport
into gigabit transport.

The interfaces that are used to convert the internal communication transport of gigabit
transport are Small Form Factor Transceivers (SFF), also often called Small Form Pluggable
(SFP). See Figure 3-6 on page 40. Gigabit Interface Converters (GBIC) are no longer used
on current models although the term GBIC is still sometimes incorrectly used to describe
these connections.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 39

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 3-6 Small Form Pluggable (SFP) with LC connector fiber cable

Obviously, the particular connectors used to connect a fiber cable to a component will depend
upon the receptacle into which they are being plugged.

LC connector

Connectors that plug into SFF or SFP devices are called LC connectors. The two fibers each
have their own part of the connector. The connector is keyed to ensure correct polarization
when connected, that is, transmit to receive and vice-versa.

The main advantage that these LC connectors have over the SC connectors is that they are of
a smaller form factor, and so manufacturers of Fibre Channel components are able to provide
more connections in the same amount of space.

All DS3500 series products use SFP transceivers and LC fiber cables. See Figure 3-7.

Figure 3-7 LC fiber cable connector

Best practice: When you are not using an SFP, it is best to remove it from the port on the
DS3500 storage controller and replace it with a cover. Similarly, unused cables must be
stored with ends covered, which will help eliminate risk of dirt or particles contaminating
the connection while not in use.

Interoperability of 2 Gbps, 4 Gbps, and 8 Gbps devices

The Fibre Channel standard specifies a procedure for speedy auto-detection. Therefore, if a
4 Gbps port on a switch or device is connected to a 2 Gbps port, it must negotiate down and
the link will run at 2 Gbps. If there are two 8 Gbps ports on either end of a link, the negotiation
runs the link at 8 Gbps if the link is up to specifications. A link that is too long or “dirty” can end
up running at 4 Gbps, even with 8 Gbps ports at either end, so care must be taken with cable
lengths distances and connector quality is sound.

40 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

The same rules apply to 8 Gbps devices relative to 4 Gbps and 2 Gbps environments. The
8 Gbps devices have the ability to automatically negotiate back down to either 4 Gbps or 2
Gbps, depending upon the attached device and the link quality. A 4 Gbps device has the
ability to automatically negotiate back down to either 2 Gbps or 1Gbps. If the link does
unexpectedly negotiate to a slower speed than expected, then the causes or reasons for this
ought to be investigated and remedied.

The DS3500 Storage System now has 8 Gbps functionality; there are several switches and
directors that operate at this speed.

Note: On certain fiber switch vendor models, it might be necessary to configure the port to
a specific speed of 2, 4, or 8 Gbps to obtain the required speed instead of leaving
“auto-detection” on the port.

FC host attachment methods

With FC SAN environments there are two methods of host attachment to the storage system
that are commonly used; direct-attached and switch-attached (fabric).

Direct-attached DS3500

Figure 3-8 shows a simple direct-attached configuration. Two host servers are connected to a
dual-controller DS3500. Each server uses two FC HBAs, so this is a fully redundant setup. If
an HBA, FC cable, or RAID controller fails, the host servers will still have access to logical
drives. This type of setup is suitable for a two-node Microsoft Cluster Server configuration.

Figure 3-8 Host servers attached to DS3500 FC direct connect

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 41

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Switch-attached DS3500

As the DS3500 subsystem can support many more than two host servers, let us consider a
larger configuration. For more than two servers, a SAN switch is required. Figure 3-9 displays
a sample configuration with four host servers attached to a dual controller DS3500. Each host
has two FC HBAs for redundant I/O path support.

Figure 3-9 DS3500 connected to hosts through FC switch SAN

If Enhanced Remote Mirroring (ERM) feature is planned to be used, the fourth FC port on
each controller must be dedicated to for this connection. No host servers should be zoned or
configured to used these ports. ERM can only be used with a SAN switch environment.

Cable management and labeling

Cable management and labeling for solutions should cover both the installation, routing and
labeling of all cables including power, ethernet LAN network, SAS for both host attachments
and backend expansions and fiber channel. Cable management and labeling needs have
expanded from the traditional labeling of network connections to management and labeling of
most cable connections between your servers, disk subsystems, multiple network
connections, and power for all the subsystems involved in your solution. In many cases
different components of the solution may have differing cable needs. Multiple unique systems
could be located in the same rack; or a system could span across multiple racks. In some
cases the solution could be configured where components might not be physically located in
the same room, building, or site.

42 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Cable planning

Successful cable management planning includes three basic activities: site planning (before
your solution is installed), cable routing, and cable labeling.

Site planning

Having adequate site planning completed before your solution is installed will result in a
reduced chance of installation problems. Significant attributes covered by site planning are
location specifications, electrical considerations, raised/non-raised floor determinations, and
determination of cable lengths. Consult the documentation for each component of your
solution for special site planning considerations.

Cable routing

With effective cable routing, you can keep your solution's cables organized, reduce the risk of
damaging cables, and allow for affective service and support. Use the following guidelines to
assist with cable routing:

򐂰 When installing cables to devices mounted on sliding rails:

– Run the cables neatly along equipment cable-management arms and tie the cables to

the arms. (Obtain the cable ties locally.)

Note: Do not use cable-management arms for fiber cables.

– Take particular care when attaching fiber optic cables to the rack. See the instructions

included with your fiber optic cables for guidance on minimum radius, handling, and

care of fiber optic cables.
– Run the cables neatly along the rack rear corner posts.
– Use cable ties to secure the cables to the corner posts.
– Make sure the cables cannot be pinched or cut by the rack rear door.
– Run internal cables that connect devices in adjoining racks through the open rack

sides.
– Run external cables through the open rack bottom.
– Leave enough slack so that the device can be fully extended without putting a strain on

the cables.
– Tie the cables so that the device can be retracted without pinching or cutting the

cables.

򐂰 To avoid damage to your fiber optic cables, follow these guidelines:

– Use great care when utilizing cable management arms.
– When attaching to a device on slides, leave enough slack in the cable so that it does

not bend to a radius smaller than that as advised by your fiber optic cable guide when

extended or become pinched when retracted.
– Route the cable away from places where it can be snagged by other devices in the

rack.
– Do not overtighten the cable straps or bend the cables to a radius smaller than that as

advised by your fiber optic cable guide.
– Do not put excess weight on the cable at the connection point and be sure that it is well

supported. For example, a cable that goes from the top of the rack to the bottom

have a method of support other than the strain relief boots built into the cable.

must

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 43

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

– For long cable runs, ensure that enough slack is made for rack movement in

accordance with your computer room standards for earthquake proofing.

Additional information for routing cables for IBM Netfinity® Rack products can be found in IBM
Netfinity Rack Planning and Installation Guide, part number 24L8055. This publication
includes pictures providing more details about how to set up the cable routing.

Cable labeling

When labeling your solution, follow these tips:

򐂰 As you install cables in the rack, label each cable with the appropriate identification.
򐂰 Remember to attach labels to any cables that you replace.
򐂰 Document deviations from the label scheme you use. Keep a copy with your Change

Control Log book.

򐂰 Comply with an existing cable naming convention or define and adhere to a simple logical

naming convention

An example of label naming convention might include these attributes:

򐂰 The function, to help identify the purpose of the cable.
򐂰 Location information must be broad to specific (for example, the site/building to a specific

port on a server or hub).

Other cabling mistakes

Avoid making these common mistakes in cabling:

򐂰 Leaving cables hanging from connections with no support.
򐂰 Not using dust caps.
򐂰 Not keeping connectors clean. (Certain cable manufacturers require the use of lint-free

alcohol wipes in order to maintain the cable warranty.)

򐂰 Leaving cables on the floor where people might kick or trip over them.
򐂰 Not removing old cables when they are no longer needed or planned for future use.

Tip: Collect all SFP, HBA, and cable dust caps, store in a dust free container, to be
used for future cabling work. Do not re-use dust caps which have been left loose in the
rack or computer room.

3.2.5 Fibre Channel adapters

We now review topics related to Fibre Channel adapters:

򐂰 Placement on the host system bus
򐂰 Distributing the load among several adapters
򐂰 Queue depth
򐂰 Driver selection

Host system bus

Today, there is a choice of high-speed adapters for connecting disk drives. Fast adapters can
provide better performance. The HBA must be placed in the fastest supported slot available.

Important: Do not place all the high-speed Host Bus Adapters (HBAs) on a single system
bus; otherwise, the computer bus becomes the performance bottleneck.

44 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

It is always a best practice to distribute high-speed adapters across several buses. When you
use PCI adapters, make sure that you first review your system specifications. Certain systems
include a PCI adapter placement guide.

The number of adapters you can install depends on the number of PCI slots available on your
server, but also on what traffic volume you expect on your SAN. The rationale behind multiple
adapters is either redundancy (failover) or load sharing.

Failover

When multiple adapters are installed on the host system and used with a multipath driver, the
multipath driver checks to see if all the available paths to the storage server are still
functioning. In the event of an HBA or cabling failure, the path is changed to the other HBA,
and the host continues to function without loss of data or functionality.

In general, all operating systems support two paths to the DS3500 Storage System. Microsoft
Windows 2003 and 2008 and Linux support up to four paths to the storage controller. AIX with
MPIO can support more than four paths to the controller modules however, too many paths
can delay failover when it actually is needed due to a controller failure. Care should be taken
to ensure that there are enough redundant paths without having too many. In general having
four paths provides good level of redundancy for load balancing and still allows for timely
controller failover when the need arises.

Load balancing

Load balancing or load sharing means distributing I/O requests from the hosts between
multiple adapters, which can be done by assigning LUNs to both the DS3500 controllers A
and B alternatively (see also 3.3.6, “Logical drives and controller ownership” on page 63).

Figure 3-10 shows the principle for a load-sharing setup. A multipath driver checks all
available paths to the controller. In Figure 3-10, that is two paths (red and blue). The driver
forces the data down all paths in a
check for the workload on a single path, but moves the data down in a
(round-robin).

round-robin scheme, which means that it does not really

rotational manner

Figure 3-10 Load sharing approach for multiple HBAs

The RDAC drivers for Linux supports round-robin load balancing.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 45

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Note: In a cluster environment, you need a single path to the each of the controllers (A and

B) of the DS3500 Storage System. However, if the cluster software and host application
can do persistent reservations, you can keep multiple paths and the multipath driver will
route the I/O request using the appropriate path to the reserved logical drive.

Queue depth

There are a number of locations in the SAN solution that can have queue depth settings
within their control. The queue depth is the maximum number of IO commands that can be
queued for processing at the system, SAN network or the storage at one point in time.

For the host this value might be adjustable at the device level and HBA level. Some host
operating systems only use the HBA level setting. Care should be taken when setting these
levels as response time performance can be impacted by too high a value, as well as

status being returned by the storage when its capabilities are exceeded.

busy

device

For QLogic based HBAs, the queue depth is known as
with either QLogic SANsurfer or in the BIOS of the QLogic-based HBA by pressing Ctl+Q
during the boot process.

For the storage there are values at the controller as well as drive level. These values can
varied between code levels and performance enhancement features.

For the latest firmware for the DS3500 controller, see the following Web site:

http://www.ibm.com/systems/storage/disk/

3.2.6 Disk expansion enclosures

The DS3500 Storage Systems offer the EXP3500 expansion enclosures for expanding the
subsystem beyond the internal drive count of the DS3500. There are two models of
expansions to chose from; the EXP3512 and the EXP3524. These expansions connect to the
DS3500 storage system through the 6 Gbps SAS expansion ports on the controller modules.
Figure 3-11 and Fig are examples of the cable connections needed to attach the DS3500 with
dual controllers to an EXP3500.

execution throttle, which can be set

Figure 3-11 DS3500 and EXP3500 cable attachments for dual controllers

46 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Figure 3-12 DS3500 and multiple EXP3500 cable attachments for dual controllers

In looking at Figure 3-12 it is important to notice the direction of the cabling scheme used
when adding the additional expansions to the loops. The DS3500 uses a “top down-bottom
up” cabling configuration. This aids in ensuring the availability of the arrays when they are
configured in the “zig-zag” pattern in the event of an enclosure being loss in the configuration.
See section 3.3.3, “Array configuration” on page 57 for further details on array layouts.

When planning for which enclosure to use with your DS3500, you must look at your
applications and workload type that you will need to provide. With transaction based
applications the workload will want to be able to perform a high number of IO so having the
greater number of spindles provided by the EXP3524 may prove to be a great advantage.
With throughput based applications the need is generally for higher capacity drives and more
bandwidth. With the new larger drives spindle count does not need to be high to meet this
goal. For this environment the EXP3512 can provide the disk capacity at a lower price point.
Careful planning can allow for the mixing of these two expansions in the storage system to
meet a mixed environment need.

The EXP3512 and EXP3524 enclosures support both SAS and SAS nearline 6 Gbps drives.
The EXP3512 drives are of a 3.5 inch footprint, while the EXP3524 drives are a 2.5 inch.

Important: There is no migration path of disks from the previous DS3000 products to the
new DS3500 as the 6 Gbps will not support the earlier 3 Gbps hardware.

Enclosure IDs

It is very important to correctly set the tray (enclosure) IDs. They are used to differentiate
multiple EXP enclosures that are connected to the same DS3500 Storage System. Each EXP
enclosure must use a unique value. The DS3500 Storage Manager (SM) uses the tray IDs to
identify each EXP enclosure.

For the EXP3500, the enclosure ID is indicated by a dual seven-segment LED located on the
back of each ESM next to the other ESM indicator lights. The storage server firmware
automatically sets the enclosure ID number. If needed, you can change the enclosure ID

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 47

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

setting through the DS3500 storage management software only. There are no switches on the
EXP3500 or EXP810 chassis to manually set the enclosure ID.

Figure 3-13 Enclosure ID LEDs for EXP3500

Enclosure guidelines

The base controller unit and each expansion enclosure has an ID number associated with it.
The ID allows each enclosure to be identified properly to the base controller unit.

Because the base and all enclosures are connected by Fibre Channel loop, it is necessary for
each ID address to be distinct and unique for I/O to flow properly. An ID address is composed
of two digits: a tens digit (x10) and a ones digit (x1). Enclosure IDs are typically between the
values of x00 and x77.

The EXP3500 is automatically assigned enclosure IDs.

Because the storage system follows a specific address assignment scheme for the drives,
you have to observe a few guidelines when assigning enclosure IDs to expansion units if you
were to assign these manually. Failure to adhere to these guidelines can cause issues with
I/O error recovery, and make the troubleshooting of certain drive communication issues more
difficult:

򐂰 Whenever possible, maintain an even number of expansion units on the DS3500 Storage

System and configure it for enclosure loss protection. Add EXP expansion units in pairs
and (ideally) for the DS3500 Storage System in groups of four.

򐂰 Add drives into an expansion enclosure in pairs.

3.3 Planning your storage structure

It is important to configure a storage system in accordance to the needs of the user workload.
An important question and primary concern for most users or storage administrators is how to
configure the storage subsystem to achieve the best performance. There is no simple answer
to fit all cases, and no single guideline for storage performance optimization that is valid in
every environment and for every particular situation. In this section we provide a general (and
less detailed) performance discussion.

Also, in this section, we discuss the use of RAID protection, and review other aspects of the
system configuration that can help optimize the storage capacity and resilience of the system.
In particular, we review array configuration, and enclosure loss protection.

48 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

3.3.1 Selecting drives

The speed and the type of the drives used will impact the performance. Typically, the faster
the drive, the higher the performance. This increase in performance comes at a cost; the
faster drives typically cost more than the lower performance drives.

The DS3500 Storage System currently supports the following types of SAS drives for the two
different models of chassis as shown below in Table 3-2.

Table 3-2 DS3500 families HDD support

Drives Supported DS3512/EXP3512 DS3524/EXP3524

SAS 15KRPM 300GB, 450GB,

600GB

SAS 15KRPM FDE 600GB None

SAS 10KRPM None 300GB

SAS 10KRPM FDE None 300GB

Nearline SAS

7.5KRPM

Maximum drives 12/96 24/96

Storage system
capacity (max)

1TB, 2TB 500GB

450 GB SAS / 1 TB
SATA

73GB, 146GB

450 GB SAS / 1 TB
SATA

The Full Disk Encryption (FDE) drives; are drives with built-in disk encryption hardware that
prevents unauthorized access to the data on a drive that is physically removed from the
storage subsystem.

Best practice: Generally it is best to use the fastest drives available for best performance.
This can be critical to transaction based (high IOPS) workloads.

The speed of the drive is measured by the number or revolutions per minute (RPM). A 15K
drive rotates 15,000 times per minute. With higher speeds, the drives tend to be denser,
because a large diameter plate driving at such speeds is likely to wobble. With the faster
speeds, greater throughput is possible.

Seek time is the measure of how long it takes for the drive head to move to the correct sectors
on the drive to either read or write data. It is measured in thousands of a second (milliseconds
or ms). The faster the seek time, the quicker data can be read from or written to the drive. The
average seek time reduces when the speed of the drive increases. Typically, a 7.2K drive will
have an average seek time of around 9 ms, a 10K drive will have an average seek time of
around 5.5 ms, and a 15K drive will have an average seek time of around 3.5 ms.

Command queuing (or queue depth) allows for multiple commands to be outstanding to the
disk drive at the same time. The drives have a queue where outstanding commands can be
dynamically rescheduled or re-ordered, along with the necessary tracking mechanisms for
outstanding and completed portions of workload. The DS3500 provides a drive command
queue depth of four operations per disk. The
depth for all drives to 16.

Avoid using the SAS nearline drives for high IOPS operations. SAS nearline can, however, be
used for streaming and archiving applications. These are both very good uses for the slower
RPM drives, where high throughput rates are required, at a lower cost. If properly configured,

High Performance Tier increases the queue

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 49

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

these drives can push very high throughput numbers with large host IO blocksizes and
sequential workloads.

3.3.2 Understanding RAID types

In this section we introduce arrays, logical drives and associated terminology, and then
describe the various RAID levels that are supported by the IBM System Storage DS3500
storage subsystem. RAID is an acronym for Redundant Array of Independent Disks, and is a
storage solution in which part of the total storage capacity is used to store redundant
information about user data stored on the remainder of the storage capacity.

RAID relies on a series of configurations, called levels, to determine how user data and
redundancy data are written to and retrieved from the drives. RAID Level 1, RAID Level 10,
RAID Level 3, RAID Level 5, and RAID Level 6 write redundancy data to the drive media for
fault tolerance. The redundancy data might be an exact copy of the data (mirrored) or an error
correcting code derived from the data. If a drive fails, you can use the redundancy data to
quickly reconstruct information on a replacement drive.

DS3500 arrays and RAID levels

An array is a set of drives that the system logically groups together to provide one or more
logical drives to an application host or cluster. The DS3500 storage subsystem supports
RAID levels 0, 1, 10, 3, 5 and 6. Each of these RAID levels offers a different compromise
among capacity, performance and data redundancy. The attributes of each of these RAID
levels is described in more detail over the following pages of this book.

Note that the maximum number of physical drives in a RAID 0, 1 or 10 array is limited by the
maximum number of physical drives that can be installed in a fully populated DS3500 storage
subsystem, which is 96 drives. The maximum number of physical drives in a RAID 3, 5 or 6
array is always 30 drives.

The DS3500 storage subsystem is able to dynamically change the RAID level without
requiring downtime. This feature is called Dynamic RAID Migration (DRM).

Each RAID array contains one or more associated logical drives. A logical drive is the basic
structure that you create to store data on the storage subsystem. Each logical drives appears
as a separate physical drive to the operating system on the host server. The DS3500 storage
subsystem supports a maximum of 256 logical drives for the entire storage subsystem.

We now briefly describe the main features of each of the various RAID levels that are
supported by the DS3500 storage subsystem.

RAID 0: Data striping

RAID 0 (Figure 3-14 on page 51) is also known as data striping. In this RAID level, the data is
striped sequentially across all participating physical drives. RAID Level 0 is only designed to
increase performance and has no data redundancy. The DS3500 storage subsystem
supports a minimum of one drive and a maximum of 96 drives in a RAID 0 array.

RAID 0 is well-suited for applications that require fast access to non-critical data, or data that
can be easily restored from backup. Increasing the number of disk drives in the array will
increase the data access performance of the array.

50 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Attention: The failure of a single disk in a RAID 0 array will cause the failure of the entire
array and all of the associated logical drives, and access to all data on the array will be lost.
For this reason, the best practise is to never use RAID Level 0 for critical applications that
require high availability.

Figure 3-14 RAID 0

RAID 1 and RAID 10: Disk mirroring and disk mirroring with striping

RAID 1 is also known as disk mirroring and is a mirrored pair of drives without parity. RAID
Level 1 uses exactly two drives to mirror the data between them. A RAID 10 (Figure 3-15 on
page 52) array is automatically created when you create a RAID 1 array with four or more
drives (two pairs of drives). RAID 10 is also known as RAID 1+0 and disk mirroring with
striping, and it implements block interleave data striping and mirroring. In RAID 10, data is
striped across the physical disk drives, and each of those drives is then mirrored to a second
drive. The DS3500 storage subsystem supports a maximum of 96 drives in a RAID 10 array.

Note: RAID Level 1 is a specific implementation of RAID Level 10 that uses exactly two
drives to mirror the data between them. A RAID Level 10 array is automatically created
when you select four or more drives in a RAID 1 array.

RAID Levels 1/10 provide good redundancy; in the case of a single disk failure in each
mirrored pair, the array and associated logical drives become degraded but all the data is still
available and accessible from the second drive of the mirrored pair.

For each pair of mirrored drives, read operations can be performed from either physical disk
of the mirrored pair. Write operations are performed by writing to both physical disks of the
mirrored pair. In this manner small blocksize writes can be completed very quickly, making
this RAID type a great solution for a high write intensive application when data protection is
desired. For example, this RAID type is generally preferred by database administrators.

However, because the data is mirrored, the capacity of the associated logical drives on a
RAID 1 or RAID 10 array is 50% of the physical capacity of the hard disk drives in the array.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 51

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 3-15 RAID 10. A Raid 1 array will only consist of one mirrored pair (#1 Mirrorset)

We recommend these best practises for using RAID 1/10 arrays:

򐂰 Use a two drive RAID 1 array for the disks that contain your operating system. It is a good

choice, because the operating system can usually fit on one disk.

򐂰 Use RAID 1 for transaction logs. Typically, the database server transaction log can fit on

one disk drive. In addition, the transaction log performs mostly sequential writes. Only
rollback operations cause reads from the transaction logs. Therefore, we can achieve a
high rate of performance by isolating the transaction log on its own RAID 1 array.

򐂰 Use write caching on RAID 1/10 arrays. Because a RAID 1/10 write will not complete until

both writes have been completed on both disks in a mirrored pair, the performance of
writes can be improved through the use of a write cache. Always ensure that you use
battery-backed write cache.

򐂰 The performance of RAID 10 is comparable to RAID 0 for sequential I/Os but RAID 10

provides data redundancy through disk mirroring.

Note: There are no guaranteed choices as to which type of RAID to use, because this is
very much dependent on the workload read and write activity. A good general guide might
be to consider using RAID 1 if random writes exceed about 25%, with a peak sustained I/O
rate that exceeds 50% of the storage subsystem’s capacity

When comparing RAID Level 10 with RAID Level 5:
򐂰 RAID 10 uses two write IOs to write a single block of data (one write IO to each drive in the

mirrored pair). RAID 5 requires two read IOs (read original data and parity) and then two
write IOs to write the same block of data. For this reason, random writes are significantly
faster on RAID 10 compared to RAID 5.

򐂰 RAID 10 rebuilds take less time than RAID 5 rebuilds. If one drive fails, RAID 10 rebuilds it

by copying all the data on the mirrored drive to a replacement/hotspare drive. RAID 5
rebuilds a failed disk by merging the contents of the surviving disks in an array and writing
the result to a spare.

52 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

RAID 3: Data striping with a dedicated parity drive

A RAID 3 array uses data striping with a dedicated parity drive. Similar to RAID 0 data
striping, information written to disk is split into chunks (a fixed amount of data), and each
chunk is written out to the same physical position on separate disks (in parallel). This
architecture requires parity information to be written for each stripe of data; RAID 3 uses a
dedicated physical drive for storing parity data. If any one disk drive in the array fails, the array
and associated logical drives become degraded, but all data is still accessible by the host
application.

However, with RAID 3, the dedicated parity drive is a performance bottleneck during writes.
Because each write operation requires the parity to be re-computed and updated, this means
that the parity drive is accessed every time a block of data is written to the array. Because of
this, RAID 3 is rarely used today in the industry and RAID 5 has taken it’s place. The DS3500
storage subsystem supports a maximum of 30 drives in a RAID 3 array.

RAID 5: Data striping with distributed parity

Like RAID Level 3, RAID Level 5 also uses parity for data protection but unlike RAID 3 it does
not use a dedicated parity drive. Instead, the parity blocks are evenly distributed across all
physical disk drives in the array, as shown in Figure 3-16. The failure of a single physical drive
in a RAID 5 array will cause the array and associated logical drives to be degraded, but all the
data will remain accessible to the host application. This level of data redundancy is known as
n+1 redundancy because the data remains accessible after a single drive failure. When you
create a RAID 5 array, the capacity of the array is reduced by the equivalent capacity of one
drive (for parity storage). The DS3500 storage subsystem requires a minimum of three and
supports a maximum of 30 drives in a RAID 5 array.

Figure 3-16 RAID 5

RAID Level 5 is best used in environments requiring high availability and fewer writes than
reads.

RAID Level 5 can be good for multi-user environments, such as database or file system
storage where the typical I/O size is small and there is a high proportion of read activity.
Applications with a low read percentage and a high random write percentage may not perform

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 53

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

14817007

12816016

10815025

8814034

6813133

4612122

2411111

Y: T o tal #
of IOs for
RAID 1 ( =
Ax2 )

X: Total #
of IOs for
RAID 5 ( =
B+C+D+E )

E: T ot al #
of RAID 5
Write
Parity IOs

D: Total #
of RAID 5
Write Data
IOs

C: Total #
of RAID 5
Read Parity
IOs

B: Total #
of RAID 5
Read Data
IOs

A: Total #
of dri ves
writte n

as well on RAID 5 logical drives because parity data must be recalculated for each write
operation and then written to each drive in the array.

Use write caching on RAID 5 arrays, because each RAID 5 write will not be completed until at
least two read IOs (one data one parity) and two write IOs (one data and one parity) have
occurred. The write performance penalty may be mitigated by using battery-backed write
cache. RAID 5 arrays with caching can give as good as performance as any other RAID level,
and with some workloads, the striping effect can provide better performance than RAID 1/10.
Applications that require high throughput sequential write I/Os are an example of one such
workload. In this situation, a RAID Level 5 array can be configured to perform just one
additional parity write when using “full stripe writes” (also known as “full stride writes”) to
perform a large write I/O when compared to the two writes per data drive (self, and its mirror)
that are needed for each write I/O with a RAID 1 array.

You must configure the RAID 5 array with certain number of physical drives in order to take
advantage of full stripe writes. This is illustrated for the case of a RAID 5 array with 8 total
drives (7 data + 1 parity) in the Figure 3-17.

Figure 3-17 Table illustrating the potential performance advantages of RAID 5 full stripe writes

Column A lists the number of drives that are being written to. Column Y is the number of write
I/Os for a RAID 1 and will always be twice the value of A. Columns B,C,D,E contain the
numbers of read data/parity and write data/parity I/Os required for the number of drives that
are being written to. You can see that for seven drives, no read I/Os are required for RAID 5
arrays because the full stripe is being written at once. This substantially reduces the total
number of I/Os (column X) required for each write operation.

The decrease in the overhead read operations with the full stripe write operation is the
advantage you are looking for. You must be very careful when implementing this type of layout
to ensure that your data pattern does not change, and decrease its effectiveness. However,
this layout might work well for you in a large sequential write environment. Due to the small
size of segments, reads might suffer, so mixed I/O environments might not fare well, which
might be worth testing if your writes are high.

When the IBM DS system storage detects that it is receiving contiguous full stripe writes it will
switch internally to an even faster write capability known as Fast Stripe Write Through. In this
method of writing the DS system storage uses the disk as the mirror device for the cache
write and shortens the write process. This method of writing can increase throughput as
much as 30% on the system storage.

This does however, require that the following rules are being met by the IO pattern:

54 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

򐂰 All write IO's are full stripe writes (no partial stripe writes can be requested)
򐂰 Write IO's are and sequential and contiguous in nature so no seeks are required.

If any interruptions in this pattern are detected the writes will revert back to the standard full
stripe write model which still gives a benefit to the RAID5 for large sequential writes over the
RAID1.

RAID 6: Data striping with dual distributed parity

RAID 6 (see Figure 3-18) is a RAID level with dual rotational parity that is distributed across
the drives in the array. A RAID 6 array has n+2 redundancy, which means that data remains
accessible to the host application after two concurrent disk drives failures in the array. RAID 6
achieves n+2 redundancy because it calculates two sets of parity information for each block
of data (P+Q) that is striped across the disks. The DS3500 storage subsystem performs the
P+Q parity calculations in hardware.

There is no performance penalty for read operations from a RAID 6 array, but there is a
performance penalty for write operations because two sets of parity information (P+Q) must
be calculated for each write operation. The write penalty in RAID Level 6 may be mitigated by
using battery-backed write caching.

The DS3500 storage subsystem requires a minimum of five drives and supports a maximum
of 30 drives in a RAID Level 6 array.

Figure 3-18 RAID 6

The general characteristics of a RAID Level 6 array are:

򐂰 It uses disk striping with dual distributed parity: two independent parity blocks per stripe

(P+Q) that are calculated in hardware for each stripe of data.

򐂰 It has very high data (n+2) redundancy: the array and associated logical drives can survive

the simultaneous loss of two physical disks without losing data.

򐂰 It requires two sets of parity data for each write operation, resulting in a significant

decrease in write performance. Using battery-backed write cache may mitigate the impact
of this write penalty.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 55

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

򐂰 It has additional costs because of the extra disk capacity required by using two parity

blocks per stripe: compared to a RAID 5 array, additional drives are required to achieve
that same usable logical drive capacity.

򐂰 Any application that has high read request rates and average write request rates such as

Transaction servers, Web servers, data mining applications, and Exchange servers will
benefit from RAID 6.

RAID Levels summary

In this section we summarize the general characteristics of the various RAID levels supported
by the DS3500 storage subsystem. The following note and Table 3-3 summarize this
information.

Summary: The general performance characteristics of the various RAID levels are:

򐂰 RAID 0 offers high performance, but does not provide any data redundancy.
򐂰 RAID 1/10 offers high performance for write-intensive applications.
򐂰 RAID 3 is good for large data transfers in applications, such as multimedia or medical

imaging, that write and read large sequential chunks of data.

򐂰 RAID 5 is good for multi-user environments, such as database or file system storage,

where the typical I/O size is small, and there is a high proportion of read activity.

򐂰 RAID 6 offers high availability with performance slightly lower than RAID 5.

Table 3-3 RAID levels comparison

RAID Description Application Advantage Disadvantage

0 Stripes data across

multiple drives.

1/10 The drive data is

mirrored to another
drive.

3 Drives operate

independently with
data blocks
distributed among all
drives. Parity is
written to a dedicated
drive.

5 Drives operate

independently with
data and parity blocks
distributed across all
drives in the group.

IOPS
Mbps

IOPS Performance, as

Mbps High performance for

IOPS
Mbps

Performance, due to
parallel operation of the
access.

multiple requests can
be fulfilled
simultaneously.

large, sequentially
accessed files (image,
video, and graphics).

Good for reads, small
IOPS, many concurrent
IOPS, and random I/Os.

No redundancy. If one
drive fails, the data is
lost.

Storage costs are
doubled.

Degraded
performance with 8-9
I/O threads, random
IOPS, and smaller,
more numerous
IOPS.

Writes are particularly
demanding.

56 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

RAID Description Application Advantage Disadvantage

6 Stripes blocks of data

and parity across an
array of drives and
calculates two sets of
parity information for
each block of data.

IOPS
Mbps

Good for multi-user
environments, such as
database or file system
storage, where typical
I/O size is small, and in
situations where
additional fault
tolerance is required.
This is the most reliable
RAID level on the
DS5000 storage
subsystem

Slower in writing data,
complex RAID
controller
architecture.

RAID Levels reliability considerations

At first glance, both RAID 3 and RAID 5 appear to provide good protection against drive
failure. With today’s high-reliability drives, it appears unlikely that a second drive in an array
will fail (causing data loss) before an initial failed drive can be replaced. But if you look at
RAID 6 and calculate the possibility of data loss, the chance to loose data is theoretically
much less than on RAID 3 and RAID 5.

However, field experience has shown that when a RAID 3 or RAID 5 array fails, it is not
usually due to two drives in the array experiencing complete failure. Instead, most failures are
caused by one drive going bad, and a single block somewhere else in the array that cannot be
read reliably.

This problem is exacerbated by using large arrays with RAID 5. This
data loss when the information to rebuild the stripe is not available. The end effect of this
issue will of course depend on the type of data and how sensitive it is to corruption. While
most storage subsystems (including the DS3500 storage subsystem) have mechanisms in
place to try to prevent this from happening, they cannot work 100% of the time.

Any selection of RAID type should take into account the cost of downtime. Simple
calculations tell us that RAID 3 and RAID 5 are going to suffer from failures more often than
RAID 10. (Exactly how often is subject to many variables and is beyond the scope of this
book.) The money saved by economizing on drives can be easily overwhelmed by the
business cost of a crucial application going down until it can be restored from backup.

No data protection method is 100% reliable, and even if RAID were faultless, it will not protect
your data from accidental corruption or deletion by program error or operator error. Therefore,
all crucial data should be backed up by the appropriate software, according to business
needs.

3.3.3 Array configuration

Before you can start using the physical disk space, you must configure it. You divide your
(physical) disk drives into arrays and create one or more logical drives inside each array.

In simple configurations, you can use all of your drive capacity with just one array and create
all of your logical drives in that particular array. However, the following drawbacks exist:

stripe kill can lead to

򐂰 If you experience a (physical) drive failure, the rebuild process affects all logical drives,

and the overall system performance goes down.

򐂰 Read/write operations to various logical drives are still being made to the same set of

physical hard drives.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 57

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

The array configuration is crucial to performance. You must take into account all the logical
drives inside the array, because all logical drives inside the array will impact the same
physical disks. If you have two logical drives inside an array and they both are high
throughput, then there can be contention for access to the physical drives as large read or
write requests are serviced. It is crucial to know the type of data that each logical drive is used
for and try to balance the load so contention for the physical drives is minimized. Contention is
impossible to eliminate unless the array only contains one logical drive.

Number of drives

In the transaction intense environment, it is more important to ensure that there are enough
disk drives configured to perform the I/Os demanded by the host application, than to focus on
the amount of possible storage space on the storage subsystem.

Use Table 3-2 on page 49 to determine what drive type, speed and capacity is best suited for
your workload and environment needs.

Transaction intensive workload

In a transaction intense environment, you want to have higher drive numbers involved.
This can be done by creating larger arrays with more disks. The storage subsystems can
have a maximum of 30 drives per RAID 5 array/logical drive and 96 drives for a full subsystem
RAID 10 array.

With a RAID 10 array, the logical drive size can be configured to encompass the entire array,
although operating system limitations on maximum logical drive size might restrict the
usefulness of this capability. Though there are circumstances where this model can work well,
it is best that smaller arrays be used to better balance the drive usage and access patterns
across the controllers and to avoid contention and intermixed IO patterns.

Configuring multiple LUNs striped across a single large array can be used to make use of all
the capacity. However, with this layout, consideration should be given to the workload types
for which these LUNs will be used, so as not to mix throughput and transaction based IO on
the same array.

Another factor to consider is congestion when accessing the drives on the back-end loops.
This situation can be avoided by using multiple arrays.

Generally, an array of 8 to 16 disks provides the best performance for RAID 5 workloads that
are OLTP based.

Best practice: For high transaction environments requiring highest redundancy and
protection, logical drives should be built on arrays with 8 to 12 disks when using RAID 5. or
RAID 10. Spreading the arrays evenly across the two controllers in a dual controller
environment will give you the best balance of the workload.

For large size databases, consider using the host volume management software to spread
the workload evenly across multiple arrays/LUNs to evenly balance the workload on all. Build
the volume across sets of logical drives laid out per the RAID type in the previous discussion.
In using multiple arrays, you will also be able to increase the controllers which are involved in
handling the load, therefore getting full use of the storage subsystems resources.

For example: If needing to build a database that is 1 TB in size, you can use five 300 GB
drives in a 4+1 parity RAID 5 single array/logical drive; or you can create two RAID 5 arrays of
8+1 parity using 73 GB drives, giving two 584 GB logical drives on which to build the 1 TB
database. In most cases the second method for large databases will work best, as it brings
twelve more disks into play for handling the high host application transaction workload.

58 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Large throughput workload

In the large throughput environment, it typically does not take high numbers of disks to reach
the maximum sustained throughput. Considering that this type of workload is usually made of
sequential I/O, which reduces disk latency, in most cases about 20 to 28 drives are enough to
reach the maximum throughput.

This does, however, require that the drives be spread evenly across the DS3500 to best utilize
the system bandwidth. The storage subsystem is optimized in its firmware to give increased
throughput when the load is spread across all parts. Here, bringing all the DS3500 resources
into play is extremely important. Keeping the drives loops, and bus busy with high data
throughput is the winning answer, which is also a perfect model for using the high capacity
drives, as we are looking to push a large volume of data and it will likely be large blocks of
sequential reads and writes.

Consider building smaller arrays which are 4+P or 8+P in size with single logical drives for
higher combined throughput. If multiple logical drives are to be created on the array it is best
to not exceed the number of data drives in the array. The higher the number of logical drives
the greater the chance for contention for the drives.

Best practice: For high throughput, logical drives should be built on arrays with 4+1, or
8+1 drives in them when using RAID 5. Data drive number and
host I/O blocksize for full stripe write. Use multiple logical drives on separate arrays for
maximum throughput.

segment size must equal

An example configuration for this environment is to have a single logical drive /array with 16+1
parity 300 GB disks doing all the transfers through one single path and controller; An
alternative consists of two 8+1 parity defined to the two controllers using separate paths,
doing two separate streams of heavy throughput in parallel and filling all the channels and
resources at the same time, which keeps the whole server busy with a cost of one additional
drive.

Further improvements can be gained by splitting the two 8+1 parity into four 4+1 parity arrays
giving four streams, but the addition of three drives is needed. A main consideration here is to
plan for the array data drive count to be a number such that the host I/O blocksize can be
evenly spread using one of the storage subsystem’s segment size selections, which will
enable the full stripe write capability discussed in the next section.

Regardless of the workload type or the RAID type being used for the array group, in all cases
building the array using as equal a number of odd and even drive slots is advantageous to the
performance of the array and it's LUNs. This is frequently done by using a "diagonal" or
"orthogonal" layout across all the expansions used to attain enclosure loss protection as well
(shown in Figure 3-20 on page 62).

3.3.4 Hot Spare drives

A hot spare drive is like a replacement drive installed in advance. Hot spare disk drives
provide additional protection that might prove to be essential in case of a disk drive failure in a
fault tolerant array.

When possible, split the hot spares so that they are in separate enclosures and are not on the
same drive loops (see Figure 3-19 on page 60).

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 59

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Attention: SAS nearline drives cannot be used as hot spares for the SAS 10KRPM or SAS

15KRPM drives. When intermixing SAS and SAS nearline you will want to create hot
spares for both types of drives.

Best practice: When assigning disks as hot spares, make sure that they have enough
storage capacity. If the failed disk drive is larger than the hot spare, reconstruction is not
possible. A best practice is to use the largest drive size for the drive type installed as hot
spare drives. See 10.3, “Set hot spare drive” on page 268 for more details on defining hot
spare drives.

Figure 3-19 Hot spare coverage with alternating loops

Planning for hot spares

Even though disk drives are becoming increasingly more reliable, disk failures can occur. To
protect your data in case of a drive failure, you should primarily use a redundant RAID level,
for example, RAID 1, 5, or 10. This way, your data will remain intact should a hard drive fail.
However, the array will then operate in a degraded state until the failed disk drive is replaced
and data reconstruction completes on the new disk drive.

To ensure as high availability as possible, we strongly recommend that you use hot spare
drives. Under normal circumstances, the hot spare drives do not participate in any array.
When one of the disk drives fails, the host spare will automatically be used as a replacement
drive. Data reconstruction onto the hot spare will take place immediately. When
reconstruction finishes, the array will again be in a fully operational status and the hot spare
will become a member of the array.

Depending on your configuration, it might be wise to use more than just one hot spare. In
small DS3500 configurations with a relatively low number of disk drives, one hot spare might
suffice. But for larger configurations, we recommend that you define several hot spares.

60 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Best practice: For best practice configuration it is recommended that you have one
hotspare for every 18 - 20 disk drives in the system.

When defining hot spare drives, you need to consider the following rules:

򐂰 Hot spares must be of the same drive type (SAS or SAS nearline).
򐂰 Hot spares must be of the same or larger size as the failed drive.

For example, if you use a mixture of different disk drive sizes in your arrays, the hot spare
drive size must be large enough so that it can effectively replace any of the drives of the same
type in the system that it is protecting. This also means that in a mixed drive type environment
you would need to have a minimum of two hot spares, one SAS and one SAS nearline.

The following methods are available to allocate hot spare drives in the storage subsystem:
򐂰 Automatic assignment: The storage subsystem automatically calculates the number of hot

spare drives needed and allocates accordingly. This can be used on unconfigured storage
subsystems.

򐂰 Explicit assignment: The hot spare drives are manually selected and assigned.

Use the IBM DS Storage Manager to configure the above options.

3.3.5 Enclosure loss protection planning

Enclosure loss protection will enable your system to be more resilient against hardware
failures. Enclosure loss protection means that you spread your protection arrays across
multiple enclosures rather than in one enclosure so that a failure of a single enclosure does
not take an array offline.

By default, the automatic configuration is enabled. However, this is not a best practice for the
method of creating arrays. Instead, we use the manual method, because this allows for more
configuration options to be available at creation time.

Best practice: Manual array configuration allows for greater control over the creation of
arrays.

Figure 3-20 on page 62 shows an example of the enclosure loss protection. If enclosure
number 2 fails, the array with the enclosure loss protection can still function (in a degraded
state), because the other drives are not affected by the failure.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 61

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Figure 3-20 Example of enclosure loss protection

In the example shown in Figure 3-21, if enclosure number 1 fails, the entire array becomes
inaccessible.

Figure 3-21 An array without enclosure loss protection

Best practice: When possible, plan to use enclosure loss protection for your arrays.

62 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

3.3.6 Logical drives and controller ownership

Logical drives, sometimes simply referred to as logical volumes or Logical Unit Numbers
(LUNs), are the logical segmentation of arrays. A logical drive or LUN is a logical structure
that you create on a storage subsystem for data storage. A logical drive is defined over a set
of drives called an array group which has a defined RAID level and capacity. The attributes of
the array are hidden from the host computer which is only presented with logical drive details
including capacity.

The DS3500 provides great flexibility in terms of configuring arrays and logical drives.
However, when assigning logical drives to the systems, it is very important to remember that
the DS3500 uses a preferred controller ownership approach for communicating with logical
drives, which means that every logical drive is owned by only one controller at a time. It is,
therefore, important at the system level to make sure that traffic is correctly balanced between
controllers with a dual controller DS3500 system, which is a fundamental principle for a
correct setup of the storage system.

Balancing traffic

Balancing traffic is unfortunately not always a trivial task. For example, if an application
requires a large disk space to be located and accessed in one chunk, it becomes harder to
balance traffic by spreading the smaller volumes among controllers. This can sometimes be
aided through the use of host based volume management software products.

In addition, typically, the load across controllers and logical drives is constantly changing. The
logical drives and data accessed at any given time depend on which applications and users
are active during that time period, hence the importance of monitoring the system.

Best practice: Here are guidelines for logical drive assignment and storage partitioning:

򐂰 Assign logical drives across both controllers to balance controller utilization.
򐂰 Use the manual method of creating logical drives, which allows greater flexibility for

configuration settings, such as enclosure loss protection and utilizing all drive loops.

򐂰 Avoid mixing different workload types (transaction based and throughput based) on the

same array of disks.

򐂰 Always leave a small amount of free space in the array after the logical drives have

been created.

Enhanced remote mirror (ERM) considerations

A secondary logical drive in a remote mirror will always have the same preferred owner as the
associated primary logical drive. For example, if controller A owns the primary logical drive in
the primary storage subsystem, controller A owns the associated secondary logical drive in
the secondary storage subsystem. If controller ownership changes on the primary logical
drive, then this will cause a corresponding controller ownership change of the secondary
logical drive also. For greater details on ERM see the IBM Midrange System Storage Copy
Services Guide, SG24-7822.

3.3.7 Storage partitioning

Storage partitioning adds a high level of flexibility to the DS3500 Storage System. The
DS3500 comes with four partitions by default, which can be expanded to a maximum of 64.
See 3.4, “Planning for premium features” on page 70 on for discussion on premium feature

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 63

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

needs. It enables you to connect multiple and heterogeneous host systems to the same
storage server, either in stand-alone or clustered mode. The term storage partitioning is
somewhat misleading, because it actually represents a host or a group of hosts and the
logical drives they access.

Without storage partitioning, the logical drives configured on a DS3500 Storage System can
only be accessed by a single host system or by a single cluster, which can lead to inefficient
use of storage server hardware unless the use of the DS3500 Storage System is dedicated to
a single host (for example, SVC attachment, where it is seen as a single host).

Storage partitioning, on the other hand, allows the creation of “sets”, containing the hosts with
their host bus adapters and the logical drives. We call these sets
systems can only access their assigned logical drives, just as though these logical drives
were locally attached to them. Storage partitioning adapts the SAN idea of globally accessible
storage to the local-storage-minded operating systems.

Storage partitioning allows mapping and masks the logical drive or LUN (that is why it is also
referred to as “LUN masking”), which means that after the logical drive is assigned to a host,
it is hidden from all other hosts connected to the same storage server. Therefore, access to
that logical drive is exclusively reserved for that host.

It is a good practice to configure storage partitioning prior to connecting multiple hosts.
Operating systems such as Windows will write their signatures to any device it can access.

storage partitions. The host

Heterogeneous host support means that the host systems can run various operating
systems. But be aware that all host systems within a particular storage partition have
unlimited access to all logical drives assigned to the partition. Therefore, file systems or disk
structure on these logical drives must be compatible with host systems. To ensure this, it is
best to run the same operating system on all hosts within the same partition. In certain
special cases it may be needed to run operating systems which will be able to mount foreign
file systems and share logical drives. In these cases care should be taken to ensure the
partition members are properly defined and configured to share.

Storage partition topology is a collection of topological elements (default group, host groups,
hosts, and host ports) shown as nodes in the topology view of the mappings view. To map a
logical drive or LUN to a specific host server or group of hosts, each component of the
storage partition must be defined.

A storage partition contains several components:

򐂰 Host groups
򐂰 Hosts
򐂰 Host ports
򐂰 Logical drive mappings

A

host group is a collection of hosts that are allowed to access certain logical drives, for

example, a cluster of two systems.

A

host is a single system that can be mapped to a logical drive.

A

host port is the FC port of the host bus adapter (HBA) on the host system. The host port is

identified by its world-wide name (WWN). A single host can contain more than one host port.
If the servers are attached using full redundancy, each server will have two host bus adapters,
that is, it needs two host ports within the same host system. It is possible to have a host with
a single HBA, but for redundancy, it must be able to access both DS3500 controllers, which
can be achieved by SAN zoning.

64 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

The DS3500 Storage System only communicates through the use of the WWN. The storage
system is not aware of which host bus adapters are in the same server or in servers that have
a certain relationship, such as a cluster. The host groups, the hosts, and their host ports
reflect a logical view of the physical connections of the SAN, as well as the logical connection
between servers, such as clusters.

With the logical setup defined as previously described, mappings are specific assignments of
logical drives to particular host groups or hosts.

The storage partition is the combination of all these components. It ensures correct access to
the various logical drives even if there are several hosts or clusters connected.

The default host group is a placeholder for hosts that are defined but have not been mapped.
The default host group is also normally used only when storage partitioning is not enabled. If
this is the case, then only one type of operating system must be sharing the logical drives.

Every unassigned logical drive is mapped to the undefined mappings group, which means
that no host (or host port, to be precise) can access these logical drives until they are
mapped.

With Storage Manager, it is possible to have up to 64 storage partitions on a DS3500, which
allows the storage subsystem to provide storage capacity to a number of different
heterogeneous hosts, allowing for greater flexibility and scalability. Note that on DS3500
models, the number of partitions also depends on the premium feature licence that has been
purchased.

For the maximum number of logical drives per partitions for a specific host type see Table 3-4.

Table 3-4 Host logical drive restrictions

Operating system Maximum number of logical drives supported

per partition

Windows Server 2003 32

Windows Server 2008 32

HP-UX 32

AIX 32

Linux 32

Every mapping of a logical drive to a new host or host group creates a new storage partition.
If additional logical drives are required for an existing host or host group, a new storage
partition is not required. For example, a cluster with two nodes with redundant I/O paths gets
configured as one host group with two hosts. Each host then has two host ports for
redundancy, and several logical drives are mapped to this host group. All these components
represent one storage partition. If another single host system is attached to the same storage
system and other logical drives are mapped to that host, another storage partition then has to
be created for it. If a new logical drive is created and mapped to either the cluster or the single
host, it uses an existing storage partition.

Note: There are limitations as to how many logical drives you can map per host. DS3500
series storage servers will support up to 256 logical drives (including the “access” logical
drive) per partition (although there are also restrictions, depending on the host operating
system) and a maximum of two partitions per host. Keep these limitations in mind when
planning the DS3500 Storage System installation.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 65

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Storage partitioning considerations

By default an “access” logical drive is created mapped to each partition created. If in-band
management is being used, then the access logical drive must be mapped to the storage
partition for the managing host. For all others this logical drive can be removed.

In a security-sensitive environment, you can assign a password to the storage system as well.

Note: Each host with separately assigned storage will use a storage partition. Each host
group with separately assigned storage to be shared by the group will also use a storage
partition.

In order to configure the storage partitioning correctly, you need the WWN of your host HBAs.
Mapping is done on a WWN basis. Depending on your HBA, you can obtain the WWN either
from the BIOS or QLogic SANsurfer tool if you have QLogic cards. Emulex adapters and IBM
adapters for IBM System p and IBM System i® servers have a sticker on the back of the card.
The WWN is also usually printed on the adapter itself or the box that the adapter was shipped
in.

If you are connected to a hub or switch, check the Name Server Table of the hub or switch to
identify the WWN of the HBAs.

When planning your partitioning, keep in mind that:

򐂰 By default the DS3500 comes with four partitions
򐂰 In a cluster environment, you need to use host groups.
򐂰 You can optionally purchase additional partitions (maximum of 64).

When planning for your storage partitioning, create a table of planned partitions and groups
so that you can clearly map out and define your environment.

Best practice: If you have a single server in a host group that has one or more LUNs
assigned to it, do the mapping to the host and not the host group. All servers with the same
host type (for example, Windows servers) can be in the same group if you want, but by
mapping the storage at the host level, you can define what specific server accesses which
specific logical drives.

However, if you have a host cluster, you will need to assign the shared logical drives at a
host group level, so that all of the host servers in that host group have access to these
logical drives for host cluster failover to work. This practice is also used when creating a
partition for shared filesystem servers (GPFS™ for example).

Table 3-5 shows an example of a storage partitioning plan, which clearly shows the host
groups, hosts, port names, WWN of the ports, and the operating systems used in that
environment. Other columns can be added to the table for future references, such as HBA
BIOS levels, driver revisions, and switch ports used, all of which can then form the basis of a
change control log.

Table 3-5 Sample plan for storage partitioning

Host group Host name Port name WWN OS type

Windows 2003 Windows Host MailAdp_A 200000E08B28773C Windows 2003

66 IBM System Storage DS3500: Introduction and Implementation Guide

MailAdp_B 200000E08B08773C

Non-Clustered

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Linux Linux_Host LinAdp_A 200100E08B27986D Linux

LinAdp_B 200000E08B07986D

POWER6® AIX_Host AIXAdp_A 20000000C926B6D2 AIX

AIXAdp_B 20000000C926B08

Heterogeneous hosts

When implementing a DS3500 Storage System solution, a mixture of host servers with
various operating systems can be supported, and both clustered and non-clustered variants
of the same operating systems. In general, all host accessing logical drives in a single storage
partition would want to be configured for the same operating system. Very seldom would this
not be the case, and great care must be taken when creating those environments.

Important: Non-shared storage can only be supported with storage partitioning enabled.

Delete the access logical drive (31)

The DS3500 Storage System will automatically create a logical drive for each host attached
(logical drive id 31). This drive is used for in-band management, so if you do not plan to
manage the DS3500 Storage System from that host, you should delete this logical drive. This
will also give you one more logical drive to use for host user data.

Important: In all cases when not using in-band management you should delete the access
logical driver for the host partition.

3.3.8 Segment size

The segment size is the maximum amount of data that is written or read from a disk per
operation before the next disk in the array is used. Segment size can be very different for
different workload types. It is important to know the host IO blocksize that will be used when
deciding on the segment size that will be set.

For small host I/Os, as mentioned earlier, set the segment size larger than the host I/O size.
Doing this makes it unnecessary to access a second drive for a single small host I/O. For
certain storage subsystems, having the segment size equal to the host I/O size is preferred,
which is
segment size that is equal to the host IO size can result in multiple disk operations due to
alignment differences. This can be easily remedied with a larger segment size selection for
small IO blocksizes.

There is no advantage in using a smaller segment size with RAID 1; only in instances of large
throughput blocksizes does this help with RAID 5 (which we discuss later). Because only the
data that is to be written to the disk is written to cache for an I/O, there is no cache penalty
encountered either. As mentioned earlier in the host sections, aligning data on segment
boundaries is very important for performance. With larger segment sizes, there are less
occasions to have misaligned boundaries impacting your performance, as more small I/O
boundaries reside within a single segment decreasing the chance of a host I/O spanning
multiple drives. This technique can be used to help mask the effect of poor layout of the host
data on the disks due to boundary differences.

not the case with the Midrange Storage Subsystems. In many cases, using the

Best practice: For most high transaction workloads with the Midrange Storage
Subsystems, the segment size of 128 KB (default) works best.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 67

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

With high throughput workload, the focus is on moving larger but fewer I/Os. This workload is
generally sequential in nature.

Best practice: In the high throughput environment, you want the stripe size to be equal to,
or an even multiple of, the host I/O size.

The total of all the segments for one pass of all the back-end data disks is a
segment sizes that can equal the I/O size might be desired to accomplish the higher
throughput you are looking for. For high read throughput, you want to have large segments
(128 K or higher) to get the most from each stripe. For example if the host I/O is 512 KB, and
the write is to a RAID 10 array, you want to use a segment size of 512 KB to limit the disk
operations as much as possible.

However, when the workload is high writes, and we are using RAID 5, we can use a method
known as
RAID 5 the parity is based on the value calculated for a stripe. As discussed earlier in 3.3.2,
“Understanding RAID types” on page 50, when the I/O being written is spread across the
entire stripe width, no reads are required to calculate the parity; and the I/O completes with
fewer back-end I/Os being required.

This design can use a smaller segment size to align the host I/O size with the size of the
stripe width. For instance with our previous example, if the array is a 4+P RAID 5 you want to
use a 128 KB segment size to achieve a full stripe write. This type of management requires
that very few host I/Os not equal a full stripe width.

Tips: The possible segment sizes available are 8 KB, 16 KB, 32 KB, 64 KB, 128 KB,
256 KB, and 512 KB:

򐂰 Storage Manager sets a default segment size to 128KB.
򐂰 For database applications, a segment size of 128 KB or 256KB (for data warehousing)

򐂰 In a large file environment, such as media streaming or CAD, 128 KB or more are

full stripe (stride) write, which can work well to improve your performance. With

have shown to be more effective.

preferable with a focus on full stripe writes.

stripe. So, large

򐂰 For a Web server or file and print server, the a range of 64KB to 128KB should provide

best results.

Note: A performance testing schedule must be undertaken in the environment before
going into production with a given segment size. Segment size can be dynamically
changed, but only by rewriting the data, which consumes bandwidth and impacts
performance. Plan this configuration carefully to avoid having to reconfigure the options
chosen.

3.3.9 Media scan

Media scan is a background process that checks the physical disks for defects by reading the
raw data from the disk and writing it back, which detects possible problems caused by bad
sectors of the physical disks before they disrupt normal data reads or writes. This process is
sometimes known as

Media scan continuously runs in the background, using spare cycles to complete its work.
The default media scan is for a scan every 30 days, that is, the maximum time media scan will
have to complete the task. During the scan process, the DS3500 calculates how much longer
the scan process will take to complete, and adjusts the priority of the scan to ensure that the

68 IBM System Storage DS3500: Introduction and Implementation Guide

data scrubbing.

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

scan completes within the time setting allocated. After the media scan has completed, it will
start over again and reset its time for completion to the current setting. This media scan
setting can be reduced, however if the setting is too low, priority will be given to media scan
over host activity to ensure that the scan completes in the allocated time. This scan can
impact on performance, but improve data integrity.

Media scan must be enabled for the entire storage subsystem. The system wide enabling
specifies the duration over which the media scan will run. The logical drive enabling specifies
whether or not to do a redundancy check as well as media scan.

A media scan can be considered a surface scan of the hard drives, whereas a redundancy
check scans the blocks of a RAID 3, 5, or 6 logical drive and compares it against the
redundancy data. In the case of a RAID 1 logical drive, then the redundancy scan compares
blocks between copies on mirrored drives.

We have seen no effect on I/O with a 30 day setting unless the processor is utilized in excess
of 95%. The length of time that it will take to scan the LUNs depends on the capacity of all the
LUNs on the system and the utilization of the controller.

Note: If you change the media scan duration setting, the changes will not take effect until
the current media scan cycle completes or the controller is reset.

3.3.10 Cache parameters

Cache memory is an area of temporary volatile storage (RAM) on the controller that has a
faster access time than the drive media. This cache memory is shared for read and write
operations.

Efficient use of the RAID controller cache is essential for good performance of the DS3500.
See 9.5.2, “Change Cache Settings” on page 247 for detailed explanation about the cache
parameter settings.

Cache blocksize selection

On the DS3500, the cache blocksize is a variable value that can be set to 4 K, 8 K, or 16 K.
The general default setting is 8 K. The main goals with setting this value are to minimize the
number of cache IO’s needed, and at the same time, not waste space. This value is a storage
subsystem wide parameter, and when set, it is the value to be used by all cache operations.

For example, if the I/O of greatest interest is taken from your database operations during the
day rather than from your weekly backups, you want to tune this value to handle the high
transactions best. Knowing that the higher transactions will have smaller I/O sizes, using the
4 K settings is generally best for transaction intense environments.

Best practice: Set the cache blocksize to 4 K for the DS3500, normally for transaction
intense environments.

In a throughput intense environment, as we discussed earlier, you want to get as much data
into cache as possible. In this environment it is generally best to use the 16 K blocksize for the
cache.

Best practice: Set the cache blocksize to 16 K for the DS3500 system, normally for
throughput intense environments.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 69

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

In mixed workload environments, you must decide which workload type is most critical and
set the system wide settings to best handle your business needs.

Best Practice: Set the cache blocksize to 8 K for the DS3500, normally for mixed workload
environments.

Tip: Throughput operations, though impacted by smaller cache blocksize, can still perform
reasonably if all other efforts have been accounted for. Transaction based operations are
normally the higher concern, and therefore must be the focus for setting the server wide
values if applicable.

Cache flush control settings

In addition to the cache blocksize, the DS3500 also has a cache control, which determines
the amount of data that can be held in write cache. With the
determine what level of write cache usage can be reached before the server will start to flush
the data to disk, and at what level the flushing will stop.

By default, these parameters are set to the value of “80” for each, which means that the
server will wait until 80% of the write cache is used before it will flush the data to disk. In a
fairly active write environment, this value might be far too high. You can adjust these settings
up and down until you find a particular value that best suits your environment. If the values
are not the same, then back-end drive inactive time increases, and you have surging with
peaks and valleys occurring instead of a steady usage of back-end disks.

cache flush settings, you can

You can also vary the maximum amount of time the write data can remain in cache prior to
being forced out, and written to disks. This value by default is set to ten seconds but can be
changed by using the Storage Manager (SM) command line interface command:

‘set logical Drive [LUN] cacheflushModifier=[new_value];’

Best practice: Begin with Start/Stop flush settings of 50/50, and adjust from there. Always
keep the values for Start and Stop equal to each other.

3.4 Planning for premium features

The premium features that come with the DS3500 Storage Manager software are enabled by
purchasing a premium feature license key for each feature. When planning for any of the
premium features, it is a good idea to document what are the goals and rationale for
purchasing the feature, which clearly defines from the outset what you want to achieve and
why. There are many different premium features available for different types of expanded
capabilities. The following is a list of the currently available features for the DS3500:

򐂰 Storage Partitioning
򐂰 High Performance Tier
򐂰 Maximum 96 drive support
򐂰 RAID 6 Logical Drive support
򐂰 Full Disk Encryption
򐂰 External Key Management
򐂰 FlashCopy
򐂰 VolumeCopy
򐂰 Enhanced Remote Mirroring

70 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Some points that may play an important part with premium features and should be
considered are the following:

򐂰 Which premium features will you need
򐂰 What additional resources will they require
򐂰 Additional configuration needs
򐂰 Amount of free space to support
򐂰 Retention of copies needed
򐂰 Automated or manual copies
򐂰 Disaster recovery or backup operation needs

Then document the needs and requirements to support all the features being implemented.
Many of these premium features are purely to enable special capabilities within the DS3500;
however, features like the Copy Services can require additional resources to be able to
implement them and need to be considered when planning and implementing your
environment.

3.4.1 FlashCopy

A FlashCopy logical drive is a point-in-time image of a logical drive. It is the logical equivalent
of a complete physical copy, but you can create it much more quickly than a physical copy.
Additionally, it requires less disk space. In DS3500 Storage Manager, the logical drive from
which you are basing the FlashCopy, called the base logical drive, must be a standard logical
drive in the storage server. Typically, you create a FlashCopy so that an application (for
example, an application to take backups) can access the FlashCopy and read the data while
the base logical drive remains online and user-accessible. The DS3500 can have up to eight
FlashCopy volumes.

Plan carefully with regard to the space available to make a FlashCopy of a logical drive even
though FlashCopy takes only a small amount of space compared to the base image.

3.4.2 VolumeCopy

The VolumeCopy feature is a firmware-based mechanism for replicating logical drive data
within a storage subsystem. This feature is designed as a system management tool for tasks,
such as relocating data to other drives for hardware upgrades or performance management,
data backup, and restoring snapshot logical drive data.

A VolumeCopy creates a complete physical replication of one logical drive (source) to another
(target) within the same storage subsystem. The target logical drive is an exact copy or clone
of the source logical drive. VolumeCopy can be used to clone logical drives to other arrays
inside the DS3500 Storage System.

The VolumeCopy premium feature must be enabled by purchasing a Feature Key. For efficient
use of VolumeCopy, FlashCopy must be installed as well.

3.4.3 Enhanced Remote Mirroring

The Enhanced Remote Mirroring (ERM) option is used for online, real-time replication of data
between storage subsystems over a remote distance. ERM mirrors by creating a full logical
drive image on a secondary of the logical drive that is being mirrored from the primary. The
DS3500 can have up to eight mirrored pair relationships. The DS3500 can mirror to another
DS3500 or another member of the IBM Midrange DS Storage System family running 7.70.xx
code or higher. At the current time these are the only supported configurations.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 71

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

Operating modes for ERM

Enhanced Remote Mirroring (formerly named “Remote Volume Mirroring”) offers three
operating modes:

򐂰 Metro mirroring:

Metro mirroring is a synchronous mirroring mode. Any host write requests are written to
the primary (local) storage subsystem and then to the secondary (remote) storage
subsystem. The remote storage controller acknowledges the write request operation to the
local storage controller, which reports a write completion to the host. This mode is called
synchronous. The host application does not get the write request result until the write
request has been executed on both (local and remote) storage controllers.

򐂰 Global copy:

This mode copies a non-synchronous, remote copy function designed to complete write
operations on the primary storage system before they are received by the secondary
storage system. This capability is designed to prevent primary performance from being
affected by wait time from writes on the secondary system. Therefore, the primary and
secondary copies can be separated by long distances. This function is appropriate for
remote data migration, offsite backups, and transmission of inactive database logs at
virtually unlimited distances.

򐂰 Global mirroring:

This mode is a two-site remote data mirroring function designed to maintain a complete
and consistent remote mirror of data asynchronously at virtually unlimited distances with
virtually no degradation of application response time. Separating data centers by longer
distances helps provide protection from regional outages. This asynchronous technique
can help achieve better performance for unlimited distances by allowing the secondary
site to trail in data currency a few seconds behind the primary site. With Global Mirror,
currency can be configured to be as little as three to five seconds with respect to host I/O.
This two-site data mirroring function is designed to provide a high-performance,
cost-effective global distance data replication and disaster recovery solution.

The Enhanced Remote Mirroring has also been equipped with new functions for better
business continuance solution design and maintenance tasks.

A minimum of two storage subsystems is required. One storage subsystem can have primary
volumes being mirrored to arrays on other storage subsystems and hold secondary volumes
from other storage subsystems. Also note that because replication is managed on a
per-logical drive basis, you can mirror multiple individual logical drives from a primary storage
subsystem to different appropriate secondary logical drives which are located on several
separate remote storage subsystems. However, only one primary (source) and one
secondary (target) member can exist in any mirrored pair relationship; and a logical drive
cannot be a member of more than one mirror relationship at a time.

Planning considerations for ERM

Here are various planning considerations to keep in mind:

򐂰 DS3500 Storage Systems (minimum of two)
򐂰 Switched fabric networks between sites
򐂰 Distances between sites (ensure that it is supported)
򐂰 Switches, directors or multi-protocol routers used for connections
򐂰 Redundancy
򐂰 Additional storage space requirements

72 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Note: ERM requires a dedicated switched fabric connection per controller to be attached
to Host port 4 on both A and B controllers of the DS3500 FC HIC option.

This same dedication is required at both the source and target ends of the ERM solution.

3.4.4 Drive Security

This is a new premium feature where Full Disk Encryption (FDE) protects the data on the
disks only when the drives are removed from storage subsystem enclosures. Drive Security
requires security capable drives (FDE) and provide access to data only through a controller
that has the correct security key when Drive Security is enabled.

Requirements

Businesses must comply with a growing number of corporate standards and government
regulations, Drive security is one tool that can enhance security thus complying with these
new standards and regulations.

Full Disk Encryption

Full Disk Encryption (FDE) does not prevent someone from copying the data in the storage
subsystems through Fibre Channel host port connections when the drives are unlocked and
operating. FDE also does not prevent unauthorized management access. A security capable
drive encrypts data during writes and decrypts data during reads. FDE prevents the physical
removal of the disk from the DS3500 system and interpreting data it contained. The FDE
drive with Drive Security enabled will be locked on power up and will only unlock after
successful authentication with the DS3500 system.

The Encryption Key is generated by the drive and never leaves the drive, so it always stays
secure. It is stored in encrypted form performing symmetric encryption and decryption of data
at full disk speed with no impact on disk performance. Each FDE drive uses its own unique
encryption key which is generated when the disk is manufactured and regenerated when
required by the storage administrator using the DS3500 Disk Encryption Manager.

The security enabled drives can be used as normal drives and intermixed in an array with
drives of equal type and capacity when this feature is not enabled. This new feature is
detailed in IBM Midrange System Storage Hardware Guide, SG24-7676.

3.4.5 Obtaining premium features key

You can generate the feature key file by using the premium feature activation tool that is
located at the following Web site:

http://www-912.ibm.com/PremiumFeatures/jsp/keyInput.jsp

The key can then be added to your DS3500 system as detailed in “Premium Features” on
page 200.

3.5 Additional planning considerations

In this section, we review additional elements to consider when planning your DS3500
Storage Systems using a Logical Volume Manager and virtualization options.

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 73

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

3.5.1 Planning for systems with LVM: AIX example

Many modern operating systems implement the concept of a Logical Volume Manager (LVM)
that can be used to manage the distribution of data on physical disk devices.

The Logical Volume Manager controls disk resources by mapping data between a simple and
flexible logical view of storage space and the actual physical disks. The Logical Volume
Manager does this by using a layer of device driver code that runs above the traditional
physical device drivers. This logical view of the disk storage is provided to applications and is
independent of the underlying physical disk structure.

74 IBM System Storage DS3500: Introduction and Implementation Guide

Draft Document for Review March 28, 2011 12:24 pm 7914DS3KPlanning_090710.fm

Figure 3-22 illustrates the layout of those components in the case of the AIX Logical Volume
Manager.

Figure 3-22 AIX Logical Volume Manager

Hierarchy of structures in disk storage

A hierarchy of structures is used to manage the actual disk storage, and there is a well
defined relationship among these structures.

In AIX, each individual disk drive is called a physical volume (PV) and has a name, usually
/dev/hdiskx (where x is a unique integer on the system). In the case of the DS3500 Storage
System, such physical volumes correspond to a LUN.

򐂰 Every physical volume in use belongs to a volume group (VG) unless it is being used as a

raw storage device.

򐂰 Each physical volume is divided into physical partitions (PPs) of a fixed size for that

physical volume.

򐂰 Within each volume group, one or more logical volumes (LVs) are defined. Logical

volumes are groups of information located on physical volumes. Data on logical volumes
appear contiguous to the user, but can be spread (striped) on multiple physical volumes.

򐂰 Each logical volume consists of one or more logical partitions (LPs). Each logical partition

corresponds to at least one physical partition (see Figure 3-23 on page 76). If mirroring is
specified for the logical volume, additional physical partitions are allocated to store the
additional copies of each logical partition.

򐂰 Logical volumes can serve a number of system purposes (paging, for example), but each

logical volume that holds ordinary systems, user data, or programs, contains a single
journaled file system (JFS or JFS2). Each file system consists of a pool of page-size
blocks. In AIX Version 4.1 and later, a given file system can be defined as having a
fragment size of less than 4 KB (512 bytes, 1 KB, or 2 KB).

Chapter 3. IBM System Storage DS3500 Storage System planning tasks 75

7914DS3KPlanning_090710.fm Draft Document for Review March 28, 2011 12:24 pm

LP1 LP8 LP9 LP8 LP9

Logical Volume - Iv04 Logical Volume - mirrlv

PP8

PP4 2

PP1 9

PP1 8

PP4 5

PP1 8

PP4 5

Physical Volume

(/dev/hdisk9)

LP: Logical Partitions
PP: Phy si cal Par t it io n s

Physical
Partitions

Physical
Partitions

Physical
Partitions

Physical Volume

Physical Volume

Physical Volume

ROOTVG

ROOTVG

Logical
Partitions

Logical Volume

hdisk1

hdisk2

hdisk0

Three physical disks in the box form
a single volume group (rootvg).

Figure 3-23 Relationships between LP and PP

The Logical Volume Manager controls disk resources by mapping data between a simple
and flexible logical view of storage space and the actual physical disks. The Logical
Volume Manager does this by using a layer of device driver code that runs above the
traditional physical device drivers. This logical view of the disk storage is provided to
applications and is independent of the underlying physical disk structure (Figure 3-24).

Figure 3-24 AIX LVM conceptual view

76 IBM System Storage DS3500: Introduction and Implementation Guide

Best practice: When using a DS3500 Storage System with operating systems that have a
built-in LVM, or if a LVM is available, you need to make use of the LVM.

The AIX LVM provides a number of facilities or policies for managing both the performance
and availability characteristics of logical volumes. The policies that have the greatest impact
on performance in general disk environment are the intra-disk allocation, inter-disk allocation,
write scheduling, and write-verify policies.