HPSS
Installation Guide
High Performance Storage System
Release 6.2
July 2008 (Revision 2.0)
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 1
© Copyright (C) 1992, 2008 International Business Machines Corporation, The Regents of the University of
California, Los Alamos National Security, LLC, Lawrence Livermore National Security, LLC, Sandia
Corporation, and UT-Battelle.
All rights reserved.
Portions of this work were produced by Lawrence Livermore National Security, LLC, Lawrence Livermore
National Laboratory (LLNL) under Contract No. DE-AC52-07NA27344 with the U.S. Department of Energy
(DOE); by the University of California, Lawrence Berkeley National Laboratory (LBNL) under Contract No.
DE-AC02-05CH11231 with DOE; by Los Alamos National Security, LLC, Los Alamos National Laboratory
(LANL) under Contract No. DE-AC52-06NA25396 with DOE; by Sandia Corporation, Sandia National
Laboratories (SNL) under Contract No. DE-AC04-94AL85000 with DOE; and by UT-Battelle, Oak Ridge
National Laboratory (ORNL) under Contract No. DE-AC05-00OR22725 with DOE. The U.S. Government has
certain reserved rights under its prime contracts with the Laboratories.
DISCLAIMER
Portions of this software were sponsored by an agency of the United States Government. Neither the United
States, DOE, The Regents of the University of California, Los Alamos National Security, LLC, Lawrence
Livermore National Security, LLC, Sandia Corporation, UT-Battelle, nor any of their employees, makes any
warranty, express or implied, or assumes any liability or responsibility for the accuracy, completeness, or
usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not
infringe privately owned rights .
Printed in the United States of America.
HPSS Release 6.2
July 2008 (Revision 2.0)
High Performance Storage System is a trademark of International Business Machines Corporation.
IBM is a registered trademark of International Business Machines Corporation.
IBM, DB2, DB2 Universal Database, AIX, pSeries, and xSeries are trademarks or registered trademarks of
International Business Machines Corporation.
AIX and RISC/6000 are trademarks of International Business Machines Corporation.
UNIX is a registered trademark of the Open Group.
Linux is a registered trademark of Linus Torvalds in the United States and other countries.
Kerberos is a trademark of the Massachusetts Institute of Technology.
Java is a registered trademark of Sun Microsystems, Incorporated in the United States and other countries.
ACSLS is a trademark of Sun Microsystems, Incorporated.
Microsoft Windows is a registered trademark of Microsoft Corporation.
NFS, Network File System, and ACSLS are trademarks of Sun Microsystems, Inc.
DST is a trademark of Ampex Systems Corporation.
Other brands and product names appearing herein may be trademarks or registered trademarks of third parties.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 2
Table of Contents
Chapter 1. Release 6.2...........................................................................................................15
1.1.New Features.................................................................................................................15
1.1.1.DCE Replacement .............................................................................................................................15
1.1.2.Linux Support ...................................................................................................................................15
1.1.3.Security..............................................................................................................................................15
1.1.4.SCSI PVR .........................................................................................................................................15
1.1.5.HPSS VFS Interface...........................................................................................................................15
1.1.6.GridFTP Interface..............................................................................................................................15
1.1.7.SAN3P/PIO Support .........................................................................................................................15
1.1.8.Additional hpssadm Configuration Options.......................................................................................15
1.1.9.Additional hpssadm operations..........................................................................................................16
1.1.10.Additional Library and Device Support...........................................................................................16
1.1.11.SAN Virtual Volume ID Mapping...................................................................................................16
1.1.12.Drive Pools.......................................................................................................................................17
1.1.13.FTP Enhancement............................................................................................................................17
1.1.14.mkhpss Enhancement.......................................................................................................................17
1.1.15.DB2 Monitoring...............................................................................................................................17
1.1.16.File Family enhancements................................................................................................................17
1.1.17.SSM Configuration Files Consolidation..........................................................................................17
1.1.18. Mover Enhancement.......................................................................................................................18
1.2.Retired Features ............................................................................................................18
1.3.Deferred Features...........................................................................................................18
1.4.HPSS Changes ..............................................................................................................18
1.4.1.Documentation Organization Changes...............................................................................................18
1.4.2.Metadata Changes..............................................................................................................................18
1.4.3.DMAP Gateway Changes .................................................................................................................19
1.4.3.1.Creating an XDSM Fileset........................................................................................................19
1.4.3.2.Viewing DMAP Gateway XDSM Fileset Information..............................................................20
1.4.3.3.Viewing Core Server XDSM Fileset Information.....................................................................20
1.4.4.SSM Changes.....................................................................................................................................21
1.4.4.1.Changes Affecting Sites Upgrading Directly from 4.5............................................................21
1.4.4.2.Changes Affecting Sites Upgrading from 5.1...........................................................................27
Chapter 2. HPSS Basics........................................................................................................35
2.1.Introduction....................................................................................................................35
2.2.HPSS Capabilities.........................................................................................................35
2.2.1.Network-centered Architecture..........................................................................................................35
2.2.2.High Data Transfer Rate....................................................................................................................35
2.2.3.Parallel Operation..............................................................................................................................35
2.2.4.Based on Standard Components.........................................................................................................36
2.2.5.Data Integrity Through Transaction Management.............................................................................36
2.2.6.Multiple Hierarchies and Classes of Services....................................................................................36
2.2.7.Storage Subsystems............................................................................................................................36
2.3.HPSS Components........................................................................................................37
2.3.1.HPSS Files, Filesets, Volumes, Storage Segments and Related Metadata .......................................38
2.3.2.HPSS Servers.....................................................................................................................................40
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 3
2.3.3.HPSS Storage Subsystems.................................................................................................................44
2.3.4.HPSS Infrastructure ..........................................................................................................................44
2.3.5.HPSS User Interfaces ........................................................................................................................46
2.3.6.HPSS Management Interfaces...........................................................................................................46
2.3.7.HPSS Policy Modules .......................................................................................................................47
2.4.HPSS Hardware Platforms............................................................................................48
2.4.1.Server Platforms ...............................................................................................................................48
2.4.2.Client Platforms ................................................................................................................................48
2.4.3.Mover Platforms................................................................................................................................49
Chapter 3. HPSS Planning...................................................................................................51
3.1.Overview.......................................................................................................................51
3.1.1.HPSS System Architecture.................................................................................................................51
3.1.2.HPSS Configuration Planning............................................................................................................52
3.1.3.Purchasing Hardware and Software...................................................................................................54
3.1.4.HPSS Operational Planning...............................................................................................................55
3.1.5.HPSS Deployment Planning..............................................................................................................55
3.2.Requirements and Intended Uses for HPSS..................................................................56
3.2.1.Storage System Capacity....................................................................................................................56
3.2.2.Required Throughputs........................................................................................................................56
3.2.3.Load Characterization........................................................................................................................56
3.2.4.Usage Trends.....................................................................................................................................56
3.2.5.Duplicate File Policy..........................................................................................................................57
3.2.6.Charging Policy..................................................................................................................................57
3.2.7.Security..............................................................................................................................................57
3.2.7.1.Cross Realm Access..................................................................................................................57
3.2.8.High Availability Option....................................................................................................................58
3.3.Prerequisite Software Considerations ...........................................................................58
3.3.1.Prerequisite Software Overview.........................................................................................................58
3.3.1.1.DB2...........................................................................................................................................58
3.3.1.2.Kerberos....................................................................................................................................58
3.3.1.3.LDAP and IBM Kerberos..........................................................................................................59
3.3.1.4.Java...........................................................................................................................................59
3.3.2.Prerequisite Summary By HPSS Node Type.....................................................................................59
3.3.2.1.HPSS Server Nodes...................................................................................................................59
3.3.2.1.1.AIX Requirements....................................................................................................................................................59
3.3.2.1.2.Linux Requirements.................................................................................................................................................60
3.3.2.2.HPSS Mover Nodes .................................................................................................................60
3.3.2.2.1.AIX Requirements....................................................................................................................................................60
3.3.2.2.2.Linux Requirements.................................................................................................................................................60
3.3.2.2.3.Solaris Requirements..............................................................................................................................................61
3.3.2.2.4.IRIX Requirements..................................................................................................................................................61
3.3.2.3.HPSS Client Nodes...................................................................................................................61
3.3.2.3.1.SSM Client Requirements.......................................................................................................................................61
3.3.2.3.2.Client API Requirements.........................................................................................................................................61
3.3.2.3.3.FTP/PFTP Client Requirements.............................................................................................................................61
3.3.2.4.HPSS HDM Nodes (Linux only)...............................................................................................61
3.4.Hardware Considerations ..............................................................................................62
3.4.1.Network Considerations.....................................................................................................................62
3.4.2.Robotically Mounted Tape.................................................................................................................62
3.4.2.1.IBM 3494..................................................................................................................................63
3.4.2.2.Drive-Controlled LTO Libraries (IBM 3582, IBM 3583, IBM 3584, Spectralogic T120).......63
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 4
3.4.2.3.STK L40, STK SL500, STK SL8500..........................................................................................63
3.4.2.4.STK............................................................................................................................................63
3.4.2.5.ADIC AML ...............................................................................................................................63
3.4.3.Manually Mounted Tape....................................................................................................................63
3.4.4.Tape Devices......................................................................................................................................63
3.4.4.1.Multiple Media Support............................................................................................................64
3.4.5.Disk Devices......................................................................................................................................67
3.4.6.Special Bid Considerations................................................................................................................68
3.5.HPSS Sizing Considerations.........................................................................................68
3.5.1.HPSS User Storage Space..................................................................................................................69
3.5.2.HPSS Infrastructure Storage Space....................................................................................................69
3.5.3.HPSS Filesystems..............................................................................................................................71
3.5.3.1./opt/hpss....................................................................................................................................71
3.5.3.2./var/hpss....................................................................................................................................71
3.5.3.3./var/hpss/adm/core....................................................................................................................72
3.5.3.4./var/hpss/hpssdb........................................................................................................................72
3.5.3.5./var/hpss/hpssdb/subsys1 & subsysX........................................................................................72
3.5.3.6./db2/backups/cfg.......................................................................................................................72
3.5.3.7./db2/backups/subsys1 & subsysX..............................................................................................73
3.5.3.8./db2/log/cfg...............................................................................................................................73
3.5.3.9./db2/log/subsys1 & subsysX......................................................................................................73
3.5.3.10./db2/mirror-log/cfg.................................................................................................................73
3.5.3.11./db2/mirror-log/subsys1 & subsysX........................................................................................73
3.5.3.12./db2/mirror-backup/cfg...........................................................................................................73
3.5.3.13./db2/mirror-backup/subsys1 & subsysX.................................................................................73
3.5.3.14.SUBSYS1 Database Allocation...............................................................................................73
3.5.4.HPSS Metadata Space .......................................................................................................................74
3.5.4.1.SMS versus DMS Space............................................................................................................74
3.5.4.2.'CFG' Database Allocation.......................................................................................................74
3.5.4.3.'SUBSYS' Database Allocation.................................................................................................74
3.5.4.4.DB2 Disk Space........................................................................................................................77
3.5.5.System Memory and Disk Space........................................................................................................78
3.5.5.1.Operating System Disk Spaces..................................................................................................78
3.5.5.2.System Disk Space Requirements for Running SSM.................................................................78
3.5.5.3.System Memory and Paging Space Requirements....................................................................78
3.6.HPSS Interface Considerations......................................................................................79
3.6.1.Client API .........................................................................................................................................79
3.6.2.FTP.....................................................................................................................................................79
3.6.3.Parallel FTP.......................................................................................................................................80
3.6.4.XFS....................................................................................................................................................80
3.7.HPSS Server Considerations.........................................................................................80
3.7.1.Core Server........................................................................................................................................81
3.7.2.Migration/Purge Server......................................................................................................................83
3.7.3.Gatekeeper.........................................................................................................................................84
3.7.4.Location Server .................................................................................................................................86
3.7.5.PVL....................................................................................................................................................86
3.7.6.PVR....................................................................................................................................................86
3.7.6.1.STK PVR...................................................................................................................................87
3.7.6.2.LTO PVR...................................................................................................................................87
3.7.6.3.3494 PVR..................................................................................................................................88
3.7.6.4.AML PVR..................................................................................................................................88
3.7.6.5.Operator PVR...........................................................................................................................88
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 5
3.7.6.6.SCSI PVR..................................................................................................................................88
3.7.7.Mover ................................................................................................................................................89
3.7.7.1.AIX Asynchronous I/O..............................................................................................................89
3.7.7.2.Tape Devices.............................................................................................................................89
3.7.7.2.1.AIX...........................................................................................................................................................................89
3.7.7.2.2.Solaris......................................................................................................................................................................89
3.7.7.2.3.IRIX..........................................................................................................................................................................90
3.7.7.2.4.Linux........................................................................................................................................................................90
3.7.7.3.Disk Devices..............................................................................................................................90
3.7.7.4.Performance..............................................................................................................................91
3.7.8.Logging Service.................................................................................................................................91
3.7.9.Startup Daemon..................................................................................................................................92
3.7.10.Storage System Management...........................................................................................................92
3.8.Storage Subsystem Considerations................................................................................94
3.9.Storage Policy Considerations ......................................................................................94
3.9.1.Migration Policy ...............................................................................................................................94
3.9.1.1.Migration Policy for Disk.........................................................................................................94
3.9.1.2.Migration Policy for Tape........................................................................................................95
3.9.2.Purge Policy ......................................................................................................................................95
3.9.3.Accounting Policy and Validation ....................................................................................................96
3.9.4.Security Policy...................................................................................................................................98
3.9.4.1.Client API..................................................................................................................................98
3.9.4.2.FTP/PFTP.................................................................................................................................98
3.9.4.3.XFS............................................................................................................................................98
3.9.4.4.Name Space...............................................................................................................................98
3.9.4.5.Security Audit............................................................................................................................99
3.9.5.Logging Policy...................................................................................................................................99
3.9.6.Location Policy .................................................................................................................................99
3.9.7.Gatekeeping.......................................................................................................................................99
3.10.Storage Characteristics Considerations ....................................................................101
3.10.1.Storage Class..................................................................................................................................102
3.10.1.1.Media Block Size Selection...................................................................................................103
3.10.1.2.Virtual Volume Block Size Selection (disk)...........................................................................103
3.10.1.3.Virtual Volume Block Size Selection (tape)..........................................................................103
3.10.1.4.Stripe Width Selection...........................................................................................................103
3.10.1.5.Blocks Between Tape Marks Selection (tape only)...............................................................104
3.10.1.6.Minimum Storage Segment Size Selection (disk only)..........................................................105
3.10.1.7.Maximum Storage Segment Size Selection (disk only).........................................................105
3.10.1.8.Maximum VVs to Write (tape only).......................................................................................106
3.10.1.9.Average Number of Storage Segments (disk only)................................................................106
3.10.1.10.PV Estimated Size / PV Size Selection................................................................................106
3.10.1.11.Optimum Access Size Selection...........................................................................................106
3.10.1.12.Some Recommended Parameter Values for Supported Storage Media..............................106
3.10.1.12.1.Disk Media Parameters....................................................................................................................................107
3.10.1.12.2.Tape Media Parameters....................................................................................................................................107
3.10.2.Storage Hierarchy..........................................................................................................................109
3.10.3.Class of Service..............................................................................................................................110
3.10.3.1.Selecting Minimum File Size.................................................................................................110
3.10.3.2.Selecting Maximum File Size................................................................................................110
3.10.3.3.Selecting Stage Code............................................................................................................110
3.10.3.4.Selecting Optimum Access Size.............................................................................................111
3.10.3.5.Selecting Average Latency....................................................................................................111
3.10.3.6.Selecting Transfer Rate.........................................................................................................112
3.10.3.7.StripeLength and StripeWidth Hints.....................................................................................112
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 6
3.10.4.File Families...................................................................................................................................112
3.11.HPSS Performance Considerations...........................................................................112
3.11.1.DB2................................................................................................................................................112
3.11.2.Bypassing Potential Bottlenecks ...................................................................................................113
3.11.3.Configuration.................................................................................................................................113
3.11.4.FTP/PFTP......................................................................................................................................114
3.11.5.Client API......................................................................................................................................115
3.11.6.Core Server ...................................................................................................................................115
3.11.7.Location Server..............................................................................................................................115
3.11.8.Logging..........................................................................................................................................115
3.11.9.Cross Realm Trust..........................................................................................................................115
3.11.10.Gatekeeping.................................................................................................................................115
3.11.11.XFS .............................................................................................................................................116
3.11.12.HPSS VFS Interface.....................................................................................................................116
3.12.HPSS Metadata Backup Considerations ...................................................................117
3.13.HPSS Security Considerations..................................................................................117
Chapter 4. System Preparation..........................................................................................119
4.1.General Setup..............................................................................................................119
4.2.Setup Filesystems........................................................................................................120
4.2.1.DB2 Filesystem................................................................................................................................120
4.2.2.HPSS Filesystem..............................................................................................................................121
4.3.Setup Tape Libraries....................................................................................................121
4.3.1.Special LTO Considerations............................................................................................................121
4.3.2.IBM 3584.........................................................................................................................................121
4.3.3.3494..................................................................................................................................................122
4.3.4.STK..................................................................................................................................................123
4.3.5.AML.................................................................................................................................................123
4.4.Verify Tape Drives......................................................................................................124
4.4.1.AIX...................................................................................................................................................124
4.4.2.Solaris..............................................................................................................................................125
4.4.3.IRIX.................................................................................................................................................126
4.4.4.Linux................................................................................................................................................126
4.5.Setup Disk Drives........................................................................................................126
4.5.1.AIX...................................................................................................................................................127
4.5.2.Linux................................................................................................................................................128
4.5.3.IRIX.................................................................................................................................................128
4.6.Setup Network Parameters..........................................................................................129
4.6.1.HPSS.conf Configuration File.........................................................................................................132
4.6.2.SP/x Switch Device Buffer Driver Buffer Pools..............................................................................133
Chapter 5. HPSS Installation and Infrastructure Configuration...................................135
5.1.Prepare for Installation................................................................................................135
5.1.1.Distribution Media...........................................................................................................................135
5.1.2.Software Installation Packages.........................................................................................................135
5.1.3.Create Owner Account for HPSS Files............................................................................................136
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 7
5.1.4.Installation Target Directory Preparation........................................................................................136
5.2.Install Prerequisite Software........................................................................................137
5.2.1.Install Java........................................................................................................................................137
5.2.2.Install MIT Kerberos (If Using Kerberos Authentication)..............................................................137
5.2.3.Install LDAP (If Using LDAP Authorization).................................................................................137
5.2.4.Install Prerequisite Software for XFS HDM....................................................................................138
5.3.Install HPSS/DB2 and Configure HPSS Infrastructure...............................................139
5.3.1.Install and Configure HPSS - Root Subsystem Machine.................................................................139
5.3.1.1.Pre-Installation Configuration...............................................................................................139
5.3.1.2.Install HPSS Documentation and DB2 Software....................................................................141
5.3.1.3.Set Up DB2 Permanent License..............................................................................................142
5.3.1.4.Configure HPSS Security Services .........................................................................................143
5.3.1.4.1.Configure UNIX Authentication and UNIX Authorization..................................................................................143
5.3.1.4.2.Configure Kerberos Authentication and UNIX Authorization.............................................................................146
5.3.1.4.3.Configure Kerberos Authentication and LDAP Authorization............................................................................149
5.3.1.5.Configure DB2 Services..........................................................................................................152
5.3.1.5.1.Remote DB2 Client Access & Fileset Creation/Deletion.....................................................................................157
5.3.1.6.Configure Other Services........................................................................................................158
5.3.1.7.Create Configuration Bundle..................................................................................................159
5.3.2.Install and Configure HPSS – Secondary Subsystem Machine.....................................................160
5.3.2.1.Pre-Installation Configuration...............................................................................................160
5.3.2.2.Install HPSS Documentation and DB2 Software on a subsystem..........................................161
5.3.2.3.Set Up DB2 Permanent License..............................................................................................162
5.3.2.4.Install Configuration Bundle..................................................................................................163
5.3.2.5.Configure HPSS Security Services .........................................................................................164
5.3.2.6.Configure DB2 Services..........................................................................................................165
5.3.2.7.Configure Other Services........................................................................................................167
5.3.3.Install and Configure HPSS – Mover/Client Machine....................................................................168
5.3.3.1.Install Mover/Client source code............................................................................................168
5.3.3.2.Install Configuration Bundle..................................................................................................169
5.3.3.3.Create /var/hpss subdirectories..............................................................................................169
5.3.3.4.Modify Kerberos Configuration File, If Necessary................................................................169
5.3.3.5.Check Time and IP Address....................................................................................................169
5.4.Post Installation Procedures.........................................................................................169
5.5.HPSS Documentation & Manual Page Setup..............................................................170
5.5.1.Documentation and SSM Help Package..........................................................................................170
5.5.2.Manual Page Setup...........................................................................................................................171
5.6.Define HPSS Environment Variables..........................................................................171
5.7.Tune DB2....................................................................................................................171
5.8.Install and Build HPSS Source Code...........................................................................172
5.8.1.Construct and Build the HPSS Base Source Tree ...........................................................................172
5.8.1.1.Construct the HPSS Source Tree............................................................................................172
5.8.1.2.Build the HPSS Base Source Tree..........................................................................................172
5.8.1.3.Generate and Bind the DB2 Helper Program........................................................................173
5.8.2.Construct and Build the HPSS Mover/Client Source Tree..............................................................174
5.8.2.1.Construct the HPSS Mover/Client Source Tree.....................................................................174
5.8.2.2.Build the HPSS Mover/Client Source Tree.............................................................................174
5.8.3.Construct and Build the HPSS HDM Source Tree..........................................................................175
5.8.3.1.Construct the HPSS HDM Source Tree..................................................................................175
5.8.3.2.Build the HPSS HDM Source Tree.........................................................................................175
5.9.Supporting Both Unix and Kerberos Authentication for SSM....................................175
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 8
Chapter 6. Upgrading to HPSS Release 6.2 .....................................................................179
6.1.Special Instructions for Upgrading to HPSS 6.2.2......................................................179
6.2.Planning for the HPSS 6.2 Upgrade............................................................................180
6.2.1.Metadata changes in HPSS 6.2........................................................................................................180
6.2.2.Upgrade Requirements and Limitations...........................................................................................182
6.2.3.New Authentication and Authorization Mechanisms.......................................................................182
6.2.3.1.Authentication Mechanisms....................................................................................................183
6.2.3.2.Authorization Mechanisms......................................................................................................183
6.2.4.New HPSS 6.2 System Files............................................................................................................184
6.2.5.Testing the Metadata Conversion.....................................................................................................184
6.2.6.Estimating the Metadata Conversion Time (for 4.5 upgrades only)................................................184
6.2.6.1.Running Time for the Long Running Metadata Conversion Utilities (for 4.5 upgrades only).....
185
6.2.7.Capturing the Metadata Conversion Output.....................................................................................185
6.2.8.DB2 Configuration and Tuning (for 4.5 upgrades only)..................................................................186
6.2.9.Overview of the Upgrade Utilities...................................................................................................187
6.2.9.1.HPSS 4.5 and HPSS 5.1 Upgrade Utilities.............................................................................187
6.2.9.2.HPSS 4.5 Upgrade Utilities....................................................................................................188
6.2.9.3.HPSS 5.1 Upgrade Utilities....................................................................................................189
6.3.HPSS 6.2 Upgrade Procedures....................................................................................190
6.3.1.Verify Prerequisites..........................................................................................................................190
6.3.2.Acquire Software.............................................................................................................................190
6.3.3.Install Authentication and Authorization Mechanisms....................................................................191
6.3.4.Install or Upgrade DB2....................................................................................................................193
6.3.5.Upgrade AIX....................................................................................................................................194
6.3.6.Install or Upgrade Java....................................................................................................................194
6.3.7.Save Current HPSS Code and Configuration Files..........................................................................194
6.3.8.Prepare HPSS 6.2 Code...................................................................................................................194
6.3.8.1.Install HPSS 6.2 Distribution Image......................................................................................195
6.3.8.2. Compile HPSS 6.2 Source Code (if necessary).....................................................................195
6.3.8.3.Disable Binaries, temporarily.................................................................................................196
6.3.9.Set Environment Variables..............................................................................................................196
6.3.10.Setup Authentication and Authorization........................................................................................199
6.3.11.Pre-Conversion System Check.......................................................................................................202
6.3.12.Take a full backup of SFS or DB2.................................................................................................203
6.3.13.Upgrade from HPSS 4.5 to HPSS 6.2............................................................................................203
6.3.13.1.Prepare for the Conversion..................................................................................................203
6.3.13.2.Run db_convert_collect_info to Collect Metadata Information .........................................203
6.3.13.3.Convert Configuration Metadata .........................................................................................204
6.3.13.4.Convert Subsystem Metadata ...............................................................................................204
6.3.13.5.Run the Long Running Utilities............................................................................................205
6.3.13.6.Create Core Server ACLs.....................................................................................................209
6.3.13.7.Terminate the Scripting Session...........................................................................................210
6.3.13.8.Modify Permissions on Devices............................................................................................210
6.3.14.Verify HPSS 4.5 Conversion Results ............................................................................................211
6.3.14.1.Capture Session Output........................................................................................................211
6.3.14.2.Run db_convert_size_check..................................................................................................211
6.3.14.3.Run db_convert_ns_check....................................................................................................212
6.3.14.4.Run db_convert_address_check...........................................................................................212
6.3.14.5.Terminate Scripting Session.................................................................................................213
6.3.15.Upgrade from HPSS 5.1 to HPSS 6.2 ..........................................................................................213
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 9
6.3.16.Enable DB2 Backup......................................................................................................................215
6.3.17.Perform the DCE Export: hpss_dce_export..................................................................................215
6.3.18.Perform the Unix, LDAP or Kerberos Import...............................................................................215
6.3.19.Prepare the 6.2 System...................................................................................................................217
6.3.19.1.Tune DB2 for normal operations..........................................................................................218
6.3.19.2.Modify Accounting, if applicable..........................................................................................218
6.3.19.3.Update FTP Configuration Files..........................................................................................218
6.3.19.4.Populate the HPSS.conf files................................................................................................218
6.3.19.5.Copy the rc.hpss Script to /etc.............................................................................................218
6.3.19.6.Run the bind Script ..............................................................................................................218
6.3.19.7.Create Default Server Security ACLs...................................................................................218
6.3.19.8.Create SSM User Ids.............................................................................................................219
6.3.19.9.Create Location Server Endpoints........................................................................................220
6.3.19.10.Perform Additional Remote Mover Configuration.............................................................220
6.3.20.Bring up the HPSS 6.2 Servers......................................................................................................220
6.3.20.1.Invoke the SSM System Manager, Startup Daemon and prerequisite software ..................221
6.3.20.2.Update HPSS Configurations...............................................................................................222
6.3.20.3.Dump Accounting Metadata, if applicable...........................................................................223
6.3.20.4.Start HPSS 6.2 Servers.........................................................................................................224
6.3.21.Verify 6.2 System...........................................................................................................................224
6.3.22.Clean Up After a 4.5 to 6.2 Upgrade.............................................................................................225
6.3.23.Clean Up After a 5.1 to 6.2 Upgrade.............................................................................................225
6.3.24.Revert HPSS 6.2 System to Prior Release ....................................................................................225
6.3.24.1.Revert the HPSS 6.2 System to Version 4.5..........................................................................225
6.3.24.2.Revert the HPSS 6.2 System to Version 5.1..........................................................................226
6.4.Metadata Conversion Troubleshooting Procedures.....................................................227
6.4.1.HPSS 4.5 to 6.2 Conversion Utility Errors and Warnings...............................................................227
6.4.1.1.db_convert_collect_info Errors..............................................................................................227
6.4.1.2.db_config_convert, db_subsys_convert, and db_lr_convert Errors and Warnings............228
6.4.2.HPSS 5.1 to 6.2 Conversion Utility Errors......................................................................................231
6.4.2.1.hpss_md_convert_51 Errors...................................................................................................231
6.4.2.2.hpss_init_server_acls Errors..................................................................................................232
6.5.HPSS 4.5 Conversion Utilities Output........................................................................232
6.5.1.Interpreting Output from the 4.5 Conversion Utility.......................................................................232
6.5.2.Examples of HPSS 4.5 Conversion Utility Output.........................................................................234
6.5.2.1.db_convert_collect_info Output.............................................................................................234
6.5.2.2.db_config_convert Output......................................................................................................234
6.5.2.3.db_subsys_convert Output......................................................................................................238
6.5.2.4.Long Running Conversion Utilities Output............................................................................241
Appendix A. Glossary of Terms and Acronyms...............................................................245
Appendix B. References......................................................................................................256
Appendix C. Developer Acknowledgments.......................................................................258
Appendix D. HPSS.conf Configuration File.....................................................................259
D.1. PFTP Client Stanza...................................................................................................259
D.2. PFTP Client Interfaces Stanza..................................................................................264
D.3. Multinode Table Stanza............................................................................................266
D.4. Network Option Stanza.............................................................................................268
D.5. PFTP Daemon Stanza...............................................................................................273
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 10
D.6. Transfer Agent Stanza..............................................................................................287
D.7. Stanzas Reserved for Future Use..............................................................................291
Appendix E. hpss_env_defs.h.............................................................................................293
Appendix F. /var/hpss files.................................................................................................309
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 11
List of Figures
Figure 1. File Migration and Stage Operations..................................................................37
Figure 2. Class of Service / Hierarchy / Storage Class.......................................................38
Figure 3. HPSS Components................................................................................................40
Figure 4. HPSS Generic Configuration...............................................................................52
Figure 5. Basic HPSS Metadata & Filesystem Allocation.................................................70
Figure 6. The Relationship of Various Server Data Structures........................................82
Figure 7. Relationship of Class of Service, Storage Hierarchy, and Storage Class.......102
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 12
List of Tables
Table 1. HPSS Client Interface and Mover Platforms.......................................................49
Table 2. Supported Platform/Driver/Tape Drive Combinations......................................64
Table 3. Cartridge/Drive Affinity Table.............................................................................66
Table 4. Paging Space Info...................................................................................................79
Table 5. Gatekeeping Call Parameters..............................................................................100
Table 6. Suggested Block Sizes for Disk............................................................................107
Table 7. Suggested Block Sizes for Tape...........................................................................108
Table 8. Network Options...................................................................................................131
Table 9. Installation Package Sizes and Disk Requirements...........................................136
Table 10. Runing Times for Long Running Metadata Conversion Utilities..................185
Table 11. 6.2 Default Server Configuration Parameters.................................................222
Table 12. PFTP Client Stanza Fields.................................................................................259
Table 13. PFTP Client Interfaces Stanza Fields...............................................................264
Table 14. Multinode Table Stanza Fields..........................................................................266
Table 15. Network Options Stanza Fields.........................................................................269
Table 16. PFTP Daemon Stanza Description....................................................................273
Table 17. Transfer Agent Stanza Description..................................................................287
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 13
Preface
About this book
The HPSS Installation Guide is for use both at system installation time as well as throughout the
lifetime of the system. It will guide system administrators through the planning and installation of a
new HPSS system. It also guides system administrators through the conversion process to upgrade
existing HPSS systems to Release 6.2. It serves as a reference whenever the system is reconfigured
by the addition, deletion, or modification of hosts, tape libraries, devices, or other components.
Chapter 1 discusses HPSS changes for Release 6.2.
Chapter 2 gives an overview of HPSS technology.
Chapters 3-5 guide administrators of new HPSS systems through planning, system
preparation, HPSS software installation, and configuration of the HPSS infrastructure.
Chapter 6 guides administrators of existing 4.5 or 5.1 HPSS systems through the conversion
process, bringing those systems up to Release 6.2.
Conventions Used in This Book
Example commands that should be typed at a command line will be proceeded by a percent sign (‘%’)
and be presented in a boldface courier font:
% sample command
Example command output and example contents of ASCII files will be presented in a courier font:
sample file line 1
sample file line 2
Any text preceded by a pound sign (‘#’) should be considered comment lines:
# This is a comment
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 14
Chapter 1. Release 6.2
This chapter summarizes HPSS changes for Release 6.2 into four categories: new features, retired
features, deferred features, and changed features. Changes since release 4.5 and 5.1 are described.
1.1. New Features
This section describes the new HPSS features added to Release 6.2.
1.1.1. DCE Replacement
Previous HPSS releases used DCE to provide authenticated client/server remote procedure calls.
Since DCE is no longer offered as a commercial product, it has been replaced in HPSS with a
solution based on commercially available authentication services and ONC RPC technology.
The new HPSS infrastructure provides four primary sub-systems: POSIX Threads, Universal Unique
Identifiers (UUIDs), ONC Remote Procedure Calls and a collection of security services. The security
services include support for Kerberos and an LDAP based registry service. For more information on
the new HPSS infrastructure, see Section 2.3.4: HPSS Infrastructure on page 44.
1.1.2. Linux Support
All HPSS servers are fully supported on Linux.
1.1.3. Security
The HPSS Security implementation, which was based on DCE security, has been replaced with Unix
or MIT Kerberos authentication and with Unix or LDAP authorization. Refer to Chapter 2: Security
and System Access of the HPSS Management Guide and Section 3.2.7: Security on page 57 for more
information.
1.1.4. SCSI PVR
Support for SCSI connected tape libraries has been added to HPSS 6.2 The SCSI PVR can be used to
control any SCSI command based tape library.
1.1.5. HPSS VFS Interface
The HPSS VFS Interface provides a POSIX I/O interface to the HPSS filesystem. The root of the
HPSS directory tree or a subdirectory can be mounted as a filesytem. HPSS files can be accessed by
any software that complies with the POSIX I/O API by using the HPSS VFS Interface. For more
information, please refer to Section 13.4: HPSS VFS Interface Configuration of the HPSS
Management Guide.
1.1.6. GridFTP Interface
The GridFTP interface uses the PIO interface to transfer data into/from HPSS.
1.1.7. SAN3P/PIO Support
PIO can be used to issue I/O requests to direct attached HPSS SAN3P disk cache.
1.1.8. Additional hpssadm Configuration Options
• Class of Service (COS)
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 15
• Storage Class (but not subsystem-specific storage class options)
• Global configuration
• Accounting policy
• Location policy
• All server configuration. Newly supported options include Core Server, Gatekeeper, Location
Server, Log Daemon, Migration/Purge Server, PVL, all PVRs, and SSM. The Mover, Log
Client, and Startup Daemon were previously supported and still are.
• Modifying devices and drives. This is the equivalent functionality of modifying fields on the
device info and drive info screens in the GUI.
1.1.9. Additional hpssadm operations
• Import/export
• Create/delete resources
• Task monitoring. A new "task" command displays the status of any long running background
tasks submitted during the session, such as imports or resource creates.
• Specifying volumes from a file for import/export/delete/move. hpssadm, like the GUI, now
allows the user to submit a file listing the target volumes for import, export, delete resources,
and cartridge move operations. This is an alternative to the previous capability of building the
list on the screen with the Fill Count, Fill Increment, and Volume Label fields, which is still
provided. As a result of adding this capability, the group labels on the structures have
changed so scripts which use the old method to specify the volume list will need a slight
modification. See the hpssadm man page for examples.
1.1.10. Additional Library and Device Support
• Spectra Logic library
• ADIC Scalar i500 library
• IBM and HP LTO Generation 3 and 4 drives. Tape encryption has not been certified with
HPSS.
• SAIT-1 drives
• IBM TS1120 (3592 Generation 2 and 3) drives. Tape encryption has not been certified with
HPSS.
• Sun StorageTek T10000 drives
• Sun StorageTek 9840D drives
• DataDirect Networks (DDN} disks
1.1.11. SAN Virtual Volume ID Mapping
A feature to consistently map device pathnames to volumes. A unique ID is assigned to each volume.
The IDs are then mapped to device pathnames.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 16
1.1.12. Drive Pools
HPSS provides HPSS end clients the ability to direct tape read I/O requests to a predefined group of
tape drives referred to as a Drive Pool. This ability helps HPSS administrators manage tape drive
scheduling and thus availability. For more information, please refer to Section 7.3: Drive Pools of the
HPSS Management Guide.
1.1.13. FTP Enhancement
• FTP application mode: FTP can be compiled as either a 32-bit or a 64-bit application, by
setting the BIT64_SUPPORT flag on or off in the Makefile.macros file, when building with
the HPSS FTP source extracted with "build-ftp" option. The 64-bit Kerberos libraries will be
required to support Kerberos authentication.
• PFTP client is now supported on Solaris 10 (Intel/AMD only).
1.1.14. mkhpss Enhancement
A new option is provided by mkhpss to configure the DB2 log mirroring capability to better protect
the DB2 transaction log files. DB2 log mirroring allows the database to write an identical second
copy of log files to a different path. The DB2 log files are essential in protecting a database and
enable transactions occurring after a database backup to be recovered. Loss of the DB2 log files can
result in significant impact to the operation of the system, extensive downtime, and potential loss of
data. It is vital that DB2 log files be stored on highly reliable disk systems. In addition, DB2 log
mirroring must be used to protect the DB2 log files on two separate disk systems. The disk systems
must also be protected using RAID1 or RAID5 configurations. This mkhpss enhancement can only
be used for new HPSS installations. For existing sites, please consult with your HPSS Support
Representative for assistance.
1.1.15. DB2 Monitoring
A new feature to have the HPSS DB2 Log Monitor periodically checks the DB2 transaction log files
to make sure that the primary and mirror logs are congruent and performing normally. It also scans
the DB2 diagnostic log for indications of problem with the DB2 log files. For each identified
problem, it will issue an HPSS alarm and indicate the path of the diagnostic log and the line number
where the error is reported. The administrator should then open the DB2 diagnostics file and
examine the reported problem in detail.
1.1.16. File Family enhancements
The number of allowed file families is increased to 4,294,967,295. In addition, authorized callers can
assign the File Family ID of a file via the hpss_FileSetAttributes Client API function, independently
of the underlying fileset.
1.1.17. SSM Configuration Files Consolidation
The SSM configuration files ssm_krb5.conf and ssm_unix.conf have been consolidated into a single
configuration file called ssm.conf. Users may continue using their existing configuration file by
specifying the "-m" option to the hpssadm.pl or hpssgui.pl script, or they may rename their existing
configuration file to ssm.conf. For environments with multiple authentication mechanisms, it may be
desirable to maintain multiple configuration files.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 17
1.1.18. Mover Enhancement
Multiple Movers for one or more HPSS instances can now be configured to run on the same machine.
The -c <alternate var path> flag is added to the Mover entry in the inetd configuration file to specify
an alternate "/var/hpss" path to be used by the Mover. In addition, the -s flag is added to enable
syslog logging for the Mover. This feature is intended for multiple HPSS test systems to share the
same set of Mover machines.
1.2. Retired Features
• HPSS is no longer supported fully on Solaris. HPSS Mover and HPSS clients are still being
supported on Solaris.
• The settapestats utility program is no longer supported. The 6.2 Core Server calculates tape
volume statistics when it starts.
• STK RAIT PVR is no longer supported.
• LTO PVR is no longer supported on Linux. The SCSI PVR should be used for LTO drives on
Linux.
• Non-DCE Gateway is no longer supported.
• Federated Name Space and Remote Site Policy are not supported in 6.2.
• Mirrored Filesets are no longer supported.
• The terminology of "Non-DCE" Movers is no longer being used. It is now referred to as the
Mover.
• The terminology of "Non-DCE" Client API is no longer being used. It is now referred to as
the Client API.
• The hpss_PVrr utility is no longer supported. The source code for this utility can be found
in /opt/hpss/tools/unsupported.
• The hpss.clean command is no longer supported.
• The HPSS Metadata Backup Tools are no longer supported.
1.3. Deferred Features
XFS is not supported in HPSS 6.2. XFS references have been left in the HPSS documentation to
support the option of re-enabling XFS in future releases.
1.4. HPSS Changes
This section describes the changes made to HPSS 6.2.
1.4.1. Documentation Organization Changes
The SSM Guide is now merged with the Management Guide.
1.4.2. Metadata Changes
• Added the AUTHZACL table to maintain security ACLs for HPSS servers and for SSM client
access to SSM.
• Deleted the DISKSPACE table. The disk space allocation maps are now maintained in the
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 18
Core Server's memory image
• Modified the DMG table. In support of the new HPSS RPC library, the TCP port was
eliminated and the Program and Version numbers were added to DMG specific configuration.
• Modified the DMGFILESET table. The TCP port was eliminated and the TCP hostname and
RPC endpoint information was added.
• Modified the GATEKEEPER table. The length of site policy pathname was changed from 127
to 1023 characters.
• Modified the LSPOLICY table. The group name was eliminated and realm name was added.
Changed length of local site name from 31 to 255 characters.
• Modified the MOVERDEVICE table. In support of the new SAN3P capabilities, SanID was
added to mover device configuration. Since this is new to HPSS 6.2, these will be set to 0 for
all mover devices.
• The NDCG table is now obsolete. It has been renamed, but left intact.
• Modified the NFS table. In support of the new authentication mechanisms, the credential
object ID has been eliminated. Privileged caller principal length was changed from 15 to 255
characters.
• SSSTATS table is now obsolete. It has been renamed, but left intact.
• Modified the SERVER table. In support of the new HPSS RPC library, the authorization
service and authentication service information has been eliminated. Request queue size
information, RPC program, version number, and authentication mechanism information has
been added, along with new index definitions.
• Added the SERVERINTERFACES table. This table is populated by the metadata converter
programs with newly created server interface information.
• Modified the SITE table. The descriptive and LS group names have been eliminated, and site
and realm names were added. The pre-6.2 SITE table will be renamed to PRE62_SITE but the
metadata in the table will not be converted into the new SITE table since remote sites are no
longer supported.
• Modified the STORAGESEGDISK table. A new index is required for this table to support
the new disk space allocation mechanism.
1.4.3. DMAP Gateway Changes
Since DFS is no longer supported, the logic to support DFS in the DMAP Gateway has been
removed. DMAP Gateway will only support XFS in 6.2. Changes were also made in the underlying
RPC mechanism used for communication with the HDM. The ability to delete the DMAP Gateway
and Core Server components of an XDSM Fileset have been removed. Changes made to the SSM
windows to support XFS are described below.
X F S i s n o t s u p p o r t e d i n H P S S 6 . 2 . X F S r e f e r e n c e s h a v e b e e n l e f t i n t h e H P S S d o c u m e n t a t i o n t o s u p p o r t t h e o p t i o n o f r e - e n a b l i n g X F S i n f u t u r e r e l e a s e s .
1.4.3.1. Creating an XDSM Fileset
• The “Create HPSS/XDSM Fileset” window has been renamed ”Create XDSM Fileset”. This
window is currently not accessible since XFS is not supported.
• Fields specific to DFS fileset creation have been removed. Since HPSS managed XFS
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 19
filesystems do not support mirrored namespaces, fields which were specific to managing
mirrored filesets are also no longer available. This includes the following configuration
options:
· Global Mount Point
· Local Mount Point
· Fileset Owner
· Fileset Permissions
• Communication between the HDM and the DMAP Gateway now uses the HPSS RPC library.
Fields which were used to specify TCP communication information have been removed.
1.4.3.2. Viewing DMAP Gateway XDSM Fileset Information
• Fields and functions which were needed for DFS or mirrored fileset management have been
removed. This includes the following changes:
· The ability to change the name of an XDSM Fileset.
· The ability to change the fileset state (i.e.: mounted, pre-unmounted etc.).
· The "Name Server Object Handle of Fileset Mountpoint" data is no longer shown.
· The ability to delete the DMAP Gateway portion of an XDSM Fileset independently from
the core server is no longer supported.
• Fileset information and associated statistics are now shown on a single window. The
"Requests via TRPC (HPSS) Interface" statistics were specific to mirrored filesets and have
been removed.
• The RPC endpoint for the HDM is shown instead of the "HPSS/DMAP TCP Hostname" and
"HPSS/DMAP TCP Port" fields. This reflects the change in the underlying communication
mechanism used between the HDM and the DMAP Gateway. Additionally, these fields may
no longer be modified from the DMAP Gateway Fileset Information window.
1.4.3.3. Viewing Core Server XDSM Fileset Information
• The name of this window has been changed from "Name Server Fileset Information" to "Core
Server Fileset Information".
• The ability to modify the following fields from this window is no longer allowed:
· Fileset ID
· Fileset Name
· User and Group ID
· Fileset Type
· Permissions
• The following informational fields have been removed:
· Name Server Object Handle
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 20
1.4.4. SSM Changes
Significant changes were made in SSM between Releases 4.5 and 5.1 and again between 5.1 and 6.2.
For the reader's convenience, all changes between 4.5 and 6.2 are summarized in Section 3.3.4.1:
Changes Affecting Sites Upgrading Directly from 4.5. Changes between 5.1 and 6.2 are summarized
in Section 3.3.4.2: Changes Affecting Sites Upgrading from 5.1. The tables indicate whether each
change affects the SSM Server (the System Manager), the SSM GUI hpssgui, and/or the HPSS
command line utility hpssadm. Sites need consult only the table which pertains to them.
1.4.4.1. Changes Affecting Sites Upgrading Directly from 4.5
Changes since 4.5 Server GUI ADM
The SSM System Manager and Data Server were consolidated in release
6.2 into a single server, the SSM System Manager.
n
Java is now required for the SSM Graphical User Interface (hpssgui). Java
is no longer required by the SSM server (the new consolidated SSM
System Manager).
n n
SSL is no longer supported with SSM. This means it is no longer
necessary to create an SSL certificate (the /var/hpss/ssm/ds.cer file) or
import it into the certificate authority file ($JAVA_ROOT/jre/lib/cacerts)
on each SSM client machine.
n n n
The Java Security Manager is no longer used with SSM. This means the
Java Policy files are no longer required. In 4.5, these files were the
java.policy.ds and the java.policy.hpssadm.
n n n
The hpssadm.jar and mobjects.jar files are no longer used. All classes are
now in a single jar file hpss.jar.
n n
The start_ssm script is no longer available. The SSM System Manager is
started with the rc.hpss script. The rc.hpss script has been moved from a
default location of /etc to /opt/hpss/bin.
n
The start_ssm_session script is no longer available. In HPSS 6.2, the SSM
GUI is started with the hpssgui startup script. There are perl and vbs
versions of this script, named with an extension to indicate the type of
script: hpssgui.pl and hpssgui.vbs.
n
There are new versions of the hpssadm startup scripts in perl and vbs.
These are named with an extension to indicate the type of script:
hpssadm.pl and hpssadm.vbs. The ksh and bat scripts are no longer
supported.
n
Options to the new hpssgui startup scripts differ considerably from the
options to the old start_ssm_session script. Options to the hpssadm startup
scripts have also changed significantly. See the man pages for details.
Following are some highlights:
n n
The scripts are dependent upon an SSM configuration file
(ssm.conf), created by mkhpss, which contains some site-specific
configuration values. The SSM client script will attempt to ready
the SSM configuration file in the current working directory unless
the pathname to this file is specified by the -m option.
n n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 21
Changes since 4.5 Server GUI ADM
The SSM client scripts now use an internal polling mechanism for
getting window updates (as opposed to being notified of the update
by the server). This means that:
• The SSM client application no longer requires a second
port for two-way communication,
• The rate of polling can be fine-tuned by the user (see
the -A, -G, -L -M, -W options).
n n
The scripts are dependent upon a login.conf file and, if using
Kerberos authentication, a krb5.conf file. Both files are created by
hpssuser. The scripts will look in the current working directory for
these files unless they are specified on the command line (-C and k options).
n n
It is no longer necessary to add a security provider entry to the java
system java.security file.
n n
The pathname to the java executable can be specified by using the
-j option. If this option isn't specified, then the script will use the
java executable in $JAVA_BIN if it is valid. If $JAVA_BIN/java
is not defined, then the script will use 'java' which will use the
client's $PATH.
n n
Effective with HPSS 5.1.1, the hpssgui script supports customizing
the user's background color or Look & Feel using the -b, -F and -T
options.
n
The SSM user can specify the number of alarms to display on the
alarm list using the -N option.
n n
The SSM user can specify the pathname to the hpss.jar using the -P
option.
n n
The SSM user can specify the path to search for application
preferences using the -i option. This path is now local to the SSM
client machine; preferences are no longer stored on the SSM server
machine.
n n
Sites that are converting from a previous release of HPSS should replace
all their hpssgui and hpssadm startup scripts on all client platforms with
the new versions.
n n
In HPSS 6.2, ssh tunneling communications between the SSM client and
server is supported.
n n n
The SSM clients can contact the System Manager across a Virtual Private
Network connection (VPN). See the -p and -h options on the hpssgui and
hpssadm man pages.
n n n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 22
Changes since 4.5 Server GUI ADM
The SSM client scripts can use ports exempted by the network
administrator as firewall exceptions. See the -n option on the
hpssadm/hpssgui man pages.
The port on which the System Manager will listen may be controlled by
setting the $HPSS_SSM_SERVER_LISTEN_PORT environment variable.
The default setting is 0 (zero) which means that the port will be chosen by
the RPC portmapper. See Section 3.3.6: Using SSM Through a Firewall,
of the HPSS Management Guide.
n n n
Access to port 88 is needed for Kerberos authentication. Access to port
111 is needed if the RPC portmapper is needed to find the System
Manager.
n n n
The user_authorization.dat and hpssadm.config files provided in HPSS 4.5
are no longer available.
In 6.2, all authorized SSM users and their privilege levels are defined in
the DB2 user acl AUTHZACL table. See Section 3.3.2.1: hpssuser Utility
and Section 3.3.2.2: SSM User Authorization in the HPSS Management
Guide for details.
n n n
The hpssgui and hpssadm programs will now optionally record every error
and informational message in an ASCII log file. The "-S" option for both
programs is used to specify whether to create the session log and what to
name it.
n n
The basic and specific server config structures, along with the server's log
policy, have been combined in 5.1 into a single structure.
n n
SSM now represents the three basic Core Server volume structures, the
physical volume, virtual volume, and storage map, as a single combined
Core Server volume structure.
n n
The delog utility is no longer available via the SSM GUI. This feature has
been retired.
n
The ability to broadcast messages to other SSM GUI users is not yet
available.
n
Alarms and events are no longer acknowledged from the SSM GUI.
n
The SSM clients can specify the number of items to display for the Alarms
and Events list. See the -N option on the hpssadm/hpssgui man page and
Section 9.6: Managing SSM Alarms and Events in the HPSS Management
Guide.
n n
The SSM clients keep an internal cached copy of the Alarms and Events
list which is maintained by regular polling requests to the System
Manager. Each polling request returns only “new” messages, those which
have not already been retrieved by the SSM client in a previous polling
request. If there are no new messages in the System Manager log cache,
no messages are returned. This will help performance of the display of
this window. See the -A, -G and -N options of the hpssadm/hpssgui man
pages and Section 9.6.5: Controlling SSM Log Message handling in the
HPSS Management Guide.
n n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 23
Changes since 4.5 Server GUI ADM
The ability to see which users are logged into SSM, referred to in HPSS
4.5 as a list of SSM "consoles", is available in 6.2 as part of the System
Manager Statistics window from both the hpssgui and hpssadm.
n n
The menu bar has been reorganized extensively.
The "Set Keyboard", "Show Sammi Environment", "View Sammi Errlog",
and "About Sammi" items from the 4.5 Session menu and the "SSM
consoles" item from the 4.5 Monitor menu are no longer available.
Some of the information about the gui session and data server which was
formerly available from these screens is now available in 6.2 from the
"System Manager Statistics" and "User Session Information" items from
the "SSM Information" submenu of the "Monitor" menu. Some
information is also available in the new user session log. See Chapter 3:
Using SSM in the HPSS Management Guide for a full description of the
layout of the menu and a description of the functions available from each
menu item.
n
The Monitor, Operations, and Configure menus are available in a menu
tree on the bottom half of the Health and Status window. The tree is
organized exactly as the corresponding items from the menu bar, and the
functionality of each item is the same as its corresponding item on the
menu bar.
n
The View menu of the Health and Status screen allows some portions of
the screen to be hidden or redisplayed.
n
The PVR Cartridge Threshold is included in the HPSS Status section of
the Health and Status information. When the status is something other
than OK, the PVR Information button on the GUI screen will become
activated; clicking this button will bring up a window for each PVR
cartridge that is not healthy.
n n
To support users with color blindness or other visual problems, all text is
now displayed on white or light gray backgrounds. Color (green, yellow,
red) is still used to indicate the severity level of alarms and component
statuses, but the color is confined to a small sphere displayed next to the
text. For example, a major alarm might be displayed on the Alarms and
Events screen as:
● May 2, 2005 11:26:13 AM Major PVL Bad things happened
n
More than one preference may now be defined for each list. See the
description of the "preferences combo box" in Section 3.6: Common
Window Elements of the HPSS Management Guide for details.
n
Note that the mechanism for selecting columns to be displayed in each list
window has moved from the list's Preferences window to the "Column
View" menu of the list window itself. See the description of the "Column
View" menu item in Section 3.6:Common Window Elements of the HPSS
Management Guide for details.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 24
Changes since 4.5 Server GUI ADM
Column ordering is now controlled by dragging columns to the
desired location. The modified order is preserved automatically in the
user preferences across hpssgui restarts.
n
When messages have been written to the status bar, the most recent
messages can be viewed in the status bar's tooltip. Rolling the mouse over
the status bar without clicking gives a tooltip that says, "Click mouse in
status bar to view messages" if there are status messages to view. If there
are no status messages then the tooltip says, "No status messages". This
message stays up for about 4 seconds or until the user moves the mouse
out of the status bar area. To view up to the last STATUS_MSG_MAX
(30) messages that have been written to the status bar, click on the status
bar. The tooltip that is displayed will show up to the last 30 messages and
will remain visible until the mouse is moved out of the status bar or for 10
minutes.
n
Most user updates to fields on information windows are no longer sent to
the server immediately or automatically. The Update button on the
window must be clicked to submit the update to the server. Fields
modified by the user will be flagged by a diskette icon to indicate the
window copy has been modified but not saved.
n
The list of volumes for import, export, resource deletes, and cartridge
move operations may now be specified from an external input file as an
alternative to building the list on the hpssgui screen or within the hpssadm
operation.
n n
Since the basic and specific server configuration structures have been
combined into a single structure, there is no longer a need for a
-server_type option to the hpssadm config command. This option is no
longer used.
The types of configuration structures supported by the -type option now
are specified by the title names used on the config struct windows. See the
hpssadm man page for a list of supported config types.
Any type name may be abbreviated so long as the abbreviation is unique.
See the hpssadm man page for details.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 25
Changes since 4.5 Server GUI ADM
The HPSS 4.5 hpssadm commands
• pvr_cartridge
• pvl_volume
• ss_pv
• ss_map
• ss_vv
have been replaced by a single command "volume". The volume command
has a required option "-type" for which these types may be specified:
• PVL Volume Information
• PVR Cartridge Information
• CS Disk Volume
• CS Tape Volume
The new volume command behaves similarly to the config command,
where the types are the titles of the window screens for the structures. The
type names may be abbreviated so long as the abbreviation is unique.
The volume command has several subcommands which support displaying
and updating the volume, importing and exporting volumes, creating and
deleting resources, and moving cartridges.
There is no longer a separate command for locking or unlocking a volume
in 6.2. This has been replaced by the practice of setting the VV Condition
and Retired fields structure on the storage map in the disk or tape volume
structure. (This is a core server design change.) The fields of the map
structure may be modified with the update command. The update
command may also be used on any of the volume commands to modify any
settable field.
n
A new hpssadm command "ssm" is supplied in 6.2 for shutting down the
System Manager and for displaying the status of the System Manager and
of the hpssadm session.
n
All available columns of each SSM list (server, device, storage class, pvl
job, alarm) are now displayed by the hpssadm list commands in the system
default order. At this time hpssadm has no facility for customizing the list
format as the gui does.
n
Field layout and grouping for each structure displayed by the hpssadm now
matches the order and grouping used by the corresponding window of the
SSM GUI.
n
Most hpss structures have changed so hpssadm scripts used before 5.1 may
need to be modified to use the new field and subfield names.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 26
Changes since 4.5 Server GUI ADM
The hpssadm config command now supports the additional structures:
• Class of Service Config
• Storage Class (but not subsystem-specific storage class options)
• Global Config
• Accounting Policy
• Location Policy
• All server configurations; servers which were not supported before are
the core server, gatekeeper, location server, log daemon,
migration/purge server, pvl, all pvrs, and ssm. The mover, log client
and startup daemon are still supported as previously.
n
A new command “task” displays the status of any long running background
tasks submitted during the session, such as imports or resource creates.
n
A new subcommand “update” has been added to the hpssadm device
command for updating the mover device and pvl drive objects.
n
1.4.4.2. Changes Affecting Sites Upgrading from 5.1
Changes since 5.1 Server GUI ADM
The SSM System Manager and Data Server were consolidated in release
6.2 into a single server, the SSM System Manager.
n
Java is no longer used in the server side of SSM (the new consolidated
SSM System Manager).
n
SSL is no longer supported with SSM. This means it is no longer
necessary to create an SSL certificate (the /var/hpss/ssm/ds.cer file) or
import it into the certificate authority file ($JAVA_ROOT/jre/lib/cacerts)
on each ssm client machine.
n n n
The Java Security Manager is no longer used with SSM. This means the
Java policy files are no longer required. In 5.1, these files were the
java.policy.ds and the java.policy.ssmuser.
n n n
The start_ssm script is no longer available. The SSM System Manager is
started with the rc.hpss script. The rc.hpss script has been moved from a
default location of /etc to /opt/hpss/bin.
n
There are new versions of the hpssgui and hpssadm startup scripts in perl
and vbs. These are named with an extension to indicate the type of script:
hpssadm.pl, hpssadm.vbs, hpssgui.pl and hpssgui.vbs. The ksh and bat
scripts are no longer supported.
The scripts are no longer distributed as templates; they will be packaged
into the $HPSS_PATH_BIN (default /opt/hpss/bin) directory.
n n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 27
Changes since 5.1 Server GUI ADM
Options to the hpssgui and hpssadm startup scripts have changed
significantly. See the man pages for details. Following are some
highlights:
n n
The scripts are dependent upon an ssm configuration file
(ssm.conf), created by mkhpss, which contains some site-specific
configuration values. The SSM client script will attempt to read
the ssm configuration file in the current working directory unless
the pathname to this file is specified by the -m option.
n n
The SSM client scripts now use an internal polling mechanism for
getting window updates (as opposed to being notified of the update
by the server). This means that:
• the SSM client application no longer requires a second
port for two-way communication,
• using SSM across an ssh tunnel is simpler since you
now need to set up the tunnel only in one direction,
• the rate of polling can be fine-tuned by the user (see the
-A, -G, -L -M, -W options).
n n
The scripts are dependent upon a login.conf file and, if using
Kerberos authentication, a krb5.conf file. The hpssuser program
creates both of these files. The scripts will look in the current
working directory for these files unless they are specified on the
command line (-C and -k options).
n n
It is no longer necessary to add a security provider entry to the java
system java.security file.
n n
The pathname to the java executable can be specified by using the
-j option. If this option isn't specified, then the script will use the
java executable in $JAVA_BIN if it is valid. If $JAVA_BIN/java
is not defined, then the script will use 'java' which will use the
client's $PATH.
n n
Effective with HPSS 5.1.1, the hpssgui script supports customizing
the user's background color or Look & Feel using the -b, -F and -T
options. Sites which did not apply the 5.1 patch would not have
had this functionality.
n
The SSM user can specify the number of alarms to display on the
alarm list using the -N option.
n n
The SSM user can specify the pathname to the hpss.jar using the P option.
n n
The SSM user can specify the path to search for application
preferences using the -i option. This path is now local to the SSM
client machine; preferences are no longer stored on the SSM server
machine.
n n
Sites which are converting from a previous release of HPSS should be
certain to replace all their hpssgui and hpssadm startup scripts on all client
platforms with the new versions.
n n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 28
Changes since 5.1 Server GUI ADM
The SSM client script options for connecting to the System Manager
across a Virtual Private Network connection (VPN) have changed. See the
-p and -h options on the hpssgui and hpssadm man pages.
n n n
The SSM client script option for using ports exempted by the network
administrator as firewall exceptions has changed; see the -n option on the
hpssadm/hpssgui man pages.
The port on which the System Manager will listen may be controlled by
setting the $HPSS_SSM_SERVER_LISTEN_PORT environment variable.
The default setting is 0 (zero) which means that the port will be chosen by
the RPC portmapper. See Section 3.3.6:Using SSM Through a Firewall,
of the HPSS Management Guide.
n n n
Access to port 88 is needed for Kerberos authentication and access to port
111 is needed if the RPC portmapper is needed to find the System
Manager.
n n n
The ssmuser.config file provided in HPSS 5.1 is no longer available. In
6.2, all authorized SSM users and their privilege levels are defined in the
DB2 user acl AUTHZACL table. See Section 3.3.2.1: hpssuser Utility and
Section 3.3.2.2: SSM User Authorization in the HPSS Management Guide
for details.
n n n
SSM now represents the three basic Core Server volume structures, the
physical volume, virtual volume, and storage map, as a single combined
Core Server volume structure.
n n
Access to the delog utility via the SSM GUI, deferred in 5.1, has now been
permanently retired.
n
The ability to broadcast messages to other SSM GUI users is not yet
available.
n
The SSM clients can specify the number of items to display for the Alarms
and Events list. See the -N option on the hpssadm/hpssgui man page and
Section 9.6: Managing SSM Alarms and Events in the HPSS Management
Guide.
n n
The SSM clients keep an internal cached copy of the Alarms and Events
list which is maintained by regular polling requests to the System
Manager. Each polling request returns only “new” messages, those which
have not already been retrieved by the SSM client in a previous polling
request. If there are no new messages in the System Manager log cache,
no messages are returned. This will help performance of the display of
this window. See the -A, -G and -N options of the hpssadm/hpssgui man
pages and Section 9.6.5: Controlling SSM Log Message Handling in the
HPSS Management Guide.
n n
The ability to see which users are logged into SSM, referred to in HPSS
4.5 as a list of SSM "consoles", was not yet available in 5.1. It is now
available in 6.2 as part of the System Manager Statistics window from both
the hpssgui and hpssadm.
n n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 29
Changes since 5.1 Server GUI ADM
The menu bar has been reorganized slightly.
The "Data Server Statistics" menu item has been replaced by the "System
Manager Statistics" menu item, available from the Monitor->SSM
Information menu path.
"Column View" was added to the menu bar for SSM windows that display
an SSM table. See Section 3.6: Common Window Elements of the HPSS
Management Guide. In HPSS 5.1, this ability was part of the Preference
window associated with the SSM table window.
The “User Session Log” is no longer available from the SSM GUI; the
user can still view this information by using the -S option of the SSM
client script and viewing the ASCII text of the session log. Since this was
removed from the menu bar, the Monitor->Logging menu path which
contained two sub-items (Log Files Info and User Session Log) was
replaced by Monitor->Log Files Information menu path.
The Monitor->Lookup HPSS Objects->Security menu path option has
been removed.
The Operations->Filesets & Junctions->Create XDSM Fileset menu path
option was added. However, this option is currently hidden since XFS is
not supported.
Since the two SSM Servers (Data Server and System Manager) were
merged into the single System Manager. all references to the Data Server
in the Operations->Shutdown menu were removed.
Configure->Restricted Users menu path option was added. It currently
supports listing the restricted users and a “Reload List” button which
causes the Core Servers to reread the HPSS_RESTRICTED_USER_FILE.
See Chapter 2: Security and System Access in the HPSS Management
Guide for a full description of restricting user access to HPSS.
Since the SSM Guide was merged into the HPSS Management Guide and
HPSS Installation Guide, the Help->HPSS Books->SSM Guide menu path
option was removed.
See Chapter 3: Using SSM in the HPSS Management Guide for a full
description of the layout of the menu and a description of the functions
available from each menu item.
n
The PVR Cartridge Threshold is included in the HPSS Status section of
the Health and Status information. When the status is something other
than OK, the PVR Information button on the GUI screen will become
activated; clicking this button will bring up a window for each PVR
cartridge that is not healthy.
n n
The "Sick" preference may no longer be edited.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 30
Changes since 5.1 Server GUI ADM
The mechanism for selecting columns to be displayed in each list window
has moved from the list's Preferences window to the "Column View" menu
of the list window itself. See the description of the "Column View" menu
item in Section 3.6: Common Window Elements of the HPSS Management
Guide for details.
n
Column ordering is now controlled solely by dragging columns to the
desired location. The modified order is now preserved automatically in the
user preferences across hpssgui restarts.
n
List tables have a field that shows the number of displayed and total items
in the list in the format X/Y where X is the number of items displayed and
Y is the total number of items in the list. The field is left justified under
the table. The X and Y values will differ if preferences are set to filter
some items out of the list.
n
The button panel to the right of the list can be hidden or displayed by
clicking the tall, thin button between the list and button panel labeled '||'. If
the button is pressed when the panel is displayed, the button panel will
hide, allowing more space for the list. The button panel may be redisplayed by pressing the '||' button again.
n
Sites which are converting from HPSS 5.1 should remove all old user
preference files. New preference files will be created automatically for
the user when he first logs in. The user will be required to reset any
preference settings (i.e. preferences will not be converted).
n
The System Manager connection indicator is not displayed on every
window; it is only displayed on the Health and Status Window.
n
When messages have been written to the status bar, the most recent
messages can be viewed in the status bar's tooltip. Rolling the mouse over
the status bar without clicking gives a tooltip that says, "Click mouse in
status bar to view messages" if there are status messages to view. If there
are no status messages then the tooltip says, "No status messages". This
message stays up for about 4 seconds or until the user moves the mouse
out of the status bar area. To view up to the last 30
(STATUS_MSG_MAX) messages that have been written to the status bar,
click on the status bar. The tooltip that is displayed will show up to the last
30 messages and will remain visible until the mouse is moved out of the
status bar or for 10 minutes.
n
The list of volumes for import, export, resource deletes, and cartridge
move operations may now be specified from an external input file as an
alternative to building the list on the hpssgui screen or within the hpssadm
operation.
n n
Most hpss structures have changed so hpssadm scripts used before 5.1 may
need to be modified to use the new field and subfield names.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 31
Changes since 5.1 Server GUI ADM
Since SSM now represents the three Core Server volume structures as a
single structure, the types specified to the hpssadm volume command have
changed. The 5.1 types:
• Disk Storage Map Information
• Disk Physical Volume Information
• Disk Virtual Volume Information
have been replaced in 6.2 by the single type:
• CS Disk Volume
n
Similarly, the 5.1 types:
• Tape Storage Map Information
• Tape Physical Volume Information
• Tape Virtual Volume Information
have been replaced in 6.2 by the single type:
• CS Tape Volume
n
The hpssadm volume command now supports volume import and export,
resource creation and deletion, and cartridge moves.
n
On the Active Storage Classes window, the “Force Migrate” has been
replaced with a “Migration Controls” drop-down allowing the
Administrator or Operator to perform migration operations on all the
selected storage classes. Likewise, the “Force Purge” has been replaced
with a “Purge Controls” drop-down allowing the Administrator to perform
purge operations on all the selected storage classes.
n
The "-ds" option is no longer available for the ssm "info" or "shutdown"
commands. A new option "-sm" is available for the "info" command.
n
The hpssadm config command now supports the additional structures:
• Class of Service Config
• Storage Class (but not subsystem-specific storage class options)
• Global Config
• Accounting Policy
• Location Policy
• All server configurations; servers which were not supported before are
the core server, gatekeeper, location server, log daemon,
migration/purge server, pvl, all pvrs, and ssm. The mover, log client
and startup daemon are still supported as previously.
n
A new hpssadm command “task” displays the status of any long running
background tasks submitted during the session, such as imports or resource
creates.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 32
Changes since 5.1 Server GUI ADM
A new subcommand “update” has been added to the hpssadm device
command for updating the mover device and pvl drive objects.
n
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 33
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 34
Chapter 2. HPSS Basics
2.1. Introduction
The High Performance Storage System (HPSS) provides hierarchical storage management and
services for very large storage environments. HPSS may be of interest to organizations having present
and future scalability requirements that are very demanding in terms of total storage capacity, file
sizes, data rates, number of objects stored, and numbers of users. HPSS is part of an open, distributed
environment based on remote procedure calls, Kerberos, LDAP directory systems and DB2 which
form the infrastructure of HPSS. HPSS is the result of a collaborative effort by leading US
Government supercomputer laboratories and industry to address very real, urgent high-end storage
requirements. HPSS is offered commercially by IBM.
HPSS provides scalable parallel storage systems for highly parallel computers as well as traditional
supercomputers and workstation clusters. Concentrating on meeting the high end of storage system
and data management requirements, HPSS is scalable and designed to store up to petabytes (1015) of
data and to use network-connected storage devices to transfer data at rates up to multiple gigabytes
(109) per second.
HPSS provides a large degree of control for the customer to manage the hierarchical storage system.
Using configuration information defined by the customer, HPSS organizes storage devices into
multiple storage hierarchies. Based on policy information defined by the customer and actual usage
information, data are then moved to the appropriate storage hierarchy and to appropriate levels in the
storage hierarchy.
2.2. HPSS Capabilities
A primary goal of HPSS is to move large files between storage devices and parallel or clustered
computers at speeds many times faster than today’s commercial storage system software products and
to do this in a way that is more reliable and manageable than is possible with current systems. In
order to accomplish this goal, HPSS is designed and implemented based on the concepts described in
the following subsections.
2.2.1. Network-centered Architecture
The focus of HPSS is the network, not a single processor as in conventional storage systems. HPSS
provides servers that can be distributed across a high performance network to provide scalability and
parallelism. The basis for this architecture is the IEEE Mass Storage System Reference Model,
Version 5.
2.2.2. High Data Transfer Rate
HPSS achieves high data transfer rates by eliminating overhead normally associated with data
transfer operations. In general, HPSS servers establish and control transfer sessions but are not
involved in actual transfer of data.
2.2.3. Parallel Operation
The HPSS Client Application Program Interface (Client API) supports parallel or sequential access to
storage devices by clients executing parallel or sequential applications. HPSS also provides a Parallel
File Transfer Protocol. HPSS can even manage data transfers in a situation where the number of data
sources differs from the number of destination sources. Parallel data transfer is vital in situations that
demand fast access to very large files.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 35
2.2.4. Based on Standard Components
HPSS runs on UNIX and is written in ANSI C and Java. It uses remote procedure calls, a selectable
security service (Kerberos or UNIX), UNIX or LDAP for user configuration information, and DB2 as
the basis for its portable, distributed, transaction-based architecture. These components are offered on
many vendors’ platforms.
The full HPSS system has been implemented on IBM AIX and LINUX platforms, and some
components of HPSS have been ported to other platforms. Refer to Section 2.4: HPSS Hardware
Platforms on page 48 and Section 3.3: Prerequisite Software Considerations on page 58 for specific
information.
To aid vendors and users in porting HPSS to new platforms, HPSS source code is available.
2.2.5. Data Integrity Through Transaction Management
Transactional metadata management, provided by DB2, enables a reliable design that protects user
data both from unauthorized use and from corruption or loss. A transaction is an atomic grouping of
metadata management functions such that either all of them take place together or none of them takes
place. Journaling makes it possible to back out any partially complete transactions if a failure occurs.
Transaction technology is common in relational data management systems but not in storage systems.
It is the key to maintaining reliability and security while scaling upward into a large, distributed
storage environment.
2.2.6. Multiple Hierarchies and Classes of Services
Most other storage management systems support simple storage hierarchies consisting of one kind of
disk and one kind of tape. HPSS provides multiple configurable hierarchies, which are particularly
useful when inserting new storage technologies over time. As new disks or tapes are added, new
classes of service can be set up. HPSS files reside in a particular class of service which users select
based on parameters such as file size and performance. A class of service is implemented by a storage
hierarchy which in turn consists of multiple storage classes, as shown in Figure 2. Storage classes are
used to logically group storage media to provide storage for HPSS files. A hierarchy may be as
simple as a single tape, or it may consist of two or more levels of disk and local tape. The user can
even set up classes of service so that data from an older type of tape is subsequently migrated to a
new type of tape. Such a procedure allows migration to new media over time without having to copy
all the old media at once.
2.2.7. Storage Subsystems
Storage subsystems (or simply, “subsystems”) may be used to increase the scalability of HPSS in
handling concurrent requests or to meet local political needs. Each subsystem contains a single Core
Server. If migration and purge are needed for the subsystem, then it will also contain a Migration /
Purge Server. In addition, if HPSS is to be used as a backing store for a 'linked' filesystem such as
XFS, a DMAP Gateway will be required. All other servers are subsystem-independent.
Data stored within HPSS is assigned to different subsystems based on pathname resolution. A
pathname consisting of ‘/’ resolves to the root Core Server which is specified in the global
configuration file. However, if the pathname contains junction components, it may resolve to a Core
Server in a different subsystem. For example, the pathname ‘/JunctionToSubsys2/mydir’ could
lead to a fileset managed by the Core Server in subsystem 2. Sites which do not wish to partition their
HPSS through the use of subsystems will run HPSS with a single subsystem.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 36
2.3. HPSS Components
The components of HPSS include files, filesets, junctions, virtual volumes, physical volumes, storage
segments, metadata, servers, infrastructure, user interfaces, a management interface, and policies.
Media and file metadata are represented by data structures that describe the attributes and
characteristics of storage system components such as files, filesets, junctions, storage segments, and
volumes. Servers are the processes that control the logic of the system and control movement of the
data. The HPSS infrastructure provides the services that are used by all the servers for operations
such as sending messages and providing reliable transaction management. User interfaces provide
several different views of HPSS to applications with different needs. The management interface
provides a way to administer and control the storage system and implement site policy.
These HPSS components are discussed below in Sections 2.3.1 through 2.3.7.
Figure 1. File Migration and Stage Operations
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 37
Figure 2. Class of Service / Hierarchy / Storage Class
2.3.1. HPSS Files, Filesets, Volumes, Storage Segments and Related
Metadata
The various metadata constructs used to describe the HPSS namespace and HPSS storage are
described below:
• Files (Bitfiles). Files in HPSS, called bitfiles in deference to IEEE Mass Storage Reference
Model terminology, are logical strings of bytes, even though a particular bitfile may have a
structure imposed by its owner. This unstructured view decouples HPSS from any particular
file management system that host clients of HPSS might use. HPSS bitfile size is limited to2
64
- 1 bytes.
Each bitfile is identified by a machine-generated name called a bitfile ID. It may also have a
human readable name. It is the job of the HPSS Core Server (discussed in Section 2.3.2) to
map a human readable name to a bitfile's ID.
• Filesets. A fileset is a logical collection of files that can be managed as a single administrative
unit, or more simply, a disjoint directory tree. A fileset has two identifiers: a human readable
name and a numeric fileset ID. Both identifiers are unique to a given realm.
• Junctions. A junction is a Core Server object, much like a symbolic link to a directory, that is
used to point to a fileset. This fileset may belong to the same Core Server or to a different
Core Server. When pointing to a different Core Server, junctions allow HPSS users to traverse
to different subsystems.
• File Families. HPSS files can be grouped into families. All files in a given family are
recorded on a set of tapes assigned to the family. Only files from the given family are
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 38
recorded on these tapes. HPSS supports grouping files on tape volumes only. Families can
only be specified by associating the family with a fileset. All files created in the fileset belong
to the family. When one of these files is migrated from disk to tape, it is recorded on a tape
with other files in the same family. If no tape virtual volume is associated with the family, a
blank tape is reassigned from the default family. The family affiliation is preserved when
tapes are repacked.
• Physical Volumes. A physical volume is a unit of storage media on which HPSS stores data.
The media can be removable (e.g., cartridge tape, optical disk) or non-removable (magnetic
disk). Physical volumes may also be composite media, such as RAID disks, but must be
represented by the host OS as a single device.
Physical volumes are not visible to the end user. The end user stores bitfiles into a logically
unlimited storage space. HPSS, however, must implement this storage on a variety of types
and quantities of physical volumes.
For a list of the tape physical volume types supported by HPSS, see Table 2 in Chapter 3.
• Virtual Volumes. A virtual volume is used by the Core Server to provide a logical abstraction
or mapping of physical volumes. A virtual volume may include one or more physical volumes.
Striping of storage media is accomplished by the Core Servers by collecting more than one
physical volume into a single virtual volume. A virtual volume is primarily used inside of
HPSS, thus hidden from the user, but its existence benefits the user by making the user’s data
independent of device characteristics. Virtual volumes are organized as strings of bytes up to
2
64-1
bytes in length that can be addressed by an offset into the virtual volume.
• Storage Segments. A storage segment is an abstract storage object which is mapped onto a
virtual volume. Each storage segment is associated with a storage class (defined below) and
has a certain measure of location transparency. The Core Server (discussed in Section 2.3.2)
uses both disk and tape storage segments as its primary method of obtaining and accessing
HPSS storage resources. Mappings of storage segments onto virtual volumes are maintained
by the HPSS Core Servers.
• Storage Maps. A storage map is a data structure used by the Core Server to manage the
allocation of storage space on virtual volumes.
• Storage Classes. A storage class defines a set of characteristics and usage parameters to be
associated with a particular grouping of HPSS virtual volumes. Each virtual volume and its
associated physical volumes belong to a single storage class in HPSS. Storage classes in turn
are grouped to form storage hierarchies (see below). An HPSS storage class is used to
logically group storage media to provide storage for HPSS files with specific intended usage,
similar size and usage characteristics.
• Storage Hierarchies. An HPSS storage hierarchy defines the storage classes on which files in
that hierarchy are to be stored. A hierarchy consists of multiple levels of storage, with each
level representing a different storage class. Files are moved up and down the hierarchy via
migrate and stage operations based on usage patterns, storage availability, and site policies.
For example, a storage hierarchy might consist of a fast disk, followed by a fast data transfer
and medium storage capacity robot tape system, which in turn is followed by a large data
storage capacity but relatively slow data transfer tape robot system. Files are placed on a
particular level in the hierarchy depending upon the migration levels that are associated with
each level in the hierarchy. Multiple copies are controlled by this mechanism. Also data can
be placed at higher levels in the hierarchy by staging operations. The staging and migrating of
data is shown in Figure 1: File Migration and Stage Operations.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 39
• Class of Service (COS). Each bitfile has an attribute called Class Of Service. The COS
defines a set of parameters associated with operational and performance characteristics of a
bitfile. The COS results in the bitfile being stored in a storage hierarchy suitable for its
anticipated and actual size and usage characteristics. Figure 2: Class of
Service/Hierarchy/Storage Class shows the relationship between COS, storage hierarchies,
and storage classes.
2.3.2. HPSS Servers
HPSS servers include the Core Server, Migration/Purge Server, Gatekeeper, Location Server, Log
Client, Log Daemon, Physical Volume Library, Physical Volume Repository, Mover, Storage System
Manager, and, possibly, a Data Migration Gateway. Figure 3: HPSS Components provides a
simplified view of the HPSS system. Each major server component is shown, along with the basic
control communications paths (thin arrowed lines). Infrastructure items (those components that “glue
together” the distributed servers) are shown at the top of the cube. These infrastructure items are
discussed in Section 2.3.4: HPSS Infrastructure on page 44. HPSS user interfaces (the clients listed
in the figure) are also discussed in Section 2.3.5: HPSS User Interfaces on page 47.
Figure 3. HPSS Components
• Core Server. The Core Server provides several key sets of functionality.
First, the Core Server provides translation between human-oriented names and HPSS object
identifiers. Name space objects managed by the Core Server are filesets, junctions, directories,
files, hard links, and symbolic links. The Core Server provides access verification to objects
and mechanisms for manipulating access to these objects via a Portable Operating System
Interface (POSIX) view of the name space. This name space is a hierarchical structure
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 40
consisting of directories, files, and links. These name space objects may exist within filesets
that are connected via junctions.
Second, the Core Server provides the abstraction of logical bitfiles to its clients. A bitfile is
identified by a Core Server generated name called a bitfile ID. Clients may reference portions
of a bitfile by specifying the bitfile ID and a starting address and length. The Core Server
supports random access to files and sparsely written files. It supports parallel reading and
writing of data to bitfiles and performs the mapping of logical portions of bitfiles onto
physical storage devices. The Core Server supports the migration, purging, and staging of data
in a storage hierarchy (though the migration/purge policies are implemented through the
Migration/Purge Server, a client to the Core Server).
Third, the Core Server provides a hierarchy of storage objects: storage segments, virtual
volumes, and physical volumes. The Core Server translates storage segment references into
virtual volume references and then into physical volume references, handles the mapping of
physical resources into striped virtual volumes to allow parallel I/O to that set of resources,
and schedules the mounting and dismounting of removable media through the Physical
Volume Library (see below).
• Migration/Purge Server (MPS). The MPS allows a site to implement its storage management
policies by managing the placement of data on HPSS storage media using site-defined
migration and purge policies. By making appropriate calls to its Core Server, an MPS copies
data to lower levels in the hierarchy (migration), removes data from the current level once
copies have been made (purge), or moves data between volumes at the same level (lateral
move). Based on the hierarchy configuration, MPS can be directed to create duplicate copies
of data when it is being migrated from disk or tape. This is done by copying the data to
multiple lower levels in the storage hierarchy.
There are three types of migration: disk migration, tape file migration, and tape volume
migration. The designation disk or tape refers to the type of storage class that migration is
running against. See Section 3.7.2: Migration/Purge Server on page 83 for a more complete
discussion of the different types of migration.
MPS runs migration on each storage class periodically using the time interval specified in the
migration policy for that class. See Section 2.3.7: HPSS Policy Modules on page 47 for
details on migration and purge policies. Migration runs can be started automatically when the
warning or critical space thresholds for the storage class are exceeded. In addition, migration
runs can be started manually by an administrator.
Purge runs are started automatically on each storage class when the free space in that class
falls below the percentage specified in the purge policy. Purge runs may also be started
manually.
Disk Migration/Purge:
The purpose of disk migration is to make one or more copies of disk files to lower levels in
the hierarchy. The number of copies depends on the configuration of the hierarchy. For disk,
migration and purge are separate operations. It is common for disk storage class which have
been configured for migration to also be configured for purge as well. Once a file has been
migrated (copied) downwards in the hierarchy, it becomes eligible for purge, which
subsequently removes the file from the higher level and allows the disk space to be reused.
Tape File Migration:
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 41
The purpose of tape file migration is to make an additional copy (or multiple additional
copies) of a file, in a tape storage class, to a lower level in the hierarchy. It is also possible to
move files downwards instead of copying them. In this case there is no duplicate copy
maintained. There is no separate purge component to tape file migration. Empty volumes
must be reclaimed using the reclaim utility.
Tape Volume Migration:
The purpose of tape volume migration is to free tape volumes for reuse. Tape volumes are
selected based on being in the EOM map state and containing the most unused space (caused
by users overwriting or deleting files). The remaining segments on these volumes are either
migrated downwards to the next level in the hierarchy, or are moved laterally to another tape
volume at the same level. This results in empty tape volumes which may then be reclaimed.
Note that there is no purge component to tape volume migration. All of the operations use a
move instead of a copy semantic.
• Gatekeeper (GK). The Gatekeeper provides two main services:
· It provides sites with the ability to schedule the use of HPSS resources using the
Gatekeeping Service.
· It provides sites with the ability to validate user accounts using the Account Validation
Service.
Both of these services allow sites to implement their own policy.
The default Gatekeeping Service policy is to not do any gatekeeping. Sites may choose to
implement a policy for monitoring authorized callers, creates, opens, and stages. The Core
Server will call the appropriate GK API depending on the requests that the site-implemented
policy is monitoring.
The Account Validation Service performs authorizations of user storage charges. A site may
perform no authorization, default authorization, or site-customized authorization depending
on how the Accounting Policy is set up and whether or not a site has written site-specific
account validation code. Clients call this service when creating files, changing file
ownership, or changing accounting information. If Account Validation is enabled, the
Account Validation Service determines if the user is allowed to use a specific account or
gives the user an account to use, if needed. The Core Server also calls this service to perform
an authorization check just before account-sensitive operations take place.
• Location Server (LS). The Location Server acts as an information clearinghouse to its clients
through the HPSS Client API to enable them to locate servers and gather information from
both local and remote HPSS systems. Its primary function is to allow a client to determine a
server's location and, by knowing other information about the server such as its object UUID,
determine its server type or its subsystem id. This allows a client to contact the appropriate
server. Usually the Location Server is only used by the Core Server or the Gatekeeper.
• Physical Volume Library (PVL). The PVL manages all HPSS physical volumes. It is in
charge of mounting and dismounting sets of physical volumes, allocating drive and cartridge
resources to satisfy mount and dismount requests, providing a mapping of physical volume to
cartridge and of cartridge to Physical Volume Repository (PVR), and issuing commands to
PVRs to perform physical mount and dismount actions. A primary function of the PVL is the
support for atomic mounts of sets of cartridges for parallel access to data. Atomic mounts are
implemented by the PVL, which waits until all necessary cartridge resources for a request are
available before issuing mount commands to the PVRs.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 42
• Physical Volume Repository (PVR). PVRs manage HPSS cartridges. Though an HPSS
system may contain multiple PVRs, each cartridge is managed by only one. PVRs provide
APIs for clients to request cartridge mounts and dismounts and query the status of cartridges.
For convenience, PVRs are often configured in one-to-one correspondence to tape libraries.
For information on the types of tape libraries supported by HPSS PVRs, see Section 3.4.2:
Robotically Mounted Tape on page 62.
An Operator PVR is provided for cartridges not under control of a robotic library. These
cartridges are mounted on a set of drives by operators.
• Mover (MVR). The purpose of the Mover is to transfer data from a source device to a sink
device. A device can be a standard I/O device with geometry (e.g., tape, disk) or a device
without geometry (e.g., network, memory). The MVR’s client (typically the Core Server)
describes the data to be moved and where the data is to be sent. It is the MVR’s responsibility
to actually transfer the data, retrying failed requests and attempting to optimize transfers. The
MVR supports transfers for disk devices, tape devices and a mover protocol that can be used
as a lightweight coordination and flow control mechanism for large transfers.
• Storage System Manager (SSM). SSM is the tool used by the system administrator to manage
HPSS. It enables the administrator to configure, monitor and control the resources (servers,
devices, tape libraries, and media) of HPSS in ways that conform to the management policies
of a given customer site.
Monitoring capabilities include the ability to query the values of important management
attributes of storage system resources and the ability to receive notifications of alarms and
other significant system events. Controlling capabilities include the ability to start up and shut
down servers and the ability to set the values of management attributes of storage system
resources and storage system policy parameters. Additionally, SSM can request that specific
operations be performed on resources within the storage system, such as adding and deleting
logical or physical resources. Operations performed by SSM are usually accomplished
through standard HPSS Application Program Interfaces (APIs).
SSM has three components, one of which is an HPSS server and two of which are user
interface client programs. The server is:
• SSM System Manager. Communicates with all other HPSS components requiring
monitoring or control.
The user interface clients are:
• SSM GUI (hpssgui) - Provides the HPSS administrator or operator the ability to
configure or monitor the HPSS System through a graphical user interface.
• SSM Command Line Interface (hpssadm) - Provides the HPSS administrator or
operator the ability to configure or monitor a subset of the HPSS system through a set
of interactive or batch commands.
• Data Migration Gateway (DMG). The DMG is only used if a connection to a XDSM fileset
is desired. If a site wishes to have HPSS act as a backing-store for a file system such a XFS,
that site will need the services of a DMG. The DMG helps to coordinate the movement of data
between these so-called linked filesets and HPSS.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 43
2.3.3. HPSS Storage Subsystems
The goal of storage subsystems (or just “subsystems”) is to increase the scalability of HPSS by
allowing multiple Core Servers to be used within a single HPSS system. Every HPSS system is
partitioned into one or more subsystems. Each subsystem contains a single Core Server. If migration
and purge are needed, then the subsystem should contain a single Migration/Purge Server. Each Core
Server and each Migration/Purge Server must exist within a storage subsystem. Each subsystem may
optionally be serviced by a Gatekeeper which performs site-specific user-level scheduling of HPSS
storage requests or account validation. Each Gatekeeper may service multiple subsystems. If a
subsystem wishes to utilize XDSM filesets, a DMAP Gateway must also be configured. Only one
DMAP Gateway is needed for multiple subsystems. All other servers exist independently of storage
subsystems. Sites which do not need multiple Core Servers use a single storage subsystem.
The computer that runs the Core Server for subsystem X is referred to as the “Subsystem X node”,
while the computer running the Root Core Server is known as the “Root Subsystem Node”.
Each HPSS system consists of two types of DB2 databases. The global database contains subsystemindependent data, and a subsystem database contains subsystem-dependent data. An HPSS system
has exactly one global database and one or more subsystem databases.
The definitions of classes of service, hierarchies, and storage classes apply to the entire HPSS system
(they are subsystem-independent). All classes of service, hierarchies, and storage classes are known
to all storage subsystems within HPSS. The level of resources dedicated to these entities by each
storage subsystem may differ. It is possible to disable selected classes of service within given storage
subsystems. Although the class of service definitions are global, if a class of service is disabled
within a storage subsystem then the Core Server in that subsystem never selects that class of service.
Since the Migration/Purge Server (MPS) is contained within the storage subsystem, migration and
purge operate independently in each subsystem. Each Migration/Purge Server is responsible for
migration and purge for those storage class resources contained within its particular storage
subsystem. Migration and purge runs are independent and are not synchronized. Migration and purge
for a storage class may be configured differently for each storage subsystem. It is possible to set up a
single migration or purge policy which applies to a storage class across all storage subsystems (to
make configuration easier), but it is also possible to control migration and purge differently in each
storage subsystem.
Similarly, storage class thresholds may be configured differently for each storage subsystem. It is
possible to set up a single set of thresholds which apply to a storage class across all storage
subsystems, but it is also possible to control the thresholds differently for each storage subsystem.
2.3.4. HPSS Infrastructure
The HPSS infrastructure items (see Figure 3) are those components and services used by the various
HPSS servers. The HPSS infrastructure components common among servers are discussed below.
• Remote Procedure Calls (RPC). Most HPSS servers, with the exception of the MVR,
PFTPD, and logging services (see below), communicate requests and status (control
information) via RPCs. HPSS does not use RPCs to move user data. RPCs provide a
communication interface resembling simple, local procedure calls.
• Thread Services. HPSS uses a threads package for multitasking. The threads package is vital
for HPSS to serve large numbers of concurrent users and to enable multiprocessing of its
servers.
• Transaction Management. Requests to perform actions, such as creating bitfiles or
accessing file data, result in client-server interactions between software components. The
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 44
HPSS Core Server performs most of the HPSS metadata changes using the transaction
management tools provided by DB2. For the most part, these metadata transactions are
managed entirely within the Core Server. Other servers such as MPS and PVL modify their
metadata transactionally, and those transactions are entirely contained within those servers. A
very small number of rarely performed operations require distributed transaction management,
and these are handled by DB2 as well.
Transactional integrity to guarantee consistency of server state and metadata is required in
HPSS in case a particular component fails. HPSS metadata updates utilize the transactional
capability of DB2. The selection of DB2 was based on functionality and vendor platform
support. It provides HPSS with an environment in which a job or action completes
successfully or is aborted completely.
DB2 provides a full suite of recovery options for metadata transactions. Recovery of the
database to a consistent state after a failure of HPSS or DB2 is automatic. A full suite of
database backup and maintenance tools is provided as well.
• Security. HPSS security software provides mechanisms that allow HPSS components to
communicate in an authenticated manner, to authorize access to HPSS objects, to enforce
access control on HPSS objects, and to issue log records for security-related events. The
security components of HPSS provide authentication, authorization, enforcement, and audit
capabilities for the HPSS components. Customer sites may use the default security policy
delivered with HPSS or define their own security policy by implementing their own version of
the security policy module.
• Authentication — is responsible for guaranteeing that a principal (a customer identity)
is the entity that is claimed, and that information received from an entity is from that
entity.
• Authorization — is responsible for enabling an authenticated entity access to an
allowed set of resources and objects. Authorization enables end user access to HPSS
directories and bitfiles.
• Enforcement — is responsible for guaranteeing that operations are restricted to the
authorized set of operations.
• Audit — is responsible for generating a log of security-relevant activity. HPSS audit
capabilities allow sites to monitor HPSS authentication, authorization, and file
security events. File security events include file creation, deletion, opening for I/O,
and attribute modification operations.
HPSS components that communicate with each other maintain a joint security context. The
security context for both sides of the communication contains identity and authorization
information for the peer principals as well as an optional encryption key.
Access to HPSS server interfaces is controlled through an Access Control List (ACL)
mechanism. Membership on this ACL is controlled by the HPSS administrator.
• Logging. A logging infrastructure component in HPSS provides an audit trail of server
events. Logged data includes alarms, events, requests, security audit records, status records,
and trace information. The Log Client, which may keep a temporary local copy of logged
information, communicates log messages to a central Log Daemon, which in turn maintains a
central log. Depending on the type of log message, the Log Daemon may send the message to
the SSM for display purposes. When the central HPSS log fills, messages are sent to a
secondary log file. A configuration option allows the filled log to be automatically archived to
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 45
HPSS. A delog function is provided to extract and format log records from a central or
archived log file. Delog options support filtering by time interval, record type, server, and
user.
• Accounting. The HPSS accounting system provides the means to collect usage information in
order to allow a particular site to charge its users for the use of HPSS resources. It is the
responsibility of the individual site to sort and use this information for subsequent billing
based on site-specific charging policies. For more information on the HPSS accounting policy,
refer to Section 2.3.7: HPSS Policy Modules on page 47.
2.3.5. HPSS User Interfaces
As indicated in Figure 3, HPSS provides the user with a number of transfer interfaces as discussed
below.
• File Transfer Protocol (FTP). HPSS provides an industry-standard FTP user interface.
Because standard FTP is a serial interface, data sent to a user is received serially. This does
not mean that the data within HPSS is not stored and retrieved in parallel; it means that the
Parallel FTP Daemon within HPSS must consolidate its internal parallel transfers into a serial
data transfer to the user. HPSS FTP performance in many cases will be limited not by the
speed of a single storage device but by the speed of the data path between the HPSS Parallel
FTP Daemon and the user’s FTP client.
• Parallel FTP (PFTP). The PFTP supports standard FTP commands plus extensions and is
built to optimize performance for storing and retrieving files from HPSS by allowing data to
be transferred in parallel across the network media. The parallel client interfaces have a
syntax similar to FTP but with numerous extensions to allow the user to transfer data to and
from HPSS across parallel communication interfaces established between the PFTP client and
the HPSS Movers. This provides the potential for using multiple client nodes as well as
multiple server nodes. PFTP supports transfers via TCP/IP. The PFTP client establishes a
control connection with the HPSS Parallel FTP Daemon and subsequently establishes TCP/IP
data connections directly with HPSS Movers to transfer data at rates limited only by the
underlying media, communications hardware, and software.
• Client Application Program Interface (Client API). The Client API is an HPSS-specific
programming interface that mirrors the POSIX.1 specification where possible to provide ease
of use to POSIX application programmers. Additional APIs are also provided to allow the
programmer to take advantage of the specific features provided by HPSS (e.g., storage/access
hints passed on file creation and parallel data transfers). The Client API is a programming
level interface. It supports file open/create and close operations; file data and attribute access
operations; file name operations; directory creation, deletion, and access operations; and
working directory operations. HPSS users interested in taking advantage of parallel I/O
capabilities in HPSS can add Client API calls to their applications to utilize parallel I/O. For
the specific details of this interface see the HPSS Programmer’s Reference Guide, Volume 1.
• HPSS VFS Interface. The HPSS VFS Interface presents a standard POSIX I/O interface to a
user application. This obviates the need for a user application to be rewritten against the
HPSS Client API and hence can be used “out of the box” as long as the user application is
POSIX compliant. A portion of an HPSS directory tree can be mounted on a client machine
as if it were a local POSIX-compliant filesystem.
2.3.6. HPSS Management Interfaces
HPSS provides a graphical user interface, the SSM hpssgui, for HPSS administration and operations
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 46
GUI. The hpssgui simplifies the management of HPSS by organizing a broad range of technical data
into a series of easy-to-read graphic displays. The hpssgui allows monitoring and control of virtually
all HPSS processes and resources from windows that can easily be added, deleted, moved, or
overlapped as desired.
HPSS also provides a command line SSM interface, hpssadm. This tool does not provide all the
functionality of the hpssgui, but does implement a subset of its frequently used features, such as some
monitoring and some control of servers, devices, storage classes, volumes, and alarms. It is useful for
performing HPSS administration from remote locations where network traffic is slow or difficult.
Additionally, hpssadm provides some rudimentary mass configuration support by means of the ability
to issue configuration commands from a batch script.
In addition to SSM, HPSS provides a number of command line utilities for specialized management
purposes, such as listing the volumes managed by a particular PVR or core server. See the HPSS
Management Guide Chapter 14: Management Tools for more information. See also the HPSS man
pages for descriptions of these utilities.
2.3.7. HPSS Policy Modules
There are a number of aspects of storage management that probably will differ at each HPSS site. For
instance, sites typically have their own guidelines or policies covering the implementation of
accounting, security, and other storage management operations. In order to accommodate site-specific
policies, HPSS has implemented flexible interfaces to its servers to allow local sites the freedom to
tailor management operations to meet their particular needs.
HPSS policies are implemented using two different approaches. Under the first approach, used for
migration, purge, and logging policies, sites are provided with a large number of parameters that may
be used to implement local policy. Under the second approach, HPSS communicates information
through a well-defined interface to a policy software module that can be completely replaced by a
site. Under both approaches, HPSS provides a default policy set for users.
• Migration Policy. The migration policy defines the conditions under which data is copied
from one level in a storage hierarchy to one or more lower levels. Each storage class that is to
have data copied from that storage class to a lower level in the hierarchy has a migration
policy associated with it. The MPS uses this policy to control when files are copied and how
much data is copied from the storage class in a given migration run. Migration runs are started
automatically by the MPS based upon parameters in the migration policy.
Note that the number of copies which migration makes and the location of these copies is
determined by the definition of the storage hierarchy and not by the migration policy.
• Purge Policy. The purge policy defines the conditions under which data that has already been
migrated from a disk storage class can be deleted. Purge applies only to disk storage classes. It
is common, but not necessary, for disk storage classes which have a migration policy to also
have a purge policy. Purge runs are started automatically by the MPS based upon parameters
in the purge policy.
• Logging Policy. The logging policy controls the types of messages to log. On a per server
basis, the message types to write to the HPSS log may be defined. In addition, for each server,
options to send Alarm, Event, or Status messages to SSM may be defined.
• Security Policy. Security policy defines the authorization and access controls to be used for
client access to HPSS. HPSS security policies are provided to control access (authentication)
from FTP and/or Parallel FTP using Username/Password, Ident, or Kerberos credentials.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 47
HPSS provides facilities for recording information about authentication and object
(file/directory) creation, deletion, access, and authorization events. The security audit policy
for each server determines the records that each individual server will generate. All servers
can generate authentication records.
• Accounting Policy. The accounting policy provides runtime information to the accounting
report utility and to the Account Validation service of the Gatekeeper. It helps determine what
style of accounting should be used and what level of validation should be enforced.
The two types of accounting are site-style and UNIX-style. The site-style approach is the
traditional type of accounting in use by most mass storage systems. Each site will have a sitespecific table (Account Map) that correlates the HPSS account index number with their local
account charge codes. The UNIX-style approach allows a site to use the user identifier (UID)
for the account index. The UID is passed along in UNIX-style accounting just as the account
index number is passed along in site-style accounting.
Account Validation allows a site to perform usage authorization of an account for a user. It is
turned on by enabling the Account Validation field of the Accounting Policy configuration
screen. If Account Validation is enabled, the accounting style in use at the site is determined
by the Accounting Style field. A site policy module may be implemented by the local site to
perform customized account validation operations. The default Account Validation behavior
is performed for any Account Validation operation that is not overridden by the site policy
module.
• Location Policy. The location policy defines how Location Servers at a given site will
perform, especially in regards to how often server location information is updated. All local,
replicated Location Servers update information according to the same policy.
• Gatekeeping Policy. The Gatekeeper provides a Gatekeeping Service along with an Account
Validation Service. These services provide the mechanism for HPSS to communicate
information though a well-defined interface to a policy software module that can be written by
a site. The site policy code is placed in well-defined shared libraries for the gatekeeping
policy and the accounting policy (/opt/hpss/lib/libgksite.[a|so] and /opt/hpss/lib/libacctsite.[a|
so] respectively) which are linked to the Gatekeeper. The Gatekeeping policy shared library
contains a default policy which does NO gatekeeping. Sites will need to enhance this library
to implement local policy rules if they wish to monitor and load-balance requests.
2.4. HPSS Hardware Platforms
2.4.1. Server Platforms
HPSS requires at least one AIX or Linux node for the core server components. A server node must
have sufficient processing power and memory to handle the work load.
2.4.2. Client Platforms
The full-function Client API can be ported to any platform that supports UNIX.
The PFTP client code and Client API source code for platforms other than AIX and Linux are not on
the HPSS distribution image. Maintenance of the PFTP and Client API software on platforms other
than AIX and Linux is the responsibility of the customer, unless a support agreement is negotiated
with IBM. Contact your HPSS Support Representative for information on how to obtain the needed
software.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 48
The following matrix illustrates which platforms support HPSS interfaces.
Table 1. HPSS Client Interface and Mover Platforms
Platform PFTP Client Client API
HPSS
Mover
HPSS VFS
Client
FTP Clients
IBM AIX X X X Any platform running
standard FTP clients. GUIbased Clients may not
function correctly for
some commands.
Sun Solaris
(Big Endian ONLY)
X X X
Digital UNIX X
Not Tested
Hewlett-Packard HPUX X
Not Tested
Silicon Graphics IRIX
(32-bit)
X X
Compaq Tru64 X
Not Tested
Linux (Intel) X X X X
Linux (Power PC) X X
2.4.3. Mover Platforms
Movers are used to control the logical network attachment of storage devices and are configured to
run on one or more nodes. A Mover consists of two parts: The Mover administrative process that
runs on the server node, and the remote Mover process that handles the HPSS devices and data
transfers. Movers can run on AIX, Linux, IRIX, and Solaris systems. See Table 1 above for a
detailed list of supported platforms.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 49
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 50
Chapter 3. HPSS Planning
3.1. Overview
This chapter provides HPSS planning guidelines and considerations to help the administrator
effectively plan, and make key decisions about, an HPSS system.
The planning process for HPSS must be done carefully to ensure that the resulting system
satisfies the site’s requirements and operates in an efficient manner. We recommend that the
administrator read this entire chapter before planning the system.
The following paragraphs describe the recommended planning steps for the HPSS installation,
configuration, and operational phases.
3.1.1. HPSS System Architecture
Before getting into the details of storage sizing, it is best to understand the overall HPSS system and
how the components are arranged to build the HSM. Figure 4: HPSS Generic Configuration on page
52 shows the basic architecture of an HPSS system including disk and tape resources and their
relationship to HPSS server nodes, Mover nodes, internal and external networks, and SAN
interconnections. Specifics of this architecture for a given site are developed during the proposal and
initial project planning stages of a deployment. Ideally the required space is derived from
requirements gathered from the HPSS Questionnaire document, known operational constraints,
transfer and transaction rates, and anticipated storage capacity needs. Often the disk and tape
resources are dictated by current equipment already available and budgetary constraints on what can
be purchased. Specific quantities and sizing of these resource are beyond the scope of this planning
document. These are largely defined by the above inputs and negotiations during the initial planning
meeting in which the systems engineering team draws from experience and similar deployments to
design a working architecture that balances the end-user requirements with the potential, or actual,
resources available.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 51
Figure 4. HPSS Generic Configuration
3.1.2. HPSS Configuration Planning
Before beginning the planning process, there is an important issue to consider. HPSS was designed to
optimize the transfer of large files at the expense of some small file transfer performance. If at all
possible, try to reduce the number of small files that are introduced into your HPSS system. For
example, if you plan to use HPSS to backup all of the PCs in your organization, it would be best to
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 52
aggregate the individual files into large individual files before moving them into the HPSS name
space.
The following planning steps must be carefully considered for the HPSS infrastructure configuration
and the HPSS configuration phases:
1. Identify the site’s storage requirements and policies, such as the initial storage system size,
anticipated growth, usage trends, average file size, expected throughput, backup policy, and
availability. For more information, see Section 3.2: Requirements and Intended Uses for
HPSS on page 56.
2. Define the architecture of the entire HPSS system to satisfy the above requirements. The
planning should:
• Identify the nodes to be configured as part of the HPSS system.
• Identify the disk and tape storage devices to be configured as part of the HPSS system
and the nodes/networks to which each of the devices will be attached. Storage devices
can be assigned to a number of nodes to allow data transfers to utilize the devices in
parallel without being constrained by the resources of a single node. This capability
also allows the administrator to configure the HPSS system to match the device
performance with the network performance used to transfer the data between the
HPSS Movers and the end users (or other HPSS Movers in the case of internal HPSS
data movement for migration and staging). Refer to Section 3.4 : Hardware
Considerations on page 62 for more discussions on the storage devices and networks
supported by HPSS.
• Identify the HPSS subsystems to be configured and how resources will be allocated
among them. Refer to Section 3.8: Storage Subsystem Considerations on page 94 for
more discussion on subsystems.
• Identify the HPSS servers to be configured and the node where each of the servers
will run. Refer to Section 3.7: HPSS Server Considerations on page 80 for more
discussions on the HPSS server configuration.
• Identify the HPSS user interfaces (e.g. FTP, PFTP, Client API) to be configured and
the nodes where the components of each user interface will run. Refer to Section 3.6:
HPSS Interface Considerations on page 79 for more discussion on the user interfaces
supported by HPSS.
3. Ensure that the prerequisite software has been obtained, installed, and configured properly in
order to satisfy the target HPSS architecture. Refer to Section 3.3: Prerequisite Software
Considerations on page 58 for more information on the HPSS prerequisite software
requirements.
4. Determine the DB2 disk storage space needed to satisfy the requirements of the HPSS
system, and verify there is sufficient free space in the file systems to meet those needs. Refer
to Section 3.5.4: HPSS Metadata Space on page 74 for more discussions of metadata sizing
requirements.
5. Verify that each of the identified nodes has sufficient resources to handle the work loads to
be imposed on the node. Refer to Section 3.5.5: System Memory and Disk Space on page 78
for more discussions on the system resource requirements.
6. Plan the design of the HPSS storage characteristics and HPSS storage space to satisfy the
site’s requirements:
• Plan for file families, if any. Refer to Section 3.10.4: File Families on page 112 for
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 53
more information about configuring families.
• Plan for filesets and junctions, if any. Refer to Chapter 10: Filesets and Junctions in
the HPSS Management Guide for more information.
• Plan for HPSS storage classes. Refer to Section 3.10.1: Storage Class on page 102 for
more information on the storage class configuration.
• Plan for HPSS storage hierarchies. Refer to Section 3.10.2: Storage Hierarchy on
page 109 for more information on the storage hierarchy configuration.
• Plan for HPSS classes of service. Refer to Section 3.10.3: Class of Service on page
110 for more information on the Class of Service configuration.
• Plan the migration and purge policy for each storage class. Refer to Section 3.9.1:
Migration Policy on page 94 and Section 3.9.2: Purge Policy on page 95 for more
information.
• Determine the amount of user data storage space needed for each storage class. Refer
to Section 3.5.1: HPSS User Storage Space on page 69 for more information on the
HPSS storage space considerations.
• Identify the disk and tape media to be imported into HPSS.
7. Define the location policy to be used. Refer to Section 3.9.6: Location Policy on page 99 for
more information.
8. Define the accounting policy to be used. Refer to Section 3.9.3: Accounting Policy and
Validation on page 96 for more information on the Accounting Policy configuration.
9. Define the logging policy for the each of the HPSS servers. Refer to Section 3.9.5: Logging
Policy on page 99 for more information on the Logging Policy configuration.
10. Define the security policy for the HPSS system. Refer to Section 3.9.4: Security Policy on
page 98 for more information on the Security Policy for HPSS.
11. Determine if a Gatekeeper will be required. It is required if a site wants to do Account
Validation or Gatekeeping. Refer to Section 3.9.3: Accounting Policy and Validation on page
96 and Section 3.9.7: Gatekeeping on page 99 for more information.
3.1.3. Purchasing Hardware and Software
It is recommended that hardware be purchased only after the HPSS configuration has been planned.
Purchasing the hardware prior to the planning process may result in performance and utilization
issues that could easily be avoided by advance planning.
If deciding to purchase Sun, SGI, or Linux servers for storage purposes, note that the operating
system limitations will only allow a fixed number of raw devices to be configured per logical unit
(disk drive or disk array). Sun's Solaris operating system currently allows only eight partitions per
logical unit (one of which is used by the system) unless the optional volume manager is used. SGI's
IRIX operating system currently allows only sixteen partitions per logical unit. Linux operating
system limits SCSI disks to 15 partitions and limits IDE disks to 63 partitions unless LVM is used.
These limits can potentially impact the utilization of a disk drive or disk array.
Refer to Section 3.5: HPSS Sizing Considerations on page 68 for more information on calculating the
number and size of devices that will be needed to meet your requirements.
Refer to Section 3.3: Prerequisite Considerations on page 58 for more information on the required
software that will be needed to run HPSS.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 54
3.1.4. HPSS Operational Planning
The following planning steps must be carefully considered for the HPSS operational phase:
1. Define the site guidelines for the HPSS users and SSM users.
• Each HPSS user who uses the storage services provided by HPSS should be assigned
an Accounting ID and one or more appropriate Classes of Service (COS) to store files.
• Each SSM user (administrator or operator) should be assigned an appropriate SSM
security level. The SSM security level defines what functions each SSM user can
perform on HPSS through SSM. Refer to Section 2.3: SSM User Security, Section
3.3.2: Creating the SSM User Accounts, and Section 12.1.1.6: Add an SSM User ID of
the HPSS Management Guide for more information on setting up the security level for
an SSM user.
2. Define the site procedures for repacking and reclaiming HPSS tape volumes. Define the tape
consolidation and reuse goals. For instance, define a tape utilization factor and plan to
repack those tapes that fall below that limit. The limits can be set on a per Storage Class
basis. Also decide when, or if empty tapes will be reclaimed. Refer to Section 8.1.4:
Repacking Tape Virtual Volumes and Section 8.1.5: Reclaiming HPSS Tape Virtual Volumes
(both in the HPSS Management Guide) for more information.
3. Define the site policy and procedure for generating the accounting reports. Take into
consideration how often an accounting report needs to be generated, how the accounting
information from the report will be used to produce the desired cost accounting, and whether
the accounting reports need to be archived. Refer to Section 3.9.3: Accounting Policy and
Validation on page 96 and Section 12.2: Accounting Policy of the HPSS Management Guide
for more information on defining an Accounting Policy and generating accounting reports.
4. Determine if gatekeeping (monitoring or load-balancing) will be required. If so, define and
write the site policy code for gatekeeping. Refer to Section 3.9.7: Gatekeeping on page 99 for
more information on gatekeeping, and refer to the HPSS Programmers Reference for
guidelines on implementing the Site Interfaces for the Gatekeeping Service.
3.1.5. HPSS Deployment Planning
The successful deployment of an HPSS installation is a complicated task which requires reviewing
client/system requirements, integration of numerous products and resources, proper training of users/
administrators, and extensive integration testing in the client environment.
Consider first, creating a set of documents to ensure the resources and intended configuration of
those resources can adequately meet the expectations required of the system. The next step in this
process is to coordinate the availability and readiness of the resources before the actual installation of
HPSS begins. Each one of the products/resources that HPSS uses must be installed, configured and
tuned for the final system to function and perform as expected. Once installed, a series of tests
should be planned and performed to verify that the system can meet the demands of the final
production environment. Finally, proper training of those administrating the system, as well as those
who will use it, is necessary to make a smooth transition to production usage.
To help the HPSS system administrators in all of these tasks, a set of procedures have been developed
to supplement this document. The HPSS Deployment Process contains a detailed outline of what is
required to bring up an HPSS system, from an initial introduction and review of the environment to
production use. This document is provided to customers at the initial HPSS planning meeting. The
deployment procedures include a time line plus checklist that the HPSS customer installation/system
administration team should use to keep the deployment of an HPSS system on track. This is the same
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 55
guide that the HPSS support/deployment team uses to monitor and check the progress of an
installation.
3.2. Requirements and Intended Uses for HPSS
This section provides some guidance for the administrator to identify the site’s requirements and
expectations of HPSS. Issues such as the amount of storage needed, access speed and data transfer
speed, typical usage, security, expected growth, data backup, and conversion from an old system must
be factored into the planning of a new HPSS system.
3.2.1. Storage System Capacity
The amount of HPSS user data storage space the administrator must plan for includes the following
considerations:
• The amount of user data storage space required to support new user accounts.
• The amount of user data storage space required to support expected growth of current
accounts.
• The amount of metadata storage space required to support storage management activities such
as migration and repack.
• The amount of user data storage space required to support duplicate copies of user files.
Another component of storage space planning is the amount of space needed for HPSS system
metadata. Refer to Section 3.5.1: HPSS User Storage Space on page 69 and Section 3.5.4: HPSS
Metadata Space on page 74 for more information on determining the needed storage space and
metadata space.
3.2.2. Required Throughputs
Determine the required or expected throughput for the various types of data transfers that users will
perform. Some users want quick access to small amounts of data. Other users have huge amounts of
data they want to transfer quickly, but are willing to wait for tape mounts, etc. In all cases, plan for
peak loads that can occur during certain time periods. These findings must be used to determine the
type of storage devices and network to be used with HPSS to provide the needed throughput.
3.2.3. Load Characterization
Understanding the kind of load users are putting on an existing file storage system provides input that
can be used to configure and schedule the HPSS system. What is the distribution of file sizes? How
many files and how much data is moved in each category? How does the load vary with time (e.g.,
over a day, week, month)? Are any of the data transfer paths saturated?
Having this storage system load information helps to configure HPSS so that it can meet the peak
demands. Also based on this information, maintenance activities such as migration, repack, and
reclaim can be scheduled during times when HPSS is less busy.
3.2.4. Usage Trends
To configure the system properly the growth rates of the various categories of storage, as well as the
growth rate of the number of files accessed and data moved in the various categories must be known.
Extra storage and data transfer hardware must be available if the amount of data storage and use are
growing rapidly.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 56
3.2.5. Duplicate File Policy
The policy on duplicating user data files impacts the amount of data stored and the amount of data
moved. If all user files are duplicated, the system will require twice as much tape storage. If users
perform their own duplication of files, the system may consume a smaller amount of storage space.
Users can be given control over duplication of their files by allowing them a choice between
hierarchies which provide duplication and hierarchies which do not.
3.2.6. Charging Policy
HPSS does not charge users for the use of storage system resources. Instead, it collects information
that a site can use to implement a charging policy.
3.2.7. Security
Authentication and authorization between HPSS servers is done through use of either UNIX or
Kerberos security tools for authentication and either UNIX or LDAP for authorization services. By
default, servers are authenticated using the Kerberos authentication service, and authorization
information is obtained from the UNIX authorization service. The default protection level passes
authentication tokens on the first remote procedure call to a server. The authentication service,
authorization service, and protection level for each server can be configured to raise or lower the
security of the system. Two cautions should be noted: (1) raising the protection level to packet
integrity or packet privacy will require additional processing for each RPC, and (2) lowering the
authentication service to none effectively removes the HPSS authentication and authorization
mechanisms. Lowering the authentication service level should only be done in a trusted environment.
Each HPSS server authorizes and enforces access to its interfaces through access control lists stored
in the AUTHZACL table. To modify server state, control access is required. Generally, this is only
given to the Kerberos principal associated with the HPSS system administrative component.
Additional Kerberos principals can be allowed or denied access by setting permissions appropriately.
See Section 2.1: HPSS Server Security ACLs of the HPSS Management Guide for more information.
Security auditing in each server may be configured to record all, none, or some security events. Some
sites may choose to log every client connection; every bitfile creation, deletion, and open; and every
file management operation. Other sites may choose to log only errors. See the security information
fields in the general server configuration (Section 5.2: Server Configuration of the HPSS
Management Guide) for more details.
User access to HPSS interfaces depends on the interface being used. Access through the native Client
API uses the UNIX or Kerberos authentication services and UNIX or LDAP authorization services
described above. FTP or Parallel FTP access may utilize the HPSS password file, a configurable
password file, or the Kerberos credentials. Additional FTP access is available using Ident, or
Kerberos GSS credentials. Refer to the FTP section of the HPSS User’s Guide for additional details.
3.2.7.1. Cross Realm Access
Kerberos provides facilities for secure communication between multiple Kerberos Realms (domains)
referred to as Trusted “Cross Realm” access. These features use the Kerberos facilities to provide a
trusted environment between cooperating locations. HPSS uses the Kerberos Cross Realm features
for authentication. The procedures for inter-connecting Kerberos Realms are outlined in Section
1.5.3: Cross Realm Cookbook of the HPSS Management Guide. The HPSS Parallel FTP program can
utilize the Kerberos and HPSS Cross Realm access features.
The Generic Security Service (GSS) FTP, available from the Massachusetts Institute of Technology,
and the HPSS Parallel FTP applications can take advantage of the Cross Realm access features for
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 57
authentication and authorization (subject to certain caveats – See FTP documentation for details).
The pftp_client binary must be built using the distributed source code. However, it is the site's
responsibility to obtain the necessary Kerberos components.
ACLs entries in the AUTHZACL table and/or ACLs on HPSS directories and files may need to be
added for appropriate foreign_user and/or foreign_group entries.
3.2.8. High Availability Option
The High Availability component allows HPSS to inter-operate with IBM’s HACMP for AIX
software. When configured with the appropriate redundant hardware, it allows failures of individual
system components (network adapters, hard disks, core server nodes, power supplies, etc.) to be
overcome, allowing the system to resume servicing requests with minimal downtime.
The specifics of each HA system can vary greatly such that no single formula applies to all situations.
However, additional hardware will be required for each HA system (e.g., redundant core server
nodes, extra core server hard disks, RS232 cables, etc.). Also, HACMP for AIX software is required.
A highly available (HA) HPSS system requires additional systems engineering and testing. It is
available as a special bid item for installing and configuring HPSS on a High Availability system.
For further information on HA HPSS, please contact your HPSS Customer Service Representative.
For more information on HACMP for AIX, please see the HACMP for AIX 5L web site at:
http://www-03.ibm.com/systems/p/software/hacmp.html.
3.3. Prerequisite Software Considerations
This section defines the prerequisite requirements for HPSS. Some products must be obtained
separately from HPSS and installed prior to the HPSS installation and configuration.
3.3.1. Prerequisite Software Overview
This section describes the prerequisite software packages required by HPSS and provides information
to obtain them. Refer to Section 3.3.2: Prerequisite Summary By HPSS Node Type on page 59 for
details on software versions.
3.3.1.1. DB2
HPSS uses the DB2 Universal Database Enterprise Server Edition by IBM Corporation to manage all
HPSS metadata. DB2 software is included in the HPSS distribution. Refer to Section 5.3.1.2: Install
HPSS Documentation and DB2 Software on page 141 for more information. The required DB2
FixPak must be downloaded and installed after the DB2 base is installed. DB2 FixPaks can be
downloaded from the DB2 Universal Database for Linux, UNIX, and Windows webpage
at http://www - 306.ibm.com/software/data/db2/udb/support/downloadadv8.html .
3.3.1.2. Kerberos
HPSS uses Massachusetts Institute of Technology (MIT) Kerberos to implement Kerberos
authentication. MIT Kerberos is a network authentication protocol designed to provide
authentication for client/server applications by using secret-key cryptography. A free implementation
of this protocol can be downloaded from the MIT's website (http://web.mit.edu/kerberos/). Refer to
Section 5.2.2: Install MIT Kerberos (If Using Kerberos Authentication) on page 137 for more
information.
For Linux, Kerberos is installed as part of the operating system.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 58
If UNIX authentication will be used, this product is not required.
3.3.1.3. LDAP and IBM Kerberos
HPSS can be configured to use an LDAP directory to store its authorization information such as
users' names, UIDs, GIDs, and home directories. The supported LDAP server product for this release
is IBM Tivoli Directory Server. It can be downloaded from the
http://www - 306.ibm.com/software/tivoli/resource - center/security/code-directory - server.jsp webpage .
Installing the IBM Kerberos client is only required if LDAP is being used for authorization and the
LDAP daemon will be used for Kerberos authentication. This option is supported only on AIX. The
fileset for the IBM Kerberos client is located on the AIX Expansion Pack CD.
If UNIX authorization will be used, this product is not required.
3.3.1.4. Java
HPSS uses the Java 2 Standard Edition, version 1.4.2, to implement the SSM graphical user
interface, hpssgui, the SSM command line interface, hpssadm and the mkhpss utility.
The Java product required for the AIX platform can be downloaded from the IBM Developer Kits for
AIX, Java Technology Edition webpage,
http://www - 106.ibm.com/developerworks/java/jdk/aix.index.html .
The Java product required for the Linux and Windows platform can be downloaded from Sun
Microsystems' download webpage, http://java.sun.com/j2se/1.4.2/download.html.
3.3.2. Prerequisite Summary By HPSS Node Type
This section provides a summary list of prerequisite software required for HPSS. It also lists the
software versions which have been verified with HPSS 6.2.
3.3.2.1. HPSS Server Nodes
This section describes the prerequisite software required for each server node.
3.3.2.1.1. AIX Requirements
Each AIX server node must have the following installed:
• IBM RS/6000 (eServer pSeries) with a minimum of 2 GB RAM
• AIX 5.2 with ML9 + APAR IY89387 or AIX 5.3 with ML5 + APAR IY89429 (32-bit or 64-
bit)
• DB2 UDB V8.2 Enterprise Server Edition (ESE) for AIX with FixPak 4. Note that FixPak 12
for DB2 8.1 is the same as DB2 UDB Version 8.2 with FixPak 4.
• Java JDK 1.4.2 for AIX. Upgrade to at least IBM Java build ca142-20060421 (SR 5) which
is APAR IY84053.
• MIT Kerberos 1.3.5 (if planning to use Kerberos authentication)
• IBM LDAP 5.1 (if planning to use LDAP authorization)
• IBM Kerberos 1.3 (if planning to use Kerberos authentication with LDAP)
• C compiler for AIX, version 7.0.0.8 (if planning to recompile HPSS code on this node)
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 59
• IBM ATL: atldd.driver 6.5.2.0 and Atape.driver 10.2.8.0 (if planning to control the IBM tape
library and drives from this node)
3.3.2.1.2. Linux Requirements
Each Linux server node must have the following installed:
• Linux machine (eServer zSeries) with a minimum of 2 GB RAM
• Red Hat Enterprise Linux AS release 4 (Nahant Update 3, kernel 2.6.9-34.ELsmp)
• DB2 UDB V8.2 Enterprise Server Edition (ESE) for Linux (32-bit) with ixPak 4. Note that
FixPak 12 for DB2 8.1 is the same as DB2 UDB Version 8.2 with FixPak 4.
• Java JDK 1.4.2_06 for Linux. Upgrade to J2SE 1.4.2_06 or later within the Java 1.4.2 set of
fixes/releases.
• MIT Kerberos 1.3.4-10 (if planning to use Kerberos authentication - installed as part of the
operating system)
• IBM LDAP 5.2 (if planning to use LDAP authorization)
• C compiler for Linux: gcc-3.4.5 (if planning to recompile HPSS code on this node)
• IBM ATL: ibmatl-5.3.9.0-0 (if planning to use IBM tape library and drives from this node)
3.3.2.2. HPSS Mover Nodes
A Mover consists of two processes: the mover administrative process that runs on the server node,
and the remote mover process that handles the HPSS devices and data transfers. To maximize
performance, the remote mover should not be placed on a node with DB2 and HPSS subsystem
servers.
Since HPSS security, logging and metadata services are preformed on the server node, no additional
software, like DB2 or HPSS servers, need to be installed on the remote Mover node.
3.3.2.2.1. AIX Requirements
Each AIX Mover node must have the following prerequisites:
• IBM RS/6000 (eServer pSeries) with a minimum of 1 GB RAM
• AIX 5.2 with ML9 + APAR IY89387 or AIX 5.3 with ML5 + APAR IY89429 (32-bit or 64-
bit)
• C compiler for AIX, version 7.0.0.8 (if planning to recompile HPSS code)
• IBM ATL: ibmatl-6.5.2.0 and Atape.driver 9.1.0.0 (if planning to use IBM tape library and
drives from this node)
3.3.2.2.2. Linux Requirements
Each Linux Mover node must have the following prerequisites:
• Linux machine (eServer zSeries) with a minimum of 1 GB RAM
• Red Hat Enterprise Linux AS release 4 (Nahant Update 3, kernel 2.6.9-34.ELsmp)
• IBM ATL: ibmatl-6.5.2.0 (if planning to use IBM tape library from this node)
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 60
• 1 GB RAM
3.3.2.2.3. Solaris Requirements
Each Solaris Mover node must have the following prerequisites:
• Solaris UltraSPARC based processor
• Solaris 8+ (32-bit or 64-bit)
• C compiler: Forte Developer 7 C 5.4 2002/03/09 (if planning to recompile Mover code)
3.3.2.2.4. IRIX Requirements
Each IRIX Mover node must have the following prerequisites:
• SGI machine
• IRIX 6.5 (with latest/recommended patch set)
• C compiler (if planning to recompile Mover code)
3.3.2.3. HPSS Client Nodes
This section describes the prerequisite requirements for running HPSS clients.
3.3.2.3.1. SSM Client Requirements
The client node where the SSM hpssgui and hpssadm applications run must meet the following
requirements:
• Supported platforms: AIX , Linux, Windows
• J2RE or J2SE 1.4.2 for the appropriate platforms
3.3.2.3.2. Client API Requirements
The client node where HPSS Client API applications run must meet the following requirements:
• Supported platforms. Refer to Table 1: HPSS Client Interface and Mover Platforms for
complete list.
• C compiler (if planning to recompile client application)
3.3.2.3.3. FTP/PFTP Client Requirements
The client node where HPSS FTP and PFTP run must meet the following requirements:
• Supported platforms. Refer to Table 1: HPSS Client Interface and Mover Platforms for
complete list.
• C and yacc compilers (if planning to recompile client code)
3.3.2.4. HPSS HDM Nodes (Linux only)
• SUSE Linux Enterprise Server (SLES) version 9 Service Pack 2 or greater
• MIT Kerberos 1.3.5
• dmapi-2.2.1-0.2 or higher
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 61
• xfsdump-2.2.25-0.2 or higher
• xfsprogs-2.6.25-0.2 or higher
3.4. Hardware Considerations
This section describes the hardware infrastructure needed to operate HPSS and includes
considerations about infrastructure installation and operation that may impact HPSS.
3.4.1. Network Considerations
Because of its distributed nature and high-performance requirements, an HPSS system is highly
dependent on the networks providing connectivity among the HPSS servers and clients.
For control communications (i.e., all communications except the actual transfer of data) among the
HPSS clients and servers, HPSS requires TCP/IP services. Since control requests and replies are
relatively small in size, a low-latency network usually is well suited to handling the control path.
The data path is logically separate from the control path and may also be physically separate,
although this is not required. For the data path, HPSS supports the same TCP/IP networks as those
supported for the control path. For supporting large data transfers, the latency of the network is less
important than the overall data throughput.
HPSS also supports a special data path option that may indirectly affect network planning because it
may off-load or shift some of the networking load. This option uses the shared memory data transfer
method, which provides for intra-node transfers between either Movers or Movers and HPSS clients
via a shared memory segment.
Along with shared memory, HPSS also supports a Local File Transfer data path for client transfers
that involve HPSS Movers that have access to the client's file system. In this case, the HPSS Mover
can be configured to transfer the data directly to or from the client’s file.
3.4.2. Robotically Mounted Tape
All HPSS PVRs are capable of sharing a robot with other tape management systems but care must be
taken when allocating drives among multiple robot users. If it is necessary to share a drive between
HPSS and another tape management system, the drive can be configured in the HPSS PVR but left in
the LOCKED state until it is needed. When needed by HPSS, the drive should be set to UNLOCKED
and should not be used by any other tape management system while in this state. This is critical
because HPSS periodically polls all of its unlocked drives even if they are not currently mounted or
in use.
Generally, only one HPSS PVR is required per robot. However, it is possible for multiple PVRs to
manage a single robot in order to provide drive and tape partitions within a robot. The drives in the
robot must be partitioned among the PVRs and no drive should be configured in more than one PVR.
Each tape is assigned to exactly one PVR when it is imported into the HPSS system and will only be
mounted in drives managed by that PVR.
The tape libraries supported by HPSS are:
• IBM 3494
• IBM 3582, 3583, 3584 (LTO). These libraries are now being called the IBM TS3500.
• Spectralogic T120 (LTO, SAIT)
• STK L40 (LTO)
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 62
• STK SL500 and SL8500
• STK Tape Libraries that support ACSLS
• ADIC i500
• ADIC AML (supported by special bid only)
3.4.2.1. IBM 3494
The 3494 PVR supports Ethernet and RS-232 (TTY) attached robots. If appropriately configured,
multiple robots can be accessible from a single node.
3.4.2.2. Drive-Controlled LTO Libraries (IBM 3582, IBM 3583, IBM
3584, Spectralogic T120)
The IBM 3582, IBM 3583, IBM 3584, and Spectralogic T120 Tape Libraries and Robots must be
attached to Linux or AIX workstation through a SCSI interface. In each case, the library shares a
SCSI channel with one of the drives, so at least one of the drives in the library must be connected to
the workstation. This workstation must be an HPSS node running the PVR. The tape driver must be
installed on this node. The LTO PVR and SCSI PVR (if using SCSI-3 interface) are used to
communicate with these libraries.
3.4.2.3. STK L40, STK SL500, STK SL8500
The SCSI PVR is used to communicate with the STK L40, STK SL500, and STK SL8500.
3.4.2.4. STK
The STK PVR must be able to communicate with STK’s ACSLS server. HPSS requires ACSLS
version 7.0.0. For the PVR to communicate with the ACSLS server, it must have a TCP/IP
connection to the server (e.g. Ethernet) and STK’s SSI software must be running on the node with the
PVR. Multiple STK Silos can be connected via pass through ports and managed by a single ACSLS
server. This collection of robots can be managed by a single HPSS PVR.
3.4.2.5. ADIC AML
The AML PVR is supported by special bid only.
The Distributed AML Server (DAS) client components on the AIX workstations must be able to
communicate (via a TCP/IP connected network) with DAS Client components on the node
controlling the robot in order to request DAS services. The AML PVR is used to communicate with
the ADIC AML.
3.4.3. Manually Mounted Tape
An Operator PVR is used to manage a homogeneous set of manually mounted drives. Tape mount
requests will be displayed on an SSM screen.
3.4.4. Tape Devices
The tape devices/drives supported by HPSS are listed in Table 2.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 63
9840C drives should not be used in conjunction with either 9840A or 9840B drives.
Table 2. Supported Platform/Driver/Tape Drive Combinations
Platform Driver Device(s)
AIX IBM 3580 (Gen3, Gen4), 3592 (Gen2,
Gen3)
Native 3580 (Gen3, Gen4), 9840 (C, D),
9940 (A, B), T10000 (A, B)
Linux Native 3580 (Gen3, Gen4), 3592 (Gen2,
Gen3), 9840 (C, D), 9940 (A, B),
T10000 (A, B)
IRIX Native 3580 (Gen3)
The “Driver” column uses the following abbreviations:
IBM IBM SCSI Tape Device Driver (Atape)
Native AIX, IRIX, or Linux native SCSI Tape Device Driver
Older tape drives (3590, 3590E, 3590H, 9840 (A & B), 3580 (Gen1 & Gen2), 3592 (Gen1 &
Gen2), DST-312, DST-314) will continue to be supported for existing HPSS sites until they
can be upgraded.
3.4.4.1. Multiple Media Support
HPSS supports multiple types of media for certain drives. Listed in the following table is a preference
list for each media type that can be mounted on more than one drive type. When the PVL starts, it
determines the drive type that each type of media may be mounted on. It makes these decisions by
traversing each media type’s list and using the first drive type from the list that it finds configured in
the system. For example, referring to Table 2, it can be seen that a single-length 3590E tape will
mount on a double-length 3590E drive if and only if there are no single-length 3590E drives
configured in the system.
IBM 3592 and 3580 tape technologies support multi-generation reads and writes. The HPSS PVL
which is responsible for creating the mount list, does not know the difference between a read and a
write request. Thus the PVL will first attempt to mount a particular generation of cartridge into the
same generation of drive. If the drives matching the generation of the cartridge are all busy, then it
will attempt to mount the cartridge into the next generation drive (if available). For example, if a
system has 3580 (LTO) Gen2, 3580 (LTO) Gen3, and 3580 (LTO) Gen4 cartridges and drives, a
3580 (LTO) Gen2 cartridge would be mounted into a 3580 (LTO) Gen2 drive (if not busy). If all the
3580 (LTO) Gen2 drives were busy, it would then attempt to mount the 3580 (LTO) Gen2 cartridge
into a 3580 (LTO) Gen3 drive.
Since 3580 LTO tape drives support reading (but not writing) of two previous generations (e.g. 3580
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 64
(LTO) Gen4 drive can read 3580 (LTO) Gen4, 3580 (LTO) Gen3, and 3580 (LTO) Gen2 cartridges,
but can only write 3580 (LTO) Gen4 and 3580 (LTO) Gen3 cartridges), HPSS will mount a 3580
(LTO) Gen2 cartridge into a 3580 (LTO) Gen4 drive only if 3580 (LTO) Gen2 drives are not defined
in HPSS and 3580 (LTO) Gen3 drives are either busy or not defined. In this case, it is up to the HPSS
Administrator to make sure these 3580 (LTO) Gen2 cartridges are read only.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 65
Table 3. Cartridge/Drive Affinity Table
Cartridge Type Drive Preference List
AMPEX DST-312 AMPEX DST-312
AMPEX DST-314
AMPEX DST-314 AMPEX DST-314
Single-Length 3590 Single-Length 3590
Double-Length 3590
Single-Length 3590E
Double-Length 3590E
Single-Length 3590H
Double-Length 3590H
Double-Length 3590 Double-Length 3590
Double-Length 3590E
Double-Length 3590H
Single-Length 3590E Single-Length 3590E
Double-Length 3590E
Double-Length 3590H
Double-Length 3590E Double-Length 3590E
Double-Length 3590H
Single-Length 3590H Single-Length 3590H
Double-Length 3590H
Double-Length 3590H Double-Length 3590H
3580 (LTO) Gen 1 3580 (LTO) Gen 1
3580 (LTO) Gen 2
3580 (LTO) Gen 3
3580 (LTO) Gen 2 3580 (LTO) Gen 2
3580 (LTO) Gen 3
3580 (LTO) Gen 4
3580 (LTO) Gen 3 3580 (LTO) Gen 3
3580 (LTO) Gen 4
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 66
3580 (LTO) Gen 4 3580 (LTO) Gen 4
3592 J1A Short Tape
3592 J1A Standard Tape
3592 J1A
3592 EO5
3592 EO6
3592 EO5 JJ Short Tape
3592 EO5 JA Standard Tape
3592 EO5 JB XL Tape
3592 EO5
3592 EO6
3592 EO6 JJ Short Tape
3592 EO6 JA Standard Tape
3592 EO6 JB XL Tape
3592 EO6
STK 9840A STK 9840A
STK 9840B
STK 9840C
STK 9840D
STK 9840B STK 9840B
STK 9840C
STK 9840D
STK 9840C STK 9840C
STK 9840D
STK 9840D STK 9840D
STK 9940A STK 9940A
STK 9940B
STK 9940B STK 9940B
STK T10000A STK T10000A
STK T10000B
STK T10000B STK T10000B
Note that the PVL’s choices are made at startup time, and are not made on a mount-to-mount basis.
Therefore a single-length 3590E cartridge will never mount on a double-length 3590E drive if a
single-length 3590E drive was configured in the system when the PVL was started.
3.4.5. Disk Devices
HPSS supports locally-attached disk devices, including those devices attached via SCSI, SSA or
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 67
Fibre Channel. For these devices, operating system disk partitions of the desired size must be created
(e.g., AIX logical volume or Linux/Solaris/IRIX disk partition), and the raw device name must be
used when creating the Mover Device configuration (see Section 7.1: Configure a New Device &
Drive of the HPSS Management Guide for details on configuring storage devices).
3.4.6. Special Bid Considerations
The following options are available by special bid only:
• ADIC AML Tape Libraries
• Sony GY-8240
• HPSS High Availability
3.5. HPSS Sizing Considerations
There are two types of storage space that must be planned for: HPSS User Storage Space & HPSS
Infrastructure Storage Space.
HPSS User Storage Space is the disk and tape storage resources that will store user data. Typically,
disk storage is allocated at the top level of storage hierarchies and is used as a disk cache for user
data. Tape storage is usually allocated to lower levels of storage hierarchies and is used for the longterm, permanent storage of user data.
HPSS Infrastructure Storage Space is the disk space allocated to file systems that contain
executables, log files, server metadata (DB2 database), backups, and other HPSS supporting files and
data. Tape resources outside of HPSS are usually required for backups of the operating system,
HPSS specific file systems, and HPSS metadata unless other backup processes have been configured.
During the HPSS planning phase, it is important to assess how much disk space will be required to
support the HPSS production environment. The first step in this process is to understand the various
metadata tables managed by the HPSS system. The sections that follow explain the metadata table
layout and how to best estimate disk space allocation to support these tables.
How these resources are interconnected to the overall system is just as important as the amount of
disk and/or number of tape drives/cartridges allocated. For instance, if there are terabytes of disk
storage in the form of several FC disk arrays and 50 enterprise type tape drives, but only one mover
and a couple of FC adapters, it is unlikely that the storage system will be able to adequately move
data in/out and within the system to meet anyone's demands and expectations. The "data pipes"
between the storage resources must be adequately provisioned to allow for efficient transfer of data;
including those times of peak demand. In the other extreme, one or both of the storage ends can be
under allocated and waste the overall potential of the infrastructure. If there are too few tape drives,
data stalls on the disk resources preventing new files from being transferred into the storage system,
or from being staged back from tape media in a timely manner when the user requests access to it.
In regard to metadata space, the key consideration is that HPSS is a database application and its
performance is directly correlated to how well, or poorly, it uses the database and the storage behind
it. Referring to Figure 5, it is probably a waste to allocate 14 36GB drives for HPSS filesystems and
the DB2 database. Except for the largest HPSS systems containing hundreds of millions of files, the
disk space will never be fully utilized. However, the overriding concern isn't space utilization, it is
database transaction performance, which is a function of the efficiency of the disks, controllers, and
adapters supporting the database operations. This is the reason that so much attention is focused on
the HPSS Core Server and the disk subsystem attached to it. High performance disk and tape systems
for user data storage have to be balanced with high performance file systems supporting HPSS and its
databases.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 68
Starting with HPSS 6.2 there are many enhancements to the storage system to take advantage of
Storage Area Networks. Though separated in Figure 4, in reality there is usually only one SAN at an
installation and all the resources are attached to it. Besides the HPSS Movers being connected to
SAN, the end-user clients are often SAN attached as well. The result is that the data paths take
greater advantage of the SAN architecture and fewer store-and-forward operations are needed
through Movers (i.e. clients transfer data across SAN directly to disk resources, the mover just
manages the transfer) and less traffic across the TCP/IP network infrastructure. Adequately
provisioning the "data pipes" is still critical, but the equation has changed to rely more heavily on the
SAN to carry the data traffic.
3.5.1. HPSS User Storage Space
HPSS files are stored on the media that is defined and allocated to HPSS. Enough storage space must
be provided to meet the demands of the user environment. HPSS assists in the management of space
by providing SSM screens with information about total space and used space in all of the defined
storage classes. In addition, alarms can be generated automatically based on configurable threshold
values to indicate when space used in a given Storage Class has reached a threshold level. In a
hierarchy where data is being migrated from one hierarchy level to a lower one, management of space
in the Storage Class provided is done via the migration and purge policies that are provided. The
basic factors involved are the total amount of media space available in the Storage Class being
migrated and the rate at which this space is used. This will drive how the migration and purge
policies are set up for the Storage Class. For more details on this, see Section 3.9.1: Migration Policy
on page 94 and Section 3.9.2: Purge Policy on page 95. Failure to provide enough storage space to
satisfy a user request results in the user receiving a NO SPACE error. It is important to understand
that the Core Server writes files only to the top level of the COS hierarchy. If the top level does not
have sufficient free space for the write operation, it will fail, regardless of the amount of free space in
lower levels.
3.5.2. HPSS Infrastructure Storage Space
Figure 5: Basic HPSS Metadata and Filesystem Allocation represents what is typically needed to
support a basic HPSS server node configuration. Specific function and required space allocation for
the defined items will be discussed in following sections. For now, the diagram helps to define the
road map between raw, physical disks and the required filesystems and logical volumes/partitions to
operate an HPSS system.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 69
Figure 5. Basic HPSS Metadata & Filesystem Allocation
On the left hand side of the diagram, the raw physical volumes are shown attached to the disk array
controller. The configuration of the disks by the controller and its software should be divided into
three separate LUNs: 1) HPSS Filesystems and DB2 Backups, 2) DB2 Logs, 3) and the DB2 Tables.
One disk may be kept as as a "hot spare" in the event that one of the other disks fails. This allows the
system to automatically replace the failed media and rebuild the data without immediate intervention
from an operator or administrator. Recommended configurations for the LUNs is to use RAID 5,
which is good for small, random data access, for the first and third LUN. For the LUN associated
with DB2 Logs, it is suggested that RAID 1 (Mirroring) be used since these logs are typically very
large and accessed sequentially by the system. Once defined, these LUNs are assigned to the core
server host via the LVM (AIX) into a Volume Group or by the operating system (Linux) into a hard
drive identifier. The last step is to allocate individual filesystems and logical
volumes(AIX)/partitions(Linux) as defined on the right-hand side of the diagram.
HPSS requires the use of DB2 log mirroring. This protects the active DB2 log from possible loss due
to the primary metadata disk array failure or other unavailability. To safe guard the DB2 log,
separate disk/controller components are required for the LUNs providing storage for the filesystem
where the log is written as well as where the archived copies of the logs are stored before being
backed up to tape media. It is imperative that the components be totally separate from the primary
metadata disk array (i.e. different disks, different controllers, different path such as the FC
connections) to provide protection against failures. Specific resource assignment for the the
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 70
mirrored log components will need to be determined by HPSS and the customer based on transaction
performance requirements. Potentially, disk resources primarily allocated for HPSS disk cache can
be used or the site may want to dedicate a second disk array for this purpose to prevent any possible
interference.
3.5.3. HPSS Filesystems
The following sections describe the various filesystems used by HPSS.
3.5.3.1. /opt/hpss
The HPSS software is installed in the /opt/hpss directory. The installation package sizes and disk
requirements are listed in Table 10: Installation Package Sizes and Disk Requirements.
3.5.3.2. /var/hpss
See Appendix F: /var/hpss Files for a more detailed explanation of directories and files located in
/var/hpss.
The /var/hpss directory tree is the default location of a number of HPSS configuration files and other
files needed by the servers. It is recommended that this filesystem be at least 1GB in size.
Within the /var/hpss filesystem the following sub-directories exist:
• The /var/hpss/etc is the default directory where some additional UNIX configuration files are
placed. These files are typically very small.
• The /var/hpss/ftp is the default directory where several PFTP Daemon files are maintained.
There are three sub-directories required: adm (contains the hpss_ftpd.log file), daemon, and
daemon/ftpd where the ftp.pids-hpss_class file will be placed.
• The /var/hpss/tmp is the default directory where the Startup Daemon creates a lock file for
each of the HPSS servers it brought up in the node. HPSS may also write diagnostic log files
and disk allocation maps in this directory, when configured to do so. The lock files are very
small, but the logs and disk maps may be several tens of kilobytes, or larger.
It is up to the administrator to remove unneeded reports to prevent the /var/hpss
filesystem from filling.
• The /var/hpss/log is the default directory where the Log Daemon creates two central log files.
The size of these log files is specified in the Log Daemon specific configuration. By default,
these files are 5 MB each. In this same directory, the Log Client will continually write a
circular ASCII log whose size is defined by the specific configuration. By default, the size of
this file is also 5 MB.
• If MPS is configured to produce a migration/purge report, it will generate a report every 24
hours. The size of these reports varies depending on the number of files being migrated and
purged during the report cycle. These reports are placed in the /var/hpss/mps directory by
default and will need to be removed when no longer needed.
It is up to the administrator to remove unneeded reports to prevent the /var/hpss
filesystem from filling.
• If the Gatekeeper is configured to do Gatekeeping Services, then the administrator may wish
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 71
to create a site policy configuration file, usually named /var/hpss/gk/gksitepolicy. The size of
this file depends on the site-implemented gatekeeping policy. If the Gatekeeper Service is not
used, there is a minimal amount of disk space used in this directory.
• If an Accounting report is requested, a report file and a checkpoint file are created in the
directory specified in the Accounting Policy, usually /var/hpss/acct. The sizes of these files
depend upon the number of files stored in HPSS.
It is up to the administrator to remove unneeded reports to prevent the /var/hpss
filesystem from filling.
• If SSM is configured to buffer alarm and event messages in a disk file, a file to store alarms
and events will be created in /var/hpss/ssm directory. This alarm file is approximately 5 MB
in size.
3.5.3.3. /var/hpss/adm/core
/var/hpss/adm/core is the default directory where HPSS servers put “core” files if they terminate
abnormally. Core files may be large, so it is recommended that there should be at least 2 GB reserved
for this purpose on the server node and at least 1 GB on Mover nodes.
It is up to the administrator to remove unneeded core files to prevent the /var/hpss
filesystem from filling.
3.5.3.4. /var/hpss/hpssdb
The /var/hpss/hpssdb filesystem stores the DB2 instance configuration information and 'CFG'
database tables. Specifics on its size are described in the 'CFG' Database Allocation section below.
The recommended minimum filesystem size is 1GB.
3.5.3.5. /var/hpss/hpssdb/subsys1 & subsysX
The /var/hpss/hpssdb/subsys1 filesystem stores the configuration and SMS temporary tables
allocated for the primary HPSS subsystem database. Specifics on the size of this filesystem are
described below in the 'SUBSYS' Database Allocation section. If multiple subsystems are defined in
the system, then separate filesystems should be allocated for each. Conventionally, subsystem
filesystems are numbered in sequence starting with '1'.
3.5.3.6. /db2/backups/cfg
The /db2/backups/cfg filesystem temporarily stores backup images of the CFG log archives and
database as they are generated. The backup files are then transferred to long-term media, such as
tape, using a backup file manager such as TSM. Details are described in the DB2 Disk Space section.
The recommended minimum filesystem size is 2GB.
Ensure that it has sufficient space to store the data generated by each DB2 backup;
otherwise the backup will fail. If the required backup space exceeds the maximum size for a
JFS filesystem (approximately 63.8 GB), consider using a JFS2 filesystem. Other options
include having DB2 compress the backup data as it is taken or having DB2 store the backup data
onto more than one filesystems.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 72
3.5.3.7. /db2/backups/subsys1 & subsysX
Similar to /db2/backups/cfg, the /db2/backups/subsys1 filesystem temporarily stores backup images
of the subsystem archived logs and database as they are generated. The backup files are then
transferred to long-term media, such as tape, using a backup file manager such as TSM. Details are
described in the Section 3.5.4.4: DB2 Disk Space on page 77. Separate filesystems are allocated for
each subsystem defined in the system.
Ensure that it has sufficient space to store the data generated by each backup; otherwise the
backup will fail. If the required backup space exceeds the maximum size for a JFS
filesystem (about 63.8 GB), consider using a JFS2 filesystem. Other options include having
DB2 compress the backup data as it is taken or having DB2 store the backup data onto more than
one filesystems.
3.5.3.8. /db2/log/cfg
Though small in comparison to the DB2 subsystem logging area, the /db2/log/cfg filesystem is
recommended to store the 'CFG' transaction logs in a separate filesystem from the configuration and
'CFG' tables.
3.5.3.9. /db2/log/subsys1 & subsysX
The subsys1 and subsequent subsystem databases require a separate filesystem to store the DB2
transaction logs. The /db2/log/subsys1 filesystem size is discussed in Section 3.5.4.4: DB2 Disk
Space on page 77.
3.5.3.10. /db2/mirror-log/cfg
This filesystem is similar to that of /db2/log/cfg, but the storage resources for the DB2 log must be
separate from the resource of the primary copy. DB2 is configured so that it writes to both the
primary and secondary log copies as transactions are registered by the database.
3.5.3.11. /db2/mirror-log/subsys1 & subsysX
This filesystem is similar to that of /db2/log/subsys1, but the storage resources for the DB2 log must
be separate from the resource of the primary copy. DB2 is configured so that it writes to both the
primary and secondary log copies as transactions are registered by the database.
3.5.3.12. /db2/mirror-backup/cfg
This filesystem is smaller than /db2/backups/cfg since it will only contain archived copies of the
mirrored copy of the 'CFG' logs. Similar to /db2/backups/cfg, it needs to be managed by a backup
tool such as TSM.
3.5.3.13. /db2/mirror-backup/subsys1 & subsysX
This filesystem is smaller than /db2/backups/subsys1 since it will only contain archived copies of the
mirrored copy of the 'SUBSYS1' logs. Similar to /db2/backups/subsys1, it needs to be managed by a
backup tool such as TSM.
3.5.3.14. SUBSYS1 Database Allocation
The recommended allocation of disk space for the SUBSYS1 (and other subsystems) database is
shown in Figure 5: on page . By default, mkhpss will distribute the database tables and indexes
across the 9 logical volumes/partitions shown in the diagram. Specifics of mkhpss are described in
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 73
Section 5.3.1.2: Install HPSS Documentation and DB2 Software on page 141. The tables and indexes
are separated into separate logical volumes/partitions to ease future expansion of the database and to
maximize performance of database operations.
For Linux, access to a /dev/hdxy partition is through the Linux buffered I/O system. While this is an
appropriate access method for a filesystem that supports journaled logging, for DB2 and Mover
accesses, non-buffered IO is required. Linux, up to release RHEL 4.0, has provided 'raw device'
access mode through the use of the 'raw' command and interface. Before DB2 uses the partitions
defined in the Figures 5 through Figure 8, the mapping of the /dev/hdxy device and the raw interface
must be configured by the administrator.
Though RHEL 4.0 and later supports the LVM manager, HPSS configurations have not attempted to
use logical volumes created on Linux by LVM for DB2 storage and their use is not recommended at
this time.
3.5.4. HPSS Metadata Space
During the HPSS planning phase, it is important to properly assess how much disk space will be
required by DB2 to store HPSS metadata. The first step in this process is to understand the metadata
tables managed by DB2. The sections that follow explain the metadata table layout and how best to
estimate disk space allocation to support these tables.
3.5.4.1. SMS versus DMS Space
DB2 table spaces can be allocated either as System Managed Space (SMS) or Database Managed
Space (DMS). The SMS allocation method uses a local filesystem which is managed by the operating
system. DB2 creates files in the filesystem to store the tables and indexes. In the DMS allocation
method, DB2 manages raw disk space directly, bypassing the operating system's buffers and
filesystem interface. We recommend the use of SMS for storing short or seldom used tables, and
DMS for storing large tables frequently used tables.
Tables used to define the configuration and policies of the system are typically small. These are
contained in the configuration or global database, usually named 'CFG', and should be allocated in
SMS space. Tables such as the BITFILE or STORAGESEGTAPE tables can be quite large and their
placement must be optimized for performance. These tables are contained in one or more subsystem
databases, usually called 'SUBSYSx' (where x is the subsystem number), and should be allocated in
DMS space.
3.5.4.2. 'CFG' Database Allocation
By default, mkhpss will store the DB2 related files for HPSS in the /var/hpss/hpssdb directory. As
recommended above, this directory should be a separate filesystem of RAID disks. The amount of
space needed will vary somewhat depending upon the number of subsystems in use. For a site with
only one subsystem, the amount of space should be about 1GB. For each additional subsystem, an
additional 1/2GB should be allocated.
3.5.4.3. 'SUBSYS' Database Allocation
HPSS support representatives will provide sites with the “6.2 Sizing Spreadsheet” to help determine
the amount of disk resources required to store the HPSS metadata and what the allocation should be
across the DB2 tablespaces. The total amount of disk space should be allocated as a 'raw logical
volume' on AIX or allocated as a raw device on Linux. The space should be allocated on a RAID or
mirrored disk device. The total space can span several logical devices as necessary. DB2 space can be
allocated using mkhpss which provides the option to define the space or use existing equally sized
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 74
disk devices. The “6.2 Sizing Spreadsheet” input tab is shown below.
Based on the input, the resulting output is show below:
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 75
Definitions for the DB2 tables are as follows:
Bitfile Disk Allocation Maps. (BFDISKALLOCREC) For each bitfile stored on disk, one or more
rows will be created in the Disk Allocation Maps table. The number of rows is determined by the
storage segment size in the Storage Class in which the files are stored and the average file size stored
in that Storage Class.
Bitfile Disk Segments. (BFDISKSEG) For a bitfile stored on disk, one or more rows will be
created in the bitfile disk segment table. Because bitfile disk segments track the contiguous
pieces of a bitfile, there normally will be only one disk segment row per file.
Bitfile Tape Segments. (BFTAPESEG) For a bitfile stored on tape, one or more rows will be created
in the bitfile tape segment table. Under normal conditions, one bitfile tape segment row is expected
for each tape on which the file is stored. It is safe to assume for each bitfile stored on tape, two bitfile
segment records are needed.
Bitfiles. (BITFILE) One row is created in the bitfile metadata table for each bitfile. The amount of
space allocated for this DB2 table will normally limit how many bitfiles can be created. Regardless of
how many copies of a bitfile exist and whether the bitfile is spread across disk and/or tape, only one
bitfile row is created for the file.
Name Space Objects. (NSOBJECT) Each name space object (file, fileset, directory, hard link,
junction or soft link) uses one row in the NSOBJECT table.
Name Space Text. (NSTEXT) A name space text row exists for each name space object that has a
comment or is a symbolic link. These text rows are variable length with a maximum size of 1023
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 76
bytes.
Disk Storage Segments. (STORAGESEGDISK) Expect the size of the disk storage segment
metadata table to be quite volatile. As files are added to HPSS, disk storage segments will be created,
and as files are migrated to tape and purged from disk, they will be deleted. A rough estimate of the
number of disk storage segments can be obtained by estimating the number of files that will be
resident on disk in the subsystem for each Storage Class, then multiplying by the Average Number of
Segments parameter of the Storage Class.
Tape Storage Segments. (STORAGESEGTAPE) The tape storage segment metadata table grows as
the number of files stored in the subsystem increases. Storage segments are created to describe
contiguous chunks of data written on tapes. Segment sizes may vary greatly. The number of storage
segments found on a given tape is not limited by the Core Server. This number is a function of the
length of the files written on the tape, the VV block size, the size of the data buffer used to write the
tape, and other factors.
HPSS support representatives will help sites use this spreadsheet as they size the metadata disk
resources and allocate those resources for the various DB2 tablespaces.
3.5.4.4. DB2 Disk Space
This section explains the local file system disk space requirements to support DB2, other than for
database tables.
The disk space that DB2 requires falls into the following categories:
DB2 log
The DB2 log requires a filesystem in which transaction log files are stored. The number of
log files and size of the files is controlled by several database configuration parameters.
LOGFILSIZ determines the size of each log file and LOGPRIMARY + LOGSECOND
determine the total number of log files that could exist in the logging filesystem that you specify with
NEWLOGPATH. Each database should log to its own filesystem and must utilize a RAID device for
storing the data. Also, it is required to use the DB2 MIRRORLOGPATH configuration variable and
define a separate filesystem to store the mirrored log file.
When DB2 is first started, log files are placed in the /var/hpss/<instance_name>/NODE0000/...
directory. After DB2 is started, use the database configuration parameters NEWLOGPATH and
MIRRORLOGPATH to direct log files to a prepared location.
To determine the amount of disk space required for the DB2 log files, run:
% db2 get db cfg for <db name>
Look for the LOGFILSIZ variable, which is the size of each log file, and multiply it by the total
number of logs for this database (LOGPRIMARY + LOGSECOND). The result will be the number
of 4KB blocks needed. To convert this to the number of bytes, multiply the result by 4096. Calculate
this for each database that utilizes this particular log filesystem to determine the total necessary disk
space.
The optimal type of disk used for the log filesystem(s) is a low-latency disk since the I/O is
performed in many small chunks. To ensure full recoverability in the event of media failure on the
log disk, it is important to mirror the log on a separate physical disk.
DB2 database home directories
DB2 requires approximately 700MB of disk space for installation. Upon installation each database
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 77
created will require approximately 10MB of disk space in a filesystem that should be used only for
database home directories and other DB2 home directories (such as the DB2 Administration Server
and Fenced User). This filesystem should be protected by RAID since DB2 may require information
in the database home directory to properly recover a database.
DB2 backups
Full backups of DB2 databases may be larger than the databases themselves. The amount of local
disk space needed for backups will depend largely on the backup strategy.
When using the DB2 to TSM strategy, local disk is not needed to backup DB2 databases. DB2
communicates directly with the TSM API to deliver backup images and archived log files directly to
TSM. This strategy is highly recommended.
3.5.5. System Memory and Disk Space
The following sections discuss recommendations and requirements for disk space, system memory,
and paging space.
3.5.5.1. Operating System Disk Spaces
It is recommended that all operating system logical volumes/partitions be mirrored. This is true of
the HPSS server and Mover nodes. Both AIX and Linux support this type of configuration.
3.5.5.2. System Disk Space Requirements for Running SSM
The SSM Graphical User Interface, hpssgui, and Command Line Interface, hpssadm, have an option
to create session log files. hpssgui records all status bar and popup messages issued during the
session in its log. hpssadm records all prompts, error and informational messages, and requested
output (lists, managed objects, configuration structures, etc.) issued during the session in its log. Old
session log files should be removed periodically to avoid filling the file system. This is typically
done with a cron job. For example, the following command will remove all files from /tmp which
have not been accessed within the previous seven days:
% find /tmp -atime +7 -exec rm {} \;
The creation of the session log files is controlled by the -S option to the hpssgui and hpssadm startup
scripts. See their man pages for details.
3.5.5.3. System Memory and Paging Space Requirements
The memory and disk space requirements for the nodes where the HPSS Servers will execute
depends on the configuration of the servers, the nodes that each server will run on, and the amount
of concurrent access they are configured to handle.
At least 2GB of memory is recommended for nodes that will run one or more HPSS servers (and most
likely a DB2 server), excluding the HPSS Movers. More memory is required for systems that run
most of the servers on one node and/or support many concurrent users. The memory available to
HPSS and DB2 servers is critical to providing acceptable response times to end user operations. Disk
space requirements are primarily covered by Section 3.5.4: HPSS Metadata Space on page 74 for
DB2 space, and the preceding subsections under Section 3.5.5: System Memory and Disk Space on
page 78 for the individual HPSS servers. Sufficient disk space should be allocated for the paging
space, using recommendations in the system documentation for the amount of memory configured.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 78
The amount of memory for nodes running HPSS Movers, and no DB2 servers, is dependent on the
number and types of devices configured on the Mover node, the expected usages of those devices,
and the configuration of the Movers. In general, Movers supporting disk devices will require more
memory than Movers supporting tape devices because disk devices are likely to have more
outstanding requests. At least 1GB of memory should be configured on the Mover nodes. More
memory is required for nodes that support many devices, especially disks, and have large numbers of
concurrent end-user requests. Additionally, the size of the Mover's internal buffers impacts the
Mover's memory requirements. Two buffers are used by each Mover process to handle I/O requests.
Paging space should be sized according to the following rules:
Table 4. Paging Space Info
Amount of physical memory Minimum recommended amount of paging space
memory <= 256MB 2 * amount of physical memory
256MB < memory <= 1GB 512MB + ((amount of physical memory – 256MB) * 1.25)
1GB < memory <= 2GB 1.5 * amount of physical memory
2GB < memory 1 * amount of physical memory
3.6. HPSS Interface Considerations
This section describes the user interfaces to HPSS and the various considerations that may impact the
use and operation of HPSS.
3.6.1. Client API
The HPSS Client API provides a set of routines that allow clients to access the functions offered by
HPSS. The API consists of a set of calls that are comparable to the file input/output interfaces
defined by the POSIX standard (specifically ISO/IEC 9945-1:1990 or IEEE Standard 1003.1-1990),
as well as extensions provided to allow access to the extended capabilities offered by HPSS.
The Client API is built on top of the HPSS security layer (either UNIX or Kerberos). It must be run
on a platform that supports the Core Server's security layer. For example if the Core Server is using
Kerberos authentication then users on the client platform must be able to authenticate themselves
with the Core Server's Kerberos realm. To access HPSS from client platforms that do not support the
Core Server's security layer, FTP or Parallel FTP must be used.
The Client API allows clients to specify the amount of data to be transferred with each request. The
amount requested can have a considerable impact on system performance and the amount of metadata
generated when writing directly to a tape storage class. See Section 3.9.6: Location Policy on page 99
and Section 3.11: HPSS Performance Considerations on page 112 for further information.
The details of the Application Program Interface are described in the HPSS Programmer’s Reference
Guide.
3.6.2. FTP
HPSS provides an FTP daemon that supports standard FTP clients. Extensions are also provided to
allow additional features of HPSS to be utilized and queried. Extensions are provided for specifying
Class of Service to be used for newly created files, as well as directory listing options to display
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 79
Class of Service and Accounting Code information. In addition, the chgrp, chmod, and chown
commands are supported as quote site options.
The FTP daemon is built on top of the Client API and must be run on a node that supports Kerberos
clients. Note that FTP clients can run on computers that do not have Kerberos installed.
The size of the buffer, used for reading and writing HPSS files, can be specified in the FTP daemon
configuration. The buffer size selected can have a considerable impact on both system performance
and the amount of metadata generated when writing directly to a tape Storage Class. See Section
3.9.6: Location Policy on page 99 and Section 3.11: HPSS Performance Considerations on page 112
for further information.
The GSSFTP from MIT is supported if the HPSS FTP Daemon is appropriately configured. This
client provides credential-based authentication and “Cross Realm” authentication to enhance security
and “password-less” FTP features.
Refer to the HPSS User’s Guide for details of the FTP interface.
3.6.3. Parallel FTP
The FTP daemon also supports the HPSS Parallel FTP (PFTP) protocol, which allows the PFTP
client to utilize the HPSS parallel data transfer mechanisms. This provides the capability for the
client to transfer data directly to the HPSS Movers (i.e., bypassing the FTP Daemon), as well as the
capability to stripe data across multiple client data ports (and potentially client nodes). Data transfers
are supported through TCP/IP. Support is also provided for performing partial file transfers.
The FTP protocol is supported by the HPSS FTP Daemon. Refer to Section 13.2: FTP Daemon
Configuration of the HPSS Management Guide for configuration information. No additional
configuration of the FTP Daemon is required to support PFTP clients.
The client side executable for PFTP is pftp_client. pftp_client supports TCP based transfers.
Because the client executable is a superset of standard FTP, standard FTP requests can be issued as
well as the PFTP extensions. Authentication using either username/password or Kerberos credentials
is configurable.
Refer to the HPSS User’s Guide for details of the PFTP interface.
3.6.4. XFS
XFS for Linux is an open source filesystem from SGI based on SGI's XFS filesystem for IRIX.
The impression of an infinitely large XFS filesystem can be achieved by using HPSS as a back end
to XFS. HPSS transparently archives inactive XFS data into HPSS storage which frees up XFS disk
space for active data.
It is well suited to sites with large numbers of small files or clients who wish to use NFS to access
HPSS data. However, the files can only be accessed through XFS (or NFS via XFS) and cannot be
accessed with HPSS utilities such as parallel FTP.
3.7. HPSS Server Considerations
Servers are the internal components of HPSS that provide the system's functionality. They must be
configured correctly to ensure that HPSS operates properly. This section outlines key considerations
that should be kept in mind when planning the server configuration for an HPSS system.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 80
3.7.1. Core Server
The Core Server is responsible for managing the HPSS name space (files, directories, links, etc.),
bitfiles, and storage (physical volumes, virtual volumes, etc.) for a single subsystem. Each of these
areas of responsibility are outlined in greater detail below.
Core Server at large
The Core Server uses POSIX threads to service concurrent requests. The Core Server accepts
requests from any authenticated client; however, certain Core Server functions can be performed only
by trusted clients. Trusted clients are those for whom control permission has been set in the Core
Server's ACL entry for the client. Higher levels of trust are granted to clients who have both control
and write permission set in their ACL entry. Refer to Section 2.1: HPSS Server Security ACLs of the
HPSS Management Guide for information concerning the ACL for the Core Server.
The Core Server can be configured to allow or disallow super-user privileges (root access). When the
Core Server is configured to allow root access, the UID of the super-user is configurable.
HPSS systems configured with multiple subsystems employ multiple Core Servers and multiple
metadata databases. Though the servers are separate, each Core Server in a given HPSS realm must
share the fileset global metadata table.
Name Space
The HPSS Core Server maintains the HPSS name space in system metadata. Refer to Section 3.5:
HPSS Sizing Considerations on page 68 for details on sizing the name space. Refer to Section 14.8:
DB2 Space Shortage of the HPSS Management Guide for information on handling a metadata space
shortage. By utilizing multiple storage subsystems, it is possible to distribute large name spaces
across multiple Core Servers. Junctions between the names spaces of these storage subsystems can be
used to "join" these subsystems.
Bitfiles
The Core Server provides a view of HPSS as a collection of files. It provides access to these files and
maps the files onto underlying storage objects.
When a Core Server is configured, it is assigned a server ID. This value should never be changed
because it is embedded in the ID of each bitfile and storage segments it uses. HPSS expects to be able
to extract the Server ID from any bitfile ID and connect to that server to access the file.
The Core Server maps bitfiles to their underlying physical storage by maintaining information that
maps a bitfile to the storage segments that contain its data. For additional information, see Section
3.7.1: Core Server on page 81. The relationship of bitfiles to storage segments and other structures is
shown in Figure 9: The Relationship of Various Server Data Structures.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 81
Figure 6. The Relationship of Various Server Data Structures
Disk Storage Allocation
Each Core Server manages disk storage units for HPSS. It maps each disk storage unit onto an HPSS
disk Physical Volume (PV) and records configuration data for the PV. Groups of one or more PVs
(disk stripe groups) are managed by the server as disk Virtual Volumes (VVs). The server also
maintains a storage map for each VV that describes which portions of the VV are in use and which
are free. Figure 9: The Relationship of Various Server Data Structures shows the relationship of
Core Server data structures such as VVs to other server data structures.
The server can manage information for any number of disk PVs and VVs; however, because a copy
of all of the PV, VV, and storage map information is kept in memory while the server runs, the size
of the server will be somewhat proportional to the number of disks it manages.
The Core Server is designed to scale up its ability to manage disks as the number of disks increase.
As long as sufficient memory and CPU capacity exist, threads can be added to the server to increase
its throughput. Additional subsystems can also be added to a system, increasing concurrency even
further.
Tape Storage Allocation
Each Core Server manages serial access magnetic tape storage media for HPSS. The server maps
each tape volume onto an HPSS tape PV and records configuration data for the PV. Groups of one or
more PVs (tape stripe groups) are managed by the server as tape VVs. The server maintains a storage
map for each VV that describes how much of each tape VV has been written and which storage
segment, if any, is currently writable in the VV. Figure 9: The Relationship of Various Server Data
Structures shows the relationship of data structures such as VVs to other server data structures.
The server can manage information for any number of tape PVs and VVs. It can also manage an
unlimited number of tape PVs, VVs, maps, and segments without impacting its memory size.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 82
The Core Server is designed to scale up its ability to manage tapes as the number of tapes increases.
As long as sufficient memory and CPU capacity exist, threads can be added to the server to increase
its throughput. Additional subsystems can also be added to a system, increasing concurrency even
further.
Note that the number of tape drives the server manages has much more to do with the throughput of
the server than the number of tape volumes the server manages. If the number of tape drives in the
system needs to increase to meet workload demands, adding a new subsystem and redistributing the
drives may be the best way to deal with the increased workload.
3.7.2. Migration/Purge Server
The Migration/Purge Server (MPS) reports storage class usage statistics and manages the amount of
free space available in storage classes by performing periodic migration and purge runs on the storage
classes. Migration copies data from the storage class on which it runs to one or more lower levels in
the storage hierarchy. Once data has been migrated, a subsequent purge run will delete the data from
the migrated storage class, if so configured. Migration is a prerequisite for purge, and MPS will never
purge data which has not previously been migrated. It is possible, but not desirable, to assign only a
migration policy and no purge policy to a disk storage class; however, this will result in data being
copied (migrated) but never deleted (purged). It is important to recognize that migration and purge
policies determine when data is migrated from a storage class and when the data is purged from that
storage class; however, the number of copies and the location of those copies is determined by the
storage hierarchy definition. The MPS uses the Core Server to both perform data movement between
hierarchy levels and gather statistics. As such, the Core Server must be running in order for the MPS
to complete its duties.
The MPS can only exist within a storage subsystem and a subsystem may be configured with no more
than one MPS. Storage hierarchies are global across all storage subsystems within an HPSS system,
but a given hierarchy may or may not be enabled within a given subsystem (a hierarchy is enabled
within a subsystem by configuring the subsystem to enable one or more classes of service which
reference that hierarchy). Note that a storage class may not be selectively migrated and purged in
different subsystems. If the hierarchy contains storage classes which require migration and purge,
then an MPS must be configured to run against those storage classes in the subsystem. This MPS will
manage migration and purge operations on only those storage resources within its assigned
subsystem. Thus, for an HPSS system with multiple storage subsystems, there may be multiple MPSs,
each operating on the resources within a particular subsystem.
Migration and purge operate differently on disk and tape storage classes. Disk migration and disk
purge are configured on a disk storage class by associating a migration policy and a purge policy with
that storage class. For tape storage classes, the migration and purge operations are combined, and are
collectively referred to as tape migration. Tape migration is enabled by associating a migration policy
with a tape storage class. Purge policies are not needed or supported on tape storage classes.
Once migration and purge policies are configured for a storage class (and the MPS is restarted), the
MPS will begin scheduling migration and purge runs for that storage class. Migration on both disk
and tape is run periodically according to the runtime interval configured in the migration policy. Disk
purge runs are not scheduled periodically, but rather are started when the percentage of space used in
the storage class reaches the threshold configured in the purge policy for that storage class. It is
critical that the hierarchies to which a storage class belongs be configured with proper migration
targets in order for migration and purge to perform as expected.
The purpose of disk purge is to maintain a given amount of free space in a disk storage class by
removing data of which copies exist at lower levels in the hierarchy. The order in which purge
records are sorted, which determines the order in which files are purged, may be configured on the
purge policy. It should be noted that all of the options except Record Create Time require additional
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 83
metadata updates and can impose extra overhead on DB2. Also, unpredictable purge behavior may be
observed if the purge record ordering is changed with existing purge records in the system until these
existing records are cleared. A purge run ends when either the supply of purge records is exhausted or
the purge target is reached.
There are two different tape migration methods, tape volume migration and tape file migration. The
method which is applied to a tape storage class is selected in the migration policy for that storage
class. Tape volume migration's goal is freeing tape volumes by moving data segments from sparsely
filled volumes either laterally (to another tape within the same storage class) or vertically (to a lower
level in the storage hierarchy). Tape file migration can be thought of as a hybrid between the disk
and tape volume migration methods. It is a file-based tape method which is able to make a single
copy of tape files to the immediately lower level in the hierarchy. For details on tape migration
options, please see Section 3.9.1.2: Migration Policy on page 95.
If, at any point during a tape file migration run, the MPS detects that the source tape volume has
become active, migration is abandoned on this volume until the next migration run. This is done in
order to avoid competing with an HPSS system user for this volume. A tape volume is deemed to be
active if any file it contains has been read or written within the access intervals specified in the
migration policy.
The MPS provides the capability of generating migration/purge report files that document the
activities of the server. The specification of the UNIX report file name prefix in the MPS server
specific configuration enables the server to create these report files. It is suggested that a complete
path be provided as part of this file name prefix. Once reporting is enabled, a new report file is started
every 24 hours. The names of the report files are made up of the UNIX file name prefix from the
server specific configuration, plus a year-month-day suffix. With reporting enabled, MPS will
generate file-level migration and purge report entries in real time. These report files can be
interpreted and viewed using the mps_reporter utility. Since the number and size of the report files
grow rapidly, each site should develop a cron job that will periodically remove the reports that are no
longer needed.
In order to efficiently perform disk migration, the MPS parallelizes the migration of files from disk to
tape. The number of files that the MPS migrates simultaneously is user configurable via the Request
Count in the Disk Migration Policy. For example, if the Request Count is set to one, then the MPS
will serially migrate files. If the Request Count is set to four, then the MPS will attempt to have four
files migrating at a time.
As previously indicated, the MPS provides the information displayed in the HPSS Active Storage
Classes window in SSM. Each MPS contributes storage class usage information for the resources
within its storage subsystem. MPS accomplishes this by polling the Core Server within its subsystem
at the interval specified in the MPS server specific configuration. The resulting output is one line for
each storage class for each storage subsystem in which that class is enabled. The MPS for a
subsystem does not report on storage classes which are not enabled within that subsystem. The
warning and critical storage class thresholds are also activated by the MPS.
3.7.3. Gatekeeper
Each Gatekeeper may provide sites with the ability to:
• Schedule the use of HPSS resources using Gatekeeping Services.
• Validate user accounts using the Account Validation Service.
If the site doesn’t want either service, then it is not necessary to configure a Gatekeeper into the
HPSS system.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 84
Sites can choose to configure zero (0) or more Gatekeepers per HPSS system. Gatekeepers are
associated with storage subsystems. Each storage subsystem can have zero or one Gatekeeper
associated with it and each Gatekeeper can support one or more storage subsystems. Gatekeepers are
associated with storage subsystems using the Storage Subsystem Configuration screen (see Section
4.2: Storage Subsystems of the HPSS Management Guide). If a storage subsystem has no Gatekeeper,
then the Gatekeeper field will be blank. A single Gatekeeper can be associated with every storage
subsystem, a group of storage subsystems, or one storage subsystem. A storage subsystem can NOT
use more than one Gatekeeper.
Every Gatekeeper has the ability to supply the Account Validation Services. A bypass flag in the
Accounting Policy metadata indicates whether or not Account Validation for an HPSS system is on
or off. Each Gatekeeper will read the Accounting Policy metadata file, so if multiple Gatekeepers are
configured and Account Validation has been turned on, then any Gatekeeper can be chosen by the
Location Server to fulfill Account Validation requests.
Every Gatekeeper has the ability to supply the Gatekeeping Service. The Gatekeeping Service
provides a mechanism for HPSS to communicate information through a well-defined interface to a
policy software module to be completely written by the site. The site policy code is placed in a welldefined site shared library for the gatekeeping policy (/opt/hpss/lib/libgksite.[a|so]) which is linked
to the Gatekeeper. The gatekeeping policy shared library contains a default policy which does NO
gatekeeping. Sites will need to enhance this library to implement local policy rules if they wish to
monitor and/or load balance requests.
The gatekeeping site policy code will determine which types of requests it wants to monitor
(authorized caller, create, open, and stage). Upon initialization, each Core Server will look for a
Gatekeeper in the storage subsystem metadata. If no Gatekeeper is configured for a particular storage
subsystem, then the Core Server in that storage subsystem will not attempt to connect to any
Gatekeeper. If a Gatekeeper is configured for the storage subsystem that the Core Server is
configured for, then the Core Server will query the Gatekeeper asking for the monitor types by
calling a particular Gatekeeping Service API. This API will then call the appropriate Site Interface
which each site can provide to determine which types of requests are to be monitored. This query by
the Core Server will occur each time the Core Server (re)connects to the Gatekeeper. The Core Server
will need to (re)connect to the Gatekeeper whenever the Core Server or Gatekeeper is restarted. Thus
if a site wants to change the types of requests it is monitoring, then it will need to restart the
Gatekeeper and Core Server.
If multiple Gatekeepers are configured for gatekeeping, then the Core Server that controls the files
being monitored will contact the Gatekeeper that is located in the same storage subsystem.
Conversely if one Gatekeeper is configured for gatekeeping for all storage subsystems, then each
Core Server will contact the same Gatekeeper.
A Gatekeeper registers five different interfaces: Gatekeeper Services, Account Validation Services,
Administrative Services, Connection Manager Services, and Real Time Monitoring Services. When
the Gatekeeper initializes, it registers each separate interface. The Gatekeeper specific configuration
will contain any pertinent data about each interface.
The Gatekeeper Service interface provides the Gatekeeping APIs which calls the site implemented
Site Interfaces. The Account Validation Service interface provides the Account Validation APIs. The
Administrative Service provides the server APIs used by SSM for viewing, monitoring, and setting
server attributes. The Connection Manager Service provides the HPSS connection management
interfaces. The Real Time Monitoring Service interface provides the Real Time Monitoring APIs.
The Gatekeeper Service Site Interfaces provide a site the mechanism to create local policy on how to
throttle or deny create, open and stage requests and which of these request types to monitor. For
example, it might limit the number of files a user has opened at one time; or it might deny all create
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 85
requests from a particular host or user. The Site Interfaces will be located in a shared library that is
linked into the Gatekeeper.
It is important that the Site Interfaces return a status in a timely fashion. Create, open, and stage
requests from MPS are timing sensitive, thus the Site Interfaces won't be permitted to delay or deny
these requests, however the Site Interfaces may choose to be involved in keeping statistics on these
requests by monitoring requests from Authorized Callers.
If a Gatekeeper should become heavily loaded, additional Gatekeepers can be configured (maximum
of one Gatekeeper per storage subsystem). In order to keep the Gatekeepers simple and fast, they do
not share state information. Thus if a site wrote a policy to allow each host a maximum of 20 creates,
then that host would be allowed to create 20 files on each storage subsystem that has a separate
Gatekeeper.
The Gatekeeper’s Real Time Monitoring Interface supports clients such as a Real Time Monitoring
utility which requests information about particular user files or HPSS Request Ids.
3.7.4. Location Server
All HPSS client API applications, which includes all end user applications, will need to contact the
Location Server at least once during initialization and usually later during execution in order to locate
the appropriate servers to contact. If the Location Server is down for an extended length of time,
these applications will eventually give up retrying their requests and become non-operational. To
avoid letting the Location Server become a single point of failure, consider replicating it, preferably
on a different node. If replicating the Location Server is not an option or desirable, consider
increasing the automatic restart count for failed servers in SSM. Since the Location Server’s requests
are short lived, and each client contacts it through a cache, performance alone is not usually a reason
to replicate the Location Server. Generally the only time a Location Server should be replicated
solely for performance reasons is if it is reporting heavy load conditions to SSM.
If any server is down for an extended length of time it is important to mark the server as nonexecutable within SSM. As long as a server is marked executable the Location Server continues to
advertise its location to clients which may try to contact it.
The Location Server must be reinitialized or recycled whenever the Location Policy or its
server configuration is modified. Note that it is not necessary to recycle the Location Server
if an HPSS server’s configuration is added, modified, or removed since this information is
periodically reread.
3.7.5. PVL
The PVL is responsible for mounting and dismounting PVs (such as tape and magnetic disk) and
queuing mount requests when required drives and media are in use. The PVL usually receives
requests from Core Server clients. The PVL accomplishes any physical movement of media that
might be necessary by making requests to the appropriate Physical Volume Repository (PVR). The
PVL communicates directly with HPSS Movers in order to verify media labels.
The PVL is not required to be co-resident with any other HPSS servers and is not a CPU-intensive
server. With its primary duties being queuing, managing requests, and association of physical
volumes with PVRs, the PVL should not add appreciable load to the system.
In the current HPSS release, only one PVL will be supported.
3.7.6. PVR
The PVR manages a set of imported cartridges, mounts and dismounts them when requested by the
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 86
PVL. It is possible for multiple HPSS PVRs to manage a single robot. This is done if it is necessary
to organize the tape drives in the robot into partitions. Each tape drive in the robot is assigned to
exactly one PVR. Additionally, each cartridge is assigned to only one PVR. The PVRs can be
configured identically and can communicate with the robot through the same interface.
The following sections describe the considerations for the various types of PVRs supported by HPSS.
3.7.6.1. STK PVR
The STK PVR communicates to the ACSLS server via STK’s SSI software.
The SSI must be started before the PVR. If the SSI is started after the PVR, the PVR should be
stopped and restarted.
If multiple STK robots are managed, SSIs that communicates with each of the robots should be
configured on separate CPUs. A PVR can be configured on each of the CPUs that is running an SSI.
If multiple STK robots are connected and are controlled by a single Library Management Unit
(LMU), a single PVR can manage the collection of robots. The PVR can be configured on any CPU
that is running an SSI.
HPSS is tested with Storage Technology Corporation’s (STK's) Automated Cartridge System Library
Software (ACSLS) Version 7.0.0.
ACSLS should be running on the workstation directly connected to the STK Silo. The HPSS STK
PVR can run on any workstation that has a TCP/IP connection to the ACSLS workstation. The
workstation running the HPSS PVR must also be running STK's Storage Server Interface (SSI)
software. This software will not be started by HPSS and should be running when HPSS is started. It
is recommended that the SSI be started by the workstation's initialization scripts every time the
workstation is booted.
The SSI requires that the system environment variables CSI_HOSTNAME and
ACSAPI_PACKET_VERSION be correctly set. Note that due to limitations in the STK Developer's
Toolkit, if the SSI is not running when the HPSS PVR is started, or if the SSI crashes while the HPSS
PVR is running, the HPSS PVR will lock up and will have to be manually terminated by issuing “kill
-9 <pid>”.
3.7.6.2. LTO PVR
The LTO PVR manages the IBM 3584 Tape Library and Robot, which mounts, dismounts and
manages LTO tape cartridges and IBM 3580 tape drives. The PVR uses the Atape driver interface to
issue SCSI commands to the library.
The SCSI control path to the library controller device (/dev/smc*) is shared with the first drive in the
library (typically /dev/rmt0). Since the PVR communicates directly to the library via the Atape
interface, the PVR must be installed on the same node that is attached to the library.
The LTO PVR operates synchronously; that is, once a request is made to the 3584 library, the request
thread does not regain control until the operation has completed or terminated. This means that other
requests must wait on an operation to complete before the PVR can issue them to the 3584.
HPSS is designed to work with the AIX tape driver (Atape) software to talk to the IBM 3584 LTO
Library over a SCSI channel. Currently HPSS is only supported for the AIX version of the Atape
driver. Please note that the PVR must run on the same node that has the Atape interface and this node
must have a direct SCSI connection to the library.
Please refer to The 3584 UltraScalable Tape Library Planning and Operator Guide and IBM Ultrium
Device Drivers Installation and User's Guide for more information.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 87
3.7.6.3. 3494 PVR
The 3494 PVR can manage an IBM 3494 tape robot attached via Ethernet or SCSI. The PVR will
create a process to receive asynchronous notifications from the robot.
At least one PVR should be created for every robot managed by HPSS. If multiple 3494 robots are
managed, care must be taken to ensure that the PVRs are configured to communicate with the correct
/dev/lmcp devices. The PVRs can run on the same CPU or different CPUs as long as the proper
/dev/lmcp devices are available.
HPSS is designed to work with IBM 3494 robots attached to an HPSS server with either RS-232 or
Ethernet control connections. Data paths to the drives will be SCSI-2 with RS-232 and Ethernet
control paths. The HPSS PVR must run on a machine with the appropriate version of Library
Manager Control Point (LMCP) device drivers installed.
3.7.6.4. AML PVR
The AML PVR is supported by special bid only.
The AML PVR can manage ADIC AML robots that use Distributed AML Server (DAS) software.
The DAS AML Client Interface (ACI) operates synchronously; that is, once a request is made to the
AML, the request process does not regain control until the operation has completed or terminated.
Therefore, the AML PVR must create a process for each service request sent to the DAS (such as
mount, dismount, eject a tape, etc.).
HPSS is designed to work with ADIC Distributed Automated Media Library Server (DAS) software
version 1.3 and the ABBA Management Unit (AMU) version 2.4.0. DAS is the ADIC software which
consists of the Automated Media Library (AML) Client Interface (ACI) and the DAS server
components. The AMU is the host computer software by which the ADIC Storage System manages
the archive database, which is based on a DB2 compatible database for an OS/2 system.
The AMU must run on a OS/2 PC host computer connected to the AML robot while the HPSS AML
PVR can run on any RS/6000 workstation that has a TCP/IP connection to the OS/2 host computer.
The workstation running the HPSS AML PVR must also contain the DAS/ACI software that is called
by the HPSS AML PVR.
Refer to ADIC DAS Installation and Administration Guide and Reference Guide AMU for additional
information.
3.7.6.5. Operator PVR
The Operator PVR displays mount requests for manually mounted drives. The mount requests are
displayed on the appropriate SSM screen
All of the drives in a single Operator PVR must be of the same type. Multiple operator PVRs can be
configured without any additional considerations.
3.7.6.6. SCSI PVR
The SCSI PVR communicates with tape libraries and robots through a generic SCSI interface. The
interface uses the SCSI-3 command set. The SCSI PVR currently supports the following medium
changers: IBM 3584, IBM Ultrium 3582, STK L40, STK SL500, STK SL8500, and Spectralogic.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 88
3.7.7. Mover
The Mover configuration is largely dictated by the hardware configuration of the HPSS system. Each
Mover can handle both disk and tape devices and must run on the node to which the storage devices
are attached. The Mover is also capable of supporting multiple data transfer mechanisms for sending
data to or receiving data from HPSS clients (e.g., TCP/IP and shared memory).
3.7.7.1. AIX Asynchronous I/O
Asynchronous I/O must be enabled manually on AIX platforms. There should be no asynchronous I/
O setup required for Solaris, Linux, or IRIX platforms.
To enable asynchronous I/O on an AIX platform, use either the chdev command:
% chdev -l aio0 -a autoconfig=available
or smitty:
% smitty aio
<select “Change / Show Characteristics of Asynchronous I/O”>
<change “STATE to be configured at system restart” to “available”>
<enter>
Asynchronous I/O on AIX must be enabled on the nodes on which the Mover will be running.
3.7.7.2. Tape Devices
All tape devices that will be used to read and write HPSS user data must be set to handle variable
block sizes to allow for the ANSI standard 80-byte volume label and file section headers. This
section describes the procedure for setting this option on each supported operating system.
3.7.7.2.1. AIX
To set the devices to use variable blocks on an AIX platform, either use the chdev command
(substituting the appropriate device name for rmt0):
% chdev -l rmt0 -a block_size=0
or smitty:
% smitty tape
<select “Change / Show Characteristics of a Tape Drive”>
<select the appropriate tape device>
<change “BLOCK size (0=variable length)” to “0”>
<enter>
Note: Take into account differences in the interface based on the specific device driver supporting
the device
3.7.7.2.2. Solaris
For Solaris, the method used to enable variable block sizes for a tape device is dependent on the type
of driver used. Supported devices include Solaris SCSI Tape Driver and IBM SCSI Tape Driver.
For the IBM SCSI Tape Driver, set the block_size parameter in the /opt/IBMtape/IBMtape.conf
configuration file to 0 and perform a reboot with the reconfiguration option. The Solaris SCSI Tape
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 89
Driver has a built-in configuration table for all HPSS supported tape drives. This configuration
provides variable block size for most HPSS supported drives. In order to override the built-in
configuration, device information can be supplied in the /dev/kernel/st.conf as global properties that
apply to each node.
Consult the tape device driver documentation for instructions on installation and configuration.
3.7.7.2.3. IRIX
Variable block sizes can be enabled for the IRIX native tape device driver by configuring the Mover
to use the tape device special file with a “v” in the name (e.g. /dev/rmt/tps5d5nsvc).
3.7.7.2.4. Linux
HPSS supports tape devices on Linux with the use of the native SCSI tape device driver (st). To
enable the loading of the Linux native tape device, uncomment the following lines in the ".config"
file and follow the procedure for rebuilding your Linux kernel.
CONFIG_SCSI=y
CONFIG_CHR_DEV_ST=y
In Linux, tape device files are dynamically mapped to SCSI IDs/LUNs on your SCSI bus. The
mapping allocates devices consecutively for each LUN of each device on each SCSI bus found at the
time of the SCSI scan, beginning at the lower LUNs/IDs/buses. The tape device file will be in this
format: /dev/st[0-31]. This will be the device name to use when configuring the HPSS device.
3.7.7.3. Disk Devices
All locally attached magnetic disk devices (e.g., SCSI, SSA) should be configured using the
pathname of the raw device (i.e., character special file).
For Linux systems, this may involve special consideration.
HPSS supports disk device on Linux with the use of the native SCSI disk device driver (sd) and the
raw device driver (raw).
The Linux SCSI Disk Driver presents disk devices to the user as device files with the following
naming convention: /dev/sd[a-h][0-8]. The first variable is a letter denoting the physical drive, and
the second is a number denoting the partition on that physical drive. Occasionally, the partition
number will be left off when the device corresponds to the whole drive. Drives can be partitioned
using the Linux fdisk utility.
The Linux raw device driver is used to bind a Linux raw character device to a block device. Any
block device may be used.
See the Linux manual page for more information on the SCSI Disk Driver, the Raw Device Driver
and the fdisk utility.
To enable the loading of the Linux native SCSI disk device, uncomment the following lines in the
configuration file and follow the procedure for rebuilding your Linux kernel.
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
Also, depending on the type of SCSI host bus adapter (HBA) that will be used, you will need to
enable one or more of the lower level SCSI drivers. For example, if you are using one of the Adaptec
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 90
HBAs with a 7000 series chip set, uncomment the following lines in the ".config" file and follow the
procedure for rebuilding your Linux kernel.
CONFIG_SCSI_AIC7XXX=y
CONFIG_AIC7XXX_CMDS_PER_DEVICE=253
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
3.7.7.4. Performance
The configuration of the Movers and attached devices can have a large impact on the performance of
HPSS because of constraints imposed by a number of factors e.g., device channel bandwidth, network
bandwidth, processor power.
A number of conditions can influence the number of Movers configured and the specific
configuration of those Movers:
• Each Mover process is built to handle a specific device interface, e.g., IBM SCSI-attached
3590/3590H/3580 drives. If multiple types of devices are to be supported, multiple Movers
must be configured.
• Each Mover currently limits the number of concurrently outstanding connections. If a large
number of concurrent requests are anticipated on the drives planned for a single Mover, the
device work load should be split across multiple Movers. This is primarily an issue for
Movers that will support disk devices.
• The planned device allocation should be examined to verify that the device allocated to a
single node will not overload that node's resource to the point that the full transfer rates of the
device cannot be achieved (based on the anticipated storage system usage). To off-load a
single node, some number of the devices and their corresponding Mover can be allocated to
other nodes.
• In general, the connectivity between the nodes on which the Movers will run and the nodes on
which the clients will run should have an impact on the planned Mover configuration. For
TCP/IP data transfers, the only functional requirement is that routes exist between the clients
and Movers; however, the existing routes and network types will be important to the
performance of client I/O operations.
• Mover to Mover data transfers, performed for migration, staging, and repack operations, also
impact the Mover configuration. For devices that support storage classes involved in
migration or staging, the Movers controlling those devices should be configured so that there
is an efficient data path among them. If Movers involved in a data transfer are configured on
the same node, the transfer will occur via a shared memory segment.
3.7.8. Logging Service
Logging Services are comprised of the Log Daemon, Log Client, and Delog processes.
If central logging is enabled (the default), log messages from all HPSS servers will be written by the
Log Daemon to a common log file. There is a single Log Daemon process per HPSS system.
The Delog process is executed as a command line utility. The central log file must be accessible
from the node on which the command is being executed. Refer to Section 9.5.2: Viewing the Central
Log of the HPSS Management Guide for detailed information on Delog.
Implementation of delog services via the SSM GUI is no longer supported. The command line utility,
hpss_delog, must be used.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 91
3.7.9. Startup Daemon
The Startup Daemon is responsible for starting, monitoring, and stopping the HPSS servers. The
Daemon responds only to requests from the SSM System Manager. It shares responsibility with each
HPSS server for ensuring that only one copy of the server runs at a given time. It helps the SSM
determine whether servers are still running, and it allows the SSM to send signals to servers.
Normally, the SSM stops servers by communicating directly with them but, in special cases, the SSM
can instruct the Startup Daemon to send a SIGKILL signal to cause the server to shut down
immediately.
If a server is configured to be restarted automatically, the Startup Daemon will restart the server when
it terminates abnormally. The Daemon can be configured to restart the server without limit, or up to a
fixed number of restarts, or not at all.
Choose a descriptive name for the Daemon that includes the name of the computer where the
Daemon will be running. For example, if the Daemon will be running on a computer named tardis,
use the descriptive name “Startup Daemon (tardis)”.
The Startup Daemon is started by running the /opt/hpss/bin/rc.hpss script. It ignores most signals
and may only be killed using the "kill -9 <pid>" command. The Startup Daemon must be run under
the root account so that it has sufficient privileges to start the HPSS servers.
The Startup Daemon runs on every HPSS server node.
3.7.10. Storage System Management
SSM has three components:
1. System Manager - Communicates with all other HPSS components requiring monitoring or
control.
2. GUI (hpssgui) - Provides the HPSS administrator or operator the ability to configure or
monitor the HPSS System through a set of windows.
3. Command Line Interface (hpssadm) - Provides the HPSS administrator or operator the ability
to configure or monitor a subset of the HPSS system through a set of interactive or batch
commands.
There can be only one SSM System Manager configured for an HPSS installation. The System
Manager is able to handle multiple SSM GUI or Command Line clients (on different hosts or on the
same host).
Starting up SSM GUI (hpssgui) directly from the HPSS server node where SSM System Manager is
running and displaying the SSM window on the user's desktop is discouraged. This is due to known
Java/X performance problems. Instead, it is recommended to install the Java and HPSS GUI client
software on the user's desktop and execute it there. See Section 3.3.5: SSM Desktop Client
Packaging of the HPSS Management Guide for more information.
There are no performance problems associated with running the SSM Command Line Interface
(hpssadm) directly on the server UNIX platform, and this is the recommended configuration.
Both the SSM GUI client, hpssgui, and the SSM Command Line client, hpssadm, may be executed
on any platform that complies with Section 3.3.2.3.1: SSM Client Requirements on page 61.
Before starting the SM, a review of SM key environment variable settings would be wise. Following
is a table of key SM environment variables along with the default value and meaning. Depending on
the size (number of servers and number of SSM clients) and activity of the HPSS system, these
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 92
values may need to be overridden in env.conf.
Key SM Environment Variables
Variable Default
Value
Functionality
HPSS_SM_SRV_CONNECT_FAIL_COUNT
3
Connection Fail Count: number of
connection failures to a server before the
Max Connection Interval takes affect (*)
HPSS_SM_SRV_CONNECT_INTERVAL_MI
N
20
Interval between attempting server
connections when Connection Fail
Count has not yet been reached
(seconds) (*)
HPSS_SM_SRV_CONNECT_INTERVAL_MA
X
60
Max Connection Interval: interval
between server connections when
Connection Fail Count has been reached
without a successful connection
(seconds) (*)
HPSS_SM_SRV_MONITOR_THREADS
5
Number of threads created to monitor
server connections
HPSS_SM_SRV_QUEUE_SIZE
5
Request Queue Size used by the System
Manager server interface - default of 5
slots in the server interface request
queue to be used when the server
interface thread pool is completely full.
The queue is used to hold RPC requests
from servers until a thread is available to
process the request.
Note that if the request queue has any
entries in it, it means that all the threads
in the server thread pool are busy and
the SM response will be degraded. If
this happens then it would be good to
increase the number of threads available
to the server interface using the
HPSS_SM_SRV_TPOOL_SIZE
variable. Increasing the size of the
queue will not help with performance.
HPSS_SM_SRV_TPOOL_SIZE
100
Thread Pool Size used by the System
Manager server interface. If the Thread
Pool is exhausted then server RPC
requests will be queued in the server
RPC Request Queue to wait for a thread
to become available. When the thread
pool is exhausted SM performance may
be degraded. Increase this value if that is
the case. Typically 1 thread per HPSS
server should be adequate. But a few
extra wouldn't hurt.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 93
Variable Default
Value
Functionality
HPSS_SM_SRV_MAX_CONNECTIONS
50
Number of HPSS server connections to
maintain at once. If this number of
connections is exceeded, then old
connections will be closed to maintain
this number of connections
* The SM attempts to throttle the connection attempts to other servers. It will attempt to reconnect
to each server every HPSS_SM_SRV_CONNECT_INTERVAL_MIN seconds until the number of
failures for that server has reached HPSS_SM_SRV_CONNECT_FAIL_COUNT. After the failure
count has been reached the SM will only try to reconnect to the server every
HPSS_SM_SRV_CONNECT_INTERVAL_MAX seconds until a successful connection is made at
which time the connection interval for the server will be set back to
HPSS_SM_SRV_CONNECT_INTERVAL_MIN.
3.8. Storage Subsystem Considerations
Storage subsystems are provided in HPSS for the purpose of increasing the scalability of the system,
particularly with respect to the Core Servers. An HPSS system consists of one or more subsystems,
and each subsystem contains its own Core Server. If multiple Core Servers are desired, this is
accomplished by configuring multiple subsystems.
Each subsystem uses a separate DB2 database. Adding a subsystem to an HPSS system means
adding an additional database that must be maintained and backed up.
3.9. Storage Policy Considerations
This section describes the various policies that control the operation of the HPSS system.
3.9.1. Migration Policy
The migration policy provides the capability for HPSS to copy (migrate) data from one level in a
hierarchy to one or more lower levels. The migration policy defines the amount of data and the
conditions under which it is migrated; however, the number of copies and the location of those copies
is determined by the storage hierarchy definition. The site administrator will need to monitor the
usage of the storage classes being migrated and adjust both the migration and purge policies to obtain
the desired results.
3.9.1.1. Migration Policy for Disk
Disk migration in HPSS copies (migrates) files from a disk storage class to one or more lower levels
in the storage hierarchy. Removing or purging of the files from disk storage class is controlled by the
purge policy. The migration and purge policies work in conjunction to maintain sufficient storage
space in the disk storage class.
When data is copied from the disk, the copied data will be marked purgeable but will not be deleted.
Data is deleted by running purge on the storage class. If duplicate copies are created, the copied data
is not marked purgeable until all copies have been successfully created. The migration policy and
purge policy associated with a disk storage class must be set up to provide sufficient free space to
deal with demand for storage. This involves setting the parameters in the migration policy to migrate
a sufficient number of files and setting the purge policy to reclaim enough of this disk space to
provide the free space desired for users of the disk storage class.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 94
Disk migration is controlled by several parameters. By default, these parameters are the same across
all subsystems. However, subsystem-specific policies may be created which override all of these
values. For a list of these parameters, refer to Section 6.4.2.2: Disk Migration Policy Configuration
in the HPSS Management Guide.
3.9.1.2. Migration Policy for Tape
There are two tape migration algorithms: tape file migration and tape volume migration. The
algorithm which MPS applies to a given tape storage class is selected in the migration policy for that
storage class.
The purpose of tape file migration is to make a second copy of files written on tape. This algorithm is
similar to disk migration, but only a single additional copy is possible. It is also possible to configure
tape file migration such that files are moved downwards in the hierarchy without keeping a second
copy.
It is possible for tape file migration to make a second copy of files written on tape. The algorithm is
similar to disk file migration, but only a single additional copy is possible. It is also possible to
configure tape file migration such that files are moved downwards in the hierarchy without keeping a
second copy.
The purpose of tape volume migration is to empty tape virtual volumes that have become full
(reached EOM) and have significant unused space in them. Unused space on a tape volume is
generated when files on that tape are deleted or overwritten. Since data can only be recorded on tapes
sequentially, vacated recording space on tapes can be reclaimed only by moving all remaining files to
other tapes.
Tape volume migration attempts to empty tapes by moving data off of the tapes to other volumes.
When a tape becomes empty, it is a candidate for reuse. The reclaim utility program resets the state
of the empty tape volumes so that they can be reused. The reclaim utility can be run from SSM, but it
should generally be set up to run on a periodic basis via the cron facility. For more information on
reclaim, see Section 8.1.5: Reclaiming HPSS Tape Virtual Volumes of the HPSS Management Guide
and the reclaim manual page.
The repack utility can also be used to create empty tapes in a storage class. The administrator should
determine whether a tape should be repacked based on the number of holes (due to file overwrite or
deletion) on the tape. If a tape storage class is at the bottom of a hierarchy, repack and reclaim must
be run periodically to reclaim wasted space. For more information on repack, see Section 8.1.4:
Repacking Tape Virtual Volumes of the HPSS Management Guide and the repack manual page.
The migration policy parameters which apply to the different tape migration algorithms in detail in
Section 6.4.2.3: Tape Migration Policy Configuration in the HPSS Management Guide.
3.9.2. Purge Policy
The purge policy allows the MPS to remove the bitfiles from disk after the bitfiles have been
migrated to a lower level of storage in the hierarchy. A purge policy cannot be defined for a tape
storage class or a disk storage class which does not support migration. Sites may or may not wish to
define a purge policy for all disk storage classes that support migration. Purging from tapes is
controlled by the "Migrate Files and Purge" flag of the tape migration policy; there is no separate
purge policy for tape storage classes.
The specification of the purge policy in the storage class configuration enables the MPS to do the
disk purging according to the purge policy for that particular storage class. Purge is run for a storage
class on a demand basis. The MPS maintains current information on total space and free space in a
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 95
storage class by periodically extracting this information from the HPSS Core Server. Based upon
parameters in the purge policy, a purge run will be started when appropriate. The administrator can
also force the start of a purge run via SSM.
The disk purge is controlled by several parameters:
• The Do not purge files accessed within <nnn> minutes parameter determines the minimum
amount of time a site wants to keep a file on disk. Files that have been accessed within this
time interval are not candidates for purge.
• The Start purge when space used reaches <nnn> percent parameter allows the amount of
free space that is maintained in a disk storage class to be controlled. A purge run will be
started for this storage class when the total space used in this class exceeds this value.
• The Stop purge when space used falls to <nnn> percent parameter allows the amount of
free space that is maintained in a disk storage class to be controlled. The purge run will
attempt to create this amount of free space. Once this target is reached, the purge run will end.
• The Purge Locks expire after <nnn> minutes parameter allows the length of time a file can
be “purge locked” before it will appear on the MPS report to be controlled. The “purge lock”
is used to prevent a file from being purged from the highest level of a hierarchy. Purge locks
only apply to a hierarchy containing a disk on the highest level. HPSS will not automatically
unlock locked files after they expire. HPSS reports the fact that they have expired in the MPS
report.
• The Purge by list box allows sites to choose the criteria used in selecting files for purge. By
default, files are selected for purge based on their migration time. Alternately, the selection of
files for purging may be based on the time the file was created or the time the file was last
accessed. Files may be purged in an unpredictable order if this parameter is changed while
there are existing purge records already in metadata until those existing files are processed.
Administrators should experiment to determine the parameter settings that will fit the needs of their
site. If a site has a large amount of disk file write activity, the administrator may want to have more
free space and more frequent purge runs. However, if a site has a large amount of file read activity,
the administrator may want to have smaller disk free space and less frequent purge runs, and allow
files to stay on disk for a longer time.
3.9.3. Accounting Policy and Validation
The purpose of the Accounting Policy is to describe how a site will charge for storage, and, in
addition, to describe the level of user authorization (validation) to be performed when maintaining
accounting information.
A site must decide which style of accounting to use before creating any HPSS files or directories.
There are two styles of accounting: UNIX-style accounting and Site-style accounting. In addition a
site may decide to customize their style of accounting by writing an accounting site policy module for
the Gatekeeper.
If a site chooses Site-style accounting and Account Validation is turned off, LDAP must be used as
the authorization mechanism. The hpssGECOS field which is maintained for each user in LDAP,
contains the account index which allows Site-style accounting to be used. However if Account
Validation is turned on then the account index comes from the Account Validation metadata (through
the Gatekeeper).
If UNIX authorization is used and Account Validation is turned off, UNIX-style accounting must be
used because there is no hpssGECOS field. The basic limitation is that if the account index is needed
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 96
out of the hpssGECOS field, it does not exist in UNIX. It only exists in LDAP.
The metadata for each file and directory in an HPSS system contains an Account field, which
determines how the storage will be charged. Each user has at least one default account index, which
is put into the Account field of all new files and directories .
When using UNIX-style accounting, the account index is the user's UID. When the user's UID is
combined with the user's Realm Id, a unique Account is created.
When using Site-style accounting, each user may have more than one account index, and may switch
among them at runtime.
Each site must decide whether it wishes to validate Accounts. However, when using UNIX-style
accounting no authorization checking need be done since the account is always the user's UID.
If Account Validation is enabled, additional authorization checks are performed when the following
events occur: when files and directories are created, when their ownership is changed, when their
account index is changed, or when a user attempts to use an account index other than their default. If
the authorization check fails, the operation fails with a permission error.
Using Account Validation is highly recommended for sites that will be accessing remote HPSS
systems. The use of Account Validation will help keep account indexes consistent. If remote sites
are not being accessed, Account Validation is still recommended as a mechanism to keep consistent
accounting information.
If UNIX-style accounting is used, at least one Gatekeeper must be configured .
For Site-style accounting, an Account Validation metadata file must be created, populated and
maintained with valid user account indexes. See the Account Validation Editor (hpss_avaledit)
manual page for details on the use of the Account Validation Editor.
If the Require Default Account field is enabled when using Site-style accounting and Account
Validation, users are required to have valid default account indexes before performing almost any
client API action. If the Require Default Account field is disabled (which is the default behavior)
users will only be required to have a valid account set when performing an operation which requires
an account to be validated such as a create, an account change operation, or an ownership change
operation.
When using Site-style accounting with Account Validation, if the Account Inheritance field is
enabled, newly created files and directories will automatically inherit their account index from their
parent directory. The account indexes can then be changed explicitly by users. This is useful when
individual users have not had default accounts set up for them or if entire directory trees need to be
charged to the same account. When Account Inheritance is disabled (which is the default) newly
created files and directories will obtain their account from the user's current session account, which is
initially set to the user's default account index. This default account index may be changed by the
user during the session.
A site may decide to customize the way they do accounting. In most cases these sites should enable
Account Validation with Site-style accounting and then implement their own site policy module
which will be linked with the Gatekeeper. See Section 3.7.3: Gatekeeper on page 84 as well as the
appropriate sections of the HPSS Programmers Reference for more information.
By default Account Validation is disabled (bypassed). If it is disabled, the style of accounting is
determined by looking up each user's hpssGECOS account information in the authorization registry.
The following instructions describe how to set up users in this case.
If a users have their default account index encoded in a string of the form AA=<default-acct-idx> in
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 97
the principal's LDAP hpssGECOS attribute, then Site-style accounting will be used. Otherwise
UNIX-style accounting will be used.
To keep the accounting information consistent, it is important to set up all users in the HPSS
Authorization services with the same style of accounting (i.e. they should all have the AA= string in
their hpssGECOS attribute or none should have this string.) The hpss_ldap_admin tool can be used to
set attributes for a user including the hpssGECOS field. For more information, see the
hpss_ldap_admin man page.
See Section 12.4: Accounting of the HPSS Management Guide for more information.
3.9.4. Security Policy
HPSS server authentication and authorization make extensive use of UNIX or Kerberos
authentication and either UNIX or LDAP authorization mechanisms. Each HPSS server has
configuration information that determines the type and level of services available to that server.
HPSS software uses these services to determine the caller identity and credentials. Server security
configuration is discussed in more detail in Section 5.2: Server Configuration of the HPSS
Management Guide.
Once the identity and credential information of a client has been obtained, HPSS servers enforce
access to their interfaces based on permissions granted by an access control list stored in the DB2
table AUTHZACL.
HPSS client interface authentication and authorization security features for end users depend on the
interface, and are discussed in the following subsections.
3.9.4.1. Client API
The Client API interface uses either UNIX username/password or Kerberos authentication and either
UNIX or LDAP authorization features. Applications that make direct Client API calls must have
valid credentials prior to making those calls. Kerberos credentials can be obtained either at the
command line level via the kinit mechanism or within the application via the sec_login_set_context
interface. UNIX credentials are determined by the HPSS rpc library based on the UNIX user id and
group id of the application process.
3.9.4.2. FTP/PFTP
By default, FTP and Parallel FTP (PFTP) interfaces use either a username/password mechanism or
Kerberos credentials to authenticate. Either UNIX or LDAP is used to authorize end users. The end
user identity credentials are obtained from the principal and account records in the appropriate
security registry.
3.9.4.3. XFS
Since XFS is a filesystem interface, it uses the standard filesystem security mechanisms - owners,
groups and UNIX mode bits to enforce security policy. For communication between the HDM and
the DMG, the regular HPSS server authentication and authorization mechanisms are used.
3.9.4.4. Name Space
Enforcement of access to HPSS name space objects is the responsibility of the Core Server. A user's
access rights to a specific name space object are determined from the information contained in the
object's ACL, and the user's credentials.
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 98
3.9.4.5. Security Audit
HPSS provides the ability to record information about authentication, file creation, deletion, access,
and authorization events. The security audit policy in each HPSS server determines what audit
records a server will generate. In general, all servers can create authentication events, but only the
Core Server will generate file events. The security audit records are sent to the log file and are
recorded as security type log messages.
3.9.5. Logging Policy
The logging policy provides the capability to control which message types are written to the HPSS
log files. In addition, the logging policy is used to control whether alarms, events, and status
messages are sent to the Storage System Manager to be displayed. Logging policy is set on a per
server basis. Refer to Section 9.2.1: Creating a Log Policy of the HPSS Management Guide for a
description of the supported message types.
If a logging policy is not explicitly defined for a server, the default log policy will be applied. The
default log policy is selected from the Global Configuration window. If no Default Log Policy entry
has been defined, only Alarm and Event messages will be logged. All Alarm, Event, and Status
messages generated by the server will also be sent to the Storage System Manager.
The administrator might consider changing a server’s logging policy under one of the following
circumstances:
• A particular server is generating excessive messages. Under this circumstance, the administrator
could use the logging policy to limit the message types being logged and/or sent to the Storage
System Manager. This will improve performance and potentially eliminate clutter from the HPSS
Alarms and Events window. Message types to disable first would be Trace messages followed by
Debug and Request messages.
• One or more servers are experiencing problems which require additional information to
troubleshoot. If Alarm, Debug, or Request message types were previously disabled, enabling these
message types will provide additional information to help diagnose the problem. HPSS support
personnel might also request that Trace messages be enabled for logging.
3.9.6. Location Policy
In past versions of HPSS, the location policy was used to provide the ability to control how often
Location Servers in an HPSS installation contacted other servers. The location policy was used to
determine how often remote Location Servers were contacted to exchange server location
information.
This location policy information is still read by the Location Server, but, in the 6.2 version of HPSS it
has no practical value. It will probably be removed in future versions of HPSS.
3.9.7. Gatekeeping
The Gatekeeping Service provides a mechanism for HPSS to communicate information through a
well-defined interface to a installation specific customized software policy module. The policy
module is placed in a shared library, /opt/hpss/lib/libgksite.[a|so], which is linked into the
Gatekeeper. The default policy module does no gatekeeping. If Gatekeeping services are desired in an
HPSS installation, this default policy module must be replaced with one that implements the desired
policy.
The locally implemented policy module determines which types of requests will be monitored
(authorized caller, create, open, and stage). Upon initialization, each Core Server looks for a
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 99
Gatekeeper configured in its storage subsystem. If one is found, the Core Server asks the Gatekeeper
for its monitor types by calling the gk_GetMonitorTypes function which calls the locally
implemented gk_site_GetMonitorTypes function which determines which types of requests to
monitor. This query by the Core Server occurs each time the Core Server connects to the Gatekeeper,
which occurs whenever the Core Server or Gatekeeper is restarted. Therefore, if a site wants to
change the types of requests to be monitored, the Core Server and Gatekeeper must be restarted.
For each type of request being monitored, the Core Server calls the appropriate Gatekeeping Service
API (gk_Create, gk_Open, gk_Stage) passing along information pertaining to the request. This
information includes:
Table 5. Gatekeeping Call Parameters
Name Description create open stage
AuthorizedCaller Whether or not the request
is from an authorized caller.
These requests cannot be
delayed or denied by the site
policy.
Y Y Y
BitFileID The unique identifier for the
file.
N/A Y Y
ClientConnectId The end client’s connection
uuid.
Y Y Y
RealmId The HPSS realm identifier
for the user.
Y Y Y
GroupId The user’s group identifier Y Y Y
HostAddr Socket information for
originating host.
Y Y Y
OpenInfo Open file status flag
(Oflag).
N/A Y N/A
StageInfo Information specific to stage
(flags, length, offset, and
storage level).
N/A N/A Y
UserId The user’s identifier. Y Y Y
Each Gatekeeping Service API will then call the appropriate Site Interface passing along the
information pertaining to the request. If the request had AuthorizedCaller set to TRUE, then the Site
"Stat" Interface will be called (gk_site_CreateStats, gk_site_OpenStats, gk_site_StageStats) and
the Site Interface will not be permitted to return any errors on these requests. Otherwise, if
AuthorizedCaller is set to FALSE, then the normal Site Interface will be called (gk_site_Create,
gk_site_Open, gk_site_Stage) and the Site Interface will be allowed to return no error or return an
error to either retry the request later or deny the request. When the request is being completed or
aborted the appropriate Site Interface will be called (gk_site_Close, gk_site_CreateComplete,
gk_site_StageComplete). Examples of when a request gets aborted are when the Core Server goes
HPSS Installation Guide July 2008
Release 6.2 (Revision 2.0) 100
Loading...