IBM HPSS User Manual

3.36 Mb
Loading...

HPSS

Installation Guide

High Performance Storage System

Release 6.2

July 2008 (Revision 2.0)

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

1

© Copyright (C) 1992, 2008 International Business Machines Corporation, The Regents of the University of California, Los Alamos National Security, LLC, Lawrence Livermore National Security, LLC, Sandia Corporation, and UT-Battelle.

All rights reserved.

Portions of this work were produced by Lawrence Livermore National Security, LLC, Lawrence Livermore National Laboratory (LLNL) under Contract No. DE-AC52-07NA27344 with the U.S. Department of Energy (DOE); by the University of California, Lawrence Berkeley National Laboratory (LBNL) under Contract No. DE-AC02-05CH11231 with DOE; by Los Alamos National Security, LLC, Los Alamos National Laboratory (LANL) under Contract No. DE-AC52-06NA25396 with DOE; by Sandia Corporation, Sandia National Laboratories (SNL) under Contract No. DE-AC04-94AL85000 with DOE; and by UT-Battelle, Oak Ridge National Laboratory (ORNL) under Contract No. DE-AC05-00OR22725 with DOE. The U.S. Government has certain reserved rights under its prime contracts with the Laboratories.

DISCLAIMER

Portions of this software were sponsored by an agency of the United States Government. Neither the United States, DOE, The Regents of the University of California, Los Alamos National Security, LLC, Lawrence Livermore National Security, LLC, Sandia Corporation, UT-Battelle, nor any of their employees, makes any warranty, express or implied, or assumes any liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights .

Printed in the United States of America.

HPSS Release 6.2

July 2008 (Revision 2.0)

High Performance Storage System is a trademark of International Business Machines Corporation. IBM is a registered trademark of International Business Machines Corporation.

IBM, DB2, DB2 Universal Database, AIX, pSeries, and xSeries are trademarks or registered trademarks of International Business Machines Corporation.

AIX and RISC/6000 are trademarks of International Business Machines Corporation. UNIX is a registered trademark of the Open Group.

Linux is a registered trademark of Linus Torvalds in the United States and other countries. Kerberos is a trademark of the Massachusetts Institute of Technology.

Java is a registered trademark of Sun Microsystems, Incorporated in the United States and other countries. ACSLS is a trademark of Sun Microsystems, Incorporated.

Microsoft Windows is a registered trademark of Microsoft Corporation.

NFS, Network File System, and ACSLS are trademarks of Sun Microsystems, Inc. DST is a trademark of Ampex Systems Corporation.

Other brands and product names appearing herein may be trademarks or registered trademarks of third parties.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

2

Table of Contents

 

Chapter 1. Release 6.2...........................................................................................................

15

1.1.New Features.................................................................................................................

15

1.1.1.DCE Replacement .............................................................................................................................

15

1.1.2.Linux Support ...................................................................................................................................

15

1.1.3.Security..............................................................................................................................................

15

1.1.4.SCSI PVR .........................................................................................................................................

15

1.1.5.HPSS VFS Interface...........................................................................................................................

15

1.1.6.GridFTP Interface..............................................................................................................................

15

1.1.7.SAN3P/PIO Support .........................................................................................................................

15

1.1.8.Additional hpssadm Configuration Options.......................................................................................

15

1.1.9.Additional hpssadm operations..........................................................................................................

16

1.1.10.Additional Library and Device Support...........................................................................................

16

1.1.11.SAN Virtual Volume ID Mapping...................................................................................................

16

1.1.12.Drive Pools.......................................................................................................................................

17

1.1.13.FTP Enhancement............................................................................................................................

17

1.1.14.mkhpss Enhancement.......................................................................................................................

17

1.1.15.DB2 Monitoring...............................................................................................................................

17

1.1.16.File Family enhancements................................................................................................................

17

1.1.17.SSM Configuration Files Consolidation..........................................................................................

17

1.1.18. Mover Enhancement.......................................................................................................................

18

1.2.Retired Features ............................................................................................................

18

1.3.Deferred Features...........................................................................................................

18

1.4.HPSS Changes ..............................................................................................................

18

1.4.1.Documentation Organization Changes...............................................................................................

18

1.4.2.Metadata Changes..............................................................................................................................

18

1.4.3.DMAP Gateway Changes .................................................................................................................

19

1.4.3.1.Creating an XDSM Fileset........................................................................................................

19

1.4.3.2.Viewing DMAP Gateway XDSM Fileset Information..............................................................

20

1.4.3.3.Viewing Core Server XDSM Fileset Information.....................................................................

20

1.4.4.SSM Changes.....................................................................................................................................

21

1.4.4.1.Changes Affecting Sites Upgrading Directly from 4.5............................................................

21

1.4.4.2.Changes Affecting Sites Upgrading from 5.1...........................................................................

27

Chapter 2. HPSS Basics........................................................................................................

35

2.1.Introduction....................................................................................................................

35

2.2.HPSS Capabilities.........................................................................................................

35

2.2.1.Network-centered Architecture..........................................................................................................

35

2.2.2.High Data Transfer Rate....................................................................................................................

35

2.2.3.Parallel Operation..............................................................................................................................

35

2.2.4.Based on Standard Components.........................................................................................................

36

2.2.5.Data Integrity Through Transaction Management.............................................................................

36

2.2.6.Multiple Hierarchies and Classes of Services....................................................................................

36

2.2.7.Storage Subsystems............................................................................................................................

36

2.3.HPSS Components........................................................................................................

37

2.3.1.HPSS Files, Filesets, Volumes, Storage Segments and Related Metadata .......................................

38

2.3.2.HPSS Servers.....................................................................................................................................

40

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

3

2.3.3.HPSS Storage Subsystems.................................................................................................................

44

2.3.4.HPSS Infrastructure ..........................................................................................................................

44

2.3.5.HPSS User Interfaces ........................................................................................................................

46

2.3.6.HPSS Management Interfaces...........................................................................................................

46

2.3.7.HPSS Policy Modules .......................................................................................................................

47

2.4.HPSS Hardware Platforms............................................................................................

48

2.4.1.Server Platforms ...............................................................................................................................

48

2.4.2.Client Platforms ................................................................................................................................

48

2.4.3.Mover Platforms................................................................................................................................

49

Chapter 3. HPSS Planning...................................................................................................

51

3.1.Overview.......................................................................................................................

51

3.1.1.HPSS System Architecture.................................................................................................................

51

3.1.2.HPSS Configuration Planning............................................................................................................

52

3.1.3.Purchasing Hardware and Software...................................................................................................

54

3.1.4.HPSS Operational Planning...............................................................................................................

55

3.1.5.HPSS Deployment Planning..............................................................................................................

55

3.2.Requirements and Intended Uses for HPSS..................................................................

56

3.2.1.Storage System Capacity....................................................................................................................

56

3.2.2.Required Throughputs........................................................................................................................

56

3.2.3.Load Characterization........................................................................................................................

56

3.2.4.Usage Trends.....................................................................................................................................

56

3.2.5.Duplicate File Policy..........................................................................................................................

57

3.2.6.Charging Policy..................................................................................................................................

57

3.2.7.Security..............................................................................................................................................

57

3.2.7.1.Cross Realm Access..................................................................................................................

57

3.2.8.High Availability Option....................................................................................................................

58

3.3.Prerequisite Software Considerations ...........................................................................

58

3.3.1.Prerequisite Software Overview.........................................................................................................

58

3.3.1.1.DB2...........................................................................................................................................

58

3.3.1.2.Kerberos....................................................................................................................................

58

3.3.1.3.LDAP and IBM Kerberos..........................................................................................................

59

3.3.1.4.Java...........................................................................................................................................

59

3.3.2.Prerequisite Summary By HPSS Node Type.....................................................................................

59

3.3.2.1.HPSS Server Nodes...................................................................................................................

59

3.3.2.1.1.AIX Requirements....................................................................................................................................................

59

3.3.2.1.2.Linux Requirements.................................................................................................................................................

60

3.3.2.2.HPSS Mover Nodes .................................................................................................................

60

3.3.2.2.1.AIX Requirements....................................................................................................................................................

60

3.3.2.2.2.Linux Requirements.................................................................................................................................................

60

3.3.2.2.3.Solaris Requirements..............................................................................................................................................

61

3.3.2.2.4.IRIX Requirements..................................................................................................................................................

61

3.3.2.3.HPSS Client Nodes...................................................................................................................

61

3.3.2.3.1.SSM Client Requirements.......................................................................................................................................

61

3.3.2.3.2.Client API Requirements.........................................................................................................................................

61

3.3.2.3.3.FTP/PFTP Client Requirements.............................................................................................................................

61

3.3.2.4.HPSS HDM Nodes (Linux only)...............................................................................................

61

3.4.Hardware Considerations ..............................................................................................

62

3.4.1.Network Considerations.....................................................................................................................

62

3.4.2.Robotically Mounted Tape.................................................................................................................

62

3.4.2.1.IBM 3494..................................................................................................................................

63

3.4.2.2.Drive-Controlled LTO Libraries (IBM 3582, IBM 3583, IBM 3584, Spectralogic T120).......

63

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

4

3.4.2.3.STK L40, STK SL500, STK SL8500..........................................................................................

63

3.4.2.4.STK............................................................................................................................................

63

3.4.2.5.ADIC AML ...............................................................................................................................

63

3.4.3.Manually Mounted Tape....................................................................................................................

63

3.4.4.Tape Devices......................................................................................................................................

63

3.4.4.1.Multiple Media Support............................................................................................................

64

3.4.5.Disk Devices......................................................................................................................................

67

3.4.6.Special Bid Considerations................................................................................................................

68

3.5.HPSS Sizing Considerations.........................................................................................

68

3.5.1.HPSS User Storage Space..................................................................................................................

69

3.5.2.HPSS Infrastructure Storage Space....................................................................................................

69

3.5.3.HPSS Filesystems..............................................................................................................................

71

3.5.3.1./opt/hpss....................................................................................................................................

71

3.5.3.2./var/hpss....................................................................................................................................

71

3.5.3.3./var/hpss/adm/core....................................................................................................................

72

3.5.3.4./var/hpss/hpssdb........................................................................................................................

72

3.5.3.5./var/hpss/hpssdb/subsys1 & subsysX........................................................................................

72

3.5.3.6./db2/backups/cfg.......................................................................................................................

72

3.5.3.7./db2/backups/subsys1 & subsysX..............................................................................................

73

3.5.3.8./db2/log/cfg...............................................................................................................................

73

3.5.3.9./db2/log/subsys1 & subsysX......................................................................................................

73

3.5.3.10./db2/mirror-log/cfg.................................................................................................................

73

3.5.3.11./db2/mirror-log/subsys1 & subsysX........................................................................................

73

3.5.3.12./db2/mirror-backup/cfg...........................................................................................................

73

3.5.3.13./db2/mirror-backup/subsys1 & subsysX.................................................................................

73

3.5.3.14.SUBSYS1 Database Allocation...............................................................................................

73

3.5.4.HPSS Metadata Space .......................................................................................................................

74

3.5.4.1.SMS versus DMS Space............................................................................................................

74

3.5.4.2.'CFG' Database Allocation.......................................................................................................

74

3.5.4.3.'SUBSYS' Database Allocation.................................................................................................

74

3.5.4.4.DB2 Disk Space........................................................................................................................

77

3.5.5.System Memory and Disk Space........................................................................................................

78

3.5.5.1.Operating System Disk Spaces..................................................................................................

78

3.5.5.2.System Disk Space Requirements for Running SSM.................................................................

78

3.5.5.3.System Memory and Paging Space Requirements....................................................................

78

3.6.HPSS Interface Considerations......................................................................................

79

3.6.1.Client API .........................................................................................................................................

79

3.6.2.FTP.....................................................................................................................................................

79

3.6.3.Parallel FTP.......................................................................................................................................

80

3.6.4.XFS....................................................................................................................................................

80

3.7.HPSS Server Considerations.........................................................................................

80

3.7.1.Core Server........................................................................................................................................

81

3.7.2.Migration/Purge Server......................................................................................................................

83

3.7.3.Gatekeeper.........................................................................................................................................

84

3.7.4.Location Server .................................................................................................................................

86

3.7.5.PVL....................................................................................................................................................

86

3.7.6.PVR....................................................................................................................................................

86

3.7.6.1.STK PVR...................................................................................................................................

87

3.7.6.2.LTO PVR...................................................................................................................................

87

3.7.6.3.3494 PVR..................................................................................................................................

88

3.7.6.4.AML PVR..................................................................................................................................

88

3.7.6.5.Operator PVR...........................................................................................................................

88

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

5

3.7.6.6.SCSI PVR..................................................................................................................................

88

3.7.7.Mover ................................................................................................................................................

89

3.7.7.1.AIX Asynchronous I/O..............................................................................................................

89

3.7.7.2.Tape Devices.............................................................................................................................

89

3.7.7.2.1.AIX...........................................................................................................................................................................

89

3.7.7.2.2.Solaris......................................................................................................................................................................

89

3.7.7.2.3.IRIX..........................................................................................................................................................................

90

3.7.7.2.4.Linux........................................................................................................................................................................

90

3.7.7.3.Disk Devices..............................................................................................................................

90

3.7.7.4.Performance..............................................................................................................................

91

3.7.8.Logging Service.................................................................................................................................

91

3.7.9.Startup Daemon..................................................................................................................................

92

3.7.10.Storage System Management...........................................................................................................

92

3.8.Storage Subsystem Considerations................................................................................

94

3.9.Storage Policy Considerations ......................................................................................

94

3.9.1.Migration Policy ...............................................................................................................................

94

3.9.1.1.Migration Policy for Disk.........................................................................................................

94

3.9.1.2.Migration Policy for Tape........................................................................................................

95

3.9.2.Purge Policy ......................................................................................................................................

95

3.9.3.Accounting Policy and Validation ....................................................................................................

96

3.9.4.Security Policy...................................................................................................................................

98

3.9.4.1.Client API..................................................................................................................................

98

3.9.4.2.FTP/PFTP.................................................................................................................................

98

3.9.4.3.XFS............................................................................................................................................

98

3.9.4.4.Name Space...............................................................................................................................

98

3.9.4.5.Security Audit............................................................................................................................

99

3.9.5.Logging Policy...................................................................................................................................

99

3.9.6.Location Policy .................................................................................................................................

99

3.9.7.Gatekeeping.......................................................................................................................................

99

3.10.Storage Characteristics Considerations ....................................................................

101

3.10.1.Storage Class..................................................................................................................................

102

3.10.1.1.Media Block Size Selection...................................................................................................

103

3.10.1.2.Virtual Volume Block Size Selection (disk)...........................................................................

103

3.10.1.3.Virtual Volume Block Size Selection (tape)..........................................................................

103

3.10.1.4.Stripe Width Selection...........................................................................................................

103

3.10.1.5.Blocks Between Tape Marks Selection (tape only)...............................................................

104

3.10.1.6.Minimum Storage Segment Size Selection (disk only)..........................................................

105

3.10.1.7.Maximum Storage Segment Size Selection (disk only).........................................................

105

3.10.1.8.Maximum VVs to Write (tape only).......................................................................................

106

3.10.1.9.Average Number of Storage Segments (disk only)................................................................

106

3.10.1.10.PV Estimated Size / PV Size Selection................................................................................

106

3.10.1.11.Optimum Access Size Selection...........................................................................................

106

3.10.1.12.Some Recommended Parameter Values for Supported Storage Media..............................

106

3.10.1.12.1.Disk Media Parameters....................................................................................................................................

107

3.10.1.12.2.Tape Media Parameters....................................................................................................................................

107

3.10.2.Storage Hierarchy..........................................................................................................................

109

3.10.3.Class of Service..............................................................................................................................

110

3.10.3.1.Selecting Minimum File Size.................................................................................................

110

3.10.3.2.Selecting Maximum File Size................................................................................................

110

3.10.3.3.Selecting Stage Code............................................................................................................

110

3.10.3.4.Selecting Optimum Access Size.............................................................................................

111

3.10.3.5.Selecting Average Latency....................................................................................................

111

3.10.3.6.Selecting Transfer Rate.........................................................................................................

112

3.10.3.7.StripeLength and StripeWidth Hints.....................................................................................

112

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

6

3.10.4.File Families...................................................................................................................................

112

3.11.HPSS Performance Considerations...........................................................................

112

3.11.1.DB2................................................................................................................................................

112

3.11.2.Bypassing Potential Bottlenecks ...................................................................................................

113

3.11.3.Configuration.................................................................................................................................

113

3.11.4.FTP/PFTP......................................................................................................................................

114

3.11.5.Client API......................................................................................................................................

115

3.11.6.Core Server ...................................................................................................................................

115

3.11.7.Location Server..............................................................................................................................

115

3.11.8.Logging..........................................................................................................................................

115

3.11.9.Cross Realm Trust..........................................................................................................................

115

3.11.10.Gatekeeping.................................................................................................................................

115

3.11.11.XFS .............................................................................................................................................

116

3.11.12.HPSS VFS Interface.....................................................................................................................

116

3.12.HPSS Metadata Backup Considerations ...................................................................

117

3.13.HPSS Security Considerations..................................................................................

117

Chapter 4. System Preparation..........................................................................................

119

4.1.General Setup..............................................................................................................

119

4.2.Setup Filesystems........................................................................................................

120

4.2.1.DB2 Filesystem................................................................................................................................

120

4.2.2.HPSS Filesystem..............................................................................................................................

121

4.3.Setup Tape Libraries....................................................................................................

121

4.3.1.Special LTO Considerations............................................................................................................

121

4.3.2.IBM 3584.........................................................................................................................................

121

4.3.3.3494..................................................................................................................................................

122

4.3.4.STK..................................................................................................................................................

123

4.3.5.AML.................................................................................................................................................

123

4.4.Verify Tape Drives......................................................................................................

124

4.4.1.AIX...................................................................................................................................................

124

4.4.2.Solaris..............................................................................................................................................

125

4.4.3.IRIX.................................................................................................................................................

126

4.4.4.Linux................................................................................................................................................

126

4.5.Setup Disk Drives........................................................................................................

126

4.5.1.AIX...................................................................................................................................................

127

4.5.2.Linux................................................................................................................................................

128

4.5.3.IRIX.................................................................................................................................................

128

4.6.Setup Network Parameters..........................................................................................

129

4.6.1.HPSS.conf Configuration File.........................................................................................................

132

4.6.2.SP/x Switch Device Buffer Driver Buffer Pools..............................................................................

133

Chapter 5. HPSS Installation and Infrastructure Configuration...................................

135

5.1.Prepare for Installation................................................................................................

135

5.1.1.Distribution Media...........................................................................................................................

135

5.1.2.Software Installation Packages.........................................................................................................

135

5.1.3.Create Owner Account for HPSS Files............................................................................................

136

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

7

5.1.4.Installation Target Directory Preparation........................................................................................

136

5.2.Install Prerequisite Software........................................................................................

137

5.2.1.Install Java........................................................................................................................................

 

137

5.2.2.Install MIT Kerberos (If Using Kerberos Authentication)..............................................................

137

5.2.3.Install LDAP (If Using LDAP Authorization).................................................................................

137

5.2.4.Install Prerequisite Software for XFS HDM....................................................................................

138

5.3.Install HPSS/DB2 and Configure HPSS Infrastructure...............................................

139

5.3.1.Install and Configure HPSS - Root Subsystem Machine.................................................................

139

5.3.1.1.Pre-Installation Configuration...............................................................................................

139

5.3.1.2.Install HPSS Documentation and DB2 Software....................................................................

141

5.3.1.3.Set Up DB2 Permanent License..............................................................................................

142

5.3.1.4.Configure HPSS Security Services .........................................................................................

143

5.3.1.4.1.Configure UNIX Authentication and UNIX Authorization..................................................................................

143

5.3.1.4.2.Configure Kerberos Authentication and UNIX Authorization.............................................................................

146

5.3.1.4.3.Configure Kerberos Authentication and LDAP Authorization............................................................................

149

5.3.1.5.Configure DB2 Services..........................................................................................................

152

5.3.1.5.1.Remote DB2 Client Access & Fileset Creation/Deletion.....................................................................................

157

5.3.1.6.Configure Other Services........................................................................................................

158

5.3.1.7.Create Configuration Bundle..................................................................................................

159

5.3.2.Install and Configure HPSS

– Secondary Subsystem Machine.....................................................

160

5.3.2.1.Pre-Installation Configuration...............................................................................................

160

5.3.2.2.Install HPSS Documentation and DB2 Software on a subsystem..........................................

161

5.3.2.3.Set Up DB2 Permanent License..............................................................................................

162

5.3.2.4.Install Configuration Bundle..................................................................................................

163

5.3.2.5.Configure HPSS Security Services .........................................................................................

164

5.3.2.6.Configure DB2 Services..........................................................................................................

165

5.3.2.7.Configure Other Services........................................................................................................

167

5.3.3.Install and Configure HPSS

– Mover/Client Machine....................................................................

168

5.3.3.1.Install Mover/Client source code............................................................................................

168

5.3.3.2.Install Configuration Bundle..................................................................................................

169

5.3.3.3.Create /var/hpss subdirectories..............................................................................................

169

5.3.3.4.Modify Kerberos Configuration File, If Necessary................................................................

169

5.3.3.5.Check Time and IP Address....................................................................................................

169

5.4.Post Installation Procedures.........................................................................................

 

169

5.5.HPSS Documentation & Manual Page Setup..............................................................

170

5.5.1.Documentation and SSM Help Package..........................................................................................

170

5.5.2.Manual Page Setup...........................................................................................................................

 

171

5.6.Define HPSS Environment Variables..........................................................................

171

5.7.Tune DB2....................................................................................................................

 

171

5.8.Install and Build HPSS Source Code...........................................................................

172

5.8.1.Construct and Build the HPSS Base Source Tree ...........................................................................

172

5.8.1.1.Construct the HPSS Source Tree............................................................................................

172

5.8.1.2.Build the HPSS Base Source Tree..........................................................................................

172

5.8.1.3.Generate and Bind the DB2 Helper Program........................................................................

173

5.8.2.Construct and Build the HPSS Mover/Client Source Tree..............................................................

174

5.8.2.1.Construct the HPSS Mover/Client Source Tree.....................................................................

174

5.8.2.2.Build the HPSS Mover/Client Source Tree.............................................................................

174

5.8.3.Construct and Build the HPSS HDM Source Tree..........................................................................

175

5.8.3.1.Construct the HPSS HDM Source Tree..................................................................................

175

5.8.3.2.Build the HPSS HDM Source Tree.........................................................................................

175

5.9.Supporting Both Unix and Kerberos Authentication for SSM....................................

175

HPSS Installation Guide

July 2008

 

Release 6.2 (Revision 2.0)

 

8

Chapter 6. Upgrading to HPSS Release 6.2 .....................................................................

179

6.1.Special Instructions for Upgrading to HPSS 6.2.2......................................................

179

6.2.Planning for the HPSS 6.2 Upgrade............................................................................

180

6.2.1.Metadata changes in HPSS 6.2........................................................................................................

180

6.2.2.Upgrade Requirements and Limitations...........................................................................................

182

6.2.3.New Authentication and Authorization Mechanisms.......................................................................

182

6.2.3.1.Authentication Mechanisms....................................................................................................

183

6.2.3.2.Authorization Mechanisms......................................................................................................

183

6.2.4.New HPSS 6.2 System Files............................................................................................................

184

6.2.5.Testing the Metadata Conversion.....................................................................................................

184

6.2.6.Estimating the Metadata Conversion Time (for 4.5 upgrades only)................................................

184

6.2.6.1.Running Time for the Long Running Metadata Conversion Utilities (for 4.5 upgrades only).....

185

 

6.2.7.Capturing the Metadata Conversion Output.....................................................................................

185

6.2.8.DB2 Configuration and Tuning (for 4.5 upgrades only)..................................................................

186

6.2.9.Overview of the Upgrade Utilities...................................................................................................

187

6.2.9.1.HPSS 4.5 and HPSS 5.1 Upgrade Utilities.............................................................................

187

6.2.9.2.HPSS 4.5 Upgrade Utilities....................................................................................................

188

6.2.9.3.HPSS 5.1 Upgrade Utilities....................................................................................................

189

6.3.HPSS 6.2 Upgrade Procedures....................................................................................

190

6.3.1.Verify Prerequisites..........................................................................................................................

190

6.3.2.Acquire Software.............................................................................................................................

190

6.3.3.Install Authentication and Authorization Mechanisms....................................................................

191

6.3.4.Install or Upgrade DB2....................................................................................................................

193

6.3.5.Upgrade AIX....................................................................................................................................

194

6.3.6.Install or Upgrade Java....................................................................................................................

194

6.3.7.Save Current HPSS Code and Configuration Files..........................................................................

194

6.3.8.Prepare HPSS 6.2 Code...................................................................................................................

194

6.3.8.1.Install HPSS 6.2 Distribution Image......................................................................................

195

6.3.8.2. Compile HPSS 6.2 Source Code (if necessary).....................................................................

195

6.3.8.3.Disable Binaries, temporarily.................................................................................................

196

6.3.9.Set Environment Variables..............................................................................................................

196

6.3.10.Setup Authentication and Authorization........................................................................................

199

6.3.11.Pre-Conversion System Check.......................................................................................................

202

6.3.12.Take a full backup of SFS or DB2.................................................................................................

203

6.3.13.Upgrade from HPSS 4.5 to HPSS 6.2............................................................................................

203

6.3.13.1.Prepare for the Conversion..................................................................................................

203

6.3.13.2.Run db_convert_collect_info to Collect Metadata Information .........................................

203

6.3.13.3.Convert Configuration Metadata .........................................................................................

204

6.3.13.4.Convert Subsystem Metadata ...............................................................................................

204

6.3.13.5.Run the Long Running Utilities............................................................................................

205

6.3.13.6.Create Core Server ACLs.....................................................................................................

209

6.3.13.7.Terminate the Scripting Session...........................................................................................

210

6.3.13.8.Modify Permissions on Devices............................................................................................

210

6.3.14.Verify HPSS 4.5 Conversion Results ............................................................................................

211

6.3.14.1.Capture Session Output........................................................................................................

211

6.3.14.2.Run db_convert_size_check..................................................................................................

211

6.3.14.3.Run db_convert_ns_check....................................................................................................

212

6.3.14.4.Run db_convert_address_check...........................................................................................

212

6.3.14.5.Terminate Scripting Session.................................................................................................

213

6.3.15.Upgrade from HPSS 5.1 to HPSS 6.2 ..........................................................................................

213

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

9

6.3.16.Enable DB2 Backup......................................................................................................................

 

215

6.3.17.Perform the DCE Export: hpss_dce_export..................................................................................

215

6.3.18.Perform the Unix, LDAP or Kerberos Import...............................................................................

215

6.3.19.Prepare the 6.2 System...................................................................................................................

 

217

6.3.19.1.Tune DB2 for normal operations..........................................................................................

218

6.3.19.2.Modify Accounting, if applicable..........................................................................................

218

6.3.19.3.Update FTP Configuration Files..........................................................................................

218

6.3.19.4.Populate the HPSS.conf files................................................................................................

218

6.3.19.5.Copy the rc.hpss Script

to /etc.............................................................................................

218

6.3.19.6.Run the bind Script ..............................................................................................................

 

218

6.3.19.7.Create Default Server Security ACLs...................................................................................

218

6.3.19.8.Create SSM User Ids.............................................................................................................

 

219

6.3.19.9.Create Location Server Endpoints........................................................................................

220

6.3.19.10.Perform Additional Remote Mover Configuration.............................................................

220

6.3.20.Bring up the HPSS 6.2 Servers......................................................................................................

220

6.3.20.1.Invoke the SSM System Manager, Startup Daemon and prerequisite software ..................

221

6.3.20.2.Update HPSS Configurations...............................................................................................

222

6.3.20.3.Dump Accounting Metadata, if applicable...........................................................................

223

6.3.20.4.Start HPSS 6.2 Servers.........................................................................................................

 

224

6.3.21.Verify 6.2 System...........................................................................................................................

 

224

6.3.22.Clean Up After a 4.5 to 6.2 Upgrade.............................................................................................

225

6.3.23.Clean Up After a 5.1 to 6.2 Upgrade.............................................................................................

225

6.3.24.Revert HPSS 6.2 System to Prior Release ....................................................................................

225

6.3.24.1.Revert the HPSS 6.2 System to Version 4.5..........................................................................

225

6.3.24.2.Revert the HPSS 6.2 System to Version 5.1..........................................................................

226

6.4.Metadata Conversion Troubleshooting Procedures.....................................................

227

6.4.1.HPSS 4.5 to 6.2 Conversion Utility Errors and Warnings...............................................................

227

6.4.1.1.db_convert_collect_info Errors..............................................................................................

227

6.4.1.2.db_config_convert, db_subsys_convert, and db_lr_convert Errors and Warnings............

228

6.4.2.HPSS 5.1 to 6.2 Conversion Utility Errors......................................................................................

231

6.4.2.1.hpss_md_convert_51 Errors...................................................................................................

231

6.4.2.2.hpss_init_server_acls Errors..................................................................................................

232

6.5.HPSS 4.5 Conversion Utilities Output........................................................................

232

6.5.1.Interpreting Output from the 4.5 Conversion Utility.......................................................................

232

6.5.2.Examples of HPSS 4.5 Conversion Utility Output.........................................................................

234

6.5.2.1.db_convert_collect_info Output.............................................................................................

234

6.5.2.2.db_config_convert Output......................................................................................................

234

6.5.2.3.db_subsys_convert Output......................................................................................................

238

6.5.2.4.Long Running Conversion Utilities Output............................................................................

241

Appendix A. Glossary of Terms and Acronyms...............................................................

245

Appendix B. References......................................................................................................

 

256

Appendix C. Developer Acknowledgments.......................................................................

258

Appendix D. HPSS.conf Configuration File.....................................................................

259

D.1. PFTP Client Stanza...................................................................................................

 

259

D.2. PFTP Client Interfaces Stanza..................................................................................

264

D.3. Multinode Table Stanza............................................................................................

 

266

D.4. Network Option Stanza.............................................................................................

 

268

D.5. PFTP Daemon Stanza...............................................................................................

 

273

HPSS Installation Guide

July 2008

 

Release 6.2 (Revision 2.0)

 

10

D.6. Transfer Agent Stanza..............................................................................................

287

D.7. Stanzas Reserved for Future Use..............................................................................

291

Appendix E. hpss_env_defs.h.............................................................................................

293

Appendix F. /var/hpss files.................................................................................................

309

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

11

 

List of Figures

 

Figure 1. File Migration and Stage Operations..................................................................

37

Figure 2.

Class of Service / Hierarchy / Storage Class.......................................................

38

Figure 3. HPSS Components................................................................................................

40

Figure 4. HPSS Generic Configuration...............................................................................

52

Figure 5.

Basic HPSS Metadata & Filesystem Allocation.................................................

70

Figure 6.

The Relationship of Various Server Data Structures........................................

82

Figure 7.

Relationship of Class of Service, Storage Hierarchy, and Storage Class.......

102

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

12

 

 

List of Tables

 

Table 1. HPSS Client Interface and Mover Platforms.......................................................

49

Table 2. Supported Platform/Driver/Tape Drive Combinations......................................

64

Table 3. Cartridge/Drive Affinity Table.............................................................................

66

Table 4. Paging Space Info...................................................................................................

79

Table 5. Gatekeeping Call Parameters..............................................................................

100

Table 6.

Suggested Block Sizes for Disk............................................................................

107

Table 7.

Suggested Block Sizes for Tape...........................................................................

108

Table 8.

Network Options...................................................................................................

131

Table 9.

Installation Package Sizes and Disk Requirements...........................................

136

Table 10. Runing Times for Long Running Metadata Conversion Utilities..................

185

Table 11. 6.2 Default Server Configuration Parameters.................................................

222

Table 12. PFTP Client Stanza Fields.................................................................................

259

Table 13. PFTP Client Interfaces Stanza Fields...............................................................

264

Table 14.

Multinode Table Stanza Fields..........................................................................

266

Table 15.

Network Options Stanza Fields.........................................................................

269

Table 16.

PFTP Daemon Stanza Description....................................................................

273

Table 17.

Transfer Agent Stanza Description..................................................................

287

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

13

Preface

About this book

The HPSS Installation Guide is for use both at system installation time as well as throughout the lifetime of the system. It will guide system administrators through the planning and installation of a new HPSS system. It also guides system administrators through the conversion process to upgrade existing HPSS systems to Release 6.2. It serves as a reference whenever the system is reconfigured by the addition, deletion, or modification of hosts, tape libraries, devices, or other components.

Chapter 1 discusses HPSS changes for Release 6.2. Chapter 2 gives an overview of HPSS technology.

Chapters 3-5 guide administrators of new HPSS systems through planning, system preparation, HPSS software installation, and configuration of the HPSS infrastructure.

Chapter 6 guides administrators of existing 4.5 or 5.1 HPSS systems through the conversion process, bringing those systems up to Release 6.2.

Conventions Used in This Book

Example commands that should be typed at a command line will be proceeded by a percent sign (‘%’) and be presented in a boldface courier font:

% sample command

Example command output and example contents of ASCII files will be presented in a courier font:

sample file line 1 sample file line 2

Any text preceded by a pound sign (‘#’) should be considered comment lines:

# This is a comment

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

14

Chapter 1. Release 6.2

This chapter summarizes HPSS changes for Release 6.2 into four categories: new features, retired features, deferred features, and changed features. Changes since release 4.5 and 5.1 are described.

1.1. New Features

This section describes the new HPSS features added to Release 6.2.

1.1.1. DCE Replacement

Previous HPSS releases used DCE to provide authenticated client/server remote procedure calls. Since DCE is no longer offered as a commercial product, it has been replaced in HPSS with a solution based on commercially available authentication services and ONC RPC technology.

The new HPSS infrastructure provides four primary sub-systems: POSIX Threads, Universal Unique Identifiers (UUIDs), ONC Remote Procedure Calls and a collection of security services. The security services include support for Kerberos and an LDAP based registry service. For more information on the new HPSS infrastructure, see Section 2.3.4: HPSS Infrastructure on page 44.

1.1.2. Linux Support

All HPSS servers are fully supported on Linux.

1.1.3. Security

The HPSS Security implementation, which was based on DCE security, has been replaced with Unix or MIT Kerberos authentication and with Unix or LDAP authorization. Refer to Chapter 2: Security and System Access of the HPSS Management Guide and Section 3.2.7: Security on page 57 for more information.

1.1.4. SCSI PVR

Support for SCSI connected tape libraries has been added to HPSS 6.2 The SCSI PVR can be used to control any SCSI command based tape library.

1.1.5. HPSS VFS Interface

The HPSS VFS Interface provides a POSIX I/O interface to the HPSS filesystem. The root of the HPSS directory tree or a subdirectory can be mounted as a filesytem. HPSS files can be accessed by any software that complies with the POSIX I/O API by using the HPSS VFS Interface. For more information, please refer to Section 13.4: HPSS VFS Interface Configuration of the HPSS Management Guide.

1.1.6. GridFTP Interface

The GridFTP interface uses the PIO interface to transfer data into/from HPSS.

1.1.7. SAN3P/PIO Support

PIO can be used to issue I/O requests to direct attached HPSS SAN3P disk cache.

1.1.8. Additional hpssadm Configuration Options

Class of Service (COS)

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

15

Storage Class (but not subsystem-specific storage class options)

Global configuration

Accounting policy

Location policy

All server configuration. Newly supported options include Core Server, Gatekeeper, Location Server, Log Daemon, Migration/Purge Server, PVL, all PVRs, and SSM. The Mover, Log Client, and Startup Daemon were previously supported and still are.

Modifying devices and drives. This is the equivalent functionality of modifying fields on the device info and drive info screens in the GUI.

1.1.9. Additional hpssadm operations

Import/export

Create/delete resources

Task monitoring. A new "task" command displays the status of any long running background tasks submitted during the session, such as imports or resource creates.

Specifying volumes from a file for import/export/delete/move. hpssadm, like the GUI, now allows the user to submit a file listing the target volumes for import, export, delete resources, and cartridge move operations. This is an alternative to the previous capability of building the list on the screen with the Fill Count, Fill Increment, and Volume Label fields, which is still provided. As a result of adding this capability, the group labels on the structures have changed so scripts which use the old method to specify the volume list will need a slight modification. See the hpssadm man page for examples.

1.1.10. Additional Library and Device Support

Spectra Logic library

ADIC Scalar i500 library

IBM and HP LTO Generation 3 and 4 drives. Tape encryption has not been certified with HPSS.

SAIT-1 drives

IBM TS1120 (3592 Generation 2 and 3) drives. Tape encryption has not been certified with HPSS.

Sun StorageTek T10000 drives

Sun StorageTek 9840D drives

DataDirect Networks (DDN} disks

1.1.11. SAN Virtual Volume ID Mapping

A feature to consistently map device pathnames to volumes. A unique ID is assigned to each volume. The IDs are then mapped to device pathnames.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

16

1.1.12. Drive Pools

HPSS provides HPSS end clients the ability to direct tape read I/O requests to a predefined group of tape drives referred to as a Drive Pool. This ability helps HPSS administrators manage tape drive scheduling and thus availability. For more information, please refer to Section 7.3: Drive Pools of the

HPSS Management Guide.

1.1.13. FTP Enhancement

FTP application mode: FTP can be compiled as either a 32-bit or a 64-bit application, by setting the BIT64_SUPPORT flag on or off in the Makefile.macros file, when building with the HPSS FTP source extracted with "build-ftp" option. The 64-bit Kerberos libraries will be required to support Kerberos authentication.

PFTP client is now supported on Solaris 10 (Intel/AMD only).

1.1.14. mkhpss Enhancement

A new option is provided by mkhpss to configure the DB2 log mirroring capability to better protect the DB2 transaction log files. DB2 log mirroring allows the database to write an identical second copy of log files to a different path. The DB2 log files are essential in protecting a database and enable transactions occurring after a database backup to be recovered. Loss of the DB2 log files can result in significant impact to the operation of the system, extensive downtime, and potential loss of data. It is vital that DB2 log files be stored on highly reliable disk systems. In addition, DB2 log mirroring must be used to protect the DB2 log files on two separate disk systems. The disk systems must also be protected using RAID1 or RAID5 configurations. This mkhpss enhancement can only be used for new HPSS installations. For existing sites, please consult with your HPSS Support Representative for assistance.

1.1.15. DB2 Monitoring

A new feature to have the HPSS DB2 Log Monitor periodically checks the DB2 transaction log files to make sure that the primary and mirror logs are congruent and performing normally. It also scans the DB2 diagnostic log for indications of problem with the DB2 log files. For each identified problem, it will issue an HPSS alarm and indicate the path of the diagnostic log and the line number where the error is reported. The administrator should then open the DB2 diagnostics file and examine the reported problem in detail.

1.1.16. File Family enhancements

The number of allowed file families is increased to 4,294,967,295. In addition, authorized callers can assign the File Family ID of a file via the hpss_FileSetAttributes Client API function, independently of the underlying fileset.

1.1.17. SSM Configuration Files Consolidation

The SSM configuration files ssm_krb5.conf and ssm_unix.conf have been consolidated into a single configuration file called ssm.conf. Users may continue using their existing configuration file by specifying the "-m" option to the hpssadm.pl or hpssgui.pl script, or they may rename their existing configuration file to ssm.conf. For environments with multiple authentication mechanisms, it may be desirable to maintain multiple configuration files.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

17

1.1.18. Mover Enhancement

Multiple Movers for one or more HPSS instances can now be configured to run on the same machine. The -c <alternate var path> flag is added to the Mover entry in the inetd configuration file to specify an alternate "/var/hpss" path to be used by the Mover. In addition, the -s flag is added to enable syslog logging for the Mover. This feature is intended for multiple HPSS test systems to share the same set of Mover machines.

1.2. Retired Features

HPSS is no longer supported fully on Solaris. HPSS Mover and HPSS clients are still being supported on Solaris.

The settapestats utility program is no longer supported. The 6.2 Core Server calculates tape volume statistics when it starts.

STK RAIT PVR is no longer supported.

LTO PVR is no longer supported on Linux. The SCSI PVR should be used for LTO drives on Linux.

Non-DCE Gateway is no longer supported.

Federated Name Space and Remote Site Policy are not supported in 6.2.

Mirrored Filesets are no longer supported.

The terminology of "Non-DCE" Movers is no longer being used. It is now referred to as the Mover.

The terminology of "Non-DCE" Client API is no longer being used. It is now referred to as the Client API.

The hpss_PVrr utility is no longer supported. The source code for this utility can be found in /opt/hpss/tools/unsupported.

The hpss.clean command is no longer supported.

The HPSS Metadata Backup Tools are no longer supported.

1.3. Deferred Features

XFS is not supported in HPSS 6.2. XFS references have been left in the HPSS documentation to support the option of re-enabling XFS in future releases.

1.4. HPSS Changes

This section describes the changes made to HPSS 6.2.

1.4.1. Documentation Organization Changes

The SSM Guide is now merged with the Management Guide.

1.4.2. Metadata Changes

Added the AUTHZACL table to maintain security ACLs for HPSS servers and for SSM client access to SSM.

Deleted the DISKSPACE table. The disk space allocation maps are now maintained in the

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

18

Core Server's memory image

Modified the DMG table. In support of the new HPSS RPC library, the TCP port was eliminated and the Program and Version numbers were added to DMG specific configuration.

Modified the DMGFILESET table. The TCP port was eliminated and the TCP hostname and RPC endpoint information was added.

Modified the GATEKEEPER table. The length of site policy pathname was changed from 127 to 1023 characters.

Modified the LSPOLICY table. The group name was eliminated and realm name was added. Changed length of local site name from 31 to 255 characters.

Modified the MOVERDEVICE table. In support of the new SAN3P capabilities, SanID was added to mover device configuration. Since this is new to HPSS 6.2, these will be set to 0 for all mover devices.

The NDCG table is now obsolete. It has been renamed, but left intact.

Modified the NFS table. In support of the new authentication mechanisms, the credential object ID has been eliminated. Privileged caller principal length was changed from 15 to 255 characters.

SSSTATS table is now obsolete. It has been renamed, but left intact.

Modified the SERVER table. In support of the new HPSS RPC library, the authorization service and authentication service information has been eliminated. Request queue size information, RPC program, version number, and authentication mechanism information has been added, along with new index definitions.

Added the SERVERINTERFACES table. This table is populated by the metadata converter programs with newly created server interface information.

Modified the SITE table. The descriptive and LS group names have been eliminated, and site and realm names were added. The pre-6.2 SITE table will be renamed to PRE62_SITE but the metadata in the table will not be converted into the new SITE table since remote sites are no longer supported.

Modified the STORAGESEGDISK table. A new index is required for this table to support the new disk space allocation mechanism.

1.4.3. DMAP Gateway Changes

Since DFS is no longer supported, the logic to support DFS in the DMAP Gateway has been removed. DMAP Gateway will only support XFS in 6.2. Changes were also made in the underlying RPC mechanism used for communication with the HDM. The ability to delete the DMAP Gateway and Core Server components of an XDSM Fileset have been removed. Changes made to the SSM windows to support XFS are described below.

XFSisnotsupportedinHPSS6.2.XFSreferenceshavebeenleftintheHPSSdocumentationtosupporttheoptionofre-enablingXFSinfuturereleases.

1.4.3.1.Creating an XDSM Fileset

The “Create HPSS/XDSM Fileset” window has been renamed ”Create XDSM Fileset”. This window is currently not accessible since XFS is not supported.

Fields specific to DFS fileset creation have been removed. Since HPSS managed XFS

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

19

filesystems do not support mirrored namespaces, fields which were specific to managing mirrored filesets are also no longer available. This includes the following configuration options:

·Global Mount Point

·Local Mount Point

·Fileset Owner

·Fileset Permissions

Communication between the HDM and the DMAP Gateway now uses the HPSS RPC library. Fields which were used to specify TCP communication information have been removed.

1.4.3.2. Viewing DMAP Gateway XDSM Fileset Information

Fields and functions which were needed for DFS or mirrored fileset management have been removed. This includes the following changes:

·The ability to change the name of an XDSM Fileset.

·The ability to change the fileset state (i.e.: mounted, pre-unmounted etc.).

·The "Name Server Object Handle of Fileset Mountpoint" data is no longer shown.

·The ability to delete the DMAP Gateway portion of an XDSM Fileset independently from the core server is no longer supported.

Fileset information and associated statistics are now shown on a single window. The "Requests via TRPC (HPSS) Interface" statistics were specific to mirrored filesets and have been removed.

The RPC endpoint for the HDM is shown instead of the "HPSS/DMAP TCP Hostname" and "HPSS/DMAP TCP Port" fields. This reflects the change in the underlying communication mechanism used between the HDM and the DMAP Gateway. Additionally, these fields may no longer be modified from the DMAP Gateway Fileset Information window.

1.4.3.3. Viewing Core Server XDSM Fileset Information

The name of this window has been changed from "Name Server Fileset Information" to "Core Server Fileset Information".

The ability to modify the following fields from this window is no longer allowed:

·Fileset ID

·Fileset Name

·User and Group ID

·Fileset Type

·Permissions

The following informational fields have been removed:

·Name Server Object Handle

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

20

1.4.4. SSM Changes

Significant changes were made in SSM between Releases 4.5 and 5.1 and again between 5.1 and 6.2. For the reader's convenience, all changes between 4.5 and 6.2 are summarized in Section 3.3.4.1:

Changes Affecting Sites Upgrading Directly from 4.5. Changes between 5.1 and 6.2 are summarized in Section 3.3.4.2: Changes Affecting Sites Upgrading from 5.1. The tables indicate whether each change affects the SSM Server (the System Manager), the SSM GUI hpssgui, and/or the HPSS command line utility hpssadm. Sites need consult only the table which pertains to them.

1.4.4.1. Changes Affecting Sites Upgrading Directly from 4.5

Changes since 4.5

Server

GUI

ADM

 

 

 

 

The SSM System Manager and Data Server were consolidated in release

n

 

 

6.2 into a single server, the SSM System Manager.

 

 

 

 

 

 

 

Java is now required for the SSM Graphical User Interface (hpssgui). Java

n

n

 

is no longer required by the SSM server (the new consolidated SSM

 

 

 

System Manager).

 

 

 

SSL is no longer supported with SSM. This means it is no longer

n

n

n

necessary to create an SSL certificate (the /var/hpss/ssm/ds.cer file) or

 

 

 

import it into the certificate authority file ($JAVA_ROOT/jre/lib/cacerts)

 

 

 

on each SSM client machine.

 

 

 

The Java Security Manager is no longer used with SSM. This means the

n

n

n

Java Policy files are no longer required. In 4.5, these files were the

 

 

 

java.policy.ds and the java.policy.hpssadm.

 

 

 

 

 

 

 

The hpssadm.jar and mobjects.jar files are no longer used. All classes are

 

n

n

now in a single jar file hpss.jar.

 

 

 

The start_ssm script is no longer available. The SSM System Manager is

n

 

 

started with the rc.hpss script. The rc.hpss script has been moved from a

 

 

 

default location of /etc to /opt/hpss/bin.

 

 

 

 

 

 

 

The start_ssm_session script is no longer available. In HPSS 6.2, the SSM

 

n

 

GUI is started with the hpssgui startup script. There are perl and vbs

 

 

 

versions of this script, named with an extension to indicate the type of

 

 

 

script: hpssgui.pl and hpssgui.vbs.

 

 

 

There are new versions of the hpssadm startup scripts in perl and vbs.

 

 

n

These are named with an extension to indicate the type of script:

 

 

 

hpssadm.pl and hpssadm.vbs. The ksh and bat scripts are no longer

 

 

 

supported.

 

 

 

Options to the new hpssgui startup scripts differ considerably from the

 

n

n

options to the old start_ssm_session script. Options to the hpssadm startup

 

 

 

scripts have also changed significantly. See the man pages for details.

 

 

 

Following are some highlights:

 

 

 

The scripts are dependent upon an SSM configuration file

 

n

n

(ssm.conf), created by mkhpss, which contains some site-specific

 

 

 

configuration values. The SSM client script will attempt to ready

 

 

 

the SSM configuration file in the current working directory unless

 

 

 

the pathname to this file is specified by the -m option.

 

 

 

 

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

21

Changes since 4.5

Server

GUI

ADM

 

 

 

 

The SSM client scripts now use an internal polling mechanism for

 

n

n

getting window updates (as opposed to being notified of the update

 

 

 

by the server). This means that:

 

 

 

The SSM client application no longer requires a second

 

 

 

port for two-way communication,

 

 

 

The rate of polling can be fine-tuned by the user (see

 

 

 

the -A, -G, -L -M, -W options).

 

 

 

 

 

 

 

The scripts are dependent upon a login.conf file and, if using

 

n

n

Kerberos authentication, a krb5.conf file. Both files are created by

 

 

 

hpssuser. The scripts will look in the current working directory for

 

 

 

these files unless they are specified on the command line (-C and -

 

 

 

k options).

 

 

 

 

 

 

 

It is no longer necessary to add a security provider entry to the java

 

n

n

system java.security file.

 

 

 

The pathname to the java executable can be specified by using the

 

n

n

-j option. If this option isn't specified, then the script will use the

 

 

 

java executable in $JAVA_BIN if it is valid. If $JAVA_BIN/java

 

 

 

is not defined, then the script will use 'java' which will use the

 

 

 

client's $PATH.

 

 

 

Effective with HPSS 5.1.1, the hpssgui script supports customizing

 

n

 

the user's background color or Look & Feel using the -b, -F and -T

 

 

 

options.

 

 

 

The SSM user can specify the number of alarms to display on the

 

n

n

alarm list using the -N option.

 

 

 

The SSM user can specify the pathname to the hpss.jar using the -P

 

n

n

option.

 

 

 

The SSM user can specify the path to search for application

 

n

n

preferences using the -i option. This path is now local to the SSM

 

 

 

client machine; preferences are no longer stored on the SSM server

 

 

 

machine.

 

 

 

Sites that are converting from a previous release of HPSS should replace

 

n

n

all their hpssgui and hpssadm startup scripts on all client platforms with

 

 

 

the new versions.

 

 

 

In HPSS 6.2, ssh tunneling communications between the SSM client and

n

n

n

server is supported.

 

 

 

The SSM clients can contact the System Manager across a Virtual Private

n

n

n

Network connection (VPN). See the -p and -h options on the hpssgui and

 

 

 

hpssadm man pages.

 

 

 

 

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

22

Changes since 4.5

Server

GUI

ADM

 

 

 

 

The SSM client scripts can use ports exempted by the network

n

n

n

administrator as firewall exceptions. See the -n option on the

 

 

 

hpssadm/hpssgui man pages.

 

 

 

The port on which the System Manager will listen may be controlled by

 

 

 

setting the $HPSS_SSM_SERVER_LISTEN_PORT environment variable.

 

 

 

The default setting is 0 (zero) which means that the port will be chosen by

 

 

 

the RPC portmapper. See Section 3.3.6: Using SSM Through a Firewall,

 

 

 

of the HPSS Management Guide.

 

 

 

Access to port 88 is needed for Kerberos authentication. Access to port

n

n

n

111 is needed if the RPC portmapper is needed to find the System

 

 

 

Manager.

 

 

 

The user_authorization.dat and hpssadm.config files provided in HPSS 4.5

n

n

n

are no longer available.

 

 

 

In 6.2, all authorized SSM users and their privilege levels are defined in

 

 

 

the DB2 user acl AUTHZACL table. See Section 3.3.2.1: hpssuser Utility

 

 

 

and Section 3.3.2.2: SSM User Authorization in the HPSS Management

 

 

 

Guide for details.

 

 

 

The hpssgui and hpssadm programs will now optionally record every error

 

n

n

and informational message in an ASCII log file. The "-S" option for both

 

 

 

programs is used to specify whether to create the session log and what to

 

 

 

name it.

 

 

 

The basic and specific server config structures, along with the server's log

 

n

n

policy, have been combined in 5.1 into a single structure.

 

 

 

SSM now represents the three basic Core Server volume structures, the

 

n

n

physical volume, virtual volume, and storage map, as a single combined

 

 

 

Core Server volume structure.

 

 

 

 

 

 

 

The delog utility is no longer available via the SSM GUI. This feature has

 

n

 

been retired.

 

 

 

The ability to broadcast messages to other SSM GUI users is not yet

 

n

 

available.

 

 

 

Alarms and events are no longer acknowledged from the SSM GUI.

 

n

 

 

 

 

 

The SSM clients can specify the number of items to display for the Alarms

 

n

n

and Events list. See the -N option on the hpssadm/hpssgui man page and

 

 

 

Section 9.6: Managing SSM Alarms and Events in the HPSS Management

 

 

 

Guide.

 

 

 

 

 

 

 

The SSM clients keep an internal cached copy of the Alarms and Events

 

n

n

list which is maintained by regular polling requests to the System

 

 

 

Manager. Each polling request returns only “new” messages, those which

 

 

 

have not already been retrieved by the SSM client in a previous polling

 

 

 

request. If there are no new messages in the System Manager log cache,

 

 

 

no messages are returned. This will help performance of the display of

 

 

 

this window. See the -A, -G and -N options of the hpssadm/hpssgui man

 

 

 

pages and Section 9.6.5: Controlling SSM Log Message handling in the

 

 

 

HPSS Management Guide.

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

23

Changes since 4.5

Server

GUI

ADM

 

 

 

 

The ability to see which users are logged into SSM, referred to in HPSS

 

n

n

4.5 as a list of SSM "consoles", is available in 6.2 as part of the System

 

 

 

Manager Statistics window from both the hpssgui and hpssadm.

 

 

 

The menu bar has been reorganized extensively.

 

n

 

The "Set Keyboard", "Show Sammi Environment", "View Sammi Errlog",

 

 

 

and "About Sammi" items from the 4.5 Session menu and the "SSM

 

 

 

consoles" item from the 4.5 Monitor menu are no longer available.

 

 

 

Some of the information about the gui session and data server which was

 

 

 

formerly available from these screens is now available in 6.2 from the

 

 

 

"System Manager Statistics" and "User Session Information" items from

 

 

 

the "SSM Information" submenu of the "Monitor" menu. Some

 

 

 

information is also available in the new user session log. See Chapter 3:

 

 

 

Using SSM in the HPSS Management Guide for a full description of the

 

 

 

layout of the menu and a description of the functions available from each

 

 

 

menu item.

 

 

 

 

 

 

 

The Monitor, Operations, and Configure menus are available in a menu

 

n

 

tree on the bottom half of the Health and Status window. The tree is

 

 

 

organized exactly as the corresponding items from the menu bar, and the

 

 

 

functionality of each item is the same as its corresponding item on the

 

 

 

menu bar.

 

 

 

The View menu of the Health and Status screen allows some portions of

 

n

 

the screen to be hidden or redisplayed.

 

 

 

The PVR Cartridge Threshold is included in the HPSS Status section of

 

n

n

the Health and Status information. When the status is something other

 

 

 

than OK, the PVR Information button on the GUI screen will become

 

 

 

activated; clicking this button will bring up a window for each PVR

 

 

 

cartridge that is not healthy.

 

 

 

To support users with color blindness or other visual problems, all text is

 

n

 

now displayed on white or light gray backgrounds. Color (green, yellow,

 

 

 

red) is still used to indicate the severity level of alarms and component

 

 

 

statuses, but the color is confined to a small sphere displayed next to the

 

 

 

text. For example, a major alarm might be displayed on the Alarms and

 

 

 

Events screen as:

 

 

 

May 2, 2005 11:26:13 AM Major PVL Bad things happened

 

 

 

 

 

 

 

More than one preference may now be defined for each list. See the

 

n

 

description of the "preferences combo box" in Section 3.6: Common

 

 

 

Window Elements of the HPSS Management Guide for details.

 

 

 

Note that the mechanism for selecting columns to be displayed in each list

 

n

 

window has moved from the list's Preferences window to the "Column

 

 

 

View" menu of the list window itself. See the description of the "Column

 

 

 

View" menu item in Section 3.6:Common Window Elements of the HPSS

 

 

 

Management Guide for details.

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

24

Changes since 4.5

Server

GUI

ADM

 

 

 

 

Column ordering is now controlled by dragging columns to the

 

n

 

desired location. The modified order is preserved automatically in the

 

 

 

user preferences across hpssgui restarts.

 

 

 

When messages have been written to the status bar, the most recent

 

n

 

messages can be viewed in the status bar's tooltip. Rolling the mouse over

 

 

 

the status bar without clicking gives a tooltip that says, "Click mouse in

 

 

 

status bar to view messages" if there are status messages to view. If there

 

 

 

are no status messages then the tooltip says, "No status messages". This

 

 

 

message stays up for about 4 seconds or until the user moves the mouse

 

 

 

out of the status bar area. To view up to the last STATUS_MSG_MAX

 

 

 

(30) messages that have been written to the status bar, click on the status

 

 

 

bar. The tooltip that is displayed will show up to the last 30 messages and

 

 

 

will remain visible until the mouse is moved out of the status bar or for 10

 

 

 

minutes.

 

 

 

 

 

 

 

Most user updates to fields on information windows are no longer sent to

 

n

 

the server immediately or automatically. The Update button on the

 

 

 

window must be clicked to submit the update to the server. Fields

 

 

 

modified by the user will be flagged by a diskette icon to indicate the

 

 

 

window copy has been modified but not saved.

 

 

 

The list of volumes for import, export, resource deletes, and cartridge

 

n

n

move operations may now be specified from an external input file as an

 

 

 

alternative to building the list on the hpssgui screen or within the hpssadm

 

 

 

operation.

 

 

 

 

 

 

 

Since the basic and specific server configuration structures have been

 

 

n

combined into a single structure, there is no longer a need for a

 

 

 

-server_type option to the hpssadm config command. This option is no

 

 

 

longer used.

 

 

 

The types of configuration structures supported by the -type option now

 

 

 

are specified by the title names used on the config struct windows. See the

 

 

 

hpssadm man page for a list of supported config types.

 

 

 

Any type name may be abbreviated so long as the abbreviation is unique.

 

 

 

See the hpssadm man page for details.

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

25

 

Changes since 4.5

Server GUI ADM

 

 

The HPSS 4.5 hpssadm commands

n

pvr_cartridge

 

pvl_volume

 

ss_pv

 

ss_map

 

ss_vv

 

have been replaced by a single command "volume". The volume command has a required option "-type" for which these types may be specified:

PVL Volume Information

 

PVR Cartridge Information

 

CS Disk Volume

 

CS Tape Volume

 

The new volume command behaves similarly to the config command,

 

where the types are the titles of the window screens for the structures. The

 

type names may be abbreviated so long as the abbreviation is unique.

 

The volume command has several subcommands which support displaying

 

and updating the volume, importing and exporting volumes, creating and

 

deleting resources, and moving cartridges.

 

There is no longer a separate command for locking or unlocking a volume

 

in 6.2. This has been replaced by the practice of setting the VV Condition

 

and Retired fields structure on the storage map in the disk or tape volume

 

structure. (This is a core server design change.) The fields of the map

 

structure may be modified with the update command. The update

 

command may also be used on any of the volume commands to modify any

 

settable field.

 

A new hpssadm command "ssm" is supplied in 6.2 for shutting down the

n

System Manager and for displaying the status of the System Manager and

 

of the hpssadm session.

 

 

 

All available columns of each SSM list (server, device, storage class, pvl

n

job, alarm) are now displayed by the hpssadm list commands in the system

 

default order. At this time hpssadm has no facility for customizing the list

 

format as the gui does.

 

Field layout and grouping for each structure displayed by the hpssadm now

n

matches the order and grouping used by the corresponding window of the

 

SSM GUI.

 

 

Most hpss structures have changed so hpssadm scripts used before 5.1 may

n

need to be modified to use the new field and subfield names.

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

26

 

Changes since 4.5

Server

GUI

ADM

 

 

 

 

The hpssadm config command now supports the additional structures:

 

 

n

Class of Service Config

 

 

 

Storage Class (but not subsystem-specific storage class options)

 

 

 

Global Config

 

 

 

Accounting Policy

 

 

 

Location Policy

 

 

 

All server configurations; servers which were not supported before are

 

 

 

 

the core server, gatekeeper, location server, log daemon,

 

 

 

 

migration/purge server, pvl, all pvrs, and ssm. The mover, log client

 

 

 

 

and startup daemon are still supported as previously.

 

 

 

A new command “task” displays the status of any long running background

 

 

n

tasks submitted during the session, such as imports or resource creates.

 

 

 

A new subcommand “update” has been added to the hpssadm device

 

 

n

command for updating the mover device and pvl drive objects.

 

 

 

 

 

 

 

 

1.4.4.2. Changes Affecting Sites Upgrading from 5.1

Changes since 5.1

Server

GUI

ADM

 

 

 

 

The SSM System Manager and Data Server were consolidated in release

n

 

 

6.2 into a single server, the SSM System Manager.

 

 

 

 

 

 

 

Java is no longer used in the server side of SSM (the new consolidated

n

 

 

SSM System Manager).

 

 

 

SSL is no longer supported with SSM. This means it is no longer

n

n

n

necessary to create an SSL certificate (the /var/hpss/ssm/ds.cer file) or

 

 

 

import it into the certificate authority file ($JAVA_ROOT/jre/lib/cacerts)

 

 

 

on each ssm client machine.

 

 

 

The Java Security Manager is no longer used with SSM. This means the

n

n

n

Java policy files are no longer required. In 5.1, these files were the

 

 

 

java.policy.ds and the java.policy.ssmuser.

 

 

 

 

 

 

 

The start_ssm script is no longer available. The SSM System Manager is

n

 

 

started with the rc.hpss script. The rc.hpss script has been moved from a

 

 

 

default location of /etc to /opt/hpss/bin.

 

 

 

There are new versions of the hpssgui and hpssadm startup scripts in perl

 

n

n

and vbs. These are named with an extension to indicate the type of script:

 

 

 

hpssadm.pl, hpssadm.vbs, hpssgui.pl and hpssgui.vbs. The ksh and bat

 

 

 

scripts are no longer supported.

 

 

 

The scripts are no longer distributed as templates; they will be packaged

 

 

 

into the $HPSS_PATH_BIN (default /opt/hpss/bin) directory.

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

27

Changes since 5.1

Server

GUI

ADM

 

 

 

 

Options to the hpssgui and hpssadm startup scripts have changed

 

n

n

significantly. See the man pages for details. Following are some

 

 

 

highlights:

 

 

 

The scripts are dependent upon an ssm configuration file

 

n

n

(ssm.conf), created by mkhpss, which contains some site-specific

 

 

 

configuration values. The SSM client script will attempt to read

 

 

 

the ssm configuration file in the current working directory unless

 

 

 

the pathname to this file is specified by the -m option.

 

 

 

 

 

 

 

The SSM client scripts now use an internal polling mechanism for

 

n

n

getting window updates (as opposed to being notified of the update

 

 

 

by the server). This means that:

 

 

 

the SSM client application no longer requires a second

 

 

 

port for two-way communication,

 

 

 

using SSM across an ssh tunnel is simpler since you

 

 

 

now need to set up the tunnel only in one direction,

 

 

 

the rate of polling can be fine-tuned by the user (see the

 

 

 

-A, -G, -L -M, -W options).

 

 

 

The scripts are dependent upon a login.conf file and, if using

 

n

n

Kerberos authentication, a krb5.conf file. The hpssuser program

 

 

 

creates both of these files. The scripts will look in the current

 

 

 

working directory for these files unless they are specified on the

 

 

 

command line (-C and -k options).

 

 

 

 

 

 

 

It is no longer necessary to add a security provider entry to the java

 

n

n

system java.security file.

 

 

 

The pathname to the java executable can be specified by using the

 

n

n

-j option. If this option isn't specified, then the script will use the

 

 

 

java executable in $JAVA_BIN if it is valid. If $JAVA_BIN/java

 

 

 

is not defined, then the script will use 'java' which will use the

 

 

 

client's $PATH.

 

 

 

Effective with HPSS 5.1.1, the hpssgui script supports customizing

 

n

 

the user's background color or Look & Feel using the -b, -F and -T

 

 

 

options. Sites which did not apply the 5.1 patch would not have

 

 

 

had this functionality.

 

 

 

The SSM user can specify the number of alarms to display on the

 

n

n

alarm list using the -N option.

 

 

 

 

 

 

 

The SSM user can specify the pathname to the hpss.jar using the -

 

n

n

P option.

 

 

 

 

 

 

 

The SSM user can specify the path to search for application

 

n

n

preferences using the -i option. This path is now local to the SSM

 

 

 

client machine; preferences are no longer stored on the SSM server

 

 

 

machine.

 

 

 

 

 

 

 

Sites which are converting from a previous release of HPSS should be

 

n

n

certain to replace all their hpssgui and hpssadm startup scripts on all client

 

 

 

platforms with the new versions.

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

28

Changes since 5.1

Server

GUI

ADM

 

 

 

 

The SSM client script options for connecting to the System Manager

n

n

n

across a Virtual Private Network connection (VPN) have changed. See the

 

 

 

-p and -h options on the hpssgui and hpssadm man pages.

 

 

 

The SSM client script option for using ports exempted by the network

n

n

n

administrator as firewall exceptions has changed; see the -n option on the

 

 

 

hpssadm/hpssgui man pages.

 

 

 

The port on which the System Manager will listen may be controlled by

 

 

 

setting the $HPSS_SSM_SERVER_LISTEN_PORT environment variable.

 

 

 

The default setting is 0 (zero) which means that the port will be chosen by

 

 

 

the RPC portmapper. See Section 3.3.6:Using SSM Through a Firewall,

 

 

 

of the HPSS Management Guide.

 

 

 

Access to port 88 is needed for Kerberos authentication and access to port

n

n

n

111 is needed if the RPC portmapper is needed to find the System

 

 

 

Manager.

 

 

 

 

 

 

 

The ssmuser.config file provided in HPSS 5.1 is no longer available. In

n

n

n

6.2, all authorized SSM users and their privilege levels are defined in the

 

 

 

DB2 user acl AUTHZACL table. See Section 3.3.2.1: hpssuser Utility and

 

 

 

Section 3.3.2.2: SSM User Authorization in the HPSS Management Guide

 

 

 

for details.

 

 

 

 

 

 

 

SSM now represents the three basic Core Server volume structures, the

 

n

n

physical volume, virtual volume, and storage map, as a single combined

 

 

 

Core Server volume structure.

 

 

 

 

 

 

 

Access to the delog utility via the SSM GUI, deferred in 5.1, has now been

 

n

 

permanently retired.

 

 

 

 

 

 

 

The ability to broadcast messages to other SSM GUI users is not yet

 

n

 

available.

 

 

 

The SSM clients can specify the number of items to display for the Alarms

 

n

n

and Events list. See the -N option on the hpssadm/hpssgui man page and

 

 

 

Section 9.6: Managing SSM Alarms and Events in the HPSS Management

 

 

 

Guide.

 

 

 

The SSM clients keep an internal cached copy of the Alarms and Events

 

n

n

list which is maintained by regular polling requests to the System

 

 

 

Manager. Each polling request returns only “new” messages, those which

 

 

 

have not already been retrieved by the SSM client in a previous polling

 

 

 

request. If there are no new messages in the System Manager log cache,

 

 

 

no messages are returned. This will help performance of the display of

 

 

 

this window. See the -A, -G and -N options of the hpssadm/hpssgui man

 

 

 

pages and Section 9.6.5: Controlling SSM Log Message Handling in the

 

 

 

HPSS Management Guide.

 

 

 

The ability to see which users are logged into SSM, referred to in HPSS

 

n

n

4.5 as a list of SSM "consoles", was not yet available in 5.1. It is now

 

 

 

available in 6.2 as part of the System Manager Statistics window from both

 

 

 

the hpssgui and hpssadm.

 

 

 

 

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

29

Changes since 5.1

Server

GUI

ADM

 

 

 

 

The menu bar has been reorganized slightly.

 

n

 

The "Data Server Statistics" menu item has been replaced by the "System

 

 

 

Manager Statistics" menu item, available from the Monitor->SSM

 

 

 

Information menu path.

 

 

 

"Column View" was added to the menu bar for SSM windows that display

 

 

 

an SSM table. See Section 3.6: Common Window Elements of the HPSS

 

 

 

Management Guide. In HPSS 5.1, this ability was part of the Preference

 

 

 

window associated with the SSM table window.

 

 

 

The “User Session Log” is no longer available from the SSM GUI; the

 

 

 

user can still view this information by using the -S option of the SSM

 

 

 

client script and viewing the ASCII text of the session log. Since this was

 

 

 

removed from the menu bar, the Monitor->Logging menu path which

 

 

 

contained two sub-items (Log Files Info and User Session Log) was

 

 

 

replaced by Monitor->Log Files Information menu path.

 

 

 

The Monitor->Lookup HPSS Objects->Security menu path option has

 

 

 

been removed.

 

 

 

The Operations->Filesets & Junctions->Create XDSM Fileset menu path

 

 

 

option was added. However, this option is currently hidden since XFS is

 

 

 

not supported.

 

 

 

Since the two SSM Servers (Data Server and System Manager) were

 

 

 

merged into the single System Manager. all references to the Data Server

 

 

 

in the Operations->Shutdown menu were removed.

 

 

 

Configure->Restricted Users menu path option was added. It currently

 

 

 

supports listing the restricted users and a “Reload List” button which

 

 

 

causes the Core Servers to reread the HPSS_RESTRICTED_USER_FILE.

 

 

 

See Chapter 2: Security and System Access in the HPSS Management

 

 

 

Guide for a full description of restricting user access to HPSS.

 

 

 

Since the SSM Guide was merged into the HPSS Management Guide and

 

 

 

HPSS Installation Guide, the Help->HPSS Books->SSM Guide menu path

 

 

 

option was removed.

 

 

 

See Chapter 3: Using SSM in the HPSS Management Guide for a full

 

 

 

description of the layout of the menu and a description of the functions

 

 

 

available from each menu item.

 

 

 

 

 

 

 

The PVR Cartridge Threshold is included in the HPSS Status section of

 

n

n

the Health and Status information. When the status is something other

 

 

 

than OK, the PVR Information button on the GUI screen will become

 

 

 

activated; clicking this button will bring up a window for each PVR

 

 

 

cartridge that is not healthy.

 

 

 

The "Sick" preference may no longer be edited.

 

n

 

 

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

30

Changes since 5.1

Server

GUI

ADM

 

 

 

 

The mechanism for selecting columns to be displayed in each list window

 

n

 

has moved from the list's Preferences window to the "Column View" menu

 

 

 

of the list window itself. See the description of the "Column View" menu

 

 

 

item in Section 3.6: Common Window Elements of the HPSS Management

 

 

 

Guide for details.

 

 

 

Column ordering is now controlled solely by dragging columns to the

 

n

 

desired location. The modified order is now preserved automatically in the

 

 

 

user preferences across hpssgui restarts.

 

 

 

 

 

 

 

List tables have a field that shows the number of displayed and total items

 

n

 

in the list in the format X/Y where X is the number of items displayed and

 

 

 

Y is the total number of items in the list. The field is left justified under

 

 

 

the table. The X and Y values will differ if preferences are set to filter

 

 

 

some items out of the list.

 

 

 

The button panel to the right of the list can be hidden or displayed by

 

n

 

clicking the tall, thin button between the list and button panel labeled '||'. If

 

 

 

the button is pressed when the panel is displayed, the button panel will

 

 

 

hide, allowing more space for the list. The button panel may be re-

 

 

 

displayed by pressing the '||' button again.

 

 

 

Sites which are converting from HPSS 5.1 should remove all old user

 

n

 

preference files. New preference files will be created automatically for

 

 

 

the user when he first logs in. The user will be required to reset any

 

 

 

preference settings (i.e. preferences will not be converted).

 

 

 

The System Manager connection indicator is not displayed on every

 

n

 

window; it is only displayed on the Health and Status Window.

 

 

 

When messages have been written to the status bar, the most recent

 

n

 

messages can be viewed in the status bar's tooltip. Rolling the mouse over

 

 

 

the status bar without clicking gives a tooltip that says, "Click mouse in

 

 

 

status bar to view messages" if there are status messages to view. If there

 

 

 

are no status messages then the tooltip says, "No status messages". This

 

 

 

message stays up for about 4 seconds or until the user moves the mouse

 

 

 

out of the status bar area. To view up to the last 30

 

 

 

(STATUS_MSG_MAX) messages that have been written to the status bar,

 

 

 

click on the status bar. The tooltip that is displayed will show up to the last

 

 

 

30 messages and will remain visible until the mouse is moved out of the

 

 

 

status bar or for 10 minutes.

 

 

 

 

 

 

 

The list of volumes for import, export, resource deletes, and cartridge

 

n

n

move operations may now be specified from an external input file as an

 

 

 

alternative to building the list on the hpssgui screen or within the hpssadm

 

 

 

operation.

 

 

 

 

 

 

 

Most hpss structures have changed so hpssadm scripts used before 5.1 may

 

 

n

need to be modified to use the new field and subfield names.

 

 

 

 

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

31

 

 

Changes since 5.1

Server

GUI

ADM

 

 

 

 

Since SSM now represents the three Core Server volume structures as a

 

 

n

single structure, the types specified to the hpssadm volume command have

 

 

 

changed. The 5.1 types:

 

 

 

 

Disk Storage Map Information

 

 

 

 

Disk Physical Volume Information

 

 

 

 

Disk Virtual Volume Information

 

 

 

have been replaced in 6.2 by the single type:

 

 

 

 

CS Disk Volume

 

 

 

 

 

 

 

Similarly, the 5.1 types:

 

 

n

 

Tape Storage Map Information

 

 

 

 

Tape Physical Volume Information

 

 

 

 

Tape Virtual Volume Information

 

 

 

have been replaced in 6.2 by the single type:

 

 

 

 

CS Tape Volume

 

 

 

 

 

 

 

The hpssadm volume command now supports volume import and export,

 

 

n

resource creation and deletion, and cartridge moves.

 

 

 

On the Active Storage Classes window, the “Force Migrate” has been

 

n

 

replaced with a “Migration Controls” drop-down allowing the

 

 

 

Administrator or Operator to perform migration operations on all the

 

 

 

selected storage classes. Likewise, the “Force Purge” has been replaced

 

 

 

with a “Purge Controls” drop-down allowing the Administrator to perform

 

 

 

purge operations on all the selected storage classes.

 

 

 

 

 

 

 

The "-ds" option is no longer available for the ssm "info" or "shutdown"

 

 

n

commands. A new option "-sm" is available for the "info" command.

 

 

 

 

 

 

 

The hpssadm config command now supports the additional structures:

 

 

n

Class of Service Config

 

 

 

Storage Class (but not subsystem-specific storage class options)

 

 

 

Global Config

 

 

 

Accounting Policy

 

 

 

Location Policy

 

 

 

All server configurations; servers which were not supported before are

 

 

 

 

the core server, gatekeeper, location server, log daemon,

 

 

 

 

migration/purge server, pvl, all pvrs, and ssm. The mover, log client

 

 

 

 

and startup daemon are still supported as previously.

 

 

 

A new hpssadm command “task” displays the status of any long running

 

 

n

background tasks submitted during the session, such as imports or resource

 

 

 

creates.

 

 

 

 

 

 

 

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

32

Changes since 5.1

Server

GUI

ADM

 

 

 

 

A new subcommand “update” has been added to the hpssadm device

 

 

n

command for updating the mover device and pvl drive objects.

 

 

 

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

33

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

34

Chapter 2. HPSS Basics

2.1. Introduction

The High Performance Storage System (HPSS) provides hierarchical storage management and services for very large storage environments. HPSS may be of interest to organizations having present and future scalability requirements that are very demanding in terms of total storage capacity, file sizes, data rates, number of objects stored, and numbers of users. HPSS is part of an open, distributed environment based on remote procedure calls, Kerberos, LDAP directory systems and DB2 which form the infrastructure of HPSS. HPSS is the result of a collaborative effort by leading US Government supercomputer laboratories and industry to address very real, urgent high-end storage requirements. HPSS is offered commercially by IBM.

HPSS provides scalable parallel storage systems for highly parallel computers as well as traditional supercomputers and workstation clusters. Concentrating on meeting the high end of storage system and data management requirements, HPSS is scalable and designed to store up to petabytes (1015) of data and to use network-connected storage devices to transfer data at rates up to multiple gigabytes (109) per second.

HPSS provides a large degree of control for the customer to manage the hierarchical storage system. Using configuration information defined by the customer, HPSS organizes storage devices into multiple storage hierarchies. Based on policy information defined by the customer and actual usage information, data are then moved to the appropriate storage hierarchy and to appropriate levels in the storage hierarchy.

2.2. HPSS Capabilities

A primary goal of HPSS is to move large files between storage devices and parallel or clustered computers at speeds many times faster than today’s commercial storage system software products and to do this in a way that is more reliable and manageable than is possible with current systems. In order to accomplish this goal, HPSS is designed and implemented based on the concepts described in the following subsections.

2.2.1. Network-centered Architecture

The focus of HPSS is the network, not a single processor as in conventional storage systems. HPSS provides servers that can be distributed across a high performance network to provide scalability and parallelism. The basis for this architecture is the IEEE Mass Storage System Reference Model, Version 5.

2.2.2. High Data Transfer Rate

HPSS achieves high data transfer rates by eliminating overhead normally associated with data transfer operations. In general, HPSS servers establish and control transfer sessions but are not involved in actual transfer of data.

2.2.3. Parallel Operation

The HPSS Client Application Program Interface (Client API) supports parallel or sequential access to storage devices by clients executing parallel or sequential applications. HPSS also provides a Parallel File Transfer Protocol. HPSS can even manage data transfers in a situation where the number of data sources differs from the number of destination sources. Parallel data transfer is vital in situations that demand fast access to very large files.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

35

2.2.4. Based on Standard Components

HPSS runs on UNIX and is written in ANSI C and Java. It uses remote procedure calls, a selectable security service (Kerberos or UNIX), UNIX or LDAP for user configuration information, and DB2 as the basis for its portable, distributed, transaction-based architecture. These components are offered on many vendors’ platforms.

The full HPSS system has been implemented on IBM AIX and LINUX platforms, and some components of HPSS have been ported to other platforms. Refer to Section 2.4: HPSS Hardware Platforms on page 48 and Section 3.3: Prerequisite Software Considerations on page 58 for specific information.

To aid vendors and users in porting HPSS to new platforms, HPSS source code is available.

2.2.5. Data Integrity Through Transaction Management

Transactional metadata management, provided by DB2, enables a reliable design that protects user data both from unauthorized use and from corruption or loss. A transaction is an atomic grouping of metadata management functions such that either all of them take place together or none of them takes place. Journaling makes it possible to back out any partially complete transactions if a failure occurs. Transaction technology is common in relational data management systems but not in storage systems. It is the key to maintaining reliability and security while scaling upward into a large, distributed storage environment.

2.2.6. Multiple Hierarchies and Classes of Services

Most other storage management systems support simple storage hierarchies consisting of one kind of disk and one kind of tape. HPSS provides multiple configurable hierarchies, which are particularly useful when inserting new storage technologies over time. As new disks or tapes are added, new classes of service can be set up. HPSS files reside in a particular class of service which users select based on parameters such as file size and performance. A class of service is implemented by a storage hierarchy which in turn consists of multiple storage classes, as shown in Figure 2. Storage classes are used to logically group storage media to provide storage for HPSS files. A hierarchy may be as simple as a single tape, or it may consist of two or more levels of disk and local tape. The user can even set up classes of service so that data from an older type of tape is subsequently migrated to a new type of tape. Such a procedure allows migration to new media over time without having to copy all the old media at once.

2.2.7. Storage Subsystems

Storage subsystems (or simply, “subsystems”) may be used to increase the scalability of HPSS in handling concurrent requests or to meet local political needs. Each subsystem contains a single Core Server. If migration and purge are needed for the subsystem, then it will also contain a Migration / Purge Server. In addition, if HPSS is to be used as a backing store for a 'linked' filesystem such as XFS, a DMAP Gateway will be required. All other servers are subsystem-independent.

Data stored within HPSS is assigned to different subsystems based on pathname resolution. A pathname consisting of ‘/’ resolves to the root Core Server which is specified in the global configuration file. However, if the pathname contains junction components, it may resolve to a Core Server in a different subsystem. For example, the pathname ‘/JunctionToSubsys2/mydir’ could

lead to a fileset managed by the Core Server in subsystem 2. Sites which do not wish to partition their HPSS through the use of subsystems will run HPSS with a single subsystem.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

36

2.3. HPSS Components

The components of HPSS include files, filesets, junctions, virtual volumes, physical volumes, storage segments, metadata, servers, infrastructure, user interfaces, a management interface, and policies. Media and file metadata are represented by data structures that describe the attributes and characteristics of storage system components such as files, filesets, junctions, storage segments, and volumes. Servers are the processes that control the logic of the system and control movement of the data. The HPSS infrastructure provides the services that are used by all the servers for operations such as sending messages and providing reliable transaction management. User interfaces provide several different views of HPSS to applications with different needs. The management interface provides a way to administer and control the storage system and implement site policy.

These HPSS components are discussed below in Sections 2.3.1 through 2.3.7.

Figure 1. File Migration and Stage Operations

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

37

Figure 2. Class of Service / Hierarchy / Storage Class

2.3.1. HPSS Files, Filesets, Volumes, Storage Segments and Related Metadata

The various metadata constructs used to describe the HPSS namespace and HPSS storage are described below:

Files (Bitfiles). Files in HPSS, called bitfiles in deference to IEEE Mass Storage Reference Model terminology, are logical strings of bytes, even though a particular bitfile may have a structure imposed by its owner. This unstructured view decouples HPSS from any particular file management system that host clients of HPSS might use. HPSS bitfile size is limited to264 - 1 bytes.

Each bitfile is identified by a machine-generated name called a bitfile ID. It may also have a human readable name. It is the job of the HPSS Core Server (discussed in Section 2.3.2) to map a human readable name to a bitfile's ID.

Filesets. A fileset is a logical collection of files that can be managed as a single administrative unit, or more simply, a disjoint directory tree. A fileset has two identifiers: a human readable name and a numeric fileset ID. Both identifiers are unique to a given realm.

Junctions. A junction is a Core Server object, much like a symbolic link to a directory, that is used to point to a fileset. This fileset may belong to the same Core Server or to a different Core Server. When pointing to a different Core Server, junctions allow HPSS users to traverse to different subsystems.

File Families. HPSS files can be grouped into families. All files in a given family are recorded on a set of tapes assigned to the family. Only files from the given family are

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

38

recorded on these tapes. HPSS supports grouping files on tape volumes only. Families can only be specified by associating the family with a fileset. All files created in the fileset belong to the family. When one of these files is migrated from disk to tape, it is recorded on a tape with other files in the same family. If no tape virtual volume is associated with the family, a blank tape is reassigned from the default family. The family affiliation is preserved when tapes are repacked.

Physical Volumes. A physical volume is a unit of storage media on which HPSS stores data. The media can be removable (e.g., cartridge tape, optical disk) or non-removable (magnetic disk). Physical volumes may also be composite media, such as RAID disks, but must be represented by the host OS as a single device.

Physical volumes are not visible to the end user. The end user stores bitfiles into a logically unlimited storage space. HPSS, however, must implement this storage on a variety of types and quantities of physical volumes.

For a list of the tape physical volume types supported by HPSS, see Table 2 in Chapter 3.

Virtual Volumes. A virtual volume is used by the Core Server to provide a logical abstraction or mapping of physical volumes. A virtual volume may include one or more physical volumes. Striping of storage media is accomplished by the Core Servers by collecting more than one physical volume into a single virtual volume. A virtual volume is primarily used inside of HPSS, thus hidden from the user, but its existence benefits the user by making the user’s data independent of device characteristics. Virtual volumes are organized as strings of bytes up to

264-1 bytes in length that can be addressed by an offset into the virtual volume.

Storage Segments. A storage segment is an abstract storage object which is mapped onto a virtual volume. Each storage segment is associated with a storage class (defined below) and has a certain measure of location transparency. The Core Server (discussed in Section 2.3.2) uses both disk and tape storage segments as its primary method of obtaining and accessing HPSS storage resources. Mappings of storage segments onto virtual volumes are maintained by the HPSS Core Servers.

Storage Maps. A storage map is a data structure used by the Core Server to manage the allocation of storage space on virtual volumes.

Storage Classes. A storage class defines a set of characteristics and usage parameters to be associated with a particular grouping of HPSS virtual volumes. Each virtual volume and its associated physical volumes belong to a single storage class in HPSS. Storage classes in turn are grouped to form storage hierarchies (see below). An HPSS storage class is used to logically group storage media to provide storage for HPSS files with specific intended usage, similar size and usage characteristics.

Storage Hierarchies. An HPSS storage hierarchy defines the storage classes on which files in that hierarchy are to be stored. A hierarchy consists of multiple levels of storage, with each level representing a different storage class. Files are moved up and down the hierarchy via migrate and stage operations based on usage patterns, storage availability, and site policies. For example, a storage hierarchy might consist of a fast disk, followed by a fast data transfer and medium storage capacity robot tape system, which in turn is followed by a large data storage capacity but relatively slow data transfer tape robot system. Files are placed on a particular level in the hierarchy depending upon the migration levels that are associated with each level in the hierarchy. Multiple copies are controlled by this mechanism. Also data can be placed at higher levels in the hierarchy by staging operations. The staging and migrating of data is shown in Figure 1: File Migration and Stage Operations.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

39

Class of Service (COS). Each bitfile has an attribute called Class Of Service. The COS defines a set of parameters associated with operational and performance characteristics of a bitfile. The COS results in the bitfile being stored in a storage hierarchy suitable for its anticipated and actual size and usage characteristics. Figure 2: Class of Service/Hierarchy/Storage Class shows the relationship between COS, storage hierarchies, and storage classes.

2.3.2. HPSS Servers

HPSS servers include the Core Server, Migration/Purge Server, Gatekeeper, Location Server, Log Client, Log Daemon, Physical Volume Library, Physical Volume Repository, Mover, Storage System Manager, and, possibly, a Data Migration Gateway. Figure 3: HPSS Components provides a simplified view of the HPSS system. Each major server component is shown, along with the basic control communications paths (thin arrowed lines). Infrastructure items (those components that “glue together” the distributed servers) are shown at the top of the cube. These infrastructure items are discussed in Section 2.3.4: HPSS Infrastructure on page 44. HPSS user interfaces (the clients listed in the figure) are also discussed in Section 2.3.5: HPSS User Interfaces on page 47.

Figure 3. HPSS Components

Core Server. The Core Server provides several key sets of functionality.

First, the Core Server provides translation between human-oriented names and HPSS object identifiers. Name space objects managed by the Core Server are filesets, junctions, directories, files, hard links, and symbolic links. The Core Server provides access verification to objects and mechanisms for manipulating access to these objects via a Portable Operating System Interface (POSIX) view of the name space. This name space is a hierarchical structure

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

40

consisting of directories, files, and links. These name space objects may exist within filesets that are connected via junctions.

Second, the Core Server provides the abstraction of logical bitfiles to its clients. A bitfile is identified by a Core Server generated name called a bitfile ID. Clients may reference portions of a bitfile by specifying the bitfile ID and a starting address and length. The Core Server supports random access to files and sparsely written files. It supports parallel reading and writing of data to bitfiles and performs the mapping of logical portions of bitfiles onto physical storage devices. The Core Server supports the migration, purging, and staging of data in a storage hierarchy (though the migration/purge policies are implemented through the Migration/Purge Server, a client to the Core Server).

Third, the Core Server provides a hierarchy of storage objects: storage segments, virtual volumes, and physical volumes. The Core Server translates storage segment references into virtual volume references and then into physical volume references, handles the mapping of physical resources into striped virtual volumes to allow parallel I/O to that set of resources, and schedules the mounting and dismounting of removable media through the Physical Volume Library (see below).

Migration/Purge Server (MPS). The MPS allows a site to implement its storage management policies by managing the placement of data on HPSS storage media using site-defined migration and purge policies. By making appropriate calls to its Core Server, an MPS copies data to lower levels in the hierarchy (migration), removes data from the current level once copies have been made (purge), or moves data between volumes at the same level (lateral move). Based on the hierarchy configuration, MPS can be directed to create duplicate copies of data when it is being migrated from disk or tape. This is done by copying the data to multiple lower levels in the storage hierarchy.

There are three types of migration: disk migration, tape file migration, and tape volume migration. The designation disk or tape refers to the type of storage class that migration is running against. See Section 3.7.2: Migration/Purge Server on page 83 for a more complete discussion of the different types of migration.

MPS runs migration on each storage class periodically using the time interval specified in the migration policy for that class. See Section 2.3.7: HPSS Policy Modules on page 47 for details on migration and purge policies. Migration runs can be started automatically when the warning or critical space thresholds for the storage class are exceeded. In addition, migration runs can be started manually by an administrator.

Purge runs are started automatically on each storage class when the free space in that class falls below the percentage specified in the purge policy. Purge runs may also be started manually.

Disk Migration/Purge:

The purpose of disk migration is to make one or more copies of disk files to lower levels in the hierarchy. The number of copies depends on the configuration of the hierarchy. For disk, migration and purge are separate operations. It is common for disk storage class which have been configured for migration to also be configured for purge as well. Once a file has been migrated (copied) downwards in the hierarchy, it becomes eligible for purge, which subsequently removes the file from the higher level and allows the disk space to be reused.

Tape File Migration:

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

41

The purpose of tape file migration is to make an additional copy (or multiple additional copies) of a file, in a tape storage class, to a lower level in the hierarchy. It is also possible to move files downwards instead of copying them. In this case there is no duplicate copy maintained. There is no separate purge component to tape file migration. Empty volumes must be reclaimed using the reclaim utility.

Tape Volume Migration:

The purpose of tape volume migration is to free tape volumes for reuse. Tape volumes are selected based on being in the EOM map state and containing the most unused space (caused by users overwriting or deleting files). The remaining segments on these volumes are either migrated downwards to the next level in the hierarchy, or are moved laterally to another tape volume at the same level. This results in empty tape volumes which may then be reclaimed. Note that there is no purge component to tape volume migration. All of the operations use a move instead of a copy semantic.

Gatekeeper (GK). The Gatekeeper provides two main services:

·It provides sites with the ability to schedule the use of HPSS resources using the Gatekeeping Service.

·It provides sites with the ability to validate user accounts using the Account Validation Service.

Both of these services allow sites to implement their own policy.

The default Gatekeeping Service policy is to not do any gatekeeping. Sites may choose to implement a policy for monitoring authorized callers, creates, opens, and stages. The Core Server will call the appropriate GK API depending on the requests that the site-implemented policy is monitoring.

The Account Validation Service performs authorizations of user storage charges. A site may perform no authorization, default authorization, or site-customized authorization depending on how the Accounting Policy is set up and whether or not a site has written site-specific account validation code. Clients call this service when creating files, changing file ownership, or changing accounting information. If Account Validation is enabled, the Account Validation Service determines if the user is allowed to use a specific account or gives the user an account to use, if needed. The Core Server also calls this service to perform an authorization check just before account-sensitive operations take place.

Location Server (LS). The Location Server acts as an information clearinghouse to its clients through the HPSS Client API to enable them to locate servers and gather information from both local and remote HPSS systems. Its primary function is to allow a client to determine a server's location and, by knowing other information about the server such as its object UUID, determine its server type or its subsystem id. This allows a client to contact the appropriate server. Usually the Location Server is only used by the Core Server or the Gatekeeper.

Physical Volume Library (PVL). The PVL manages all HPSS physical volumes. It is in charge of mounting and dismounting sets of physical volumes, allocating drive and cartridge resources to satisfy mount and dismount requests, providing a mapping of physical volume to cartridge and of cartridge to Physical Volume Repository (PVR), and issuing commands to PVRs to perform physical mount and dismount actions. A primary function of the PVL is the support for atomic mounts of sets of cartridges for parallel access to data. Atomic mounts are implemented by the PVL, which waits until all necessary cartridge resources for a request are available before issuing mount commands to the PVRs.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

42

Physical Volume Repository (PVR). PVRs manage HPSS cartridges. Though an HPSS system may contain multiple PVRs, each cartridge is managed by only one. PVRs provide APIs for clients to request cartridge mounts and dismounts and query the status of cartridges. For convenience, PVRs are often configured in one-to-one correspondence to tape libraries.

For information on the types of tape libraries supported by HPSS PVRs, see Section 3.4.2:

Robotically Mounted Tape on page 62.

An Operator PVR is provided for cartridges not under control of a robotic library. These cartridges are mounted on a set of drives by operators.

Mover (MVR). The purpose of the Mover is to transfer data from a source device to a sink device. A device can be a standard I/O device with geometry (e.g., tape, disk) or a device without geometry (e.g., network, memory). The MVR’s client (typically the Core Server) describes the data to be moved and where the data is to be sent. It is the MVR’s responsibility to actually transfer the data, retrying failed requests and attempting to optimize transfers. The MVR supports transfers for disk devices, tape devices and a mover protocol that can be used as a lightweight coordination and flow control mechanism for large transfers.

Storage System Manager (SSM). SSM is the tool used by the system administrator to manage HPSS. It enables the administrator to configure, monitor and control the resources (servers, devices, tape libraries, and media) of HPSS in ways that conform to the management policies of a given customer site.

Monitoring capabilities include the ability to query the values of important management attributes of storage system resources and the ability to receive notifications of alarms and other significant system events. Controlling capabilities include the ability to start up and shut down servers and the ability to set the values of management attributes of storage system resources and storage system policy parameters. Additionally, SSM can request that specific operations be performed on resources within the storage system, such as adding and deleting logical or physical resources. Operations performed by SSM are usually accomplished through standard HPSS Application Program Interfaces (APIs).

SSM has three components, one of which is an HPSS server and two of which are user interface client programs. The server is:

SSM System Manager. Communicates with all other HPSS components requiring monitoring or control.

The user interface clients are:

SSM GUI (hpssgui) - Provides the HPSS administrator or operator the ability to configure or monitor the HPSS System through a graphical user interface.

SSM Command Line Interface (hpssadm) - Provides the HPSS administrator or operator the ability to configure or monitor a subset of the HPSS system through a set of interactive or batch commands.

Data Migration Gateway (DMG). The DMG is only used if a connection to a XDSM fileset is desired. If a site wishes to have HPSS act as a backing-store for a file system such a XFS, that site will need the services of a DMG. The DMG helps to coordinate the movement of data between these so-called linked filesets and HPSS.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

43

2.3.3. HPSS Storage Subsystems

The goal of storage subsystems (or just “subsystems”) is to increase the scalability of HPSS by allowing multiple Core Servers to be used within a single HPSS system. Every HPSS system is partitioned into one or more subsystems. Each subsystem contains a single Core Server. If migration and purge are needed, then the subsystem should contain a single Migration/Purge Server. Each Core Server and each Migration/Purge Server must exist within a storage subsystem. Each subsystem may optionally be serviced by a Gatekeeper which performs site-specific user-level scheduling of HPSS storage requests or account validation. Each Gatekeeper may service multiple subsystems. If a subsystem wishes to utilize XDSM filesets, a DMAP Gateway must also be configured. Only one DMAP Gateway is needed for multiple subsystems. All other servers exist independently of storage subsystems. Sites which do not need multiple Core Servers use a single storage subsystem.

The computer that runs the Core Server for subsystem X is referred to as the “Subsystem X node”, while the computer running the Root Core Server is known as the “Root Subsystem Node”.

Each HPSS system consists of two types of DB2 databases. The global database contains subsystemindependent data, and a subsystem database contains subsystem-dependent data. An HPSS system has exactly one global database and one or more subsystem databases.

The definitions of classes of service, hierarchies, and storage classes apply to the entire HPSS system (they are subsystem-independent). All classes of service, hierarchies, and storage classes are known to all storage subsystems within HPSS. The level of resources dedicated to these entities by each storage subsystem may differ. It is possible to disable selected classes of service within given storage subsystems. Although the class of service definitions are global, if a class of service is disabled within a storage subsystem then the Core Server in that subsystem never selects that class of service.

Since the Migration/Purge Server (MPS) is contained within the storage subsystem, migration and purge operate independently in each subsystem. Each Migration/Purge Server is responsible for migration and purge for those storage class resources contained within its particular storage subsystem. Migration and purge runs are independent and are not synchronized. Migration and purge for a storage class may be configured differently for each storage subsystem. It is possible to set up a single migration or purge policy which applies to a storage class across all storage subsystems (to make configuration easier), but it is also possible to control migration and purge differently in each storage subsystem.

Similarly, storage class thresholds may be configured differently for each storage subsystem. It is possible to set up a single set of thresholds which apply to a storage class across all storage subsystems, but it is also possible to control the thresholds differently for each storage subsystem.

2.3.4. HPSS Infrastructure

The HPSS infrastructure items (see Figure 3) are those components and services used by the various HPSS servers. The HPSS infrastructure components common among servers are discussed below.

Remote Procedure Calls (RPC). Most HPSS servers, with the exception of the MVR, PFTPD, and logging services (see below), communicate requests and status (control information) via RPCs. HPSS does not use RPCs to move user data. RPCs provide a communication interface resembling simple, local procedure calls.

Thread Services. HPSS uses a threads package for multitasking. The threads package is vital for HPSS to serve large numbers of concurrent users and to enable multiprocessing of its servers.

Transaction Management. Requests to perform actions, such as creating bitfiles or accessing file data, result in client-server interactions between software components. The

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

44

HPSS Core Server performs most of the HPSS metadata changes using the transaction management tools provided by DB2. For the most part, these metadata transactions are managed entirely within the Core Server. Other servers such as MPS and PVL modify their metadata transactionally, and those transactions are entirely contained within those servers. A very small number of rarely performed operations require distributed transaction management, and these are handled by DB2 as well.

Transactional integrity to guarantee consistency of server state and metadata is required in HPSS in case a particular component fails. HPSS metadata updates utilize the transactional capability of DB2. The selection of DB2 was based on functionality and vendor platform support. It provides HPSS with an environment in which a job or action completes successfully or is aborted completely.

DB2 provides a full suite of recovery options for metadata transactions. Recovery of the database to a consistent state after a failure of HPSS or DB2 is automatic. A full suite of database backup and maintenance tools is provided as well.

Security. HPSS security software provides mechanisms that allow HPSS components to communicate in an authenticated manner, to authorize access to HPSS objects, to enforce access control on HPSS objects, and to issue log records for security-related events. The security components of HPSS provide authentication, authorization, enforcement, and audit capabilities for the HPSS components. Customer sites may use the default security policy delivered with HPSS or define their own security policy by implementing their own version of the security policy module.

Authentication — is responsible for guaranteeing that a principal (a customer identity) is the entity that is claimed, and that information received from an entity is from that entity.

Authorization — is responsible for enabling an authenticated entity access to an allowed set of resources and objects. Authorization enables end user access to HPSS directories and bitfiles.

Enforcement — is responsible for guaranteeing that operations are restricted to the authorized set of operations.

Audit — is responsible for generating a log of security-relevant activity. HPSS audit capabilities allow sites to monitor HPSS authentication, authorization, and file security events. File security events include file creation, deletion, opening for I/O, and attribute modification operations.

HPSS components that communicate with each other maintain a joint security context. The security context for both sides of the communication contains identity and authorization information for the peer principals as well as an optional encryption key.

Access to HPSS server interfaces is controlled through an Access Control List (ACL) mechanism. Membership on this ACL is controlled by the HPSS administrator.

Logging. A logging infrastructure component in HPSS provides an audit trail of server events. Logged data includes alarms, events, requests, security audit records, status records, and trace information. The Log Client, which may keep a temporary local copy of logged information, communicates log messages to a central Log Daemon, which in turn maintains a central log. Depending on the type of log message, the Log Daemon may send the message to the SSM for display purposes. When the central HPSS log fills, messages are sent to a secondary log file. A configuration option allows the filled log to be automatically archived to

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

45

HPSS. A delog function is provided to extract and format log records from a central or archived log file. Delog options support filtering by time interval, record type, server, and user.

Accounting. The HPSS accounting system provides the means to collect usage information in order to allow a particular site to charge its users for the use of HPSS resources. It is the responsibility of the individual site to sort and use this information for subsequent billing based on site-specific charging policies. For more information on the HPSS accounting policy, refer to Section 2.3.7: HPSS Policy Modules on page 47.

2.3.5. HPSS User Interfaces

As indicated in Figure 3, HPSS provides the user with a number of transfer interfaces as discussed below.

File Transfer Protocol (FTP). HPSS provides an industry-standard FTP user interface. Because standard FTP is a serial interface, data sent to a user is received serially. This does not mean that the data within HPSS is not stored and retrieved in parallel; it means that the Parallel FTP Daemon within HPSS must consolidate its internal parallel transfers into a serial data transfer to the user. HPSS FTP performance in many cases will be limited not by the speed of a single storage device but by the speed of the data path between the HPSS Parallel FTP Daemon and the user’s FTP client.

Parallel FTP (PFTP). The PFTP supports standard FTP commands plus extensions and is built to optimize performance for storing and retrieving files from HPSS by allowing data to be transferred in parallel across the network media. The parallel client interfaces have a syntax similar to FTP but with numerous extensions to allow the user to transfer data to and from HPSS across parallel communication interfaces established between the PFTP client and the HPSS Movers. This provides the potential for using multiple client nodes as well as multiple server nodes. PFTP supports transfers via TCP/IP. The PFTP client establishes a control connection with the HPSS Parallel FTP Daemon and subsequently establishes TCP/IP data connections directly with HPSS Movers to transfer data at rates limited only by the underlying media, communications hardware, and software.

Client Application Program Interface (Client API). The Client API is an HPSS-specific programming interface that mirrors the POSIX.1 specification where possible to provide ease of use to POSIX application programmers. Additional APIs are also provided to allow the programmer to take advantage of the specific features provided by HPSS (e.g., storage/access hints passed on file creation and parallel data transfers). The Client API is a programming level interface. It supports file open/create and close operations; file data and attribute access operations; file name operations; directory creation, deletion, and access operations; and working directory operations. HPSS users interested in taking advantage of parallel I/O capabilities in HPSS can add Client API calls to their applications to utilize parallel I/O. For the specific details of this interface see the HPSS Programmer’s Reference Guide, Volume 1.

HPSS VFS Interface. The HPSS VFS Interface presents a standard POSIX I/O interface to a user application. This obviates the need for a user application to be rewritten against the HPSS Client API and hence can be used “out of the box” as long as the user application is POSIX compliant. A portion of an HPSS directory tree can be mounted on a client machine as if it were a local POSIX-compliant filesystem.

2.3.6. HPSS Management Interfaces

HPSS provides a graphical user interface, the SSM hpssgui, for HPSS administration and operations

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

46

GUI. The hpssgui simplifies the management of HPSS by organizing a broad range of technical data into a series of easy-to-read graphic displays. The hpssgui allows monitoring and control of virtually all HPSS processes and resources from windows that can easily be added, deleted, moved, or overlapped as desired.

HPSS also provides a command line SSM interface, hpssadm. This tool does not provide all the functionality of the hpssgui, but does implement a subset of its frequently used features, such as some monitoring and some control of servers, devices, storage classes, volumes, and alarms. It is useful for performing HPSS administration from remote locations where network traffic is slow or difficult. Additionally, hpssadm provides some rudimentary mass configuration support by means of the ability to issue configuration commands from a batch script.

In addition to SSM, HPSS provides a number of command line utilities for specialized management purposes, such as listing the volumes managed by a particular PVR or core server. See the HPSS Management Guide Chapter 14: Management Tools for more information. See also the HPSS man pages for descriptions of these utilities.

2.3.7. HPSS Policy Modules

There are a number of aspects of storage management that probably will differ at each HPSS site. For instance, sites typically have their own guidelines or policies covering the implementation of accounting, security, and other storage management operations. In order to accommodate site-specific policies, HPSS has implemented flexible interfaces to its servers to allow local sites the freedom to tailor management operations to meet their particular needs.

HPSS policies are implemented using two different approaches. Under the first approach, used for migration, purge, and logging policies, sites are provided with a large number of parameters that may be used to implement local policy. Under the second approach, HPSS communicates information through a well-defined interface to a policy software module that can be completely replaced by a site. Under both approaches, HPSS provides a default policy set for users.

Migration Policy. The migration policy defines the conditions under which data is copied from one level in a storage hierarchy to one or more lower levels. Each storage class that is to have data copied from that storage class to a lower level in the hierarchy has a migration policy associated with it. The MPS uses this policy to control when files are copied and how much data is copied from the storage class in a given migration run. Migration runs are started automatically by the MPS based upon parameters in the migration policy.

Note that the number of copies which migration makes and the location of these copies is determined by the definition of the storage hierarchy and not by the migration policy.

Purge Policy. The purge policy defines the conditions under which data that has already been migrated from a disk storage class can be deleted. Purge applies only to disk storage classes. It is common, but not necessary, for disk storage classes which have a migration policy to also have a purge policy. Purge runs are started automatically by the MPS based upon parameters in the purge policy.

Logging Policy. The logging policy controls the types of messages to log. On a per server basis, the message types to write to the HPSS log may be defined. In addition, for each server, options to send Alarm, Event, or Status messages to SSM may be defined.

Security Policy. Security policy defines the authorization and access controls to be used for client access to HPSS. HPSS security policies are provided to control access (authentication) from FTP and/or Parallel FTP using Username/Password, Ident, or Kerberos credentials.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

47

HPSS provides facilities for recording information about authentication and object (file/directory) creation, deletion, access, and authorization events. The security audit policy for each server determines the records that each individual server will generate. All servers can generate authentication records.

Accounting Policy. The accounting policy provides runtime information to the accounting report utility and to the Account Validation service of the Gatekeeper. It helps determine what style of accounting should be used and what level of validation should be enforced.

The two types of accounting are site-style and UNIX-style. The site-style approach is the traditional type of accounting in use by most mass storage systems. Each site will have a sitespecific table (Account Map) that correlates the HPSS account index number with their local account charge codes. The UNIX-style approach allows a site to use the user identifier (UID) for the account index. The UID is passed along in UNIX-style accounting just as the account index number is passed along in site-style accounting.

Account Validation allows a site to perform usage authorization of an account for a user. It is turned on by enabling the Account Validation field of the Accounting Policy configuration screen. If Account Validation is enabled, the accounting style in use at the site is determined by the Accounting Style field. A site policy module may be implemented by the local site to perform customized account validation operations. The default Account Validation behavior is performed for any Account Validation operation that is not overridden by the site policy module.

Location Policy. The location policy defines how Location Servers at a given site will perform, especially in regards to how often server location information is updated. All local, replicated Location Servers update information according to the same policy.

Gatekeeping Policy. The Gatekeeper provides a Gatekeeping Service along with an Account Validation Service. These services provide the mechanism for HPSS to communicate information though a well-defined interface to a policy software module that can be written by a site. The site policy code is placed in well-defined shared libraries for the gatekeeping policy and the accounting policy (/opt/hpss/lib/libgksite.[a|so] and /opt/hpss/lib/libacctsite.[a| so] respectively) which are linked to the Gatekeeper. The Gatekeeping policy shared library contains a default policy which does NO gatekeeping. Sites will need to enhance this library to implement local policy rules if they wish to monitor and load-balance requests.

2.4. HPSS Hardware Platforms

2.4.1. Server Platforms

HPSS requires at least one AIX or Linux node for the core server components. A server node must have sufficient processing power and memory to handle the work load.

2.4.2. Client Platforms

The full-function Client API can be ported to any platform that supports UNIX.

The PFTP client code and Client API source code for platforms other than AIX and Linux are not on the HPSS distribution image. Maintenance of the PFTP and Client API software on platforms other than AIX and Linux is the responsibility of the customer, unless a support agreement is negotiated with IBM. Contact your HPSS Support Representative for information on how to obtain the needed software.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

48

The following matrix illustrates which platforms support HPSS interfaces.

Table 1. HPSS Client Interface and Mover Platforms

Platform

PFTP Client

Client API

HPSS

HPSS VFS

FTP Clients

 

 

 

Mover

Client

 

 

 

 

 

 

 

 

 

 

 

 

 

IBM AIX

X

X

X

 

Any platform running

 

 

 

 

 

standard FTP clients. GUI-

 

 

 

 

 

based Clients may not

 

 

 

 

 

function correctly for

 

 

 

 

 

some commands.

Sun Solaris

X

X

X

 

 

(Big Endian ONLY)

 

 

 

 

 

Digital UNIX

X

 

 

 

 

 

Not Tested

 

 

 

 

Hewlett-Packard HPUX

X

 

 

 

 

 

Not Tested

 

 

 

 

Silicon Graphics IRIX

X

 

X

 

 

(32-bit)

 

 

 

 

 

 

 

 

 

 

 

Compaq Tru64

X

 

 

 

 

 

Not Tested

 

 

 

 

Linux (Intel)

X

X

X

X

 

 

 

 

 

 

 

Linux (Power PC)

 

X

 

X

 

 

 

 

 

 

 

2.4.3. Mover Platforms

Movers are used to control the logical network attachment of storage devices and are configured to run on one or more nodes. A Mover consists of two parts: The Mover administrative process that runs on the server node, and the remote Mover process that handles the HPSS devices and data transfers. Movers can run on AIX, Linux, IRIX, and Solaris systems. See Table 1 above for a detailed list of supported platforms.

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

49

HPSS Installation Guide

July 2008

Release 6.2 (Revision 2.0)

50

+ 261 hidden pages