March Products 204 User Manual

———————————————————

SirFile User's Guide

(Model 204 File Monitor)

———————————————————

Sirius Software, Inc. 875 Massachusetts Avenue, Suite 21 Cambridge, MA 02139

Telephone: (617) 876-6677 FAX: (617) 234-1200 E-mail: support@sirius-software.com World Wide Web: http://sirius-software.com

March 14, 2006

——————————————————————————————————————————

—————————————————————————————————————————— ii SirFile User's Guide

—————————————————————————————————————————— Proprietary Notices ——————————————————————————————————————————

——————

Proprietary Notices

The following products:

● SirDBA

● SirFile

● SirLib

● SirMon

● SirPro

● SirScan

● UL/SPF

are proprietary products of Sirius Software, Inc.:

Sirius Software, Inc. 875 Massachusetts Avenue, Suite 21 Cambridge, Massachusetts 02139 USA

1.617.876.6677 http://sirius-software.com

Model 204™ is a proprietary product of Computer Corporation of America:

Computer Corporation of America 500 Old Connecticut Path Framingham, Massachusetts 01701 USA

1.508.270.6666 http://www.cca-int.com

—————————————————————————————————————————— SirFile User's Guide iii

—————————————————————————————————————————— Proprietary Notices ——————————————————————————————————————————

—————————————————————————————————————————— iv SirFile User's Guide

—————————————————————————————————————————— Contents ——————————————————————————————————————————

———————

Introduction to UL/SPF

UL/SPF is a family of products implemented as User Language application subsystems and designed to function together in a Model 204 online. Each product in the family can be installed and run independently, or each can be installed and run as a component of the integrated UL/SPF (User Language / System Productivity Facilities) framework.

All UL/SPF products share a common "look and feel" that is modelled upon IBM's ISPF facilities. UL/SPF supplants the sometimes arcane command language of Model 204, providing a highly productive full screen interface to a variety of common Model 204 functions. UL/SPF enhances the capabilities of Model 204 and bypasses many of its restrictions, enabling the performance of routine tasks that were previously impossible or prohibitively time-consuming.

UL/SPF comprises the following products:

SirDBA A system that analyzes Model 204 databases to determine their logical

structure, populating an internal catalog. SirDBA is distributed as a component of the Sir2000 Database Analysis Tools.

SirFile A comprehensive facility both for monitoring the physical storage utilization

of Model 204 database files and for warning users of the need for file reorganizations. SirFile maintains historical information that allows it to predict when file sizing problems will occur, allowing a DBA to take preventative action before an application outage results.

SirLib A powerful and flexible system that provides change management and

configuration control for Model 204 User Language applications. SirLib is fully integrated within the Model 204 programming environment, supporting unique Model 204 constructs such as file groups, while remaining nearly transparent to programmers.

SirMon A comprehensive facility for monitoring the performance and availability of

Model 204 online systems. SirMon combines the real time monitoring of Model 204 performance with intelligent full screen displays that facilitate System Manager duties.

SirPro A collection of powerful and easy to use tools for programmers, database

administrators, and application managers. SirPro provides programmers with powerful facilities for managing large libraries of User Language procedures, and it provides system managers with intuitive ISPF-like front ends to many arcane Model 204 system management commands.

—————————————————————————————————————————— SirFile User's Guide 1

—————————————————————————————————————————— Introduction to UL/SPF ——————————————————————————————————————————

SirScan A high performance utility that allows users in a Model 204 online to

browse the contents of its journal in real time. SirScan permits ordinary users to view journal entries generated by their own online session, and it allows users in ADMIN SCLASSes to browse journal entries for any set of users. The data is displayed in a full-screen browser with powerful searching commands and filtering options.

In addition, a number of subsystems that are not linked into the UL/SPF menuing structure may be accessed via APSY-transfer from the UL/SPF applications. One such subsystem is FACT, a utility for browsing SirFact dumps.

There are also many sample web and client-server applications developed for the Janus product family that are distributed and installed along with the UL/SPF products. These include JANCAT, an application that builds normalized views of Model 204 data for use by Janus Specialty Data Store applications, and JANSSL, a system for creating and managing SSL certificate requests.

1.1 UL/SPF Packaging and Installation Requirements

All of the UL/SPF products are Model 204 application subsystems written in User Language. UL/SPF is distributed as a set of Model 204 files in a backup format produced by the Model 204 DUMP command. As of UL/SPF Version 6.8, all User Language based products are distributed in a single Model 204 procedure file called SIRIUS.

UL/SPF makes extensive use of specialized User Language $functions provided by Sirius Software. These $functions enable the creation of powerful User Language application systems that can support complex environments with minimal server size requirements. The $functions are part of the Sirius Mods, which must be installed according to the instructions contained in Sirius Mods Installation Guide before any UL/SPF product will work.

Although all of the Sirius Functions are documented in the Sirius Functions Reference Manual, their use is controlled by a product authorization mechanism. Many functions require that a specific UL/SPF product be authorized. Some functions may only be

invoked from an authorized procedure that has been “signed” by Sirius. The SIRIUS

command may be used to examine the status of all Sirius products installed in a Model 204 online.

—————————————————————————————————————————— 2 SirFile User's Guide

—————————————————————————————————————————— Integrating UL/SPF with Other Subsystems ——————————————————————————————————————————

1.2 Integrating UL/SPF with Other Subsystems

UL/SPF and any of its constituent products can be easily integrated with other User Language subsystems. Whenever a UL/SPF component product is exiting, it first

checks to see if the global variable SIRIUS.COMM exists and has a non-null value. If so,

the UL/SPF product performs a subsystem transfer using the value in SIRIUS.COMM as the name of the target subsystem.

For example, the following code fragment lets you transfer into SIRMON, and then,

when SIRMON exits, control is transferred to the application subsystem MENUSYS, provided that NEXTPROC is the current subsystem's communication global variable:

%RC = $SETG('SIRIUS.COMM','MENUSYS') %RC = $SETG('NEXTPROC','XFER') %RC = $SETG('XFER','SIRMON') STOP

In addition to the individual User Language subsystems that implement the UL/SPF products (SirDBA, SirFile, SirLib, SirMon, SirPro, and SirScan), Sirius distributes an umbrella UL/SPF subsystem: The ULSPF subsystem provides a menu that contains entries for all UL/SPF components installed at a site.

The UL/SPF subsystems support fast path navigation. For example, a SirMon user can transfer into SirPro Option 1 by typing:

=M.1.1

1.3 Related Manuals

The following manuals are available for public download at http://sirius-software.com/maint/manlist, and they may also be obtained by contacting Sirius Software:

● SirDBA User's Guide

● SirFile User's Guide

● SirLib User's Guide

● SirMon User's Guide

● SirPro User's Guide

● SirScan User's Guide

● UL/SPF Installation and Maintenance Guide

● Sirius Mods Installation Guide

● Sirius Functions Reference Manual

● Sirius Messages Manual

The following manuals are part of the Model 204 documentation set available from the Computer Corporation of America:

—————————————————————————————————————————— SirFile User's Guide 3

—————————————————————————————————————————— Introduction to UL/SPF ——————————————————————————————————————————

● Model 204 Command Reference Manual

● Model 204 File Manager's Guide

● Model 204 System Manager's Guide

● Model 204 User Language Manual

—————————————————————————————————————————— 4 SirFile User's Guide

—————————————————————————————————————————— Overview and Setup for SirFile ——————————————————————————————————————————

—————— CHAPTER 2

Overview and Setup for SirFile

SirFile is a comprehensive facility for monitoring the physical storage utilization of Model 204 database files and warning users of the need for file reorganizations.

SirFile requires only a single screen of setup information to determine “thresholds,” after which it automatically performs a pass through the online database files, checking to see if any file has exceeded a threshold. The process that evaluates files is called the refresh process. The refresh process performs two types of checks:

file statistic is above the critical value for that statistic.

growth and to predict when thresholds will be exceeded.

During the initial database load, all files added to the SirFile database are compared to the system default thresholds. After the initial load, the user may override the system default thresholds with file-specific thresholds for files that require higher or lower levels of monitoring.

If any file table is above a threshold, SirFile flags the file with a warning, causing it to be highlighted in SirFile. Once SirFile has stored at least one sample snapshot of a file, it begins calculating the date at which each table will fill — assuming any table is showing growth. If the predicted fill date falls within a user-settable number of days, SirFile posts a prediction to the file, and the file is highlighted in the SirFile screens. When a prediction or warning is posted using a periodic refresh, a message can also be sent to the audit trail, as well as to a list of user IDs.

The file parameters that are evaluated for table-full status are ARETRIES, BFULLP, CRETRIES, CFULLP, DFULLP, EXTNADD, OVFLADD, EOVFLADD, and SPILLADD.

● It compares the current state of each file against the threshold settings to see if any

● It maintains a database of table usage information which it uses to determine rate of

The database refresh portion of SirFile is quite efficient, however it is the most expensive operation in SirFile. Therefore, several options are provided for determining when to perform a refresh, and for determining when to calculate CFULLP, generally the most expensive of the refresh calculations.

For more information, see:

invoke a periodic refresh

● “The Refresh Process” on page 11, for a complete description of the refresh process

● “Invocation Techniques for Background Monitoring” on page 33, for methods that

—————————————————————————————————————————— SirFile User's Guide 5

—————————————————————————————————————————— Overview and Setup for SirFile ——————————————————————————————————————————

● “Running SirFile under BATCH204” on page 35, for a method that can be useful for

calculating CFULLP

The SirFile refresh process is always in “learn” mode. Any file that is open when a SirFile refresh is run is automatically added to the SirFile database, using the default thresholds for comparison.

2.1 Files available for SirFile Monitoring

For SirFile to collect information on a file, it must be open in the region by some user or subsystem. To guarantee that certain files are included in the initial load, those files can be opened manually before initiating the refresh process. Because storing user-entered passwords would violate most sites' security, SirFile uses a variety of methods to make files available to itself without needing to store passwords.

1. If a file is open in the region, or SirFile manages to open the file with default privileges, the file is added to the database and no user intervention is required.

2. The first time SirFile collects data for a file it adds the file to its Application Subsystem definition (with low privileges in the “USER” SCLASS and high privileges in the “ADMIN” SCLASS). Thereafter, no password is required for SirFile to open the file. Files that are prefixed “CCA” are exempted from this operation, as these files cause problems in SUBSYSMGMT.

3. If SirFile has a file in its database that requires a password, and a non-periodic refresh is running (other than SIRFILE BATCH), you are prompted for the password.

4. If SIRFILE BATCH or a periodic refresh is running, SirFile skips any file it cannot open.

2.2 File enqueuing

Besides its APSY procedure file and the UL/SPF file SIRLOCAL, SirFile requires update access to CCASYS. All other files, even if they are defined to SirFile as optional APSY files, are closed and freed from the subsystem at the end of each user's SirFile session. This is done to reduce enqueuing problems.

—————————————————————————————————————————— 6 SirFile User's Guide

—————————————————————————————————————————— Getting Started ——————————————————————————————————————————

2.3 Getting Started

Installation of SirFile is covered in the UL/SPF Installation and Maintenance Guide. SirFile should be installed as a private application subsystem. To access the system, the privileged user types “SIRFILE” on the Model 204 command line of the region containing the software. An initial System Default thresholds screen is presented.

-------------- * * * Set threshold values for file warnings * * * ------------ ==> _ File ==> *DEFAULT

Thresholds Number of historical records to keep per file. ARETRIES ==> 1 ==> 99999 BFULLP ==> 90 CFULLP ==> 80 Minimum number of days CRETRIES ==> 100 between stored samples. DFULLP ==> 90 ==> 7 EXTNADD ==> 5000 OVFLADD ==> 5000 EOVFLADD ==> 5000 # of days advance warning SPILLADD ==> 1000 on threshold exceeded. ==> 7

Collect CFULLP only on specified files (Y/N)? ==> N

--------------------------------------------------------------------------- 1/Help 3/Quit 9/Repeat 12/Save-Refresh

System Default thresholds screen

This screen is also presented by pressing

PF11 from either the Main File menu or from

the File Tables screen, if the cursor is not on a file name or file statistics line, respectively. The System Default thresholds screen is initially populated with a set of defaults that are adequate for most file monitoring needs. The thresholds are:

ARETRIES

Table A Retries indicate a failure of the Table A hashing algorithm to find a slot on a Table A page to store field information. Table A is usually the smallest table in a Model 204 database file, but it is accessed every time a field name is referred to in User Language or IFAM code, so it is important that Table A operate as efficiently as possible. In most cases, any ARETRIES are too many, and for this reason, the ARETRIES default is very small.

BFULLP Percentage of Table B full. This is the percentage of record slots currently

used, calculated as BHIGHPG/BSIZE. Table B holds the actual data in a Model 204 file and is usually the largest table.

CRETRIES

Table C hashing algorithm retries. This statistic indicates that Table C, a hashed data structure, is becoming full, so the algorithm which determines

—————————————————————————————————————————— SirFile User's Guide 7

—————————————————————————————————————————— Overview and Setup for SirFile ——————————————————————————————————————————

placement of new pointers in the table must run repeatedly in order to find an empty slot. There is no fixed measure of how many retries indicate a critical condition for Table C.

CFULLP Percentage of Table C full. Like Table A, Table C begins to exhibit hash

retries when the data in it reaches a certain density. There is no strict formula for the maximum fullness of Table C for a given file, but a rule of thumb is that retries will begin to appear when Table C is 70 to 80% full. CFULLP is a more accurate measure of Table C full status than CRETRIES, but it is very expensive to calculate. For this reason, unless you determine

CFULLP calculation file-by-file, it is only calculated by the SIRFILE BATCH command.

DFULLP Table D percentage full. Table D can fill either as a result of index

information or User Language procedures. Percentage full in Table D is calculated as the number of D pages used (DPGUSED) divided by DSIZE.

EXTNADD

Number of Extension Records. Extension records occur when the information added to an existing record exceeds the usable space left on a Table B page. Extension records are not a sign of a file full condition pending, and in some record structures they may be unavoidable. For this reason the EXTNADD threshold may be set very high. However, extension records cause inefficiencies in storage and retrieval, and for some file designs should be taken as a sign that Table B parameters are out of balance.

OVFLADD

Addition of records to overflow areas. Overflow records may occur in either sorted or hash files.

EOVFLADD

Addition of records to extra overflow areas. Records may be added to Extra Overflow areas in sorted files when the Overflow area is full.

SPILLADD

Number of records that have “spilled” from the preferred overflow area into secondary overflow areas or the overflow area prior to the appropriate sort group. Spilled records are stored and retrieved less efficiently than other records.

Number of historical records to keep

Number of sample records to keep for each file. SirFile requires at least 1 sample to be stored for any given file in order to make predictions about file full conditions. The default value of 99999 records means that SirFile will never throw away old data, as long as the file CREATE date does not change.

—————————————————————————————————————————— 8 SirFile User's Guide

—————————————————————————————————————————— Getting Started ——————————————————————————————————————————

Minimum number of days between stored samples

Minimum number of days that must pass between samples being stored for any particular file. This setting allows users to run the refresh as often as they like without SirFile storing excessive amounts of data. The default of 7 ensures that no more than one record a week is maintained for any file unless that file breaks a threshold.

# Days Warning

Number of days prior to a predicted “threshold exceeded” condition SirFile should post a warning. During a Refresh, SirFile uses the current file table values and the historical file information to calculate when the file will exceed each threshold (if positive growth is detected in a threshold value). If the number of days predicted falls within this value, a prediction and message are posted to the file, and the file appears highlighted on the Main File menu.

Collect CFULLP only on specified files?

Specifying “Y” to this prompt allows SirFile to collect the CFULLP statistic only on files specified by the user. Collecting CFULLP requires a TABLEC command, which can be very I/O intensive.

● When “Y” is specified at this prompt, the user must specifically request

each file the TABLEC command is to be run against, entering Y at the following prompt in the File thresholds screen:

Collect CFULLP for this file

● When “N” is specified at this prompt, CFULLP calculations are

performed only by the SIRFIELD BATCH command.

The user may accept the defaults or change any of the supplied settings. Pressing

PF12 saves the settings and initiates the SirFile database Refresh process.

SirFile records are fairly small and you shouldn't be too concerned about minimizing the amount of stored data: A year's worth of data for a single file will occupy only about 1 Model 204 page if a record is kept per week.

The following commands are allowed:

OPEN xxxxxxxx Opens a file. As data can only be collected for files that SirFile

already knows about, or files that are currently open by any user, a SirFile user may wish to manually open a file before running a refresh to guarantee the file is included in the sample.

CLOSE xxxxxxxx Closes a file. Any file that the SirFile user has open can be closed,

except SIRFILE, SIRLOCAL and CCASYS (the required application subsystem files for SirFile).

—————————————————————————————————————————— SirFile User's Guide 9

—————————————————————————————————————————— Overview and Setup for SirFile ——————————————————————————————————————————

PF keys:

operation.

PF1 Accesses full-screen help.

PF3 Return to command level or previous screen.

PF9 Repeats the last command-line command.

PF12 Saves the file specific thresholds and invokes the SirFile database refresh

—————————————————————————————————————————— 10 SirFile User's Guide

—————————————————————————————————————————— The Refresh Process ——————————————————————————————————————————

—————— CHAPTER 3

The Refresh Process

The Refresh process checks file tables and updates the SirFile database. You can request a refresh operation to run once or periodically.

1. Running a refresh once always runs in the Model 204 thread of the user requesting the refresh, and can be specified either for a single file or for all files that SirFile can access. This can be invoked via:

using

invoked only in a BATCH204 jobstream, to be run when files are not open for update by any online. See “Running SirFile under BATCH204” on page 35.

2. A periodic refresh, which repeats at a specified time interval, is also called the Background Monitor. This can be invoked via:

on an sdaemon thread, that is, a different thread from the requesting user.

sdaemon thread, that is, a different thread from the requesting user, and it may be useful in the online's User 0 stream.

thread of the requesting user, and it is most often invoked by a BATCH2 or IODEV=3 thread.

● the Main File menu, using either the ADD or REFRESH command or using

PF12

● any of the File Tables, System Default thresholds, or File thresholds screens,

PF12

● the SIRFILE BATCH command. The SIRFILE BATCH command is usually

● the Background Monitor specification screen, using PF5 . This runs the refresh

● the SIRFILE BACKGROUND interval command. This runs the refresh on an

● the SIRFILE SDAEMON interval command. This runs the refresh on the

“Invocation Techniques for Background Monitoring” on page 33 and “Running SirFile under BATCH204” on page 35 further discuss considerations of periodic

refreshes.

Whenever the refresh operation is performed, it can consume considerable resources if it calculates the CFULLP (Table C percentage full) value. One of the values specified on the System Default thresholds screen is whether to allow file-by-file determination of CFULLP calculation. If file-by-file CFULLP determination is allowed, then CFULLP is calculated during any type of refresh, for exactly those files which are specified as having CFULLP calculation performed. If CFULLP calculation is not determined on a file-by-file basis, then CFULLP calculation is performed for all files by the SIRFILE BATCH command, and is not performed by any other type of refresh.

—————————————————————————————————————————— SirFile User's Guide 11

March Products 204 User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Proprietary Notices

Contents

Introduction to UL/SPF

1.1 UL/SPF Packaging and Installation Requirements

1.2 Integrating UL/SPF with Other Subsystems

1.3 Related Manuals

Overview and Setup for SirFile

2.1 Files available for SirFile Monitoring

2.2 File enqueuing

2.3 Getting Started

The Refresh Process