IBM Multiprise 3000, S/390 Multiprise 3000 Reference Manual
S/390
Internal Disk Subsystem:
Reference Guide
Multiprise 3000 Enterprise Server
R
IBM
SA22-1025-00
S/390 IBM
Internal Disk Subsystem:
Reference Guide
SA22-1025-00
Note!
Before using this information and the product it supports, be sure to read the general information under Appendix D, “Notices”
on page D-1.
First Edition (November, 1999)
This edition, SA22-1025-00, applies to the IBM Internal Disk Subsystem.
Copyright International Business Machines Corporation 1999. All rights reserved.
Note to U.S. Government Users — Documentation related to restricted rights — Use, duplication or disclosure is subject to
restrictions set forth in GSA ADP Schedule Contract with IBM Corp.
Contents
Chapter 1. Overview of the S/390 Internal Disk Subsystem ......... 1-1
Host Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
RAID Disk Arrays .................................... 1-3
HDD Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
Internal Disk RAID Fast Write ............................ 1-4
Internal Disk Facilities ................................. 1-4
CKD Emulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Control Unit and Device Emulation ........................ 1-5
LPAR Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Enhanced HDD Reliability .............................. 1-5
Search Assist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
SSID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
Long Busy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
Internal Disk Components ............................... 1-7
Shared CPC resources ............................... 1-7
SSA Adapter Card ................................. 1-8
HDD Packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8
Internal Disk Logical Volume Configurations .................... 1-8
Logical Volumes Mapped onto RAID Arrays ................... 1-8
Logical Volume Address Resolution ........................ 1-9
Internal Disk Configuration Granularity ..................... 1-10
Commands and Facilities Not Supported ..................... 1-11
IOCP Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11
RAS Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Chapter 2. Processing Commands and Presenting Status .......... 2-1
Input/Output Channel Interface ............................ 2-1
Command Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
Status Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
Status Byte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
Initial Status Byte ................................... 2-4
Status Pending Conditions ............................. 2-4
State-Change Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Effect of Subsystem and Device State on Status Presented to Command .. 2-5
Listing of Commands Appearing in Table 3 ................... 2-9
Contingent Allegiance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
Channel Command Retry ............................... 2-9
Channel Command Retry–Control Unit Initiated ................ 2-10
System and Selective Reset ............................. 2-10
Chapter 3. Elements of Data Access ....................... 3-1
Track Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
Track Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
Data Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
Track Accessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Track Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Record Accessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3
Data Transfer Commands ............................... 3-3
Formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
File Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
Copyright IBM Corp. 1999 iii
End-of-File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
Chapter 4. Command Descriptions
Channel Command Word
Exception Conditions
............................... 4-1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
. . . . . . . . . . . . . . . . . . . . . . . . 4-1
Command Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Multitrack Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Defective/Alternate Track Operations ......................... 4-3
Summary of Channel Commands ........................... 4-3
List of Channel Commands by Hex Code .................... 4-6
Addressing and Control Commands ......................... 4-7
Define Extent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8
Locate Record . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13
Locate Record Extended ............................. 4-27
Prefix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-42
Seek Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-46
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-47
Seek
Seek Cylinder
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-48
Seek Head . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-49
Recalibrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-50
Set File Mask .................................... 4-51
Set Sector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-52
Read Sector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-53
Read Device Characteristics ........................... 4-54
Read Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-61
Read Home Address ............................... 4-62
Read Record Zero ................................. 4-63
Read Count, Key, and Data ........................... 4-64
Read Key and Data ................................ 4-66
Read Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-68
Read Count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-70
Read Multiple Count, Key, and Data ...................... 4-71
Read Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-72
Read Track Data .................................. 4-74
Read IPL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-76
Search Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-77
Search Home Address Equal ........................... 4-78
Search ID Equal .................................. 4-79
Search ID High ................................... 4-80
Search ID Equal or High ............................. 4-80
Search Key Equal ................................. 4-81
Search Key High .................................. 4-82
Search Key Equal or High ............................ 4-82
Write Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-82
Write Home Address ................................ 4-84
Write Record Zero ................................. 4-85
Write Count, Key, and Data ............................ 4-86
Write Count, Key, and Data Next Track ..................... 4-88
Erase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-90
Write Update Key and Data ............................ 4-91
Write Key and Data ................................ 4-93
Write Update Data ................................. 4-95
Write Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-97
Write Full Track ................................... 4-99
iv Internal Disk Subsystem Reference Guide
Write Track Data .................................. 4-102
Sense Commands
Sense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-104
Sense ID
Miscellaneous Commands
No-Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-107
Read and Reset Buffered Log .......................... 4-108
Read Configuration Data ............................. 4-111
Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-117
Path Control Commands ............................... 4-118
Device Reserve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-118
Device Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-120
Unconditional Reserve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-121
Reset Allegiance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-123
Set Path Group ID ................................. 4-125
Sense Path Group ID ............................... 4-128
Suspend Multipath Reconnection
Subsystem Commands
Perform Subsystem Function ........................... 4-130
Sense Subsystem Status ............................. 4-136
Read Subsystem Data ............................... 4-139
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-104
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-105
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-107
........................ 4-130
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-130
Chapter 5. 24-Byte Compatibility Sense Data .................. 5-1
24-Byte Sense Data Summary ............................ 5-2
Sense Data Formats ................................. 5-2
24-Byte Sense Data Description ........................... 5-4
Sense Bytes 0, 1, and 2 .............................. 5-4
Sense Byte 3–Count ................................. 5-6
Sense Byte 4–Logical Volume Address ...................... 5-6
Sense Byte 5–Low Cylinder Address ....................... 5-7
Sense Byte 6–High Cylinder Address and Track Address ........... 5-7
Sense Byte 7–Format/Message . . . . . . . . . . . . . . . . . . . . . . . . . . 5-7
Sense Bytes 8 through 23–Format-Dependent Information .......... 5-7
Sense Byte 25–Program Action Code ....................... 5-7
Sense Bytes 26 and 27–Configuration Data ................... 5-8
Sense Bytes 29 through 31–Cylinder and Head Address ............ 5-8
24-Byte Compatibility Sense Data Formats .................... 5-10
Format 0–Program or System Checks ..................... 5-10
Format 1–Device Equipment Checks ...................... 5-11
Sense Bytes 8 through 23 for Format 1 ..................... 5-12
Format 2–Internal Disk Control Unit Errors ................... 5-13
Format 3–Allegiance Terminated ......................... 5-13
Format 4–Unrecovered Data Checks ...................... 5-14
Format 5 to Format E – Not used ........................ 5-15
Format F–Subsystem Checks .......................... 5-15
Chapter 6. ECKD 32-Byte Sense Data ...................... 6-1
ECKD 32-Byte Sense Data Summary ........................ 6-2
ECKD 32-Byte Sense Data Description ....................... 6-4
Sense Byte 0 ..................................... 6-4
Sense Byte 1 ..................................... 6-4
Sense Byte 2–Control Unit Type .......................... 6-5
Sense Byte 3–Remaining Intent Count ...................... 6-5
Sense Byte 4–Logical Volume Address ...................... 6-5
Contents v
Sense Byte 5–Device Type Code ......................... 6-6
Sense Byte 6–Content and Format
Sense Bytes 7 through 19 ............................. 6-6
Sense Bytes 20 and 21–Subsystem Identifier
Sense Bytes 22 and 23–Exception Code
Sense Byte 24–Logging and Message Control .................. 6-7
Sense Byte 25–Program Action Code ....................... 6-8
Sense Bytes 26 and 27–Configuration Data ................... 6-8
Sense Byte 28–Message Code .......................... 6-8
Sense Bytes 29 and 30–Cylinder Address .................... 6-9
Sense Byte 31–Head Address ........................... 6-9
Sense Data Dependent on Exception Class .................. 6-10
Chapter 7. Error Recovery Procedures ...................... 7-1
ERPs for 24-Byte Compatibility Sense ........................ 7-1
Error Recovery Actions ............................... 7-2
ERPs for ECKD 32-Byte Sense
ERP Actions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-6
............................ 7-5
ERP Message Requirements ............................ 7-9
........................ 6-6
.................. 6-6
..................... 6-6
Appendix A. Device Characteristics . . . . . . . . . . . . . . . . . . . . . . . A-1
Appendix B. Caching Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . B-1
Caching Algorithm Definitions ............................. B-1
Algorithms for Cache MISSES ............................. B-1
Algorithms for Cache HITS ............................... B-2
Algorithm Modifications due to Cache Hints ..................... B-2
Algorithm Special Cases ................................ B-3
Appendix C. Warranties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1
Statement of Limited Warranty ............................ C-1
IBM Agreement for Licensed Internal Code ..................... C-4
Appendix D. Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-1
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-2
Electronic Emission Notices .............................. D-2
List of Abbreviations ................................. X-1
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-3
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-7
vi Internal Disk Subsystem Reference Guide
Chapter 1. Overview of the S/390 Internal Disk Subsystem
This chapter gives an overview of the S/390 Internal Disk Subsystem, describes its
components, features, and facilities.
Internal Disk storage subsystem provides internal attachment of disk storage
integrated into the Central Processing Complex (CPC) of a S/390 CMOS processor.
Internal Disk consists of attachment hardware and microcode plus high capacity,
high performance Hard Disk Drives (HDDs) that are packaged under the same
CPC covers. The HDDs are organized into RAID arrays that insure data availability
while minimizing the cost of disk storage. The HDDs appear to the host programs
executing in a CPC as conventional Count, Key, Data (CKD) disks attached to a
CKD control unit. The programs use channel commands to manage the operations
of the disk subsystem.
The control unit and channel functions are executed within the CPC utilizing
processor and memory resources of the CPC:
A System Assist Processor (SAP) is used for the channel emulation and control
unit functions.
Disk cache space is allocated from S/390 main memory
The system Service Element (SE) is shared for configuration and service
functions.
Power, packaging, and cooling are all provided by the Multiprise 3000 (host
processor).
From an implementation perspective these functions are an extension to the I/O
subsystem within the CPC.
Unique to the Internal Disk subsystem are Hard Disk Drives (HDDs) and logic cards
that carry adapter electronics. The logic cards are called SSA adapter cards. The
Hard Disk Drives (HDDs) used in Internal Disk are 3.5 inch, fixed block disk drives
with a data capacity of 18 GB. The SSA adapter card interfaces to the Multiprise
3000 via a PCI bus and provides an industry standard SSA (Serial Storage
Architecture) interface for connection of the HDDs.
The HDDs are mounted in the frame used for the packaging of the CPC. The
18GB UltraStar 18ZX drives are mounted in the HDD enclosure. The user may
choose to expand the storage beyond the capacity of the CPC frame by adding
expansion frames. The maximum configuration has one CPC frame and two
expansion frames.
The Multiprise 3000 has a minimum Internal Disk capacity of 72GBs and a
maximum capacity (inclusive of two expansion frames) of 792GBs.
The Internal Disk Subsystem offers the following advantages:
Minimization of software changes: If an operating system supports the device
being emulated, the Internal Disk subsystem can generally be installed without
any system software changes. Any combination of 3390-1, -2, -3, -9, and
3380-J, -E, -K logical volumes can be created on a RAID array. The total
number of logical volumes that can be created is only limited by the storage
capacity available on the array and the number of unit addresses that can be
assigned to an array. The latter one is 64 for an array with 5 HDDs (4+P), and
Copyright IBM Corp. 1999 1-1
96 for an array consisting of 7 HDDs (6+P). Also, logical volumes can be
added incrementally. This can even be done while I/O operations are ongoing
to other logical volumes which already exist on this array. In the same way,
logical volumes can also be selectively deleted again, freeing up space which
can then be used to create other logical volumes again.
Preloaded software is available
Internal Disk uses a RAID-5 organization for managing a collection of disks so
that high levels of data reliability and availability are maintained.
Usable disk storage capacity:
– 72 to 216 GB in the CPC cage
– Up to 288 GB in each of the expansion cages
Enhanced reliability by Predictive Failure Analysis: A set of processes
continuously performed by the HDD is designed to predict disk drive failure
conditions before they occur.
End to end data protection
Low-cost disk subsystem: A high capacity Internal Disk subsystem with a very
low cost of ownership.
Host Processor
SSA Interface: The connection via the SSA adapter interface provides high
data transfer rates.
Concurrent Repair and Verify of HDDs.
Concurrent LIC patch for SAP microcode.
Control unit and data sharing among logical partitions.
Internal Disk RAID Fast Write (IDRFW) provides a fast write capability through
the SSA adapter card.
Internal Disk is packaged with the Multiprise 3000 which is a unique card on board
package that takes advantage of:
Integrated RAID Storage
Integrated Tape
Integrated Communications
Industry Standard PCI Adapters
S/390 ESCON Channels
S/390 Parallel Channels
The Multiprise 3000 presents a low cost server platform that maintains most of the
S/390 traditional attributes. It is a small, office environment server that provides
state of the art LAN/WAN connection capabilities at an attractive price. The base
package satisfies new customer/workloads that do not have the traditional heavy
I/O channel demands. The basic frame that houses the CPC also contains up to 24
ESCON channels. If more channels are needed, an expansion frame with 32
additional ESCON channels is available.
The Multiprise 3000 is the entry level processor for the S/390 product family
designed for installation in any standard office environment without special
provisions. The unit is powered from single phase 100 to 240 volts ac, at either 50
or 60 hertz. Dual line cords are provided, each connected to one of the two N+1
1-2 Internal Disk Subsystem Reference Guide
power supplies. Two cooling fans are also provided to assure that the power and
cooling system is fully redundant with concurrent repair. An Integrated Battery
Feature (IBF) is available.
Internal Disk in the Multiprise 3000 continues the evolution of disk capacity with the
introduction of RAID disk arrays and a new high performance 10,000 RPM Hard
Disk Drive, the 18GB UltraStar 18ZX.
RAID Disk Arrays
The Multiprise 3000 Internal Disk uses a RAID-5 organization for its disk arrays.
RAID is an acronym for Redundant Arrays of Independent Disks. The RAID
organization is a technique for managing a collection of disks so that high levels of
data reliability and availability are maintained while at the same time retaining
desirable cost and performance.
The original Internal Disk that was introduced on Multiprise systems implemented
mirroring to assure data reliability and availability. Mirroring is a recognized RAID
organization (RAID-1). However, RAID-1 requires a mirrored capacity that is equal
in size to the capacity of the primary storage.
The Multiprise 3000 Internal Disk supports RAID-5 array sizes of 4+P, and 6+P.
The 4+P array may be thought of as having 4 HDDs assigned to carry user data
and a P drive for Parity or redundancy. In reality, the redundancy is spread among
the 5 HDDs equally so there is no specific P drive. Distribution of the redundancy
among the drives avoids a performance bottleneck that would occur if one drive
were assigned all of the redundancy. The 4+P arrangement has 80% of its HDD
space available for user data. Similarly, the 6+P array may be thought of as having
6 HDDs for user data and a single Parity Drive. Almost 86% of the 6+P array can
contain user data. A RAID-1 organization uses 50% of the disk space for user data
and the other 50% for redundancy.
In the event of failure or loss of any HDD, all of its content can be reconstructed
from the content of the other HDDs in the array. Until the reconstruction is
complete, there is an exposure to loss of data since the redundancy is adequate for
only a single HDD loss. To assure that reconstruction begins without delay, spare
HDDs are included in the configuration. The spare drives are subjected to
periodical testing by the SSA adapter card to assure that they will be fully
operational if they are needed.
Another improvement in the Multiprise 3000 implementation is that management of
the RAID-5 arrays is relegated to the SSA adapter rather than being performed at
the SAP. This increases the maximum throughput for the Internal Disk subsystem.
HDD Characteristics
The high performance 18GB UltraStar 18ZX which has a rotational speed of 10,020
rpm is the HDD used on Internal Disk for the Multiprise 3000. Along with its high
rpm and lower latency, the 18GB UltraStar 18ZX has state of the art data rate and
seek times. Some general characteristics of the 18GB UltraStar 18ZX are:
Fixed block size of 524 bytes.
Chapter 1. Overview of the S/390 Internal Disk Subsystem 1-3
Drive is mounted on a carrier which facilitates installation into the HDD
enclosure and provides additional shock protection for the HDD. The carrier
includes 3 LEDs:
– Green LED indicating good power
– Another Green LED indicating activity
– Amber LED indicating a check
Other major disk drive parameters of interest can be seen in the table that follows.
Table 1-1. 18GB UltraStar 18ZX Parameters
Usable Capacity 18 GBytes
Rotational Speed 10,000 RPM
Average Read Seek 6.5 ms
Latency 2.99 ms
Media Transfer Rate 23.4 to 30.4 MB/sec
SSA Transfer Rate 40 MB/sec
HDD Buffer Capacity 4 MBytes
Internal Disk RAID Fast Write
To improve write performance, Internal Disk for Multiprise 3000 includes a 32MB
Non-Volatile RAM (NVRAM) memory. The NVRAM is incorporated on the SSA
adapter card. Some advantages of the NVRAM addition are:
The NVRAM is a centralized storage that dynamically services the most active
HDDs on the SSA adapter card.
The NVRAM operates without a dependency upon the Internal Battery Feature.
The NVRAM retains all of its data in the event of a system or room Emergency
Power Off (EPO).
The operating system should issue a Sense Subsystem Status command to
determine the RAID fast write status of the associated logical volume and issue a
message to the operator indicating that the state change occurred. The RAID fast
write status is conveyed in byte 26 of the returned data. See "Sense Subsystem
Status" on page 4-132.
The MVS and VSE operating systems allow the operator to issue a query at any
time to display the current RAID fast write status of any volume. Alternatively,
ICKDSF can be used to query the status of a RAID array that is undergoing rebuild.
This function will work for VM as well as MVS and VSE.
Internal Disk Facilities
The Internal Disk Subsystem provides the following facilities:
1-4 Internal Disk Subsystem Reference Guide
CKD Emulation
The Internal Disk Subsystem processes ECKD/CKD channel programs which
presuppose variable length Count, Key, and Data fields like those of a 3990
subsystem. Internal Disk constructs CKD track images in memory and executes the
ECKD and CKD channel programs, which operate on byte boundaries, against the
images. In contrast, the Internal Disk Subsystem HDDs are SSA devices, designed
for data transfer via fixed block data transfers rather than the variable length CKD
fields. To bridge between the fixed block disks and the variable length ECKD/CKD
architecture, the CKD track images in memory are mapped onto a series of fixed
blocks suitable for transfer to and from the disk subsystem.
Internal Disk provides the ECKD command support and the compatibility mode
CKD command support as specified in the ECKD Architecture document.
Control Unit and Device Emulation
To maintain compatibility with current software, Internal Disk emulates an existing
control unit and existing device types and models. The 3990-2 is the control unit
that is emulated. The 3990-2 appears to be attached to a dedicated ESCON
channel. The CKD emulation and the emulated ESCON attachment allow Internal
Disk to perform all channel data transfers as non-synchronous operations.
LPAR Sharing
The logical volumes are emulated 3380 model J, E, or K volumes and
3390 model 1, 2, 3, or 9 volumes. The 3380 model J has the same track format
and number of cylinders as the 3380 Standard and the 3380 model D. By
emulating 3380 model J, E, or K volumes, all of the members of the 3380 family
are emulated.
The large capacity of the 18GB UltraStar 18ZX devices coupled with the RAID
organization allows multiple logical volumes to be mapped onto a physical RAID
array. The mappings of logical volumes that have been provided are detailed in the
"Internal Disk Logical Volume Configurations" section of this overview.
LPAR sharing is the ability for individual logical volumes to be shared among logical
partitions. The control unit emulation function in the SAP provides the appropriate
serialization and ensures that required allegiances are maintained. For example, if
a logical volume is reserved to one logical partition, any I/O requests from other
logical partitions are held in abeyance until the reservation is removed. LPAR
sharing does not enable Internal Disk to be shared between LPARs on different
physical CPCs.
LPAR sharing is enabled by the customer in his definition of the I/O configuration
via HCD, VM Dynamic I/O Configuration, or IOCP.
Enhanced HDD Reliability
Chapter 1. Overview of the S/390 Internal Disk Subsystem 1-5
Background Scrubbing
Background scrubbing is provided by Internal Disk to further reduce the probability
of customer loss of data.
There is a small possibility that after a data failure occurs, the redundancy data is
also found to be unreadable, resulting in loss of data to the customer. Data
scrubbing is provided to help ensure that all the data in the disk subsystem is
periodically proven readable so that if a failure occurs, the redundancy exists to
fully restore the original data.
In the background, the SSA adapter card initiates reads of all the data on all the
HDDs in the subsystem. No data transfer to the system is involved. Problems with
the data are promptly corrected by use of the redundancy data within the RAID
array.
Predictive Failure Analysis
Internal Disk is notified of impending hardware failures by the disk drive, which is
continuously performing predictive failure analysis. This process predicts device
failure conditions before failure occurs by measuring various parameters of the
head and disk as well as device electronics. When one of the predictive functions
detects an anomaly, the disk drive notifies the attachment of the potential problem.
The attachment in turn notifies the CPC service element, which initiates a service
action request.
Search Assist
Media Maintenance
The 18GB UltraStar 18ZX HDDs used within Internal Disk incorporate their own
internal media maintenance capabilities. Internal Disk media maintenance strategy
is to capitalize on the HDD capabilities wherever possible. Media maintenance is
performed internal to the storage subsystem. Therefore, defective sectors are
reassigned internally and require no customer interaction.
The Internal Disk emulates a 3990-2 control unit. Many of the channel programs
that were written for the 3990 find a target record by repetitively executing a
Search, TIC (Transfer in Channel) loop at every record until either the search is
satisfied or an exception such as End of Cylinder or No Record Found occurs. An
example of such a channel program is the PDS Directory Search channel program
which has the following form:
Seek
Define Extent
Search ID Equal
TIC *-1 (To the previous command)
Read Count
Search Key Equal or High
TIC *-2 (To the previous Read Count command)
Read Data or other command
The above program has two loops in it. Internal Disk detects both loops. Each of
the CCWs is executed in sequence, however, unlike a channel attached outboard
subsystem, Internal Disk is able to detect the TIC command along with its
parameters. The address to which the TIC command is pointing reveals a
Search/TIC loop. The parameters from the Search commands are preserved within
the subsystem, so that a search algorithm to find the desired record can be
1-6 Internal Disk Subsystem Reference Guide
SSID
executed when a loop is detected. With only one transfer of command parameters
from the application program, either the desired record is found or an exception
condition occurs. A second transfer of parameters may then be needed to structure
the ending conditions for the loop, such as providing Read Count information that
points to the correct final record.
Since Internal Disk has access to the channel program and its parameters,
detection of the loops does not depend on mode settings or hints from supervisory
software.
Subsystem identifier (SSID) support is provided for the unique identification of
Internal Disk control units within a customer installation.Three fixed default values
of SSID are provided:
An SSID of x'FD00' is assigned to the logical subsystem that services the disks
within the CPC frame.
An SSID of x'FD01' is assigned to the logical subsystem that services the disks
within the first expansion frame.
An SSID of x'FD02' is assigned to the logical subsystem that services the disks
within the second expansion frame.
Long Busy
The 3990 mod 2 uses the long busy indication to indicate to the host that an
extended operation is being executed at the device. Internal Disk uses the long
busy indication in some rare instances of subsystem failure as a means for having
CCW chains redriven from the host system.
The 3990 Models 2 and 3 documentation used “State Change Pending” to describe
the condition that is called “Long Busy” in this document.
Internal Disk Components
The following section describes the basic components of the Internal Disk
Subsystem. The Internal Disk is ordered by capacity. Based on the capacity,
varying numbers of components are required.
An Internal Disk Subsystem consists of the attachment integrated in the CPC and
some number of HDDs configured into RAID arrays. The major components
comprising an Internal Disk Subsystem are the SSA adapter cards, 18GB UltraStar
18ZX disk drives, and packaging for the HDDs within the CPC frame or expansion
frames.
Shared CPC resources
The CPC provides physical mounting facilities as well as power, cooling and
service panel facilities for the Internal Disk Subsystem. Special Licensed Internal
Code (LIC) running on the System Assist Processor (SAP) performs channel,
control unit (3990-2), and device (3380-J/E/K or 3390-1/2/3/9) emulation functions
and manages a portion of the CPC memory as a subsystem read cache across all
Internal Disk Subsystem devices. The read cache capacity can range from 32MB to
2GB. The 2GB cache can only be selected on a Multiprise 3000 which has 4GB
Chapter 1. Overview of the S/390 Internal Disk Subsystem 1-7
total storage, not on 1GB and 2GB models. The size can be changed by the
operator in 32MB increments.
SSA Adapter Card
The SSA adapter card supplies an SSA interface that operates at up to 40MB/sec
for connection of the HDDs. Two levels of physical interface conversion occur
between the Multiprise 3000 main memory and the SSA HDDs. The host Multiprise
3000 provides a Self Timed Interface (STI). A special purpose electronics bridge
called the STI-PCI bridge converts the STI to the upper interface of the SSA
adapter card. The SSA adapter card accepts a 32 bit address and data PCI
interface as its system interface.
The functionality of the SSA adapter card is far more than simple interface
conversion.
The SSA adapter card handles management of the RAID-5 organization. In
addition, the card manages both a 64MB RAM, used as a disk cache, and 32MB of
NVRAM used as fast write cache. Having these microcode intensive functions
performed at the SSA adapter card frees the SAP to do more channel and control
unit emulation, thus increasing the transaction processing power of the Internal Disk
subsystem.
Other features of the SSA Adapter Card include:
Support of 2 SSA loops
Independent Packet Network (IPN) transactions
Hardware scatter/gather
Full duplex communication
HDD hot plugging
Error detection and correction
Data scrubbing
HDD Packaging
An HDD for use on Internal Disk is mounted on a carrier which facilitates
installation and provides additional shock protection. The HDD and its carrier are
then mounted into either the frame that contains the CEC, or one of the two
possible expansion frames.
Internal Disk Logical Volume Configurations
Logical Volumes Mapped onto RAID Arrays
A specific number of HDDs form a RAID array. The RAID array could be regarded
as a single very large capacity device. Operating Systems currently recognize
3380 and 3390 volumes, however, changes would be needed to directly recognize
this new, large capacity device. To make the RAID array usable with minimal
impact on existing S/390 software, Internal Disk divides the space into multiple
3380 and/or 3390 logical volumes and provides addressability to the logical
volumes.
The size of the logical volumes vary. (For example, the 3390-2 is twice as large as
a 3390-1). As building blocks for the logical volumes, each array is first divided into
a sequence of storage units. Storage units then become the grains from which
1-8 Internal Disk Subsystem Reference Guide
logical volumes are built. The number of storage units per array depends on the
size of the array. For the supported array sizes, these are the corresponding
number of storage units:
Table 1-2. Storage Units Provided by HDDs
Number of HDDs in the Array.
4+P 73
6+P 109
Note: *Does not include spare HDDs
The following types of logical volumes can be mapped to an array, each requiring
the number of storage units shown:
Table 1-3. Storage Units Required for Logical Volume Type
Type of Logical Volume Required Number of Storage Units
3390-1 1
3390-2 2
3390-3 3
3390-9 9
3380-J 1
3380-E 2
3380-K 3
*
Number of Storage Units
There are no constraints on the types of logical volumes that can be mapped to an
array. However, the number of logical volumes are restricted by the storage units
available and the availability of logical volume addresses. For example, a 4+P array
has 73 possible storage units and 64 possible logical volume addresses, therefore:
24 logical volumes of 3390-3 can be accommodated using 72 of the 73 storage
units and 24 of the 64 logical volumes. 25 logical volumes of 3390-3 would
exceed the number of storage units available.
64 logical volumes of 3390-1 can be accommodated using 64 of the 73 storage
units and all 64 logical volumes. 65 logical volumes of 3390-1 would exceed
the number of logical volume addresses provided.
Logical Volume Address Resolution
The following details the rules for number of logical volume addresses that are
supported.
A particular logical volume is reached via an addressing hierarchy:
CHPID Address
Only one CHPID address is used for all of the logical volumes on a Multiprise 3000.
The address is assigned as x'FD'.
Chapter 1. Overview of the S/390 Internal Disk Subsystem 1-9
Control Unit Logical Address
Each of the three enclosures of a fully populated Multiprise 3000 contains a disk
subsystem with a logical 3990 mod 2 control unit.
The logical control unit that controls the HDD enclosure in the CPC frame is
assigned control unit logical address x'0'.
The logical control unit that controls the HDD enclosure in the first expansion
frame is assigned control unit logical address x'1'.
The logical control unit that controls the HDD enclosure in the second
expansion frame is assigned control unit logical address x'2'.
Device Address (on the logical control unit)
Each array is assigned a fixed, contiguous range of device addresses. The number
of device addresses for a 4+P array is 64 and for a 6+P array it is 96. The three
possible arrays in a CPC frame are assigned the device addresses as follows:
First array, 4+P, addresses 0 to 63 (x'00' to x'3F).
Second array, 4+P, addresses 64 to 127 (x'40' to x'7F').
Third array, 4+P, addresses 128 to 191 (x'80' to x'BF').
The three possible arrays in an expansion frame are assigned the device
addresses as follows:
First array, 4+P, addresses 0 to 63 (x'00' to x'3F').
Second array, 6+P, addresses 64 to 159 (x'40' to x'9F').
Third array, 6+P, addresses 160 to 255 (x'A0' to x'FF').
Internal Disk Configuration Granularity
The Internal Disk subsystem may consist of as little as one CPC frame containing
one RAID array, and as much as the CPC frame plus two expansion frames with a
total of nine RAID arrays in the three frames.
Growth of a configuration within a frame is straight forward. The entry CEC frame
has one RAID array. Arrays may be added, one at a time, up to the maximum in
the frame of 3 RAID arrays. One SSA loop on a SSA adapter card manages all of
the HDDs within the CPC frame. The SSA adapter card is capable of supporting
two SSA loops, so one of the loops is unused in the single frame configuration.
Similarly, the expansion frames may contain 1, 2, or 3 RAID arrays. The supported
configurations in the CPC frame:
Table 1-4. Configurations in CPC Frame
Number Configuration Approximate Gross Disk Capacity
B1 4+P, Spare 4 * 18 = 72
B2 4+P, 4+P, Spare 8 * 18 = 144
B3 4+P, 4+P, 4+P, Spare 12 * 18 = 216
(GByte)
When the first expansion frame is added, another SSA adapter card is also added.
If a second expansion frame is added, the SSA adapter card in the CPC frame
services the HDD enclosure in that frame.
1-10 Internal Disk Subsystem Reference Guide
Table 1-5. Configurations in CPC Frame
Number Configuration Approximate Gross Disk Capacity
E1 4+P, Spare 4 * 18 = 72
E2 4+P, 6+P, Spare 10 * 18 = 180
E3 4+P, 6+P, 6+P, Spare 16 * 18 = 288
Commands and Facilities Not Supported
Internal Disk does not support the following commands and facilities of the 3990
Model 2 storage control unit:
3380 track compatibility mode
Alternate/defective track assignment
Device Support Track Facility
Diagnostic Track Facility
Diagnostic Control
Write Special Home Address
Read Special Home Address
Buffered log data
Multipath Reconnection
Write Inhibit
Fencing/Unfencing
Service Information Message (SIM)
Variable length Record Zero key field. Internal Disk supports only a zero length
for Record Zero key field.
Variable length Record Zero data field. Internal Disk supports only a fixed eight
byte length for Record Zero data field.
(GByte)
IOCP Definitions
The Input/Output Configuration Program (IOCP) utility is used to specify Internal
Disk configuration to the hardware. IOCP changes for Internal Disk include using
the DSD (Direct System Device) channel type and unique rules for control units and
devices defined on the new channel:
A channel path type of DSD (Direct System Device) is defined for Internal Disk
interfaces. The DSD CHPID can be dedicated or shared, or reconfigurable
(REC).
There is only one valid DSD path on the Multiprise 3000. It is x'FD'.
IOCP requires that the control unit type defined on a DSD CHPID begin with
"3990". Other characters (e.g., -2) may follow 3990, but the "UNIT=" keyword
value must always begin with 3990.
If multiple control units are defined on a CHPID, each must have a unique
Control Unit Logical Address specified. The "CUADD" keyword is used for this
and the value must be between 0-2. Each SSA loop is defined as a unique
CUADD value.
An Internal Disk control unit defined on a DSD CHPID must contain only one
path. The devices defined on a DSD control unit represent the logical volumes
Chapter 1. Overview of the S/390 Internal Disk Subsystem 1-11
defined for each RAID-5 array. Each logical volume may only be defined to one
control unit.
The Self Timed Interface (STI) connection between the processor and the SSA
adapter is specified as the single DSD CHPID. Control unit (CNTLUNIT) statements
are defined for each SSA loop in the storage facility. I/O device (IODEVICE)
statements are needed for each logical volume defined on each RAID-5 array on a
SSA loop.
For details about the IOCP specifications of Internal Disk, see Input/Output
Configuration Program User's Guide and ESCON Channel- to-Channel Reference
(GC38-0401).
RAS Characteristics
Internal Disk is a feature of the CPC platforms with which it is packaged. The RAS
characteristics of Internal Disk are intended to complement the RAS Characteristics
and Service Strategy of the host.
1-12 Internal Disk Subsystem Reference Guide
Chapter 2. Processing Commands and Presenting Status
An understanding of the Enterprise Systems Architecture/390 channels is
assumed throughout this manual. The only points discussed are those points
requiring further clarity, or where the subsystem has an option.
The following I/O instructions in ESA/390 affect the subsystem:
Clear Subchannel
Halt Subchannel
Resume Subchannel
Start Subchannel
Test Subchannel
Reset Channel path
Channel Subsystem Call - Reset Control Unit.
The Manual IBM Enterprise Systems Architecture/390 Principles of Operation
describes the above instructions.
Internal Disk Subsystem supports all count-key-data (CKD) and extended
count-key-data (ECKD) channel commands defined in this manual.
Input/Output Channel Interface
Command Processing
The Internal Disk Subsystem accepts and processes channel commands when:
The host system is powered on and initialized.
The logical path has been established between the Internal Disk Subsystem
and the host system.
The logical volume address is within the range supported by the control unit.
The physical RAID array associated with the logical volume address is installed
and on-line.
A Logical Volume has been configured at the device address.
If any of these conditions are not satisfied, the Internal Disk Subsystem will not
process the command. The subsystem response depends on which conditions are
not satisfied.
Status Presentation
Internal Disk Subsystem generates status conditions that it presents to the host
channel.
Status Byte
The eight bits of status information make up the status byte. The status byte
contains information that shows the status of the subsystem or logical volume. The
status byte contents are shown in Table 2-1 on page 2-2.
Copyright IBM Corp. 1999 2-1
Table 2-1. Status Byte
Bit Name
0 Attention
1 Status Modifier
2 Control-Unit End
3 Busy
4 Channel End
5 Device End
6 Unit Check
7 Unit Exception
Attention (Bit 0)
Bit 0 is set to a ‘1’:
With device end and unit exception to show a state transition. (See
“State-Change Interruption” on page 2-5.)
With unit check to show that the addressed logical volume has been
conditioned with a Set Special Intercept Condition order of the Perform
Subsystem Function command on any interface with the same path group ID
as the interface processing the command. See “Set Special Intercept Condition
- PSF Order Code x'1B'” on page 4-132.
Status Modifier (Bit 1)
Bit 1 is set to a ‘1’:
With channel end and device end to show a Search High, Search Equal, or a
Search High or Equal command completed and the condition is satisfied.
With unit check and channel end to show an unusual condition from the last
operation. The last channel command must be retried. If device end is on,
immediate retry is requested. (See “Channel Command Retry” on page 2-9.)
Control Unit End (Bit 2)
Bit 2 is set to zero:
There are no Control Unit Busy instances that need to be ended via the Control
Unit End bit.
Busy (Bit 3)
Bit 3 is set to a ‘1’:
When the logical volume is busy.
When the logical volume is reserved to another logical path or path group.
When pending status is being presented during a channel initiated selection
busy is included with the pending status.
2-2 Internal Disk Subsystem Reference Guide
Channel End (Bit 4)
Bit 4 is set to a ‘1’:
At the end of parameter transfer or data transfer of each command.
At the end of command execution for commands that have no parameter or
data transfer.
Device End (Bit 5)
Bit 5 is set to a ‘1’:
When alone or with channel end to show that a logical volume is available for
use.
With attention and unit exception to show a state change. (See “State-Change
Interruption” on page 2-5.)
Unit Check (Bit 6)
Bit 6 is set to a ‘1’ when the subsystem detects an unusual or error condition. If
status modifier is not set with unit check, one of following combinations of channel
end, device end, and unit check define the unusual or error condition:
Unit check alone occurs as initial status for a command if the command cannot
be processed. (See “Unit Check Status” on page 2-4.)
Device end and unit check occur when the channel has already accepted
channel end for this command, and unit check has occurred.
Unit check is presented with attention if the logical volume has been
conditioned by the Set Special Intercept Condition order of the Perform
Subsystem Function command on any interface with the same path group ID
as the interface processing the command.
Channel end, device end and unit check are for all other unit check conditions.
If status modifier and channel end are set with unit check, command retry is
requested. If immediate retry is requested, device end is included. (See “Channel
Command Retry” on page 2-9.)
Note: When Internal Disk presents a unit check to the first command in a chain,
examine the sense data to determine if the unit check is associated with the
failing channel command word (CCW), or is an asynchronous operation
error just being presented on a current CCW chain.
Unit Exception (Bit 7)
Bit 7 is set to a ‘1’:
With channel end and device end to show a read command, other than Read
Track or Read Multiple CKD, processed a data area of an end-of-file record.
The data area does not transfer to the channel.
With channel end and device end to show that an update write command
outside a Locate Record or Locate Record Extended domain tried to update the
data area of an end-of-file record. (See page 4-82 for the update write
commands.)
With attention and device end to show a state change. (See “State-Change
Interruption” on page 2-5.)
Chapter 2. Processing Commands and Presenting Status 2-3
Initial Status Byte
The initial status byte is zero for all valid non-immediate commands with the
following exceptions:
Logical volume is busy – Busy Bit alone is returned.
Status condition is pending – The pending status combination plus Busy Bit is
Busy Status
The logical volume is:
Busy.
Reserved to another logical path or path group.
Has pending status to present.
Unit Check Status
The control unit presents unit check in the initial status for any of the following:
A command reject.
A long busy condition exists for the logical volume or subsystem.
A resetting event occurred since the last Start Subchannel to this logical
returned. See “Status Pending Conditions.”
volume on this channel interface.
Status Pending Conditions
Status pending is status that the subsystem has, but which has not yet been
accepted by the channel.
Status Pending in the Subsystem
Assume that a status condition for the addressed logical volume is pending in the
subsystem. The status pending is presented as initial status to the next initial
selection, and the busy bit is set to a ‘1’ in the status byte. The active busy bit
shows that the logical volume is busy because of the outstanding status. The
status pending is cleared.
Status is pending in the subsystem if:
A cancel was signaled after initial status was accepted, but before channel end
was accepted for any CCW. The ending status for the operation is pending
when the operation is complete.
Note: In some DASD control units, pending status for a device makes the control
unit busy to all other devices attached to the control unit. This is not the
case in this subsystem. In this subsystem, the logical volume is busy and
the control unit is available for use.
Status Pending in the Logical Volume
Status pending at a logical volume is presented as initial status if the logical volume
is not busy. The busy bit is included. The status conditions that can be pending at
the logical volume are device end alone, or attention with device end and unit
exception for a state-change interruption, or device end and unit check.
Status pending at the logical volume is cleared when the channel accepts the
status, or it is reset by selective reset or system reset.
Status is pending at the logical volume if:
2-4 Internal Disk Subsystem Reference Guide
Channel end has been previously accepted and the operation is now complete,
but device end has not been presented to the channel.
Busy status has previously been sent and the logical volume is now not busy.
The resulting device end has not been presented to the channel.
The logical volume changed states, and that status has not been presented to
the channel. See “State-Change Interruption.”
Address Associated with Pending Status
All status conditions are associated with a specific logical volume address.
State-Change Status
State-Change Interruption
A state-change interruption is the combination of three device status bits: attention,
device end, and unit exception. As with all status, busy can also be included.
Note: Unit check may also be included if the logical volume has previously
received a Perform Subsystem Function command with a Set Special
Intercept Condition order.
The term state-change interruption is used in this manual for the former term
pack-change interruption. A state-change interruption occurs for a state change in
the subsystem or the logical volume. The following conditions cause a
state-change interruption:
Logical volume goes from not-ready to the ready state.
Long busy condition no longer exists for the logical volume.
Set System Characteristics order changes stored values for a path group. This
state change is issued only to channels in the path group that changed. See
"Set System Characteristics - PSF Order Code x'1D'" on page 4-129.
State-change interruptions are sent to each LPAR.
Note: For a state-change interruption, a host program can issue the following
CCWs to determine the current status of the subsystem:
Sense ID
Read Device Characteristics
Read Configuration Data
Sense Subsystem Status
Effect of Subsystem and Device State on Status Presented to
Command
The status presented to commands depends on:
The particular command code.
The state of the subsystem.
The state of the device addressed.
Most channel commands are concerned with data transfer control to cause data to
be moved to or from a disk track (or disk track image for a cache hit). Examples of
these commands are Seek, Set Sector, Read Data, and Write Key & Data. Data
transfer command chains are issued by application programs or by data
management programs acting on the behalf of the application. In almost all cases
these CCW chains have Define Extent or Set File Mask as the first command of the
Chapter 2. Processing Commands and Presenting Status 2-5
chain. These command chains may compete for data within a single logical
volume. Only one data transfer chain is active on a logical volume at a time.
I/O Queuing
Multiple LPARs may issue CCW chains that require the same logical volume.
Internal Disk provides for queuing of command chains from the various LPARs and
provides for orderly execution of the chains. In Table 2-2 on page 2-8 the action
IOQueued indicates that Internal Disk is queuing the chain. The queuing discipline
is FIFO. While the chain is queued the channel emulator in Internal Disk maintains
the subchannel in a Start Pending state.
Bypass Commands
Vital to the design of LPAR sharing are a set of commands which are required to
be executed although the logical volume is busy on behalf of another LPAR. (In
the Internal Disk design these commands are called Bypass Commands because
they bypass the logical volume busy conditions that cause normal data transfer
commands to be terminated or queued.) The specific commands that are executed
while the logical volume is busy on behalf of another LPAR are:
Mnemonic Command Name Hex
SPID Set Path Group ID AF
SNID Sense Path Group ID 34
RSTA Reset Allegiance 44
UR Unconditional Reserve 14
RDC Read Device Characteristics 64
RCD Read Configuration Data FA
SNSID Sense ID E4
SNSS Sense Subsystem Status 54
SNS Sense with contingent allegiance 04
Other Special Commands
Another set of commands has the property of executing although the logical volume
is either Long Busy or Not Available. These commands do not execute through
logical volume busy conditions. They are:
Mnemonic Command Name Hex
RES Device Reserve B4
REL Device Release 94
PSF Perform Subsystem Function 27
RSSD Read Subsystem Data 3E
Three other commands have special architectural and historical significance and
are executed somewhat differently from the data transfer commands. They are:
Mnemonic Command Name Hex
NOP No-Operation 03
RIPL Read IPL 02
SMR Suspend Multipath Reconnection 5B
Note: The Suspend Multipath Reconnect command is allowed to precede
commands that otherwise MUST be the first command in a CCW chain. In
the Internal Disk implementation, SMR does not bypass logical volume busy
conditions. Therefore, if a command in the list of bypass commands is
chained from an SMR, the CCW chain will NOT be a bypass chain. When
the logical volume is busy, such a chain will be queued and executed later,
as with normal data transfer chains.
2-6 Internal Disk Subsystem Reference Guide
Table Column Definitions
The table that follows is intended to show the actions taken for the bypass
commands particularly. Table 2-2 on page 2-8 summarizes the status presented
to commands when the subsystem or device is in one of the states described in the
following text.
HDD Long Busy. This state exists when the device is doing an operation that
exceeds the time for normal busy. On Internal Disk, there are some recovery
actions taken at the SSA adapter card that will cause Long Busy.
Logical Volume Busy on Different LPAR. The logical volume is currently
executing a data transfer chain on behalf of another LPAR.
Logical Volume Reserved to a Different LPAR. The logical volume is
currently Reserved on behalf of another LPAR.
Logical Volume Intervention Required. Causes for Intervention Required can
be found on page 5-4.
Logical Volume Chain UR-ed. The logical volume had allegiance to this
LPAR, and another LPAR issued an Unconditional Reserve or Reset Allegiance
command. The allegiance was broken but the resulting Unit Check status has
not yet been presented.
Logical Volume Reset Notify. A system reset has been received and
executed. The reset has removed any allegiance that was in place for the
resetting LPAR. Unit Check status is presented to the next initial selection by
the resetting LPAR to indicate that a reset occurred.
An exception is made for Read IPL. Unit check is not presented in initial
status. Instead the command is Accepted and allowed to execute. No Reset
Notification occurs in this case.
Contingent Allegiance to This LPAR. Either subsystem or logical volume
contingent allegiance exists for this LPAR due to a unit check that was
presented earlier.
Logical Volume Contingent Allegiance to Different LPAR. Logical volume
contingent allegiance exists for a different LPAR due to a unit check that was
presented while executing a non-bypass CCW chain.
Subsystem Contingent Allegiance to Different LPAR. Subsystem contingent
allegiance exists for a different LPAR due to a unit check that was presented to
that LPAR while executing a bypass CCW chain.
Chapter 2. Processing Commands and Presenting Status 2-7
Command
Mnemonic
Subsystem
Contingent
Logical
Volume
Contingent
Contingent
Allegiance
Different
LPAR
Allegiance
Different
LPAR
Allegiance
to This
LPAR
NOP
SPID
Accepted
Accepted
IOQueued
Accepted
Accepted
(6)
Accepted
SNIDURRSTA
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
RIPL
RES
Accepted
Accepted
IOQueued
IOQueued
Accepted
Accepted
REL
SNS (1)
SNS (2)
SNS (3)
Accepted
Accepted
Accepted
Accepted
IOQueued
IOQueued
AcceptedNAAccepted
AcceptedNAAccepted
Accepted
SNSS
RDC
Accepted
Accepted
Accepted
Accepted
Accepted
SNSID
RCD
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
PSF
RSSD
Accepted
Accepted
IOQueued
IOQueued
Accepted
Accepted
SMR
All other
commands
Accepted
Accepted
IOQueued
IOQueued
Accepted
Accepted
Logical
Volume
Reset
Notify
Unit Check
Unit Check
Unit Check
Logical
Volume
Chain
UR-ed
Unit Check
Unit Check
Unit Check
Subsystem or Logical Volume State
Logical
Volume
Intervention
Required
or IDFW
Exposed
Unit Check
(4)
Accepted
Accepted
Logical
Volume
Reserved
Different
LPAR
IOQueued
Accepted
Accepted
Logical
Volume
Busy on
Different
LPAR
IOQueued
Accepted
Accepted
Accepted
HDD
Long Busy
Unit Check
Accepted
Unit Check
Unit Check
Accepted
Unit Check
Unit Check
Unit Check
Accepted
Accepted
Unit Check
Accepted
Accepted
IOQueued
Accepted
Accepted
IOQueued
Accepted
Accepted
Unit Check
(5)
Unit Check
Unit Check
(4)
Accepted
IOQueued
IOQueued
Accepted
Unit Check
Unit Check
Unit Check
Unit Check
Accepted
Unit Check
(4)
IOQueued
IOQueued
IOQueued
IOQueued
Accepted
Accepted
Accepted
Unit Check
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Unit Check
Unit Check
Unit Check
Unit Check
Unit Check
Unit Check
Accepted
Accepted
Accepted
IOQueued
IOQueued
IOQueued
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Unit Check
Unit Check
Unit Check
Unit Check
Unit Check
Unit Check
Accepted
Accepted
Accepted
IOQueued
IOQueued
IOQueued
Accepted
IOQueued
IOQueued
Accepted
Accepted
Accepted
Unit Check
Unit Check
Unit Check
Unit Check
Accepted
Unit Check
(4)
IOQueued
IOQueued
IOQueued
IOQueued
Accepted
Unit Check
Table 2-2. Effect of Subsystem and Device State on Status Presented to Commands
Command
Mnemonic
NOP
SPID
SNIDURRSTA
2-8 Internal Disk Subsystem Reference Guide
RIPL
RES
REL
SNS (1)
SNS (2)
SNS (3)
SNSS
RDC
SNSID
RCD
PSF
RSSD
SMR
All other
commands
(1) No contingent allegiance for this interface.
(2) Contingent allegiance at subsystem level for this interface.
(3) Contingent allegiance at logical volume level for this interface.
(4) Sense data reflects Intervention Required or possible loss of IDFW data.
(5) The Reset Notification Mark is reset.
(6) Contingent Connection and Sense Data are preserved through the NOP.
Listing of Commands Appearing in Table 3
A full listing of all of the command names and mnemonics that are executed by
Internal Disk appears in Table 7 on page 4-3. For convenience in referring to the
preceding table a list of only the commands that appear in the table is shown
below. The commands are listed in the order in which they appear in the table.
Mnemonic Command Name Hex
NOP No-Operation 03
SNID Set Path Group ID AF
SPID Sense Path Group ID 34
UR Unconditional Reserve 14
RSTA Reset Allegiance 44
RIPL Read IPL 02
RES Device Reserve B4
REL Device Release 94
SNS Sense 04
SNSS Sense Subsystem Status 54
RDC Read Device Characteristics 64
SNSID Sense ID E4
RCD Read Configuration Data FA
PSF Perform Subsystem Function 27
RSSD Read Subsystem Data 3E
SMR Suspend Multipath Reconnection 5B
Contingent Allegiance
Contingent allegiance is established after the channel accepts a status byte
containing unit check. In Internal Disk, the contingent allegiance may be logical
volume contingent allegiance, which impacts other LPARs attempting to use the
logical volume, or it may be subsystem contingent allegiance on behalf of a
particular LPAR and logical volume.
There are no cases in Internal Disk where a contingent allegiance at one logical
volume impacts the use of other logical volumes.
The contingent allegiance lasts until a command other than No-Op receives an
initial status byte of all zeros for the channel and logical volume that generated the
unit check, or a selective or system reset occurs.
Channel Command Retry
Channel command retry is an Internal Disk initiated action that retries a command
without an I/O interruption. When Internal Disk is ready to retry the command, it
presents device end. For channel command retry, the status byte contains:
Unit check (bit 6)
Status modifier (bit 1)
Channel end (bit 4).
If device end (bit 5) is on with the retry request, the retry is immediate. If the
control unit must delay reconnection to complete other activity before it is ready for
reconnection, device end will not be included in the retry status but will be
presented when Internal Disk is ready for reconnection.
As the control unit processes host command chains, an error can occur in the
control unit or disk subsystem, or a resource may not be available. The control unit
Chapter 2. Processing Commands and Presenting Status 2-9
uses channel command retry to disconnect from the channel and recover from the
error or wait for the required resource.
Channel Command Retry–Control Unit Initiated
The control unit initiates retries for delayed selection and device orientation.
Delayed Selection and Device Orientation
Internal Disk accepts initial selection without selecting the physical device. The
command chain proceeds until an operation requires the device. When Internal
Disk determines that it needs the physical device, it presents channel command
retry to the channel. At the same time, Internal Disk selects and orients the device
to the correct position. If the device is doing an internal operation, the channel
remains in retry status until the device becomes available and activity completes.
When the device activity is complete, Internal Disk presents device end. The
channel then sends the command again, and the operation proceeds normally.
System and Selective Reset
System reset affects all status and allegiances for that logical path on all attached
logical volumes. Selective reset affects the status and allegiance of only the
selected logical volume.
The effect of system and selective reset depends on the operation in progress. For
example, asynchronous (internal) operations complete normally. For read hits in
cache, the operation stops without affecting the cache image. For write operations,
Internal Disk terminates the writing to the cache and discards the record. The
modified record is not written to the logical volume.
Effect of System Reset
A system reset affects all logical volumes that have allegiance to the logical path
issuing the reset. Status is cleared for all affected logical volumes. Logical
volumes that were Reserved are Released.
A system reset causes the path group ID for the logical path to reset. The
parameters associated with Set System Characteristics - PSF Order Code x'1D' are
also cleared by the system reset. See "Set System Characteristics - PSF Order
Code x'1D'" on page 4-129 for a description of the parameters.
Effect of Selective Reset
Selective reset affects only the chain in process at the time of the selective reset.
The chain processing is terminated and ending status is cleared. It does not alter
the reserved state of a logical volume or the valid path group ID for an interface.
Reset Notification
The control unit notifies the host that a system reset has been received. Any
system resetting event on Internal Disk causes a later channel initiated selection to
be unit checked. The sense data contains format 0, message 8 with a program
action code of X'16'.
The 3990 Model 2 has a procedure where the first selection on the interface will be
unit checked with sense data that includes equipment check and environmental
data present. This causes the event to be logged, which is appropriate for a stand
2-10 Internal Disk Subsystem Reference Guide
Loading...