IBM Ultra320 User Manual

PCI-X Dual Channel Ultra320 SCSI RAID Adapter
Installation an d Using Guid e
SA23-1324-01
PCI-X Dual Channel Ultra320 SCSI RAID Adapter
Installation an d Using Guid e
SA23-1324-01
A
©
Note
Before using this information and the product it supports, be sure to read the general information under ″Product Warranties and Notices included with your system unit.
Second Edition (August 2004)
Before using this information and the product it supports, read the information in “Safety Information” on page v and Appendix B, “Notices,” on page 33.
reader’s comment form is provided at the back of this publication. If the form has been removed, address comments to Publications Department, Internal Zip 9561, 11501 Burnet Road, Austin, Texas 78758-3493. To send comments electronically, use this commercial internet address: aix6kpub@austin.ibm.com. Any information that you supply may be used without incurring any obligation to you.
International Business Machines Corporation 2003, 2004. All rights reserved.
Note to U.S. Government Users Restricted Rights--Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents

Safety Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v
Handling Static Sensitive Devices . . . . . . . . . . . . . . . . . . . . . . . . . vii
About This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
ISO 9000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
Highlighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
References to AIX and Linux Operating Systems . . . . . . . . . . . . . . . . . . . . .ix
Related Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x
Chapter 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Verifying Operating System Requirements . . . . . . . . . . . . . . . . . . . . . . . .2
Using AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Using Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Chapter 2. Preparing for Installation . . . . . . . . . . . . . . . . . . . . . . . . .3
Checking Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Using AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Using Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Preparing for a PCI Slot-Length Installation . . . . . . . . . . . . . . . . . . . . . . .3
Determining Your Cable Requirements . . . . . . . . . . . . . . . . . . . . . . . . .4
Internal Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
External Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Chapter 3. Installing the Device Driver Software (AIX) . . . . . . . . . . . . . . . . . .5
Chapter 4. Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX) . . . . . . . .7
Installing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Connecting Internal SCSI Backplanes and Devices . . . . . . . . . . . . . . . . . . . .7
Connecting External SCSI Storage Peripheral Devices . . . . . . . . . . . . . . . . . . .8
Chapter 5. Setting Up the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX) . . . . . . .9
Verifying the Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Verifying AIX Software Installation . . . . . . . . . . . . . . . . . . . . . . . . . .9
Verifying Hardware Installation . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Updating Hard Disk Drive and Adapter Microcode . . . . . . . . . . . . . . . . . . .10
Configuring the PCI-X Dual Channel Ultra320 SCSI RAID Adapter . . . . . . . . . . . . . .10
Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s . . . . . . . . . . . . .10
Wide Bus Enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Quick Arbitration and Select (QAS) Capability . . . . . . . . . . . . . . . . . . . . .11
Problem Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Chapter 6. Using Disk Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Using AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Using Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID
Adapter (Linux ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ipr Device Driver . . . . .15
Connecting External or Internal Devices . . . . . . . . . . . . . . . . . . . . . . .15
Verifying the Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Verifying Linux Software Installation . . . . . . . . . . . . . . . . . . . . . . . . .16
iii
iv
Installing the Downloaded Device Driver and Utility Software . . . . . . . . . . . . . . .17
Verifying Hardware Installation . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Updating Hard Disk Drive and Adapter Microcode . . . . . . . . . . . . . . . . . . . .20
Configuring the PCI-X Dual Channel Ultra320 SCSI Adapter . . . . . . . . . . . . . . . . .20
Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s . . . . . . . . . . . . .20
Wide Bus Enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Quick Arbitration and Select (QAS) Capability . . . . . . . . . . . . . . . . . . . . .21
Setting Bus Speed at Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Problem Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ibmsis Device Driver . . . .22
Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter . . . . . . . . . . . . . .22
Verifying the Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Updating Hard Disk Drive and Adapter Microcode . . . . . . . . . . . . . . . . . . .26
Configuring the PCI-X Dual Channel Ultra320 SCSI Adapter . . . . . . . . . . . . . . . .26
Problem Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Appendix A. Communications Statements . . . . . . . . . . . . . . . . . . . . . .29
Federal Communications Commission (FCC) Statement . . . . . . . . . . . . . . . . . .29
European Union (EU) Statement . . . . . . . . . . . . . . . . . . . . . . . . . . .29
International Electrotechnical Commission (IEC) Statement . . . . . . . . . . . . . . . . .29
United Kingdom Telecommunications Safety Requirements . . . . . . . . . . . . . . . . .29
Avis de conformité aux normes du ministère des Communications du Canada . . . . . . . . . .30
Canadian Department of Communications Compliance Statement . . . . . . . . . . . . . .30
VCCI Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Electromagnetic Interference (EMI) Statement - Taiwan . . . . . . . . . . . . . . . . . .30
Radio Protection for Germany . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Appendix B. Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Appendix C. Connecting SCSI Hard Disk Drives to the Adapter . . . . . . . . . . . . . .35
Determining Current SCSI Hard Disk Drive Addresses . . . . . . . . . . . . . . . . . . .36
Appendix D. Connection Examples . . . . . . . . . . . . . . . . . . . . . . . . .37
Internal Hard Disk Drive Connection . . . . . . . . . . . . . . . . . . . . . . . . .37
Internal and Single External Storage Peripheral Connections . . . . . . . . . . . . . . . .37
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide

Safety Information

DANGER
electrical outlet that is not correctly wired could place hazardous voltage on metal parts of
An the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock.
Before installing or removing signal cables, ensure that the power cables for the system unit and all attached devices are unplugged.
When adding or removing any additional devices to or from the system, ensure that the power cables for those devices are unplugged before the signal cables are connected. If possible, disconnect all power cables from the existing system before you add a device.
Use one hand, when possible, to connect or disconnect signal cables to prevent a possible shock from touching two surfaces with different electrical potentials.
During an electrical storm, do not connect cables for display stations, printers, telephones, or station protectors for communication lines.
v
vi
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
v
v
v Do
v Do
v
v
it is
v

Handling Static Sensitive Devices

Attention: Static electricity can damage this device and your system unit. To avoid damage, keep this
device in its anti-static protective bag until you are ready to install it. To reduce the possibility of electrostatic discharge, follow the precautions listed below:
Limit your movement. Movement can cause static electricity to build up around you.
Handle the device carefully, holding it by its edges or its frame.
not touch solder joints, pins, or exposed printed circuitry. not leave the device where others can handle and possibly damage the device.
While the device is still in its anti-static package, touch it to an unpainted metal part of the system unit
for at least two seconds. (This drains static electricity from the package and from your body.)
Remove the device from its package and install it directly into your system unit without setting it down. If
necessary to set the device down, place it on its static-protective package. (If your device is an adapter, place it component-side up.) Do not place the device on your system unit cover or on a metal table.
Take additional care when handling devices during cold weather, as heating reduces indoor humidity and increases static electricity.
vii
viii
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
v
v
If
v

About This Book

This book provides information about installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter and device driver support software, as well as cabling requirements. Use this book in conjunction with your specific system unit and operating system documentation.

ISO 9000

ISO 9000 registered quality systems were used in the development and manufacturing of this product.

Highlighting

The following highlighting conventions are used in this book:
Bold
Identifies commands, subroutines, keywords, files, structures, directories, and other items whose names are predefined by the system. Also identifies graphical objects such as buttons, labels, and icons that the user selects.
Italics
Monospace
Identifies parameters whose actual names or values are to be supplied by the user. Identifies examples of specific data values, examples of text similar to what you might see
displayed, examples of portions of program code similar to what you might write as a programmer, messages from the system, or information you should actually type.

References to AIX and Linux Operating Systems

This document contains references to the AIX and Linux operating system. If you are using another operating system, consult the appropriate documentation for that operating system.
This document may describe hardware features and functions. While the hardware supports them, the realization of these features and functions depends upon support from the operating system. AIX and Linux provide this support. If you are using another operating system, consult the appropriate documentation for that operating system regarding support for those features and functions.

Related Publications

The following publications contain related information:
System unit documentation for information specific to your hardware configuration Operating system documentation for information specific to your software configuration and disk arrays.
your operating system is AIX, refer to the PCI-X SCSI RAID Controller Reference Guide for AIX, order
number SA23-1323. If your operating system is Linux, refer to the PCI-X SCSI RAID Controller
Reference Guide for Linux, order number SA23-1327 PCI Adapter Placement Reference, order number SA38-0538 (for the latest version, you may need to
contact your marketing representative)
ix
v
v
v
v
v
x

Trademarks

The following terms are trademarks of International Business Machines Corporation in the United States, other countries, or both:
AIX AIX 5L
Eserver
pSeries RS/6000
company, product, and service names may be trademarks or service marks of others.
Other
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
an
1
2
3
4
5

Chapter 1. Overview

The PCI-X Dual Channel Ultra320 SCSI RAID Adapter is a 133 MHz 64-bit PCI-X adapter. It allows you to connect SCSI hard disk drives in RAID 0, 5 and 10 configurations to system units with a Peripheral Component Interconnect (PCI) bus. RAID (Redundant Array of Independent Disks) provides a way to distribute stored data among two or more hard disk drives inside or attached to a host system. This adapter supports non-RAID disks formatted to 512 bytes/sector, RAID disks formatted to 522 bytes/sector, and other devices including tape and optical devices.
Note: This guide refers to various hardware and software features and functions. The support for these
features and functions depends upon the limitations of your hardware and software. AIX and Linux
support all functions mentioned. If you are using another operating system, consult the appropriate
documentation for that operating system regarding support for the mentioned features and
functions. This guide also uses the terms channel and bus interchangeably.
The PCI-X Dual Channel Ultra320 SCSI RAID Adapter has two channels, 0 and 1. Each channel has both
internal and an external connector. Only one internal or external connector can be used on each
channel.
Figure 1. PCI-X Dual Channel Ultra320 SCSI RAID Adapter
Channel 1 external, 68-position, VHDCI (very high density cable interconnect) Channel 0 external, 68-position, VHDCI (very high density cable interconnect) Channel 0 internal, 68-position, high-density plastic D-shell SCSI standard connector for 16-bit
attachment Channel 1 internal, 68-position, high-density plastic D-shell SCSI standard connector for 16-bit
attachment Adapter Handle
You can create, control, and manage a RAID configuration in combination with supporting host software. For more RAID information, refer to the PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323, or PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327. The PCI-X Dual Channel Ultra320 SCSI RAID Adapter supports SCSI synchronous data rates from 20 MB/s (16-bit) up to 320 MB/s (16-bit) on two independent SCSI-2 channels.
1
to
To
v
v
No
If
v
v
v
2
If the AIX or Linux operating system is installed on a disk array, the RAID disk array may be used as the boot source.
You may receive this adapter preinstalled in a hot-swap carrier for use in certain system types. This carrier allows the adapter to be installed and removed without removing the top cover of the system. For handling instructions, refer to the user’s guide for your system. If the adapter does not fit in your system, the handle may need to be removed in order for the adapter to fit properly. The handle can be removed by referring
“Preparing for a PCI Slot-Length Installation” on page 3.
The PCI-X Dual Channel Ultra320 SCSI RAID Adapter conforms to the American National Standards Institute (ANSI) SCSI-2 standard and the PCI local specification, revision 2.2.

Verifying Operating System Requirements

ensure that your operating system will support the PCI-X Dual Channel Ultra320 SCSI RAID Adapter,
use the following procedure to verify that your operating system requirements.

Using AIX

The PCI-X Dual Channel Ultra320 SCSI RAID Adapter is supported on the following levels of AIX or higher:
AIX 5L Version 5.1 with the 5100-04 Recommended Maintenance Package (APAR IY44478) AIX 5L Version 5.2 with the 5200-01 Recommended Maintenance Package (APAR IY44479)
determine which maintenance level is installed on your system, type
To
oslevel
-r. Verify that the
maintenance level displayed is 5100-04, 5200-01, or higher. If no maintenance level is installed on your system, the following is an example of the data that displays on your screen:
recommended maintenance level information is available.
updates are needed, the latest maintenance packages and update instructions can be downloaded from the pSeries and RS/6000 Support Web site. The Web address is: http://techsupport.services.ibm.com/server/support. Updates can also be ordered by calling IBM Support.

Using Linux

The PCI-X Dual Channel Ultra320 SCSI RAID Adapter is supported on the following Linux distributions:
Any distribution based on United Linux 1.0 Service Pack 3, such as SuSE Linux Enterprise Server 8 Service Pack 3 or newer
Red Hat Enterprise Linux 3 or newer SuSE Linux Enterprise Server 9 or newer
determine which release is installed on your system, refer to your Linux distributor’s documentation. If
To updates are needed, obtain them from your Linux distributor or download them from the IBM StorageIO Web Site. The Web address is: http://www-124.ibm.com/storageio/ipr/.
Note: SuSE Linux Enterprise Server 8 SP3 used the
Channel Ultra320 SCSI RAID adapter. This device driver has been replaced by the driver, which is used on all other supported distributions. Information on using the older device driver is contained in “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ibmsis Device Driver” on page 22. Refer to this section if you are still using this device driver.
ibmsis
device driver to control the PCI-X Dual
ipr
device
ibmsis
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
to
v
v
v
To
v
v
v
v
v
To
v
v An
v
In
v
v
v
In
v
v
To
1.
2.

Chapter 2. Preparing for Installation

This chapter helps you prepare to install the PCI-X Dual Channel Ultra320 SCSI RAID Adapter. Preparing
install the adapter involves the following tasks: Checking Prerequisites Preparing for a PCI Slot-Length Installation Determining Your Cable Requirements

Checking Prerequisites

install the PCI-X Dual Channel Ultra320 SCSI RAID Adapter, you may need the following items: Your system unit installation guide IBM
Eserver
Flat-blade screwdriver PCI Adapter Placement Reference, order number SA38-0538 Documentation provided with your SCSI devices
pSeries & RS/6000 Standalone Diagnostics CD-ROM (Version 5.2.0.11 or higher)
Note:
This adapter is supported on multiple systems and may require a system firmware update. Contact your service representative for further details.
connect SCSI hard disk drives to the PCI-X Dual Channel Ultra320 SCSI RAID Adapter, you need the
following cables:
External cables (for attaching external storage devices or subsystems)
interposer cable, if required for external subsystem connections
Internal cables (for attaching internal hard disk drives or backplanes)

Using AIX

addition to the items listed above, this section includes AIX-specific items needed when preparing for
installation.
AIX 5.1 Update CD-ROM (Recommended Maintenance Package 5100-04 or higher) AIX 5.2 Update CD-ROM (Recommended Maintenance Package 5200-01 or higher) PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323

Using Linux

addition to the items listed above, this section includes Linux-specific items needed when preparing for
installation.
Linux Operating System CD-ROMs PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327

Preparing for a PCI Slot-Length Installation

prepare the PCI-X Dual Channel Ultra320 SCSI RAID Adapter for a PCI slot-length installation, do the
following:
Remove the two screws that hold the handle to the adapter. Refer to Figure 1 on page 1. Remove the handle from the adapter.
3
If
to
If
A
To
4

Determining Your Cable Requirements

You must use the correct SCSI cables to properly attach SCSI hard disk drives and subsystems.
you install an Ultra320 SCSI hard disk drive or subsystem that can transfer data at up to 320 MB/s, the
maximum cable length is limited to 20 meters (approximately 60 feet). For more information about connecting SCSI hard disk drives to the PCI-X Dual Channel Ultra320 SCSI
RAID Adapter, see Appendix C, “Connecting SCSI Hard Disk Drives to the Adapter,” on page 35 and refer
the PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323 or PCI-X SCSI
RAID Controller Reference Guide for Linux, order number SA23-1327.

Internal Cabling

you plan to install internal SCSI hard disk drives, you must obtain the necessary cables for your system unit. Refer to your system unit documentation for information about installing internal SCSI backplanes, hard disk drives, and cables.
SCSI terminator must be located at the end of the internal cable. Terminators are built into the backplane within the computer.

External Cabling

External SCSI hard disk drives and subsystems have a variety of connectors. Ensure that you have cables with the correct connectors for each planned external SCSI attachment.
externally attach hard disk drives and subsystems to the PCI-X Dual Channel Ultra320 SCSI RAID Adapter, you may need a 68-pin SCSI Very High Density Connector Interface to standard 68-pin SCSI standard connector interposer cable. In most cases, the cabling is delivered with the attaching subsystem.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
To
1.
2. At
3.
4. To
5.
6.
v
OR v In
7.
8.
9. To
/
In
v
is
v
v
of

Chapter 3. Installing the Device Driver Software (AIX)

The instructions in this chapter pertain to the AIX operating system.
Note: If you are using the Linux operating system, see Chapter 7, “Installing and Setting Up Software for
the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux ),” on page 15 for information regarding installing the device driver software and configuring your system. If you have another operating system installed, refer to your operating system documentation.
install the device driver software, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
Insert the appropriate AIX 5.1 or 5.2 update CD-ROM into the media device.
start SMIT, type:
smitty install_all
Press Enter. The Install Additional Device Software window highlights the INPUT device/directory for software
option. Select or type the input device:
Press F4 to display the input device list. Select the name of the device (example: CD-ROM) that you are using and press Enter
the Entry Field, type the name of the input device you are using, and press Enter.
The INSTALL AND UPDATE FROM ALL AVAILABLE SOFTWARE window appears. Highlight the
SOFTWARE to install option.
Press F4 to display a list of the device software you can install.
display the Find window, type:
10.
the Find window, type:
devices.pci.14107802
Press Enter. The system finds and highlights this device software and displays output similar to the following example:
devices.pci.14107802
11.
Press F7 to select the device software, and press Enter.
12.
The INSTALL AND UPDATE FROM ALL AVAILABLE SOFTWARE window displays. Entry data fields
All
are automatically updated. Highlight the ACCEPT new license agreements? option. Ensure that this option is set to yes. Press Enter to accept the data.
13.
The ARE YOU SURE window displays. Press Enter to accept the data.
14.
The COMMAND STATUS window displays.
The term When
RUNNING
RUNNING
highlighted to indicate that the install and configure command is in progress.
changes to OK, scroll down to the bottom of the page and locate the Installation
Summary. After a successful installation,
SUCCESS
appears in the Result column of the summary at the bottom
the page.
Remove the installation media from the drive.
15.
16.
Press F10 to exit SMIT.
5
6
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
Be
To
1.
a.
b.
To
To
1.
2.
3.
4.

Chapter 4. Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX)

This chapter provides instructions for installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter and connecting your SCSI hard disk drives. Save this guide for future reference.
Note: If you are using Linux, see “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using
the ipr Device Driver” on page 15. If you have another operating system installed, refer to your operating system documentation for adapter installation information.
Attention
sure you have read “Handling Static Sensitive Devices” on page vii before handling your PCI-X
Dual Channel Ultra320 SCSI RAID Adapter.

Installing the Adapter

install the adapter, do the following:
Refer to the option installation documentation that shipped with your system unit to perform the following:
Some systems support the hot-plugging of adapters. Prepare your system to hot plug this adapter while system power is on. Shut down your system unit if your system does not support hot-plugging.
Install the PCI Adapter into your system unit.
2.
determine if any placement restrictions apply to your adapter, refer to the PCI Adapter Placement
Reference. Return here to complete the installation procedure.
Attention
Failure to install your PCI adapter into the correct slot can cause your adapter or system unit to function incorrectly.

Connecting Internal SCSI Backplanes and Devices

connect the internal hard-disk-drive backplanes or devices, do the following:
Stop the system and place it into the service position as described in the documentation provided with your system.
Set up and install your backplane or device with the power and signal connections as described in the documentation provided with your system.
The internal connectors for channel 0 and channel 1 are located on the top of the PCI-X Dual Channel Ultra320 SCSI RAID Adapter. These connectors are designed to fit one way only. Using the internal cable provided, connect one end of the cable to the internal connectors of the PCI-X Dual Channel Ultra320 SCSI RAID Adapter. Connect the other end of the cable to the appropriate backplane or device. The PCI-X Dual Channel Ultra320 SCSI RAID Adapter provides built-in terminators; therefore, terminators are not required at the end of the cable. Hard disk drives connect to the terminators that are built into the backplane.
Note: Do not attach any external subsystems to a channel if it is internally connected to a backplane.
Install all the internal SCSI (including non-RAID) devices by referring to the documentation provided with your device.
7
To
1.
on
2.
3.
1.
2.
3.
v
v
8
Important
After you connect the card, internal cables, and backplane, be sure to record which set of hard disk drives are connected to channels 0 and 1. The management software identifies the physical hard disk drives by channel and SCSI ID. To properly remove or add physical hard disk drives, you need the location of the hard disk drive that was removed from or added to the system.
5.
Return the system to the operating position and start the system according to the instructions in your system unit documentation.

Connecting External SCSI Storage Peripheral Devices

After you install the PCI-X Dual Channel Ultra320 SCSI RAID Adapter and reinstall the cover of your system unit, you can connect the external storage peripheral devices. This adapter has two 68-pin Very High Density (VHD) connectors. If you are using a regular 68-pin cable, you may need an interposer cable.
Note: Attachment cables are usually shipped with the attaching storage peripheral device.
connect external storage peripheral devices using the VHD interposer cable, do the following:
Connect the 68-pin VHD connector of the interposer cable to one of the external ports (channel 0 or 1)
this adapter. Use the fastening screws on the cable connector to securely attach the connector to
the adapter. Connect the other end of the VHD connector to the SCSI cable going to your external storage
peripheral device. Ensure that the external storage peripheral devices and system unit are powered on.
connect external storage peripheral devices using a 68-pin Very High Density Cable Interconnection
To connector, do the following:
Connect one end of the 68-pin VHD connector to one of the external ports (channel 0 or 1) on this adapter. Use the fastening screws on the cable connector to securely attach the connector to the adapter.
Connect the other end of the 68-pin VHD cable to your external storage peripheral device. Ensure that the external storage peripheral devices and system unit are powered on.
Notes:
The SCSI channels 0 and 1 should never be interconnected. Each channel must operate independently. When you operate with more than one adapter in a system, you cannot share external hard disk drives
between the adapters.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
v
v
To
1. If
2.
If
an
on
If no
If
To
1. If
2.
If

Chapter 5. Setting Up the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX)

This chapter provides AIX-specific hardware and software installation verification, adapter configuration, and problem-determination procedures.
Note: If you are using Linux, see “Verifying the Installation” on page 16. If you have another operating
system installed, refer to your operating system documentation for information regarding configuring your system.

Verifying the Installation

Verify the initial installation by using the following procedures:
Verifying AIX Software Installation (lslpp) Verifying Hardware Installation (lsdev)

Verifying AIX Software Installation

verify that the device driver software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter is
installed, do the following:
necessary, log in as root user.
Type:
lslpp -l devices.pci.14107802*
Press Enter.
the PCI-X Dual Channel Ultra320 SCSI RAID Adapter device driver software is installed, the following is
example of the data that displays on your screen:
devices.pci.14107802.diag 5.2.0.11 COMMITTED PCI-X Dual Channel Ultra320 SCSI RAID Adapter Diagnostics devices.pci.14107802.rte devices.pci.14107802.ucode 5.2.0.0
Note: The fileset number 5.2.0.11 displayed in the example might be different from the output displayed
your screen.
data displays on your screen, the PCI-X Dual Channel Ultra320 SCSI RAID Adapter device driver did not install. Return to Chapter 3, “Installing the Device Driver Software (AIX),” on page 5. Then continue with “Verifying Hardware Installation.”
you continue to experience problems, it may be necessary to call your system support organization.
5.2.0.11 COMMITTED PCI-X Dual Channel Ultra320 SCSI RAID Adapter Software COMMITTED PCI-X Dual Channel Ultra320 SCSI RAID Adapter Microcode

Verifying Hardware Installation

verify that your system unit recognizes the PCI-X Dual Channel Ultra320 SCSI RAID Adapter, do the
following:
necessary, log in as root user.
Type:
lsdev -Cs pci | grep sisioa
Press Enter.
the PCI-X Dual Channel Ultra320 SCSI RAID Adapter did install correctly, the following is an
example of the data that displays on your screen:
sisioa0
Available 14-08 PCI-X Dual Channel U320 SCSI RAID Adapter
9
a
be
To
1. At
2.
3.
4.
5.
10
If the message on your screen indicates your adapter is
Defined
instead of Available, shut down your machine. Check the PCI-X Dual Channel Ultra320 SCSI RAID Adapter to ensure it is installed correctly. Go to Chapter 4, “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX),” on page 7, return to this section and repeat steps 1 and 2.
Note: If the message on your screen indicates your adapter is
Defined
second time, it may be
necessary to contact your service representative.

Updating Hard Disk Drive and Adapter Microcode

Disk drives attached to this RAID adapter must contain the latest microcode level. The latest microcode level for your drive model and PCI-X Dual Channel Ultra320 SCSI RAID Adapter can
found at the Download pSeries and RS/6000 Microcode Updates Web site:
https://techsupport.services.ibm.com/server/mdownload/download.html. The drive microcode download you need should match the name of one of the drive model numbers
located under the DASD Microcode listing found on the Web site. The drive machine type and model of the drive you are updating can be identified by reading its VPD (Vital Product Data). Similarly, the name of the PCI-X Dual Channel Ultra320 SCSI RAID Adapter Microcode download you need can be found under the Adapter Microcode listing on the Web site.

Configuring the PCI-X Dual Channel Ultra320 SCSI RAID Adapter

The PCI-X Dual Channel Ultra320 SCSI RAID Adapter permits up to 320 MB/s operation with low voltage differential (LVD) devices on a wide bus. With single-end (SE) devices, it can run up to 40 MB/s. The maximum speed and bus width are automatically negotiated based on the characteristics of the attached device.
You can use SMIT to control speed and bus width for devices as described in the following sections.

Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s

The default maximum adapter speed is 320 MB/s for LVD mode and 40 MB/s for SE mode. If you are not successful in running at these speeds, you can reduce the adapter speed.
limit adapter speed, do the following:
the command prompt, type smit, and press Enter. Select Devices and press Enter. Select SCSI Adapter and press Enter. Select Change/Show Characteristics of a SCSI Adapter and press Enter. Select your SCSI RAID adapter physical bus from the list and press Enter.
Note:
After making all desired changes in the entry fields, press Enter.
This screen content varies based on the adapter that you select.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
no 6.
7.
8.
9.
If
If
1.
2.
3.
4.
5. At
at
If
If
Change / Show Characteristics of a PCI-X SCSI RAID Controller Bus
SCSI Adapter Description Status Location Adapter card SCSI ID Wide bus enabled
Maximum SCSI Bus Speed in MB/s
QAS (Quick Arbitration) Capability
scsi5 PCI-X U320 SCSI RAID> Available 14-08-00 [7] yes
320
Enable
Apply changes to DATABASE only
Select Maximum SCSI Bus Speed in MB/s. Press F4. Select the desired bus speed: 5, 10, 20, 40, 80, 160, or 320 MB/s, and press Enter. Press Enter again.
[Entry Fields]
10.
the change was successful, press F10 to exit SMIT.
you were not successful in using the procedure “Limit Adapter Speed Using the Maximum SCSI Bus
Speed in MB/s” on page 10, do the following:
Select Apply change to DATABASE only. Press F4 and select yes. Press Enter. Press F10 to exit SMIT.
this point, the adapter and its child devices must be reconfigured. To reboot your system, log in as
root user and type
shutdown -Fr
the command prompt.

Wide Bus Enabled

you are attaching an external device with an 8-bit SCSI interface cable, set Wide bus enabled to no.
Follow the steps in “Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s” on page 10.

Quick Arbitration and Select (QAS) Capability

The QAS capability is automatically controlled by the adapter.

Problem Determination

you encounter any problems with the adapter after the installation, refer to your system unit documentation for information about running diagnostics or other problem-determination procedures. AIX users should refer to the PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323.
Chapter 5. Setting Up the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX)
11
12
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
v To
v
v
v
v To
to
v
a
to
v
v If a
v
v To
to
v
a
v
v If a
be

Chapter 6. Using Disk Arrays

This chapter provides information about using the RAID-related features of this adapter. The following tasks are beyond the scope of this installation guide and are usually performed by system administrators.
use the RAID-related features of the hardware, disk arrays must be created first. AIX and Linux use and manage the disk array in the same way as it would a single non-RAID disk. For optimum performance, distribute the hard disk drives equally across the available channels.

Using AIX

For an overview of RAID and detailed procedures, refer to the PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323.
Disk drives previously installed in this system probably have 512 bytes/sector.
use the drives in disk arrays with this adapter, the disk drives must be prepared by formatting them
522 bytes/sector.
Disk drives in 512 bytes/sector format may continue to be used as standalone disks (that is, not part of
disk array) on this RAID adapter. Additionally, drives formatted to 522 bytes/sector may be reformatted
512 bytes/sector at a later time if desired for compatibility with previous adapters.
Formatting of drives to 522 bytes/sector and the creation of disk arrays may require that data previously written on the drives be saved and restored after the the disk arrays are created.
disk array is to be used as the boot device, it may be required to prepare the disks by booting from the Diagnostic CD and creating the disk array before installing AIX. You might want to perform this procedure when the original boot drive is to be used as part of a disk array.

Using Linux

For an overview of RAID and detailed procedures, refer to the PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327.
Disk drives previously installed in this system are probably JBOD drives. JBOD drives are formatted to 512 bytes/sector.
use the drives in disk arrays with this adapter, the disk drives must be prepared by formatting them
522 bytes/sector.
Disk drives in 512 bytes/sector format may continue to be used as standalone disks (that is, not part of
disk array) on this RAID adapter. Additionally, drives formatted to 522 bytes/sector for advanced function may be reformatted to 512 bytes/sector at a later time if desired for compatibility with previous adapters.
Formatting of drives to 522 bytes/sector for advanced function and the creation of disk arrays may require that data previously written on the drives be saved and restored after the disk arrays are created.
disk array is to be used as the boot device, it may be required to prepare the disks by booting into rescue mode from the Linux CD first. If your Linux distribution does not provide this feature, the Diagnostic CD may be used. You might want to perform this procedure when the original boot drive is to
used as part of a disk array.
13
14
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
If
To
1.
a.
b. To
c.
To

Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )

Previously, SuSE Linux Enterprise Server 8 SP3 used the Channel Ultra320 SCSI RAID adapter. This device driver has been replaced by the which is used on all other supported distributions. Information on using the retained in this chapter. If you are using the older
ibmsis
ibmsis
device driver to control the PCI-X Dual
ipr
device driver,
ibmsis
device driver has been
device driver, go to “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ibmsis Device Driver” on page 22 and follow the procedures shown to install and set up the PCI-X Dual Channel Ultra320 SCSI RAID Adapter.
Note:
you are using AIX, see Chapter 5, “Setting Up the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX),” on page 9. If you have another operating system installed, refer to your operating system documentation for information regarding installing the device driver software and configuring your system.

Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ipr Device Driver

Install your PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the instructions found in this section.
Note: If you are using AIX, follow the steps in Chapter 4, “Installing the PCI-X Dual Channel Ultra320
SCSI RAID Adapter (AIX),” on page 7.
Attention: Do not remove the PCI-X Dual Channel Ultra320 SCSI Adapter from its antistatic package at
this time.
install the adapter, do the following:
Refer to the option installation documentation that shipped with your system unit to perform the following:
Some systems support the hot-plugging of adapters. Prepare your system to hot plug this adapter while system power is on. Shut down your system unit if your system does not support hot-plugging.
Note:
The 2.4 Linux kernel does not support PCI hot-plugging for SCSI host adapters.
determine if any placement restrictions apply to your adapter, refer to the PCI Adapter
Placement Reference. Return here to complete the installation procedure.
Install the PCI Adapter into your system unit.
Attention
Failure to install your PCI adapter into the correct slot can cause your adapter or system unit to function incorrectly.

Connecting External or Internal Devices

connect external or internal devices, use the procedures in “Connecting Internal SCSI Backplanes and
Devices” on page 7 or “Connecting External SCSI Storage Peripheral Devices” on page 8.
Note: Attachment cables are usually shipped with the attaching subsystem or device.
15
v
v
v
v
of
1. If
2.
1. If
2.
3.
If
16

Verifying the Installation

Verify the initial installation by using the following procedures:
Verifying Linux Software Installation Verifying Hardware Installation

Verifying Linux Software Installation

Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter consists of a device driver and a set of utilities. This software is provided with the supported Linux distribution, and should have been installed along with the operating system.
Verify the initial software installation by using the following procedures:
Verifying Device Driver Version (ipr) Verifying Utility Version (iprconfig)
Verifying Device Driver Version
The PCI-X Dual Channel Ultra320 SCSI RAID Adapter requires Version 1.0.3 (or later) of the ipr device driver for 2.4 based kernels and version 2.0.9 (or later) for 2.6 based kernels. To verify the version number
ipr, do the following:
For 2.6 Linux kernels:
necessary, log in as root user.
Type:
modinfo ipr | grep version
Press Enter.
following is an example of the data that displays on your screen:
The
version:
2.0.9 DA9C6A0AA78C5D2B9D947A1
For 2.4 Linux kernels:
necessary, log in as root user.
Type:
modinfo ipr
Press Enter. Type:
strings filename | grep "ipr version"
where filename is the file name displayed after the modinfo command was typed in the previous step. Press Enter.
The following is an example of the data that displays on your screen:
ipr version=Ver. 1 Rev. 0.3 SMP
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your Linux distributor or download them from the IBM StorageIO Web site. The Web address is: http://www-124.ibm.com/storageio/ipr/.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
do
1. If
2.
If
If it is
1.
2.
3.
If
1.
2. At
3.
cd
4.
Verifying Utility Version
The PCI-X Dual Channel Ultra320 SCSI RAID Adapter requires Version 1 of the iprconfig utility for 2.4 based kernels and version 2.x.x for 2.6 based kernels. To verify the version number of the iprconfig utility,
the following:
necessary, log in as root user.
Type:
iprconfig --version
Press Enter.
The following is an example of the data that displays on your screen:
iprconfig: Ver. 1 Rev. 0.3 iprconfig: 2.0.10 (May 23, 2004)
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your Linux distributor or download them from the IBM StorageIO Web site. The Web address is: http://www-124.ibm.com/storageio/ipr/.

Installing the Downloaded Device Driver and Utility Software

Software required for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter is usually installed when the Linux operating system is installed. Software installation is not necessary unless verification has failed.
necessary to install software, obtain it from one of the following sources in the order listed: Your supported Linux operating system CD-ROM Your Linux distributor Online at the IBM StorageIO Web site: http://www-124.ibm.com/storageio/ipr/
downloading the needed software package, install the software by continuing with the instructions in
After this section. Then, return to “Verifying Linux Software Installation” on page 16, and follow the procedure to ensure that the installation was successful.
Installing Device Driver Software
The ipr device driver is available in the base 2.6 kernel as of version 2.6.7-rc1. Additional driver updates may be available on the IBM StorageIO Web site as mentioned above. These patches may be downloaded and installed like any other kernel patch. The ipr device driver is not in the base 2.4 kernel and must be downloaded from http://www-124.ibm.com/storageio/ipr/ in the form of a kernel patch if not available as part of your distribution.
you downloaded an ipr device driver patch, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
Uncompress and apply the patch by typing:
/usr/src/linux
wget http://www-124.ibm.com/storageio/ipr/patch-2.4.26-ipr-1.0.5.gz
gunzip patch-2.4.26-ipr-1.0.5.gz patch -p1 < patch-2.4.26-ipr-1.0.5
Press Enter. Compile and install the device driver. If you are compiling a 2.4 kernel, make sure IBM Power RAID
SCSI support is selected. If you are compiling a 2.6 kernel, make sure IBM Power Linux RAID adapter support is selected.
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )
17
To
If
If
1.
2. At
3.
4.
5.
6.
7. If
To
1. If
2.
3.
If
18
5.
load the module, type:
modprobe ipr
Press Enter.
Note:
you are running a RHEL 3 based distribution, add the
/etc/rc.d/rc.local
file to ensure the driver is loaded at boot time.
Installing the Utility Software
you downloaded the iprutils utility package, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
Install the following packages (if not already installed):
gcc ncurses-devel pciutils-devel glibc-devel
Rebuild the rpm package by typing:
rpmbuild –-rebuild –-target ppc iprutils-2.0.10-1.src.rpm
Press Enter. Install the compiled rpm package by typing:
rpm -Uvh iprutils-2.0.10-1.ppc.rpm
Restart the iprutils services by typing:
/etc/init.d/iprupdate restart /etc/init.d/iprdump restart
running a 2.6 kernel, restart the iprinit service by typing:
/etc/init.d/iprinit restart
modprobe ipr
line to the

Verifying Hardware Installation

verify that your system unit recognizes the PCI-X Dual Channel Ultra320 SCSI RAID Adapter, do the
following:
necessary, log in as root user.
Type:
iprconfig
Press Enter. Select the Display hardware status option and press Enter.
the PCI-X Dual Channel Ultra320 SCSI RAID Adapter installed correctly, data displays for the adapter
and each disk attached to it. The following is an example of the data that displays on your screen:
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
If
or if it is
to
to
Type option, press Enter.
Display Hardware Status
1=Display hardware resource information details
OPT Name
PCI/SCSI Location
Description
Status
--- ------ -------------------------- ------------------------- --------------­0000:02:04.0.2/ 0000:02:04.0.2/1:14:0 0000:02:04.0.2/1:6:0 0000:02:04.0.2/1:8:0 0000:02:04.0.2/1:9:0
sdt
0000:02:04.0.2/255:0:0 0000:02:04.0.2/1:10:0 0000:02:04.0.2/1:11:0 0000:02:04.0.2/1:12:0 0000:02:04.0.2/1:13:0
sdb
0000:02:04.0.2/255:0:2 0000:02:04.0.2/1:4:0
sdc
0000:02:04.0.2/255:0:3 0000:02:04.0.2/1:3:0
sdd
0000:02:04.0.2/255:0:4
PCI-X SCSI RAID Adapter Advanced Function Disk Advanced Function Disk Advanced Function Disk Advanced Function Disk RAID 5 Disk Array
RAID 5 Array Member RAID 5 Array Member RAID 5 Array Member RAID 5 Array Member
RAID 0 Disk Array
RAID 0 Array Member
RAID 0 Disk Array
RAID 0 Array Member
RAID 0 Disk Array
Operational Active Active Active Active Active Active Active Active Active Active Active Active Active Active
More...
e=Exit
the message on your screen indicates your newly installed card, shown in the example as
RAID
q=Cancel
Adapter, is
r=Refresh
t=Toggle
Not Operational
f=PageDn
instead of
b=PageUp
Operational
PCI-X SCSI
not listed, shut down your machine. Go “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ipr Device Driver” on page 15 ensure that the PCI-X Dual Channel Ultra320 SCSI RAID Adapter is installed correctly. Then return to
this section, and repeat this hardware installation procedure.
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )
19
If
1. If
2.
3.
If
If
4. If
v
v
be
To
1. At
20

Updating Hard Disk Drive and Adapter Microcode

Disk drives attached to this RAID adapter must contain the latest microcode level. Microcode updates for hard disk drives attached to the PCI-X Dual Channel Ultra320 SCSI RAID Adapter
are handled automatically by the iprupdate utility, which executes each time the system is booted, or manually by the iprconfig utility. The iprupdate utility will only automatically update microcode to adapters and devices if it is determined that they are not at the minimum supported level of microcode.
The latest microcode level for your drive model and PCI-X Dual Channel Ultra320 SCSI RAID Adapter may be shipped as part of the Linux distribution. If an update or replacement is needed, the latest microcode can be obtained from the pSeries microcode Web site: http://techsupport.services.ibm.com/server/mdownload/download.html.
you downloaded new microcode from the Web site, follow the instructions in the Readme on the Web
site to install it. If there are no instructions available, install it as follows:
necessary, log in as root user.
Install the rpm package by typing:
rpm -ivh –-ignoreos scsi.0051.rpm
Press Enter. Update the microcode based on your kernel type:
you are using a 2.4 based kernel, type:
iprupdate
Press Enter. Once the microcode update has completed, you are finished.
you are using a 2.6 based kernel, type:
iprconfig
Press Enter and go to the next step.
you are using a 2.6 based kernel, choose the Download microcode menu option and:
Select the adapters and devices you wish to update For each selected device, select the microcode image you wish to download

Configuring the PCI-X Dual Channel Ultra320 SCSI Adapter

The PCI-X Dual Channel Ultra320 SCSI Adapter permits up to 320 MB/s operation with LVD devices on a wide bus. With single-end (SE) devices, it can run up to 40 MB/s. The maximum speed and bus width are automatically negotiated based on the characteristics of the attached device.
You can use iprconfig to control speed and bus width for devices as described in the following sections.

Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s

The default maximum adapter speed is 320 MB/s for LVD mode and 40 MB/s for SE mode. If you are not successful running at these speeds, you can reduce the adapter speed by changing the Maximum Bus
Throughput.
Note: The device driver will default to 80 MB/s on the 2.4 kernel until the iprupdate utility runs and 160
MB/s on the 2.6 kernel until the iprinit utility is run. After the utilities complete, higher speeds will
attempted. These utilities are configured to run automatically each time the system is booted.
limit adapter speed, do the following:
the command prompt, type iprconfig, and press Enter.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
If
3.
4.
80
40
20
10
5
5.
6.
7.
8.
If
If
on
If
to
2.
using iprconfig version less that 2.0.0, select Work with configuration and press Enter. Select Work with SCSI bus configuration and press Enter. Select your adapter from the list by pressing 1 then press Enter.
Note: This screen content varies based on the adapter that you select.
Change SCSI Bus Configuration
Current Bus configurations are shown. To change setting hit "c" for options menu. Highlight desired option then hit Enter
c=Change Setting
Adapter Location: 000:01:01.0
BUS 0
Maximum Bus Throughput . . : 320 MB/s
BUS 1
Wide Enabled . . . . . . . : Yes
Maximum Bus Throughput . . : 320 MB/s
320 MB/s
*160 MB/s
MB/s MB/s MB/s MB/s MB/s
Press Enter to Continue
e=Exit
q=Cancel
Select Maximum Bus Throughput field under the desired bus and press c. Select the desired bus speed: 5, 10, 20, 40, 80, 160, or 320 MB/s, and press Enter.
Note:
Depending on your hardware, some of these options may not be available. Press Enter to continue. Press c to confirm the changes.

Wide Bus Enabled

you are attaching an external device with an 8-bit SCSI interface cable, set Wide bus enabled to no.
Follow the steps in “Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s” on page 20.

Quick Arbitration and Select (QAS) Capability

The QAS capability is automatically controlled by the adapter.

Setting Bus Speed at Boot

running a 2.6 kernel, a module parameter is provided to allow for setting the maximum bus speed at boot time before the iprinit utility runs. Normally this should not be needed, but if you are having difficulty booting due to running at 160 MB/s, bus speed can be lowered to 80 MB/s by using the kernel command line.
ipr.max_speed=0

Problem Determination

you encounter any problems with the adapter after the installation, refer to your system unit documentation for information about running diagnostics or other problem-determination procedures. Refer
the PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327.
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )
21
If
To
1.
a.
b. To
c.
To
22

Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ibmsis Device Driver

This chapter provides instructions for installing and verifying hardware and software that pertains to the Linux operating system using the “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ipr Device Driver” on page 15 and follow the procedures shown in that section.
ibmsis
Device Driver. If you are using the newer
ipr
device driver, go to
Note:
you are using AIX, see Chapter 5, “Setting Up the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (AIX),” on page 9. If you have another operating system installed, refer to your operating system documentation for information regarding installing the device driver software and configuring your system.

Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter

Install your PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the instructions found in this section.
Note: If you are using AIX, follow the steps in Chapter 4, “Installing the PCI-X Dual Channel Ultra320
SCSI RAID Adapter (AIX),” on page 7.
Attention: Do not remove the PCI-X Dual Channel Ultra320 SCSI Adapter from its antistatic package at
this time.
install the adapter, do the following:
Refer to the option installation documentation that shipped with your system unit to perform the following:
Some systems support the hot-plugging of adapters. Prepare your system to hot plug this adapter while system power is on. Shut down your system unit if your system does not support hot-plugging.
Note:
Placement Reference. Return here to complete the installation procedure.
Install the PCI Adapter into your system unit.
The 2.4 Linux kernel does not support PCI hot-plugging for SCSI host adapters.
determine if any placement restrictions apply to your adapter, refer to the PCI Adapter
Attention
Failure to install your PCI adapter into the correct slot can cause your adapter or system unit to function incorrectly.
Connecting External or Internal Devices
connect external or internal devices, use the procedures in “Connecting Internal SCSI Backplanes and
Devices” on page 7 or “Connecting External SCSI Storage Peripheral Devices” on page 8.
Note: Attachment cables are usually shipped with the attaching subsystem or device.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
v
v
v
v
1. If
2.
3.
If
1. If
2.

Verifying the Installation

Verify the initial installation by using the following procedures:
Verifying Linux Software Installation Verifying Hardware Installation
Verifying Linux Software Installation
Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter consists of a device driver and a set of utilities. This software is provided with the supported Linux distribution, and should have been installed along with the operating system.
Verify the initial software installation by using the following procedures:
Verifying Device Driver Version (ibmsis) Verifying Utility Version (sisconfig)
Verifying
Device Driver Version:
The PCI-X Dual Channel Ultra320 SCSI RAID Adapter requires Version 1 Revision 19 (or later) of the ibmsis device driver. To verify the version number of ibmsis, do the following:
necessary, log in as root user.
Type:
modinfo ibmsis
Press Enter. Type:
strings filename | grep "ibmsis version"
where filename is the file name displayed after the modinfo command was typed in the previous step. Press Enter.
The following is an example of the data that displays on your screen:
ibmsis version=Ver. 1 Rev. 19.10 SMP
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your Linux distributor or download them from the IBM Support Web site. The Web address is: http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p.
Verifying Utility Version:
The PCI-X Dual Channel Ultra320 SCSI RAID Adapter requires Version 1 Revision 19 (or later) of the sisconfig utility. To verify the version number of the sisconfig utility, do the following:
necessary, log in as root user.
Type:
sisconfig --version
Press Enter.
The following is an example of the data that displays on your screen:
sisconfig: Ver. 1 Rev. 19.10
Note: The version displayed in the example might be different from the output displayed on your screen.
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )
23
If it is
1.
2.
3.
If
1.
2. At
3. If
4.
5. To
If
If
1.
2. At
3. If
4.
If
To
24
If the version displayed is not at the recommended level and updates are needed, obtain them from your Linux distributor or download them from the IBM Support Web site. The Web address is: http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p.
Installing the Downloaded Device Driver and Utility Software
Software required for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter is usually installed when the Linux operating system is installed. Software installation is not necessary unless verification has failed.
necessary to install software, obtain it from one of the following sources in the order listed: Your supported Linux operating system CD-ROM Your Linux distributor Online at the IBM Support Web site: http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p
downloading the needed software package, install the software by continuing with the instructions in
After this section. Then, return to “Verifying Linux Software Installation” on page 23, and follow the procedure to ensure that the installation was successful.
Installing Device Driver Software:
you downloaded the ibmsis device driver package, do the
following:
Turn on the power to your system unit.
the system prompt, log in as root user.
necessary, untar the device driver package. Type:
tar -xzvf ibmsis-$ver.tgz
Press Enter. Compile and install the device driver. For instructions on how to compile and install the ibmsis device
driver, see the Readme file located in ibmsis/src/ directory.
load the module, type:
modprobe ibmsis
Press Enter.
Note:
Installing
you are running a United Linux based distribution (such as SuSE) add the following line to the
file /etc/rc.d/boot.local to ensure the driver is loaded at boot time:
the Utility Software:
you downloaded the sisutils utility package, do the following:
modprobe ibmsis
Turn on the power to your system unit.
the system prompt, log in as root user.
necessary, untar the utility package. Type:
tar -xzvf sisutils-$ver.tgz
Press Enter. Compile and install the sisutils utility package. For instructions on how to compile and install the utility
package, see the readme file located in the sisutils/ directory.
Note:
you are running a United Linux based distribution (such as SuSE), add the following lines to
the file /etc/rc.d/boot.local to ensure the driver is loaded at boot time:
sisdump & sisupdate
These lines should be added to the /etc/rc.d/boot.localfile after the line
modprobe
ibmsis.
Verifying Hardware Installation
verify that your system unit recognizes the PCI-X Dual Channel Ultra320 SCSI RAID Adapter, do the
following:
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
If
2.
3.
If
ID ID
If
is
or if it is
1.
necessary, log in as root user.
Type:
sisconfig
Press Enter. Select the Display disk hardware status option and press Enter.
the PCI-X Dual Channel Ultra320 SCSI RAID Adapter installed correctly, data displays for the adapter
and each disk attached to it. The following is an example of the data that displays on your screen:
Display Disk Hardware Status
Serial
Number 03000001 IBM 000012F0 IBM 00001315 IBM 000012F5 IBM 00001304 IBM E7800377 IBM
Vendor Product
5703001 ST3146807LC 090 ST3146807LC 079 ST3146807LC 070 ST3146807LC 079 01C933F1
Resource
Model Name
001
/dev/ibmsis0 Operational
205
/dev/sdc
Hardware Status
HS/Active DPY/Active DPY/Active DPY/Active Operational
Press Enter to continue.
e=Exit d=Display disk unit details
q=Cancel
r=Refresh
f=PageDn
b=PageUp
p=Display device parity status
the message on your screen indicates your newly installed card, shown in the example as /dev/ibmsis0,
Not Operational
instead of
Operational
not listed, shut down your machine. Go to “Installing the PCI-X Dual Channel Ultra320 SCSI RAID Adapter using the ipr Device Driver” on page 15 to ensure that the PCI-X Dual Channel Ultra320 SCSI RAID Adapter is installed correctly. Then return to this section, and repeat this hardware installation procedure.
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )
25
be
If
1. If
2.
3.
cd
4.
5.
To
1. At
2.
3.
4.
26

Updating Hard Disk Drive and Adapter Microcode

Disk drives attached to this RAID adapter must contain the latest microcode level. Microcode updates for hard disk drives attached to the PCI-X Dual Channel Ultra320 SCSI RAID Adapter
are handled automatically by the sisupdate utility, which executes each time the system is booted. The latest microcode level for your drive model and PCI-X Dual Channel Ultra320 SCSI RAID Adapter is
shipped as part of the Linux distribution. If an update or replacement is needed, the latest microcode can
obtained from your Linux distributor, or found as part of the device driver package at the following Web
site: http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p.
you downloaded new microcode from the Web site, install it as follows:
necessary, log in as root user.
Untar the package, type:
tar –xzvf ibmsis-$ver.tgz
Press Enter. Change directories to ibmsis/obj/firmware/. Type:
ibmsis/obj/firmware/
Install the microcode. Type:
make install
Update the microcode, type:
sisupdate
Press Enter.

Configuring the PCI-X Dual Channel Ultra320 SCSI Adapter

The PCI-X Dual Channel Ultra320 SCSI Adapter permits up to 320 MB/s operation with LVD devices on a wide bus. With single-end (SE) devices, it can run up to 40 MB/s. The maximum speed and bus width are automatically negotiated based on the characteristics of the attached device.
You can use sisconfig to control speed and bus width for devices as described in the following sections.
Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s
The default maximum adapter speed is 320 MB/s for LVD mode and 40 MB/s for SE mode. If you are not successful running at these speeds, you can reduce the adapter speed by changing the Maximum Bus
Throughput.
Note: The device driver will default to 80 MB/s until the sisconfig utility is run. After the sisconfig utility
completes, higher speeds will be attempted. The sisconfig utility is configured to run automatically each time the system is booted.
limit adapter speed, do the following:
the command prompt, type sisconfig, and press Enter. Select Work with configuration and press Enter. Select Work with SCSI bus configuration and press Enter. Select your adapter from the list by pressing 1 then press Enter.
Note: This screen content varies based on the adapter that you select.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
80
40
20
10
5
5.
6.
7.
8.
9.
If
If
to
Change SCSI Bus Configuration
Current Bus configurations are shown. To change setting hit "c" for options menu. Highlight desired option then hit Enter
c=Change Setting
/dev/ibmsis0 BUS 0
Maximum Bus Throughput . . : 320 MB/s
BUS 1
Wide Enabled . . . . . . . : Yes
Maximum Bus Throughput . . : 320 MB/s
Press Enter to Continue
320 MB/s
*160 MB/s
MB/s MB/s MB/s MB/s MB/s
e=Exit
q=Cancel
Select Maximum Bus Throughput field and under the desired bus and press c. Press F4. Select the desired bus speed: 5, 10, 20, 40, 80, 160, or 320 MB/s, and press Enter.
Note:
Depending on your hardware, some of these options may not be available. Press Enter to continue. Press c to confirm the changes.
Wide Bus Enabled
you are attaching an external device with an 8-bit SCSI interface cable, set Wide bus enabled to no.
Follow the steps in “Limit Adapter Speed Using the Maximum SCSI Bus Speed in MB/s” on page 20.
Quick Arbitration and Select (QAS) Capability
The QAS capability is automatically controlled by the adapter.

Problem Determination

you encounter any problems with the adapter after the installation, refer to your system unit
documentation for information about running diagnostics or other problem-determination procedures. Refer
the PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327.
Chapter 7. Installing and Setting Up Software for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter (Linux )
27
28
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide

Appendix A. Communications Statements

The following statement applies to this product. The statement for other products intended for use with this product appears in their accompanying documentation.

Federal Communications Commission (FCC) Statement

Note: This equipment has been tested and found to comply with the limits for a Class A digital device,
pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense.
Properly shielded and grounded cables and connectors must be used in order to meet FCC emission limits. Neither the provider nor the manufacturer is responsible for any radio or television interference caused by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment. Unauthorized changes or modifications could void the user’s authority to operate the equipment.
This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation.

European Union (EU) Statement

This product is in conformity with the protection requirements of EU Council Directive 89/336/EEC on the approximation of the laws of the Member States relating to electromagnetic compatibility. The manufacturer cannot accept responsibility for any failure to satisfy the protection requirements resulting from a non-recommended modification of the product, including the fitting of option cards supplied by third parties. Consult with your dealer or sales representative for details on your specific hardware.
This product has been tested and found to comply with the limits for Class A Information Technology Equipment according to CISPR 22 / European Standard EN 55022. The limits for Class A equipment were derived for commercial and industrial environments to provide reasonable protection against interference with licensed communication equipment.
Attention: This is a Class A product. In a domestic environment this product may cause radio
interference in which case the user may be required to take adequate measures.

International Electrotechnical Commission (IEC) Statement

This product has been designed and built to comply with IEC 60950.

United Kingdom Telecommunications Safety Requirements

This equipment is manufactured to the International Safety Standard EN60950 and as such is approved in the UK under the General Approval Number NS/G/1234/J/100003 for indirect connection to the public telecommunication network.
The network adapter interfaces housed within this equipment are approved separately, each one having its own independent approval number. These interface adapters, supplied by the manufacturer, do not use or contain excessive voltages. An excessive voltage is one which exceeds 70.7 V peak ac or 120 V dc. They
29
a
in
30
interface with this equipment using Safe Extra Low Voltages only. In order to maintain the separate (independent) approval of the manufacturer’s adapters, it is essential that other optional cards, not supplied by the manufacturer, do not use main voltages or any other excessive voltages. Seek advice from
competent engineer before installing other adapters not supplied by the manufacturer.

Avis de conformité aux normes du ministère des Communications du Canada

Cet appareil numérique de la classe A est conforme à la norme NMB-003 du Canada.

Canadian Department of Communications Compliance Statement

This Class A digital apparatus complies with Canadian ICES-003..

VCCI Statement

The following is a summary of the VCCI Japanese statement in the box above. This is a Class A product based on the standard of the Voluntary Control Council for Interference by
Information Technology Equipment (VCCI). If this equipment is used in a domestic environment, radio disturbance may arise. When such trouble occurs, the user may be required to take corrective actions.

Electromagnetic Interference (EMI) Statement - Taiwan

The following is a summary of the EMI Taiwan statement above. Warning: This is a Class A product. In a domestic environment this product may cause radio interference
which case the user will be required to take adequate measures.

Radio Protection for Germany

Dieses Gerät ist berechtigt in Übereinstimmung mit Dem deutschen EMVG vom 9.Nov.92 das EG–Konformitätszeichen zu führen.
Der Aussteller der Konformitätserklärung ist die IBM Germany. Dieses Gerät erfüllt die Bedingungen der EN 55022 Klasse A. Für diese von Geräten gilt folgende
Bestimmung nach dem EMVG:
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
Geräte dürfen an Orten, für die sie nicht ausreichend entstört sind, nur mit besonderer Genehmigung des Bundesministers für Post und Telekommunikation oder des Bundesamtes für Post und Telekommunikation betrieben werden. Die Genehmigung wird erteilt, wenn keine elektromagnetischen Störungen zu erwarten sind.
(Auszug aus dem EMVG vom 9.Nov.92, Para.3, Abs.4) Hinweis Dieses Genehmigungsverfahren ist von der Deutschen Bundespost noch nicht veröffentlicht worden.
Appendix A. Communications Statements
31
32
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
in

Appendix B. Notices

This information was developed for products and services offered in the U.S.A. The manufacturer may not offer the products, services, or features discussed in this document in other
countries. Consult the manufacturer’s representative for information on the products and services currently available in your area. Any reference to the manufacturer’s product, program, or service is not intended to state or imply that only that product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any intellectual property right of the manufacturer may be used instead. However, it is the user’s responsibility to evaluate and verify the operation of any product, program, or service.
The manufacturer may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the manufacturer.
The following paragraph does not apply to the United Kingdom or any country where such provisions are inconsistent with local law: THIS MANUAL IS PROVIDED AS IS WITHOUT
WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. The manufacturer may make improvements and/or changes in the product(s) and/or the program(s) described
this publication at any time without notice.
Information concerning products made by other than the manufacturer was obtained from the suppliers of those products, their published announcements, or other publicly available sources. The manufacturer has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to products made by other than the manufacturer. Questions on the capabilities of products made by other than the manufacturer should be addressed to the suppliers of those products.
33
34
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
8, 9, A, B, C, D, E, or F.
ID
1
2
3
4
5

Appendix C. Connecting SCSI Hard Disk Drives to the Adapter

The RAID adapter connects to a hard disk drive through one of two possible SCSI channels. The connected hard disk drives can be inside the host system unit, outside the system unit in a hard disk drive enclosure, or both. Each hard disk drive on a SCSI channel must have a unique SCSI ID of 0 through 6,
The addressing scheme is in hex notation. Address 7 is reserved for the adapter and is not to be used for another SCSI hard disk drive. Each of the SCSI channels (0 and 1) is assigned a channel ID. The channel
and SCSI ID are used to fully identify a hard disk drive connected to the adapter. For optimum overall
performance and data protection, distribute hard disk drives evenly across the two available channels.
Figure 2. PCI-X Dual Channel Ultra320 SCSI RAID Adapter
Channel 1 external, 68-position, VHDCI (very high density cable interconnect) Channel 0 external, 68-position, VHDCI (very high density cable interconnect) Channel 0 internal, 68-position, high-density plastic D-shell SCSI standard connector for 16-bit
attachment Channel 1 internal, 68-position, high-density plastic D-shell SCSI standard connector for 16-bit
attachment Adapter Handle
35
to
To
1.
2.
36

Determining Current SCSI Hard Disk Drive Addresses

Hard disk drive backplanes or external subsystems automatically set the SCSI address for you. Refer to the documentation for your system or external subsystem for their locations. For future reference, be sure
keep a list of all the SCSI addresses you use and the channels to which they are connected.
determine the SCSI addresses of any installed hard disk drives, start the RAID manager program and
list the attached physical drives. For an AIX system, see PCI-X SCSI RAID Controller Reference Guide for
AIX, order number SA23-1323. For a Linux system, see PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327. If you have another operating system installed, refer to your operating
system documentation for the correct procedures.
Notes:
The priority of SCSI addresses for the PCI-X Dual Channel Ultra320 SCSI RAID Adapter is not in sequential order. The order of priority is 7 through 0 and 15 through 8.
Address 7 is reserved for the adapter.
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
1
2
3
4
1
2
3
4

Appendix D. Connection Examples

This appendix provides examples of the different ways that you can connect SCSI hard disk drives to the PCI-X Dual Channel Ultra320 SCSI RAID Adapter.

Internal Hard Disk Drive Connection

Figure 3. Internal Hard Disk Drive Connection
SCSI Hot Pluggable Backplane System-Unique Internal Cable Channel 1 68-pin SCSI Standard Internal Connector Channel 0 68-pin SCSI Standard Internal Connector

Internal and Single External Storage Peripheral Connections

Figure 4. Internal and Single External Storage Connections
SCSI Hot Pluggable Backplane System-Unique Internal Cable Channel 1. 68-pin SCSI Standard Internal Connector Channel 0. 68-pin SCSI Standard Internal Connector (VHDCI)
37
6
7
8
9
10
11
38
5
Channel 1. 68-pin SCSI Very High Density Connector Interface External Connector (VHDCI) Channel 0. 68-pin SCSI Very High Density Connector Interface External Connector (VHDCI) Interposer Cable FC 2118 (76H0518) for use only when cables with high-density connectors are
available Cable with two VHDCI connectors, which attaches the PCI-X Dual Channel Ultra320 SCSI RAID
Adapter directly to the attaching subsystem enclosure. This cable is usually supplied with the subsystem enclosure.
Cable with two High Density Connectors, which is used with Interposer Cable FC 2118 (76H0518) External Subsystem Enclosure PCI-X Dual Channel Ultra320 SCSI RAID Adapter
PCI-X Dual Channel Ultra320 SCSI RAID Adapter: Installation and Using Guide
h
h No
Readers’ Comments We’d Like to Hear from You
PCI-X Dual Channel Ultra320 SCSI RAID Adapter Installation and Using Guide
Publication No. SA23-1324-01
Overall, how satisfied are you with the information in this book?
Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied
Overall satisfaction hhhhh
How satisfied are you that the information in this book is:
Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied Accurate hhhhh Complete hhhhh Easy to find hhhhh Easy to understand hhhhh Well organized hhhhh Applicable to your tasks hhhhh
Please tell us how we can improve this book:
Thank you for your responses. May we contact you?
Yes
When you send comments to IBM, you grant IBM a nonexclusive right to use or distribute your comments in any way it believes appropriate without incurring any obligation to you.
Name
Address
Company or Organization
Phone No.
or
___________________________________________________________________________________________________
Readers’ Comments We’d Like to Hear from You
SA23-1324-01
Fold
_________________________________________________________________________________________
and Tape
Please do not staple
Fold and Tape
NO
POSTAGE
NECESSARY
MAILED IN THE
IF UNITED STATES
Cut or Fold Along Line
BUSINESS REPLY MAIL
FIRST-CLASS MAIL PERMIT NO. 40 ARMONK, NEW YORK
POSTAGE WILL BE PAID BY ADDRESSEE
Information Development Department H6DS-9561 11501 Burnet Road Austin, TX 78758-3493
_________________________________________________________________________________________
Fold
and Tape
Please do not staple
Fold and Tape
SA23-1324-01
Fold
Cut Along Line
Part Number: 80P5517
Printed in USA
SA23-1324-01
80P5517
P/N:
(1P)
Loading...