Dual Channel SCSI RAID Enableme nt Card
Installation an d Using Guid e
SA23-1325-01
Dual Channel SCSI RAID Enableme nt Card
Installation an d Using Guid e
SA23-1325-01
A
©
Note
Before using this information and the product it supports, be sure to read the general information
under ″Product Warranties and Notices″ included with your system unit.
Second Edition (August 2004)
Before using this information and the product it supports, read the information in “Safety Information” on page v and
Appendix B, “Notices,” on page 27.
reader’s comment form is provided at the back of this publication. If the form has been removed, address
comments to Publications Department, Internal Zip 9561, 11501 Burnet Road, Austin, Texas 78758-3493. To send
comments electronically, use this commercial internet address: aix6kpub@austin.ibm.com. Any information that you
supply may be used without incurring any obligation to you.
International Business Machines Corporation 2003. 2004. All rights reserved.
Note to U.S. Government Users Restricted Rights--Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
Contents
Safety Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v
Handling Static Sensitive Devices . . . . . . . . . . . . . . . . . . . . . . . . . vii
About This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
ISO 9000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
Highlighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
References to AIX and Linux Operating Systems . . . . . . . . . . . . . . . . . . . . .ix
Related Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x
Chapter 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Verifying Operating System Requirements . . . . . . . . . . . . . . . . . . . . . . . .2
Using AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Using Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Chapter 2. Preparing for Installation . . . . . . . . . . . . . . . . . . . . . . . . .3
Using AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Using Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Chapter 3. Installing the Device Driver Software (AIX) . . . . . . . . . . . . . . . . . .5
Chapter 4. Installing the Dual Channel SCSI RAID Enablement Card . . . . . . . . . . . .7
Chapter 5. Setting Up the Dual Channel SCSI RAID Enablement Card (AIX) . . . . . . . . . .9
Verifying the Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Verifying AIX Software Installation . . . . . . . . . . . . . . . . . . . . . . . . . .9
Verifying Hardware Installation . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Updating Hard Disk Drive and Adapter Microcode . . . . . . . . . . . . . . . . . . .10
Problem Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Chapter 6. Using Disk Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Using AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Using Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Chapter 7. Installing and Setting Up Software for the Dual Channel SCSI RAID Enablement Card
(Linux) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Installing the Dual Channel SCSI RAID Enablement Card using the ipr Device Driver . . . . . . .13
Verifying Linux Software Installation . . . . . . . . . . . . . . . . . . . . . . . . .13
Installing the Downloaded Device Driver and Utility Software . . . . . . . . . . . . . . .15
Installing the Dual Channel SCSI RAID Enablement Card . . . . . . . . . . . . . . . .16
Verifying Hardware Installation . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Updating Hard Disk Drive and Adapter Microcode . . . . . . . . . . . . . . . . . . .18
Problem Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Installing the Dual Channel SCSI RAID Enablement Card using the ibmsis Device Driver . . . . . .18
Verifying Linux Software Installation . . . . . . . . . . . . . . . . . . . . . . . . .19
Installing the Downloaded Device Driver and Utility Software . . . . . . . . . . . . . . .20
Installing the Dual Channel SCSI RAID Enablement Card . . . . . . . . . . . . . . . .21
Verifying Hardware Installation . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Updating Hard Disk Drive and Adapter Microcode . . . . . . . . . . . . . . . . . . .22
Problem Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
Appendix A. Communications Statements . . . . . . . . . . . . . . . . . . . . . .23
iii
iv
Federal Communications Commission (FCC) Statement . . . . . . . . . . . . . . . . . .23
European Union (EU) Statement . . . . . . . . . . . . . . . . . . . . . . . . . . .23
International Electrotechnical Commission (IEC) Statement . . . . . . . . . . . . . . . . .23
United Kingdom Telecommunications Safety Requirements . . . . . . . . . . . . . . . . .23
Avis de conformité aux normes du ministère des Communications du Canada . . . . . . . . . .24
Canadian Department of Communications Compliance Statement . . . . . . . . . . . . . .24
VCCI Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Electromagnetic Interference (EMI) Statement - Taiwan . . . . . . . . . . . . . . . . . .24
Radio Protection for Germany . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Appendix B. Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
Safety Information
DANGER
electrical outlet that is not correctly wired could place hazardous voltage on metal parts of
An
the system or the devices that attach to the system. It is the responsibility of the customer to
ensure that the outlet is correctly wired and grounded to prevent an electrical shock.
Before installing or removing signal cables, ensure that the power cables for the system unit
and all attached devices are unplugged.
When adding or removing any additional devices to or from the system, ensure that the power
cables for those devices are unplugged before the signal cables are connected. If possible,
disconnect all power cables from the existing system before you add a device.
Use one hand, when possible, to connect or disconnect signal cables to prevent a possible
shock from touching two surfaces with different electrical potentials.
During an electrical storm, do not connect cables for display stations, printers, telephones, or
station protectors for communication lines.
v
vi
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
v
v
v Do
v Do
v
v
it is
v
Handling Static Sensitive Devices
Attention: Static electricity can damage this device and your system unit. To avoid damage, keep this
device in its anti-static protective bag until you are ready to install it. To reduce the possibility of
electrostatic discharge, follow the precautions listed below:
Limit your movement. Movement can cause static electricity to build up around you.
Handle the device carefully, holding it by its edges or its frame.
not touch solder joints, pins, or exposed printed circuitry.
not leave the device where others can handle and possibly damage the device.
While the device is still in its anti-static package, touch it to an unpainted metal part of the system unit
for at least two seconds. (This drains static electricity from the package and from your body.)
Remove the device from its package and install it directly into your system unit without setting it down. If
necessary to set the device down, place it on its static-protective package. (If your device is an
adapter, place it component-side up.) Do not place the device on your system unit cover or on a metal
table.
Take additional care when handling devices during cold weather, as heating reduces indoor humidity
and increases static electricity.
vii
viii
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
v
v
If
v
About This Book
This book provides information about installing the Dual Channel SCSI RAID Enablement Card and device
driver support software, as well as cabling requirements. Use this book in conjunction with your specific
system unit and operating system documentation.
ISO 9000
ISO 9000 registered quality systems were used in the development and manufacturing of this product.
Highlighting
The following highlighting conventions are used in this book:
Bold
Identifies commands, subroutines, keywords, files, structures, directories, and other items
whose names are predefined by the system. Also identifies graphical objects such as buttons,
labels, and icons that the user selects.
Italics
Monospace
Identifies parameters whose actual names or values are to be supplied by the user.
Identifies examples of specific data values, examples of text similar to what you might see
displayed, examples of portions of program code similar to what you might write as a
programmer, messages from the system, or information you should actually type.
References to AIX and Linux Operating Systems
This document contains references to the AIX
operating system, consult the appropriate documentation for that operating system.
This document may describe hardware features and functions. While the hardware supports them, the
realization of these features and functions depends upon support from the operating system. AIX and
Linux provide this support. If you are using another operating system, consult the appropriate
documentation for that operating system regarding support for those features and functions.
®
and Linux operating system. If you are using another
Related Publications
The following publications contain related information:
System unit documentation for information specific to your hardware configuration.
Operating system documentation for information specific to your software configuration and disk arrays.
your operating system is AIX, refer to the PCI-X SCSI RAID Controller Reference Guide for AIX, order
number SA23-1323. If your operating system is Linux, refer to the PCI-X SCSI RAID Controller
Reference Guide for Linux, order number SA23-1327.
PCI Adapter Placement Reference, order number SA38-0538 (for the latest version, you may need to
contact your marketing representative).
ix
v
v
v
v
v
x
Trademarks
The following terms are trademarks of International Business Machines Corporation in the United States,
other countries, or both:
AIX
AIX 5L
Eserver
pSeries
RS/6000
Other company, product, and service names may be trademarks or service marks of others.
™
™
®
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
or
Chapter 1. Overview
The Dual Channel SCSI RAID Enablement Card adds RAID 0, 5, or 10 disk array capability to systems
containing the appropriate supporting dual channel integrated SCSI adapter. It provides an internal RAID
(Redundant Array of Independent Disks) solution on supporting systems with internal multiple disk drives
packs of drives. Internally attached Ultra320 devices can run at a data rate up to 320 MB/s on systems
that have internal backplanes capable of running Ultra320 speeds.
Note: This guide refers to various hardware and software features and functions. The support for these
features and functions depends upon the limitations of your hardware and software. AIX and Linux
support all functions mentioned. If you are using another operating system, consult the appropriate
documentation for that operating system regarding support for the mentioned features and
functions.
Figure 1. Dual Channel SCSI RAID Enablement Card
1
v
v
No
If
v
v
v
2
Verifying Operating System Requirements
Ensure that your operating system will support the Dual Channel SCSI RAID Enablement Card, use the
following appropriate procedure to verify your operating system requirements.
Using AIX
The Dual Channel SCSI RAID Enablement Card is supported on the following levels of AIX or higher:
AIX 5L Version 5.1 with the 5100-04 Recommended Maintenance Package (APAR IY44478)
AIX 5L Version 5.2 with the 5200-01 Recommended Maintenance Package (APAR IY44479)
determine which maintenance level is installed on your system, type
To
oslevel
-r. Verify that the
maintenance level displayed is 5100-04, 5200-01, or higher. If no maintenance level is installed on your
system, the following is an example of the data that displays on your screen:
recommended maintenance level information is available.
updates are needed, the latest maintenance packages and update instructions can be downloaded from
the pSeries and RS/6000 Support Web site. The Web address is:
http://techsupport.services.ibm.com/server/support. Updates can also be ordered by calling IBM Support.
Using Linux
The Dual Channel SCSI RAID Enablement Card is supported on the following Linux distributions:
Any distribution based on United Linux 1.0 Service Pack 3, such as SuSE Linux Enterprise Server 8
Service Pack 3 or newer
Red Hat Enterprise Linux 3 or newer
SuSE Linux Enterprise Server 9 or newer
determine which release is installed on your system, refer to your Linux distributor’s documentation. If
To
updates are needed, obtain them from your Linux distributor or download them from the IBM StorageIO
Web Site. The Web address is: http://www-124.ibm.com/storageio/ipr/.
Note:
SuSE Linux Enterprise Server 8 SP3 used the
Channel RAID Enablement Card. This device driver has been replaced by the
which is used on all other supported distributions. Information on using the older
driver is contained in “Installing the Dual Channel SCSI RAID Enablement Card using the ibmsis
Device Driver” on page 18. Refer to this section if you are still using this device driver.
ibmsis
device driver to control the PCI-X Dual
ipr
device driver,
ibmsis
device
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
v
v
v
In
v
v
v
In
v
v
Chapter 2. Preparing for Installation
This chapter helps you prepare to install the Dual Channel SCSI RAID Enablement Card. To install the
Dual Channel SCSI RAID Enablement Card, you may need the following items:
Your system unit installation guide
Documentation provided with your SCSI devices
IBM
Eserver
pSeries & RS/6000 Standalone Diagnostics CD-ROM (Version 5.2.0.11 or higher)
Note:
This card is supported on multiple systems and may require a system firmware update. Contact
your service representative for further details.
Using AIX
addition to the preceding items listed, you will need the following AIX-specific items:
AIX 5.1 Update CD-ROM (Recommended Maintenance Package 5100-04 or higher)
AIX 5.2 Update CD-ROM (Recommended Maintenance Package 5200-01 or higher)
PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323
Using Linux
addition to the preceding items listed, you will need the following Linux-specific items:
Linux Operating System CD-ROMs
PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327
3
4
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
To
1.
2. At
3.
4. To
5.
6.
v
OR v In
7.
8.
9. To
/
In
v
is
v
v
of
Chapter 3. Installing the Device Driver Software (AIX)
The instructions in this chapter pertain to the AIX operating system.
Note: If you are using the Linux operating system, see Chapter 7, “Installing and Setting Up Software for
the Dual Channel SCSI RAID Enablement Card (Linux),” on page 13 for information regarding
installing the device driver software and configuring your system. If you have another operating
system installed, refer to your operating system documentation.
install the device driver software, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
Insert the appropriate AIX 5.1 or 5.2 update CD-ROM into the media device.
start SMIT, type:
smitty install_all
Press Enter.
The Install Additional Device Software window highlights the INPUT device/directory for software
option.
Select or type the input device:
Press F4 to display the input device list. Select the name of the device (example: CD-ROM) that
you are using and press Enter
the Entry Field, type the name of the input device you are using, and press Enter.
The INSTALL AND UPDATE FROM ALL AVAILABLE SOFTWARE window appears. Highlight the
SOFTWARE to install option.
Press F4 to display a list of the device software you can install.
display the Find window, type:
10.
the Find window, type:
devices.pci.14107802
Press Enter. The system finds and highlights this device software, and displays output similar to the
following example:
devices.pci.14107802
11.
Press F7 to select the device software, and press Enter.
12.
The INSTALL AND UPDATE FROM ALL AVAILABLE SOFTWARE window displays. Entry data fields
All
are automatically updated. Highlight the ACCEPT new license agreements? option. Ensure that this
option is set to yes. Press Enter to accept the data.
13.
The ARE YOU SURE window displays. Press Enter to accept the data.
14.
The COMMAND STATUS window displays.
The term
When
RUNNING
RUNNING
highlighted to indicate that the install and configure command is in progress.
changes to OK, scroll down to the bottom of the page and locate the Installation
Summary.
After a successful installation,
SUCCESS
appears in the Result column of the summary at the bottom
the page.
Remove the installation media from the drive.
15.
16.
Press F10 to exit SMIT.
5
6
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
Be
To
1.
2.
3.
4.
5.
If
Chapter 4. Installing the Dual Channel SCSI RAID Enablement Card
This chapter provides instructions for installing the Dual Channel SCSI RAID Enablement Card if you are
using the AIX or Linux operating system. Save this guide for future reference.
Note: If you are using the Linux operating system, the device driver and utility software must first be
installed before the Dual Channel SCSI RAID Enablement Card. For information regarding installing
software and configuring your system, see Chapter 7, “Installing and Setting Up Software for the
Dual Channel SCSI RAID Enablement Card (Linux),” on page 13. If you have another operating
system installed, refer to your operating system documentation.
Attention
sure you have read “Handling Static Sensitive Devices” on page vii before handling your Dual
Channel SCSI RAID Enablement Card.
install the adapter, refer to the option installation documentation that shipped with your system unit to
perform the following:
Shut down your system and place it into service mode as described in the documentation provided
with your system.
Remove the front bezel as described in the service as described in the documentation provided with
your system.
Plug the card into the connector on the system board as described in the documentation provided with
your system.
Replace the front bezel as described in the service as described in the documentation provided with
your system.
Return the system unit to the operating position and power it on. If the integrated SCSI controller was
used as the boot device prior to the card installation, the controller will still boot your system.
Note:
the system unit does not power on, repeat this installation procedure. If you continue
experiencing problems, contact your service representative.
7
8
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
v
v
To
1. If
2.
If
C F
C F
C F
on
If
If
To
1. If
2.
Chapter 5. Setting Up the Dual Channel SCSI RAID Enablement Card (AIX)
This chapter provides AIX-specific system hardware and software installation verification, adapter
configuration, and problem- determination procedures.
Note: If you are using the Linux operating system, see Chapter 7, “Installing and Setting Up Software for
the Dual Channel SCSI RAID Enablement Card (Linux),” on page 13 for information regarding
installing the device driver software and configuring your system. If you have another operating
system installed, refer to your operating system documentation.
Verifying the Installation
Verify the initial installation by using the following procedures:
Verify AIX Software Installation (lslpp)
Verify Hardware Installation (lsdev)
Verifying AIX Software Installation
verify that the device driver software for the Dual Channel SCSI RAID Enablement Card is installed, do
the following:
necessary, log in as root user.
Type:
lslpp -l devices.pci.14107802*
Press Enter.
the Dual Channel SCSI RAID Enablement Card device driver software is installed, the following is an
example of the data that displays on your screen:
Fileset
---------------------------------------------------------------------------------------------------------devices.pci.14107802.diag 5.2.0.11
devices.pci.14107802.rte
devices.pci.14107802.ucode 5.2.0.0
Note: The fileset number 5.2.0.11 displayed in the example might be different from the output displayed
your screen.
this information displays but you continue to have problems, or if no data displays on your screen, the
Dual Channel SCSI RAID Enablement Card device driver did not install. Return to Chapter 3, “Installing
the Device Driver Software (AIX),” on page 5. Then continue with “Verifying Hardware Installation.”
Level
5.2.0.11
State Type Description (Uninstaller)
PCI-X Dual Channel Ultra320 SCSI Adapter Diagnostics
PCI-X Dual Channel Ultra320 SCSI Adapter Software
PCI-X Dual Channel Ultra320 SCSI Adapter Microcode
you continue to experience problems, it may be necessary to call your system support organization.
Refer to your operating system documentation for instructions.
Verifying Hardware Installation
verify that your system unit recognizes the Dual Channel SCSI RAID Enablement Card, do the
following:
necessary, log in as root user.
Type:
lsdev -C | grep sis
Press Enter.
9
3. To
4.
If
If
Go to
a
If
to
10
If the Dual Channel SCSI RAID Enablement Card did install correctly, the following is an example of
the data that displays on your screen:
sisioa0
sisraid0
Available 1H-08
Available 1H-08-ff
PCI-X Dual Channel U320 SCSI RAID Adapter
PCI-X U320 SCSI RAID Adapter Logical bus
You may also see a message on your screen that shows the non-RAID adapter that existed before the
integrated SCSI controller was converted to a RAID adapter, sisscsia0, is now
Defined
instead of
Available, as shown:
sisscsia0
Defined 1H-08
PCI-X Dual Channel Ultra320 SCSI Adapter
remove the non-RAID adapter, type:
rmdev -Rdl sisscsia0
Press Enter.
Type:
lsdev -C | grep scsi
Press Enter.
the Dual Channel SCSI RAID Enablement Card did install correctly, the following is an example of
the data that displays on your screen:
scsi4
scsi5
the message on your screen indicates that your adapter is
Available 1H-08-00
Available 1H-08-01
PCI-X U320 SCSI RAID Adapter Physical bus
PCI-X U320 SCSI RAID Adapter Physical bus
Defined
instead of Available, shut down
your machine. Check the Dual Channel SCSI RAID Enablement Card to ensure it is installed correctly.
Chapter 4, “Installing the Dual Channel SCSI RAID Enablement Card,” on page 7, then return to
this section and repeat steps 1 and 2.
Note: If the message on your screen indicates that your adapter is
Defined
second time, contact
your service representative.
Updating Hard Disk Drive and Adapter Microcode
Disk drives attached to this RAID adapter must contain the latest microcode level.
The latest microcode level for your drive model and Dual Channel SCSI RAID Enablement Card can be
found at the Download pSeries and RS/6000 Microcode Updates Web site:
https://techsupport.services.ibm.com/server/mdownload/download.html.
The drive microcode download you need should match the name of one of the drive model numbers
located under the DASD Microcode listing found on the Web site. The drive machine type and model of
the drive you are updating can be identified by reading its VPD (Vital Product Data). Similarly, the name of
the Dual Channel SCSI RAID Enablement Card microcode download you need can be found under the
Adapter Microcode listing on the Web site.
Problem Determination
you encounter any problems with the adapter after the installation, refer to your system unit
documentation for information about running diagnostics or other problem-determination procedures. Refer
the PCI-X SCSI RAID Controller Reference Guide for AIX, order number SA23-1323.
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
v To
v
v
v
v To
to
v
a
to
v
v If a
v
v To
to
v
a
v
v If a
be
Chapter 6. Using Disk Arrays
This chapter provides information about using the RAID-related features of this adapter. The following
tasks are beyond the scope of this installation guide and are usually performed by system administrators.
use the RAID-related features of the hardware, disk arrays must be created first.
AIX and Linux use and manage the disk array in the same way as it would a single non-RAID disk.
For optimum performance, distribute the hard disk drives equally across the available channels.
Using AIX
For an overview of RAID and detailed procedures, refer to the PCI-X SCSI RAID Controller Reference
Guide for AIX, order number SA23-1323.
Disk drives previously installed in this system probably have 512 bytes/sector.
use the drives in disk arrays with this adapter, the disk drives must be prepared by formatting them
522 bytes/sector.
Disk drives in 512 bytes/sector format may continue to be used as standalone disks (that is, not part of
disk array) on this RAID adapter. Additionally, drives formatted to 522 bytes/sector may be reformatted
512 bytes/sector at a later time if desired for compatibility with previous adapters.
Formatting of drives to 522 bytes/sector and the creation of disk arrays may require that data previously
written on the drives be saved and restored after the the disk arrays are created.
disk array is to be used as the boot device, it may be required to prepare the disks by booting from
the Diagnostic CD and creating the disk array before installing AIX. You might want to perform this
procedure when the original boot drive is to be used as part of a disk array.
Using Linux
For an overview of RAID and detailed procedures, refer to the PCI-X SCSI RAID Controller Reference
Guide for Linux, order number SA23-1327.
Disk drives previously installed in this system are probably JBOD drives. JBOD drives are formatted to
512 bytes/sector.
use the drives in disk arrays with this adapter, the disk drives must be prepared by formatting them
522 bytes/sector.
Disk drives in 512 bytes/sector format may continue to be used as standalone disks (that is, not part of
disk array) on this RAID adapter. Additionally, drives formatted to 522 bytes/sector for advanced
function may be reformatted to 512 bytes/sector at a later time if desired for compatibility with previous
adapters.
Formatting of drives to 522 bytes/sector for advanced function and the creation of disk arrays may
require that data previously written on the drives be saved and restored after the disk arrays are
created.
disk array is to be used as the boot device, it may be required to prepare the disks by booting into
rescue mode from the Linux CD first. If your Linux distribution does not provide this feature, the
Diagnostic CD may be used. You might want to perform this procedure when the original boot drive is to
used as part of a disk array.
11
12
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
If
v
v
1. If
2.
1. If
2.
3.
Chapter 7. Installing and Setting Up Software for the Dual Channel SCSI RAID Enablement Card (Linux)
Previously, SuSE Linux Enterprise Server 8 SP3 used the
Channel Ultra320 SCSI RAID adapter. This device driver has been replaced by the
which is used on all other supported distributions. Information on using the
retained in this chapter. If you are using the older
ibmsis
ibmsis
device driver to control the PCI-X Dual
ipr
device driver,
ibmsis
device driver has been
device driver, go to “Installing the Dual Channel
SCSI RAID Enablement Card using the ibmsis Device Driver” on page 18 and follow the procedures
shown to install and set up the Dual Channel SCSI RAID Enablement Card. If you are using the newer
device driver, use the procedures detailed in “Installing the Dual Channel SCSI RAID Enablement Card
using the ipr Device Driver” below.
Note:
you are using AIX, see Chapter 5, “Setting Up the Dual Channel SCSI RAID Enablement Card
(AIX),” on page 9. If you have another operating system installed, refer to your operating system
documentation for information regarding installing the device driver software and configuring your
system.
Installing the Dual Channel SCSI RAID Enablement Card using the ipr Device Driver
Verifying Linux Software Installation
Software for the Dual Channel SCSI RAID Enablement Card consists of a device driver and a set of
utilities. This software is provided with the supported Linux distribution, and should have been installed
along with the operating system.
Verify the initial software installation by using the following procedures:
Verifying Device Driver Version (ipr)
Verifying Utility Version (iprconfig)
ipr
Verifying Device Driver Version
The Dual Channel SCSI RAID Enablement Card requires Version 1.0.3 (or later) of the ipr device driver
for 2.4 based kernels and Version 2.0.9 (or later) for 2.6 based kernels. To verify the version number of
ipr, do the following:
For 2.6 Linux kernels:
necessary, log in as root user.
Type:
modinfo ipr | grep version
Press Enter.
following is an example of the data that displays on your screen:
The
version:
For 2.4 Linux kernels:
necessary, log in as root user.
Type:
modinfo ipr
Press Enter.
Type:
2.0.9 DA9C6A0AA78C5D2B9D947A1
13
If
14
strings filename | grep "ipr version"
where filename is the file name displayed after the modinfo command was typed in the previous step.
Press Enter.
The following is an example of the data that displays on your screen:
ipr version=Ver. 1 Rev. 0.3 SMP
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your
Linux distributor or download them from the IBM StorageIO Web site. The Web address is:
http://www-124.ibm.com/storageio/ipr/.
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
1. If
2.
If
If it is
1.
2.
3.
If
1.
2. At
3.
cd
4.
5.
Verifying Utility Version
The Dual Channel SCSI RAID Enablement Card requires Version 1.0.3 (or later) of the iprconfig utility for
2.4 based kernels and version 2.x.x for 2.6 based kernels. To verify the version number of the iprconfig
utility, do the following:
necessary, log in as root user.
Type:
iprconfig --version
Press Enter.
The following is an example of the data that displays on your screen:
iprconfig: Ver. 1 Rev. 0.3
iprconfig: 2.0.10 (May 23, 2004)
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your
Linux distributor or download them from the IBM StorageIO Web site. The Web address is:
http://www-124.ibm.com/storageio/ipr/.
Installing the Downloaded Device Driver and Utility Software
Software required for the Dual Channel SCSI RAID Enablement Card is usually installed when the Linux
operating system is installed. Software installation is not necessary unless verification has failed.
necessary to install software, obtain it from one of the following sources in the order listed:
Your supported Linux operating system CD-ROM
Your Linux distributor
Online at the IBM StorageIO Web site: http://www-124.ibm.com/storageio/ipr/
downloading the needed software package, install the software by continuing with the instructions in
After
this section. Then, return to “Verifying Linux Software Installation” on page 13, and follow the procedure to
ensure that the installation was successful.
Installing Device Driver Software
The ipr device driver is available in the base 2.6 kernel as of Version 2.6.7-rc1. Additional driver updates
may be available on the IBM StorageIO Web site as mentioned above. These patches may be
downloaded and installed like any other kernel patch. The ipr device driver is not in the base 2.4 kernel
and must be downloaded from http://www-124.ibm.com/storageio/ipr/ in the form of a kernel patch if not
available as part of your distribution.
you downloaded the ipr device driver patch, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
Uncompress and apply the patch by typing:
/usr/src/linux
wget http://www-124.ibm.com/storageio/ipr/patch-2.4.26-ipr-1.0.5.gz
gunzip patch-2.4.26-ipr-1.0.5.gz
patch -p1 < patch-2.4.26-ipr-1.0.5
Compile and install the device driver. If you are compiling a 2.4 kernel, make sure IBM Power RAID
SCSI support is selected. If you are compiling a 2.6 kernel, make sure IBM Power Linux RAID
adapter support is selected.
Reboot your system.
Chapter 7. Installing and Setting Up Software for the Dual Channel SCSI RAID Enablement Card (Linux)
15
If
1.
2. At
3.
4.
5.
6.
7. If
If
To
1. If
2.
3.
If
16
Installing the Utility Software
you downloaded the iprutils utility package, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
Install the following packages (if not already installed):
gcc
ncurses-devel
pciutils-devel
glibc-devel
Rebuild the rpm package by typing:
rpmbuild –-rebuild –-target ppc iprutils-2.0.10-1.src.rpm
Press Enter.
Install the compiled rpm package by typing:
rpm -Uvh iprutils-2.0.10-1.ppc.rpm
Restart the iprutils services by typing:
/etc/init.d/iprupdate restart
/etc/init.d/iprdump restart
running a 2.6 kernel, restart the iprinit service by typing:
/etc/init.d/iprinit restart
Installing the Dual Channel SCSI RAID Enablement Card
you have successfully verified your software installation, install your Dual Channel SCSI RAID
Enablement Card. Follow the steps in Chapter 4, “Installing the Dual Channel SCSI RAID Enablement
Card,” on page 7.
Verifying Hardware Installation
verify that your system unit recognizes the Dual Channel SCSI RAID Enablement Card, do the
following:
necessary, log in as root user.
Type:
iprconfig
Press Enter.
Select the Display hardware status option and press Enter.
the Dual Channel SCSI RAID Enablement Card installed correctly, data displays for the adapter and
each disk attached to it. The following is an example of the data that displays on your screen:
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
If
or if it is
to
Display Hardware Status
Type option, press Enter.
1=Display hardware resource information details
OPT Name
PCI/SCSI Location
Description
Status
--- ------ -------------------------- ------------------------- --------------0000:02:04.0.2/
0000:02:04.0.2/1:14:0
0000:02:04.0.2/1:6:0
0000:02:04.0.2/1:8:0
0000:02:04.0.2/1:9:0
sdt
0000:02:04.0.2/255:0:0
0000:02:04.0.2/1:10:0
0000:02:04.0.2/1:11:0
0000:02:04.0.2/1:12:0
0000:02:04.0.2/1:13:0
sdb
0000:02:04.0.2/255:0:2
0000:02:04.0.2/1:4:0
sdc
0000:02:04.0.2/255:0:3
0000:02:04.0.2/1:3:0
sdd
0000:02:04.0.2/255:0:4
PCI-X SCSI RAID Adapter
Advanced Function Disk
Advanced Function Disk
Advanced Function Disk
Advanced Function Disk
RAID 5 Disk Array
RAID 5 Array Member
RAID 5 Array Member
RAID 5 Array Member
RAID 5 Array Member
RAID 0 Disk Array
RAID 0 Array Member
RAID 0 Disk Array
RAID 0 Array Member
RAID 0 Disk Array
Operational
Active
Active
Active
Active
Active
Active
Active
Active
Active
Active
Active
Active
Active
Active
More...
e=Exit
the message on your screen indicates your newly installed card, shown in the example as
RAID
q=Cancel
Adapter, is
r=Refresh
t=Toggle
Not Operational
f=PageDn
instead of
b=PageUp
Operational
PCI-X SCSI
not listed, shut down your machine. Go
Chapter 4, “Installing the Dual Channel SCSI RAID Enablement Card,” on page 7 to ensure that the
Dual Channel SCSI RAID Enablement Card is installed correctly. Then return to this section, and repeat
this hardware installation procedure.
Chapter 7. Installing and Setting Up Software for the Dual Channel SCSI RAID Enablement Card (Linux)
17
by
be
If
1. If
2.
3.
If
If
4. If
v
v
If
to
If
18
Updating Hard Disk Drive and Adapter Microcode
Disk drives attached to this RAID adapter must contain the latest microcode level.
Microcode updates for hard disk drives attached to the Dual Channel SCSI RAID Enablement Card are
handled automatically by the iprupdate utility, which executes each time the system is booted, or manually
the iprconfig utility. The iprupdate utility will only automatically update microcode to adapters and
devices if it is determined that they are not at the minimum supported level of microcode.
The latest microcode level for your drive model and Dual Channel SCSI RAID Enablement Card may be
shipped as part of the Linux distribution. If an update or replacement is needed, the latest microcode can
obtained from the pSeries microcode Web site:
http://techsupport.services.ibm.com/server/mdownload/download.html.
you downloaded new microcode from the Web site, follow the instructions in the Readme on the Web
site to install it. If there are no instructions available, install it as follows:
necessary, log in as root user.
Install the rpm package by typing:
rpm -ivh –-ignoreos sisraid.0047.rpm
Press Enter.
Update the microcode based on your kernel type:
you are using a 2.4 based kernel, type:
iprupdate
Press Enter. Once the microcode update has completed, you are finished.
you are using a 2.6 based kernel, type:
iprconfig
Press Enter and go to the next step.
you are using a 2.6 based kernel, choose the Download microcode menu option and:
Select the adapters and devices you wish to update
For each selected device, select the microcode image you wish to download
Problem Determination
you encounter any problems with the adapter after the installation, refer to your system unit
documentation for information about running diagnostics or other problem-determination procedures. Refer
the PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327.
Installing the Dual Channel SCSI RAID Enablement Card using the ibmsis Device Driver
This chapter provides instructions for installing and verifying software that pertains to the Linux operating
system using the
ibmsis
device driver. If you are using the newer
ipr
driver, see “Installing the Dual
Channel SCSI RAID Enablement Card using the ipr Device Driver” on page 13. The required software
should be installed before you install the Dual Channel SCSI RAID Enablement Card.
Note:
you are using AIX, see Chapter 5, “Setting Up the Dual Channel SCSI RAID Enablement Card
(AIX),” on page 9. If you have another operating system installed, refer to your operating system
documentation for information regarding installing the device driver software and configuring your
system.
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
v
v
1. If
2.
3.
If
Verifying Linux Software Installation
Software for the Dual Channel SCSI RAID Enablement Card consists of a device driver and a set of
utilities. This software is provided with the supported Linux distribution, and should have been installed
along with the operating system.
Verify the initial software installation by using the following procedures:
Verifying Device Driver Version (ibmsis)
Verifying Utility Version (sisconfig)
Verifying Device Driver Version
The Dual Channel SCSI RAID Enablement Card requires Version 1 Revision 19 (or later) of the ibmsis
device driver. To verify the version number of ibmsis, do the following:
necessary, log in as root user.
Type:
modinfo ibmsis
Press Enter.
Type:
strings filename | grep "ibmsis version"
where filename is the file name displayed after the modinfo command was typed in the previous step.
Press Enter.
The following is an example of the data that displays on your screen:
ibmsis version=Ver. 1 Rev. 19.10 SMP
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your
Linux distributor or download them from the IBM Support Web site. The Web address is:
http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p.
Chapter 7. Installing and Setting Up Software for the Dual Channel SCSI RAID Enablement Card (Linux)
19
1. If
2.
If
If it is
1.
2.
3.
If
1.
2. At
3. If
4.
5.
If
1.
2. At
3. If
4.
20
Verifying Utility Version
The Dual Channel SCSI RAID Enablement Card requires Version 1 Revision 19 (or later) of the sisconfig
utility. To verify the version number of the sisconfig utility, do the following:
necessary, log in as root user.
Type:
sisconfig --version
Press Enter.
The following is an example of the data that displays on your screen:
sisconfig: Ver. 1 Rev. 19.10
Note: The version displayed in the example might be different from the output displayed on your screen.
the version displayed is not at the recommended level and updates are needed, obtain them from your
Linux distributor or download them from the IBM Support Web site. The Web address is:
http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p.
Installing the Downloaded Device Driver and Utility Software
Software required for the Dual Channel SCSI RAID Enablement Card is usually installed when the Linux
operating system is installed. Software installation is not necessary unless verification has failed.
necessary to install software, obtain it from one of the following sources in the order listed:
Your supported Linux operating system CD-ROM
Your Linux distributor
Online at the IBM Support Web site: http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p
downloading the needed software package, install the software by continuing with the instructions in
After
this section. Then, return to “Verifying Linux Software Installation” on page 13, and follow the procedure to
ensure that the installation was successful.
Installing Device Driver Software
you downloaded the ibmsis device driver package, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
necessary, untar the device driver package. Type:
tar -xzvf ibmsis-$ver.tgz
Press Enter.
Compile and install the device driver. For instructions on how to compile and install the ibmsis device
driver, see the readme file located in the ibmsis/src/ directory.
Reboot your system.
Installing the Utility Software
you downloaded the sisutils utility package, do the following:
Turn on the power to your system unit.
the system prompt, log in as root user.
necessary, untar the utility package. Type:
tar -xzvf sisutils-$ver.tgz
Press Enter.
Compile and install the sisutils utility package. For instructions on how to compile and install the utility
package, see the readme file located in the sisutils/ directory.
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
If
To
1. If
2.
3.
If
ID ID
If
is
or if it is
Installing the Dual Channel SCSI RAID Enablement Card
you have successfully verified your software installation, install your Dual Channel SCSI RAID
Enablement Card. Follow the steps in Chapter 4, “Installing the Dual Channel SCSI RAID Enablement
Card,” on page 7.
Verifying Hardware Installation
verify that your system unit recognizes the Dual Channel SCSI RAID Enablement Card, do the
following:
necessary, log in as root user.
Type:
sisconfig
Press Enter.
Select the Display disk hardware status option and press Enter.
the Dual Channel SCSI RAID Enablement Card installed correctly, data displays for the adapter and
each disk attached to it. The following is an example of the data that displays on your screen:
Display Disk Hardware Status
Serial
Number
03000001 IBM
000012F0 IBM
00001315 IBM
000012F5 IBM
00001304 IBM
E7800377 IBM
Vendor Product
5709001
ST3146807LC 090
ST3146807LC 079
ST3146807LC 070
ST3146807LC 079
01C933F1
Resource
Model Name
001
/dev/ibmsis0 Operational
205
/dev/sdc
Hardware
Status
HS/Active
DPY/Active
DPY/Active
DPY/Active
Operational
Press Enter to continue.
e=Exit
d=Display disk unit details
q=Cancel
r=Refresh
f=PageDn
b=PageUp
p=Display device parity status
the message on your screen indicates your newly installed card, shown in the example as /dev/ibmsis0,
Not Operational
instead of
Operational
not listed, shut down your machine. Go to Chapter 4,
“Installing the Dual Channel SCSI RAID Enablement Card,” on page 7 to ensure that the Dual Channel
SCSI RAID Enablement Card is installed correctly. Then return to this section, and repeat this hardware
installation procedure.
Chapter 7. Installing and Setting Up Software for the Dual Channel SCSI RAID Enablement Card (Linux)
21
as
If
1. If
2.
3.
cd
4.
5.
If
to
22
Updating Hard Disk Drive and Adapter Microcode
Disk drives attached to this RAID adapter must contain the latest microcode level.
Microcode updates for hard disk drives attached to the Dual Channel SCSI RAID Enablement Card are
handled automatically by the sisupdate utility, which executes each time the system is booted.
The latest microcode level for your drive model and Dual Channel SCSI RAID Enablement Card is shipped
part of the Linux distribution. If an update or replacement is needed, the latest microcode can be
obtained from your Linux distributor, or found as part of the device driver package at the following Web
site: http://www6.software.ibm.com/dl/linuxsdd/linuxsdd-p.
you downloaded new microcode from the Web site, install it as follows:
necessary, log in as root user.
Untar the package, type:
tar –xzvf ibmsis-$ver.tgz
Press Enter.
Change directories to ibmsis/obj/firmware/. Type:
ibmsis/obj/firmware/
Install the microcode. Type:
make install
Update the microcode. Type:
sisupdate
Press Enter.
Problem Determination
you encounter any problems with the adapter after the installation, refer to your system unit
documentation for information about running diagnostics or other problem-determination procedures. Refer
the PCI-X SCSI RAID Controller Reference Guide for Linux, order number SA23-1327.
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
Appendix A. Communications Statements
The following statement applies to this product. The statement for other products intended for use with this
product appears in their accompanying documentation.
Federal Communications Commission (FCC) Statement
Note: This equipment has been tested and found to comply with the limits for a Class A digital device,
pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection
against harmful interference when the equipment is operated in a commercial environment. This
equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in
accordance with the instruction manual, may cause harmful interference to radio communications.
Operation of this equipment in a residential area is likely to cause harmful interference in which
case the user will be required to correct the interference at his own expense.
Properly shielded and grounded cables and connectors must be used in order to meet FCC emission
limits. Neither the provider nor the manufacturer is responsible for any radio or television interference
caused by using other than recommended cables and connectors or by unauthorized changes or
modifications to this equipment. Unauthorized changes or modifications could void the user’s authority to
operate the equipment.
This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
(1) this device may not cause harmful interference, and (2) this device must accept any interference
received, including interference that may cause undesired operation.
European Union (EU) Statement
This product is in conformity with the protection requirements of EU Council Directive 89/336/EEC on the
approximation of the laws of the Member States relating to electromagnetic compatibility. The
manufacturer cannot accept responsibility for any failure to satisfy the protection requirements resulting
from a non-recommended modification of the product, including the fitting of option cards supplied by third
parties. Consult with your dealer or sales representative for details on your specific hardware.
This product has been tested and found to comply with the limits for Class A Information Technology
Equipment according to CISPR 22 / European Standard EN 55022. The limits for Class A equipment were
derived for commercial and industrial environments to provide reasonable protection against interference
with licensed communication equipment.
Attention: This is a Class A product. In a domestic environment this product may cause radio
interference in which case the user may be required to take adequate measures.
International Electrotechnical Commission (IEC) Statement
This product has been designed and built to comply with IEC 60950.
United Kingdom Telecommunications Safety Requirements
This equipment is manufactured to the International Safety Standard EN60950 and as such is approved in
the UK under the General Approval Number NS/G/1234/J/100003 for indirect connection to the public
telecommunication network.
The network adapter interfaces housed within this equipment are approved separately, each one having its
own independent approval number. These interface adapters, supplied by the manufacturer, do not use or
contain excessive voltages. An excessive voltage is one which exceeds 70.7 V peak ac or 120 V dc. They
23
a
in
24
interface with this equipment using Safe Extra Low Voltages only. In order to maintain the separate
(independent) approval of the manufacturer’s adapters, it is essential that other optional cards, not
supplied by the manufacturer, do not use main voltages or any other excessive voltages. Seek advice from
competent engineer before installing other adapters not supplied by the manufacturer.
Avis de conformité aux normes du ministère des Communications du Canada
Cet appareil numérique de la classe A est conforme à la norme NMB-003 du Canada.
Canadian Department of Communications Compliance Statement
This Class A digital apparatus complies with Canadian ICES-003..
VCCI Statement
The following is a summary of the VCCI Japanese statement in the box above.
This is a Class A product based on the standard of the Voluntary Control Council for Interference by
Information Technology Equipment (VCCI). If this equipment is used in a domestic environment, radio
disturbance may arise. When such trouble occurs, the user may be required to take corrective actions.
Electromagnetic Interference (EMI) Statement - Taiwan
The following is a summary of the EMI Taiwan statement above.
Warning: This is a Class A product. In a domestic environment this product may cause radio interference
which case the user will be required to take adequate measures.
Radio Protection for Germany
Dieses Gerät ist berechtigt in Übereinstimmung mit Dem deutschen EMVG vom 9.Nov.92 das
EG–Konformitätszeichen zu führen.
Der Aussteller der Konformitätserklärung ist die IBM Germany.
Dieses Gerät erfüllt die Bedingungen der EN 55022 Klasse A. Für diese von Geräten gilt folgende
Bestimmung nach dem EMVG:
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
Geräte dürfen an Orten, für die sie nicht ausreichend entstört sind, nur mit besonderer Genehmigung des
Bundesministers für Post und Telekommunikation oder des Bundesamtes für Post und Telekommunikation
betrieben werden. Die Genehmigung wird erteilt, wenn keine elektromagnetischen Störungen zu erwarten
sind.
(Auszug aus dem EMVG vom 9.Nov.92, Para.3, Abs.4)
Hinweis
Dieses Genehmigungsverfahren ist von der Deutschen Bundespost noch nicht veröffentlicht worden.
Appendix A. Communications Statements
25
26
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
in
Appendix B. Notices
This information was developed for products and services offered in the U.S.A.
The manufacturer may not offer the products, services, or features discussed in this document in other
countries. Consult the manufacturer’s representative for information on the products and services currently
available in your area. Any reference to the manufacturer’s product, program, or service is not intended to
state or imply that only that product, program, or service may be used. Any functionally equivalent product,
program, or service that does not infringe any intellectual property right of the manufacturer may be used
instead. However, it is the user’s responsibility to evaluate and verify the operation of any product,
program, or service.
The manufacturer may have patents or pending patent applications covering subject matter described in
this document. The furnishing of this document does not give you any license to these patents. You can
send license inquiries, in writing, to the manufacturer.
The following paragraph does not apply to the United Kingdom or any country where such
provisions are inconsistent with local law: THIS MANUAL IS PROVIDED ″AS IS″ WITHOUT
WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain
transactions; therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically
made to the information herein; these changes will be incorporated in new editions of the publication. The
manufacturer may make improvements and/or changes in the product(s) and/or the program(s) described
this publication at any time without notice.
Information concerning products made by other than the manufacturer was obtained from the suppliers of
those products, their published announcements, or other publicly available sources. The manufacturer has
not tested those products and cannot confirm the accuracy of performance, compatibility or any other
claims related to products made by other than the manufacturer. Questions on the capabilities of products
made by other than the manufacturer should be addressed to the suppliers of those products.
27
28
Dual Channel SCSI RAID Enablement Card: Installation and Using Guide
h
h No
Readers’ Comments — We’d Like to Hear from You
Dual Channel SCSI RAID Enablement Card
Installation and Using Guide
Publication No. SA23-1325-01
Overall, how satisfied are you with the information in this book?
Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied
Overall satisfaction hhhhh
How satisfied are you that the information in this book is:
Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied
Accurate hhhhh
Complete hhhhh
Easy to find hhhhh
Easy to understand hhhhh
Well organized hhhhh
Applicable to your tasks hhhhh
Please tell us how we can improve this book:
Thank you for your responses. May we contact you?
Yes
When you send comments to IBM, you grant IBM a nonexclusive right to use or distribute your comments in any
way it believes appropriate without incurring any obligation to you.
Name
Address
Company or Organization
Phone No.
or
___________________________________________________________________________________________________
Readers’ Comments — We’d Like to Hear from You
SA23-1325-01
Fold
_________________________________________________________________________________________
and Tape
Please do not staple
Fold and Tape
NO
POSTAGE
NECESSARY
MAILED IN THE
IF
UNITED STATES
Cut or Fold
Along Line
BUSINESS REPLY MAIL
FIRST-CLASS MAIL PERMIT NO. 40 ARMONK, NEW YORK
POSTAGE WILL BE PAID BY ADDRESSEE
Information Development
Department H6DS-9561
11501 Burnet Road
Austin, TX 78758-3493
_________________________________________________________________________________________
Fold
and Tape
Please do not staple
Fold and Tape
SA23-1325-01
Fold
Cut
Along Line
Part Number: 80P5514
Printed in USA
SA23-1325-01
80P5514
P/N:
(1P)
Loading...