Sun Microsystems PCI User Manual

Sun™ Dual Port 4x IB Host Channel
Adapter PCI-X Installation and
User’s Guide
Sun Microsystems, Inc. www.sun.com
Part No. 819-1280-10 September 2005, Revision A
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
limitation, theseintellectual propertyrights mayinclude oneor more ofthe U.S.patents listedat http://www.sun.com/patents andone or more additionalpatents orpending patentapplications inthe U.S.and in other countries.
This documentand theproduct towhich itpertains are distributedunder licensesrestricting theiruse, copying,distribution, and decompilation. Nopart ofthe productor ofthis document may be reproducedin anyform byany meanswithout priorwritten authorizationof Sun andits licensors,if any.
Third-party software, including fonttechnology, is copyrightedand licensedfrom Sun suppliers. Parts ofthe productmay bederived from Berkeley BSDsystems, licensedfrom the University of California. UNIX is a registeredtrademark in
the U.S.and inother countries,exclusively licensedthrough X/Open Company, Ltd. Sun, SunMicrosystems, theSun logo,Java, AnswerBook2,docs.sun.com, SunVTS, Sun Fire, SunSolve,and Solarisare trademarksor registered
trademarks ofSun Microsystems,Inc. inthe U.S.and in other countries. All SPARC trademarksare usedunder licenseand are trademarks orregisteredtrademarks ofSPARCInternational, Inc.in theU.S. andin other
countries. Productsbearing SPARC trademarksare based upon an architecturedeveloped bySun Microsystems, Inc. The OPENLOOK andSun™ GraphicalUser Interfacewas developedby SunMicrosystems, Inc. for its users and licensees. Sun acknowledges
the pioneeringefforts ofXeroxin researchingand developingthe conceptof visualor graphicaluser interfaces for the computer industry.Sun holds anon-exclusive licensefrom Xerox to theXerox Graphical User Interface, which license also covers Sun’s licensees whoimplement OPEN LOOK GUIsand otherwisecomply withSun’s writtenlicense agreements.
U.S. GovernmentRights—Commercial use.Government usersaresubject tothe SunMicrosystems, Inc.standard licenseagreementand applicable provisionsof theFAR and itssupplements.
DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANYIMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FORA PARTICULAR PURPOSEOR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Copyright 2005Sun Microsystems,Inc., 4150Network Circle, Santa Clara,Californie 95054,Etats-Unis. Tous droitsréservés. Sun Microsystems,Inc. ales droits de propriétéintellectuels relatants à latechnologie quiest décritdans cedocument. Enparticulier, et sansla
limitation, cesdroits depropriétéintellectuels peuventinclure unou plusdes brevetsaméricains énumérésà http://www.sun.com/patentset un oules brevetsplus supplémentaires ou lesapplications debrevet en attente dans les Etats-Unis et dans les autres pays.
Ce produitou documentest protégé par uncopyright etdistribué avecdes licencesqui enrestreignentl’utilisation, lacopie, ladistribution, etla décompilation. Aucunepartie dece produitou documentne peut êtrereproduite sousaucune forme,par quelque moyen que ce soit, sans l’autorisation préalableet écritede Sunet deses bailleursde licence,s’il yen a.
Le logicieldétenu pardes tiers,et quicomprend la technologie relative auxpolices decaractères, estprotégé par un copyright et licencié par des fournisseurs deSun.
Des partiesde ceproduit pourront être dérivéesdes systèmesBerkeley BSD licenciés par l’Université de Californie. UNIX est une marque déposée auxEtats-Unis etdans d’autrespays etlicenciée exclusivement par X/Open Company, Ltd.
Sun, SunMicrosystems, lelogo Sun,Java, AnswerBook2,docs.sun.com, SunVTS, Sun Fire, SunSolve,et Solarissont desmarques defabrique ou des marquesdéposées deSun Microsystems, Inc. auxEtats-Unis etdans d’autres pays.
Toutes les marquesSPARC sont utiliséessous licenceet sontdes marques de fabriqueou desmarques déposéesde SPARC International,Inc. aux Etats-Uniset dansd’autres pays.Les produits portant lesmarques SPARCsont baséssur unearchitecturedéveloppée parSun Microsystems, Inc.
L’interfaced’utilisation graphiqueOPEN LOOKet Sun™a étédéveloppée parSun Microsystems, Inc. pourses utilisateurset licenciés.Sun reconnaît lesefforts de pionniersde Xeroxpour larechercheet ledéveloppement duconcept desinterfaces d’utilisationvisuelle ougraphique pour l’industriede l’informatique.Sun détientune licensenon exclusivede Xerox sur l’interfaced’utilisation graphiqueXerox, cettelicence couvrant égalementles licenciéesde Sunqui mettenten placel’interface d’utilisation graphiqueOPEN LOOKet quien outre se conforment aux licencesécrites deSun.
LA DOCUMENTATION EST FOURNIE "EN L’ÉTAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONTFORMELLEMENT EXCLUES,DANS LAMESURE AUTORISEEPAR LA LOIAPPLICABLE, YCOMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

Contents

Regulatory Compliance Statements xi
Declaration of Conformity xv
Preface xvii
How This Book Is Organized xvii
Using UNIX Commands xviii
Shell Prompts xviii
Typographic Conventions xix
Related Documentation xix
Documentation, Support, and Training xx
Third-Party Web Sites xx
Contacting Sun Technical Support xx
Sun Welcomes Your Comments xxi
1. Product Overview 1
Product Features 1
Hardware Description 1
I/O Interfaces 3
InfiniBand Interface 3
PCI-X Interface 3
iii
LED Displays 3
Hardware, Software, and Cable Requirements 5
2. Adapter Installation 7
IB-HCA Bus and Slot Selection 7
Installing the Adapter 7
Verifying the Installation 9
3. Configuring IP Over InfiniBand 11
IPoIB Configuration 11
Configuring IPoIB 11
Managing IB Devices for IPoIB 13
4. InfiniBand Software Overview 15
InfiniBand Software 15
User Direct Access Programming Library 16
IP over InfiniBand Driver 16
InfiniBand Transport Framework 17
InfiniBand Management Framework 17
Tavor HCA Driver 17
Related Software Documentation 18
Manuals 18
Man Pages 18
A. InfiniBand Cable Pin Assignments 19
B. Diagnostic Software and Troubleshooting 23
SunVTS Diagnostic Testing 23
Using the SunVTS ibhcatest 24
Using the ibhcatest 25
Troubleshooting Tasks 25
iv Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005
Other Useful Utilities 26
cfgadm 26
snoop 27
netstat 27
kstat 27
Index 29
Contents v
vi Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005

Figures

FIGURE 1-1 Sun Dual Port 4x IB Host Channel Adapter PCI-X Card 2
FIGURE 1-2 LED Layout 4
FIGURE 2-1 Typical Adapter-to-Switch Configuration 9
FIGURE 4-1 InfiniBand Software Stack 16
FIGURE A-1 Sun 4x-to-12x InfiniBand Cable Pin Assignments 21
vii
viii Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005

Tables

TABLE 1-1 IB-HCA LEDs 3
TABLE 1-2 Hardware, Software, and Cable Requirements 5
TABLE B-1 SunVTS Documentation 24
ix
x Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005

Regulatory Compliance Statements

Your Sun product is marked to indicate its compliance class:
• Federal Communications Commission (FCC) — USA
• Industry Canada Equipment Standard for Digital Equipment (ICES-003) — Canada
• Voluntary Control Council for Interference (VCCI) — Japan
• Bureau of Standards Metrology and Inspection (BSMI) — Taiwan Please read the appropriate section that corresponds to the marking on your Sun product before attempting to install the
product.

FCC Class ANotice

This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
1. This device may not cause harmful interference.
2. This device must accept any interference received, including interference that may cause undesired operation. Note: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of
the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy, and if it is not installed and used in accordance with the instruction manual, it may cause harmful interference to radio communications. Operation of this equipment in a residential areais likely tocause harmful interference,in which casethe user willbe required to correct the interference at his own expense.
Modifications: Anymodifications madeto thisdevice thatare notapproved by Sun Microsystems, Inc.may voidthe authority granted to the user by the FCC to operate this equipment.

FCC Class B Notice

This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
1. This device may not cause harmful interference.
2. This device must accept any interference received, including interference that may cause undesired operation. Note: This equipment has been tested and found to comply with the limits for a Class B digital device, pursuant to Part 15 of
the FCC Rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiate radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If this equipment does cause harmful interference to radio or television reception,which can bedetermined by turningthe equipment off and on, the user is encouraged to try to correct the interference by one or more of the following measures:
• Reorient or relocate the receiving antenna.
• Increase the separation between the equipment and receiver.
• Connect the equipment into an outlet on a circuit different from that to which the receiver is connected.
• Consult the dealer or an experienced radio/television technician for help. Modifications: Anymodifications madeto thisdevice thatare notapproved by Sun Microsystems, Inc.may voidthe authority
granted to the user by the FCC to operate this equipment.

ICES-003 Class A Notice - Avis NMB-003, Classe A

This Class A digital apparatus complies with Canadian ICES-003. Cet appareil numérique de la classe A est conforme à la norme NMB-003 du Canada.
xi

ICES-003 Class B Notice - Avis NMB-003, Classe B

This Class B digital apparatus complies with Canadian ICES-003. Cet appareil numérique de la classe B est conforme à la norme NMB-003 du Canada.

BSMI Class A Notice

The following statement is applicable to products shipped to Taiwan and marked as Class A on the product compliance label.
xii Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005

CCC Class A Notice

The following statement is applicable to products shipped to China and marked with “Class A” on the product’s compliance label.
GOST-R Certification Mark
Regulatory Compliance Statements xiii
xiv Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005

Declaration of Conformity

Compliance Model Number: IB-HCA Product Family Name: Sun Dual Port 4x IB Host Channel Adapter PCI-X
EMC
USA—FCC Class A
This equipment complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
1. This equipment may not cause harmful interference.
2. This equipment must accept any interference that may cause undesired operation.

European Union

This equipment complies with the following requirements of the EMC Directive 89/336/EEC:
As Telecommunication Network Equipment (TNE) in Both Telecom Centers and Other Than Telecom Centers per (as applicable):
EN300-386 V.1.3.1 (09-2001) Required Limits:
EN55022/CISPR22 Class A EN61000-3-2 Pass EN61000-3-3 Pass EN61000-4-2 6 kV (Direct), 8 kV (Air) EN61000-4-3 3 V/m 80-1000MHz, 10 V/m 800-960 MHz, and 1400-2000 MHz EN61000-4-4 1 kV AC and DC Power Lines, 0.5 kV Signal Lines EN61000-4-5 2 kV AC Line-Gnd, 1 kV AC Line-Line and Outdoor Signal Lines, 0.5 kV Indoor signal Lines > 10m EN61000-4-6 3 V EN61000-4-11 Pass
As Information Technology Equipment (ITE) Class A per (as applicable):
EN55022:1998/CISPR22:1997 Class A EN55024:1998 Required Limits:
EN61000-4-2 4 kV (Direct), 8 kV (Air) EN61000-4-3 3 V/m EN61000-4-4 1 kV AC Power Lines, 0.5 kV Signal and DC Power Lines EN61000-4-5 1 kV AC Line-Line and Outdoor Signal Lines, 2 kV AC Line-Gnd, 0.5 kV DC Power Lines EN61000-4-6 3 V EN61000-4-8 1 A/m
EN61000-4-11 Pass EN61000-3-2 Pass EN61000-3-3 Pass

Safety: This equipment complies with the following requirements of the Low Voltage Directive 73/23/EEC:

EC Type Examination Certificates:
EN60950-1:2001, 1st Edition TÜV Rheinland Certificate No. IEC 60950-1:2001, 1st Edition CB Scheme Certificate No. Evaluated to all CB Countries UL 60950-1, 1st Edition, CSA C22.2 No. 60950-1-03 File: Vol. Sec.
Supplementary Information: This product was tested and complies with all the requirements for the CE Mark.
/S/ /S/
Dennis P. Symanski DATE Manager, Compliance Engineering Sun Microsystems, Inc. 4150 Network Circle, MPK15-102 Santa Clara, CA 95054 U.S.A. Tel: 650-786-3255 Fax: 650-786-3723
Donald Cameron DATE Program Manager Sun Microsystems Scotland, Limited Blackness Road, Phase I, Main Bldg Springfield, EH49 7LR Scotland, United Kingdom Tel: +44 1 506 672 395 Fax: +44 1 506 670 011
xv
Loading...
+ 35 hidden pages