Sun Microsystems Ethernet PCI-X Adapter User Manual

Download

Sun™10-Gigabit Ethernet PCI-X

Adapter Installation and User’s

Guide

Sun Microsystems, Inc. www.sun.com

Part No. 819-0938-12 December 2005, Revision A

Submit comments about this document at: http://www.sun.com/hwdocs/feedback

Copyright 2005 Sun Microsystems, Inc., 4150 Network Circle,SantaClara,California95054,U.S.A.Allrightsreserved. Sun Microsystems, Inc. has intellectual propertyrightsrelatingtotechnologythatisdescribedinthisdocument.Inparticular, and without

limitation, these intellectual property rights may include one or moreoftheU.S.patentslistedathttp://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and in other countries.

This document and the product to which it pertains aredistributedunderlicensesrestrictingtheiruse,copying,distribution,and decompilation. No part of the product or of this document may be reproducedinanyformbyanymeanswithoutpriorwrittenauthorizationof Sun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Parts of the product may be derived fromBerkeleyBSDsystems,licensedfromtheUniversityofCalifornia.UNIXisaregistered trademark in

the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd. Sun, Sun Microsystems, the Sun logo, AnswerBook2, docs.sun.com, Sun Fire,andSolarisaretrademarksorregistered trademarks of Sun

Microsystems, Inc. in the U.S. and in other countries. All SPARC trademarks are used under license and are trademarks or registeredtrademarksofSPARC International, Inc. in the U.S. and in other

countries. Products bearing SPARC trademarks are based upon an architecturedevelopedbySunMicrosystems,Inc. The OPEN LOOK and Sun™ Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sunacknowledges

the pioneering efforts of Xeroxinresearchinganddevelopingtheconceptofvisualorgraphicaluser interfaces for the computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements.

U.S. Government Rights—Commercial use. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements.

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTYOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Copyright 2005 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, Californie 95054, Etats-Unis. Tousdroits réservés. Sun Microsystems, Inc. a les droits de propriété intellectuels relatants à la technologie qui est décrit dans ce document. En particulier, et sans la

limitation, ces droits de propriété intellectuels peuvent inclure un ou plus des brevets américains énumérés à http://www.sun.com/patents et un ou les brevets plus supplémentaires ou les applications de brevet en attente dans les Etats-Unis et dans les autres pays.

Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a.

Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun.

Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company,Ltd.

Sun, Sun Microsystems, le logo Sun, AnswerBook2, docs.sun.com,Sun Fire, et Solaris sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.

Toutes les marques SPARCsont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARCInternational, Inc. aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARCsont basés sur une architecture développée par Sun Microsystems, Inc.

L’interfaced’utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sun reconnaît les efforts de pionniers de Xerox pour la recherche et le développement du concept des interfaces d’utilisation visuelle ou graphique pour l’industrie de l’informatique. Sun détient une license non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette licence couvrant également les licenciées de Sun qui mettent en place l’interface d ’utilisation graphique OPEN LOOK et qui en outre se conforment aux licences écrites de Sun.

LA DOCUMENTATION EST FOURNIE "EN L’ÉTAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

Please

Recycle

Preface xiii

1. Product Overview 1

Product Description 1

Hardware and Software Requirements 2

Features 3

2. Installing the Adapter 5

Installing the Adapter Without Dynamic Reconfiguration 5

▼ To Install the Adapter 6 ▼ To Install the Optical Transceiver 7 ▼ To Verify the Hardware Installation 9

Rebooting the System 10

Installing the Adapter With Dynamic Reconfiguration 11

Using the Installation Script 11

▼ To Install the Software Using the Installation Script 11

Verifying the Software Installation 14

Configuring the Network Host Files 15

3. Configuring Driver Parameters 19

10-Gigabit Ethernet Device Driver Parameters 19

iii

10-Gigabit Ethernet Driver Parameter Values and Definitions 20

Flow Control Parameters 20

Interrupt Parameters 21

Jumbo Frames 22

▼ To Configure Jumbo Frames Using ndd 22

▼ To Configure Jumbo Frames in a Linux Environment 23

Setting ixge Driver Parameters 23

Setting Parameters Using the ndd Utility 23

▼ To Specify Device Instances for the ndd Utility 24

Noninteractive and Interactive Modes 24

▼ To Use the ndd Utility in Noninteractive Mode 24 ▼ To Use the ndd Utility in Interactive Mode 25

Setting Parameters Using the ixge.conf File 26

▼ To Set Driver Parameters Using a ixge.conf File 27

10-Gigabit Ethernet Driver Operating Statistics 29

▼ To Check Link Partner Settings 32 ▼ To Discover Device Capabilities 34 ▼ To Discover Link Settings 35

Configuration Checking 35

4. Configuring VLANs 37

Overview of VLANs 37

Configuring VLANs 39

▼ To Configure Static VLANs 40

A. Specifications 43

Performance Specifications 43

Physical Characteristics 44

Power Requirements 44

iv Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

B. Diagnostic Software 45

SunVTS Diagnostic Software 45

Updating SunVTS to Recognize the Adapter 46

▼ To Update SunVTS to Recognize the Adapter 46

Using the SunVTS netlbtest 47

▼ To Use the netlbtest 47

C. Installing the Software Manually 49

▼ To Install the Driver Software Manually 49

D. Installing the Sun 10-Gigabit Ethernet Device Driver on Linux Platforms 53

Setting Up the Driver in a Linux Platform 53

▼ To Set Up the Driver in a Linux Platform 53

Setting Driver Parameters in a Linux Environment 56

▼ To Set ixge Parameters in a Linux Environment 57 ▼ To Configure VLANs in a Linux Environment 59 ▼ To Configure Bonding for Multiple ixge Interfaces 60

Index 61

Contents v

vi Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Tables

TABLE 1-1 Hardware and Software Requirements for Sun SPARC Servers 2 TABLE 3-1 ixge Driver Parameter, Status, and Descriptions 20 TABLE 3-2 Read-Write Flow Control Keyword Descriptions 21 TABLE 3-3 RX Blanking Register for Alias Read 21 TABLE 3-4 accept-jumbo Parameters 22 TABLE 3-5 Read-Only ixge Device Capabilities 29 TABLE 3-6 Receive and Transmit Parameters 30 TABLE A-1 Performance Specifications 43 TABLE A-2 Physical Characteristics 44 TABLE A-3 Power Requirements 44 TABLE B-1 SunVTS Documentation 46 TABLE C-1 Files and Directories on the CD-ROM 50 TABLE D-1 Tunable ixge Driver Parameters for Linux Operating Systems 58

vii

viii Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Figures

FIGURE 1-1 Sun 10-Gigabit Ethernet PCI-X Adapter with Optical Transceiver Installed 2 FIGURE 1-2 Sun 10-Gigabit Ethernet PCI-X Adapter Fiber Optic Channels 3 FIGURE 4-1 Example of Servers Supporting Multiple VLANs With Tagging Adapters 38 FIGURE 4-2 Ethernet Tag Header Format 40

x Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Declaration of Conformity

Compliance Model Number: Kirkwood Product Family Name:

EMC

USA—FCC Class A

This equipment complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:

1. This equipment may not cause harmful interference.

2. This equipment must accept any interference that may cause undesired operation.

European Union

This equipment complies with the following requirements of the EMC Directive 89/336/EEC:

As Telecommunication Network Equipment (TNE) in Both Telecom Centers and Other Than Telecom Centers per (as applicable):

EN300-386 V.1.3.1 (09-2001) Required Limits:

EN55022/CISPR22 Class A EN61000-3-2 Pass EN61000-3-3 Pass EN61000-4-2 6 kV (Direct), 8 kV (Air) EN61000-4-3 3 V/m 80-1000MHz, 10 V/m 800-960 MHz, and 1400-2000 MHz EN61000-4-4 1 kV AC and DC Power Lines, 0.5 kV Signal Lines EN61000-4-5 2 kV AC Line-Gnd, 1 kV AC Line-Line and Outdoor Signal Lines, 0.5 kV Indoor signal Lines > 10m EN61000-4-6 3 V EN61000-4-11 Pass

As Information Technology Equipment (ITE) Class A per (as applicable):

EN55022:1998/CISPR22:1997 Class A EN55024:1998 Required Limits:

EN61000-4-2 4 kV (Direct), 8 kV (Air) EN61000-4-3 3 V/m EN61000-4-4 1 kV AC Power Lines, 0.5 kV Signal and DC Power Lines EN61000-4-5 1 kV AC Line-Line and Outdoor Signal Lines, 2 kV AC Line-Gnd, 0.5 kV DC Power Lines EN61000-4-6 3 V EN61000-4-8 1 A/m

EN61000-4-11 Pass EN61000-3-2:1995 + A1, A2, A14 Pass EN61000-3-3:1995 Pass

Safety: This equipment complies with the following requirements of the Low Voltage Directive 73/23/EEC:

EC Type Examination Certificates:

EN60950:2000, 3rd Edition TÜV Rheinland Certificate No. xxxxxxxxxxxx IEC 60950:2000, 3rd Edition CB Scheme Certificate No. xxxxxxxxxxxx Evaluated to all CB Countries UL 60950, 3rd Edition, CSA C22.2 No. 60950-00 File: Vol. Sec. UL 60950, 3rd Edition, CSA C22.2 No. 950-00 File: Vol. Sec. FDA DHHS Accession Number (Monitors Only)

Supplementary Information: This product was tested and complies with all the requirements for the CE Mark.

Sun 10-GB Ethernet PCIX (X5558A

/S/ /S/ Dennis P. Symanski DATE

Manager, Compliance Engineering Sun Microsystems, Inc. 4150 Network Circle, MPK15-102 Santa Clara, CA 95054 U.S.A. Tel: 650-786-3255 Fax: 650-786-3723

Pamela J. Dullaghan DATE Quality Program Manager Sun Microsystems Scotland, Limited Springfield, Linlithgow West Lothian, EH49 7LR Scotland, United Kingdom Tel: +44 1 506 672 395 Fax: +44 1 506 670 011

xii Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Preface

The Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide provides installation instructions for the Sun 10-Gigabit Ethernet PCI-X adapter. This document also describes how to configure the driver software.

These instructions are designed for enterprise system administrators with experience installing network hardware and software.

How This Book Is Organized

Chapter 1 provides a description of the adapter, including hardware and software.

Chapter 2 describes how to install the adapter in your system, verify that it is correctly installed, and install the driver software using the automated script.

Chapter 3 describes how to configure the driver parameters used by the Sun 10Gigabit Ethernet PCI-Xadapter.

Chapter 4 explains VLANs in detail, and provides configuration instructions and examples.

Appendix A provides specifications for the Sun 10-Gigabit Ethernet PCI-Xadapter.

Appendix B provides an overview of the SunVTS diagnostic application and instructions for updating the SunVTS software to recognize the adapter.

Appendix C describes how to install the driver software manually.

Appendix D explains how to install and configure the Sun 10-Gigabit Ethernet PCIX device driver in a Linux environment.

xiii

Using UNIX Commands

This document might not contain information on basic UNIX®commands and procedures such as shutting down the system, booting the system, and configuring devices. Refer to the following for this information:

■ Software documentation that you received with your system

■ Solaris™ Operating System documentation, which is at:

http://docs.sun.com

Shell Prompts

Shell Prompt

C shell machine-name% C shell superuser machine-name# Bourne shell and Korn shell $ Bourne shell and Korn shell superuser #

xiv Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Typographic Conventions

Typeface

AaBbCc123 The names of commands, files,

AaBbCc123

AaBbCc123 Book titles, new words or terms,

* The settings on your browser might differ from these settings.

Meaning Examples

Edit your.login file.

and directories; on-screen computer output

What you type, when contrasted with on-screen computer output

words to be emphasized. Replace command-line variables with real names or values.

Use ls -a to list all files.

% You have mail. % su

Password:

Read Chapter 6 in the User’s Guide. These are called class options. Yo u must be superuser to do this. To delete a file, type rm filename.

Accessing Sun Documentation

You can view, print, or purchase a broad selection of Sun documentation, including localized versions, at:

http://www.sun.com/documentation

Third-Party Web Sites

Sun is not responsible for the availability of third-party web sites mentioned in this document. Sun does not endorse and is not responsible or liable for any content, advertising, products, or other materials that are available on or through such sites or resources. Sun will not be responsible or liable for any actual or alleged damage or loss caused by or in connection with the use of or reliance on any such content, goods, or services that are available on or through such sites or resources.

Preface xv

Contacting Sun Technical Support

If you have technical questions about this product that are not answered in this document, go to:

http://www.sun.com/service/contacting

Sun Welcomes Your Comments

Sun is interested in improving its documentation and welcomes your comments and suggestions. You can submit your comments by going to:

http://www.sun.com/hwdocs/feedback

Please include the title and part number of your document with your feedback:

Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide, part number 8xxxxxx-xx

xvi Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

CHAPTER

Product Overview

This chapter contains an overview of the Sun 10-Gigabit Ethernet PCI-X adapter, including:

■ “Product Description” on page 1

■ “Hardware and Software Requirements” on page 2

■ “Features” on page 3

Product Description

The Sun 10-Gigabit Ethernet PCI-X adapter is a low profile 133 MHz 10 gigabit Ethernet card utilizing the Intel 82597EX PCI-X MAC controller with XFP-based 10gigabit Ethernet optics.

FIGURE 1-1 shows the adapter with the optical transceiver inserted.

Note – The adapter must be inserted into the system first. Then insert the optical

transceiver into the adapter.

FIGURE 1-1 Sun 10-Gigabit Ethernet PCI-X Adapter with Optical Transceiver Installed

Hardware and Software Requirements

Before installing the adapter, make sure your system meets the required hardware and software.

TABLE 1-1 lists the supported hardware and software for Sun SPARC

systems.

TABLE1-1 Hardware and Software Requirements for Sun SPARC Servers

Requirements Hardware or Software

Hardware Sun Fire™ V440, 490, 880, V890, T2000

Sun Fire V40z, X4200

Operating System Solaris 9 9/04, and 10 SPARC Operating System

RedHat Enterprise Linux 3.0 and 4.0 SuSe Linux Enterprise Server 9.0 SP1

Note – The preceding information is up-to-date as of the printing of this manual.

Visit Sun on the World Wide Web at http://www.sun.com/ for the latest information.

2 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

FIGURE 1-2 Sun 10-Gigabit Ethernet PCI-X Adapter Fiber Optic Channels

Features

The Sun 10-Gigabit Ethernet PCI-X adapter addresses the following requirements and provides additional features and benefits:

■ One full-duplex 10 gigabit Ethernet interface that uses a small form factor

pluggable (XFP) optical transceiver duplex LC fiber connector

■ PCI-X -133MHz compatible and PCI-66 MHz compliant

■ Uses the Intel 82597EX PCI-X MAC controller

■ Uses XAUI to XFI Serdes

■ Dynamic reconfiguration (DR) and redundancy/failover support

■ kstat and snoop support

■ Support for promiscuous and multicast modes

■ Jumbo frames support up to 8000 bytes

■ Low profile 64-bit PCI-X plug-in card

■ PCI Local Bus Rev 2.3-compliant (6.6-inch x 2.536-inch short card)

■ Implements PCI-X addendum to PCI Local Bus specifications, Revision 1.0A

■ IPv4, IPv6, and IPMP support

■ TCP, UDP, and IP hardware checksum offload

■ 802.1Q VLAN support

■ IEEE 802.3ad link aggregation (trunking)

■ NDD physical layer management configuration capability

Chapter 1 Product Over view 3

4 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

CHAPTER

Installing the Adapter

This chapter describes how to install the Sun 10-Gigabit Ethernet PCI-X adapter in your system and verify that it has been installed correctly. The chapter then describes how to install the driver software using the automated script. Finally, this chapter describes how to edit the network host files after installing the adapter on your system.

This chapter contains the following sections:

■ “Installing the Adapter Without Dynamic Reconfiguration” on page 5

■ “Installing the Adapter With Dynamic Reconfiguration” on page 11

■ “Using the Installation Script” on page 11

■ “Verifying the Software Installation” on page 14

■ “Configuring the Network Host Files” on page 15

Note – If you are installing the Sun 10-Gigabit Ethernet PCI-X adapter in a machine

running the Solaris x86 Operating System Dynamic Reconfiguration (DR) is not supported.

Installing the Adapter Without Dynamic Reconfiguration

Caution – If your are installing the Sun 10-Gigabit Ethernet PCI-X adapter into a

machine running Solaris 10, you must install the software before you install the hardware. Failure to install the software before installing the hardware will allow the Sun 10-Gigabit Ethernet PCI-X adapter to attach to an older version of the software. See “Using the Installation Script” on page 11.

▼ To Install the Adapter

The following instructions describe the basic tasks required to install the adapter. Refer to your system installation or service manual for detailed PCI adapter installation instructions.

1. Halt and power off your system.

2. Power off all of the peripherals connected to your system.

3. Open the system unit.

4. Attach the antistatic wrist strap to yourself and to the system.

Attach the adhesive copper strip of the antistatic wrist strap to the metal casing of the power supply. Wrap the other end twice around your wrist, with the adhesive side against your skin.

5. Holding the PCI adapter by the edges, unpack it and place it on an antistatic surface.

6. Using a No. 2 Phillips screwdriver, remove the PCI filler panel from the slot in which you want to insert the PCI adapter.

Save the filler panel screw for Step 9.

7. Insert the adapter into the PCI opening.

Holding the PCI adapter by the edges, align the adapter edge connector with a 66 MHz PCI slot or a PCI-X slot. Slide the adapter face plate into the small slot at the end of the PCI opening.

8. Applying even pressure at both corners of the adapter, push the PCI adapter until it is firmly seated in the slot.

Caution – Do not use excessive force when installing the adapter into the PCI slot.

You might damage the adapter’s PCI connector. If the adapter does not seat properly when you apply even pressure, remove the adapter and carefully reinstall it.

9. Secure the adapter to the PCI slot using the screw you removed in Step 6.

10. Close the system.

Note – Green LEDs do not light until the correct driver is installed.

6 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Install the Optical Transceiver

The Sun 10-Gigabit Ethernet PCI-Xadapter requires an optical transceiver to create and Ethernet connection. The optical transceiver Part Number: 135-1172-01 is available from Sun Microsystems.

1. Pull the white locking handle into the full horizontal position.

You will feel the handle click into position when it is fully opened.

2. Holding the optical transceiver by the edges, align the transceiver with the slot in the Sun 10-Gigabit Ethernet PCI-Xadapter and slide it into the opening.

3. Applying even pressure at both corners of the transceiver, push the transceiver until it is firmly seated in the slot.

Chapter 2 Installing the Adapter 7

4. Push the handle closed to lock the optical transceiver in place.

5. Detach the wrist strap.

Caution – If you pull the locking handle down when the optical transceiver is

installed, remove the optical transceiver entirely and reinstall it. The handle operates an internal lock. Pulling the handle down can disconnect the optical transceiver, even though it might appear to be connected.

6. Connect the Ethernet cable.

8 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Verify the Hardware Installation

After you have installed the Sun 10-Gigabit Ethernet adapter, but before you boot your system, perform the following tasks to verify the installation. Refer to the your Solaris documentation for the detailed instructions.

Note – Verification is not required if your system supports dynamic reconfiguration

(DR). Verification is not supported if your system is running Solaris x86 software.

1. Power on the system, and when the banner appears, press the Stop-A key sequence to interrupt the boot process and display the OpenBoot (ok) prompt.

2. List the network devices on your system.

ok show-devs

The output from this command is extensive. Look for ethernet@ and

network@ to

see the network devices. In this example, the ethernet@1 device is the Sun 10Gigabit Ethernet adapter, shown in bold italics for ease of use.

ok show-devs

.............................................................

/packages /pci@9,600000/pci@2

/pci@9,600000/ethernet@1

/pci@9,600000/pci@2/network@3 /pci@9,600000/pci@2/network@2 /pci@9,600000/pci@2/network@1 /pci@9,600000/pci@2/network@0 /pci@9,700000/usb@1,3

................................................................

/pci@9,600000/ethernet@1 identifies the 10-gigabit Ethernet port.

Note – If you do not see the device listed, check that the adapter is properly seated.

If necessary, reinstall the adapter.

3. View the device that you installed.

Using the previous example, type:

cd /pci@9,600000/ethernet@1

Chapter 2 Installing the Adapter 9

4. View the .properties file for a list of device properties.

The .properties command displays the specific information about the installed adapter. If you are using the Sun 10-Gigabit Ethernet adapter, your output will be similar to the following:

ok cd /pci@9,600000/ethernet@1 ok .properties assigned-addresses 83000810 00000000 00100000 00000000 00008000 81000820 00000000 00000300 00000000 00000020 reg 00000800 00000000 00000000 00000000 00000000 03000810 00000000 00000000 00000000 00008000 01000820 00000000 00000000 00000000 00000020 compatible 70 63 69 38 30 38 36 2c 31 30 34 38 2e 38 30 38 name ethernet 66mhz-capable devsel-speed 00000001 class-code 00020000 interrupts 00000001 latency-timer 00000040 cache-line-size 00000010 max-latency 00000000 min-grant 000000ff subsystem-id 00007036 subsystem-vendor-id 0000108e revision-id 00000002 device-id 00001048 vendor-id 00008086 ok

5. Type the following when you finish looking at the .properties values:

ok device-end

Rebooting the System

After verifying the adapter installation, use the boot -r command to perform a reconfiguration boot on your system.

ok boot -r

10 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Installing the Adapter With Dynamic Reconfiguration

If you have a Sun Enterprise™ system that supports dynamic reconfiguration (DR), you do not have to reboot your system after installing the adapter.

The process of adding and configuring an adapter with DR involves connecting the attachment point and configuring its occupant. In most cases, the cfgadm(1M) command can perform both steps at once.

Note – If you have a Sun Enterprise system that supports dynamic reconfiguration

(DR), refer to the Sun Enterprise Dynamic Reconfiguration User’s Guide and your system’s documentation for further information about dynamic reconfiguration. DR is not supported in the Solaris x86 operating system.

Using the Installation Script

The Sun 10-Gigabit Ethernet CD provides automated installation for the Sun 10Gigabit Ethernet PCI-X driver software. In most cases, you can use the following procedure and execute the installation script. If you have any problems, see Appendix C for manual installation procedures.

The script first checks to ensure that the software driver package has already been installed. If the package is installed, the script verifies the patch level and updates the driver with the updated patch if necessary. If the packages are not installed on Solaris 9 or 10 software, the installation script installs the latest prepatched packages. If the system does not require updating because the patch level is at or beyond the current installed version, the script exits with an appropriate message.

▼ To Install the Software Using the Installation

Script

1. At the system console, become superuser (root).

2. Insert the Sun 10-Gigabit Ethernet Adapter CD into a CD-ROM drive that is connected to your system.

Chapter 2 Installing the Adapter 11

■ If your system is running Sun Enterprise Volume Manager™, it should

automatically mount the CD-ROM to the /cdrom/cdrom0 directory.

■ If your system is not running Sun Enterprise Volume Manager, mount the CD-

ROM as follows:

# mount -F hsfs -o ro /dev/dsk/c0t6d0s2 /cdrom/cdrom0

Note – If you are installing the software from the download site, follow the online

download instructions.

3. Change to the directory were the installation script resides:

# cd /media-path/

where:

■ media-path = /cdrom/cdrom0, if you are installing from the CD-ROM.

■ unzipped 10-Gigabit-path/Sun_10-Gigabit_Ethernet_Driver, if you

downloaded the driver from the Sun download center.

4. Execute the installation script:

# ./install

Note – After the installation script runs, you see a summary of the software package

or patch that was installed. You can also find a log of the installation in /var/tmp. The log file information is displayed when the installation completes unless the software meets or exceeds the required package or patch level.

You’ll see the following output

CODE EXAMPLE 2-1 Typical Install Script Output

# ./install

10-Gigabit Ethernet Adapter Driver for sparc Installation.

12 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

CODE EXAMPLE 2-1 Typical Install Script Output (Continued)

*** Checking for correct version of 10-Gigabit Ethernet Driver Packages...

*** Checking for correct version of VLAN Utility Driver Packages.. *** Checking for correct version of VLAN Utility Driver Patches...

The following optional supporting driver(s) can be used with 10Gigabit Ethernet Adapter Driver:

VLAN Utility Driver software (Installed)

This script is about to take the following actions:

- Install 10-Gigabit Ethernet Adapter Driver packages.

- Install VLAN Ethernet Utility Patch: 114600-04.

Press return to continue, or 'Q' followed by a return to quit:

*** Installing patch 114600-04 for Solaris 9...

Checking installed patches... Verifying sufficient filesystem capacity (dry run method)... Installing patch packages...

Patch number 114600-04 has been successfully installed. See /var/sadm/patch/114600-04/log for details

Patch packages installed: SUNWvld SUNWvldu SUNWvldx

*** Installing Sun 10-Gigabit Ethernet packages... Copyright 2004 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. System configuration files modified but ixge driver not loaded or attached.

Installation of <SUNWixgea> was successful.

Chapter 2 Installing the Adapter 13

CODE EXAMPLE 2-1 Typical Install Script Output (Continued)

Installation completed.

10-Gigabit Ethernet Driver packages have been installed.

Optional VLAN Utility has been updated using 114600-04. It may be necessary to reboot the system to reload the updated driver.

A log of this Install can be found at: /var/tmp/10_Gigabit.install.2004.12.10.1029

5. Reboot the system to load the driver:

# reboot -r

6. Verify the installation to ensure that the driver installed and loaded properly.

Verifying the Software Installation

Unless you have the latest version of the driver software installed, the green LEDs on the adapter will not go on. Be sure that you have version 1.30 or later installed.

● Use the modinfo command to verify that you have the current driver (v1.30 or

later) installed.

If you type grep IXGE, using uppercase letters, you will get only the specific driver information:

# modinfo | grep IXGE 267 1331c11 11e0e 271 1 ixge (IXGE 10Gb Ethernet Driver v1.30)

14 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Configuring the Network Host Files

After installing the driver software, you must create a hostname.ixgenumber file where number = the instance number of the ixge interface.) for the adapter’s Ethernet interface. You must also create both an IP address and a host name for its Ethernet interface in the /etc/hosts file.

Caution – Ensure that the optical transceiver is correctly seated before configuring

the interface.

1. At the command line, use the grep command to search the /etc/path_to_inst file for ixge interfaces.

# grep ixge /etc/path_to_inst "/pci@9,600000/ethernet@1" 0 "ixge"

In the example above, the device instance is from a Sun 10-Gigabit Ethernet PCI-X adapter. For clarity, the instance number is in bold italics.

2. Use the ifconfig command to set up the adapter’s ixge interface.

Use the ifconfig command to assign an IP address to the network interface. Type the following at the command line, replacing ip-address with the adapter’s IP address:

# ifconfig ixge0 plumb ip-address up

Caution – If the optical transceiver is unplugged after the interface has been

configured using the ifconfig command, you must unconfigure and reconfigure the interface.

On Solaris operating systems, use the following commands to unconfigure and reconfigure the interface:

# ifconfig ixge0 down # ifconfig ixge0 unplumb # ifconfig ixge0 plumb ip-address up

Chapter 2 Installing the Adapter 15

On Linux operating systems, use the following commands:

ifconfig ethnumber down ifconfig eth

number up

Where number corresponds to the device instance of the Sun 10-Gigabit Ethernet PCI-X adapter you plan to configure.

Refer to the ifconfig(1M) man page and the Solaris or Linux documentation for more information.

■ If you want a setup that remains the same after you reboot, create an

/etc/hostname.ixgenumber file, where number corresponds to the instance number of the ixge interface you plan to use.

To use the adapter’s ixge interface in the Step 1 example, create an /etc/hostname.ixge0 file, where 0 is the number of the ixge interface. If the instance number were 1, the filename would be /etc/hostname.ixge1.

■ Do not create an /etc/hostname.ixgenumber file for a Sun 10-Gigabit Ethernet

adapter interface you plan to leave unused.

■ The /etc/hostname.ixgenumber file must contain the hostname and IP address

for the appropriate ixge interface.

■ The host name and IP address must be listed in the /etc/hosts file.

■ The host name must be different from any other host name of any other interface,

for example: /etc/hostname.ixge0 and /etc/hostname.ixge1 cannot share the same host name.

The following example shows the /etc/hostname.ixgenumber file required for a system called zardoz that has a Sun 10-Gigabit Ethernet adapter (zardoz-11).

# cat /etc/hostname.hme0 zardoz # cat /etc/hostname.ixge0 zardoz-11

16 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

3. Create an appropriate entry in the /etc/hosts file for each active ixge interface.

For example:

# cat /etc/hosts # # Internet host table #

127.0.0.1 localhost

129.144.10.57 zardoz loghost

129.144.11.83 zardoz-11

Chapter 2 Installing the Adapter 17

18 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

CHAPTER

Configuring Driver Parameters

This chapter describes how to configure the driver parameters used by the Sun 10Gigabit Ethernet PCI-Xadapter. This chapter contains the following sections:

■ “10-Gigabit Ethernet Device Driver Parameters” on page 19

■ “Setting ixge Driver Parameters” on page 23

■ “10-Gigabit Ethernet Driver Operating Statistics” on page 29

10-Gigabit Ethernet Device Driver Parameters

The ixge device driver controls the Sun 10-Gigabit Ethernet PCI-X adapter. The ixge driver is attached to the UNIX pci name property pci8086,1048 for the Sun 10-Gigabit Ethernet PCI-X adapter (8086 is the vendor ID and 1048 is the PCI device ID).

You can manually configure the ixge device driver parameters to customize each Sun 10-Gigabit Ethernet adapter device in your system. This section provides an overview of the capabilities of the 10-Gigabit Ethernet driver used in the adapter, lists the available ixge device driver parameters, and describes how to configure these parameters.

10-Gigabit Ethernet Driver Parameter Values and Definitions

TABLE 3-1 describes the parameters and settings for the ixge device driver.

TABLE3-1 ixge Driver Parameter, Status, and Descriptions

Parameter Status Description

instance Read and write Device instance adv-asmpause-cap Read and write Flow control parameter adv-pause-cap Read and write Flow control parameter rx-intr-pkts Read and write Receive interrupt blanking values rx-intr-time Read and write Receive interrupt blanking values accept-jumbo Read and write Enable jumbo frames

Flow Control Parameters

The ixge device sources (transmits) and terminates (receives) pause frames conforming to the IEEE 802.3x Frame Based Link Level Flow Control Protocol. In response to received flow control frames, the ixge device can slow down its transmit rate. The ixge device is capable of sourcing flow control frames by requesting the link partner to slow down, provided that the link partner supports this feature. By default, the driver advertises both transmit and receive pause capability during autonegotiation.

20 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

TABLE 3-2 provides flow control keywords and describes their function.

TABLE3-2 Read-Write Flow Control Keyword Descriptions

Keyword Description

adv_asmpause_cap The adapter supports asymmetric pause, which means it

can pause only in one direction. 0=Off (default) 1=On

adv_pause_cap This parameter has two meanings, each depending on the

value of adv_asmpause_cap. (Default=0)

•Ifadv_asmpause_cap = 1 while adv_pause_cap =1,

pauses are received.

•Ifadv_asmpause_cap = 1 while adv_pause_cap =0,

pauses are transmitted.

•Ifadv_asmpause_cap = 0 while adv_pause_cap =1,

pauses are sent and received.

•Ifadv_asmpause_cap = 0, then adv_pause_cap

determines whether Pause capability is on or off.

Interrupt Parameters

TABLE 3-3 describes the receive interrupt blanking values.

TABLE3-3 RX Blanking Register for Alias Read

Field Name Values Description

rx_intr_pkts 0 to 511 Interrupt after this number of packets have arrived since

the last packet was serviced. A value of zero indicates no packet blanking. (Default=3)

rx_intr_time 0 to 32768 Interrupt after 4.5 US ticks have elapsed since the last

packet was serviced. A value of zero indicates no time blanking. (Default=0)

Chapter 3 Configuring Driver Parameters 21

Jumbo Frames

The variable accept-jumbo enables the driver to instruct the upper layers that the driver is operating in Jumbo-frame mode.

TABLE3-4 accept-jumbo Parameters

Parameter Values Description

accept-jumbo 1 Enables Jumbo-frame mode and sets MTU to

greater than 1500.

0 Jumbo frames is not enabled. MTU is at

1500.

Refer to the documentation that came with your switch for exact commands to configure jumbo frames support.

▼ To Configure Jumbo Frames Using ndd

1. Unplumb the interface you want to configure Jumbo Frames on.

2. Kill any snoop or netstat processes running over the interface to be configured.

3. Set the instance number:

% ndd -set /dev/ixge instance 0

In this example, the instance number set is 0.

4. Set the instance to accept jumbo frames:

% ndd -set /dev/ixge accept-jumbo 1

5. Plumb the interface up:

% ifconfig ixge0 plumb xxx.xxx.xx.xxx up

where

xxx.xxx.xx.xxx = the IP address of the interface.

22 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Configure Jumbo Frames in a Linux

Environment

1. Ensure that the ixge software is installed.

2. Plumb the Sun 10-Gigabit Ethernet PCI-X interface:

# modprobe ixge # ifconfig eth2 xxx.xxx.xx.xxx up # ifconfig eth2 mtu 8170

where

xxx.xxx.xx.xxx = the IP address of the interface.

Setting ixge Driver Parameters

You can set the ixge device driver parameters in two ways:

■ Using the ndd utility

■ Using the ixge.conf file

If you use the ndd utility, the parameters are valid only until you reboot the system. This method is good for testing parameter settings.

To set parameters so they remain in effect after you reboot the system, create a /platform/sun4u/kernel/drv/ixge.conf file and add parameter values to this file when you need to set a particular parameter for a device in the system.

Setting Parameters Using the ndd Utility

Use the ndd utility to configure parameters that are valid until you reboot the system. The ndd utility supports any networking driver that implements the Data Link Provider Interface (DLPI).

The following sections describe how you can use the ixge driver and the ndd utility to modify (with the -set option) or display (without the -set option) the parameters for each ixge device.

Chapter 3 Configuring Driver Parameters 23

▼ To Specify Device Instances for the ndd Utility

Before you use the ndd utility to get or set a parameter for a ixge device, you must specify the device instance for the utility.

1. Check the /etc/path_to_inst file to identify the instance associated with a

particular device.

# grep ixge /etc/path_to_inst "/pci@9,600000/ethernet@1" 0 "ixge"

In this example, the 10-Gigabit Ethernet instance is from the installed adapter. The instance number is in bold italics for clarity.

2. Use the instance number to select the device.

# ndd -set /dev/ixge instance number

The device remains selected until you change the selection.

Noninteractive and Interactive Modes

You can use the ndd utility in two modes:

■ Noninteractive

■ Interactive

In noninteractive mode, you invoke the utility to execute a specific command. Once the command is executed, you exit the utility. In interactive mode, you can use the utility to get or set more than one parameter value. Refer to the ndd(1M) man page for more information.

▼ To Use the ndd Utility in Noninteractive Mode

This section describes how to modify and display parameter values.

24 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

● To modify a parameter value, use the -set option.

If you invoke the ndd utility with the -set option, the utility passes value, which must be specified, down to the named /dev/ixge driver instance, and assigns the value to the parameter:

# ndd -set /dev/ixge parameter value

When you change any parameter, a message similar to the following appears:

ixge0: xcvr addr:0x01 - link up 10000 Mbps full duplex

● To display the value of a parameter, specify the parameter name and omit the

value.

When you omit the -set option, the utility queries the named driver instance, retrieves the value associated with the specified parameter, and prints it:

# ndd /dev/ixge parameter

▼ To Use the ndd Utility in Interactive Mode

● To modify a parameter value in interactive mode, specify ndd /dev/ixge:

# ndd /dev/ixge name to get/set? (Enter the parameter name or ? to view all parameters)

After you enter the parameter name, the ndd utility prompts you for the parameter value (see

TABLE 3-1 through TABLE 3-6).

Chapter 3 Configuring Driver Parameters 25

● To list all the parameters supported by the ixge driver, type ?.

TABLE 3-1 through TABLE 3-6 for parameter descriptions.

See

# ndd -get /dev/ixge \? ? (read only) instance (read and write) adv_asmpause_cap (read and write) adv_pause_cap (read and write) rx_intr_pkts (read and write) rx_intr_time (read and write) accept_jumbo (read and write) laggr_multistream (read and write)#

Note – By default, autonegotiation is set to off. This adapter operates only at full-

duplex.

Setting Parameters Using the ixge.conf File

Specify the driver parameter properties for each device by creating a ixge.conf file in the /platform/sun4u/kernel/drv directory. Use a ixge.conf file when you need to set a particular parameter for a device in the system. The parameters you can set are the read and write parameters listed in “10-Gigabit Ethernet Driver Parameter Values and Definitions” on page 20.

Note – Configuring the parameters by putting ndd commands in rcX.d scripts is

not supported.

The man pages for prtconf(1M) and driver.conf(4) include additional details. The next procedure shows an example of setting parameters in a ixge.conf file.

● To access any man page, type the man command plus the name of the man page.

For example, to access man pages for prtconf(1M), type:

% man prtconf

26 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Set Driver Parameters Using a ixge.conf

File

1. Obtain the hardware path names for the ixge devices in the device tree. a. Check the /etc/driver_aliases file to identify the name associated with a

particular device:

# grep ixge /etc/driver_aliases ixge "pci8086,1048.108e.7036"

Note – ixge "pci8086,1048.108e.7036" is used by the Sun 10-Gigabit Ethernet

adapter.

b. Locate the path names and the associated instance numbers in the

/etc/path_to_inst file.

# grep ixge /etc/path_to_inst "/pci@9,600000/ethernet@1" 0 "ixge"

Chapter 3 Configuring Driver Parameters 27

In this example:

■ The first part within the double quotes specifies the hardware node name in

the device tree.

■ The number not enclosed in quotes is the instance number (shown in bold

italics).

■ The last part in double quotes is the driver name.

■ In the hardware node name, the last component after the last / character and

before the @ character is the device name.

■ The path name before the last component is the parent name.

■ The number after the final @ character within quotes is the unit-address.

To identify a PCI device unambiguously in the ixge.conf file, use the name, parent name, and the unit-address for the device. Refer to the pci(4) man page for more information about the PCI device specification.

2. Set the parameters for the above devices in the

/platform/sun4u/kernel/drv/ixge.conf file.

In the following example, the accept-jumbo parameter is set to enable jumbo frames for all Sun 10-Gigabit Ethernet devices. See the driver.conf(4) man page for more information.

accept-jumbo=1;

In the following example, the accept-jumbo parameter is set to enable jumbo frames is set for a single instance of the Sun 10-Gigabit Ethernet device.

In this example:

■ parent = "/pci@8,600000"

■ unit-address = "0"

name="pci8086,1048.108e.7036" parent="pci@8,600000" unit-address= "0" accept-jumbo=1;

Note – If you omit the name=, parent=, and unit-address= definitions, the

settings become global to all Sun 10-Gigabit Ethernet instances.

3. Save the ixge.conf file.

28 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

10-Gigabit Ethernet Driver Operating Statistics

These statistics are part of the statistics presented by the kstat command.

TABLE 3-5 describes the read-only Media Independent Interface (MII) capabilities.

These parameters define the capabilities of the hardware. The Gigabit Media Independent Interface (GMII) supports all of the following capabilities.

TABLE3-5 Read-Only ixge Device Capabilities

Parameter Description (Local interface Capabilities)

cap_autoneg 0 = Not capable of autonegotiation

1 = Autonegotiation capable

cap_10gfdx Local interface full-duplex capability

0 = Not 10000 Mbit/sec full-duplex capable 1 = 10000 Mbit/sec full-duplex capable

cap_10ghdx Local interface half-duplex capability

0 = Not 10000 Mbit/sec half-duplex capable 1 = 10000 Mbit/sec half-duplex capable

cap_1000fdx Local interface full-duplex capability

0 = Not 1000 Mbit/sec full-duplex capable 1 = 1000 Mbit/sec full-duplex capable

cap_1000hdx Local interface half-duplex capability

0 = Not 1000 Mbit/sec half-duplex capable 1 = 1000 Mbit/sec half-duplex capable

cap_100fdx Local interface full-duplex capability

0 = Not 100 Mbit/sec full-duplex capable 1 = 100 Mbit/sec full-duplex capable

cap_100hdx Local interface half-duplex capability

0 = Not 100 Mbit/sec half-duplex capable 1 = 100 Mbit/sec half-duplex capable

cap_10fdx Local interface full-duplex capability

0 = Not 10 Mbit/sec full-duplex capable 1 = 10 Mbit/sec full-duplex capable

Chapter 3 Configuring Driver Parameters 29

TABLE3-5 Read-Only ixge Device Capabilities (Continued)

Parameter Description (Local interface Capabilities)

cap_10hdx Local interface half-duplex capability

0 = Not 10 Mbit/sec half-duplex capable 1 = 10 Mbit/sec half-duplex capable

cap_asm_pause Local interface flow control capability

0 = Not asymmetric pause capable 1 = Asymmetric pause (from the local device) capable

cap_pause Local interface flow control capability

0 = Not Symmetric pause capable 1 = Symmetric pause capable

Table describes the read-only ixge device capabilities.

TABLE 3-6 describes the kstat ixge:1 receive and transmit parameters:

TABLE3-6 Receive and Transmit Parameters

Parameter Description

lb_mode Copy of the Loopback mode the device is in, if any. rev_id Revision ID of the 10-Gigabit Ethernet device useful for

recognition of device being used in the field.

rx_allocb_fail Number of times the driver failed to allocate OS receive buffer. rx_hdr_drops Number of times an entire page of packets less than 256 bytes

each was dropped because the driver was unable to map a new page to replace it.

rx_hdr_pkts Number of packets received that were less than 256 bytes. rx_inits Number of packets received rx_jumbo_pkts Number of jumbo packets received. rx_len_mm Receive length error count. rx_msgdup_fail Number of times driver failed to duplicate a packet on receipt. rx_mtu_drops Number of times an entire page of packets greater than 256 bytes

and less than 1514 each was dropped because the driver was unable to map a new page to replace it.

rx_new_hdr_pgs Number of pages that were filled with packets less than 256 bytes

that got replaced during reception.

rx_new_mtu_pgs Number of pages that were filled with packets greater than 256

bytes and less than 1514 that got replaced during reception.

rx_new_nxt_pgs Number of pages that contained packets that were split across

pages that got replaced during reception.

30 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

TABLE3-6 Receive and Transmit Parameters

Parameter Description

rx_new_pages Number of pages that got replaced during reception. rx_no_buf Number of times the hardware cannot receive data because there

is no more receive buffer space.

rx_nocanput Number of packets dropped due to full STREAMS message

queue.

rx_no_comp_wb Number of times the hardware cannot post completion entries for

received data.

rx_nxt_drops Number of times a page with a split packet was dropped because

the driver was unable to map a new one to replace it.

rx_ov_flow Number of packets dropped due to full h/w FIFO. rx_pkts_dropped Number of packets dropped due to service FIFO. rx_reused_pgs Number of times driver able to reuse a system page. rx_split_pkts Number of packets that were split across two pages. snaptime System snapshot time in seconds. trunk_mode A value of 1 indicates that a link is aggregated. tx_allocb_fail Number of times driver failed to duplicate a packet on transmit. tx_ddi_pkts Number of packets that used DDI_DMA. tx_dma_bind_fail Number of times a page table entry was not available to enable

the driver to map the kernel memory to device-accessible memory for transmission.

tx_dvma_pkts Number of packets that used dvma. tx_hdr_pkts Number of packets that used bcopy (buffer copy). tx_jumbo_pkts Number of packets greater that Ethernet MTU of 1522. tx_max_pend Maximum number of transmits pending on any of the four

queues.

tx_msgdup_fail Number of times driver failed to duplicate a packet for

promiscuous stream.

tx_no_desc Number of tx packets dropped due to full descriptor ring.

_starts Number of times that the driver attempted to transmit a packet.

tx xcvr_addr GMII/MII physical layer device address for management

interface.

Chapter 3 Configuring Driver Parameters 31

TABLE3-6 Receive and Transmit Parameters

Parameter Description

xcvr_id GMII/MII physical layer device Identification Decimal copy of

MII registers 2 and 3.

xcvr_inits Number of physical layer re-initializations incremented each time

link parameters are changed using NDD.

xcvr_inuse Type of Transceiver/SERDES which is always PCS (3).

▼ To Check Link Partner Settings

Type the kstat command as superuser:

CODE EXAMPLE 3-1 Output from kstat Command

# kstat ixge:1 module: ixge instance: 1 name: ixge1 class: net brdcstrcv 0 brdcstxmt 0 cap_1000fdx 0 cap_1000hdx 0 cap_100fdx 0 cap_100hdx 0 cap_100T4 0 cap_10fdx 0 cap_10gfdx 1 cap_10ghdx 0 cap_10hdx 0 cap_asmpause 0 cap_autoneg 0 cap_pause 0 crc_err 0 crtime 147.15892392 ierrors 2 ifspeed 10000000000 ipackets 0 ipackets64 0 lb_mode 0 link_asmpause 0 link_duplex 2 link_pause 0 link_speed 10000 link_T4 0 link_up 1 local_faults 1 mac_mtu 0

32 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

CODE EXAMPLE 3-1 Output from kstat Command (Continued)

multircv 0 multixmt 0 norcvbuf 0 noxmtbuf 0 obytes 0 obytes64 0 oerrors 0 opackets 0 opackets64 0 pci_bad_ack_err 0 pci_bus_speed 100 pci_dmarz_err 0 pci_dmawz_err 0 pci_drto_err 0 pci_err 0 pci_parity_err 0 pci_rma_err 0 pci_rta_err 0 pcix_bus_mode 1 promisc off qos_mode 0 rbytes 0 rbytes64 0 remote_faults 1 rev_id 28726 rx_allocb_fail 0 rx_hdr_drops 0 rx_hdr_pkts 0 rx_inits 0 rx_jumbo_pkts 0 rx_len_mm 0 rx_msgdup_fail 0 rx_mtu_drops 0 rx_mtu_pkts 0 rx_new_hdr_pgs 0 rx_new_mtu_pgs 0 rx_new_nxt_pgs 0 rx_new_pages 0 rx_no_buf 0 rx_nocanput 0 rx_no_comp_wb 0 rx_nxt_drops 0 rx_ov_flow 0 rx_pkts_dropped 0 rx_reused_pgs 0 rx_split_pkts 0 rx_tag_err 0

Chapter 3 Configuring Driver Parameters 33

CODE EXAMPLE 3-1 Output from kstat Command (Continued)

rx_taskq_waits 0 snaptime 7883.09014984 trunk_mode 0 tx_allocb_fail 0 tx_ddi_pkts 0 tx_dma_bind_fail 0 tx_dvma_pkts 0 tx_hdr_pkts 0 tx_inits 0 tx_jumbo_pkts 0 tx_max_pend 0 tx_msgdup_fail 0 tx_nocanput 0 tx_no_desc 0 tx_starts 0 tx_uflo 0 xcvr_addr 0 xcvr_id 0 xcvr_inits 1 xcvr_inuse 3

▼ To Discover Device Capabilities

● Use the kstat command to discover device capabilities.

# kstat ixge:1 | grep cap_ cap_1000fdx 0 cap_1000hdx 0 cap_100fdx 0 cap_100hdx 0 cap_100T4 0 cap_10fdx 0 cap_10gfdx 1 cap_10ghdx 0 cap_10hdx 0 cap_asmpause 0 cap_autoneg 0 cap_pause 0

34 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Discover Link Settings

● Use the kstat command to discover link settings.

# kstat ixge:1 | grep link link_asmpause 0 link_duplex 2 link_pause 0 link_speed 10000 link_T4 0 link_up 1

Configuration Checking

The jumbo frame configuration checking occurs at Layer 2 or Layer 3, depending on the configuration method.

Layer 2 Configuration Checking

You can view the MTU configuration of a ixge instance at any time using the kstat command. The kstat mac_mtu variable represents the complete size of the Ethernet frame, which includes the Ethernet header, maximum payload, and crc. This value is consistent with the definition of MTU presented by switch vendors.

Use the following command to get the information:.

# kstat ixge:0 | grep mac_mtu mac_mtu 8192

Layer 3 Configuration Checking

You can check the Layer 3 configuration by invoking the ifconfig command on a preconfigured interface as shown in the previous examples:.

# ifconfig ixge138000 ixge138000: flags= 201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 8170 index 5

inet 192.12.38.69 netmask ffffff00 broadcast 192.12.38.255

ether 0:3:ba:c4:f7:1d

Chapter 3 Configuring Driver Parameters 35

36 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

CHAPTER

Configuring VLANs

This chapter explains Virtual Local Area Networks (VLANs) in detail and provides configuration instructions and examples. It contains the following sections:

■ “Overview of VLANs” on page 37

■ “Configuring VLANs” on page 39

With multiple VLANs on an adapter, a server with a single adapter can have a logical presence on multiple IP subnets. By default, 128 VLANs can be defined for each VLAN-aware adapter on your server. However, this number can be increased by changing the system parameters.

If your network does not require multiple VLANs, you can use the default configuration, in which case no further configuration is necessary.

Note – If you change any of the VLAN configuration parameters, you must reboot

the system before the changes take effect. If you make changes and do not reboot, you may experience configuration problems.

Overview of VLANs

VLANs enable you to split your physical LAN into logical subparts, providing an essential tool for increasing the efficiency and flexibility of your network.

VLANs are commonly used to separate groups of network users into manageable broadcast domains, to create logical segmentation of workgroups, and to enforce security policies among each logical segment. Each defined VLAN behaves as its own separate network, with its traffic and broadcasts isolated from the others, increasing the bandwidth efficiency within each logical group.

VLAN 1 VLAN 2 VLAN 3

Although VLANs are commonly used to create individual broadcast domains and/or separate IP subnets, it can be useful for a server to have a presence on more than one VLAN simultaneously. Several Sun products support multiple VLANs on a per port or per interface basis, allowing very flexible network configurations.

FIGURE 4-1 shows an example network that uses VLANs.

Main Server Accounting Server (VLAN 3)

Adapter

Gigabit/Tagged

(All VLANs)

Shared Media Segment

Software PC 1 (VLAN 2)

FIGURE 4-1 Example of Servers Supporting Multiple VLANs With Tagging Adapters

Software PC 2 (VLAN 2)

Engineering PC 3 (VLAN 1)

Accounting PC 4 (VLAN 3)

The example network has the following features:

The physical LAN network consists of a switch, two servers, and five clients. The LAN is logically organized into three different VLANs, each representing a different IP subnet.

■ VLAN 1 is an IP subnet consisting of the Main Server, Client 3, and Client 5. This

represents an engineering group.

38 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Engineering/ Software PC 5 Adapter Gigabit/Tagged (VLAN 1 & 2)

■ VLAN 2 includes the Main Server, Clients 1 and 2 by means of a shared media

segment, and Client 5. This is a software development group.

■ VLAN 3 includes the Main Server, the Accounting Server and Client 4. This is an

accounting group.

The Main Server is a high-use server that needs to be accessed from all VLANs and IP subnets. The server has a Sun 10-Gigabit Ethernet adapter installed. All three IP subnets are accessed by means of the single physical adapter interface. The server is attached to one of the switch’s Gigabit Ethernet ports, which is configured for VLANs 1, 2, and 3. Both the adapter and the connected switch port have tagging turned on. Because of the tagging VLAN capabilities of both devices, the sever is able to communicate on all three IP subnets in this network, but continues to maintain broadcast separation between all of them. The following list describes the components of this network:

■ The Accounting Server is available to VLAN 3 only. It is isolated from all traffic

on VLANs 1 and 2. The switch port connected to the server has tagging turned off.

■ Clients 1 and 2 are attached to a shared media hub that is then connected to the

switch. They belong to VLAN 2 only, and are logically in the same IP subnet as the Main Server and Client 5. The switch port connected to this segment has tagging turned off.

■ Client 3 is a member of VLAN 1, and can communicate only with the Main Server

and Client 5. Tagging is not enabled on Client 3’s switch port.

■ Client 4 is a member of VLAN 3, and can communicate only with the servers.

Tagging is not enabled on Client 4’s switch port.

■ Client 5 is a member of both VLANs 1 and 2, and has a Sun 10-Gigabit Ethernet

adapter installed. It is connected to switch port 10. Both the adapter and the switch port are configured for VLANs 1 and 2 and have tagging enabled.

VLAN tagging is only required to be enabled on switch ports that create trunk links to other VLAN-aware Ethernet switches, or on ports connected to tag-capable endstations, such as servers or workstations with VLAN-aware adapters.

Configuring VLANs

VLANs can be created according to various criteria, but each VLAN must be assigned a VLAN tag or VLAN ID (VID). The VID is a 12-bit identifier between 1 and 4094 that identifies a unique VLAN. For each network interface (ixge0, ixge1, ixge2 and so on), 4094 possible VLAN IDs can be selected. Only 512 unique IDs can be used simultaneously. Because IP subnets are commonly used, it is best to use IP

Chapter 4 Configuring VLANs 39

subnets when setting up a VLAN network interface. This means that each VID assigned to a VLAN interface of a physical network interface will belong to different subnets.

Tagging an Ethernet frame requires the addition of a tag header to the frame. The header is inserted immediately following the destination MAC address and the Source MAC address. The tag header consists of two bytes of Ethernet Tag Protocol identifier (TPID, 0x8100) and two bytes of Tag Control Information (TCI).

FIGURE 4-2

shows the Ethernet Tag Header format.

Octet

1 2

3 4

3 bits

TPID (0x8100

12 bytes

bit

User_priority

FIGURE 4-2 Ethernet Tag Header Format

CFI

VID

By default, a single VLAN is configured for every port, which groups all ports into the same broadcast domain, just as if there were no VLANs at all, VLAN tagging for the switch port turned off.

Note – If you configure a VLAN virtual device for an adapter, all traffic sent or

received by that adapter must be in VLAN-tagged format.

▼ To Configure Static VLANs

1. Create one hostname6.ixgenumber file for each VLAN that will be configured for

each adapter on the server.

Use the following naming format, which includes both the VID and the physical point of attachment (PPA):

VLAN logical PPA = 1000 * VID + Device PPA

ixge123000 = 1000*123 + ixge

This format limits the maximum number of PPAs (instances) you can configure to 1000 in the /etc/path_to_inst file.

For example, on a server with the Sun 10-Gigabit Ethernet adapter having an instance of 0, belonging to a member of two VLANs, with VID 123 and 224, you would use ixge123000 and ixge224000, respectively, as the two VLAN PPAs.

40 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

2. Use the ifconfig(1M) to configure a VLAN virtual device, for example:

# ifconfig ixge123000 plumb up # ifconfig ixge4000 plumb up

The output of ifconfig -a on a system having VLAN devices ixge123000 and ixge224000:

# ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 129.144.131.91 netmask ffffff00 broadcast 129.144.131.255 ether 8:0:20:a4:4f:b8 ixge123000: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 199.199.123.3 netmask ffffff00 broadcast 199.199.123.255 ether 8:0:20:a4:4f:b8 ixgee224000: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 199.199.224.3 netmask ffffff00 broadcast 199.199.224.225 ether 8:0:20:a4:4f:b8

3. On the switch, set VLAN tagging and set VLAN ports to coincide with the VLANs you’ve set up on the server.

Using the examples in Step 2, you would set up VLAN ports 123 and 224 on the switch.

Refer to the documentation that came with your switch for specific instructions for setting VLAN tagging and ports.

Chapter 4 Configuring VLANs 41

42 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

APPENDIX

Specifications

This appendix provides specifications for the Sun 10-Gigabit Ethernet PCI-X adapter.

■ “Performance Specifications” on page 43

■ “Physical Characteristics” on page 44

■ “Power Requirements” on page 44

Performance Specifications

TABLEA-1 Performance Specifications

Feature Specification

PCI-X clock 66 to 133 MHz max

PCI clock 66 MHz

PCI I/O voltage 3.3V

PCI data burst transfer rate Up to 64-byte bursts

PCI data/address width 64 bit

PCI modes Master/slave

Maximum Ethernet transfer rate 10 Gbps Ethernet

Optics XG-Base-SR, XG-Base-LR and IEEE 802.ae, 2002

compliant

Physical Characteristics

TABLEA-2 Physical Characteristics

Dimension Measurement

Length 6.6 inches (167.64 mm)

Width 2.536 inches (64.61 mm)

Height Height on component side: 0.57 inches (14.48 mm)

Height on back side :0.105 inches (2.67 mm)

Power Requirements

TABLEA-3 Power Requirements

Specification Measurement

Maximum power consumption 11 Watts

Voltage 3.3V and 5V

44 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

APPENDIX

Diagnostic Software

This appendix provides an overview of the SunVTS diagnostic application and instructions for updating the SunVTS software to recognize the adapter. This appendix contains the following sections:

■ “SunVTS Diagnostic Software” on page 45

■ “Updating SunVTS to Recognize the Adapter” on page 46

■ “Using the SunVTS netlbtest” on page 47

SunVTS Diagnostic Software

The SunVTS software executes multiple diagnostic hardware tests from a single user interface and is used to verify the configuration and functionality of most hardware controllers and devices. The SunVTS software operates primarily from a graphical user interface, enabling test parameters to be set quickly and easily while a diagnostic test operation is being performed.

Note – SunVTS diagnostic software is not currently available for Solaris x86

Operating Systems.

The SunVTS nettest diagnostic can be used to test all of the networking interfaces on the system, including the interfaces on the adapter.

To use the nettest diagnostic, you must have the SunVTS software installed on your system. Refer to your Solaris documentation for installation instructions.

Refer to the SunVTS documentation (listed in run and monitor the nettest diagnostic. These SunVTS documents are available online at the following URL:

TABLE B-1) for instructions on how to

http://www.sun.com/products-n-solutions/hardware/docs/Software /system_administration/tools/testing_valid/sunvts/index.html

Select the document for the Solaris release on your system.

TABLEB-1 SunVTS Documentation

Title Description

SunVTS User’s Guide Describes the SunVTS diagnostic environment

SunVTS Test Reference Manual Describes each SunVTS test (including the nettest)

and describes the various test options and command-line arguments

SunVTS Quick Reference Provides an overview of the user interface

Updating SunVTS to Recognize the Adapter

Current versions of SunVTS do not recognize the Sun 10-Gigabit Ethernet PCI-X adapter. You will need to update the SunVTS configuration.

▼ To Update SunVTS to Recognize the Adapter

1. Plug in a loopback cable.

2. Ensure that the SunVTS software and the ixge driver are installed on your

system.

3. Add the following lines to the /opt/SUNWvts/lib/conf/netlbtest.conf and

/opt/SUNWvts/lib/conf/nettest.conf files:

ixge ixge 10GbE

46 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Using the SunVTS netlbtest

You must have the Ethernet card and the device driver installed, a loopback connector in place, and Intervention mode enabled before running netlbtest. netlbtest cannot run if the network interface is connected to a live network, and requires that the Ethernet device be configured offline before running the test. Use the ifconfig(1M) command to bring the Ethernet device down before running netlbtest.

▼ To Use the netlbtest

1. Ensure that the SunVTS software and the ixge driver are installed on your system.

2. Plug in a loopback cable.

3. Unplumb the interface from the system, using the ifconfig command:

# ifconfig ixgeinstance down # ifconfig ixgeinstance unplumb

Where Refer to SunVTS documentation for instructions on how to run netlbtest.

instance is the instance number of the interface.

Appendix B Diagnostic Software 47

48 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

APPENDIX

Installing the Software Manually

This appendix describes how to install the driver software manually.

The Sun 10-Gigabit Ethernet Adapter CD contains the driver software required to operate the adapter. When you update your Solaris operating system, refer to the documentation that shipped with the Solaris Supplement CD-ROM for a listing of the available network drivers. Install the Sun 10-Gigabit Ethernet driver for your updated version of the Solaris Operating System.

▼ To Install the Driver Software Manually

1. Become superuser.

2. Insert the Sun 10-Gigabit Ethernet Adapter CD into a CD-ROM drive that is connected to your system.

■ If your system is running Sun Enterprise Volume Manager™, the system should

automatically mount the CD-ROM to the /cdrom/cdrom0 directory.

■ If your system is not running Volume Manager, mount the CD-ROM as follows:

# mkdir /cdrom # mkdir /cdrom/cdrom0 # mount -F hsfs -o ro /dev/dsk/c0t6d0s2 /cdrom/cdrom0

You will see the following files and directories in the

/cdrom/sun_10-gigabit_ethernet_driver directory or the /cdrom/cdrom0 directory, depending on how you mounted the CD-ROM.

TABLEC-1 Files and Directories on the CD-ROM

File or Directory Contents

Copyright U.S. copyright file FR_Copyright French copyright file 10_GigabitEthernet/Linux/SRPM/ Contains the Sun 10-Gigabit Ethernet software source

rpm applicable to Linux operating systems:

• sun-10-gigabit-kernel-1.0.version-1.src.rpm

10_GigabitEthernet/Solaris_OS-VER /ARCH/Packages/

Contains the Sun 10-Gigabit Ethernet software packages applicable to SPARC architecture software:

• SUNWixgea—adb macros

• SUNWixged.u—adapter driver

• SUNWixgedu—adapter driver header files

The optional VLAN packages:

• SUNWvld—VLAN utility routines

• SUNWvldx—VLAN utility routines (64-bit, s9)

• SUNWvldu—VLAN utility headers

Note – If you intend to use VLAN, you must install VLAN packages when you

install the software packages.

3. To use the optional VLAN Utility with the 10-Gigabit Ethernet driver on the Solaris 9 Operating System, verify the proper patch revision using the following command:

# showrev -p | grep 114600

If the patch version is less then 114600-04, update the VLAN Utility software by installing the 114600-04 patch.

50 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

4. Install the patch:

# patchadd 10_GigabitEthernet/Solaris_9/sparc/Patches/114600-04

5. Check to see if vlan drivers are installed on the system:

■ If you are using a Solaris SPARC system:

# /usr/bin/pkginfo SUNWvld SUNWvldx system SUNWvld Sun Ethernet Vlan Utility Routines system SUNWvldx Sun Ethernet Vlan Utility Routines (64-bit)

■ If you do not see the previous output, the VLAN Utility software is not installed

on your system. You may install the optional VLAN packages if desired in the following step.

■ If you do see the previous output, do not reinstall the VLAN Utility software.

Select only the Sun 10 Gigabit Ethernet Driver packages.

6. Install the software packages by typing the following at the command line,

replacing the OS_VER with your version of the Solaris operating system:

■ If your system is running Solaris 9 or 10 software type the following:

# /usr/sbin/pkgadd -d /cdrom/cdrom0/10_GigabitEthernet/Solaris_OS_VER/ARCH/Packages/

where OS_VER is9or10andARCH is sparc. For example, if your system is running the Solaris 9 SPARC operating system, you

would type the following:

# /usr/sbin/pkgadd -d /cdrom/cdrom0/10_GigabitEthernet/Solaris_9/sparc/Packages/

Appendix C Installing the Software Manually 51

■ If your system is running Solaris 9 x86 software type the following:

# pkgadd -d /Solaris_9/i386/Packages/

A menu similar to the following displays:

The following packages are available: 1 SUNWixgea Sun 10 Gigabit Ethernet Adapter Driver adb Macros (sparc) 1.0,REV=2004.12.01. 2 SUNWixged.u Sun 10 Gigabit Ethernet Adapter Driver (sparc.sun4u) 1.0,REV=2004.12.01. 3 SUNWixgedu Sun 10 Gigabit Ethernet Adapter Driver Headers (sparc) 1.0,REV=2004.12.01. 4 SUNWvld Sun Ethernet Vlan Utility Routines (sparc) 1.0,REV=2001.09.19.9.0 5 SUNWvldu Sun Ethernet Vlan Utility Headers (sparc) 1.0,REV=2001.09.19.9.0 6 SUNWvldx Sun Ethernet Vlan Utility Routines (64-bit) (sparc) 1.0,REV=2001.09.19.9.0

Select package(s) you wish to process (or 'all' to process all packages). (default: all) [?,??,q]:

7. Select the packages you want to install:

■ Press Return or type all to accept the default and install all packages (if neither

the 10-Gigabit Ethernet Driver nor VLAN Utility packages are installed).

■ Type the numbers of all but the optional packages if you prefer not to install

them.

Note – Do not install VLAN packages if you previously determined that they are

already present on your system.

See

TABLE C-1 to identify the optional packages.

52 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

APPENDIX

Installing the Sun 10-Gigabit Ethernet Device Driver on Linux Platforms

This appendix explains how to install the Sun 10-Gigabit Ethernet PCI-X device driver in a Linux environment. This appendix contains the following sections:

■ “Setting Up the Driver in a Linux Platform” on page 53

■ “Setting Driver Parameters in a Linux Environment” on page 56

Setting Up the Driver in a Linux Platform

If your system uses the Linux operating system you will need to perform the following procedure to be sure the Sun 10-Gigabit Ethernet device driver is properly installed and loaded.

▼ To Set Up the Driver in a Linux Platform

1. Build the 10-Gigabit Ethernet driver:

# rpmbuild --rebuild sun-10-gigabit-kernel-1.0.version.src.rpm

where the version is the minor version number such as:

sun-10-gigabit-kernel-1.0.82.6-1.src.rpm

2. Build the man pages (optional):

# rpmbuild --rebuild sun-10-gigabit-1.0.version.src.rpm

3. Change to the rpm directory:

■ For RedHat, use the following command:

# cd /usr/src/redhat/RPMS/arch

■ For SuSe Linux Enterprise Server, use the following command:

# cd /usr/src/packages/RPMS/arch

4. Install the 10-Gigabit Ethernet rpms, using the same command for both RedHat and Suse:

# rpm -ivh sun-10-gigabit-kernel-1.0.version.arch.rpm

5. Use the depmod command to register the Sun 10-Gigabit Ethernet module:

# depmod

6. Load ixge driver for all instances:

# modprobe ixge

54 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

7. Verify that the driver is loaded.

In the following example, the ixge driver is shown in bold italic. On your system the driver will be indistinguishable from the other modules.

# lsmod

Module Size Used by Not tainted

ixge 53592 0 (unused)

ide-cd 36616 0 (autoclean) ipv6 285176 -1 (autoclean) key 78344 0 (autoclean) [ipv6] st 31380 0 (autoclean) (unused) sr_mod 16812 0 (autoclean) (unused) cdrom 34664 0 (autoclean) [ide-cd sr_mod] sg 40384 0 (autoclean) joydev 7136 0 (unused) evdev 6496 0 (unused) input 7488 0 [joydev evdev] usb-ohci 23728 0 (unused) usbcore 87648 1 [usb-ohci] bcm5700 194456 1 lvm-mod 81248 0 (autoclean) reiserfs 248344 1 mptscsih 44144 2 mptbase 46400 3 [mptscsih] sd_mod 16192 4 scsi_mod 133028 5 [st sr_mod sg mptscsih sd_mod]

8. Run the dmesg command to see which devices the ixge driver was mapped to:

9. Add the ixge interfaces to the /etc/modules.conf file.

alias eth2 ixge

alias eth3 ixge

Appendix D Installing the Sun 10-Gigabit Ethernet Device Driver on Linux Platforms 55

10. Use the ethtool command to check the parameter configurations that apply to the ixge driver.

# ethtool eth3

Supported ports: [ FIBRE ] Supported link modes: Supports auto-negotiation: No Advertised link modes: Not reported Advertised auto-negotiation: No Speed: Unknown! (10000) Duplex: Full Port: FIBRE PHYAD: 0 Transceiver: externel Auto-negotiation: off Link detected: yes

Setting Driver Parameters in a Linux Environment

In a Linux operating system, the driver parameters can only be set at the time the driver is loaded. If you have already loaded the driver and have not set the parameters at the same time. Remove the driver and reinstall it.

56 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Set ixge Parameters in a Linux Environment

1. Remove the Sun 10-Gigabit Ethernet PCI-X device driver, if it is already loaded.

# rmmod ixge

2. Load the Sun 10-Gigabit Ethernet PCI-X device driver:

# modprobe ixge option=VAL1, VAL2, . . .

or you can use the insmod command:

# insmod ixge option=VAL1, VAL2, . . .

where VAL1 applies to the first ixge interface and VAL2 to the next interface and so on.

Appendix D Installing the Sun 10-Gigabit Ethernet Device Driver on Linux Platforms 57

TABLE D-1 lists the tunable ixge driver parameters for Linux operating systems, and

describes their function.

TABLED-1 Tunable ixge Driver Parameters for Linux Operating Systems

Keyword Description

FlowControl Valid Range: 0-3 (0=none, 1=Rx only, 2=Tx only, 3=Rx&Tx)

Default: Read from the EEPROM If EEPROM is not detected, default is 3. This parameter controls the automatic generation (Tx) and response

(Rx) to Ethernet PAUSE frames.

RxDescriptiors Valid Range: 64-4096

Default Value: 1024 This value is the number of receive descriptors allocated by the driver.

Increasing this value allows the driver to buffer more incoming packets. Each descriptor is 16 bytes. A receive buffer is also allocated for each descriptor and can be either 2048, 4056, 8192, or 16384 bytes, depending on the MTU setting. When the MTU size is 1500 or less, the receive buffer size is 2048 bytes. When the MTU is greater than 1500 the receive buffer size will be either 4056, 8192, or 16384 bytes. The maximum MTU size is 16114.

RxIntDelay Valid Range: 0-65535 (0=off)

Default Value: 72 This value delays the generation of receive interrupts in units of 0.8192

microseconds. Receive interrupt reduction can improve CPU efficiency if properly tuned for specific network traffic. Increasing this value adds extra latency to frame reception and can end up decreasing the throughput of TCP traffic. If the system is reporting dropped receives, this value may be set too high, causing the driver to run out of available receive descriptors.

TxDescriptors Valid Range: 64-4096

Default Value: 256 This value is the number of transmit descriptors allocated by the driver.

Increasing this value allows the driver to queue more transmits. Each descriptor is 16 bytes.

XsumRX Valid Range: 0-1

Default Value: 1 A value of 1 indicates that the driver should enable IP checksum offload

for received packets (both UDP and TCP) to the adapter hardware.

58 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

▼ To Configure VLANs in a Linux Environment

1. Ensure that the ixge module is loaded:

# modprobe ixge

2. Plumb the Sun 10-Gigabit Ethernet PCI-X interface:

# ifconfig eth2 xxx.xxx.xx.xxx up

xxx.xxx.xx.xxx = the IP address of the interface.

where

3. Insert the vlan module:

# /sbin/insmod 8021q

4. Add vlan instance (VID):

# sbin/vconfig add eth2 5

5. Configure ixge vlan (eth2 in this example):

# ifconfig eth2.5 xxx.xxx.xx.xxx up

where

xxx.xxx.xx.xxx = the IP address of the interface.

Appendix D Installing the Sun 10-Gigabit Ethernet Device Driver on Linux Platforms 59

▼ To Configure Bonding for Multiple ixge

Interfaces

1. Modify /etc/modules.conf file by adding these lines:

alias bond0 bonding options bonding max_bonds=2 mode=4 miimon=1000

Where:

bond0 is the bonding device. max_bonds is the number of bond interfaces to be created. mode specifies the bonding policies. miimon is the frequency in milliseconds that MII link monitoring will occur.

Refer to Linux documentation for more information.

2. Load the bonding driver:

# modprobe bonding

3. Configure bond0 interface. In this example, bond0 is the master of the two interfaces eth2 and eth3.

# ip addr add 192.12.38.64/24 brd + dev bond0 # ip link set dev bond0 up # ifenslave bond0 eth2 eth3

60 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Index

Symbols /etc/hostname.ixgenumber file 16 A assigning an IP address 15 autonegotiation

setting

disabling 26 C configuring device driver parameters 19 configuring the driver parameters 19 configuring the network host files 15 configuring VLANs 39 D documentation

SunVTS 46 Dynamic Reconfiguration 5 dynamic reconfiguration 11 Dynamic Reconfiguration (DR) 11 E editing the network host files 5 F files and directories on the CD-ROM 50 flow control keywords 21 H hardware

requirements 2 I ifconfig -a 41 ifconfig command 15 installing the adapter with DR 11 installing the adapter without DR 6

ixge driver parameter, status, and descriptions 20 L link partner settings

checking 32 N ndd utility 24 P parameter values

how to modify and display 24 parameters and settings 20 physical dimensions 44 R receive interrupt blanking values 20, 21 S setting ixge driver parameters

using ndd

using ixge.conf 23

software

requirements 2 SunVTS

documentation 45 SunVTS diagnostic application 45 V verify the installation 9 virtual device 41 VLAN ID 39 VLAN naming format 40 VLANs 37

62 Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide • December 2005

Sun Microsystems Ethernet PCI-X Adapter User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Contents

Tables

Figures

Declaration of Conformity

European Union

Preface

How This Book Is Organized

Using UNIX Commands

Shell Prompts

Typographic Conventions

Accessing Sun Documentation

Third-Party Web Sites

Contacting Sun Technical Support

Sun Welcomes Your Comments

Product Overview

Product Description

Hardware and Software Requirements

Features

Installing the Adapter

Installing the Adapter Without Dynamic Reconfiguration

▼ To Install the Adapter

▼ To Install the Optical Transceiver

▼ To Verify the Hardware Installation

Rebooting the System

Installing the Adapter With Dynamic Reconfiguration

Using the Installation Script

Verifying the Software Installation

Configuring the Network Host Files

Configuring Driver Parameters

10-Gigabit Ethernet Device Driver Parameters

10-Gigabit Ethernet Driver Parameter Values and Definitions

Flow Control Parameters

Interrupt Parameters

Jumbo Frames

Setting ixge Driver Parameters

Setting Parameters Using the ndd Utility

Noninteractive and Interactive Modes

Setting Parameters Using the ixge.conf File

10-Gigabit Ethernet Driver Operating Statistics

▼ To Check Link Partner Settings

▼ To Discover Device Capabilities

▼ To Discover Link Settings

Configuration Checking

Configuring VLANs

Overview of VLANs

Configuring VLANs

▼ To Configure Static VLANs

Specifications

Performance Specifications

Physical Characteristics

Power Requirements

Diagnostic Software

SunVTS Diagnostic Software

Updating SunVTS to Recognize the Adapter

▼ To Update SunVTS to Recognize the Adapter

Using the SunVTS netlbtest

Installing the Software Manually

▼ To Install the Driver Software Manually

Installing the Sun 10-Gigabit Ethernet Device Driver on Linux Platforms

Setting Up the Driver in a Linux Platform

▼ To Set Up the Driver in a Linux Platform

Setting Driver Parameters in a Linux Environment

▼ To Configure VLANs in a Linux Environment

Index