Sun Microsystems T6320 User Manual 2

Download

Sun Blade™T6320 XAUI Pass-

Through Fabric Expansion Module

User’s Guide

Sun Microsystems, Inc. www.sun.com

Part No. 820-4777-11 July 2008, Revision A

Submit comments about this document at: http://www.sun.com/hwdocs/feedback

Sun Microsystems,Inc. has intellectual property rightsrelating to technology embodied in the productthat is described in this document. In particular,and withoutlimitation, these intellectual property rightsmay includeone or moreof theU.S. patentslisted at http://www.sun.com/patents andone ormore additional patents or pendingpatent applicationsin theU.S. and in other countries.

Sun, SunMicrosystems, the Sun logo, SunVTS, and Solarisare trademarks or registered trademarksof Sun Microsystems,Inc. orits subsidiaries, inthe U.S.and othercountries.

UNIX isa registered trademarkin the U.S. and other countries, exclusivelylicensed through X/OpenCompany, Ltd.

UNIX®. PCI-Express®. The Adobe. logo and the PostScriptlogo are trademarksor registered trademarksof Adobe Systems, Incorporated.

Products covered byand informationcontained in this service manual are controlled byU.S. ExportControl laws and may besubject tothe export orimport lawsin othercountries. Nuclear, missile, chemicalbiological weaponsor nuclearmaritime end uses or end users, whether direct orindirect, are strictlyprohibited. Export or reexport tocountries subject to U.S. embargoor toentities identiﬁed on U.S. export exclusion lists, including,but notlimited to,the denied persons and specially designated nationalslists isstrictly prohibited.

Use ofany spare orreplacement CPUs is limited to repair orone-for-one replacement ofCPUs inproducts exported incompliance withU.S. export laws.Use ofCPUs asproduct upgrades unless authorized bythe U.S.Government is strictly prohibited.

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULARPURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Sun Microsystems,Inc. détient les droits depropriété intellectuels relatifsà latechnologie incorporéedans le produitqui estdécrit dansce document. Enparticulier,et ce sans limitation, ces droits depropriété intellectuelle peuventinclure un ou plus des brevets américainslistés à l’adresse http://www.sun.com/patents et un ou lesbrevets supplémentaires oules applicationsde brevet enattente auxEtats - Unis et dansles autres pays.

Sun, SunMicrosystems, le logo Sun, SunVTS, et Solarissont desmarques de fabrique ou des marques déposées de Sun Microsystems,Inc., ou ses ﬁliales,aux Etats-Uniset dansd’autres pays.

Toutes lesmarques SPARCsont utiliséessous licence et sont des marques defabrique ou des marques déposéesde SPARC International, Inc. aux Etats-Uniset dansd’autres pays. Les produits portantles marques SPARC sont baséssur unearchitecturedéveloppée parSun Microsystems, Inc.

UNIX estune marque déposéeaux Etats-Uniset dansd’autres pays etlicenciée exlusivementpar X/OpenCompany, Ltd.

UNIX®. PCI-Express®. Le logo Adobe. et le logo PostScript sontdes marques defabrique oudes marques déposéesde AdobeSystems, Incorporated.

Ce produit est soumis à la législation américaine en matière de contrôle des exportations et peut être soumis à la règlementation en vigueur dans d’autres pays dans le domaine des exportations et importations. Les utilisations , ou utilisateurs ﬁnaux, pour des armes nucléaires, des missiles, des armes biologiques et chimiques ou du nucléaire maritime, directement ou indirectement, sont strictement interdites. Les exportations ou reexportations vers les pays sous embargo américain, ou vers des entités ﬁgurant sur les listes d’exclusion d’exportation américaines, y compris, mais demanière non exhaustive, laliste de personnes qui font objet d’unordre de ne pas participer, d’une façon directe ou indirecte, aux exportations des produits ou des services qui sont régis par la législation américaine en matière de contrôle des exportations et la liste de ressortissants spéciﬁquement désignés, sont rigoureusement interdites. L’utilisation de pièces détachées ou d’unités centrales de remplacement est limitée aux réparations ou à l’échange standard d’unités centrales pour les produits exportés, conformément à la législation américaine enmatière d’exportation. Sauf autorisation par les autoritésdes Etats-Unis,l’utilisation d’unités centrales pour procéderà des mises à jour de produits est rigoureusement interdite.

LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENTEXCLUES, DANSLA MESURE AUTORISEE PAR LALOI APPLICABLE,Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFACON.

Contents 3

Preface 5

1. Product Overview 1

Product Description 1

Hardware and Software Requirements 3

Features 4

2. Installing the Fabric Expansion Module 5

Installing the FEM 5

Verifying the Hardware Installation 6

3. Configuration Information 9

A. Specifications 11

Performance Specifications 11

Physical Characteristics 12

Power Requirements 12

B. Diagnostic Software 13

SunVTS Diagnostic Software 13

Using the SunVTS xnetlbtest 14

xnetlbtest Test Requirements 14

4 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

Preface

This guide describes the features, installation, and test of the Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module (FEM).

These instructions are designed for enterprise system administrators with experience installing network hardware and software.

For supported systems, see the “Hardware and Software Requirements” on page 3.

How This Document Is Organized

Chapter 1 provides an overview of the Sun Blade T6320 XAUI Pass-Through Fabric

Expansion Module features.

Chapter 2 describes how to install the Sun Blade T6320 XAUI Pass-Through Fabric

Expansion Module in your system and to verify that it has been installed correctly.

Chapter 3 describes how to find more information on configuring the FEM.

Appendix A lists the specifications for the Sun Blade T6320 XAUI Pass-Through Fabric

Expansion Module.

Appendix B provides an overview of the SunVTS™ diagnostic application and

instructions for updating the SunVTS software to recognize the FEM.

Using UNIX Commands

This document might not contain information about basic UNIX®commands and procedures such as shutting down the system, booting the system, and configuring devices. Refer to the following for this information:

■ Software documentation that you received with your system

■ Solaris Operating System documentation, which is at:

http://docs.sun.com

Shell Prompts

Shell Prompt

C shell machine-name%

C shell superuser machine-name#

Bourne shell and Korn shell $

Bourne shell and Korn shell superuser #

6 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

Typographic Conventions

Typeface

AaBbCc123 The names of commands, files,

AaBbCc123 What you type, when contrasted

AaBbCc123 Book titles, new words or terms,

* The settings on your browser might differ from these settings.

Meaning Examples

Edit your.login file. and directories; on-screen computer output

with on-screen computer output

words to be emphasized. Replace command-line variables with real names or values.

Use ls -a to list all files.

% You have mail.

Password:

Read Chapter 6 in the User’s Guide.

These are called class options.

Yo u must be superuser to do this.

To delete a file, type rm filename.

Product Overview

This chapter provides an overview of the Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module (FEM), including:

■ “Product Description” on page 1

■ “Hardware and Software Requirements” on page 3

■ “Features” on page 4

Product Description

The Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module (FEM) connects to the Sun Blade T6320 server module motherboard by two square NexLev connectors (

FIGURE 1-1).

The FEM transmits two sets of x4 XAUI 10-Gigabit Ethernet lanes from the Sun Blade™ T6320 server module to the Sun Blade 6000 10GbE Multi-Fabric Network Express Module (

FIGURE 1-2).

FIGURE 1-1 FEM Module for the Sun Blade T6320 Server Module

2 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

FIGURE 1-2 Signal Diagram of the Sun Blade T6320 XAUI Pass-Through Fabric

Expansion Module

Hardware and Software Requirements

Before installing the FEM, ensure that your system meets the minimum hardware and software requirements in

TABLE 1-1 Minimum Hardware, Software and Firmware Versions

System or OS Software or Firmware Version

Sun Blade T6320 server module System Firmware 7.0.9.c with patch 127581-01 or later.

Sun Blade 6000 Modular System Chassis Sun Blade 6000 Modular System Chassis

Operating System Solaris 10 8/07 OS with patch 127127-11 and 136933-02 or later

TABLE 1-1.

Chapter 1 Product Overview 3

Features

The Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module provides the following features:

■ Supports two full-duplex 10-Gigabit Ethernet interfaces with the Sun Blade 6000

10GbE Multi-fabric Network Express Module.

■ IEEE 802.3ae 2002-compliant

■ Hardware-based flow classification for extending parallelism and virtualization to

networking

■ Supports up to eight receive DMA channels and up to 12 transmit DMA channels,

multiple receive and transmit descriptor rings, and dedicated networking hardware resources (DMA, interrupts, buffer, and more) for each thread or strand.

■ CPU/thread affinity and CPU load balancing at L1, L2, L3, and L4

■ Jumbo Frames support (up to 9KBytes)

■ IPv4, IPv6, and IPMP support

■ TCP and UDP checksum and CRC32C support

■ IEEE 802.1Q VLAN support

4 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

CHAPTER

Installing the Fabric Expansion Module

This chapter describes how to install the Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module in your system and verify that it has been installed correctly.

This chapter contains the following sections:

■ “Installing the FEM” on page 5

■ “Verifying the Hardware Installation” on page 6

Note – If you are installing the Sun Blade T6320 XAUI Pass-Through Fabric

Expansion Module into a server module running Solaris 10 software, you must install the Solaris OS and patches before you install the hardware.

Installing the FEM

The following instructions describe the basic tasks required to install the FEM inside a blade server module.

▼ To Install the FEM With the Power Off

1. Halt and power off your system.

2. Attach the adhesive copper strip of the antistatic wrist strap to the server module chassis. Wrap the other end twice around your wrist, with the adhesive side against your skin.

3. Perform an orderly shutdown of the server module

4. Remove the server module from the system chassis.

5. Remove the main cover of your system.

6. Slide the FEM card at an angle into the support bracket, then press it carefully into the connector.

FIGURE 2-1 Installing the FEM

Verifying the Hardware Installation

After you install the Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module, perform the following tasks to verify the installation.

▼ To Verify the Hardware Installation

1. Power on the system.

6 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

2. List the network devices on your system.

ok show-nets a) /pci@0/pci@0/pci@c/network@0,1 b) /pci@0/pci@0/pci@c/network@0 c) /niu@80/network@1 d) /niu@80/network@0 q) NO SELECTION Enter Selection, q to quit

You should see the network devices in c) and d) above. If you do not see the network devices, verify that the FEM is properly seated.

▼ To Reboot the System

● After verifying the FEM installation, perform a reconfiguration boot on your

system. Type the following:

ok boot -r

Chapter 2 Installing the Fabric Expansion Module 7

8 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

CHAPTER

Configuration Information

If you require more information about the following topics, refer to the Sun Dual 10GbE XFP PCIe ExpressModule User’s Guide, 820-1606.

■ Configuring the Network

■ Configuring the nxge Device Driver Parameters

■ Configuring the Jumbo Frames Feature

■ Configuring Link Aggregation

■ Configuring VLANs

The Sun Dual 10GbE XFP PCIe ExpressModule User’s Guide, 820-1606, is available at this URL:

http://docs.sun.com/app/docs/prod/10gbe.xfp.pcie~10gbe-xfppem#hic

10 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

APPENDIX

Specifications

This appendix lists the specifications for the Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module. This appendix contains the following sections:

■ “Performance Specifications” on page 11

■ “Physical Characteristics” on page 12

■ “Power Requirements” on page 12

Performance Specifications

TABLE A-1 Performance Specifications

Feature Specification

Bus type Two x4 lanes XAUI

Bus speed (x8, encoded rate) 20Gbit/sec unidirectional; 40Gbit/sec bidirectional

(theoretical)

Maximum Ethernet transfer rate 10Gbps (full-duplex)

Optics 10GBASE-SR, IEEE 802.3ae 2002 compliant

Physical Characteristics

TABLE A-2 Physical Characteristics of the Express Module

Dimension Measurement

Length 139.7 mm ( 5.5 inches)

Width 114.3 mm ( 4.5 inches)

Height 112 mm (4.4 inches)

Power Requirements

TABLE A-3 Environmental Requirements

Specification Measurement

Power consumption 3.5 W RMS nominal

5.0 W maximum

12 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008

APPENDIX

Diagnostic Software

This appendix provides an overview of the SunVTS diagnostic application and instructions for updating the SunVTS software to recognize the Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module. This appendix contains the following sections:

■ “SunVTS Diagnostic Software” on page 13

■ “Find the SunVTS documentation at:” on page 14

SunVTS Diagnostic Software

The SunVTS software executes multiple diagnostic hardware tests from a single user interface. SunVTS is used to verify the configuration and functionality of most hardware controllers and devices. The SunVTS software operates primarily from a graphical user interface, enabling test parameters to be set quickly and easily while a diagnostic test operation is being performed.

The SunVTS nettest diagnostic can be used to test all of the networking interfaces on the system, including the interfaces on the FEM.

To use the nettest diagnostic, you must have the SunVTS software installed on your system. Use SunVTS 6.4 Patch Set 3 or later. You will need to update the SunVTS configuration to recognize the FEM. Refer to the SunVTS documentation in

TABLE B-1.

TABLE B-1 SunVTS Documentation

Title Description

SunVTS 6.4 Test Reference Manual for SPARC Platforms Describes each SunVTS test (including xnetlbtest).

Describes test options and command-line arguments

SunVTS Quick Reference Card Provides an overview of the user interface

SunVTS 6.4 Patch Set 3 Release Notes Updates to the other SunVTS documents.

SunVTS 6.4 Patch Set 3 Documentation Supplement for SPARC Platforms

Find the SunVTS documentation at:

http://docs.sun.com/app/docs/prod/vts64#hic

Using the SunVTS xnetlbtest

The xnetlbtest provides functional test coverage of the network products based on Neptune Ethernet chip/core. Refer to the SunVTS 6.4 Test Reference Manual for

SPARC Platforms, for complete xnetlbltest information.

xnetlbtest Test Requirements

The NEM has an internal loopback path built in. You do not need an loopback connector. You must have the Ethernet card (the FEM) and the device driver installed. Intervention mode is enabled before running xnetlbtest. xnetlbtest cannot run and does not appear in the GUI if the network interface is connected to a live network. xnetlbtest also requires that the Ethernet device be configured offline before running the test. Use the ifconfig(1M) command to bring the Ethernet device down before running xnetlbtest. Enter the following commands to bring the interface down:

Updates to the other SunVTS documents specific to this patch set.

# ifconfig nxge0 down # ifconfig nxge0 unplumb

Where nxge0 is the 10GbE port on your server module.

14 Sun Blade T6320 XAUI Pass-Through Fabric Expansion Module User’s Guide • July 2008