Intel PCI-X User Manual

DSS NETWORKS

GigMAC PMC & PCI/PCI-X Cards

GigPMC Switch

GigPCI-Express Switch

Board and Driver Users Manual

(Includes all Intel-based models 5164, 5261, 5262, 5263-SFP, 5264, 5468, 6162,

6267-SFP and 6468)

(copper and fiber models)

Document Version 2.6, P/N 131903

Sept 2007

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

1. INTRODUCTION .................................................................................................................. 4

1.1 SEE ALSO............................................................................................................................ 4

1.2 COMPATIBILITY ...............................................................................................................4

2. MODEL NUMBERS.............................................................................................................. 5

3. KEY FEATURES................................................................................................................... 6

3. SWITCH CAPABILITIES (MODELS 5468/6468)............................................................... 7

4. OEM DEVELOPER KIT CONTENTS..................................................................................7

5. SYSTEM REQUIREMENTS................................................................................................. 9

6. HARDWARE INFORMATION .......................................................................................... 10

6.1 BOARD PHOTOS.............................................................................................................. 10

6.2 BOARD LED INDICATORS............................................................................................. 11

6.2 BOARD LED INDICATORS............................................................................................. 12

6.3 PMC CONNECTOR PIN/SIGNAL DEFINITIONS.......................................................... 15

6.4 PCI CONFIGURATION REGISTERS.............................................................................. 19

6.4.1 VENDOR AND DEVICE IDS........................................................................................ 20

6.5 EEPROM LISTINGS.......................................................................................................... 20

6.6 DEFAULT REGISTER SETTINGS .................................................................................. 22

6.7 ETHERNET FRAME LATENCY ..................................................................................... 24

7. POWER CONSUMPTION SPECS......................................................................................24

8. HARDWARE INSTALLATION ......................................................................................... 24

8.1 INSTALLATION IN PC COMPUTER.............................................................................. 25

8.2 EMBEDDED OR COMPACT PCI INSTALLATION...................................................... 25

9. COPPER CABLING AND CONNECTOR INFO............................................................... 26

9.1. FIBER CABLE SPECIFICATIONS ................................................................................. 26

9.2 COPPER RJ-45 CONNECTOR AND CABLE.................................................................. 27

10. SOFTWARE DRIVER INSTALLATION......................................................................... 28

10.1 LINUX DRIVER INSTALLATION AND USAGE........................................................ 28

10.2 VXWORKS DRIVER INSTALLATION AND USAGE ................................................ 35

10.3 DRIVER UTILITY COMMAND REFERENCE............................................................. 36

10.3.1 STATISTICS SHOW FUNCTION ............................................................................... 36

10.3.2 PCI REGISTERS SHOW FUNCTION......................................................................... 37

10.3.3 MAC REGISTERS SHOW FUNCTION...................................................................... 38

10.3.4 MAC STATISTICS SHOW FUNCTION..................................................................... 39

10.3.5 PHY REGISTERS SHOW FUNCTION....................................................................... 41

10.3.6 EEPROM SHOW FUNCTION ..................................................................................... 42

10.3.7 MAC ADDRESS SHOW FUNCTION.........................................................................43

10.3.8 BUFFER DESCRIPTOR STATUS SHOW FUNCTION............................................. 43

10.3.9 STARTING FRAME GENERATOR............................................................................44

10.3.10 STOPPING FRAME GENERATOR .......................................................................... 44

10.3.11 SWITCH STATUS SHOW FUNCTION.................................................................... 45

10.4 DPM DRIVER MANAGEMENT API............................................................................. 45

10.5 SWITCH CONFIGURATION / MANAGEMENT API..................................................47

11. TESTING AND VERIFICATION..................................................................................... 49

12. SPECIFICATIONS.............................................................................................................50

DSS NETWORKS, INC. Version: 2.6 Page: 2

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

12.1 ENVIRONMENTAL SPECIFICATIONS....................................................................... 53

13. WARRANTEE AND SUPPORT INFO.............................................................................54

DSS NETWORKS, INC. Version: 2.6 Page: 3

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

1. INTRODUCTION

The GigMAC and GigPMC-switch family of network adapter cards and switches are a high-performance, cost-effective solution for adding Gigabit Ethernet connectivity to any embedded or real-time network appliance or network access device equipped with a PCI slot or PMC mezzanine site and is ideal for embedded systems utilizing add-on modules including PCI, Compact PCI and VME based systems.

The GigMac and GigPMC-switch family includes optimized, high-performance driver support for VxWorks and Linux. Instructions for Linux driver installation are included in this manual. In addition, a companion document titled “VxWorks Users Manual and Integration Guide” is provided for technical assistance in integrating and testing the driver in an embedded real-time VxWorks environment.

1.1 SEE ALSO

Please also see the following documents on our website at www.dssnetworks.com and also included in the OEM developers kit CD:

Datasheets – please see product datasheets and other updated product information on OEM developer CD and on website.

Release Notes -- where updated information is provided on new features, compatibility, performance benchmarks, platform information and corrected problems.

VxWorks Users Manual and Integration Guide, DSS Document part no 131901. Provides technical information on integrating and testing our controllers and drivers into a VxWorks BSP and system board.

GIGFAQ.HTML – Also on website and included on OEM developer CD contains many answers to commonly asked questions regarding Gigabit Ethernet and our products including performance and system recommendations.

README.LINUX – Included on OEM developer CD contains latest driver installation and usage instructions for Linux Operating System.

netPerformance.txt -- Included on OEM developer CD contains useful information on tuning the VxWorks network protocol stack.

1.2 COMPATIBILITY

The GigMAC and GigPMC-Switch family is fully compliant with the following standards:

DSS NETWORKS, INC. Version: 2.6 Page: 4

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

 IEEE 802.3 (all sections applicable to 1000 Base T, 1000 Base SX, 1000 Base LX)  IEEE 802.1D and IEEE 802.1Q as applicable for VLAN and priority queuing support  PCI 2.2 and PCI-X 1.0 compliant  PCI low-profile specification (as applicable for model)  IEEE 1386.1 Draft 2.2  Linux driver compatibility: Standard Linux 2.4 or 2.6 kernel level network driver

module compiled and tested on Intel and PowerPC architectures up to Linux kernel

2.4.26 and 2.6.4.

 VxWorks driver compatibility: Standard Tornado 2.0.2 and Tornado 2.2/2.2.1,

VxWorks 5.4/5.5/5.5.1 Enhanced Network Driver loadable module integrated and tested on PowerPC and Intel architectures

2. MODEL NUMBERS

This user manual covers all Models of our Intel based GigMAC and GigPMC-switch PMC, PMC-X, PCI-X cards including:

PMC models

 5164 quad-port (copper)  5261-LC (fiber)  5261-RJ (copper)  5262-LC (fiber)  5262-RJ (copper)  5262-SC (fiber)  5263-SFP (fiber, copper)  5264-RJ (copper)  5468 (GigPMC hybrid switch, copper)

PCI-Express models

 6468 (GigPCI-Express hybrid switch and multiport interface, copper)

PCI-X models

 Model 6162 dual port copper PCI-X  Model 6267-SFP (fiber and copper, PCI-X)

DSS NETWORKS, INC. Version: 2.6 Page: 5

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

3. KEY FEATURES

The GigMAC (PMC and PCI), GigPMC-switch and GigPCI-E switches offer the following key features:

• Sustained throughput of 245 Mbytes/sec (1.96 Gb) per port over PCI bus using 64-bit, 66 MHZ PCI

• Sustained throughput of 118 Mbytes/sec (944 Gb) per port over PCI bus using 32-bit, 33 MHZ PCI

• Frame processing rate of up to 1,000,000 frames per second (as measured on 2 GHZ P4/Zeon running Linux 2.4 kernel)

• Very low latency, < 2 microseconds for short frames

• Support for copper and fiber interfaces

• Driver support for vxWorks and Linux including embedded Linux

• Built in management and diagnostics capabilities in drivers

• High-performance frame generator (wire and bus-speed capable)

• Performance and health monitoring statistics

• 64-bit PMC and 64-bit PCI/PCI-X low-profile card adapter form factors

• Supports 33 and 66 MHz, 32 and 64 bit PCI interfaces including PMC form factor

• Support for 133/100/66 PCI-X in Models 5262-(LC,SC,RJ), 5263-SFP, 5264,

5468, 6162 and 6267-SFP

• Support for “x8” 8-lane PCI Express in Models 6468

• Installs in any PCI or CompactPCI system with a PMC, universal PCI/PCI-X or

PCIe slot

• Ideal solution for CompactPCI and embedded 1U and 2U PC-based systems

• Utilizes either 5 or 3.3 volt pci bus power supply and contains onboard power

regulators

• Full duplex Gigabit Ethernet Interface over standard CAT5e cabling

• Complies with all PCI revision 2.2 mechanical and electrical requirements

• Fully IEEE 802.3z, IEEE 802.3ab, 802.3u and IEEE 1386 compliant

DSS NETWORKS, INC. Version: 2.6 Page: 6

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

• Compatible with all 10/100/1000BaseT hubs, switches and routers

• Burst rate of up to 256 dwords (1024 bytes) over PCI bus

• Jumbo frame support for up to 9K, 802.3x full duplex flow control with automatic

pause and priority with multiple priority queues

3. SWITCH CAPABILITIES (MODELS 5468/6468)

The models 5468 and 6468 provide an onboard Gigabit Ethernet layer 2 switch with management capabilities. The operating registers in the switch are accessible via a register-level API that is provided in the host driver and described in section 10.5 of this manual.

The switch can be programmed to support advanced features including following:

o ARL table control (4K MAC-address management) o Aging Control o VLAN management (4K VLANS) o Trunking, aggregation and failover control o Port mirroring control o QoS / 802.1P / Priority / Differential Services o Port traffic control o Switch status o PHY (transceiver) control o Jumbo frame control o Port parameters and port state control o Port rate control o Protected ports (switch partitioning) o Broadcast Storm Suppression o LED control

4. OEM DEVELOPER KIT CONTENTS

An OEM developer kit is provided with the purchase of the Gigabit Ethernet controllers which contains drivers, documentation and sample code including the following:

 Driver source code for vxWorks, Linux 2.4, PowerPC, Intel platforms  Users Manual(s)  VxWorks Users Manual and Integration Guide  Datasheets for chipset controllers (Intel, National, etc.)  TCP/UDP/IP performance test programs (vxWorks, Linux, Windows versions

included)

 TCP, UDP and raw driver performance tests

DSS NETWORKS, INC. Version: 2.6 Page: 7

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

 Driver Utilities (Linux)  High-performance frame generator (wire and bus-speed capable)  Transmit and receive callbacks (hooks) for driver-level application code  Internal and external loopback capabilities  Built-in performance instrumentation statistics  Gigabit Ethernet FAQ sheet  NetPerformance.txt protocol stack tuning guide for vxWorks

DSS NETWORKS, INC. Version: 2.6 Page: 8

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

5. SYSTEM REQUIREMENTS

Intel Platform (minimum): Pentium III 800 MHZ or faster with PC100 or PC133 SDRAM, 32-bit 33 MHZ PCI.

Intel Platform (recommended): Pentium4, Core Duo or Xeon at 1.8GHZ or faster, DDR SDRAM, 64-bit, 66 MHZ PCI or PCI-X (133/100/66 MHZ).

PowerPC Platform (minimum): PowerPC 7400 or 750 series at 500 MHZ or faster, PC100 or PC133 SDRAM, 32-bit, 33 MHZ PCI.

PowerPC Platform (recommended): PowerPC 7400 or 750 series at 1 GHZ or faster, DDR SDRAM, 64-bit, 33 or 66 MHZ PCI or PCI-X (133/100/66 MHZ)

Linux Operating system: Linux 2.4 (2.4.18 - 2.4.30), Linux 2.6 (all) Wind River:

- Wind River Workbench Development Studio / vxWorks 6.x

- Tornado 2.2/vxWorks 5.5 (or Tornado 2.0.2/vxWorks 5.4 with patch updates. PCI Configuration Library support (pciConfigLib).

DSS NETWORKS, INC. Version: 2.6 Page: 9

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

6. HARDWARE INFORMATION

6.1 BOARD PHOTOS

Model 5468 GigPMC 8-port switch

Model 6267-SFP PCI-X

Model 5262-RJ PMC

Model 5262-LC fiber PMC

DSS NETWORKS, INC. Version: 2.6 Page: 10

Model 5164 - 4-port PMC

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

Model 5262-SC

Model 6468-RJ PCIe

Model 5264-RJ

Model 5263-SFP

DSS NETWORKS, INC. Version: 2.6 Page: 11

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

6.2 BOARD LED INDICATORS

PCI / PMC Boards LED Function Table

Gigabit NIC Model

PCI-X 6267 TX RX LINK N/A PMC 5468 LNK/ACT LNK/ACT LNK/ACT LNK/ACT PMC 5164 LNK/ACT LNK/ACT LNK/ACT LNK/ACT PMC 5264 LNK/ACT LNK/ACT LNK/ACT LNK/ACT PMC 5262 TX RX LINK FD PCI-X 6162 TX RX LINK FD PCIe 6468 TBD TBD TBD TBD

Model 6267-SFP

The front panel has onboard pluggable SFP fiber optic connectors that support fiber optic cabling using 50 or 62.5 micron multimode fiber with LC type connectors. An RJ45 type 1000 Base T SFP connector is also available for this model. There are 3 LED indicators per port that provide the following as marked with silkscreen nomenclature on front panels:

 LED_LINK (link)  LED_TX (transmit)  LED_RX (receive)

Link “LNK” Indicator

On solid green to indicate auto-negotiation complete and MAC and driver level link is up.

Transmit and Receive Activity Indicators

Transmit and Receive LEDs are solid or blinking yellow to indicate activity on the link (blinking shows tx and rx activity).

Model 5262-RJ dual-port or 5261-RJ single port PMC (copper)

The front panel has two onboard RJ-45 connectors that supports the connection of Category 5e cabling with 4 LED indicators per port that provide the following as marked with silkscreen nomenclature on front panels:

 LED_1000 (1000 Mb mod)  LED_FD (full-duplex mode)

LED # 1 Green

LED # 2 Green

LED #3 Yellow

LED #4 Yellow

DSS NETWORKS, INC. Version: 2.6 Page: 12

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

 LED_TX (transmit activity)  LED_RX (receive activity)

Model 5262-LC dual-port or 5261-LC single port PMC (fiber)

The front panel has two onboard fiber LC connectors that support the connection of singlemode or multimode fiber optics with 4 LED indicators per port that provide the following as marked with silkscreen nomenclature on front panels:

 LED_1000 (1000 Mb mod)  LED_FD (full-duplex mode)  LED_TX (transmit activity)  LED_RX (receive activity)

Model 5263-SFP dual-port (fiber, copper)

The front panel has two onboard SFP connectors that support the connection of singlemode or multimode fiber optics or 1000-Base T transceivers with 4 LED indicators per port that provide the following as marked with silkscreen nomenclature on front panels:

 LED_1000 (1000 Mb mod)  LED_FD (full-duplex mode)  LED_TX (transmit activity)  LED_RX (receive activity)

Model 5164/5264 quad-port PMC with RJ-45 connector(s)

The front panel has four onboard RJ-45 connectors that support the connection of Category 5e cabling with 1 LED indicator per port that provides both a “link” and “activity” indicator as follows:

 OFF (no link)  ON (link)  BLINKING (link and activity)

Model 5468 GigPMC Switch with 4-port RJ-45 connector(s)

DSS NETWORKS, INC. Version: 2.6 Page: 13

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

 OFF (no link)  ON (link)  BLINKING (link and activity)

Model 6468 GigPCI-Express Switch with 4-port RJ-45 connector(s)

 TBD

DSS NETWORKS, INC. Version: 2.6 Page: 14

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

6.3 PMC CONNECTOR PIN/SIGNAL DEFINITIONS

PMC Connectors Pin Assignments

Pn1/Jn1 32 Bit PCI

1 TCK -12V NC 2 3 GND INITA# 4 5 INTB# INTC# NC 6 7 BUSMODE1# +5V 8 9 NC INTD# PCI-RSVD NC 10 11 GND PCI-RSVD NC 12 13 CLK GND 14 15 GND GNT# 16 17 REQ# +5V 18 19 VIO AD[31] 20 21 AD[28] AD[27] 22 23 AD[25] GND 24 25 GND C/BE[3]# 26 27 AD[22] AD[21] 28 29 AD[19] +5V 30 31 NC VIO AD[17] 32 33 FRAME# GND 34 35 GND IRDY# 36 37 DEVSEL# +5V 38 39 GND/PCIXCAP LOCK# 40 41 NC SDONE# SBO# 42 43 PAR GND 44 45 NC VIO AD[15] 46 47 AD[12] AD[11] 48 49 AD[09] +5V 50 51 GND C/BE[0]# 52 53 AD[06] AD[05] 54 55 AD[04] GND 56 57 NC VIO AD[03] 58 59 AD[02] AD[01] 60 61 AD[00] +5V 62 63 GND REQ64# 64

DSS NETWORKS, INC. Version: 2.6 Page: 15

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

Pn2/Jn2 32 Bit PCI

1 NC +12V TRST# 2 3 TMS TDO 4 5 TDI GND 6 7 GND PCI-RSVD NC 8 9 NC PCI-RSVD PCI-RSVD NC 10 11 NC BUSMODE2# +3.3V 12 13 RST# BUSMODE3# PMC_1 14 15 3.3V BUSMODE4# NC 16 17 NC PCI-RSVD GND 18 19 AD[30] AD[29] 20 21 GND AD[26] 22 23 AD[24] 3.3V 24 25 IDSEL AD[23] 26 27 +3.3V AD[20] 28 29 AD[18] GND 30 31 AD[16] C/BE[2]# 32 33 GND PMC-RSVD NC 34 35 TRDY# +3.3V 36 37 GND STOP# 38 39 PERR# GND 40 41 +3.3V SERR# 42 43 C/BE[1]# GND 44 45 AD[14] AD[13] 46 47 M66EN AD[10] 48 49 AD[08] 3.3V 50 51 AD[07] PMC-RSVD NC 52 53 +3.3V PMC-RSVD NC 54 55 NC PMC-RSVD GND 56 57 NC PMC-RSVD PMC-RSVD NC 58 59 GND PMC-RSVD NC 60 61 ACK64# +3.3V 62 63 GND PMC-RSVD NC 64

DSS NETWORKS, INC. Version: 2.6 Page: 16

GigMAC-PMC and PCI GigPMC Switch Board and Driver Users Manual

Pn3/Jn3 32 Bit PCI

1 NC PCI-RSVD GND 2 3 GND C/BE[7]# 4 5 C/BE[6]# C/BE[5]# 6 7 C/BE[4]# GND 8 9 NC VIO PAR64 10 11 AD[63] AD[62] 12 13 AD[61] GND 14 15 GND AD[60] 16 17 AD[59] AD[58] 18 19 AD[57] GND 20 21 NC VIO AD[56] 22 23 AD[55] AD[54] 24 25 AD[53] GND 26 27 GND AD[52] 28 29 AD[51] AD[50] 30 31 AD[49] GND 32 33 GND AD[48] 34 35 AD[47] AD[46] 36 37 AD[45] GND 38 39 NC VIO AD[44] 40 41 AD[43] AD[42] 42 43 AD[41] GND 44 45 GND AD[40] 46 47 AD[39] AD[38] 48 49 AD[37] GND 50 51 GND AD[36] 52 53 AD[35] AD[34] 54 55 AD[33] GND 56 57 NC VIO AD[32] 58 59 NC PCI-RSVD PCI-RSVD 60 61 NC PCI-RSVD GND 62 63 GND PCI-RSVD NC 64

DSS NETWORKS, INC. Version: 2.6 Page: 17

+ 37 hidden pages