IDT 89HPES64H16G2 User Manual

®

IDT® 89HPES64H16G2

®

PCI Express

Switch

User Manual

April 2013

6024 Silver Creek Valley Road, San Jose, California 95138

Telephone: [phone] • [phone] • FAX: [phone]

©2013 Integrated Device Technology, Inc.

Printed in U.S.A.

Integrated Device Technology, Inc. reserves the right to make changes to its products or specifications at any time, without notice, in order to improve design or performance
and to supply the best possible product. IDT does not assume any responsibility for use of any circuitry described other than the circuitry embodied in an IDT product. The
Company makes no representations that circuitry described herein is free from patent infringement or other rights of third parties which may result from its use. No license is
granted by implication or otherwise under any patent, patent rights or other rights, of Integrated Device Technology, Inc.

GENERAL DISCLAIMER

Code examples provided by IDT are for illustrative purposes only and should not be relied upon for developing applications. Any use of the code examples below is completely
at your own risk. IDT MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE NONINFRINGEMENT, QUALITY, SAFETY OR SUITABILITY
OF THE CODE, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICU­LAR PURPOSE, OR NON-INFRINGEMENT. FURTHER, IDT MAKES NO REPRESENTATIONS OR WARRANTIES AS TO THE TRUTH, ACCURACY OR COMPLETENESS
OF ANY STATEMENTS, INFORMATION OR MATERIALS CONCERNING CODE EXAMPLES CONTAINED IN ANY IDT PUBLICATION OR PUBLIC DISCLOSURE OR
THAT IS CONTAINED ON ANY IDT INTERNET SITE. IN NO EVENT WILL IDT BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, INDIRECT, PUNITIVE OR
SPECIAL DAMAGES, HOWEVER THEY MAY ARISE, AND EVEN IF IDT HAS BEEN PREVIOUSLY ADVISED ABOUT THE POSSIBILITY OF SUCH DAMAGES. The code
examples also may be subject to United States export control laws and may be subject to the export or import laws of other countries and it is your responsibility to comply with
any applicable laws or regulations.

Integrated Device Technology's products are not authorized for use as critical components in life support devices or systems unless a specific written agreement pertaining to
such intended use is executed between the manufacturer and an officer of IDT.

1. Life support devices or systems are devices or systems which (a) are intended for surgical implant into the body or (b) support or sustain life and whose failure to perform,
when properly used in accordance with instructions for use provided in the labeling, can be reasonably expected to result in a significant injury to the user.

2. A critical component is any components of a life support device or system whose failure to perform can be reasonably expected to cause the failure of the life support device
or system, or to affect its safety or effectiveness.

IDT, the IDT logo, and Integrated Device Technology are trademarks or registered trademarks of Integrated Device Technology, Inc.

CODE DISCLAIMER

LIFE SUPPORT POLICY

Notes

®

About This Manual

Introduction

This user manual includes hardware and software information on the 89HPES64H16G2, a member of
IDT’s PRECISE™ family of PCI Express® switching solutions offering the next-generation I/O interconnect
standard.

Finding Additional Information

Information not included in this manual such as mechanicals, package pin-outs, and electrical character­istics can be found in the data sheet for this device, which is available from the IDT website (www.idt.com)
as well as through your local IDT sales representative.

Content Summary

Chapter 1, “PES64H16G2 Device Overview,” provides a complete introduction to the performance
capabilities of the 89HPES64H16G2. Included in this chapter is a summary of features for the device as
well as a system block diagram and pin description.

Chapter 2, “Architectural Overview,” provides a high level architectural overview of the PES64H16G2
device.

Chapter 3, “Switch Core,” provides a description of the PES64H16G2 switch core.

Chapter 4, “Clocking,” provides a description of the PES64H16G2 clocking architecture.

Chapter 5, “Reset and Initialization,” describes the PES64H16G2 reset operations and initialization

procedure.

Chapter 6, “Switch Partitions,” describes how the PES64H16G2 supports up to 16 active switch parti­tions.

Chapter 7, “Link Operation,” describes the operation of the link feature including polarity inversion,
link width negotiation, and lane reversal.

Chapter 8, “SerDes,” describes basic functionality and controllability associated with the Serialiazer­Deserializer (SerDes) block in PES64H16G2 ports.

Chapter 9, “Theory of Operation,” describes the general operational behavior of the PES64H16G2.

Chapter 10, “Hot-Plug and Hot-Swap,” describes the behavior of the hot-plug and hot-swap features

in the PES64H16G2.

Chapter 11, “Power Management,” describes the power management capability structure located in
the configuration space of each PCI-to-PCI bridge in the PES64H16G2.

Chapter 12, “General Purpose I/O,” describes how the 32 General Purpose I/O (GPIO) pins may be
individually configured as general purpose inputs, general purpose outputs, or alternate functions.

Chapter 13, “SMBus Interfaces,” describes the operation of the 2 SMBus interfaces on the
PES64H16G2.

Chapter 14, “Multicast,” describes how the multicast capability enables a single TLP to be forwarded
to multiple destinations.

Chapter 15, “Register Organization,” describes the organization of all the software visible registers in
the PES64H16G2 and provides the address space for those registers.

Chapter 16, “PCI to PCI Bridge and Proprietary Port Specific Registers,” lists the Type 1 configura­tion header registers in the PES64H16G2 and provides a description of each bit in those registers.

PES64H16G2 User Manual 1 April 5, 2013

IDT

Notes

1 2 3 4

high-to-low

transition

low-to-high

transition

single clock cycle

Chapter 17, “Switch Control and Status Registers,” lists the switch control and status registers in the
PES64H16G2 and provides a description of each bit in those registers.

Chapter 18, “JTAG Boundary Scan,” discusses an enhanced JTAG interface, including a system logic
TAP controller, signal definitions, a test data register, an instruction register, and usage considerations.

Signal Nomenclature

To avoid confusion when dealing with a mixture of “active-low” and “active-high” signals, the terms
assertion and negation are used. The term assert or assertion is used to indicate that a signal is active or
true, independent of whether that level is represented by a high or low voltage. The term negate or negation
is used to indicate that a signal is inactive or false.

To define the active polarity of a signal, a suffix will be used. Signals ending with an ‘N’ should be inter­preted as being active, or asserted, when at a logic zero (low) level. All other signals (including clocks,
buses and select lines) will be interpreted as being active, or asserted when at a logic one (high) level.

To define buses, the most significant bit (MSB) will be on the left and least significant bit (LSB) will be on
the right. No leading zeros will be included.

Throughout this manual, when describing signal transitions, the following terminology is used. Rising
edge indicates a low-to-high (0 to 1) transition. Falling edge indicates a high-to-low (1 to 0) transition. These
terms are illustrated in Figure 1.

Figure 1 Signal Transitions

Numeric Representations

To represent numerical values, either decimal, binary, or hexadecimal formats will be used. The binary
format is as follows: 0bDDD, where “D” represents either 0 or 1; the hexadecimal format is as follows:
0xDD, where “D” represents the hexadecimal digit(s); otherwise, it is decimal.

The compressed notation ABC[x|y|z]D refers to ABCxD, ABCyD, and ABCzD.

The compressed notation ABC[x..y]D refers to ABCxD, ABC(x+1)D, ABC(x+2)D,... ABCyD.

Data Units

The following data unit terminology is used in this document.

PES64H16G2 User Manual 2 April 5, 2013

IDT

Notes

0 1 2 3

bit 0bit 31

Address of Bytes within Words: Big Endian

3 2 1 0

bit 0bit 31

Address of Bytes within Words: Little Endian

Ter m Words By tes Bi ts

Byte 1/2 1 8

Word 1 2 16

Doubleword (Dword) 2 4 32

Quadword (Qword) 4 8 64

Table 1 Data Unit Terminology

In quadwords, bit 63 is always the most significant bit and bit 0 is the least significant bit. In double­words, bit 31 is always the most significant bit and bit 0 is the least significant bit. In words, bit 15 is always
the most significant bit and bit 0 is the least significant bit. In bytes, bit 7 is always the most significant bit
and bit 0 is the least significant bit.

The ordering of bytes within words is referred to as either “big endian” or “little endian.” Big endian
systems label byte zero as the most significant (leftmost) byte of a word. Little endian systems label byte
zero as the least significant (rightmost) byte of a word. See Figure 2.

Figure 2 Example of Byte Ordering for “Big Endian” or “Little Endian” System Definition

Register Terminology

Software in the context of this register terminology refers to modifications made by PCIe root configura­tion writes, writes to registers made through the slave SMBus interface, or serial EEPROM register initial­ization. See Table 2.

PES64H16G2 User Manual 3 April 5, 2013

IDT

Notes

Type Abbreviation Description

Hardware Initialized HWINIT Register bits are initialized by firmware or hardware mechanisms

such as pin strapping or serial EEPROM. (System firmware hard­ware initialization is only allowed for system integrated devices.)
Bits are read-only after initialization and can only be reset (for
write-once by firmware) with reset.

Read Only and Clear RC Software can read the register/bits with this attribute. Reading the

value will automatically cause the register/bit to be reset to zero.
Writing to a RC location has no effect.

Read Clear and Write RCW Software can read the register/bits with this attribute. Reading the

value will automatically cause the register/bits to be reset to zero.
Writes cause the register/bits to be modified.

Reserved Reserved The value read from a reserved register/bit is undefined. Thus,

software must deal correctly with fields that are reserved. On
reads, software must use appropriate masks to extract the defined
bits and not rely on reserved bits being any particular value. On
writes, software must ensure that the values of reserved bit posi­tions are preserved. That is, the values of reserved bit positions
must first be read, merged with the new values for other bit posi­tions and then written back.

Read Only RO Software can only read registers/bits with this attribute. Contents

are hardwired to a constant value or are status bits that may be
set and cleared by hardware. Writing to a RO location has no
effect.

Read and Write RW Software can both read and write bits with this attribute.

Read and Write Clear RW1C Software can read and write to registers/bits with this attribute.

However, writing a value of zero to a bit with this attribute has no
effect. A RW1C bit can only be set to a value of 1 by a hardware
event. To clear a RW1C bit (i.e., change its value to zero) a value
of one must be written to the location. An RW1C bit is never
cleared by hardware.

Read and Write when
Unlocked

Sticky Sticky Register/bits with this designation take on their initial value as a

Switch Sticky SWSticky Register/bits with this designation take on their initial value as a

RWL Software can read the register/bits with this attribute. Writing to

register/bits with this attribute will only cause the value to be modi­fied if the REGUNLOCK bit in the SWCTL register is set. When
the REGUNLOCK bit is cleared, writes are ignored and the regis­ter/bits are effectively read-only.
Fields with this attribute are implicitly SWSticky (i.e., their value is
preserved across all resets, except switch fundamental reset).

result of a switch fundamental reset or partition fundamental reset.
Other resets have no effect.

result of a switch fundamental reset. Other resets have no effect.

Table 2 Register Terminology

PES64H16G2 User Manual 4 April 5, 2013

IDT

Notes

Use of Hypertext

In Chapter 15, Tables 15.4 through 15.6 contain register names and page numbers highlighted in blue
under the Register Definition column. In pdf files, users can jump from this source table directly to the regis­ters by clicking on the register name in the source table. Each register name in the table is linked directly to
the appropriate register in the register section of Chapters 14 and 15. To return to the source table after
having jumped to the register section, click on the same register name (in blue) in the register section.

Reference Documents

[1] PCI Express Base Specification Revision 2.0., December 20, 2006, PCI-SIG.

[2] Multicast Engineering Change Notice to [1]., May 8, 2008, PCI-SIG.

[3] Internal Error Reporting Engineering Change Notice to [1]., April 24, 2008, PCI-SIG.

[4] SMBus Specification, Version 2.0, August 3, 2000, SBS Implementers Forum.

Revision History

October 6, 2008: Initial publication of preliminary user manual.

January 12, 2009: On page 3-5, revised Port Arbitration section. In Table 9.10, under Description for

Function in D3Hot state, changed reference to 11-1 instead of 10-1.

January 22, 2009: In Chapter 13, Table 13.17, changed the description for bit USA. In Chapter 16,
PCIEDCTL register, changed the description for bit ERO.

February 9, 2009: In Chapter 1: Table 1.4, for Port 0 Serial Data Receive/Transmit signals, deleted
statement that port 0 is the upstream port; Table 1.8, revised Description for SWMODE[3:0]; Table 1.10,
added “3.3V is preferred” for signal V

Software Management of Link Speed.

February 18, 2009: In Chapter 7, added a note under L2/L3 Ready in Link States section. In Chapter
16, modified Description for REG and EREG fields in the ECFGADDR register, GADDR field in the
GASAADDR register, DATA field in the GASADATA register, and RSE field in the SECSTS register. In
Chapters 15 and 16, added PHYLSTATE0 (0x540) register.

April 6, 2009: In Chapter 5, revised text in Partition Hot Reset section and added text in several sub­sections of Switch Mode Dependent Initialization section. Revised Chapter 6, Switch Partitions, including
new sections Partition State Change, and Partition and Port Configuration. Made changes in register fields
in Chapters 16 and 17 (Bridge and Switch Registers), to conform with device specification and validation.

April 21, 2009: In Table 1.8, deleted reference to pull-down value of 251K ohm resistor for all
PxMERGEN pins. In Footnote 1 for Table 1.11, internal resistor pull-down value was changed to 91K ohms.
In Chapter 17, changed “Bit x in this field corresponds to GPIO pin (x+31)” to “Bit x in this field corresponds
to GPIO pin (x+32)” in the GPIOFUNC1, GPIOCFG1, and GPIOD1 registers. Changed title for Table 13.13.

April 27, 2009: ZB silicon was added to Table 1.3.

May 6, 2009: In Chapter 5, under section Switch Fundamental Reset, deleted bullet referencing

SWFRST bit.

May 14, 2009: In Table 1.11, changed CML to HCSL for PCIe reference clocks.

May 28, 2009: In Chapter 7, revised Crosslink section. In Chapter 8, Tables 8.2, 8.3 and 8.4, changed

column title of TX_EQ_MODE to reflect the register field used to control TX equalization depending on the
operating mode of the link (e.g., TX_EQ_3DBG1). In Chapter 13, revised Introduction section and deleted
references to LAERR bit in Table 13.3, Table 13.17, and Figure 13.8. In Chapter 15, added section Partial­Byte Access to Word and DWord Registers. In Chapter 17, added bit BDISCARD to the Switch Control
register and changed bit 26 in the SMBus Status register from LAERR to Reserved.

I/O. In Chapter 7, deleted footnote in 2nd paragraph under section

DD

PES64H16G2 User Manual 5 April 5, 2013

IDT

Notes

June 16, 2009: In Chapter 5, revised Table 5.1 and revised text in sections Partition Hot Reset and Port
Mode Change Reset. In Chapter 6, revised text in the following sections: Partition State, Partition State
Change via Other Methods, Port Operating Mode Change via EEPROM Loading, Port Operating Mode
Change via Other Methods, and Dynamic Reconfiguration. Also, deleted section Hot Reset. In Chapter 7,
revised Crosslink section. In Chapter 17, revised the description of the STATE bit in the Switch Partition x
Control register and Switch Partition x Status register, and revised the description of the OMA bit in the
Switch Port x Control register. Also, removed reference to RDETECT bit in Hot-Plug Configuration Control
register.

June 22, 2009: In Table 1.11, System Pins section, changed CLKMODE[1:0] to pull-up.

July 30, 2009: In Chapter 17, Switch Registers, changed bits 19:18 in the SMBus Control register from

SSMBMODE to Reserved.

September 22, 2009: Modified Chapter 4, Clocking. In Chapter 8, SerDes, modified Table 8.2 and
added Note before Figure 8.1. Modified section Transaction Layer Error Pollution in Chapter 9, Theory of
Operation. In Chapter 17, Switch Registers, modified description of the LANESEL field in the SxCTL
register and modified description of the RXEQZ and RXEQB fields in the SxRXEQLCTL register.

September 28, 2009: ZC silicon was added to Table 1.3.

November 6, 2009: In Chapter 3, Switch Core, modified text and figures in Operation section. In

Chapter 4, Clocking, modified Introduction section. In Chapter 6, Switch Partitions, added new sections:
Partition State Change Latency, Port Operating Mode Change Latency, and Dynamic Reconfiguration. In
Chapter 15, Register Organization, added new section Register Side-Effects. In Chapter 16, Bridge Regis­ters, modified description for DVADJ bit in the Requester Metering Control register.

November 11, 2009: In Chapter 8, SerDes, deleted settings greater than 0x0F in Tables 8.7 and 8.8.

December 7, 2009: In Chapter 7, added reference in section Link width Negotiation to the MAXLNK-

WDTH field in the PCI Express Link Capabilities register. In Chapter 15, added new sub-section Limitations
under Register Side-Effects. In Chapter 16, modified Description for the MAXLNKWDTH field in the
PCIELCAP register and added field RCVD_OVRD to the SerDes Configuration register. In Chapter 17,
added field DDDNC to the Switch Control register and modified Description for the BLANK field in the
SMBus Status register.

December 14, 2009: Deleted all references to support for Weighted Round Robin arbitration.

January 21, 2010: Removed Preliminary from title.

February 10, 2010: In Chapter 5, added new Port Merging section. In Table 1.8, added reference to Port

Merging section in PxxMERGEN pin description.

December 8, 2010: In Chapter 18, deleted PERSTN, GLK1, and SMODE from Table 18.1.

February 2, 2011: In Table 9.13, revised text in Action Taken column for ACS Source Validation. In

Chapter 16, added footnote to STAS bit in PCISTS and SECSTS registers.

February 17, 2011: In Table 13.10, changed Type from Output to Input.

May 18, 2011: In Chapter 8, section Low-Swing Transmitter Voltage Mode, the reference in the first

paragraph to the LSE bit being in the SerDes Control register was changed to the SerDes Configuration
register.

June 28, 2011: In Chapter 17, added bit 26,

TX_SLEW_C, to the SerDes x Transmitter Lane Control 0 register.

July 8, 2011: In Chapter 16, removed table footnotes from PCISTS and SECSTS registers, added
Reserved bits 31:24 to AERUEM and AERUESV registers, and added last sentence to each description in
the PCIESCTLIV register. In Chapter 17, added FEN and

FCAPSEL fields to SWPART[x]CTL register and

SWPORT[x]CTL registers, added PFAILOVER and SFAILOVER fields to SWPART[X]STS register and
SWPORT[x]STS register, adjusted bit fields in GPIOCFG1 and GPIOD1 registers, and added SSMBMODE
field to the SMBUSCTL register.

PES64H16G2 User Manual 6 April 5, 2013

IDT

Notes

August 31, 2011: In Chapter 2, page 2-1, added bullet to explain behavior of an odd numbered port
when it is merged with its even counterpart. In Chapter 6, added text to Downstream Switch Port section. In
Chapter 7, revised text in section Link Width Negotiation in the Presence of Bad Lanes. In Chapter 8,
revised Table 8.1 and text under this table, revised text in section Programmable De-emphasis Adjustment,
added headings to Figures 8.1 through 8.3, and added paragraph after Figure 8.3. In Chapter 14, revised
text in the Introduction section. In Chapter 16, changed type of MAXLNKSPD field in the PCIELCAP
register from RWL to RO, revised Description for MAXGROUP field in MCCAP register, changed lower to
upper in Description for MCBLKALLH and MCBLKUTH registers. In Chapter 17, deleted references to
Failover capability from several registers.

September 9, 2011: In Chapter 8, added additional reference in last paragraph of section Driver Voltage
Level and Amplitude Boost.

February 7, 2012: In Chapter 13, added footnote for RERR and WERR bits in Table 13.15.

February 23, 2012: Added paragraph after Table 13.15 to explain use of DWord addresses.

January 31, 2013: In Figure 13.8, changed No-ack to Ack between DATALM and DATAUM.

April 5, 2013: In Chapter 17, added USSBRDELAY register.

PES64H16G2 User Manual 7 April 5, 2013

IDT

Notes

PES64H16G2 User Manual 8 April 5, 2013

Notes

Table of Contents

®

About This Manual

Introduction ....................................................................................................................................1

Content Summary ..........................................................................................................................1

Signal Nomenclature .....................................................................................................................2

Numeric Representations ..............................................................................................................2

Data Units ...................................................................................................................................... 2

Register Terminology ..................................................................................................................... 3

Use of Hypertext ............................................................................................................................ 5

Reference Documents ................................................................................................................... 5

Revision History .............................................................................................................................5

PES64H16G2 Device Overview

Introduction .....................................................................................................................................1-1

Features.......................................................................................................................................... 1-1

Logic Diagram................................................................................................................................. 1-5

System Identification....................................................................................................................... 1-6

Vendor ID................................................................................................................................1-6

Device ID ................................................................................................................................1-6

Revision ID .............................................................................................................................1-6

JTAG ID..................................................................................................................................1-6

SSID/SSVID............................................................................................................................1-6

Device Serial Number Enhanced Capability........................................................................... 1-6

Pin Description................................................................................................................................ 1-7

Pin Characteristics........................................................................................................................1-18

Architectural Overview

Introduction .....................................................................................................................................2-1

Switch Partitioning ..........................................................................................................................2-2

Dynamic Reconfiguration................................................................................................................ 2-3

Switch Core

Introduction .....................................................................................................................................3-1

Switch Core Architecture ................................................................................................................3-1

Ingress Buffer .........................................................................................................................3-1

Egress Buffer.......................................................................................................................... 3-2

Crossbar Interconnect ............................................................................................................3-3

Datapaths ...............................................................................................................................3-3

Packet Ordering.............................................................................................................................. 3-4

Arbitration........................................................................................................................................3-5

Port Arbitration........................................................................................................................ 3-6

Cut-Through Routing ......................................................................................................................3-6

Request Metering............................................................................................................................3-8

Operation..............................................................................................................................3-10

Completion Size Estimation.................................................................................................. 3-12

Internal Errors ...............................................................................................................................3-13

Switch Time-Outs .................................................................................................................3-14

Memory SECDED ECC Protection....................................................................................... 3-14

PES64H16G2 User Manual i April 5, 2013

IDT Table of Contents

Notes

End-to-End Data Path Parity Protection ............................................................................... 3-14

Clocking

Introduction ..................................................................................................................................... 4-1

Port Clocking Mode ........................................................................................................................4-1

Reset and Initialization

Introduction ..................................................................................................................................... 5-1

Boot Configuration Vector............................................................................................................... 5-2

Switch Fundamental Reset............................................................................................................. 5-3

Switch Mode Dependent Initialization..................................................................................... 5-6

Port Merging ................................................................................................................................... 5-7

Partition Resets ..............................................................................................................................5-8

Partition Fundamental Reset .................................................................................................. 5-8

Partition Hot Reset .................................................................................................................5-9

Partition Upstream Secondary Bus Reset .............................................................................. 5-9

Partition Downstream Secondary Bus Reset .......................................................................5-10

Port Mode Change Reset .............................................................................................................5-10

Switch Partitions

Introduction ..................................................................................................................................... 6-1

Switch Partitions ............................................................................................................................. 6-1

Partition Configuration ............................................................................................................ 6-1

Partition State ......................................................................................................................... 6-2

Partition State Change ...........................................................................................................6-3

Switch Ports.................................................................................................................................... 6-4

Switch Port Mode ...................................................................................................................6-4

Port Operating Mode Change......................................................................................................... 6-7

Common Operating Mode Change Behavior .........................................................................6-9

No Action Mode Change Behavior ....................................................................................... 6-14

Reset Mode Change Behavior .............................................................................................6-14

Hot Reset Mode Change Behavior....................................................................................... 6-15

Partition and Port Configuration.................................................................................................... 6-15

Static Reconfiguration ..........................................................................................................6-15

Dynamic Reconfiguration .....................................................................................................6-16

Link Operation

Introduction ..................................................................................................................................... 7-1

Polarity Inversion ............................................................................................................................7-1

Lane Reversal................................................................................................................................. 7-1

Link Width Negotiation.................................................................................................................... 7-5

Link Width Negotiation in the Presence of Bad Lanes ...........................................................7-6

Dynamic Link Width Reconfiguration..............................................................................................7-6

Link Speed Negotiation................................................................................................................... 7-6

Link Speed Negotiation in the PES64H16G2 ......................................................................... 7-7

Software Management of Link Speed ....................................................................................7-8

Link Retraining................................................................................................................................ 7-9

Link Down ..................................................................................................................................... 7-10

Slot Power Limit Support ..............................................................................................................7-10

Upstream Port ......................................................................................................................7-10

Downstream Port..................................................................................................................7-10

Link States .................................................................................................................................... 7-11

Active State Power Management .................................................................................................7-11

PES64H16G2 User Manual ii April 5, 2013

IDT Table of Contents

Notes

L0s ASPM............................................................................................................................. 7-12

L1 ASPM ..............................................................................................................................7-12

L1 ASPM Entry Rejection Timer ...................................................................................................7-13

Link Status .................................................................................................................................... 7-14

De-emphasis Negotiation .............................................................................................................7-14

Crosslink ....................................................................................................................................... 7-15

Hot Reset Operation on a Crosslink .................................................................................... 7-16

Link Disable Operation on a Crosslink .................................................................................7-16

Gen1 Compatibility Mode .............................................................................................................7-16

SerDes

Introduction ..................................................................................................................................... 8-1

SerDes Numbering and Port Association .......................................................................................8-1

SerDes Transmitter Controls ..........................................................................................................8-1

Driver Voltage Level and Amplitude Boost ............................................................................. 8-1

De-emphasis ..........................................................................................................................8-2

Slew Rate ............................................................................................................................... 8-2

PCI Express Low-Swing Mode ............................................................................................... 8-2

Receiver Equalization ..................................................................................................................... 8-3

Programming of SerDes Controls................................................................................................... 8-3

Programmable Voltage Margining and De-Emphasis..................................................................... 8-3

SerDes Transmitter Control Registers.................................................................................... 8-4

Transmit Margining using the PCI Express Link Control 2 Register............................................. 8-12

Low-Swing Transmitter Voltage Mode..........................................................................................8-13

Receiver Equalization Controls..................................................................................................... 8-14

SerDes Power Management.........................................................................................................8-15

Theory of Operation

Introduction ..................................................................................................................................... 9-1

Transaction Routing........................................................................................................................ 9-1

Interrupts......................................................................................................................................... 9-1

Downstream Port Interrupts.................................................................................................... 9-2

Legacy Interrupt Emulation..................................................................................................... 9-2

Access Control Services................................................................................................................. 9-3

Error Detection and Handling .........................................................................................................9-6

Physical Layer Errors .............................................................................................................9-7

Data Link Layer Errors............................................................................................................ 9-7

Transaction Layer Errors ........................................................................................................9-8

Routing Errors ......................................................................................................................9-16

Bus Locking .................................................................................................................................. 9-17

Hot-Plug and Hot-Swap

Introduction ................................................................................................................................... 10-1

Hot-Plug Signals ........................................................................................................................... 10-3

Port Reset Outputs .......................................................................................................................10-5

Power Enable Controlled Reset Output................................................................................ 10-5

Power Good Controlled Reset Output .................................................................................. 10-6

Hot-Plug Events............................................................................................................................ 10-6

Legacy System Hot-Plug Support................................................................................................. 10-7

Hot-Swap ...................................................................................................................................... 10-8

PES64H16G2 User Manual iii April 5, 2013

IDT Table of Contents

Notes

Power Management

Introduction ................................................................................................................................... 11-1

PME Messages............................................................................................................................. 11-3

PCI Express Power Management Fence Protocol .......................................................................11-3

Upstream Switch Port or Downstream Switch Port Mode ....................................................11-3

Power Budgeting Capability.......................................................................................................... 11-4

General Purpose I/O

Introduction ................................................................................................................................... 12-1

GPIO Configuration ......................................................................................................................12-1

Configured as an Input .........................................................................................................12-1

Configured as an Output ......................................................................................................12-1

Configured as an Alternate Function .................................................................................... 12-1

SMBus Interfaces

Introduction ................................................................................................................................... 13-1

Master SMBus Interface ............................................................................................................... 13-1

Initialization...........................................................................................................................13-1

Serial EEPROM.................................................................................................................... 13-2

Initialization from Serial EEPROM........................................................................................ 13-2

Programming the Serial EEPROM .......................................................................................13-5

I/O Expanders....................................................................................................................... 13-6

Slave SMBus Interface ...............................................................................................................13-14

Initialization.........................................................................................................................13-14

SMBus Transactions ..........................................................................................................13-15

Multicast

Introduction ................................................................................................................................... 14-1

Addressing and Routing ...............................................................................................................14-1

Multicast TLP Determination ................................................................................................14-1

Multicast TLP Routing ..........................................................................................................14-4

Multicast Egress Processing ................................................................................................14-4

Register Organization

Introduction ................................................................................................................................... 15-1

Partial-Byte Access to Word and DWord Registers .............................................................15-2

Register Side-Effects............................................................................................................15-2

Address Maps...............................................................................................................................15-3

PCI-to-PCI Bridge Registers................................................................................................. 15-3

Capability Structures ............................................................................................................15-3

IDT Proprietary Port Specific Registers.............................................................................. 15-10

Switch Configuration and Status Registers ........................................................................15-12

PES64H16G2 User Manual iv April 5, 2013

IDT Table of Contents

Notes

PCI to PCI Bridge and Proprietary Port Specific Registers

Type 1 Configuration Header Registers .......................................................................................16-1

PCI Express Capability Structure ...............................................................................................16-11

Power Management Capability Structure ...................................................................................16-27

Message Signaled Interrupt Capability Structure .......................................................................16-29

Subsystem ID and Subsystem Vendor ID ..................................................................................16-31

Extended Configuration Space Access Registers ......................................................................16-31

Advanced Error Reporting (AER) Enhanced Capability..............................................................16-32

Device Serial Number Enhanced Capability............................................................................... 16-41

PCI Express Virtual Channel Capability .....................................................................................16-42

Power Budgeting Enhanced Capability ......................................................................................16-47

ACS Extended Capability ...........................................................................................................16-49

Multicast Extended Capability..................................................................................................... 16-52

Proprietary Port Specific Registers............................................................................................. 16-57

Port Control and Status Registers ...................................................................................... 16-57

Internal Error Control and Status Registers........................................................................ 16-59

Physical Layer Control and Status Registers .....................................................................16-67

Power Management Control and Status Registers ............................................................16-70

Request Metering ...............................................................................................................16-70

Global Address Space Access Registers ........................................................................... 16-72

Switch Configuration and Status Registers

Switch Control and Status Registers ............................................................................................17-1

Internal Switch Timer .................................................................................................................... 17-3

Switch Partition and Port Registers ..............................................................................................17-4

Protection...................................................................................................................................... 17-7

SerDes Control and Status Registers...........................................................................................17-7

General Purpose I/O Registers................................................................................................... 17-15

Hot-Plug and SMBus Interface Registers ................................................................................... 17-19

JTAG Boundary Scan

Introduction ................................................................................................................................... 18-1

Test Access Point ......................................................................................................................... 18-1

Signal Definitions .......................................................................................................................... 18-1

Boundary Scan Chain...................................................................................................................18-3

Test Data Register (DR) ...............................................................................................................18-5

Boundary Scan Registers.....................................................................................................18-6

Instruction Register (IR)................................................................................................................18-7

EXTEST................................................................................................................................ 18-8

SAMPLE/PRELOAD............................................................................................................. 18-8

BYPASS ............................................................................................................................... 18-8

CLAMP ................................................................................................................................. 18-9

IDCODE................................................................................................................................ 18-9

VALIDATE ............................................................................................................................ 18-9

EXTEST_TRAIN................................................................................................................... 18-9

EXTEST_PULSE................................................................................................................ 18-10

RESERVED........................................................................................................................ 18-10

Usage Considerations ........................................................................................................18-10

PES64H16G2 User Manual v April 5, 2013

IDT Table of Contents

Notes

PES64H16G2 User Manual vi April 5, 2013

Notes

List of Tables

®

Table 1.1 Initial Configuration Register Settings for PES64H16G2..................................................... 1-4

Table 1.2 PES64H16G2 Device IDs.................................................................................................... 1-6

Table 1.3 PES64H16G2 Revision ID................................................................................................... 1-6

Table 1.4 PCI Express Interface Pins.................................................................................................. 1-7

Table 1.5 Reference Clock Pins ..........................................................................................................1-9

Table 1.6 SMBus Interface Pins .......................................................................................................... 1-9

Table 1.7 General Purpose I/O Pins.................................................................................................... 1-9

Table 1.8 System Pins....................................................................................................................... 1-13

Table 1.9 Test Pins............................................................................................................................ 1-16

Table 1.10 Power, Ground, and SerDes Resistor Pins .......................................................................1-16

Table 1.11 Pin Characteristics.............................................................................................................1-18

Table 3.1 IFB Buffer Sizes................................................................................................................... 3-1

Table 3.2 EFB Buffer Sizes .................................................................................................................3-2

Table 3.3 Replay Buffer Storage Limit................................................................................................. 3-3

Table 3.4 Packet Ordering Rules in the PES64H16G2 .......................................................................3-5

Table 3.5 Conditions for Cut-Through Transfers ................................................................................. 3-7

Table 3.6 Request Metering Decrement Value..................................................................................3-11

Table 4.1 Initial Port Clocking Mode and Slot Clock Configuration State............................................ 4-2

Table 5.1 PES64H16G2 Reset Precedence........................................................................................5-1

Table 5.2 Boot Configuration Vector Signals....................................................................................... 5-2

Table 5.3 Switch Mode Dependent Register Initialization ...................................................................5-6

Table 7.1 Crosslink Port Groups........................................................................................................7-15

Table 7.2 Gen1 Compatibility Mode: bits cleared in training sets...................................................... 7-17

Table 8.1 SerDes Transmit Level Controls in the S[x]TXLCTL0 and S[x]TXLCTL1 Registers............8-5

Table 8.2 SerDes Transmit Driver Settings in Gen1 Mode..................................................................8-6

Table 8.3 SerDes Transmit Driver Settings in Gen2 Mode with -3.5dB de-emphasis......................... 8-7

Table 8.4 SerDes Transmit Driver Settings in Gen2 Mode with -6.0dB de-emphasis......................... 8-8

Table 8.5 Transmitter Slew Rate Settings .........................................................................................8-11

Table 8.6 PCI Express Transmit Margining Levels Supported by the PES64H16G2........................ 8-12

Table 8.7 SerDes Transmit Drive Swing in Low Swing Mode at Gen1 Speed ..................................8-13

Table 8.8 SerDes Transmit Drive Swing in Low Swing Mode at Gen2 Speed ..................................8-14

Table 9.1 Switch Routing Methods......................................................................................................9-1

Table 9.2 Downstream Port Interrupts................................................................................................. 9-2

Table 9.3 Downstream to Upstream Port Interrupt Routing Based on Device Number.......................9-3

Table 9.4 Prioritization of ACS Checks for Request TLPs...................................................................9-5

Table 9.5 Prioritization of ACS Checks for Completion TLPs.............................................................. 9-6

Table 9.6 TLP Types Affected by ACS Checks................................................................................... 9-6

Table 9.7 Physical Layer Errors...........................................................................................................9-7

Table 9.8 Data Link Layer Errors......................................................................................................... 9-7

Table 9.9 Transaction Layer Errors associated with the PCI-to-PCI Bridge Function......................... 9-9

Table 9.10 Conditions Handled as Unsupported Requests (UR) by the PCI-to-PCI Bridge Function. 9-11

Table 9.11 Ingress TLP Formation Checks Associated with the PCI-to-PCI Bridge Function.............9-11

Table 9.12 Egress Malformed TLP Error Checks................................................................................9-12

Table 9.13 ACS Violations for Ports Operating in Downstream Switch Port Mode ............................. 9-13

Table 9.14 Prioritization of Transaction Layer Errors ..........................................................................9-14

Table 10.1 Port Hot Plug Signals.........................................................................................................10-3

Table 10.2 Negated Value of Unused Hot-Plug Output Signals.......................................................... 10-4

Table 11.1 PES64H16G2 Power Management State Transition Diagram .......................................... 11-2

Table 12.1 GPIO Pin Configuration..................................................................................................... 12-1

PES64H16G2 User Manual vii April 5, 2013

IDT List of Tables

Notes

Table 12.2 General Purpose I/O Pin Alternate Function .....................................................................12-2

Table 12.3 GPIO Alternate Function Pins ...........................................................................................12-3

Table 13.1 Serial EEPROM SMBus Address...................................................................................... 13-2

Table 13.2 PES64H16G2 Compatible Serial EEPROMs ....................................................................13-2

Table 13.3 Serial EEPROM Initialization Errors ..................................................................................13-5

Table 13.4 I/O Expander Function Allocation...................................................................................... 13-6

Table 13.5 I/O Expander Default Output Signal Value........................................................................ 13-7

Table 13.6 Pin Mapping for I/O Expanders 0 through 7 ......................................................................13-9

Table 13.7 Pin Mapping I/O Expander 8 ...........................................................................................13-10

Table 13.8 Pin Mapping I/O Expander 9 ...........................................................................................13-11

Table 13.9 Pin Mapping I/O Expander 10 .........................................................................................13-12

Table 13.10 I/O Expander 11 - Partition Fundamental Reset Inputs................................................... 13-12

Table 13.11 I/O Expander 12 - Link Up Status....................................................................................13-13

Table 13.12 I/O Expander 13 - Link Activity Status ............................................................................. 13-14

Table 13.13 Slave SMBus Address..................................................................................................... 13-14

Table 13.14 Slave SMBus Command Code Fields ............................................................................. 13-15

Table 13.15 CSR Register Read or Write Operation Byte Sequence .................................................13-16

Table 13.16 CSR Register Read or Write CMD Field Description.......................................................13-17

Table 13.17 Serial EEPROM Read or Write Operation Byte Sequence .............................................13-17

Table 13.18 Serial EEPROM Read or Write CMD Field Description................................................... 13-18

Table 15.1 Global Address Space Organization .................................................................................15-1

Table 15.2 Default PCI Capability List Linkage ................................................................................... 15-4

Table 15.3 Default PCI Express Capability List Linkage .....................................................................15-4

Table 15.4 PCI-to-PCI Bridge Configuration Space Registers............................................................ 15-6

Table 15.5 Proprietary Port Specific Registers.................................................................................. 15-11

Table 15.6 Switch Configuration and Status .....................................................................................15-13

Table 18.1 JTAG Pin Descriptions ......................................................................................................18-2

Table 18.2 Boundary Scan Chain........................................................................................................ 18-3

Table 18.3 Instructions Supported by the JTAG Boundary Scan........................................................ 18-8

Table 18.4 System Controller Device Identification Register ..............................................................18-9

PES64H16G2 User Manual viii April 5, 2013

Notes

List of Figures

®

Figure 1.1 PES64H16G2 Block Diagram ............................................................................................1-4

Figure 1.2 PES64H16G2 Logic Diagram ............................................................................................1-5

Figure 2.1 PES64H16G2 Block Diagram ............................................................................................2-1

Figure 2.2 Transparent PCIe Switch ...................................................................................................2-2

Figure 2.3 Partitionable PCI Express Switch ......................................................................................2-2

Figure 3.1 Crossbar Connection to Port Ingress and Egress Buffers .................................................3-3

Figure 3.2 Architectural Model of Arbitration .......................................................................................3-6

Figure 3.3 PCIe Switch Static Rate Mismatch ....................................................................................3-9

Figure 3.4 PCIe Switch Static Rate Mismatch ....................................................................................3-9

Figure 3.5 Request Metering Count and Initial Value Loaded ..........................................................3-10

Figure 3.6 Decrement Value and Decrement Value Adjustment .......................................................3-11

Figure 3.7 Request Metering Counter Decrement Operation ............................................................3-12

Figure 3.8 Non-Posted Read Request Completion Size Estimate Computation ...............................3-12

Figure 4.1 Logical Representation of the PES64H16G2 Clocking Architecture ..................................4-1

Figure 5.1 Switch Fundamental Reset with Serial EEPROM Initialization ..........................................5-5

Figure 5.2 Fundamental Reset Using RSTHALT to Keep Device in Quasi-Reset State .....................5-5

Figure 6.1 Allowable Partition State Transitions .................................................................................6-2

Figure 7.1 Unmerged Port Lane Reversal for Maximum Link Width of x4 ..........................................7-2

Figure 7.2 Unmerged Port Lane Reversal for Maximum Link Width of x2 ..........................................7-2

Figure 7.3 Merged Port Lane Reversal for Maximum Link Width of x2) ..............................................7-3

Figure 7.4 Merged Port Lane Reversal for Maximum Link Width of x4 ...............................................7-4

Figure 7.5 Merged Port Lane Reversal for Maximum Link Width of x8 ...............................................7-5

Figure 7.6 PES64H16G2 ASPM Link Sate Transitions .....................................................................7-11

Figure 8.1 De-emphasis Applied on Link as a Function of the Fine de-emphasis and Transmit

Drive Level Controls, when the PHY Operates in Gen1 Data Rate with -3.5 dB

Nominal de-emphasis ......................................................................................................8-10

Figure 8.2 De-emphasis Applied on Link as a Function of the Fine de-emphasis and Transmit

Drive Level Controls, when the PHY Operates in Gen2 Data Rate with -3.5 dB

Nominal de-emphasis ......................................................................................................8-10

Figure 8.3 De-emphasis Applied on Link as a Function of the Fine de-emphasis and Transmit

Drive Level Controls, when the PHY Operates in Gen1 Data Rate with -6.0 dB

Nominal de-emphasis ......................................................................................................8-11

Figure 9.1 ACS Source Validation Example .......................................................................................9-4

Figure 9.2 ACS Peer-to-Peer Request Re-direct at a Downstream Port ............................................9-4

Figure 9.3 ACS Upstream Forwarding Example .................................................................................9-5

Figure 9.4 Error Checking and Logging on a Received TLP .............................................................9-15

Figure 10.1 Hot-Plug on Switch Downstream Slots Application ..........................................................10-1

Figure 10.2 Hot-Plug with Switch on Add-In Card Application ............................................................10-2

Figure 10.3 Hot-Plug with Carrier Card Application ............................................................................10-2

Figure 10.4 Power Enable Controlled Reset Output Mode Operation ................................................10-5

Figure 10.5 Power Good Controlled Reset Output Mode Operation ...................................................10-6

Figure 10.6 PES64H16G2 Hot-Plug Event Signalling .........................................................................10-8

Figure 11.1 PES64H16G2 Power Management State Transition Diagram .........................................11-2

Figure 13.1 Split SMBus Interface Configuration ................................................................................13-1

Figure 13.2 Single Double Word Initialization Sequence Format ........................................................13-3

Figure 13.3 Sequential Double Word Initialization Sequence Format .................................................13-4

Figure 13.4 Configuration Done Sequence Format ............................................................................13-4

Figure 13.5 Slave SMBus Command Code Format ..........................................................................13-15

Figure 13.6 CSR Register Read or Write CMD Field Format ............................................................13-16

PES64H16G2 User Manual ix April 5, 2013

IDT List of Figures

Notes

Figure 13.7 Serial EEPROM Read or Write CMD Field Format ........................................................13-18

Figure 13.8 CSR Register Read Using SMBus Block Write/Read Transactions with PEC Disabled 13-19
Figure 13.9 Serial EEPROM Read Using SMBus Block Write/Read Transactions with PEC

Disabled .........................................................................................................................13-19

Figure 13.10 CSR Register Write Using SMBus Block Write Transactions with PEC Disabled .........13-19

Figure 13.11 Serial EEPROM Write Using SMBus Block Write Transactions with PEC Disabled .....13-20

Figure 13.12 Serial EEPROM Write Using SMBus Block Write Transactions with PEC Enabled ......13-20

Figure 13.13 CSR Register Read Using SMBus Read and Write Transactions with PEC Disabled ..13-20

Figure 14.1 Multicast Group Address Ranges ....................................................................................14-2

Figure 14.2 Multicast Group Address Region Determination ..............................................................14-3

Figure 15.1 PCI-to-PCI Bridge Configuration Space Organization .....................................................15-5

Figure 15.2 Proprietary Port Specific Register Organization ............................................................15-10

Figure 15.3 Switch Configuration and Status Space Organization ...................................................15-12

Figure 18.1 Diagram of the JTAG Logic ..............................................................................................18-1

Figure 18.2 State Diagram of the TAP Controller ...............................................................................18-2

Figure 18.3 Diagram of Observe-only Input Cell .................................................................................18-6

Figure 18.4 Diagram of Output Cell ....................................................................................................18-6

Figure 18.5 Diagram of Bidirectional Cell ............................................................................................18-7

Figure 18.6 Device ID Register Format ...............................................................................................18-9

PES64H16G2 User Manual x April 5, 2013

Notes

Register List

®

ACSCAP - ACS Capability Register (0x324)........................................................................................ 16-49

ACSCTL - ACS Control Register (0x326)............................................................................................. 16-51

ACSECAPH - ACS Extended Capability Header (0x320) .................................................................... 16-49

ACSECV - ACS Egress Control Vector (0x328)................................................................................... 16-52

AERCAP - AER Capabilities (0x100) ................................................................................................... 16-32

AERCEM - AER Correctable Error Mask (0x114) ................................................................................ 16-38

AERCES - AER Correctable Error Status (0x110) ............................................................................... 16-37

AERCTL - AER Control (0x118)........................................................................................................... 16-40

AERHL1DW - AER Header Log 1st Doubleword (0x11C) ................................................................... 16-40

AERHL2DW - AER Header Log 2nd Doubleword (0x120)................................................................... 16-40

AERHL3DW - AER Header Log 3rd Doubleword (0x124).................................................................... 16-40

AERHL4DW - AER Header Log 4th Doubleword (0x128).................................................................... 16-41

AERUEM - AER Uncorrectable Error Mask (0x108) ............................................................................ 16-34

AERUES - AER Uncorrectable Error Status (0x104) ........................................................................... 16-33

AERUESV - AER Uncorrectable Error Severity (0x10C)...................................................................... 16-36

BAR0 - Base Address Register 0 (0x010).............................................................................................. 16-4

BAR1 - Base Address Register 1 (0x014).............................................................................................. 16-4

BCTL - Bridge Control Register (0x03E) .............................................................................................. 16-10

BCVSTS - Boot Configuration Vector Status (0x0004) .......................................................................... 17-2

BIST - Built-in Self Test Register (0x00F) .............................................................................................. 16-4

CAPPTR - Capabilities Pointer Register (0x034) ................................................................................... 16-9

CCODE - Class Code Register (0x009) ................................................................................................. 16-3

CLS - Cache Line Size Register (0x00C)............................................................................................... 16-4

DID - Device Identification Register (0x002) .......................................................................................... 16-1

ECFGADDR - Extended Configuration Space Access Address (0x0F8) ............................................. 16-31

ECFGDATA - Extended Configuration Space Access Data (0x0FC)................................................... 16-32

EEPROMINTF - Serial EEPROM Interface (0x0AD0).......................................................................... 17-24

EROMBASE - Expansion ROM Base Address Register (0x038)........................................................... 16-9

GASAADDR - Global Address Space Access Address (0xFF8).......................................................... 16-72

GASADATA - Global Address Space Access Data (0xFFC)................................................................ 16-72

GASAPROT - Global Address Space Access Protection (0x0700)........................................................ 17-7

GPECTL - General Purpose Event Control (0x0AE8).......................................................................... 17-26

GPESTS - General Purpose Event Status (0x0AEC) .......................................................................... 17-26

GPIOAFSEL0 - General Purpose I/O Alternate Function Select 0 (0x0A98) ....................................... 17-16

GPIOAFSEL1 - General Purpose I/O Alternate Function Select 1 (0x0A9C)....................................... 17-17

GPIOCFG0 - General Purpose I/O Configuration 0 (0x0AA8) ............................................................. 17-18

GPIOCFG1 - General Purpose I/O Configuration 1 (0x0AAC)............................................................. 17-18

GPIOD0 - General Purpose I/O Data 0 (0x0AB0) ................................................................................ 17-18

GPIOD1 - General Purpose I/O Data 1 (0x0AB4) ................................................................................ 17-19

GPIOFUNC0 - General Purpose I/O Function 0 (0x0A90)................................................................... 17-15

GPIOFUNC1 - General Purpose I/O Function 1 (0x0A94)................................................................... 17-16

HDR - Header Type Register (0x00E).................................................................................................... 16-4

HPCFGCTL - Hot-Plug Configuration Control (0x0ABC) ..................................................................... 17-20

HPSIGMAP - Hot-Plug GPIO Signal Map (0x0AB8) ............................................................................ 17-19

IERRORCTL - Internal Error Reporting Control (0x480) ...................................................................... 16-59

IERRORMSK - Internal Error Reporting Mask (0x488) ........................................................................ 16-60

IERRORSEV - Internal Error Reporting Severity (0x48C).................................................................... 16-63

IERRORSTS - Internal Error Reporting Status (0x484) ....................................................................... 16-59

IERRORTST - Internal Error Reporting Test (0x490)........................................................................... 16-65

PES64H16G2 User Manual xi April 5, 2013

IDT Register List

Notes

INTRLINE - Interrupt Line Register (0x03C) ...........................................................................................16-9

INTRPIN - Interrupt PIN Register (0x03D) ..............................................................................................16-9

IOBASE - I/O Base Register (0x01C)......................................................................................................16-5

IOBASEU - I/O Base Upper Register (0x030).........................................................................................16-8

IOEXPADDR0 - SMBus I/O Expander Address 0 (0x0AD8).................................................................17-24

IOEXPADDR1 - SMBus I/O Expander Address 1 (0x0ADC) ................................................................17-25

IOEXPADDR2 - SMBus I/O Expander Address 2 (0x0AE0) .................................................................17-25

IOEXPADDR3 - SMBus I/O Expander Address 3 (0x0AE4) .................................................................17-25

IOLIMIT - I/O Limit Register (0x01D).......................................................................................................16-6

IOLIMITU - I/O Limit Upper Register (0x032)..........................................................................................16-8

L1ASPMRTC - L1 ASPM Rejection Timer Control (0x710) ..................................................................16-70

LANESTS0 - Lane Status 0 (0x51C).....................................................................................................16-67

LANESTS1 - Lane Status 1 (0x520) .....................................................................................................16-68

MBASE - Memory Base Register (0x020)...............................................................................................16-7

MCBARH- Multicast Base Address High (0x33C).................................................................................16-54

MCBARL- Multicast Base Address Low (0x338)...................................................................................16-53

MCBLKALLH- Multicast Block All High (0x34C)....................................................................................16-55

MCBLKALLL- Multicast Block All Low (0x348)......................................................................................16-55

MCBLKUTH - Multicast Block Untranslated High (0x354) ....................................................................16-56

MCBLKUTL- Multicast Block Untranslated Low (0x350).......................................................................16-55

MCCAP - Multicast Capability (0x334)..................................................................................................16-53

MCCAPH - Multicast Enhanced Capability Header (0x330) .................................................................16-52

MCCTL- Multicast Control (0x336)........................................................................................................16-53

MCOVRBARH- Multicast Overlay Base Address High (0x35C)............................................................16-56

MCOVRBARL- Multicast Overlay Base Address Low (0x358)..............................................................16-56

MCRCVH- Multicast Receive High (0x344)...........................................................................................16-54

MCRCVL- Multicast Receive Low (0x340) ............................................................................................16-54

MLIMIT - Memory Limit Register (0x022)................................................................................................16-7

MSIADDR - Message Signaled Interrupt Address (0x0D4)...................................................................16-30

MSICAP - Message Signaled Interrupt Capability and Control (0x0D0) ...............................................16-29

MSIMDATA - Message Signaled Interrupt Message Data (0x0DC)......................................................16-30

MSIUADDR - Message Signaled Interrupt Upper Address (0x0D8) .....................................................16-30

PBUSN - Primary Bus Number Register (0x018)....................................................................................16-5

PCICMD - PCI Command Register (0x004)............................................................................................16-1

PCIECAP - PCI Express Capability (0x040) .........................................................................................16-11

PCIEDCAP - PCI Express Device Capabilities (0x044) ........................................................................16-11

PCIEDCAP2 - PCI Express Device Capabilities 2 (0x064) ...................................................................16-24

PCIEDCTL - PCI Express Device Control (0x048)................................................................................16-13

PCIEDCTL2 - PCI Express Device Control 2 (0x068)...........................................................................16-24

PCIEDSTS - PCI Express Device Status (0x04A) ................................................................................16-14

PCIEDSTS2 - PCI Express Device Status 2 (0x06A) ...........................................................................16-24

PCIELCAP - PCI Express Link Capabilities (0x04C) ............................................................................16-14

PCIELCAP2 - PCI Express Link Capabilities 2 (0x06C) .......................................................................16-25

PCIELCTL - PCI Express Link Control (0x050).....................................................................................16-16

PCIELCTL2 - PCI Express Link Control 2 (0x070)................................................................................16-25

PCIELSTS - PCI Express Link Status (0x052)......................................................................................16-18

PCIELSTS2 - PCI Express Link Status 2 (0x072).................................................................................16-27

PCIESCAP - PCI Express Slot Capabilities (0x054) .............................................................................16-19

PCIESCAP2 - PCI Express Slot Capabilities 2 (0x074) ........................................................................16-27

PCIESCTL - PCI Express Slot Control (0x058).....................................................................................16-21

PCIESCTL2 - PCI Express Slot Control 2 (0x078)................................................................................16-27

PCIESCTLIV - PCI Express Slot Control Initial Value (0x420)..............................................................16-57

PCIESSTS - PCI Express Slot Status (0x05A) .....................................................................................16-23

PCIESSTS2 - PCI Express Slot Status 2 (0x07A) ................................................................................16-27

PCIEVCECAP - PCI Express VC Enhanced Capability Header (0x200) ..............................................16-42

PES64H16G2 User Manual xii April 5, 2013

IDT Register List

Notes

PCISTS - PCI Status Register (0x006) ...................................................................................................16-2

PHYLCFG0 - Phy Link Configuration 0 (0x530)....................................................................................16-68

PHYLSTATE0 - Phy Link State 0 (0x540).............................................................................................16-69

PHYPRBS - Phy PRBS Seed (0x55C)..................................................................................................16-69

PLTIMER - Primary Latency Timer (0x00D)............................................................................................16-4

PMBASE - Prefetchable Memory Base Register (0x024) .......................................................................16-7

PMBASEU - Prefetchable Memory Base Upper Register (0x028)..........................................................16-8

PMCAP - PCI Power Management Capabilities (0x0C0)......................................................................16-27

PMCSR - PCI Power Management Control and Status (0x0C4) ..........................................................16-28

PMLIMIT - Prefetchable Memory Limit Register (0x026) ........................................................................16-7

PMLIMITU - Prefetchable Memory Limit Upper Register (0x02C) ..........................................................16-8

PVCCAP1- Port VC Capability 1 (0x204)..............................................................................................16-42

PVCCAP2- Port VC Capability 2 (0x208)..............................................................................................16-42

PVCCTL - Port VC Control (0x20C) ......................................................................................................16-43

PVCSTS - Port VC Status (0x20E) .......................................................................................................16-43

PWRBCAP - Power Budgeting Capabilities (0x280).............................................................................16-47

PWRBD - Power Budgeting Data (0x288).............................................................................................16-48

PWRBDSEL - Power Budgeting Data Select (0x284)...........................................................................16-48

PWRBDV[7:0] - Power Budgeting Data Value [7:0] (0x300 - 0x31C) ...................................................16-48

PWRBPBC - Power Budgeting Power Budget Capability (0x28C) .......................................................16-48

RID - Revision Identification Register (0x008) ........................................................................................16-3

RMCOUNT - Requester Metering Count (0x88C).................................................................................16-71

RMCTL - Requester Metering Control (0x880) .....................................................................................16-70

S[15:0]CTL - SerDes x Control................................................................................................................17-8

S[15:0]RXEQLCTL - SerDes x Receiver Equalization Lane Control.....................................................17-15

S[15:0]TXLCTL0 - SerDes x Transmitter Lane Control 0........................................................................17-9

S[15:0]TXLCTL1 - SerDes x Transmitter Lane Control 1......................................................................17-12

SBUSN - Secondary Bus Number Register (0x019) ...............................................................................16-5

SECSTS - Secondary Status Register (0x01E) ......................................................................................16-6

SERDESCFG - SerDes Configuration (0x510) .....................................................................................16-67

SLTIMER - Secondary Latency Timer Register (0x01B).........................................................................16-5

SMBUSCTL - SMBus Control (0x0ACC)...............................................................................................17-23

SMBUSSTS - SMBus Status (0x0AC8) ................................................................................................17-21

SNUMCAP - Serial Number Capabilities (0x180) .................................................................................16-41

SNUMLDW - Serial Number Lower Doubleword (0x184) .....................................................................16-41

SNUMUDW - Serial Number Upper Doubleword (0x188).....................................................................16-41

SSIDSSVID - Subsystem ID and Subsystem Vendor ID (0x0F4) .........................................................16-31

SSIDSSVIDCAP - Subsystem ID and Subsystem Vendor ID Capability (0x0F0) .................................16-31

SUBUSN - Subordinate Bus Number Register (0x01A)..........................................................................16-5

SWCTL - Switch Control (0x0000) ..........................................................................................................17-1

SWPART[15:0]CTL - Switch Partition x Control......................................................................................17-4

SWPART[15:0]STS - Switch Partition x Status .......................................................................................17-4

SWPORT[15:0]CTL - Switch Port x Control ............................................................................................17-5

SWPORT[15:0]STS - Switch Port x Status .............................................................................................17-6

USSBRDELAY - Upstream Secondary Bus Reset Delay (0x008C)........................................................17-3

VCR0CAP- VC Resource 0 Capability (0x210).....................................................................................16-43

VCR0CTL- VC Resource 0 Control (0x214)..........................................................................................16-44

VCR0STS - VC Resource 0 Status (0x218)..........................................................................................16-44

VCR0TBL0 - VC Resource 0 Port Arbitration Table Entry 0 (0x240) ....................................................16-45

VCR0TBL1 - VC Resource 0 Port Arbitration Table Entry 1 (0x244) ....................................................16-46

VCR0TBL2 - VC Resource 0 Port Arbitration Table Entry 2 (0x248) ....................................................16-46

VCR0TBL3 - VC Resource 0 Port Arbitration Table Entry 3 (0x24C) ...................................................16-47

VID - Vendor Identification Register (0x000)...........................................................................................16-1

PES64H16G2 User Manual xiii April 5, 2013

IDT Register List

Notes

PES64H16G2 User Manual xiv April 5, 2013

Notes

®

Chapter 1

PES64H16G2 Device

Overview

Introduction

The 89HPES64H16G2 is a member of the IDT PRECISE™ family of PCI Express® switching solutions.
The PES64H16G2 is a 64-lane, 16-port system interconnect switch optimized for PCI Express Gen2 packet
switching in high-performance applications, supporting multiple simultaneous peer-to-peer traffic flows.
Target applications include servers, storage, communications, embedded systems, and multi-host or intelli­gent I/O based systems with inter-domain communication.

Utilizing standard PCI Express Gen2 interconnect, the PES64H16G2 provides the most efficient system
interconnect switching solution for applications requiring high throughput, low latency, and simple board
layout with a minimum number of board layers. Each lane is capable of 5 GT/s of bandwidth in both direc­tions and is fully compliant with PCI Express Base specification 2.0.

Features



High Performance Non-Blocking Switch Architecture

– 64-lane 16-port PCIe switch

• Eight x8 ports switch ports each of which can bifurcate to two x4 ports (total of sixteen x4 ports)

– Integrated SerDes supports 5.0 GT/s Gen2 and 2.5 GT/s Gen1 operation
– Delivers up to 64 GBps (512 Gbps) of switching capacity
– Supports 128 Bytes to 2 KB maximum payload size
– Low latency cut-through architecture
– Supports one virtual channel and eight traffic classes



Standards and Compatibility

– PCI Express Base Specification 2.0 compliant
– Implements the following optional PCI Express features

• Advanced Error Reporting (AER) on all ports

• End-to-End CRC (ECRC)

• Access Control Services (ACS)

• Power Budgeting Enhanced Capability

• Device Serial Number Enhanced Capability

• Sub-System ID and Sub-System Vendor ID Capability

• Internal Error Reporting ECN

• Multicast ECN

• VGA and ISA enable

• L0s and L1 ASPM

•ARI ECN

– Compatible with IDT 89HPES64H16 PCIe Gen1 switch



Port Configurability

– x4 and x8 ports

• Ability to merge adjacent x4 ports to create a x8 port

– Automatic per port link width negotiation

(x8  x4  x2  x1)

– Crosslink support
– Automatic lane reversal

– Autonomous and software managed link width and speed control
– Per lane SerDes configuration

PES64H16G2 User Manual 1 - 1 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

• De-emphasis

• Receive equalization

• Drive strength



Switch Partitioning

– IDT proprietary feature that creates logically independent switches in the device
– Supports up to 16 fully independent switch partitions
– Configurable downstream port device numbering
– Supports dynamic reconfiguration of switch partitions

• Dynamic port reconfiguration — downstream, upstream

• Dynamic migration of ports between partitions

• Movable upstream port within and between switch partitions



Initialization / Configuration

– Supports Root (BIOS, OS, or driver), Serial EEPROM, or SMBus switch initialization
– Common switch configurations are supported with pin strapping (no external components)
– Supports in-system Serial EEPROM initialization/programming



Quality of Service (QoS)

– Port arbitration

• Round robin

– Request metering

• IDT proprietary feature that balances bandwidth among switch ports for maximum system
throughput

– High performance switch core architecture

• Combined Input Output Queued (CIOQ) switch architecture with large buffers



Multicast

– Compliant to the PCI-SIG multicast ECN
– Supports arbitrary multicasting of Posted transactions
– Supports 64 multicast groups
– Independent multicast support within each switch partition



Clocking

– Supports 100 MHz and 125 MHz reference clock frequencies
– Flexible clocking modes

• Common clock

• Non-common clock



Hot-Plug and Hot Swap

– Hot-plug controller on all ports

• Hot-plug supported on all downstream switch ports

– All ports support hot-plug using low-cost external I
– Configurable presence detect supports card and cable applications
– GPE output pin for hot-plug event notification

• Enables SCI/SMI generation for legacy operating system support

– Hot-swap capable I/O



Power Management

– Supports D0, D3hot and D3 power management states
– Active State Power Management (ASPM)

• Supports L0, L0s, L1, L2/L3 Ready and L3 link states

• Configurable L0s and L1 entry timers allow performance/power-savings tuning

– Supports PCI Express Power Budgeting Capability
– SerDes power savings

• Supports low swing / half-swing SerDes operation

2

C I/O expanders

PES64H16G2 User Manual 1 - 2 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

• SerDes optionally turned-off in D3hot

• SerDes associated with unused ports are turned-off

• SerDes associated with unused lanes are placed in a low power state



32 General Purpose I/O



Reliability, Availability and Serviceability (RAS)

– ECRC support
– AER on all ports
– SECDED ECC protection on all internal RAMs
– End-to-end data path parity protection
– Checksum Serial EEPROM content protected
– Autonomous link reliability (preserves system operation in the presence of faulty links)

– Ability to generate an interrupt (INTx or MSI) on link up/down transitions



Test and Debug

– On-chip link activity and status outputs available for Port 0 (upstream port)
– Per port link activity and status outputs available using external I

– SerDes test modes
– Supports IEEE 1149.6 AC JTAG and IEEE 1149.1 JTAG



Power Supplies

– Requires only two power supply voltages (1.0 V and 2.5 V)
– No power sequencing requirements



Packaged in a 35mm x 35mm 1156-ball Flip Chip BGA with 1mm ball spacing

– Compatible with IDT 89HPES64H16 PCIe Gen1 switch

Note: For pin compatibility issues, contact the IDT help desk at [e-mail].

2

C I/O expander for all other ports

PES64H16G2 User Manual 1 - 3 April 5, 2013

IDT PES64H16G2 Device Overview

64 PCI Express Lanes

Up to 8 x8 ports or 16 x4 Ports

16-Port Switch Core

Frame Buffer

Route Table

Port

Arbitration

Scheduler

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

DL/Transaction Layer

SerDes

x8/x4/x2/x1

Figure 1.1 PES64H16G2 Block Diagram

P o r t

PCIELCAP

MAXLNKWDTH

P o r t

PCIELCAP

MAXLNKWDTH

P or t 0 0 x 4 P or t 8 0 x 4

P or t 1 0 x 4 P or t 9 0 x 4

P o rt 2 0 x 4 P o rt 1 0 0 x 4

P o rt 3 0 x 4 P o rt 1 1 0 x 4

P o rt 4 0 x 4 P o rt 1 2 0 x 4

P o rt 5 0 x 4 P o rt 1 3 0 x 4

P o rt 6 0 x 4 P o rt 1 4 0 x 4

P o rt 7 0 x 4 P o rt 1 5 0 x 4

Table 1.1 Initial Configuration Register Settings for PES64H16G2

PES64H16G2 User Manual 1 - 4 April 5, 2013

IDT PES64H16G2 Device Overview

PE00TP[3:0]

Global

Reference Clocks

GCLKN[1:0]
GCLKP[1:0]

JTAG_TCK

GPIO[31:0]

32

General Purpose

I/O

VDDCORE

V

DD

I/O

V

DD

PEA

Power/Ground

MSMBADDR[4:1]

MSMBCLK
MSMBDAT

4

SSMBADDR[5,3:1]

SSMBCLK
SSMBDAT

4

Master

SMBus Interface

Slave

SMBus Interface

GCLKFSEL

RSTHALT

System

Pins

JTAG_TDI
JTAG_TDO
JTAG_TMS
JTAG_TRST_N

JTAG Pins

V

SS

SWMODE[3:0]

4

CLKMODE[1:0]

PERSTN

MSMBSMODE

PE00RP[3:0]
PE00RN[3:0]

PCI Express

Switch

SerDes Input

PE00TN3:[0]

PCI Express

Switch

SerDes Output

Port 0

Port 0

PE01RP[3:0]
PE01RN[3:0]

PCI Express

Switch

SerDes Input

PE01TP[3:0]
PE01TN[3:0]

PCI Express

Switch

SerDes Output

Port 1

Port 1

......

PE15RP[3:0]
PE15RN[3:0]

PCI Express

Switch

SerDes Input

PE15TP[3:0]
PE15TN[3:0]

PCI Express

Switch

SerDes Output

Port 15

Port 15

PES64H16G2

REFRES[15:0]

SerDes

Reference

Resistors

VDDPEHA

VDDPETA

......

2

P01MERGEN

P23MERGEN
P45MERGEN
P67MERGEN

P89MERGEN

P1011MERGEN
P1213MERGEN
P1415MERGEN

REFRESPLL

Logic Diagram

PES64H16G2 User Manual 1 - 5 April 5, 2013

Figure 1.2 PES64H16G2 Logic Diagram

IDT PES64H16G2 Device Overview

System Identification

Vendor ID

All vendor IDs in the device are hardwired to 0x111D which corresponds to Integrated Device Technology, Inc.

Device ID

The PES64H16G2 device ID is shown in Table 1.2.

PCIe Device Device ID

0x07 0x8077

Table 1.2 PES64H16G2 Device IDs

Revision ID

The revision ID in the PES64H16G2 is set to the same value in all mode. The value of the revision ID is determined in one place and is easily modi-

fied during a metal mask change. The revision ID will start at 0x0 and will be incremented with each all-layer or metal mask change.

Revision ID Description

0x0 Corresponds to ZA silicon

0x1 Corresponds to ZB silicon

0x2 Corresponds to ZC silicon

Table 1.3 PES64H16G2 Revision ID

JTAG ID

The JTAG ID is:

– Version: Same value as Revision ID. See Table 1.3
– Part number: Same value as base Device ID. See Table 1.2.
– Manufacture ID: 0x33
– LSB: 0x1

SSID/SSVID

The PES64H16G2 contains the mechanisms necessary to implement the PCI-to-PCI bridge Subsystem ID and Subsystem Vendor ID capability
structure. However, in the default configuration the Subsystem ID and Subsystem Vendor ID capability structure is not enabled. To enable this capa­bility, the SSID and SSVID fields in the Subsystem ID and Subsystem Vendor ID (SSIDSSVID) register must be initialized with the appropriate ID
values. the Next Pointer (NXTPTR) field in one of the other enhanced capabilities should be initialized to point to this capability. Finally, the Next
Pointer (NXTPTR) of this capability should be adjusted to point to the next capability if necessary.

Device Serial Number Enhanced Capability

The PES64H16G2 contains the mechanisms necessary to implement the PCI express device serial number enhanced capability. However, in the
default configuration this capability structure is not enabled. To enable the device serial number enhanced capability, the Serial Number Lower
Doubleword (SNUMLDW) and the Serial Number Upper Doubleword (SNUMUDW) registers should be initialized. The Next Pointer (NXTPTR) field in
one of the other enhanced capabilities should be initialized to point to this capability. Finally, the Next Pointer (NXTPTR) of this capability should be
adjusted to point to the next capability if necessary.

PES64H16G2 User Manual 1 - 6 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Pin Description

The following tables list the functions of the pins provided on the PES64H16G2. Some of the functions
listed may be multiplexed onto the same pin. The active polarity of a signal is defined using a suffix. Signals
ending with an “N” are defined as being active, or asserted, when at a logic zero (low) level. All other signals
(including clocks, buses, and select lines) will be interpreted as being active, or asserted, when at a logic
one (high) level. Differential signals end with a suffix “N” or “P.” The differential signal ending in “P” is the
positive portion of the differential pair and the differential signal ending in “N” is the negative portion of the
differential pair.

Signal Type Name/Description

PE00RP[3:0]

PE00RN[3:0]

PE00TP[3:0]
PE00TN[3:0]

PE01RP[3:0]

PE01RN[3:0]

PE01TP[3:0]
PE01TN[3:0]

PE02RP[3:0]

PE02RN[3:0]

PE02TP[3:0]
PE02TN[3:0]

PE03RP[3:0]

PE03RN[3:0]

PE03TP[3:0]
PE03TN[3:0]

PE04RP[3:0]

PE04RN[3:0]

PE04TP[3:0]
PE04TN[3:0]

PE05RP[3:0]

PE05RN[3:0]

PE05TP[3:0]
PE05TN[3:0]

PE06RP[3:0]

PE06RN[3:0]

PE06TP[3:0]
PE06TN[3:0]

PE07RP[3:0]

PE07RN[3:0]

I PCI Express Port 0 Serial Data Receive. Differential PCI Express receive

pairs for port 0.

O PCI Express Port 0 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 0.

I PCI Express Port 1 Serial Data Receive. Differential PCI Express receive

pairs for port 1. When port 0 is merged with port 1, these signals become
port 0 receive pairs for lanes 4 through 7.

O PCI Express Port 1 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 1. When port 0 is merged with port 1, these signals
become port 0 transmit pairs for lanes 4 through 7.

I PCI Express Port 2 Serial Data Receive. Differential PCI Express receive

pairs for port 2.

O PCI Express Port 2 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 2.

I PCI Express Port 3 Serial Data Receive. Differential PCI Express receive

pairs for port 3. When port 2 is merged with port 3, these signals become
port 2 receive pairs for lanes 4 through 7.

O PCI Express Port 3 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 3. When port 2 is merged with port 3, these signals
become port 2 transmit pairs for lanes 4 through 7.

I PCI Express Port 4 Serial Data Receive. Differential PCI Express receive

pairs for port 4.

O PCI Express Port 4 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 4.

I PCI Express Port 5 Serial Data Receive. Differential PCI Express receive

pairs for port 5. When port 4 is merged with port 5, these signals become
port 4 receive pairs for lanes 4 through 7.

O PCI Express Port 5 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 5. When port 4 is merged with port 5, these signals
become port 4 transmit pairs for lanes 4 through 7.

I PCI Express Port 6 Serial Data Receive. Differential PCI Express receive

pairs for port 6.

O PCI Express Port 6 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 6.

I PCI Express Port 7 Serial Data Receive. Differential PCI Express receive

pairs for port 7. When port 6 is merged with port 7, these signals become
port 6 receive pairs for lanes 4 through 7.

Table 1.4 PCI Express Interface Pins (Part 1 of 2)

PES64H16G2 User Manual 1 - 7 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

PE07TP[3:0]
PE07TN[3:0]

PE08RP[3:0]

PE08RN[3:0]

PE08TP[3:0]
PE08TN[3:0]

PE09RP[3:0]

PE09RN[3:0]

PE09TP[3:0]
PE09TN[3:0]

PE10RP[3:0]

PE10RN[3:0]

PE10TP[3:0]
PE10TN[3:0]

PE11RP[3:0]

PE11RN[3:0]

PE11TP[3:0]
PE11TN[3:0]

PE12RP[3:0]

PE12RN[3:0]

PE12TP[3:0]
PE12TN[3:0]

PE13RP[3:0]

PE13RN[3:0]

PE13TP[3:0]
PE13TN[3:0]

PE14RP[3:0]

PE14RN[3:0]

PE14TP[3:0]
PE14TN[3:0]

PE15RP[3:0]

PE15RN[3:0]

PE15TP[3:0]
PE15TN[3:0]

O PCI Express Port 7 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 7. When port 6 is merged with port 7, these signals
become port 6 transmit pairs for lanes 4 through 7.

I PCI Express Port 8 Serial Data Receive. Differential PCI Express receive

pairs for port 8.

O PCI Express Port 8 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 8.

I PCI Express Port 9 Serial Data Receive. Differential PCI Express receive

pairs for port 9. When port 8 is merged with port 9, these signals become
port 8 receive pairs for lanes 4 through 7.

O PCI Express Port 9 Serial Data Transmit. Differential PCI Express trans-

mit pairs for port 9. When port 8 is merged with port 9, these signals
become port 8 transmit pairs for lanes 4 through 7.

I PCI Express Port 10 Serial Data Receive. Differential PCI Express

receive pairs for port 10.

O PCI Express Port 10 Serial Data Transmit. Differential PCI Express

transmit pairs for port 10.

I PCI Express Port 11 Serial Data Receive. Differential PCI Express

receive pairs for port 11. When port 10 is merged with port 11, these sig­nals become port 10 receive pairs for lanes 4 through 7.

O PCI Express Port 11 Serial Data Transmit. Differential PCI Express

transmit pairs for port 11. When port 10 is merged with port 11, these sig­nals become port 10 transmit pairs for lanes 4 through 7.

I PCI Express Port 12 Serial Data Receive. Differential PCI Express

receive pairs for port 12.

O PCI Express Port 12 Serial Data Transmit. Differential PCI Express

transmit pairs for port 12.

I PCI Express Port 13 Serial Data Receive. Differential PCI Express

receive pairs for port 13. When port 12 is merged with port 13, these sig­nals become port 12 receive pairs for lanes 4 through 7.

O PCI Express Port 13 Serial Data Transmit. Differential PCI Express

transmit pairs for port 13. When port 12 is merged with port 13, these sig­nals become port 12 transmit pairs for lanes 4 through 7.

I PCI Express Port 14 Serial Data Receive. Differential PCI Express

receive pairs for port 14.

O PCI Express Port 14 Serial Data Transmit. Differential PCI Express

transmit pairs for port 14.

I PCI Express Port 15 Serial Data Receive. Differential PCI Express

receive pairs for port 15. When port 14 is merged with port 15, these sig­nals become port 14 receive pairs for lanes 4 through 7.

O PCI Express Port 15 Serial Data Transmit. Differential PCI Express

transmit pairs for port 15. When port 14 is merged with port 15, these sig­nals become port 14 transmit pairs for lanes 4 through 7.

Table 1.4 PCI Express Interface Pins (Part 2 of 2)

PES64H16G2 User Manual 1 - 8 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

GCLKN[1:0]
GCLKP[1:0]

Signal Type Name/Description

MSMBADDR[4:1] I Master SMBus Address. These pins determine the SMBus address of the

MSMBCLK I/O Master SMBus Clock. This bidirectional signal is used to synchronize

MSMBDAT I/O Master SMBus Data. This bidirectional signal is used for data on the mas-

SSMBADDR[5,3:1] I Slave SMBus Address. These pins determine the SMBus address to

SSMBCLK I/O Slave SMBus Clock. This bidirectional signal is used to synchronize trans-

SSMBDAT I/O Slave SMBus Data. This bidirectional signal is used for data on the slave

I Global Reference Clock. Differential reference clock input pair. This clock

is used as the reference clock by on-chip PLLs to generate the clocks
required for the system logic. The frequency of the differential reference
clock is determined by the GCLKFSEL signal.

Table 1.5 Reference Clock Pins

serial EEPROM from which configuration information is loaded.

transfers on the master SMBus.

ter SMBus.

which the slave SMBus interface responds.

fers on the slave SMBus.

SMBus.

Table 1.6 SMBus Interface Pins

Signal Type Name/Description

GPIO[0] I/O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: PART0PERSTN
Alternate function pin type: Input/Output
Alternate function: Assertion of this signal initiated a partition fundamental
reset in the corresponding partition.

GPIO[1] I/O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: PART1PERSTN
Alternate function pin type: Input/Output
Alternate function: Assertion of this signal initiated a partition fundamental
reset in the corresponding partition.

GPIO[2] I/O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: PART2PERSTN
Alternate function pin type: Input/Output
Alternate function: Assertion of this signal initiated a partition fundamental
reset in the corresponding partition.

GPIO[3] I/O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: PART3PERSTN
Alternate function pin type: Input/Output
Alternate function: Assertion of this signal initiated a partition fundamental
reset in the corresponding partition.

Table 1.7 General Purpose I/O Pins (Part 1 of 5)

PES64H16G2 User Manual 1 - 9 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

GPIO[4] I General Purpose I/O.

GPIO[5] O General Purpose I/O.

GPIO[6] I General Purpose I/O.

GPIO[7] I/O General Purpose I/O.

GPIO[8] I General Purpose I/O.

GPIO[9] I General Purpose I/O.

GPIO[10] I General Purpose I/O.

GPIO[11] I General Purpose I/O.

GPIO[12] I General Purpose I/O.

GPIO[13] I General Purpose I/O.

GPIO[14] O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
1st Alternate function — Reserved
2nd Alternate function pin name: P0LINKUPN
2nd Alternate function pin type: Output
2nd Alternate function: Port 0 Link Up Status output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: GPEN
1st Alternate function pin type: Output
1st Alternate function: Hot-plug general purpose even output.
2nd Alternate function pin name: P0ACTIVEN
2nd Alternate function pin type: Output
2nd Alternate function: Port 0 Link Active Status Output.

This pin can be configured as a general purpose I/O pin.

This pin can be configured as a general purpose I/O pin.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: IOEXPINTN
Alternate function pin type: Input
Alternate function: IO expander interrupt.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0APN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 0 Attention Push Button Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0PDN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 0 Presence Detect Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0PFN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 0 Power Fault Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0PWRGDN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 0 Power Good Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0MRLN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 0 Manually Operated Retention
latch Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0AIN
Alternate function pin type: Output
Alternate function: Hot Plug Signal Group 0 Attention Indicator Output.

Table 1.7 General Purpose I/O Pins (Part 2 of 5)

PES64H16G2 User Manual 1 - 10 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

GPIO[15] O General Purpose I/O.

GPIO[16] O General Purpose I/O.

GPIO[17] O General Purpose I/O.

GPIO[18] I General Purpose I/O.

GPIO[19] I General Purpose I/O.

GPIO[20] I General Purpose I/O.

GPIO[21] I General Purpose I/O.

GPIO[22] I/O General Purpose I/O.

GPIO[23] O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0PIN
Alternate function pin type: Output
Alternate function: Hot Plug Signal Group 0 Power Indicator Output.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0PEP
Alternate function pin type: Output
Alternate function: Hot Plug Signal Group 0 Power Enable Output.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP0RSTN
Alternate function pin type: Output
Alternate function: Hot Plug Signal Group 0 Reset Output.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP1APN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 1 Attention Push Button Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP1PDN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 1 Presence Detect Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP1PFN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 1 Power Fault Input.

This pin can be configured as a general purpose I/O pin.
Alternate function pin name: HP1PWRGDN
Alternate function pin type: Input
Alternate function: Hot Plug Signal Group 1 Power Good Input.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP1MRLN
1st Alternate function pin type: Input
1st Alternate function: Hot Plug Signal Group 1 Manually Operated Reten­tion.
2nd Alternate function pin name: P1LINKUPN
2nd Alternate function pin type: Output
2nd Alternate function: Port 1 Link Up Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP1AIN
1st Alternate function pin type: Output
1st Alternate function: Hot Plug Signal Group 1 Attention Indicator Output.
2nd Alternate function pin name: P1ACTIVEN
2nd Alternate function pin type: Output
2nd Alternate function: Port 1 Link Active Status Output.

Table 1.7 General Purpose I/O Pins (Part 3 of 5)

PES64H16G2 User Manual 1 - 11 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

GPIO[24] O General Purpose I/O.

GPIO[25] O General Purpose I/O.

GPIO[26] O General Purpose I/O.

GPIO[27] I/O General Purpose I/O.

GPIO[28] I/O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP1PIN
1st Alternate function pin type: Output
1st Alternate function: Hot Plug Signal Group 1 Power Indicator Output.
2nd Alternate function pin name: P2LINKUPN
2nd Alternate function pin type: Output
2nd Alternate function: Port 2 Link Up Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP1PEP
1st Alternate function pin type: Output
1st Alternate function: Hot Plug Signal Group 1 Power Enable Output.
2nd Alternate function pin name: P2ACTIVEN
2nd Alternate function pin type: Output
2nd Alternate function: Port 2 Link Active Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP1RSTN
1st Alternate function pin type: Output
1st Alternate function: Hot Plug Signal Group 1 Reset Output.
2nd Alternate function pin name: P3LINKUPN
2nd Alternate function pin type: Output
2nd Alternate function: Port 3 Link Up Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP2APN
1st Alternate function pin type: Input
1st Alternate function: Hot Plug Signal Group 2 Attention Push Button
Input.
2nd Alternate function pin name: P3ACTIVEN
2nd Alternate function pin type: Output
2nd Alternate function: Port 3 Link Active Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP2PDN
1st Alternate function pin type: Input
1st Alternate function: Hot Plug Signal Group 0 Presence Detect Input.
2nd Alternate function pin name: P4LINKUPN
2nd Alternate function pin type: Output
2nd Alternate function: Port 4 Link Up Status Output.

Table 1.7 General Purpose I/O Pins (Part 4 of 5)

PES64H16G2 User Manual 1 - 12 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

GPIO[29] I/O General Purpose I/O.

GPIO[30] I/O General Purpose I/O.

GPIO[31] I/O General Purpose I/O.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP2PFN
1st Alternate function pin type: Input
1st Alternate function: Hot Plug Signal Group 2 Power Fault Input.
2nd Alternate function pin name: P4ACTIVEN
2nd Alternate function pin type: Output
2nd Alternate function: Port 4 Link Active Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP2PWRGDN
1st Alternate function pin type: Input
1st Alternate function: Hot Plug Signal Group 2 Power Good Input.
2nd Alternate function pin name: P5LINKUPN
2nd Alternate function pin type: Output
2nd Alternate function: Port 5 Link Up Status Output.

This pin can be configured as a general purpose I/O pin.
1st Alternate function pin name: HP2MRLN
1st Alternate function pin type: Input
1st Alternate function: Hot Plug Signal Group 2 Manually Operated Reten­tion Latch Input.
2nd Alternate function pin name: P5ACTIVEN
2nd Alternate function pin type: Output
2nd Alternate function: Port 5 Link Active Status Output.

Table 1.7 General Purpose I/O Pins (Part 5 of 5)

Signal Type Name/Description

CLKMODE[1:0] Clock Mode. These signals determine the port clocking mode used by

ports of the device.

GCLKFSEL I Global Clock Frequency Select. These signals select the frequency of

the GCLKP and GCLKN signals.
0x0 100 MHz
0x1 125 MHz

MSMBSMODE I Master SMBus Slow Mode. The assertion of this pin indicates that the

master SMBus should operate at 100 KHz instead of 400 KHz. This value
may not be overridden.

P01MERGEN I Port 0 and 1 Merge. P01MERGEN is an active low signal. It is pulled low

internally.
When this pin is low, port 0 is merged with port 1 to form a single x8 port.
The Serdes lanes associated with port 1 become lanes 4 through 7 of port

0. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 0 and port 1 are not merged, and each operates
as a single x4 port.

P23MERGEN I Port 2 and 3 Merge. P23MERGEN is an active low signal. It is pulled low

internally.
When this pin is low, port 2 is merged with port 3 to form a single x8 port.
The Serdes lanes associated with port 3 become lanes 4 through 7 of port

2. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 2 and port 3 are not merged, and each operates
as a single x4 port.

Table 1.8 System Pins (Part 1 of 3)

PES64H16G2 User Manual 1 - 13 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

P45MERGEN I Port 4 and 5 Merge. P45MERGEN is an active low signal. It is pulled low

P67MERGEN I Port 6 and 7 Merge. P67MERGEN is an active low signal. It is pulled low

P89MERGEN I Port 8 and 9 Merge. P89MERGEN is an active low signal. It is pulled low

P1011MERGEN I Port 10 and 11 Merge. P1011MERGEN is an active low signal. It is pulled

P1213MERGEN I Port 12 and 13 Merge. P1213MERGEN is an active low signal. It is pulled

P1415MERGEN I Port 14 and 15 Merge. P1415MERGEN is an active low signal. It is pulled

internally.
When this pin is low, port 4 is merged with port 5 to form a single x8 port.
The Serdes lanes associated with port 5 become lanes 4 through 7 of port

4. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 4 and port 5 are not merged, and each operates
as a single x4 port.

internally.
When this pin is low, port 6 is merged with port 7 to form a single x8 port.
The Serdes lanes associated with port 7 become lanes 4 through 7 of port

6. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 6 and port 7 are not merged, and each operates
as a single x4 port.

internally.
When this pin is low, port 8 is merged with port 9 to form a single x8 port.
The Serdes lanes associated with port 9 become lanes 4 through 7 of port

8. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 8 and port 9 are not merged, and each operates
as a single x4 port.

low internally.
When this pin is low, port 10 is merged with port 11 to form a single x8 port.
The Serdes lanes associated with port 11 become lanes 4 through 7 of port

10. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 10 and port 11 are not merged, and each oper­ates as a single x4 port.

low internally.
When this pin is low, port 12 is merged with port13 to form a single x8 port.
The Serdes lanes associated with port 13 become lanes 4 through 7 of port

12. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 12 and port 13 are not merged, and each oper­ates as a single x4 port.

low internally.
When this pin is low, port 14 is merged with port 15 to form a single x8 port.
The Serdes lanes associated with port 15 become lanes 4 through 7 of port

14. Refer to section Port Merging on page 5-7 for details.
When this pin is high, port 14 and port 15 are not merged, and each oper­ates as a single x4 port.

Table 1.8 System Pins (Part 2 of 3)

PES64H16G2 User Manual 1 - 14 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

PERSTN I Global Reset. Assertion of this signal resets all logic inside PES64H16G2.

RSTHALT I Reset Halt. When this signal is asserted during a PCI Express fundamental

SWMODE[3:0] I Switch Mode. These configuration pins determine the PES64H16G2

reset, PES64H16G2 executes the reset procedure and remains in a reset
state with the Master and Slave SMBuses active. This allows software to
read and write registers internal to the device before normal device opera­tion begins. The device exits the reset state when the RSTHALT bit is
cleared in the SWCTL register by an SMBus master.

switch operating mode. Note: These pins should be static and not change
following the negation of PERSTN.
0x0 - Single partition
0x1 - Single partition with Serial EEPROM initialization
0x2 through 0x7 - Reserved
0x8 - Single partition with port 0 selected as the upstream port (port 2 dis-

abled)

0x9 - Single partition with port 2 selected as the upstream port (port 0 dis-

abled)

0xA - Single partition with Serial EEPROM initialization and port 0 selected

as the upstream port (port 2 disabled)

0xB - Single partition with Serial EEPROM initialization and port 2 selected

as the upstream port (port 0 disabled)
0xC - Multi-partition
0xD - Multi-partition with Serial EEPROM initialization
0xE - Reserved
0xF - Reserved

Table 1.8 System Pins (Part 3 of 3)

PES64H16G2 User Manual 1 - 15 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

JTAG_TCK I JTAG Clock. This is an input test clock used to clock the shifting of data

JTAG_TDI I JTAG Data Input. This is the serial data input to the boundary scan logic or

JTAG_TDO O JTAG Data Output. This is the serial data shifted out from the boundary

JTAG_TMS I JTAG Mode. The value on this signal controls the test mode select of the

JTAG_TRST_N I JTAG Reset. This active low signal asynchronously resets the boundary

into or out of the boundary scan logic or JTAG Controller. JTAG_TCK is
independent of the system clock with a nominal 50% duty cycle.

JTAG Controller.

scan logic or JTAG Controller. When no data is being shifted out, this signal
is tri-stated.

boundary scan logic or JTAG Controller.

scan logic and JTAG TAP Controller. An external pull-up on the board is
recommended to meet the JTAG specification in cases where the tester
can access this signal. However, for systems running in functional mode,
one of the following should occur:

1) actively drive this signal low with control logic

2) statically drive this signal low with an external pull-down on the board

Table 1.9 Test Pins

Signal Type Name/Description

REFRES00 I/O Port 0 External Reference Resistor. Provides a reference for the Port 0

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES01 I/O Port 1 External Reference Resistor. Provides a reference for the Port 1

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES02 I/O Port 2 External Reference Resistor. Provides a reference for the Port 2

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES03 I/O Port 3 External Reference Resistor. Provides a reference for the Port 3

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES04 I/O Port 4 External Reference Resistor. Provides a reference for the Port 4

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES05 I/O Port 5 External Reference Resistor. Provides a reference for the Port 5

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES06 I/O Port 6 External Reference Resistor. Provides a reference for the Port 6

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES07 I/O Port 7 External Reference Resistor. Provides a reference for the Port 7

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

Table 1.10 Power, Ground, and SerDes Resistor Pins (Part 1 of 2)

PES64H16G2 User Manual 1 - 16 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Signal Type Name/Description

REFRES08 I/O Port 8 External Reference Resistor. Provides a reference for the Port 8

REFRES09 I/O Port 9 External Reference Resistor. Provides a reference for the Port 9

REFRES10 I/O Port 10 External Reference Resistor. Provides a reference for the Port 10

REFRES11 I/O Port 11 External Reference Resistor. Provides a reference for the Port 11

REFRES12 I/O Port 12 External Reference Resistor. Provides a reference for the Port 12

REFRES13 I/O Port 13 External Reference Resistor. Provides a reference for the Port 13

REFRES14 I/O Port 14 External Reference Resistor. Provides a reference for the Port 14

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRES15 I/O Port 15 External Reference Resistor. Provides a reference for the Port 15

SerDes bias currents and PLL calibration circuitry. A 3 kOhm +/- 1% resis­tor should be connected from this pin to ground.

REFRESPLL I/O PLL External Reference Resistor. Provides a reference for the PLL bias

currents and PLL calibration circuitry. A 3K Ohm +/- 1% resistor should be
connected from this pin to ground.

V

CORE I Core V

DD

I/O I I/O V

V

DD

PEA I PCI Express Analog Power. Serdes analog power supply (1.0V).

V

DD

V

PEHA I PCI Express Analog High Power. Serdes analog power supply (2.5V).

DD

PETA I PCI Express Transmitter Analog Voltage. Serdes transmitter analog

V

DD

Power supply for core logic (1.0V).

DD.

LVTTL I/O buffer power supply (2.5V or preferred 3.3V).

DD.

power supply (1.0V).

V

SS

I Ground.

Table 1.10 Power, Ground, and SerDes Resistor Pins (Part 2 of 2)

PES64H16G2 User Manual 1 - 17 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Pin Characteristics

Note: Some input pads of the switch do not contain internal pull-ups or pull-downs. Unused SMBus
and System inputs should be tied off to appropriate levels. This is especially critical for unused control
signal inputs which, if left floating, could adversely affect operation. Also, floating pins can cause a
slight increase in power consumption. Unused Serdes (Rx and Tx) pins should be left floating. Finally,
No Connection pins should not be connected.

Function Pin Name Type Buffer

PCI Express
Interface

PE00RN[3:0] I PCIe

PE00RP[3:0] I

PE00TN[3:0] O

PE00TP[3:0] O

PE01RN[3:0] I

PE01RP[3:0] I

PE01TN[3:0] O

PE01TP[3:0] O

PE02RN[3:0] I

PE02RP[3:0] I

PE02TN[3:0] O

PE02TP[3:0] O

PE03RN[3:0] I

PE03RP[3:0] I

PE03TN[3:0] O

PE03TP[3:0] O

PE04RN[3:0] I

PE04RP[3:0] I

PE04TN[3:0] O

PE04TP[3:0] O

PE05RN[3:0] I

PE05RP[3:0] I

PE05TN[3:0] O

PE05TP[3:0] O

PE06RN[3:0] I

PE06RP[3:0] I

PE06TN[3:0] O

PE06TP[3:0] O

PE07RN[3:0] I

PE07RP[3:0] I

PE07TN[3:0] O

PE07TP[3:0] O

PE08RN[3:0] I

PE08RP[3:0] I

PE08TN[3:0] O

differential

2

I/O

Type

Serial Link

Internal

Resistor

Notes

1

Table 1.11 Pin Characteristics (Part 1 of 3)

PES64H16G2 User Manual 1 - 18 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Function Pin Name Type Buffer

PCI Express
Interface (Cont.)

SMBus MSMBADDR[4:1] I LVTTL Input pull-down

PE08TP[3:0] O PCIe

PE09RN[3:0] I

PE09RP[3:0] I

PE09TN[3:0] O

PE09TP[3:0] O

PE10RN[3:0] I

PE10RP[3:0] I

PE10TN[3:0] O

PE10TP[3:0] O

PE11RN[3:0] I

PE11RP[3:0] I

PE11TN[3:0] O

PE11TP[3:0] O

PE12RN[3:0] I

PE12RP[3:0] I

PE12TN[3:0] O

PE12TP[3:0] O

PE13RN[3:0] I

PE13RP[3:0] I

PE13TN[3:0] O

PE13TP[3:0] O

PE14RN[3:0] I

PE14RP[3:0] I

PE14TN[3:0] O

PE14TP[3:0] O

PE15RN[3:0] I

PE15RP[3:0] I

PE15TN[3:0] O

PE15TP[3:0] O

GCLKN[1:0] I HCSL Diff. Clock

GCLKP[1:0] I

MSMBCLK I/O STI

MSMBDAT I/O STI pull-up on

SSMBADDR[5,3:1] I Input pull-up

SSMBCLK I/O STI pull-up on

SSMBDAT I/O STI pull-up on

differential

I/O

Type

Serial Link

Input

Internal

Resistor

3

1

Refer to Table

PES64H16G2

Data Sheet

Notes

10 in the

pull-up on

board

board

board

board

Table 1.11 Pin Characteristics (Part 2 of 3)

PES64H16G2 User Manual 1 - 19 April 5, 2013

IDT PES64H16G2 Device Overview

Notes

Function Pin Name Type Buffer

General Pur-

GPIO[31:0] I/O LVTTL STI,

pose I/O

I/O

Type

High Drive

Internal

Resistor

pull-up

System Pins CLKMODE[1:0] I LVTTL Input pull-up

GCLKFSEL I pull-down

MSMBSMODE I pull-down

P01MERGEN I pull-down

P23MERGEN I pull-down

P45MERGEN I pull-down

P67MERGEN I pull-down

P89MERGEN I pull-down

P1011MERGEN I pull-down

P1213MERGEN I pull-down

P1415MERGEN I pull-down

PERSTN I STI

RSTHALT I Input pull-down

SWMODE[3:0] I pull-down

EJTAG / JTAG JTAG_TCK I LVTTL STI pull-up

JTAG_TDI I STI pull-up

JTAG_TDO O

JTAG_TMS I STI pull-up

JTAG_TRST_N I STI pull-up

SerDes Refer­ence Resistors

REFRES00 I/O Analog

REFRES01 I/O

REFRES02 I/O

REFRES03 I/O

REFRES04 I/O

REFRES05 I/O

REFRES06 I/O

REFRES07 I/O

REFRES08 I/O

REFRES09 I/O

REFRES10 I/O

REFRES11 I/O

REFRES12 I/O

REFRES13 I/O

REFRES14 I/O

REFRES15 I/O

REFRESPLL I/O

Notes

1

1.

Internal resistor values under typical operating conditions are 92K  for pull-up and 91K for pull-down.

2.

All receiver pins set the DC common mode voltage to ground. All transmitters must be AC coupled to the media.

3.

Schmitt Trigger Input (STI).

PES64H16G2 User Manual 1 - 20 April 5, 2013

Table 1.11 Pin Characteristics (Part 3 of 3)

Notes

®

Chapter 2

Port

SerDes

Switch Core

GPIO

Controller

Ma ster

SM Bus

Interface

Slave

SM Bus

Interface

Reset

Controller

GPIO

Master SMBus

Slave SM Bus

Reset and Bo ot
Conf ig uration Ve ctor

Port

SerDes

Port

SerD es

Port Port Port

SerDes SerD es SerDes

PCI Ex pr es s P ort s

PCI E xpr ess Port s

….

….

Port

SerDes

Switch Core

GPIO

Controller

Ma ster

SM Bus

Interface

Slave

SM Bus

Interface

Reset

Controller

GPIO

Master SMBus

Slave SM Bus

Reset and Bo ot
Conf ig uration Ve ctor

Port

SerDes

Port

SerD es

Port Port Port

SerDes SerD es SerDes

PCI Ex pr es s P ort s

PCI E xpr ess Port s

….

….

Architectural Overview

Introduction

This section provides a high level architectural overview of the PES64H16G2. An architectural block

diagram of the PES64H16G2 is shown in Figure 2.1.

PES64H16G2 User Manual 2 - 1 April 5, 2013

Figure 2.1 PES64H16G2 Block Diagram

The PES64H16G2 contains sixteen x4 ports labeled port 0 through port 15. An even port n and its odd

counterpart, port n+1, may be merged to create a single x8 port.

– When ports are merged, the odd numbered port is not logically visible in the PCI Express hier-

archy associated with the port.

All ports support 2.5 GT/s (i.e., Gen1) and 5.0 GT/s (i.e., Gen2) operation.

At a high level, the PES64H16G2 consists of ports and a switch core. A port consists of a logic that
performs functions associated with the physical, data link, and transactions layers described in the PCIe
base 2.0 specification. In addition, a port performs switch application layer functions such as TLP routing
using route map tables, processing configuration read and write requests, etc.

The switch core is responsible for transferring TLPs between ports. Its main functions are: input buff­ering, maintaining per port ingress and egress flow control information, port and VC arbitration, scheduling,
and forwarding TLPs between ports.

Since the PES64H16G2 represents a single architecture optimized for both fan-out and system inter­connect applications, its switch core is based on a non-blocking crossbar.

IDT Architectural Overview

Notes

Virtual PCI Bus

P2P

Bridge

Upstream

Port

P2P

Bridge

P2P

Bridge

P2P

Bridge

P2P

Bridge

Downstream Ports

P2P

Bridge

Partition 1 – Virtual PCI Bus

P2P

Bridge

Partition 1

Upstream Port

P2P

Bridge

P2P

Bridge

P2P

Bridge

P2P

Bridge

Partition 2 – Virtual PCI Bus

P2P

Bridge

P2P

Bridge

P2P

Bridge

Partition 3 – Virtual PCI Bus

P2P

Bridge

P2P

Bridge

P2P

Bridge

Partition 1

Downstream Ports

Partition 2

Downstrea m Ports

Partition 3

Downstream Ports

Partition 2

Upstream Port

Partition 3

Upstream Port

Switch Partitioning

The logical view of a PCIe switch is shown in Figure 2.2. A PCI switch contains one upstream port and
one or more downstream ports. Each port is associated with a PCI-to-PCI bridge. All
associated with a PCIe switch are interconnected by a virtual PCI bus.

– The primary side of the upstream port’s

PCI-to-PCI bridge is associated with the external link, while

the secondary side connects to the virtual PCI bus.

– The primary side of a downstream port’s

PCI-to-PCI bridge is connected to the virtual PCI bus,

while the secondary side is associated with the external link.

PCI-to-PCI bridges

Figure 2.2 Transparent PCIe Switch

The PES64H16G2 is a partitionable PCIe switch. This means that in addition to operating as a standard
PCI express switch, the PES64H16G2 ports may be partitioned into groups that logically operate as
completely independent PCIe switches.

Figure 2.3 illustrates a three partition PES64H16G2 configuration.

Figure 2.3 Partitionable PCI Express Switch

PES64H16G2 User Manual 2 - 2 April 5, 2013

IDT Architectural Overview

Notes

Each partition operates logically as a completely independent PCIe switch that implements the behavior
and capabilities outlined in the PCI Express Base specification required of a switch.

The PES64H16G2 supports boot-time (i.e., Fundamental Reset) and runtime configuration of ports into
partitions. Boot time configuration may be performed via serial EEPROM, external SMBus master, or soft­ware executing on a root port (e.g., BIOS, OS, drive or hypervisor).

Runtime reconfiguration allows the number of active partitions in the device and assignment of ports to
partitions to be modified while the system is active. Runtime reconfiguration does not affect partitions
whose configuration is not modified.

Partitioning is described in detail in Chapter 6, Switch Partitions.

Dynamic Reconfiguration

The PES64H16G2 supports two forms of dynamic reconfiguration. The first is reconfiguration of the
ports associated with a switch partition. The second is reconfiguration of the operating mode of a port. Parti­tion and port reconfiguration may be initiated by software executing on a root or SMBus master.

Dynamic reconfiguration of the operating mode of a port or partition configuration is described in section
Port Operating Mode Change on page 6-7.

PES64H16G2 User Manual 2 - 3 April 5, 2013

IDT Architectural Overview

Notes

PES64H16G2 User Manual 2 - 4 April 5, 2013

Notes

®

Chapter 3

Switch Core

Introduction

This chapter provides an overview of the PES64H16G2’s Switch Core. As shown in Figure 2.1 in the
Architectural Overview chapter, the Switch Core interconnects switch ports. The Switch Core’s main func­tion is to transfer TLPs among these ports efficiently and reliably. In order to do so, the Switch Core
provides buffering, ordering, arbitration, and error detection services.

Switch Core Architecture

The Switch Core is based on a non-blocking crossbar design optimized for system interconnect (i.e.,
peer-to-peer) as well as fanout (i.e., root-to-endpoint) applications. At a high level, the Switch Core is
composed of ingress buffers, a crossbar fabric interconnect, and egress buffers. These blocks are comple­mented with ordering, arbitration, and error handling logic (not shown in the figure).

Each port has dedicated ingress and egress buffer. The ingress buffer stores data received or generated
by the port. The egress buffer stores data that will be sent to the port. The crossbar interconnect is a matrix
of pathways, capable of concurrently transferring data among all possible port pairs (e.g., port 0 can
transfer data to port 1 at the same time port 2 transfers data to port 3).

As packets are received from the link they are stored in the corresponding ingress buffer. After under­going ordering and arbitration, they are transferred to the corresponding egress buffer via the crossbar
interconnect. The presence of egress buffers provides head-of-line-blocking (HOLB) relief when an egress
port is congested. For example, a packet received on port 0 that is destined to port 1 may be transferred
from port 0’s ingress buffer to port 1’s egress buffer even if port 1 does not have sufficient egress link
credits. This transfer allows subsequent packets received on port 0 to be transmitted to their destination.

Ingress Buffer

When a packet is received from the link, the ingress port’s Application Layer determines the packet’s
route and subjects it to TC/VC mapping. The packet is then stored in the appropriate IFB, together with its
routing and handling information (i.e., the packet’s descriptor). The IFB consists of three queues. These
queues are the posted transaction queue (PT queue), the non-posted transaction queue (NP queue), and
the completion transaction queue (CP queue).

– The queues for the IFB are implemented using a descriptor memory and a data memory.
– When two x4 ports are merged to create a x8 port, the descriptor and data memories for both x4

ports are merged.

The default size of each of these queues is shown in Table 3.1.

Port

Mode

x4

Bifurcated

IFB

Queue

Posted 6176 Bytes and up to 64

Non Posted 1024 Bytes and up to 64

Completion 6176 Bytes and up to 64

Total Size and

Limitations

(per-port)

TLPs

TLPs

TLPs

Table 3.1 IFB Buffer Sizes (Part 1 of 2)

Advertised

Data

Credits

386 64

64 64

386 64

Advertised

Header
Credits

PES64H16G2 User Manual 3 - 1 April 5, 2013

IDT Switch Core

Notes

Port

Mode

x8

Merged

IFB

Queue

Posted 12352 Bytes and up to 127

Non Posted 2048 Bytes and up to 127

Completion 12352 Bytes and up to 127

Total Size and

Limitations

(per-port)

TLPs

TLPs

TLPs

Table 3.1 IFB Buffer Sizes (Part 2 of 2)

Advertised

Data

Credits

772 127

128 127

772 127

Advertised

Header
Credits

Egress Buffer

The EFBs provide head-of-line-blocking (HOLB) relief to the IFBs by allowing packets to be stored in an
egress port’s EFB even if the port’s link does not have sufficient credits to accept the packet. HOLB relief
allows subsequent packets in the IFB to be transferred to their destinations efficiently. As packets are trans­ferred from an IFB to an EFB, they are subjected to the egress port’s TC/VC mapping and stored in the
EFB.

Each EFB consists of three queues. These are the posted queue, non-posted queue, and completion
queue. The use of these queues allows for packet re-ordering to improve transmission efficiency on the
egress link. Refer to section Packet Ordering on page 3-4 for details.

– The queues for both EFBs are implemented using a descriptor memory and a data memory.
– When two x4 ports are merged to create a x8 port, the descriptor and data memories for both x4

ports are merged.

The default size of each of these queues is shown in Table 3.2.

Port

Mode

x4

Bifurcated

x8

Merged

EFB

Queue

Posted 6176 Bytes and up to 64 TLPs

Non Posted 1024 Bytes and up to 64 TLPs

Completion 6176 Bytes and up to 64 TLPs

Posted 12352 Bytes and up to 128 TLPs

Non Posted 2048 Bytes and up to 128 TLPs

Completion 12352 Bytes and up to 128 TLPs

Table 3.2 EFB Buffer Sizes

Total Size and Limitations

(per-port)

In addition to providing HOLB relief, the EFB is used as a dynamically sized replay buffer. This allows for
efficient use of the egress buffer space: when transmitted packets are not being acknowledged by the link
partner the replay buffer grows to allow further transmission; when transmitted packets are successfully
acknowledged by the link partner the replay buffer shrinks and this space is used as egress buffer space to
provide maximum HOLB relief to the IFBs. Assuming a link partner issues acknowledges at the rates
recommended in the PCI Express 2.0 spec, the replay buffer naturally grows to the optimal size for the
port’s link width and speed. Table 3.3 shows the maximum number of TLPs that may be stored in the EFB’s
replay buffer.

PES64H16G2 User Manual 3 - 2 April 5, 2013

IDT Switch Core

Notes

Switch Core

PT Queue

NP Queue

CP Queue

IFB

Port 0 Ingress Buffers

O
r
d
e
r
i
n
g

PT Queue

NP Queue

CP Queue

IFB

Port 15 Ingress Buffers

O
r
d
e
r
i
n
g

PT Queue

NP Queue

CP Queue

EFB

Port 0 Egress Buffers

O
r
d
e
r
i
n
g

PT Queue

NP Queue

CP Queue

EFB

Port 15 Egress Buffers

O
r
d
e
r
i
n
g

16 x 16

Crossbar

Port

Mode

x4

Replay Buffer Storage

Limit

32 TLPs

Bifurcated

x8

64 TLPs

Merged

Table 3.3 Replay Buffer Storage Limit

Crossbar Interconnect

The crossbar is a 16x16 matrix of pathways, capable of concurrently transferring data between a
maximum of 16 port pairs. The crossbar interconnects the port ingress buffers to the egress buffers. It
provides two data-interfaces per port, one for the port’s ingress buffers and one for the port’s egress
buffers.

Figure 3.1 shows the interface between the crossbar and a port’s ingress and egress buffers. The
crossbar is able to support 16 simultaneous data transfers. This architecture is well suited for system inter­connect applications, as it allows simultaneous full-duplex communication between up to 16 peer devices.

Datapaths

As mentioned earlier, the Switch Core interfaces with 16 switch ports. The interface between each port

Figure 3.1 Crossbar Connection to Port Ingress and Egress Buffers

and the switch core can be logically divided into ingress data interface, egress data interface.

The ingress data interface transfers data received by the port from the PCIe link into the switch-core.
The egress data interface transfers data from the switch-core to the port. All data paths through the ingress
data interface, crossbar interconnect and egress data interface are 160-bits wide instead of the required

PES64H16G2 User Manual 3 - 3 April 5, 2013

128-bits (i.e., a x8 Gen2 port requires a throughput of 128-bits per clock cycle). On the ingress data inter­face, the Switch Core receives data from the port at a rate determined by the operational mode of the port
(merged or bifurcated) and the width and speed of the port’s link. Packets received from the port are stored

IDT Switch Core

Notes

in the appropriate IFB queue. After being queued in an IFB1 and undergoing ordering and arbitration, all
data transferred through the crossbar interconnect is transferred in a continuous TLP manner (i.e., the data
path is never multiplexed).

This choice of datapath width implies that the crossbar has 20% higher throughput than the throughput
required to service all ports. This “over-speed” ensures that inter-port messages (i.e., internal messages
exchanged by ports for switch management) do not affect the throughput of the PCIe links.

On the egress interface, data in the EFB is read by the port’s data link layer (i.e., DL) when it is chosen
to be transmitted on the link. If the port is in merged mode, the DL allocates all clock cycles to read data
from the EFB. However, depending on the negotiated link width not all clock cycles may be used to transfer
data. If the port is in bifurcated mode, the DL reads data from the appropriate EFB (i.e., each port has a
dedicated EFB). Again, depending on the negotiated link width, not all clock cycles may be used to transfer
data.

Packet Ordering

The PCI Express specification 2.0 contains packet ordering rules to ensure the producer/consumer
model is honored across a PCIe hierarchy and to prevent deadlocks. The Switch Core performs packet
ordering on a per-port basis, at the output of the ingress and egress buffers of each port (refer to Figure

3.1).

Applying ordering rules at the output of the ingress buffer (i.e., before the crossbar) is done to ensure
that packets are ordered regardless of their destination port. This guarantees that the producer/consumer
model is met when the data transfer involves any number of peers.

Applying ordering rules at the output of the egress buffer is done to allow packets in the EFB to be re­ordered for deadlock prevention and efficient transmission on the link without violating the PCIe ordering
rules. Without this ordering logic, packets in the EFB would need to be transmitted in the order they were
received by the EFB. If the oldest packet in the EFB lacked sufficient link credits for its departure, head-of­line blocking would occur at the EFB. The presence of ordering logic at the EFB reliefs potential head-of­line blocking by allowing other packets to be transmitted, as long as ordering rules are not violated.

Table 3.4 shows the ordering rules honored by the Switch Core. Note that the PES64H16G2 honors the
relaxed-ordering attribute in packets as shown in the table.

1.

Please refer to section Cut-Through Routing on page 3-6 for further information on conditions for cut-through

transfers to occur.

PES64H16G2 User Manual 3 - 4 April 5, 2013

IDT Switch Core

Notes

Row Pass Column?

Posted

Request

Non Posted

Request

Completion

Request

Memory

Write or

Message

Request

Read

Request

IO or Config­uration Write

Request

Read Com-

pletion

IO or Config­uration Write

Completion

Posted

Request

Memory

Write or

Message

Request

No YesYesYesYes

No No No Yes Yes

No No No Yes Yes

‘Yes’ if packet

has RO bit

set; Else ‘No’

Non-Posted Request Completion

IO or

Read

Request

Yes Yes No No

Yes Yes No No

Configur

ation

Write

Request

Read

Comple-

tion

IO or

Configur

ation

Write

Comple-

tion

Table 3.4 Packet Ordering Rules in the PES64H16G2

Arbitration

Packets stored in the ingress buffers are subject to arbitration as they are moved towards the egress
port. The switch core performs all packet arbitration functions in the switch. Architecturally, arbitration is
done at the egress ports. Each port has a dedicated arbitration configuration as programmed in the port’s
VC Capability Structure.

Packets undergo two levels of arbitration at an egress port:

– Port arbitration within a VC
– VC arbitration for access to the egress link

Figure 3.2 shows the architectural model of arbitration. The following sub-sections describe arbitration in
detail.

PES64H16G2 User Manual 3 - 5 April 5, 2013

IDT Switch Core

Notes

Switch Core

Port 0 IFB

VC 0

Port 1 IFB

VC 0

Port 15 IFB

VC 0

EFB

VC 0

VC 0

Port

Arbitration

VC

Arbitration

(not

applicable)

Port 0 Egress Arbitration

TC/

VC

Map

EFB

VC 0

VC 0

Port

Arbitration

VC

Arbitration

(not

applicable)

Port 15 Egress Arbitrati on

TC/

VC

Map

Figure 3.2 Architectural Model of Arbitration

Port Arbitration

Each egress port does port arbitration among multiple ingress ports for packets. Each egress port
contains a port arbiter. Ingress port(s) that wish to transfer one or more packets to the egress port partici­pate in arbitration. Prior to participating in port arbitration, each ingress port does packet ordering. Based on
this, each ingress port selects zero, one, or multiple packets as candidates for transfer towards the EFBs.
Port arbitration is done according to the configuration of the egress port’s VC Capability Structure. The
PES64H16G2 ports operating in upstream switch port or downstream switch port mode support port arbitra­tion using Hardware Fixed Round-Robin (default). The arbitration algorithm is programmed independently
for each of the ports via the VC Capability Structure located in the port’s configuration space.

Cut-Through Routing

The PES64H16G2 utilizes a combined input and output buffered cut-through switching architecture to
forward PCIe TLPs between switch ports. Cut-through means that while a TLP is being received on an
ingress link, it can be simultaneously routed across the switch and transferred on the egress link. The entire
TLP need not be received and buffered prior to starting the routing process (i.e., store-and-forward). This
reduces the latency experienced by packets as they are transferred across the switch.

Typically, cut-through occurs when a TLP is received on an ingress link whose bandwidth is greater than
or equal to the bandwidth of the egress link. For example, a TLP received on a x4 Gen2 port and destined
to a x1 Gen2 port is cut-through the switch. This rule ensures that the ingress link has enough bandwidth to
prevent ‘underflow’ of the egress link. In addition to this, the PES64H16G2 does “adaptive cut-through”,
meaning that packets are cut-through even if the egress link bandwidth is greater than the ingress link
bandwidth. In this case, the cut-through transfer starts when the ingress port has received enough quantity
of the packet such that the packet can be sent to the egress link without underflowing this link. The ingress

PES64H16G2 User Manual 3 - 6 April 5, 2013

IDT Switch Core

Notes

and egress link bandwidth is determined by the negotiated speed and width of the links. Table 3.5 shows
the conditions under which cut-through and adaptive-cut-through occur. When the conditions are met, cut-

1

through is performed across the IFB, crossbar

, and EFB.

Note that a packet undergoing a cut-through transfer across the Switch Core may be temporarily
delayed by the presence of prior packets in the IFB and/or EFB. In this case, the packet starts cutting­through as soon as it becomes unblocked by prior packets.

When cut-through routing of a packet is not possible, the packet is fully buffered in the appropriate IFB
prior to being transferred to the EFB and towards the egress link (i.e., store-and-forward operation). Once
the packet is stored in the IFB, there is no necessity to fully store it in the EFB as it is transferred towards
the egress link.

Ingress

Link

Speed

2.5 Gbps x8 2.5 x8, x4, x2, x1 Always

Ingress

Link

Width

x4 2.5 x4, x2, x1 Always

x2 2.5 x2, x1 Always

Egress

Link

Speed

5.0 x4, x2, x1 Always

5.0 x2, x1 Always

5.0 x1 Always

Egress

Link

Width

x8 At least 50% of packet is in IFB

x8 At least 50% of packet is in IFB

x4 At least 50% of packet is in IFB

x8 At least 75% of packet is in IFB

x4 At least 50% of packet is in IFB

x8 At least 75% of packet is in IFB

x2 At least 50% of packet is in IFB

x4 At least 75% of packet is in IFB

x8 At least 100% of packet is in IFB

Conditions for Cut-

Through

x1 2.5 x1 Always

x2 At least 50% of packet is in IFB

x4 At least 75% of packet is in IFB

x8 Never (100% of packet is in IFB)

5.0 x1 At least 50% of packet is in IFB

x2 At least 75% of packet is in IFB

x4 Never (100% of packet is in IFB)

x8

Table 3.5 Conditions for Cut-Through Transfers (Part 1 of 2)

1.

During cut-through transfers, the crossbar maintains the connection between the appropriate IFB and EFB

through-out the duration of the transfer.

PES64H16G2 User Manual 3 - 7 April 5, 2013

IDT Switch Core

Notes

Ingress

Link

Speed

5.0 Gbps x8 2.5 x8, x4, x2, x1 Always

Ingress

Link

Width

x4 2.5 x8, x4, x2, x1 Always

x2 2.5 x8 At least 50% of packet is in IFB

x1 2.5 x8 At least 75% of packet is in IFB

Egress

Link

Speed

5.0 x8, x4, x2, x1 Always

5.0 x8 At least 50% of packet is in IFB

5.0 x8 At least 75% of packet is in IFB

5.0 x8 Never (100% of packet is in IFB)

Egress

Link

Width

x4, x2, x1 Always

x4, x2, x1 Always

x4 At least 50% of packet is in IFB

x2, x1 Always

x4 At least 50% of packet is in IFB

x2, x1 Always

x4 At least 75% of packet is in IFB

x2 At least 50% of packet is in IFB

Conditions for Cut-

Through

x1 Always

Table 3.5 Conditions for Cut-Through Transfers (Part 2 of 2)

Request Metering

Request metering may be used to reduce congestion in PCI express switches caused by a static rate
mismatch. Request metering is available on all PES64H16G2 switch ports but is disabled by default. A
static rate mismatch is a mismatch in the capacity of the path from a component injecting traffic into the
fabric (e.g., a Root Complex) and the ultimate destination (e.g., an Endpoint).

An example of a static rate mismatch in a PCIe fabric is a x8 root injecting traffic destined to a x1
endpoint. PCIe fabrics are typically no more than one switch deep. Therefore, static rate mismatches typi­cally occur within a switch due to asymmetric link rates. Figure 3.3 illustrates the effect of congestion on
PCIe fabric caused by a static rate mismatch. In this example there are two endpoints issuing memory read
requests to a root. Endpoint A has a x1 link to the switch, while endpoint B and the root complex have a x8
link.

Memory read request TLPs are three or four DWords in size. A single memory read request may result
in up to 4 KB of completion data being returned to the requester. Depending on system architecture and
configured maximum payload size, this completion data may be returned as a single completion TLP or
may be returned as a series of small (e.g., 64B data) TLPs.

Consider an example where Endpoints A and B are injecting read request to the root at a high rate and
the root is able to inject completion data into the fabric at a rate higher than which may be supported by
endpoint A’s egress link. The result is that the endpoint A’s EFB and the root’s IFB may become filled with
queued completion data blocking completion data to endpoint B.

PES64H16G2 User Manual 3 - 8 April 5, 2013

IDT Switch Core

Notes

Root Port

IFB

Switch Core

Endpoint A

EFB

Endpoint B

EFB

Root

(x8)

Endpoint A

(x1)

Endpoint B

(x8)

Request

1

Request

2

Request

3

Time

Request

1

Request

2

Request

3

Time

Estimate of Request 1

Completion Transfer Time

Estimate of Request 2

Completion Transfer Time

(a) Request Injection without Request Metering

(b) Request Injection with Request Metering

If read requests are injected sporadically or at a low rate, then buffering within the switch may be used to
accommodate short lived contention and allow completions to endpoints to proceed without interfering. If
read requests are injected at a high rate, then no amount of buffering in the switch will prevent completions
from interfering.

PCIe has no end-to-end QoS mechanisms. Therefore, it is common for Endpoints to be designed to
inject requests into a fabric at high rates. Request metering is a congestion avoidance mechanism that
limits the request injection rate into a fabric. Although this example illustrates the effect of a static rate
mismatch in an I/O connectivity application, similar situations may occur in system interconnect applica­tions.

Figure 3.3 PCIe Switch Static Rate Mismatch

The request metering operation is illustrated Figure 3.4. Figure 3.4(a) shows requests injection without
request metering. Figure 3.4(b) shows requests injection with request metering. Request metering is imple­mented by logic at the interface between the IFB and the switch core arbiter. When a request reaches the
head of the non-posted IFB queue, request metering logic examines the request and estimates the amount
of time that the associated completion TLPs will consume on the endpoint link (i.e., completion transfer
time). The request is then allowed to proceed and a timer is initiated with the estimated completion transfer
time. The next request from that IFB is not allowed to proceed until the timer has expired.

PES64H16G2 User Manual 3 - 9 April 5, 2013

Figure 3.4 PCIe Switch Static Rate Mismatch

IDT Switch Core

Notes

0 DW ords Fractional Part

Sign 13 bits 11 bits

Request Metering C ount
(24 bits)

0 DW ords Nibbles

Init ial V alu e L oa d e d into
Request Metering C ounter

0000 0000

Sign 13 bits 8 bits

3 bits

The request metering implementation in the PES64H16G2 makes a number of simplifying assumptions
that may or may not be true in all systems. Therefore, it should be expected that some amount of parameter
tuning may be required to achieve optimum performance.

Note that tuning of the request metering mechanism should take into account the completion timeout
value of the associated requesters (i.e., request metering should be tuned such that a requester’s comple­tion timeout value is not violated).

Operation

The completion transfer timer is implemented using a counter. The counter is loaded with an estimate of
the number of DWords that will be transferred on the link in servicing the completion and is decremented at
a rate that corresponds to the number of DWords that will be transferred on the link in a 4ns period.

Request metering is enabled on an input port when the Enable (EN) bit is set in the Requester Metering
Control (RMCTL) register. An non-posted request TLP is allowed to be transferred into the switch core
when the request metering counter is zero.

When a request is transferred into the switch core, the request metering counter is loaded with a value
that estimates the number of DWords associated with the corresponding completion(s). The method for
determining this value is described in section Completion Size Estimation on page 3-12.

– The request metering counter is a 24-bit counter. The count represents a signed-magnitude fixed-

point 0:13:11 number (i.e., a positive number with 13 integer bits and 11 fractional bits) but is
treated by the logic as a 24-bit unsigned integer.

– The value loaded into the request metering counter for the last non-posted request is available in

the Count (COUNT) field of the Request Metering Counter (RMCOUNT) register.

The requester metering initial counter value computed as described in section Comple­tion Size Estimation on page 3-12 is a sign-magnitude fixed point 0:13:3 number (i.e., a
positive number with 13 integer bits and 3 fractional bits).

The least significant eight fractional bits of the initial counter value are always implicitly
zero.

Figure 3.5 shows the request metering count and its initial value.

PES64H16G2 User Manual 3 - 10 April 5, 2013

Figure 3.5 Request Metering Count and Initial Value Loaded

The request metering counter is decremented by a value that corresponds to the number of DWords
transferred on the link per 4ns period. The value is equal to the sum of the decrement value plus the value
of the Decrement Value Adjustment (DVADJ) field in the RMCTL register.

The decrement value is a sign-magnitude fixed-point 0:4:3 number (i.e., an positive number with 4
integer bits and 3 fractional bits), determined by the port’s negotiated link width and speed as shown in
Table 3.6.

– The least significant eight fractional bits of the decrement value are always implicitly zero.

IDT Switch Core

Notes

0 DWords Nibbles

Sign 4 bits 3 bits

Fixed Decrement Value

0 DWords Fractional Part

Sign 4 bits 11 bits

Decrement Value Adjustment

0000 0000

8 bits

0 DWords Fractional Part

Sign 4 bits 11 bits

Actual Decrement Value

The Decrement Value Adjustment (DVADJ) field represents a sign-magnitude fixed point 0:4:11 number
(i.e., a positive fixed-point number with 4 integer bits and 11 fractional bits).

– DVADJ field provides fine grain programmable adjustment of the value by which the counter is

decremented.

– The sign bit in the DVADJ field should not be set to negative (i.e., 0b1).

Figure 3.6 shows the decrement value and the decrement value adjustment.

The counter stops decrementing when it reaches zero or when a rollover occurs (i.e., the decrement
causes it to become negative).

The computation that occurs on each clock tick by the request metering counter is shown Figure 3.7.

Figure 3.6 Decrement Value and Decrement Value Adjustment

Link

Width

Link

Speed

Decrement Value Notes

x1 Gen 1 0x02 Corresponds to 1 Byte per clock tick

x2 Gen 1 0x04 Corresponds to 2 Bytes per clock tick

x4 Gen 1 0x08 Corresponds to 4 Bytes per clock tick

x8 Gen 1 0x10 Corresponds to 8 Bytes per clock tick

x1 Gen 2 0x04 Corresponds to 2 Bytes per clock tick

x2 Gen 2 0x08 Corresponds to 4 Bytes per clock tick

x4 Gen 2 0x10 Corresponds to 8 Bytes per clock tick

x8 Gen 2 0x20 Corresponds to 16 Bytes per clock tick

Table 3.6 Request Metering Decrement Value

PES64H16G2 User Manual 3 - 11 April 5, 2013

IDT Switch Core

Notes

tmp = RequestMeteringCounter

RequestMeteringCounter -= (DecrementValue[LinkSpeed,LinkWidth] +
RMCTL.DVADJ)

if (tmp < RequestMeteringCounter) {

RequestMeteringCounter = 0

DataDWords = Length / 4 *4

If (DataDWords == 0) {

CompletionSizeEstimate = 3

} else if (DataDWords <= CnstLimit) {

CompletionSizeEstimate = DataDWords + 1

} else {

OverheadDWords = (DataDWords >> OverheadFactor)

CompletionSizeEstimate = DataDWords + OverheadDWords

}

Figure 3.7 Request Metering Counter Decrement Operation

Completion Size Estimation

This section describes the value that is loaded into the request metering counter when a request is
transferred into the switch core. This value is referred to as the completion size estimate. The completion
size estimate is based on the type of non-posted request as described below. The request metering counter
is a 24-bit counter that represents a fixed point 0:13:11 number (i.e., an unsigned number with 13 integer
bits and 11 fractional bits).

The completion size estimate is a 0:13:3 number. The least significant eight fractional bits of the comple­tion size estimate are always implicitly zero.

Non-Posted Writes

The completion size estimate is 0x0018 which corresponds to 3 DWords (3 DWord header)

Non-Posted Reads

The completion size estimate is based on the Length field in the read request header and is computed
as shown in Figure 3.8. All arithmetic in this section is performed using an implicit 0:13:3 representation and
all values are implicitly converted to this value.

Figure 3.8 Non-Posted Read Request Completion Size Estimate Computation

The number of data DWords in a non-posted request TLP is estimated by the number of PCI Express
data credits required by the corresponding completion(s). Each PCI Express data credit is 4 DWords or 16
bytes.

– The first line in Figure 3.8 computes the number of DWords required by the completion(s) using

the number of required PCI Express data credits. This corresponds to PCI Express completion
data credits multiplied by 4.

PES64H16G2 User Manual 3 - 12 April 5, 2013

IDT Switch Core

Notes

If the number of data DWords is zero, then the completion size is estimated to be three DWords (i.e., a
0:13:3 representation value of 0x0018).

– Otherwise, if the number of required data DWords is less than the Constant Limit (CNSTLIMIT)

field in the RMCTL register, then the completion size is estimated as the number of required data
DWords plus one.

– Otherwise, if the number of required data DWords is greater than CNSTLIMIT, then the completion

size is estimated using OverheadDWords as described below.

OverheadDWords represents the number of DWords of link overhead. This includes the header, data
link layer overhead, and physical layer overhead of the completion TLP(s) associated with this request.
Ideally, OverheadDWords would be set to the number of completion TLPs associated with the request
multiplied by the TLP overhead. Unfortunately, this requires a multiplication. Therefore, the following esti­mate may be used.

A completion header is 3 DWords. There are 2 DWords of additional overhead associated with a TLP.
Therefore a reasonable estimate of the overhead is 5 DWords. In many systems, completions are 64-bytes
in size (i.e., 16 DWords in size).

OverheadDWords = (Length / 16) * 5.

– This is approximately equal to OverheadDWords = (Length / 16) * 4.
– This may be simplified to (Length / 4) and may be computed as (Length >> 2).

Thus, an acceptable value for OverheadFactor in many systems is 2. The OverheadFactor value used in
computing the completion size estimate is contained in the Overhead Factor (OVRFACTOR) field in the
RMCTL register.

Internal Errors

Internal errors are errors associated with a PCI Express interface that occurs within a component and
which may not be attributable to a packet or event on the PCI Express interface itself or on behalf of trans­actions initiated on PCI Express.

The PES64H16G2 classifies the following IDT proprietary switch errors as internal errors.

– Switch core time-outs
– Single and double bit internal memory ECC errors
– End-to-end data path parity protection errors

Internal errors are reported by the port in which they are detected through AER as outlined in the PCI­SIG Internal Error Reporting ECN. The reporting of internal errors may be disabled by clearing the Internal
Error Reporting Enable (IERROREN) bit in the port’s Internal Error Reporting Control (IERRORCTL)
register. When internal error reporting is disabled, the following AER fields become read-only:

– Uncorrectable Internal Error Mask (UIE) field in the AERUEM register
– Uncorrectable Internal Error Severity (UIE) field in the AERUESV register
– Correctable Internal Error Mask (CIE) field in the AERCEM register
– Header Log Overflow Mask (HLO) field in the AERCEM register

The PES64H16G2 does not support recording of headers for uncorrectable internal errors. When an
uncorrectable internal error is reported by AER, a header of all ones is recorded.

Corresponding to each possible internal error source is a status bit in the Internal Error Reporting Status
(IERRORSTS) register. A bit is set in the status register when the corresponding internal error is detected.
Associated with each internal error status bit in the IERRORSTS register is a mask bit in the Internal Error
Reporting Mask (IERRORMSK) register. When a mask bit is set in this register, the setting of the corre­sponding status bit is masked from generating an internal error.

Each internal error status bit has an associated severity bit in the IERRORSEV register. When an
unmasked internal error is detected, the error is reported as dictated by the corresponding severity bit (i.e.,
either an Uncorrectable Internal Error or a Correctable Internal Error). When an uncorrectable or correct­able internal error is reported, the corresponding AER status bit is set and process as dictated by the PCIe
base specification and Internal Error Reporting ECN.

PES64H16G2 User Manual 3 - 13 April 5, 2013

IDT Switch Core

Notes

To facilitate testing of software error handlers, any bit in the IERRORSTS register may be set by writing
a one to the corresponding bit position in the Internal Error Test (IERRORTST) register. Once a bit is set in
the ERRORSTS register, it is processed as though the actual error occurred (e.g., reported by AER).

Switch Time-Outs

The switch core discards any TLP that reaches the head of an IFB or EFB queue and is more than 64
seconds old. This includes posted, non-posted, completion and inserted TLPs. If during processing of a
TLP with broadcast routing a switch core time-out occurs, then the switch core will abort processing of the
TLP. This may result in the broadcast TLP being transmitted on some but not all downstream pots.

Memory SECDED ECC Protection

PCI Express provides reliable hop-by-hop communication between interconnected devices, such as
roots, switches, and endpoints, by utilizing a 32-bit Link CRC (LCRC), sequence numbers, and a link level
retransmission protocol. While this mechanism provides reliable communication between interconnected
devices, it does not protect against corruption that may occur inside of a device. PCI Express defines an
optional end-to-end data integrity mechanism that consists of appending a 32-bit end-to-end CRC (ECRC)
computed at the source over the invariant fields of a Transaction Layer Packet (TLP) that is checked at the
ultimate destination of the TLP. While this mechanism provides end-to-end error detection, unfortunately it
is an optional PCI Express feature and has not been implemented in many North-Bridges and endpoints. In
addition, the ECRC mechanism does not cover variant fields within a TLP.

Since deep sub-micron devices are known to be susceptible to single-event-upsets, a mechanism is
desired that detects errors that occur within a PCI Express switch.

The PES64H16G2 protects all memories (i.e., both data and control structures) with a Single Error
Correction with Double Error Detection (SECDED) Error Correcting Code (ECC). The objective of this
memory protection is to prevent silent data corruption. Single bit errors are automatically corrected and
optionally reported while double bit errors are optionally reported.

Double bit errors are uncorrectable memory errors that may compromise the integrity of control and data
structures. Detection of a double bit error may result in further modification of one or more memory bits in
the data quantity in which the error was detected (i.e., single bit error correction is not disabled when a
double bit error is detected and a double bit error may result in one or more single bit corrections).

Associated with each port are five memories: IFB control, IFB data, and EFB control, EFB data, and
Replay Buffer Control. Each port contains memory error control and status registers that are used to
manage memory errors associated with that port.

When a single or double bit error is detected in a memory, the status bit corresponding to the memory in
which the error was detected is set in the Internal Error Reporting Status (IERRORSTS) register.

A double bit error detected by a memory associated with TLP data (i.e., IFB or EFB data) results in the
TLP being nullified when it reaches the DL layer of an egress port. The TLP is nullified by inverting the
computed LCRC and ending the packet with an EDB symbol. Nullified TLPs received by a link partner are
discarded. Although the TLP is nullified, flow control credits associated with the egress port may not be
correctly updated. Thus, double bit errors could result in a flow control credit leak.

The DL layer never replays a TLP with a sequence number different from that initially used. If a double
bit error is detected during a DL layer replay, then all TLPs in the replay buffer are flushed.

If a double bit error is detected by an internal memory in a TLP that targets a function in the switch (e.g.,
a configuration read or write request to the PCI-to-PCI bridge function), then the TLP is discarded.

End-to-End Data Path Parity Protection

In addition to memory ECC protection, the PES64H16G2 supports end-to-end data path parity protec­tion. Data flowing into the PES64H16G2 is protected by the LCRC. Within the Data Link (DL) layer of the
switch ingress port, the LCRC is checked and a 32-bit DWord even parity is computed on the received TLP
data. If an LCRC error is detected at this point, the link level retransmission protocol is used to recover from
the error by forcing a retransmission by the link partner.

PES64H16G2 User Manual 3 - 14 April 5, 2013

IDT Switch Core

Notes

As the TLP flows through the switch, its alignment or contents may be modified. In all such cases, parity
is updated and not recomputed. Hence, any error that occurs is propagated and not masked by a parity
regeneration. When the TLP reaches the DL layer of the switch egress port, parity is checked and in parallel
an LCRC is computed. If the TLP is parity error free, then the LCRC and TLP contents are known to be
correct and the LCRC is used to protect the packet through the lower portion of the DL layer, PHY layer, and
link transmission.

If a parity error is detected by the DL layer of an egress port, then the TLP is nullified by inverting the
computed LCRC and ending the packet with an EDB symbol. Nullified TLPs received by the link-partner are
discarded. In addition to nullifying the TLP, the End-to-End Parity Error (E2EPE) bit is set in the Internal
Error Status (IERRORSTS) register. The DL layer never replays a TLP with a sequence number different
from that initially used. If a parity error is detected during a DL layer replay, then all TLPs in the replay buffer
are flushed.

In addition to TLPs that flow through the switch, cases exist in which TLPs are produced and consumed
by the switch (e.g., configuration requests and responses). Whenever a TLP is produced by the switch,
parity is computed as the TLP is generated. Thus, error protection is provided on produced TLPs as they
flow through the switch. In addition, parity is checked on all consumed TLPs. If an error is detected, the TLP
is discarded and an error is reported by setting the E2EPE bit in the IERRORSTS register.

A parity error reported at a switch port cannot be definitively used to identify the location within the
device at which the fault occurred as the fault may have occurred at another port, in the switch core, or may
have been generated locally (i.e., for ingress TLPs to the switch core which are consumed by the port such
as Type 0 configuration read requests on the upstream port).

PES64H16G2 User Manual 3 - 15 April 5, 2013

IDT Switch Core

Notes

PES64H16G2 User Manual 3 - 16 April 5, 2013

Notes

®

Chapter 4

SerDes

Quad

Port 0

Stack

SerDes

Quad

Port 1
Stack

SerDes

Quad

Port 2

Stack

SerDes

Quad

Port 3
Stack

SerDes

Quad

Port 14
Stack

SerDes

Quad

Port 15

Stack

Switch Core

PLL

...

GCLK

Port 0 Port 1

Port 2

Port 3 Port 14

Port 15

Clocking

Introduction

Figure 4.1 provides a logical representation of the PES64H16G2 clocking architecture. The
PES64H16G2 has a single differential global reference clock input (GCLK).

Figure 4.1 Logical Representation of the PES64H16G2 Clocking Architecture

The differential global reference clock input (GCLK) is driven into the device on the GCLKP[1:0] and
GCLKN[1:0] pins.

– The nominal frequency of the global reference clock input may be selected by the Global Clock

Frequency Select (GCLKFSEL) pin to be either 100 MHz or 125 MHz.

– Both global reference clock differential inputs should be driven with the same frequency. There

are no skew requirements between the GCLKP[0]/GCLKN[0] and GCLKP[1]/GCLKN[1] inputs.
Any constant phase difference is acceptable.

– The Global Clock supports Spread Spectrum Clocking (SSC).
– The global reference clock input is provided to each SerDes quad and to an on-chip PLL.

• The on-chip PLL uses this clock to generate a 250 MHz core clock that is used by internal switch

logic (e.g., switch core, portion of a stack, etc.).

• The PLL within each SerDes quad generates a 5.0 GHz clock used by the SerDes analog
portion (PMA) and a 250 MHz clock used by the digital portion (PCS).

Port Clocking Mode

Port clocking refers to the clock that a port uses to receive and transmit serial data. All ports in the switch
use the global reference clock (GCLK) input for receiving and transmitting serial data. The switch does not
introduce any requirements on the global reference clock input beyond those imposed by PCI express.
Depending on the system configuration, a port may employ the common Refclk or separate Refclk architec­tures defined by the PCIe Base specification.

The port clocking mode of a port is determined by the state of the CLKMODE[1:0] pins in the boot
configuration vector as shown in Table 4.1. This field determines the initial value of the Slot Clock Configu­ration (SCLK) field in each port’s PCI Express Link Status (PCIELSTS) register. The SCLK field controls the

PES64H16G2 User Manual 4 - 1 April 5, 2013

IDT Clocking

Notes

advertisement of whether or not the port uses the same reference clock source as the link partner. A one in
the SCLK field indicates that the port and its link partner use the same reference clock source. This is
defined as Common Clock Configuration by the PCI Express Base Specification. A zero in the SCLK field
indicates that the port and its link partner do not use the same reference clock source.

CLKMODE[1:0] Value in

Boot Configuration Vector

000

110

201

311

Table 4.1 Initial Port Clocking Mode and Slot Clock Configuration State

Port 0

SCLK

SCLK for Ports other

than Port 0

PES64H16G2 User Manual 4 - 2 April 5, 2013

IDT Clocking

PES64H16G2 User Manual 4 - 3 April 5, 2013

IDT Clocking

Notes

PES64H16G2 User Manual 4 - 4 April 5, 2013

Notes

®

Chapter 5

Reset and Initialization

Introduction

This chapter describes the PES64H16G2 resets and initialization. There are two classes of
PES64H16G2 resets. The first is a switch fundamental reset which is the reset used to initialize the entire
device. The second class is referred to as partition resets. This class has multiple sub-categories. Partition
resets are associated with a specific PES64H16G2 switch partition and corresponds to those resets defined
in the PCI express base specification. Switch resets are described in section Switch Fundamental Reset on
page 5-3 while partition resets are described in section Partition Resets on page 5-8.

When multiple resets are initiated concurrently, the precedence shown in Table 6.1 is used to determine
which one is acted upon.

– Reset types and causes are described in detail in the following sections.

• A partition reset affects the partition and ports associated with that partition.

• A switch fundamental reset affects the entire device

• A port reset affects only that one port

– When a high priority and low priority reset are initiated concurrently and the condition causing the

high priority reset ends prior to that causing the low priority reset, then the device/partition/port
immediately transitions to the reset associated with low priority reset condition.

• If the high priority and low priority resets share the same reset type, then the device/partition/
port remains in the corresponding reset when the high priority reset condition ends.

• If the high priority and low priority reset have different reset types, then the device/partition/port
transitions to the low priority reset type when the high priority reset condition ends.

Priority Reset Type Reset cause

1

(Highest)

2 Port mode change reset Port operating mode change and OMA field set to port

3 Partition fundamental

4 Partition fundamental

5 Partition hot reset Reception of TS1 ordered sets on upstream port indicat-

6 Partition hot reset Data link layer of the upstream port transitioning to

7 Partition upstream sec-

8

(Lowest)

Switch fundamental reset Global reset pin (PERSTN) assertion

reset in the corresponding SWPORTxCTL register

Assertion of partition fundamental reset pin (PARTxPER-

reset

reset

ondary bus reset

Partition downstream sec­ondary bus reset

Table 5.1 PES64H16G2 Reset Precedence

STN)

Directed by STATE field value in SWPARTxCTL register

ing a hot reset

DL_Down state

Setting of the SRESET bit in the partition’s upstream port
PCI-to-PCI bridge BCTL register

Setting of the SRESET bit in the in the corresponding
port’s PCI-to-PCI bridge BCTL register

Registers and fields designated as Switch Sticky (SWSticky) take on their initial value as a result of the

following resets. Other resets have no effect on registers and fields with this designation.

– Switch Fundamental Reset

PES64H16G2 User Manual 5 - 1 April 5, 2013

IDT Reset and Initialization

Notes

Registers and fields designated as Sticky (Sticky) take on their initial value as a result of the following

resets. Other resets have no effect on registers and fields with this designation.

All fields designated and Read Write when Unlocked (RWL) are implicitly SWSticky. Their value is

preserved across all resets except a switch fundamental reset.

Boot Configuration Vector

A boot configuration vector consisting of the signals listed in Table 5.2 is sampled during a switch funda­mental reset. Since the boot configuration vector is only sampled during a switch fundamental reset, the
value of signals that make up the boot configuration vector is ignored and their state outside of a switch
fundamental reset sequence has no effect on the operation of the device.

While basic switch operation may be configured using signals in the boot configuration vector, advanced
switch features require more complex initialization. This initialization may be performed by an external
SMBus device via the slave SMBus interface or may be performed automatically via the serial EEPROM.

See Chapter 13, SMBus Interfaces, for a description of the slave SMBus interface and serial EEPROM
operation.

As noted in table Table 5.2, some of the initial values specified by the boot configuration vector may be
overridden by software, serial EEPROM, or an external SMBus device. The state of all of the boot configu­ration signals in Table 5.2 sampled during a switch fundamental reset may be determined from the Boot
Configuration Status (BCVSTS) register.

– Switch Fundamental Reset
– Partition Fundamental Reset

Signal

GCLKFSEL N Global Clock Frequency Select.

CLKMODE[1:0] Y Clock Mode.

MSMBADDR[4:1] N Master SMBus Address.

MSMBMODE Y Master SMBus Slow Mode.

P01MERGEN N Ports 0 and 1 Merge.

P23MERGEN N Ports 2 and 3 Merge.

P45MERGEN N Ports 4 and 5 Merge.

May Be

Overridden

Name/Description

These pins specifies the frequency of the GCLKP and
GCLKN signals.

These pins specify the clocking mode used by switch ports.
See Table 4.1 for a definition of the encoding of these signals.
The value of these signals may be overridden by modifying
the Port Clocking Mode (PCLKMODE) register.

SMBus address of the serial EEPROM from which configura­tion information is loaded.

This pin specifies the operating frequency of the master
SMBus interface.The value of this signal may be overridden
by modifying the Master SMBus Clock Prescalar (MSMBCP)
field in the SMBus Control (SMBUSCTL) register.

This pin specifies whether ports 0 and 1 are merged.

This pin specifies whether ports 2 and 3 are merged.

This pin specifies whether ports 4 and 5 are merged.

P67MERGEN N Ports 6 and 7 Merge.

This pin specifies whether ports 6 and 7 are merged.

Table 5.2 Boot Configuration Vector Signals (Part 1 of 2)

PES64H16G2 User Manual 5 - 2 April 5, 2013

IDT Reset and Initialization

Notes

Signal

P89MERGEN N Ports 8 and 9 Merge.

P1011MERGEN N Ports 10 and 11 Merge.

P1213MERGEN N Ports 12 and 13 Merge.

P1415MERGEN N Ports 14 and 15 Merge.

RSTHALT Y Reset Halt.

SSMBADDR[5,3:1] N Slave SMBus Address.

SWMODE[3:0] N Switch Mode.

May Be

Overridden

Name/Description

This pin specifies whether ports 8 and 9 are merged.

This pin specifies whether ports 10 and 11 are merged.

This pin specifies whether ports 12 and 13 are merged.

This pin specifies whether ports 14 and 15 are merged.

When this pin is asserted during a switch fundamental reset
sequence, the PES64H16G2 remains in a reset state with the
Master and Slave SMBuses active. This allows software to
read and write registers internal to the device before normal
device operation begins. The device exits the reset state
when the RSTHALT bit is cleared in the SWCTL register by
an SMBus master.

SMBus address of the switch on the slave SMBus.

These pins specify the switch operating mode.

Table 5.2 Boot Configuration Vector Signals (Part 2 of 2)

Switch Fundamental Reset

A switch fundamental reset may be cold or warm. A cold switch fundamental reset occurs following a
device being powered-on and assertion of the global reset (PERSTN) signal. A warm switch fundamental
reset occurs when a switch fundamental reset is initiated while power remains applied. The PES64H16G2
behaves in the same manner regardless of whether the switch fundamental reset is cold or warm.

A switch fundamental reset may be initiated by any of the following conditions.

– A cold switch fundamental reset initiated by application of power (i.e., a power-on) followed by

assertion of the global reset (PERSTN) signal.

– A warm switch fundamental reset initiated by assertion of PERSTN while power remains applied.

When a switch fundamental reset is initiated, the following sequence is executed.

1. Wait for the switch fundamental reset condition to clear (e.g., negation of PERSTN).

2. On negation of PERSTN, sample the boot configuration vector signals shown in Table 5.2.

3. All registers are initialized to their default value.

4. The Register Unlock (REGUNLOCK) bit is set in the Switch Control (SWCTL) register.

5. The on-chip PLL and SerDes are initialized (i.e., PLL lock).

6. The master SMBus operating frequency is determined by examining the MSMBMODE state
sampled in the boot configuration vector. The master SMBus interface is initialized. The master
SMBus address is specified by the MSMBADDR[4:1] bits in the boot configuration vector.

7. The slave SMBus is taken out of reset and initialized. The slave SMBus address is specified by the
SSMBADDR[5,3:1] signals in the boot configuration vector.

8. Within 20 ms after the switch fundamental reset condition clears, the reset signal to the stacks is
negated and link training begins on all ports. While link training takes place, execution of the reset
sequence continues.

PES64H16G2 User Manual 5 - 3 April 5, 2013

IDT Reset and Initialization

Notes

9. Within 100 ms following clearing of the switch fundamental reset condition, the following occurs.

10. If the sampled Switch Mode (SWMODE[3:0]) state corresponds to a mode that supports serial

11. If the RSTHALT bit in the SWCTL register is set (e.g., due to the assertion of the RSTHALT signal in

– All ports that have PCI Express base specification compliant link partners have completed link

training.

– All ports are able to receive and process TLPs.

EEPROM initialization, then the contents of the serial EEPROM are read and appropriate switch
registers are updated.

– While the contents of the EEPROM are read, the switch responds to all configuration request with

configuration-request-retry-status completion.

– If a one is written by the serial EEPROM to the Full Link Retrain (FLRET) bit in any Phy Link State

0 (PHYLSTATE0) register, then link retraining is initiated on the corresponding port using the
current link parameters.

– If an error is detected during loading of the serial EEPROM, then loading of the serial EEPROM

is aborted and the RSTHALT bit is set in the SWCTL register. Error information is recorded in the
SMBUSSTS register.

– When serial EEPROM initialization completes, the EEPROM Done (EEPROMDONE) bit in the

SMBUSSTS register is set and the switch ports start processing configuration requests normally,
unless the RSTHALT bit in the SWCTL register is set.

If serial EEPROM initialization completes with an error, the RSTHALT bit in the SWCTL register
is set as described in section Initialization from Serial EEPROM on page 13-2. In this case, the
ports enter a quasi-reset state as described in step 11.

the sampled boot vector), the ports enter a quasi-reset state.

– In quasi-reset state, the port responds to all type 0 configuration request TLPs with a configura-

tion-request-retry-status completion

1

. All other TLPs are ignored (i.e., flow control credits are

returned but the TLP is discarded).

– The ports remain in quasi-reset state until the Reset Halt (RSTHALT) bit is cleared by software in

the SWCTL register.

This provides a synchronization point for a device on the slave SMBus to initialize the device.
When device initialization is completed, the slave SMBus device clears the RSTHALT bit allow­ing the device to begin normal operation.

12. The Register Unlock (REGUNLOCK) bit is cleared in the Switch Control (SWCTL) register.

13. Normal device operation begins as dictated by the SWMODE value in the boot configuration vector.

The PCI Express 2.0 specification indicates that a device must respond to Configuration Request trans­actions within 100ms from the end of Conventional Reset (cold, warm, or hot). Additionally, the PCI Express
specification indicates that a device must respond to Configuration Requests with a Successful Completion
within 1.0 second after Conventional Reset of a device. The reset sequence above guarantees that the
switch will be ready to respond successfully to configuration requests within the 1.0 second period as long
as the serial EEPROM initialization process completes within 200 ms.

– Under normal circumstances, 200 ms is more than adequate to initialize registers in the device

even with a Master SMBus operating frequency of 100 KHz.

Serial EEPROM initialization may cause writes to register fields that initiate side effects such as link
retraining. These side effects are initiated at the point at which the write occurs. Therefore, serial EEPROM
initialization should be structured in a manner so as to ensure proper configuration prior to initiation of these
side effects.

1.

This includes configuration requests to the Global Address Space Access and Data registers (GASAADDR and

GASADATA). Type 1 configuration request TLPs are handled as unsupported requests.

PES64H16G2 User Manual 5 - 4 April 5, 2013

IDT Reset and Initialization

Notes

GCLK*

Vdd

PERSTN

SerDes

Master SMBus

Slave SMBus

CDR Lock

Link Ready

Ready for Normal Operation

Ready

Serial EEPROM Initiali zation

1. Clock not shown to scale

~285 s

Switch Ports held in Quasi-Reset Mode

Link Training

PLL Reset & Lock

> 100ns

Stable
Power

Stable
GCLK

~2 s

< 100 ms

< 200 ms

Switch ports begin to process TLPs normally

SerDes

Slave SMBus

CDR Lock

Link Ready

Ready for Normal Operation

Switch Ports held in Quasi-Reset Mode

Link Training

PLL Reset & Lock

RSTHALT

Boot Vector sampled and RSTHALT bit in SWCTL register is set

RSTHALT bit in SWCTL cleared (e.g., via slave SMBus)

GCLK*

Vdd

PERSTN

1. Clock not shown to scale

> 100ns

Stable
Power

Stable
GCLK

~285 s ~2 s

< 100 ms

Switch ports begin to process TLPs normally

The operation of a switch fundamental reset with serial EEPROM initialization is illustrated in Figure 5.1.

The operation of a switch fundamental reset using RSTHALT is illustrated in Figure 5.2.

Figure 5.1 Switch Fundamental Reset with Serial EEPROM Initialization

Figure 5.2 Fundamental Reset Using RSTHALT to Keep Device in Quasi-Reset State

PES64H16G2 User Manual 5 - 5 April 5, 2013

IDT Reset and Initialization

Notes

Switch Mode Dependent Initialization

Switch modes may be subdivided into normal switch modes and test modes. The modes listed below
are normal switch modes. All other switch modes are test modes.

This section outlines switch mode specific initialization that occurs at the end of the switch fundamental
reset sequence. Table 5.3 lists the initial value of non-test mode register fields that are dependent on
normal switch modes. The effect of these initial values are described in section Single Partition Mode on
page 5-6 through section Multi-Partition Mode on page 5-7.

Some of the switch modes have an option with and without serial EEPROM initialization. Except for
serial EEPROM initialization, these switch modes are identical. Therefore, Table 5.3 lists only the version
without serial EEPROM initialization.

– Single partition
– Single partition with Serial EEPROM initialization
– Multi-partition
– Multi-partition with Serial EEPROM initialization
– Single partition with port 0 selected as the upstream port (port 2 disabled)
– Single partition with port 2 selected as the upstream port (port 0 disabled)
– Single partition with Serial EEPROM initialization and port 0 selected as the upstream port (port

2 disabled)

– Single partition with Serial EEPROM initialization and port 2 selected as the upstream port (port

0 disabled)

Switch Mode (SWMODE)

0x8 or 0xA

Register and Field

SWPART[0]CTL.STATE 0x1 0x0 0x1 0x1

SWPART[15:1]CTL.STATE 0x0 0x0 0x0 0x0

SWPORT[0]CTL.MODE 0x2 0x5 0x2 0x0

SWPORT[2]CTL.MODE 0x1 0x5 0x0 0x2

SWPORT[15:3,1]CTL.MODE 0x1 0x5 0x1 0x1

0x0 or 0x1

Single

Partition

Table 5.3 Switch Mode Dependent Register Initialization

0xC or 0xD

Multi-

Partition

SWMODE

Single

Partition with

Port 0

Upstream

Port

0x9 or 0xB

Single

Partition with

Port 2

Upstream

Port

Single Partition Mode

In single partition with port 0 upstream port, the initial values outlined in Table 5.3 result in the following
configuration:

– All switch ports are members of partition zero.
– Port zero is configured as the upstream port of partition zero. All other ports are configured as

downstream ports of partition zero.

– The initial state of partition zero is active. The initial state of all other partitions is disabled.

In this mode, it is not possible to configure partitions other than partition 0.

PES64H16G2 User Manual 5 - 6 April 5, 2013

IDT Reset and Initialization

Notes

Single Partition with Port 0 Upstream Port (Port 2 disabled)

In single partition with port 0 upstream port, the initial values outlined in Table 5.3 result in the following
configuration.

In this mode, it is not possible to configure partitions other than partition 0.

Single Partition with Port 2 Upstream Port (Port 0 disabled)

In single partition with port 2 as the upstream port, the initial values outlined in Table 5.3 result in the
following configuration.

In this mode, it is not possible to configure partitions other than partition 0.

Multi-Partition Mode

In multi-partition mode, the initial values outlined in Table 5.3 result in the following configuration.

In this mode, switch partitioning is possible.

– All switch ports, except port 2, are members of partition zero.
– Port 2 is disabled.
– Port zero is configured as the upstream port of partition zero. All other ports associated with parti-

tion zero are configured as downstream ports.

– The initial state of partition zero is active. The initial state of all other partitions is disabled.

– All switch ports, except port 0, are members of partition zero.
– Port 0 is disabled.
– Port two is configured as the upstream port of partition zero. All other ports associated with parti-

tion zero are configured as downstream ports.

– The initial state of partition zero is active. The initial state of all other partitions is disabled.

– All switch ports are configured to operate in unattached mode.
– The initial state of all partitions is disabled.

Port Merging

The switch allows merging of ports to form a single port whose link width is the aggregate sum of the
individual port widths. Port merging is only supported between an even numbered port and its subsequent
odd numbered port. The PxyMERGEN signals, sampled during switch fundamental reset, select which
ports are merged. For example, if the P45MERGEN signal is driven asserted (i.e., low) at switch funda­mental reset, then ports 4 and 5 are merged. It is not possible to change this port configuration until a
subsequent switch fundamental reset.

When two ports are merged, the even numbered port is active and its odd-numbered pair is de-acti­vated. For example, when ports 4 and 5 are merged, port 4 remains active and port 5 is de-activated. A de­activated port has the following behavior:

– All output signals associated with the port are placed in a negated state (e.g., link status and hot-

plug signals).

• The negated value of PxAIN, PxILOCKP, PxPEP, PxPIN, and PxRSTN is determined as shown
in Table 10.2.

• PxACTIVEN and PxLINKUPN are negated.

– All input signals associated with the port are ignored and have no effect on the operation of the

device.

• The state of the following hot-plug input signals is ignored: PxAPN, PxMRLN, PxPDN, PxPFN,
and PxPWRGDN.

– The port is not associated with a PCI Express link. PCI Express configuration requests targeting

the port are not possible and the port is not part of the PCI Express hierarchy.

– The port is not associated with any switch partition. The port is unaffected by the state of any

switch partition, and vice-versa.

– Unused logic is placed in a low power state.

PES64H16G2 User Manual 5 - 7 April 5, 2013

IDT Reset and Initialization

Notes

An active port behaves as described throughout the rest of this specification and may be configured in

one of several operating modes, as described in Chapter 6, Switch Partitions.

Partition Resets

A partition reset is a reset that is associated with a specific switch partition. The reset has an effect only
on those functions and switch ports associated with that switch partition. It has no effect on the operation of
other switch partitions, ports in other switch partitions, or logic not associated with a switch partition (e.g.,
Master SMBus).

A partition reset may be subdivided into four subcategories: partition fundamental reset, partition hot
reset, partition upstream secondary bus reset, and partition downstream secondary bus reset. These
subcategories correspond to resets defined by the PCI architecture.

The operation of the slave SMBus interface is unaffected by a partition reset. Using the slave SMBus to
access a register that is in the process of being reset causes the register’s default value to be returned on a
read and written data to be ignored on writes.

– All registers associated with the port remain accessible from the global address space.

1

– The port remains in this state regardless of the setting of the port’s operating mode (i.e., via the

port’s SWPORTxCTL register).

– A partition fundamental reset logically causes all logic associated with a partition to take on its

initial state, but does not cause the state of register fields denoted as SWSticky to be modified.

– A partition hot reset logically causes all logic to be returned to an initial state, but does not cause

the state of register fields denoted as Sticky or SWSticky to be modified.

– An partition upstream secondary bus reset logically causes all devices on the virtual PCI bus of a

partition to be hot reset except the upstream port (i.e., upstream PCI-to-PCI bridge).

– A partition downstream secondary bus reset causes a hot reset to be propagated on the corre-

sponding external link.

Partition Fundamental Reset

A partition fundamental reset is initiated by any of the following events.

– A switch fundamental reset.
– Assertion of a partition fundamental reset signal.
– As directed by the Switch Partition State (STATE) field in the Switch Partition (SWPARTxCTL)

register.

Associated with each partition is a partition fundamental reset input (PARTxPERSTN).

– The partition fundamental reset input for the first four partitions (i.e., partitions zero through three)

are available as GPIO alternate functions.

– The partition fundamental reset input for all partitions are available on external I/O expanders.

When a partition fundamental reset is initiated, the following sequence of actions take place.

1. All logic associated with the switch partition (i.e., switch ports, switch core, etc.) is logically
reset to its initial state.

2. All port links associated with the partition enter the ‘Detect’ state.

3. All registers and fields, except those designated as SWSticky, take on their initial value. The
value of SWSticky registers and field is preserved across a partition fundamental reset.

4. As long as the condition that initiated the partition fundamental reset persists (e.g., the
fundamental reset signal is asserted or the STATE field remains set to reset), logic associ­ated with the partition remains at this step.

5. Ports associated with the partition begin to link train and normal partition operation begins.

1.

Refer to Chapter 15, Register Organization, for details on the switch’s global address space.

PES64H16G2 User Manual 5 - 8 April 5, 2013

IDT Reset and Initialization

Notes

Partition Hot Reset

A partition hot reset is initiated by any of the following events:

When a partition hot reset is initiated the following sequence of actions take place.

1. The upstream port associated with the partition transitions its PHY LTSSM

2. Each downstream switch port associated with the partition whose link is ‘up’ propagates a hot reset

3. All register fields and registers associated with the switch partition except those designated Sticky

4. As long as the condition that initiated the partition hot reset persists, logic associated with the parti-

– Reception of TS1 ordered-sets on the partition’s upstream port indicating a hot reset.
– Data link layer of the partition’s upstream port transitions to the DL_Down state.
– As directed by the Switch Partition State (STATE) field in the Switch Partition (SWPARTxCTL)

register.

1

state to the appropriate
state (i.e., the hot reset state on reception of TS1 ordered-sets indicating hot reset or else the Detect
state).

by transmitting TS1 ordered sets with the hot reset bit set. All logic associated with the switch parti­tion (i.e., switch ports, switch core, etc.) is logically reset to its initial state.

If the link associated with a downstream port is in the Disabled LTSSM state, then a hot reset
will not

be propagated out on that port. The port will instead transition to the Detect LTSSM

state. Although not a hot reset, this has the same functional effect on downstream components.

and SWSticky, are reset to their initial value. The value of Sticky and SWSticky registers and fields
is preserved across a hot reset.

tion remains at this step.

5. Ports associated with the partition begin to link train and normal partition operation begins.

The initiation of a hot reset due to the data link layer of the upstream port transitioning to the DL_Down
state may be disabled by setting the Disable Link Down Hot Reset (DLDHRST) bit in the corresponding
Switch Partition Control (SWPARTxCTL) register. When the DLDHRST is set and the upstream port’s data
link is down, the PHY LTSSM transitions to the appropriate states but the hot reset steps described above
are not executed. As a result, the behavior of the partition is the following:

– Peer-to-peer TLP transfers between downstream ports are allowed to progress.
– TLPs not destined to a downstream port are treated as unsupported requests.
– TLPs generated by the switch and that are normally routed to the root (e.g., INTx messages) are

silently discarded.

– Downstream ports are allowed to enter and exit L0s and L1 ASPM state without regard to the

ASPM state of the upstream port (i.e., since there is no upstream port, then upstream port plays
no role in determining when a downstream port enters or exists a low power ASPM state).

Note that other hot reset trigger conditions (i.e., hot reset triggered by reception of training sets with the
hot reset bit set on the upstream port) are unaffected by the DLDHRST bit.

Partition Upstream Secondary Bus Reset

A partition upstream secondary bus reset is initiated by any of the following events.

– A one is written to the Secondary Bus Reset (SRESET) bit in the partition’s upstream port Bridge

Control (BCTL) register

2

.

When an Upstream Secondary Bus Reset occurs, the following sequence of actions take place on logic
associated with the affected partition.

1. Each downstream port whose link is up propagates the reset by transmitting TS1 ordered sets with
the hot reset bit set.

1.

The term ‘LTSSM’ refers to a port’s Link Training and Status State Machine in the Physical Layer.

2.

Note that the Bridge Control Register is only present in Type 1 Configuration Headers (i.e., PCI-to-PCI Bridge

functions).

PES64H16G2 User Manual 5 - 9 April 5, 2013

IDT Reset and Initialization

Notes

2. All registers fields in all registers associated with downstream ports, except those designated Sticky

3. All TLPs received from downstream ports and queued in the PES64H16G2 are discarded.

4. Logic in the stack and switch core associated with the downstream ports are gracefully reset.

5. Wait for software to clear the Secondary Bus Reset (SRESET) bit in the upstream port’s Bridge

6. Normal downstream port operation begins.

The operation of the upstream port is unaffected by a secondary bus reset. The link remains up and

Type 0 configuration read and write transactions that target the upstream port complete normally.

During an Upstream Secondary Bus Reset, all TLPs destined to the secondary side of the upstream

port’s PCI-to-PCI bridge are treated as unsupported requests.

The operation of the slave SMBus interface is unaffected by an Upstream Secondary Bus Reset. Using
the slave SMBus to access a register that is reset by an Upstream Secondary Bus Reset causes the
register’s default value to be returned on a read and written data to be ignored on writes.

Partition Downstream Secondary Bus Reset

A partition downstream secondary bus reset may be initiated by the following condition:

When a Downstream Secondary Bus Reset occurs, the following sequence of actions take place on
logic associated with the affected partition.

The operation of the upstream port is unaffected by a partition downstream secondary bus reset. The
operation of other downstream ports in this and other partitions is unaffected by a partition downstream
secondary bus reset. During a partition downstream secondary bus reset, Type 0 configuration read and
write transactions that target the downstream port complete normally. During a partition downstream
secondary bus reset, all TLPs destined to the secondary side of the downstream port’s PCI-to-PCI bridge
are treated as unsupported requests.

• If the link associated with a downstream port is in the Disabled LTSSM state, then a hot reset
will not be propagated out on that port. The port will instead transition to the Detect LTSSM state.
Although not a hot reset, this has the same functional effect on downstream components.

and SWSticky, are reset to their initial value. The value of fields designated Sticky or SWSticky is
unaffected by an Upstream Secondary Bus Reset.

Control Register (BCTL).

– A one is written to the Secondary Bus Reset (SRESET) bit in a downstream port’s Bridge Control

Register (BCTL).

– If the corresponding downstream port’s link is up, TS1 ordered sets with the hot reset bit set are

transmitted

– All TLPs received from corresponding downstream port and queued are discarded.
– Wait for software to clear the Secondary Bus Reset (SRESET) bit in the downstream port’s Bridge

Control Register (BCTL).

– Normal downstream port operation begins.

Port Mode Change Reset

A port mode change reset occurs when a port operating mode change is initiated and the OMA field in
the corresponding SWPORTxCTL register specifies a reset. Port mode change reset behavior is described
in section Reset Mode Change Behavior on page 6-14.

PES64H16G2 User Manual 5 - 10 April 5, 2013

Notes

®

Chapter 6

Switch Partitions

Introduction

The PES64H16G2 supports up to 16 active switch partitions. Each switch partition represents an inde­pendent PCI Express hierarchy whose operation is independent of other switch partitions.

A port may be configured to operate in one of four modes.

– Disabled port
– Downstream switch port
– Upstream switch port
– Unattached

The operating mode of a switch port may be dynamically reconfigured without affecting in any way unre­lated switch partitions.

Switch Partitions

A switch partition represents a logical container to which ports are attached. Each switch partition has an
associated ID. The PES64H16G2 supports 16 switch partitions with IDs zero through 15. A port is attached
to a switch partition by setting the Switch Partition (SWPART) field in the corresponding Switch Port x
Control (SWPORTxCTL) register to the ID of the partition to which the port should be attached and config­uring the port to be in one of the following modes.

– Downstream switch port
– Upstream switch port

A port in a disabled or unattached mode is not associated with a switch partition. In these modes, the
behavior of a port is unaffected by the state of any partition.

Partition Configuration

The following list represents valid switch partition configurations. The behavior of all other configurations
is undefined.

– A switch partition with no associated ports
– A switch partition with one upstream port and zero or more downstream ports
– A switch partition with no upstream port and one or more downstream ports

An upstream port is a port configured to operate in upstream switch port mode and is attached to the
partition. The Upstream Port (US) bit is set in the Switch Partition Status (SWPARTxSTS) register when
there is an upstream port associated with the partition. When the US bit is set in the SWPARTxSTS register,
the Upstream Port ID (USID) field contains the port ID of the upstream port of the partition.

A downstream port is a port configured to operate in downstream switch port mode and attached to the
partition. A switch partition with no associated ports represents a logical entity. The state and configuration
of such a partition has no functional effect on the operation of the device.

A switch partition with one upstream switch port and zero or more downstream ports represents a stan­dard PCI express switch configuration. Such a configuration operates as a standard PCI express switch
using the associated ports.

PES64H16G2 User Manual 6 - 1 April 5, 2013

IDT Switch Partitions

Notes

Fundamental

Reset

Disabled Active

Any State

A partition with one upstream port and no downstream ports has the following behavior.

– All received requests, except configuration requests that target the upstream port, are treated as

unsupported requests.

– All received completions are treated as unsupported requests.
– The upstream port is allowed to enter and exit L0s and L1 ASPM state without regard to the ASPM

state of a downstream port (i.e., since there are no downstream ports, they play no role in deter­mining when an upstream port enters or exists a low power ASPM state).

A partition with no upstream switch port and zero or more downstream ports operates in a manner
similar to that of a PCIe switch when the DLDHRST bit set in the SWPARTxCTL register and the data link
layer of the upstream port transitions to the DL_Down state (see section Partition Hot Reset on page 5-9 for
details).

– Peer-to-peer TLP transfers between downstream ports are allowed to progress.
– TLPs not destined to a downstream port are treated as unsupported requests.
– TLPs generated by the switch and that are normally routed to the root (e.g., INTx messages) are

silently discarded.

– Downstream ports are allowed to enter and exit L0s and L1 ASPM state without regard to the

ASPM state of the upstream port (i.e., since there is no upstream port, then upstream port plays
no role in determining when a downstream port enters or exists a low power ASPM state).

Partition State

A partition may be in one of three states: disabled, fundamental reset, and active. The state of a partition
is determined by the State (STATE) field in the Switch Partition Control (SWPARTxCTL) register. Valid state
transitions are shown in Figure 6.1. State transitions other than those shown in Figure 6.1 produce unde­fined results.

Disabled

A partition in the disabled state represents unused and idle resources. Ports associated with a disabled
partition are in a disabled mode (see section Disabled on page 6-2) regardless of the value of the Mode
(MODE) field in the Switch Port Control (SWPORTxCTL) register.

Fundamental Reset

A partition in the fundamental reset state operates in the same manner as a traditional PCIe component
would when with the fundamental reset (PERSTN) signal asserted. This corresponds to a partition funda­mental reset. See section Partition Fundamental Reset on page 5-8 for a details. The fundamental reset
state provides a software mechanism for resetting all logic and ports associated with a partition. Register

PES64H16G2 User Manual 6 - 2 April 5, 2013

values are initialized on entry to the fundamental reset state. Following initialization, register values may be
modified by masters in other partitions or via an external SMBus master.

Figure 6.1 Allowable Partition State Transitions

IDT Switch Partitions

Notes

The partition fundamental reset condition is considered to persist as long as the STATE field in the
SWPARTxCTL register remains in the fundamental reset state. Transitioning a partition from the funda­mental reset state to the active state requires that the system meet the requirements associated with a
conventional reset outlined in Section 6.6.1 in the PCIe Base specification.

Active

A partition in the active state is in the normal operating mode.

Partition State Change

This section describes requirements and restrictions that apply when modifying a partition’s state. In
general, the state of a partition may be modified at any time, except for the restrictions listed in the sub­sections below. Since a partition state change may take a significant amount of time to complete, status bits
are provided to indicate when the change has started and when it has completed.

– The Switch Partition State Change Initiated (SCI) bit in the Switch Partition Status

(SWPARTxSTS) register is set when a state change begins.

– The Switch Partition State Change Completed (SCC) bit in the Switch Partition Status

(SWPARTxSTS) register is set when a state change completes.

Partition state change requirements and restrictions differ based on the method used to program the
SWPARTxCTL register.

Partition State Change Latency

Partition state changes typically complete within a few clock cycles, unless the following conditions are
true. When the partition state change causes a fundamental reset in the partition (i.e., partition state set to
fundamental reset), the latency to complete the state change is 250 microseconds. This delay ensures that
the ingress and egress buffers of ports associated with the partition are fully drained before the partition
state change completes.

In addition, when the partition state change has the side-effect of modifying the operating mode of one
or more ports (e.g., partition is placed in the disabled state), the partition state change completes after

the
operating mode of the ports in the partition is changed. Refer to section Port Operating Mode Change on
page 6-7 for details on the latency port operating mode changes.

Partition State Change via EEPROM Loading

When modifying the state of a partition via the serial EEPROM, it is not possible to check the SCI and
SCC bits in the SWPARTxSTS register to obtain an indication of partition state change initiation and
completion. As a result, the following requirements and restrictions apply:

– The state of a partition can only be modified when there are no ports associated with the partition

(i.e., the partition is empty). Violating this rule produces undefined results.

– Prior to modifying the state of a partition, it is required that the following proprietary timer registers

be set to 0x0. This will ensure instantaneous execution of the partition state change action. The
last instructions in the EEPROM must set these timers back to their default values.

• Side Effect Delay Timer (SEDELAY register)

• Port Operating Mode Change Drain Delay Timer (POMCDELAY register)

• Reset Drain Delay Timer (RDRAINDELAY register)

• Upstream Secondary Bus Reset Delay (USSBRDELAY register)

– The GPIO alternate function associated with a partition reset signal (PARTxPERSTN) must not be

enabled

1

until all partition and port configurations are performed.

– In the serial EEPROM, the instructions that execute partition and port configurations must be

placed before the instruction that enables a GPIO alternate function associated with a partition
reset signal (PARTxPERSTN). The instruction that enables the PARTxPERSTN GPIO alternate
function may immediately follow the last instruction that configured a partition state and/or port
mode.

1.

Refer to Chapter 12 for details on enabling alternate functionality on GPIO signals.

PES64H16G2 User Manual 6 - 3 April 5, 2013

IDT Switch Partitions

Notes

Refer to section Static Reconfiguration on page 6-15 for a sample partition and port configuration
sequence programmed via the serial EEPROM.

Partition State Change via Other Methods

When modifying the state of a partition via methods other than EEPROM loading (i.e., via PCI Express
configuration requests or using the SMBus slave), the following requirements and restrictions apply:

– Once a partition state change has been initiated, it must be allowed to complete (e.g., by polling

the SCI field) before a new partition state change is initiated on the same partition, before
changing the operating mode of ports in the partition, and before adding ports into the partition.
Violating this rule produces undefined results.

• The setting of the SCC bit indicates that all changes associated with the state change in the
entire device have completed (e.g., port operating mode changes).

– The state of a partition must not be modified at the same time that the operating mode of one or

more ports in the partition is being changed. Violating this rule produces undefined results.

– The state of a partition must not be modified at the same time that one or more ports are being

added into the partition. Violating this rule produces undefined results.

– When a partition is experiencing fundamental reset caused by the assertion of the partition funda-

mental reset signal (PARTxPERSTN), the following partition state changes will not
the SCC bit will not be set) until the PARTxPERSTN signal is de-asserted:

Disabled  Active
Fundamental Reset  Active

complete (i.e.,

Switch Ports

A switch port is a logical entity that represents a PCIe link, stack logic (e.g., physical layer, data link
layer, etc.), and TLP queues (i.e., port’s IFB and EFB) required to communicate on that link. All switch ports
are completely independent. The configuration or state of one switch port in no way affects the operation or
restricts the possible configuration of another port.

Switch Port Mode

A switch port may be configured to operate in one of the following modes.

– Disabled
– Unattached
– Upstream switch port
– Downstream switch port

A port in the disabled or unattached state is not associated with any switch partition (i.e., the port is
considered unattached).

A port in an operational mode is attached to the partition specified by the Switch Partition (SWPART)
field in the corresponding Switch Port x Control (SWPORTxCTL) register. The following switch port modes
are considered operational modes.

– Upstream switch port
– Downstream switch port

In general, the state of a port is determined by the Mode (MODE) field in the Switch Port Control
(SWPORTxCTL) register. The only exception is a port that is attached to a partition that is in the disabled
state. A disabled partition causes all attached ports to be in a disabled mode regardless of the MODE field
setting.

Disabled

A port that is disabled is considered unused and is placed in a low power state. For example, the port
does not generate power management messages, it does not generate INTx or MSI interrupts, and it not
generate error messages.

PES64H16G2 User Manual 6 - 4 April 5, 2013

IDT Switch Partitions

Notes

A port in the disabled mode has the following behavior.

– All output signals associated with the port are placed in a negated state (e.g., link status and hot-

plug signals).

• The negated value of HPxAIN, HPxILOCKP, HPxPEP, HPxPIN, and HPxRSTN is determined as
shown in Table 10.2.

• PxACTIVEN and PxLINKUPN are negated.All input signals associated with the port are ignored
and have no effect on the operation of the device.

• Boot configuration vector signals are sampled during a switch fundamental reset and thus their
dynamic state has no effect on the operating mode of the port in any port mode.

For example, if the PxyMERGEN signal associated with the port was asserted during a switch
fundamental reset, then the port remains merged while disabled. The state of the following
port signals is ignored: HPxAPN, HPxMRLN, HPxPDN, HPxPFN, and HPxPWRGDN.

– Since a port in this state is not associated with a switch partition, the state of no switch partition

input signal (e.g., PARTxPERSTN) has effect on the port.

• Since the port is not associated with a switch partition in this state, the port is unaffected by the
state of any switch partition, and vice-versa.

• The LTSSM

1

immediately transitions to the Detect state and SerDes lanes associated with the

port are turned off.

• Unused logic is placed in a low power state (e.g., clock is gated).

• All registers associated with the port remain accessible from the global address space.

• PCI configuration requests to the port registers are not possible.

Register state is preserved in the disabled state. Therefore, transitioning a port from an operational state
to a disabled state and then back to an operational state results in no change in register values except for
fields that report status. All configuration registers in all functions associated with the port are accessible
through the global address space by the serial EEPROM, other ports, and the SMBus.

In this mode, all registers in the PCI-to-PCI bridge are accessible and the register space is organized as
that shown in Table 15.4 for a downstream port. Register fields whose value is dependent on port mode are
configured to operate as a downstream port.

Unattached

An unattached port is a port not associated with a switch partition and except for the link, is not opera­tional. A port in the unattached mode has the following behavior.

– The physical and data-link layers remain operational and the link behaves as an upstream port.

• The PxACTIVEN and PxLINKUPN signals are driven normally and reflect the state of the link.

• The LTSSM transitions and remains in the L0 state. If necessary, the link trains from the Detect

state.

• ASPM configuration and D-state power management is ignored.

The transaction layer functionality necessary to respond to configuration requests and manage flow
control remains active. All other logic associated with the port is disabled. For example, the port does not
generate power management messages, it does not generate INTx or MSI interrupts, and it does not
generate error messages.

All output signals associated with the port, except signals associated with the link, are placed in a
negated state.

– The negated value of HPxAIN, HPxILOCKP, HPxPEP, HPxPIN, and HPxRSTN is determined as

shown in Table 10.2.

– PxACTIVEN and PxLINKUPN are negated.
– The SerDes signals and link status and activity signals remain active.

1.

The term ‘LTSSM’ refers to a port’s Link Training and Status State Machine in the Physical Layer.

PES64H16G2 User Manual 6 - 5 April 5, 2013

IDT Switch Partitions

Notes

All input signals associated with the port, except the SerDes, are ignored and have no effect on the
operation of the device.

– Boot configuration vector signals are sampled during a switch fundamental reset and thus their

dynamic state has no effect on the operating mode of the port in any port mode.

– The state of the following port signals is ignored: HPxAPN, HPxMRLN, HPxPDN, HPxPFN, and

HPxPWRGDN.

– Since a port in this state is not associated with a switch partition, the state of no switch partition

input signal (e.g., PARTxPERSTN) has effect on the port.

Since the port is not associated with a switch partition in this state, the port is unaffected by the state of
any switch partition, and vice-versa. The port responds to received TLPs as follows.

– The port responds to all received PCI configuration requests that do not target the Global Address

Space Access Address or Data (GASAADDR or GASADATA) registers with a configuration
request retry status completion.

– The port responds to received PCI configuration requests that target the GASAADDR and GASA-

DATA normally (i.e., the requested action is performed and completion generated).

– The port responds to all other received requests by treading them as unsupported requests (i.e.,

logging error status and if appropriate, generating a completion). The port responds to comple­tions by discarding the completion and returning flow control credits.

– The port responds to all completions as unexpected completions.

All registers associated with the port remain accessible from the global address space.

– All configuration registers in the port are accessible through the global address space by the serial

EEPROM, other ports, and the SMBus.

– Although the link in this mode behaves as an upstream port, all registers in the port’s PCI-to-PCI

bridge function take on the organization and initial values shown in Table 15.4 for a downstream
port.

– Registers and fields that affect the behavior of the link (listed below) operate normally (i.e., control

fields control behavior and status fields report status) as though the port were an upstream port.

• PCIELCTL (all fields related to link/phy)

• PCIELSTS (all fields related to link/phy)

• PCIELCTL2 (all fields related to link/phy)

• PCIELSTS2 (all fields related to link/phy)

• SERDESCFG

• LANESTS[1:0]

• PHYPRBS

Registers other than those listed above operate as though the port were a downstream port and take on
the initial value of a downstream port. A port in this mode is unaffected by the following.

– The reception of TS1 ordered-sets indicating a hot reset.
– The data link layer of the port transitions to the DL_Down state.
– The reception of a Set_Slot_Power_Limit message.

Since the link operates as an upstream port, an automatic speed change is not initiated when the link
enters L0. Automatic speed change may be enabled by modifying the value of the Initial Link Speed
Change Control (ILSCC) bit in the Phy Link Configuration 0 (PHYLCFG0) register.

Upstream Switch Port

A port in upstream switch port mode behaves as the upstream port of a switch partition.

– All output signals associated with a downstream port are placed in a negated state (i.e., hot-plug

signals). The negated value of HPxAIN, HPxILOCKP, HPxPEP, HPxPIN, and HPxRSTN is deter­mined as shown in Table 10.2.

The PCI-to-PCI bridge associated with the upstream port has an associated type 1 header. The device
and function number are zero.

PES64H16G2 User Manual 6 - 6 April 5, 2013

IDT Switch Partitions

Notes

A port in the upstream switch port mode has the following behavior.

– Has the behavior of an upstream switch port defined by the PCI express base specification.
– The LTSSM is operational and behaves as an upstream port.

Downstream Switch Port

A port in downstream switch port mode behaves as the downstream port of a switch partition. The PCI­to-PCI bridge associated with a downstream switch port has an associated type 1 header. A port in the
downstream switch port mode has the following behavior.

– Has the behavior of a downstream switch port defined by the PCI express base specification.
– The LTSSM is operational and behaves as a downstream port.

The Device Number (DEVNUM) field in the SWPORTxCTL register determines the device number of
the downstream port within the switch partition. The function number of a downstream switch port is always
zero. The behavior when two or more downstream switch ports in the same partition are configured with the
same device number is undefined.

Since partitions are independent, two downstream switch ports in different partitions may share the
same device number. If the DEVNUM field is modified using a PCI configuration write request, then the
modification takes place prior to the generation of a completion for the request. The completion uses the old
(i.e., unmodified) device number.

Modifying the DEVNUM field using a PCI configuration write request does not modify the bus and device
numbers captured by the PCI-to-PCI bridge. Therefore, a subsequent Type 0 configuration write must be
performed to a PCI-to-PCI bridge following modification of its device number.

– Failure to follow this procedure will result in an incorrect value in the requester and completer ID

fields of TLPs generated by the PCI-to-PCI bridge.

– As expected, modifying the DEVNUM field via SMBus or Serial EEPROM does not modify the bus

and device numbers captured by the PCI-to-PCI bridge.

Port Operating Mode Change

The operating mode of a port is determined by the Port Mode (MODE), Switch Partition (SWPART), and
Device Number (DEVNUM) fields in the SWPORTxCTL register as well as in some cases the state (STATE)
field in the corresponding partition control (SWPARTxCTL) register. The initial operating mode of a port is
determined by the switch mode setting in the boot configuration vector sampled during a switch funda­mental reset.

The following events constitute a port operating mode change and initiate the action specified by the
Operating Mode Change Action (OMA) field in the corresponding SWPORTxCTL register.

– Any modification of the value in the MODE, SWPART, or DEVNUM fields in the SWPORTxCTL

register.

• A write of the same value already contains in these fields does not result in a port operating
mode change.

– Setting the STATE field in the SWPARTxCTL register of the partition with which the port is asso-

ciated to disabled.

When a port is disabled due to the STATE field in the corresponding partition SWPARTxCTL field being
set to disabled, modification of values in the MODE, SWPART, or DEVNUM fields that do not modify the
partition with which the port is associated result in the actions associated with an operating mode change
(e.g., OMCI and OMCC are set), but the port remains in the disabled mode during and after the operating
mode change process.

The modification of the MODE field to disabled causes the port to detach from the disabled partition;
however, the port remains in the disabled mode. All of the actions associated with an operating mode
change (e.g., OMCI and OMCC are set) continue to operate as described above. The modification of the
SWPART field causes the port to detach from the disabled partition. The new operating mode is dependent
on the MODE field and the state of the new partition with which the port is associated (if any).

PES64H16G2 User Manual 6 - 7 April 5, 2013

IDT Switch Partitions

Notes

Changing the operating mode of a port is subject to the requirements and restrictions listed in the sub­sections below. Since an operating mode change may take a significant amount of time to complete, status
bits are provided to indicate when the change has started and when it has completed.

– The Operating Mode Change Initiated (OMCI) bit in the Switch Port Status (SWPORTxSTS)

register is set when a mode change begins.

– The Operating Mode Change Completed (OMCC) bit in the Switch Port Status (SWPORTxSTS)

register is set when a mode change completes.

The port operating mode change requirements and restrictions differ based on the method used to
program the SWPORTxCTL register.

Port Operating Mode Change Latency

The latency to complete a port operating mode change is set to 2 ms if the port’s OMA field is set to ‘no
action’. If the port’s OMA field is set to ‘reset’, the latency is 2.5 ms.

Port Operating Mode Change via EEPROM Loading

When modifying the state of a port via the serial EEPROM, it is not possible to check the OMCI and
OMCC bits in the SWPORTxSTS register to obtain an indication of port operating mode change initiation
and completion. As a result, the following requirements and restrictions apply. Violating these requirements
results in undefined behavior.

Prior to modifying the operating mode of a port using EEPROM, it is required that the following proprie­tary timer registers be set to 0x0. This will ensure instantaneous execution of the port operating mode
change action which in turn facilitates following the rules stated above. The last instructions in the EEPROM
must set these timers back to their default values.

– Side Effect Delay Timer (SEDELAY register)
– Port Operating Mode Change Drain Delay Timer (POMCDELAY register)
– Reset Drain Delay Timer (RDRAINDELAY register)
– Upstream Secondary Bus Reset Delay (USSBRDELAY register)

Modifying the operating mode of a port via serial EEPROM loading requires that the OMA field be set to
‘no action’. The following port operating mode changes are allowed. All other port operating mode changes
are not allowed via serial EEPROM.

Unattached  Upstream switch port
Unattached  Downstream switch port
Unattached  Disabled

Changing the operating mode of a port from unattached to upstream switch port mode requires that the
PCI Express capabilities list and PCI Express extended capabilities list in the port’s PCI-to-PCI bridge func­tion be configured via the EEPROM to match the functionality of an upstream port (see section Capability
Structures on page 15-3 for details).

Specifically, the following register fields must be set appropriately.

– NXTPTR field in the PCI Power Management Capabilities (PMCAP) register
– NXTPTR field in the PCI Express VC Extended Capability Header (PCIEVCCAP) register

Refer to section Static Reconfiguration on page 6-15 for a sample partition and port configuration
sequence programmed via the serial EEPROM.

Port Operating Mode Change via Other Methods

When modifying the operating mode of a port via methods other than EEPROM loading (i.e., via PCI
Express configuration requests or using the SMBus slave), the following requirements and restrictions
apply:

– Once an operating mode change to a port has been initiated, this operating mode change must

be allowed to complete (e.g., by polling the OMCC field) before a new operating mode change is
initiated on the same port, or a partition state change is initiated on the partition associated with
the port.

PES64H16G2 User Manual 6 - 8 April 5, 2013

IDT Switch Partitions

Notes

– The operating mode of a port in upstream switch port mode can’t be later modified to downstream

switch port mode, except after a switch fundamental reset. This restriction holds even if the modi­fication from upstream switch port mode to downstream switch port mode goes through interme­diate states (e.g., unattached, disabled).

– Similarly, the operating mode of a port in downstream switch port mode can’t be later modified to

upstream switch port mode, except after a switch fundamental reset. This restriction holds even if
the modification from downstream switch port mode to upstream switch port mode goes through
intermediate states (e.g., unattached, disabled).

– If a port is attached to a partition, the operating mode of a port must not be modified at the same

time that the state of the partition is being changed.

– If a port is being added to a partition, this action must not be done at the same time that the state

of the partition is being changed.

– Changing the operating mode of a port from unattached to upstream switch port mode requires

that the PCI Express capabilities list and PCI Express extended capabilities list in the port’s PCI­to-PCI bridge function be configured to match the functionality of an upstream port (see section
Capability Structures on page 15-3 for details). Specifically, the following register fields must be
set appropriately.

• NXTPTR field in the PCI Power Management Capabilities (PMCAP) register

• NXTPTR field in the PCI Express VC Extended Capability Header (PCIEVCCAP) register

– The OMA field must be set to ‘reset’ or ‘no action’ for the following operating mode changes. If set

to ‘no action’, it is required that the link be fully retrained by setting the FLRET bit in the port’s
PHYLSTATE0 register after the operating mode change completes.

• Unattached port  Downstream Switch Port

• Downstream Switch port  Unattached port

– If a partition is experiencing a partition fundamental reset initiated by the assertion of the partition’s

PARTxPERSTN signal, a port outside the partition can only be added into the partition if the OMA
field is set to ‘no action’.

– If a partition is experiencing a partition hot reset initiated by the reception of TS1 ordered-sets on

the partition’s upstream port indicating a hot reset, or by the data link layer of the partition’s
upstream port transitioning to the DL_Down state, a port outside the partition can only be added
into the partition if the OMA field is set to ‘no action’.

Common Operating Mode Change Behavior

This section specifies common port operating mode change behavior that occurs regardless of the OMA
field setting. The following sections describe specific behaviors associated with specific OMA field settings.

Modifying the operating mode of a port requires that no TLP traffic flow through the port during modifica­tion of the operating mode (i.e., traffic on that port should be quiesced).A port operating mode change may
result in a port being removed from a partition, a port being added to a partition, both a removal and an
addition, or neither.

– The partition from which a port is removed is referred to as the source partition.
– The partition to which a port is added is referred to as the destination partition.
– When a port operating mode changes within a partition, it may have a substantial effect on the

partition and is thus logically viewed as a removal followed by an addition of the same port to the
same partition.

A port operating mode change that is caused only by a device number change is logically viewed as a
source partition removal, followed by a device number change, followed by a destination partition addition
to the same partition. The intent of this requirement is to make a device number change operate in the
same manner as all other port operating mode changes.

PES64H16G2 User Manual 6 - 9 April 5, 2013

IDT Switch Partitions

Notes

When a port operating mode change is initiated, the operation logically executes in the following order.

1. The OMCI bit in the SWPORTxSTS register is set.

2. The effect on the source partition, if appropriate, takes place (i.e., cleanly remove the port from the
partition).

3. The effect on the destination partition, if appropriate, takes place (i.e., cleanly add the port to the
partition).

4. The effect on the port, if appropriate, takes place (i.e., as dictated by the OMA field).

5. The OMCC bit in the SWPORTxSTS register is set.

Mode Change Effect on Source Partition

A port operating mode change that results in a port being removed from a partition has the following

effect on that partition (i.e., the “source” partition).

– If the port that is removed is the upstream port, then the partition operates without an upstream

port as described in section Partition Configuration on page 6-1.

Partition Hot Reset

See section Partition Hot Reset on page 5-9 for an overview of partition hot resets. The removal of an
upstream port that is initiating a partition hot reset (i.e., as the result of reception of TS1 ordered sets with
the hot reset bit set or DL_Down) has the effect of removing the hot reset condition on the partition. The
removal of a downstream port that is affected by a source partition hot reset has no effect on the source
partition (i.e., the hot-reset operation continues normally on the other ports in the partition).

– The removed downstream port stops participating in partition hot reset(s) after the removal of the

port has been completed.

Partition Upstream Secondary Bus Reset

See section Partition Upstream Secondary Bus Reset on page 5-9 for an overview of partition
secondary bus resets. The removal of an upstream port whose SRESET bit in the BCTL register is set has
the same effect as clearing of the SRESET bit (i.e., the partition secondary bus reset condition is removed).
The removal of a downstream port that is affected by a source partition upstream secondary bus reset has
no effect on the source partition (i.e., the hot-reset operation continues normally on the other ports in the
partition).

– The removed downstream port stops participating in the upstream secondary bus reset after the

removal of the port has been completed.

Partition Downstream Secondary Bus Reset

See section Partition Downstream Secondary Bus Reset on page 5-10 for an overview of partition
secondary bus resets. The removal of a downstream port whose SRESET bit in the BCTL register is set
has no effect on the source partition since other ports in the partition are unaffected by this type of reset
(i.e., the hot-reset is propagated to the endpoint located below the port that is being removed).

Routing

Removing a port from a partition results in the corresponding invalidation of routes to that port.

L0s ASPM

A switch partition exhibits a correlation between the L0s ASPM state of its upstream and downstream
port(s). Refer to section Active State Power Management on page 7-11 and to the PCI Express Base spec­ification for details.

Downstream port removal:

– A switch partition must initiate an exit from L0s on the transmitter of the upstream port if it detects

an exit from L0s on the receiver of any downstream port.

– Removing a downstream port from a partition removes it from affecting the L0s ASPM state of

upstream port in the source partition.

PES64H16G2 User Manual 6 - 10 April 5, 2013

IDT Switch Partitions

Notes

Upstream port removal:

– A switch partition must initiate an exit from L0s on the transmitters of all downstream ports asso-

ciated with the partition if it detects an exit from L0s on the receiver of its upstream port.

– Removing an upstream port from a partition removes it from affecting the L0s ASPM state of

downstream ports of the source partition.

L1 ASPM

A switch upstream port is not allowed to initiate entry into L1 unless all of the downstream ports associ­ated within the partition are in an L1 (or deeper) state. Refer to section Active State Power Management on
page 7-11 and to the PCI Express Base specification for details.

Downstream port removal:

– Removing a downstream port from a partition removes it from affecting the L1 ASPM state of

upstream port in the source partition.

Upstream port removal:

– Removing an upstream port from a partition has no effect on the L1 state of downstream ports

associated with the partition.

PME Synchronization

Removing a port from a partition has the affect of removing it from participation in PME synchronization
associated with the source partition. If PME synchronization is in progress, then PME synchronization
completes before the port (upstream or downstream) is removed from the partition.

For an upstream port, PME synchronization completes when the port aggregates PME_TO_Ack
messages from all downstream ports in the partition or when the upstream port abandons the aggregation.

For a downstream port, PME synchronization completes when the port notifies the upstream port of the
reception of a PME_TO_Ack message or a PME_TO_Ack timer timeout.

In this scenario, the OMCI field in the port’s SWPORTxSTS register is set after the operating mode
change is requested, and the OMCC field in this same register is set after the PME synchronization has
finished and the operating mode change completes.

Bus Locking

Removing the upstream port from a partition immediately unlocks the partition. Removing a downstream
port that is locked has the effect of unlocking the partition. Removing a downstream port that is not locked
from a locked partition has no effect on the locked nature of the partition.

INTx Interrupt Signaling

Removing an upstream port from a partition has no effect on a partition since the interrupt state is asso­ciated with the root located above the upstream port that is being removed. An INTx state change signalled
by a downstream port in the source partition has no effect on the upstream port as the latter is no longer
associated with the partition.

Removing a downstream port from a partition has the affect of removing all INTx virtual wire assertions
associated with the port (i.e., INTA, INTB, INTC and INTD from the port are negated).

Mode Change Effect on Destination Partition

A port operating mode change that results in a port being added to a partition has the following effect on
that partition (i.e., the “destination” partition). Some of the behaviors described below rely on the state of the
port. The state of the port and its corresponding configuration registers are determined by the OMA field as
described in section No Action Mode Change Behavior on page 6-14 through section Hot Reset Mode
Change Behavior on page 6-15.

Partition Hot Reset

See section Partition Hot Reset on page 5-9 for an overview of partition hot resets. The addition of an
upstream port that is initiating a hot reset (i.e., as the result of reception of TS1 ordered sets with the hot
reset bit set or DL_Down) has the effect of initiating a partition hot reset to the destination partition as

PES64H16G2 User Manual 6 - 11 April 5, 2013

IDT Switch Partitions

Notes

described in section Partition Hot Reset on page 5-9. An upstream port whose link transition to the Detect
state (i.e., DL_Down) as a result of the operating mode change may trigger a hot-reset in the destination
partition as described in section Partition Hot Reset on page 5-9.

The addition of a downstream port to a destination partition that has a partition hot reset in progress
does not have an effect on the ongoing hot reset in the destination partition (i.e., the hot-reset operation
continues normally on the other ports in the partition). The added downstream port participates in the
ongoing hot reset after the addition of the port has completed.

Partition Upstream Secondary Bus Reset

See section Partition Upstream Secondary Bus Reset on page 5-9 for an overview of partition
secondary bus resets. The addition of an upstream port whose SRESET bit in the BCTL register is set has
the effect of initiating a partition upstream secondary bus reset.

The addition of a downstream port to a destination partition that has an upstream secondary bus reset in
progress does not have an effect on the ongoing hot reset in the destination partition (i.e., the hot-reset
operation continues normally on the other ports in the partition). The added downstream port participates in
the ongoing hot reset after the addition of the port has completed.

Partition Downstream Secondary Bus Reset

See section Partition Downstream Secondary Bus Reset on page 5-10 for an overview of partition
secondary bus resets. The addition of a downstream port whose SRESET bit in the BCTL register is set has
no effect on the destination partition since other ports in the partition are unaffected by this type of reset
(i.e., the hot-reset is propagated to the endpoint located below the port that is being removed).

Routing

Adding a port to a partition results in the routing specified by configuration registers associated with that
port being enabled in the destination partition.

L0s ASPM

A switch partition exhibits a correlation between the L0s ASPM state of its upstream and downstream
port(s). Refer to section Active State Power Management on page 7-11 and to the PCI Express Base spec­ification for details.

Downstream port addition:

– A switch partition must initiate an exit from L0s on the transmitter of the upstream port if it detects

an exit from L0s on the receiver of any downstream port.

– Adding a downstream port to a partition causes it to affect the L0s ASPM state of upstream port

in the destination partition.

If the upstream port’s transmitter is in L0 and a downstream port whose receiver is in L0s is
added to the partition, then an entry to L0s is initiated on the transmitter of the upstream port
per the rules described in section Active State Power Management on page 7-11.

If the upstream port’s transmitter is in L0s and a downstream port whose receiver is in L0 is
added to the partition, then an exit from L0s is initiated on the transmitter of the upstream port.

– Adding a downstream port to a partition causes the added port to be affected by the L0s ASPM

state of the upstream port in the destination partition.

For example, if a downstream port whose transmitter is in L0 is added to a switch partition
whose upstream port receiver is in L0s, then an entry to L0s is initiated on the added port’s
transmitter per the rules described in section Active State Power Management on page 7-11.

Also, if a downstream port whose transmitter is in L0s is added to a switch partition whose
upstream port receiver is in L0, then an exit from L0s is initiated on the added port’s transmit­ter.

PES64H16G2 User Manual 6 - 12 April 5, 2013

IDT Switch Partitions

Notes

Upstream port addition:

– A switch partition must initiate an exit from L0s on the transmitters of all downstream ports asso-

ciated with the partition if it detects an exit from L0s on the receiver of its upstream port. See the
PCI Express Base specification for details.

– Adding an upstream port to a partition causes it to affect the L0s ASPM state of downstream ports

in the destination partition.

If an upstream port whose receiver is in L0s is added to a switch partition, then an entry to L0s
is initiated on the transmitter of all downstream ports in L0 per the rules described in section
Active State Power Management on page 7-11.

If an upstream port whose receiver is in L0 is added to a switch partition, then an exit from L0s
is initiated on the transmitter of all downstream ports that are in L0s.

– Adding an upstream port to a partition causes the added port to be affected by the L0s ASPM state

of downstream ports in the destination partition.

For example, if an upstream port whose transmitter is in L0 is added to a switch partition
whose downstream ports are in L0s, then an entry to L0s is initiated on the port’s transmitter
per the rules described in section Active State Power Management on page 7-11.

Also, if an upstream port whose transmitter is in L0s is added to a switch partition whose
downstream ports are in L0, then an exit from L0s is initiated on the port’s transmitter.

L1 ASPM

Downstream port addition:

– A switch upstream port is not allowed to initiate entry into L1 unless all of the downstream ports

associated with the partition are in an L1 (or deeper) state. See section Active State Power
Management on page 7-11 and to the PCI Express Base specification for details.

– Adding a downstream port to a partition causes it to affect the L1 ASPM initiation of the upstream

port.

For example, if a downstream port in L0 is added to a switch partition whose upstream port is
in L1 ASPM, then an exit from L1 is initiated on the upstream port.

– Adding a downstream port to a partition causes the added port to be affected by the ASPM state

of the upstream port in the destination partition.

For example, if a downstream port in L1 ASPM is added to a switch partition whose upstream
port is in L0, then an exit from L1 is initiated on the added port.

Upstream port addition:

– Adding an upstream port to a partition causes it to affect the L1 ASPM state of downstream ports

in the destination partition.

For example, if an upstream port in L0 is added to a switch partition, then an exit from L1 is
initiated on all downstream ports in L1 ASPM.

– Adding an upstream port to a partition where all downstream ports are in L1 ASPM causes the

upstream port to enter the L1 ASPM state per the rules described in section Active State Power
Management on page 7-11.

PME Synchronization

If PME synchronization is in progress then the port being added must be a downstream port. This down­stream port being added does not participate in PME synchronization in progress within the destination
partition. The added downstream port participates in PME synchronization initiated within the destination
partition after the addition of the port has been completed.

Bus Locking

If the destination partition is bus locked, then the port being added must be a downstream port. By the
nature that the port is added to a locked partition, it cannot be one of the two locked ports. Adding a down­stream port to a partition that is locked causes it to adopt the locked behavior associated with the destina­tion partition. The port blocks all requests from being propagated to either of the locked ports, except for
requests which map to non-VC0 on the egress port.

PES64H16G2 User Manual 6 - 13 April 5, 2013

IDT Switch Partitions

Notes

INTx Interrupt Signaling

Adding an upstream port to a partition causes the upstream port to adopt the aggregated interrupt state
of the downstream ports associated with the destination partition. This may result in the generation of
Assert_INTx and Deassert_INTx messages if the new aggregated state is different from that previously
reported to the root. Adding a downstream port to a partition causes the partition’s upstream port to aggre­gate the interrupt state of the added downstream port. This may result in the generation of Assert_INTx and
Deassert_INTx messages if the new aggregated state is different from that previously reported to the root.

Mode Change Effect on Port

A port mode change triggers the execution of the operation dictated by the OMA field value described in
section No Action Mode Change Behavior on page 6-14 through section Hot Reset Mode Change Behavior
on page 6-15. The following operations are common to all port mode changes.

Buffering

TLPs associated with the port remain queued in the ingress buffers, replay buffers, and egress buffers.

PME Synchronization

Any PME synchronization state associated with the port is reset. If the port has completed PME
synchronization, then the LTSSM transitions to the Detect state and then to the LTSSM state, if any, speci­fied by the OMA field value.

Bus Locking

Any bus locking state associated with the port is reset.

Port state or context

Any port state or context associated with the previous mode of operation (e.g., downstream switch port)
that is not relevant to the new mode of operation (e.g., upstream switch port) is cleared. All other state is
preserved.

No Action Mode Change Behavior

Modifying the operating mode of a port when the OMA field is set to no action has the following behavior
in addition to that specified by the common operating mode change behavior. The state of all registers
associated with the port are preserved (i.e., they are not reset). Status fields may change as appropriate.

If the link is up, it remains up in operating in the same LTSSM mode (i.e., upstream or downstream). If
the link is down and the new operating mode is an operational mode, then the link begins link training from
the Detect state in the specified LTSSM mode (i.e., upstream or downstream).

Reset Mode Change Behavior

Modifying the operating mode of a port when the OMA field is set to port reset has the following behavior
in addition to that specified by the common operating mode change behavior.

– All state associated with the port is reset. Registers associated with the port are reset to their initial

value except those designated SWSticky. SWSticky register and field contents are preserved.

– The port reset output is asserted (i.e., PxRSTN and/or the corresponding HPxRSTN signal if

mapped to that port).

– The port remains in a reset state for 250 µS (i.e., two times T

). The port reset output remains

PERST

asserted while in the reset state. During this time the LTSSM remains in the Detect state.

– The port reset output is negated (i.e., PxRSTN and/or the corresponding HPxRSTN signal if

mapped to that port).

– Following an exit from the reset state, if the new operating mode is an operational mode, then the

link begins link training from the Detect state in the specified LTSSM mode (i.e., upstream or
downstream).

PES64H16G2 User Manual 6 - 14 April 5, 2013

IDT Switch Partitions

Notes

Hot Reset Mode Change Behavior

Modifying the operating mode of a port when the OMA field is set to hot reset has the following behavior
in addition to that specified by the common operating mode change behavior.

– All registers associated with the port are reset to their initial value except those designated Sticky

and SWSticky. Sticky and SWSticky register and field contents are preserved.

– If the previous operating mode is set to downstream switch port and the new operating mode is

also set to downstream switch port, then the LTSSM is directed to the hot reset state. Otherwise,
the LTSSM transitions to the Detect state.

Examples:

If the previous operating mode was upstream switch port and the new operating mode is upstream
switch port or unattached, the LTSSM transitions to the Detect state and does not transition to the
hot reset state.

If the previous operating mode was downstream switch port and the new operating mode is dis­abled, then the LTSSM does not transition to the hot reset.

If the previous operating mode was disabled and the new operating mode is downstream switch
port, then the LTSSM transitions to L0 through the Detect state and does not transition to the hot
reset state.

Note that the Detect state on the link causes a hot reset on the downstream link partner.

In the above examples, if the LTSSM is unable to reach the L0 or hot reset states (e.g., due to an unpop­ulated slot), then the LTSSM transitions to the Detect state. The port remains in a reset state for 250 µs
(i.e., two times T

tional mode, then the link begins training from the Detect state in the specified LTSSM mode (i.e., upstream
or downstream).

). Following an exit from the hot reset state, if the new operating mode is an opera-

PERST

Partition and Port Configuration

The configuration of partition states and port modes may be done statically or dynamically as described
below.

Static Reconfiguration

Static configuration requires a switch fundamental reset and is nothing more than configuration
performed either through modification of the sampled boot configuration vector or initialization performed
via serial EEPROM or SMBus during the fundamental reset sequence. Static configuration of partitions and
ports via the serial EEPROM is subject to the restrictions described in section Partition State Change via
EEPROM Loading on page 6-3 and section Port Operating Mode Change via EEPROM Loading on page 6-

8.

The following is a sample EEPROM sequence to configure two partitions. Partition 0 has one upstream
port (Port 0) and 2 downstream ports (Ports 1 to 2). Partition 1 has one upstream port (Port 3) and 2 down­stream ports (Ports 4 and 5). Other ports are disabled. The sequence assumes the SWMODE signal in the
boot vector is set to 0xD (i.e., Multi-partition with serial EEPROM initialization). In this mode, all partitions
are initially disabled and all ports are initially unattached (see section Switch Mode Dependent Initialization
on page 5-6).

1. Set the following timer registers to a value of 0x0.

– Side Effect Delay Timer (SEDELAY register)
– Port Operating Mode Change Drain Delay Timer (POMCDELAY register)
– Reset Drain Delay Timer (RDRAINDELAY register)
– Upstream Secondary Bus Reset Delay (USSBRDELAY register)

2. Change the state of partition 0 to ‘Active’ by setting state field in the SWPART0CTL register.

3. Change the state of partition 1 to ‘Active’ by setting the state field in the SWPART1CTL register.

4. Add downstream ports to the partitions using the following sequence for each port.

PES64H16G2 User Manual 6 - 15 April 5, 2013

IDT Switch Partitions

Notes

– Change the port operating mode by setting the following fields in the SWPORTxCTL register. This

causes the port to be added to the selected partition.

• MODE field to ‘Downstream switch port’

• PART field to the appropriate partition (e.g., 0 or 1)

• OMA field to ‘no action’.

– Do a full link retrain on the port by setting the FLRET bit in the port’s PHYLSTATE0 register. This

will cause the port’s link to retrain from the Detect state.

5. Add the upstream port to each partition using the following sequence for each port.
– Change the port operating mode by setting the following fields in the SWPORTxCTL register. This

causes the port to be added to the selected partition.

• MODE field to ‘Upstream switch port’

• PART field to the appropriate partition (e.g., 0 or 1)

• OMA field to ‘no action’.

6. Disable the unused ports by using the following sequence for each port.
– Change the port operating mode by setting the following fields in the SWPORTxCTL register. This

causes the port to be disabled.

• MODE field to ‘Disabled’

• OMA field to ‘no action’.

• PART field to any value (this field is irrelevant for this port operating mode change).

7. Set the PCI Express capabilities and extended capabilities list for the upstream ports. This is done

by configuring the Next Pointer (NXTPTR) field appropriately in the capability header register of the
capabilities that form the two lists. Refer to section Capability Structures on page 15-3 for details.
This step is not required for the downstream ports.

– Specifically, the following register fields must be set appropriately.

• NXTPTR field in the PCI Power Management Capabilities (PMCAP) register

• NXTPTR field in the PCI Express VC Extended Capability Header (PCIEVCCAP) register

8. If the application requires partition reset control via the PARTxPERSTN input signal, enable the

appropriate GPIO alternate function as described in Chapter 12.

9. Set the following timer registers to their default values.

– Side Effect Delay Timer (SEDELAY register)
– Port Operating Mode Change Drain Delay Timer (POMCDELAY register)
– Reset Drain Delay Timer (RDRAINDELAY register)
– Upstream Secondary Bus Reset Delay (USSBRDELAY register)

The above sequence must complete before the PCI Express hierarchy is enumerated (i.e., in less than 1
second from the de-assertion of the switch fundamental reset signal (PERSTNT)). Refer to section Switch
Fundamental Reset on page 5-3 for details regarding switch fundamental reset timings.

Dynamic Reconfiguration

Dynamic reconfiguration refers to the reconfiguration of the switch partitions after the switch funda­mental reset sequence completes (i.e., run-time reconfiguration). Possible partition reconfigurations are list
below:

– A downstream port is added or removed from a partition
– An upstream port is added or removed from a partition
– The operating mode of the upstream port is modified.

Dynamic partition reconfiguration is subject to the restrictions described in section Partition State
Change via Other Methods on page 6-4 and section Port Operating Mode Change via Other Methods on
page 6-8. Partition reconfiguration may be initiated by software through modification of the operating mode
of a port.

PES64H16G2 User Manual 6 - 16 April 5, 2013

IDT Switch Partitions

Notes

A system may require software notification when a partition reconfiguration occurs. If the reconfiguration
results in the addition, removal, or change in operating mode of the upstream port associated with the parti­tion, then the system may be notified of the reconfiguration by a link down event detected by the component
upstream of the partition (i.e., the root or switch downstream port). This form of notification requires that the
OMA field in the SWPORTxCTL register be set to reset.

Dynamic Partition Reconfiguration Latency

The amount of time that the switch takes to do a partition reconfiguration depends on the reconfiguration
actions. A partition reconfiguration action may involve partition state changes and/or port operating mode
changes.

The latency to perform port operating mode changes is described in section Port Operating Mode
Change on page 6-7. This latency defaults to 2 ms or 2.5 ms, depending on the port’s operating mode
change action (OMA).

The latency to perform partition state changes is described in section Partition State Change Latency on
page 6-3. This latency is typically a few cycles, unless the partition state change has the side-effect of
causing port operating mode changes. For the latter case, the delay is as described in the previous bullet.

PES64H16G2 User Manual 6 - 17 April 5, 2013

IDT Switch Partitions

Notes

PES64H16G2 User Manual 6 - 18 April 5, 2013

Notes

®

Chapter 7

Link Operation

Introduction

Link operation in the PES64H16G2 adheres to the PCI Express 2.0 Base Specification, supporting
speeds of 2.5 GT/s and 5.0 GT/s. The PES64H16G2 contains sixteen x4 ports which may be merged in
pairs to form x8 ports. The default link width of each port is x4 and the SerDes lanes are statically assigned
to a port.

Each port supports upstream and downstream link behavior. The behavior is determined dynamically by
the port’s operating state (i.e., upstream switch port, downstream switch port). Refer to Chapter 6, Switch
Partitions, for further details. A full link retrain is defined as retraining of a link that transitions through the

Detect LTSSM

Polarity Inversion

Each port of the PES64H16G2 supports automatic polarity inversion as required by the PCIe specifica­tion. Polarity inversion is a function of the receiver and not the transmitter. The transmitter never inverts its
data. During link training, the receiver examines symbols 6 through 15 of the TS1 and TS2 ordered sets for
inversion of the PExRP[n] and PExRN[n] signals. If an inversion is detected, then logic for the receiving
lane automatically inverts received data. Polarity inversion is a lane and not a link function. Therefore, it is
possible for some lanes of link to be inverted and for others not to be inverted.

1

state.

Lane Reversal

The PCIe specification describes an optional lane reversal feature. The PES64H16G2 supports the
automatic lane reversal feature outlined in the PCIe specification. The operation of lane reversal is depen­dent on the maximum link width determined dynamically by the PHY. The maximum link width is the
minimum of:

– The value of the MAXLNKWDTH field in the port’s PCI Express Link Capabilities (PCIELCAP)

register.

– The number of consecutive lanes detected during the Detect state on which valid training sets are

received.

Lane reversal mapping for the various non-trivial maximum link width configurations supported by the
PES64H16G2 is illustrated in Figures 7.1 and 7.5.

1.

The term ‘LTSSM’ refers to a port’s Link Training and Status State Machine in the Physical Layer.

PES64H16G2 User Manual 7 - 1 April 5, 2013

IDT Link Operation

Notes

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0
lane 1
lane 2
lane 3

(a) x4 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 3
lane 2
lane 1
lane 0

(b) x4 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0
lane 1

(a) x2 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 1
lane 0

(b) x2 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0

(a) x1 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0

(b) x1 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0
lane 1

(a) x2 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 1
lane 0

(b) x2 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0

(c) x1 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]

PES64H16G2

lane 0

(d) x1 Port with lane reversal

Figure 7.1 Unmerged Port Lane Reversal for Maximum Link Width of x4

Figure 7.2 Unmerged Port Lane Reversal for Maximum Link Width of x2

PES64H16G2 User Manual 7 - 2 April 5, 2013

IDT Link Operation

Notes

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0
lane 1

(a) x2 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 1
lane 0

(b) x2 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0

(c) x1 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0

(d) x1 Port with lane reversal

Figure 7.3 Merged Port Lane Reversal for Maximum Link Width of x2)

PES64H16G2 User Manual 7 - 3 April 5, 2013

IDT Link Operation

Notes

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0
lane 1
lane 2
lane 3

(a) x4 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 3
lane 2
lane 1
lane 0

(b) x4 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0
lane 1

(c) x2 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 1
lane 0

(d) x2 Port with lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0

(e) x1 Port without lane reversal

PExRP[0]
PExRP[1]
PExRP[2]
PExRP[3]
PExRP[4]
PExRP[5]
PExRP[6]
PExRP[7]

PES64H16G2

lane 0

(f) x1 Port with lane reversal

PES64H16G2 User Manual 7 - 4 April 5, 2013

Figure 7.4 Merged Port Lane Reversal for Maximum Link Width of x4