Page 1
HyperTransport Technology Consortium
HyperTransport™ I/O
Link Specification
Revision 1.05c
Document #
HTC2002104-0005-0005
8/5/2003
Copyright 2001, 2002, 2003 HyperTransport Technology Consortium
Page 2
HyperTransport Technology Consortium
The HyperTransport Technology Consortium disclaims all warranties and liability for the use of this
document and the information contained herein and assumes no responsibility for any errors that may
appear in this document, nor does the HyperTransport Technology Consortium make a commitment to
update the information contained herein.
DISCLAMER
This document is provided “AS IS” with no warranties whatsoever, including any warranty of
merchantability, non-infringement, fitness for any particular purpose, or any warranty otherwise arising
out of any proposal, specification or sample. The HyperTransport Technology Consortium disclaims all
liability for infringement of property rights relating to the use of information in this document. No license,
express, implied, by estoppels, or otherwise, to any intellectual property rights is granted herein.
TRADEMARKS
HyperTransport is a licensed trademark of the HyperTransport Technology Consortium.
Other product names used in this publication are for identification purposes only and may be trademarks of
their respective companies.
Copyright 2001, 2002, 2003 HyperTransport Technology Consortium
Page 3
HyperTransport Technology Consortium
About HyperTransport™ Technology
HyperTransport technology is a high-speed, high-performance, point-to-point link for integrated circuits, and is
designed to meet the bandwidth needs of tomorrow’s computing and communications platforms. HyperTransport
technology helps reduce the number of buses while providing a high-performance link for PCs, workstations, and
servers, as well as numerous embedded applications and highly scalable multiprocessing systems. It is designed to
allow chips inside of PCs, networking and communications devices to communicate with each other up to 48 times
faster than with some existing bus technologies.
About the HyperTransport Technology Consortium
The HyperTransport Technology Consortium is a nonprofit corporation managed by its members. The consortium
promotes the common business interests of providers to the networking, telecommunications, computer and highperformance embedded application through the conduct of a forum for the future development and adoption of the
HyperTransport specification.
AMD, Apple Computers, Broadcom, Cisco Systems, NVIDIA, PMC-Sierra, SGI, SiPackets, Sun Microsystems, and
Transmeta are the charter members that comprise the Executive Committee of the HyperTransport Technology
Consortium.
Companies interested in the HyperTransport specification are invited to join the consortium. Members of the
consortium pay annual dues and receive a royalty-free license to HyperTransport IP, gain access to technical
documentation and may attend consortium meetings and events. To become a member, visit the consortium Web site
at www.hypertransport.org. Please review the Bylaws of the HyperTransport Technology Consortium, and complete
the online membership application.
The HyperTransport Consortium, HyperTransport Technology and combinations thereof are trademarks of
HyperTransport Consortium.
Copyright 2001, 2002, 2003 HyperTransport Technology Consortium
Page 4
HyperTransport Technology Consortium
REVISION CHANGE SECTION DATE
1.03 Initial release All 10/10/01
1.04
Fixed errata and made clarifications All
Fully defined open-drain signal behavior 2, 14, 17.5
Document Isochronous VC limitations 4.4.3, 4.4.4,
7.5.4.9, D.1
Allow RMW with count other than 1 or 3 4.4.5
Relaxed response UnitID for rejected packets 4.9.3
Updated upstream configuration requirements 4.9.4, F
Swapped Interrupt and Address chapters (5 and 9) 5, 9
Added new x86 reserved address range 5
Tighten Config and I/O space access requirements 5, B
Highlighted required registers 7.3-7.5
Expanded description of Address registers 7.3.5, 7.4.6
ISA and VGA enable bits required and documented 7.4.9.3
Require a revision ID in every function 7.5
Extend HyperTransport Capability Type field to 5 bits 7.5.3.1, 7.7.1
Documented behavior of multiple nonprefetchable
memory range register extensions
Added Isochronous, NonCoherent, and Compat bits to
Address Remapping Block
Created Revision ID Capability 7.8
Documented reset data corruption case 10.1
Documented behavior when all UnitIDs are consumed 12.3
Combined Appendices B and C B
Relaxed PCI ordering, added HyperTransport to PCI
command mapping
Added PCI-X ordering rules and command mapping B.4
Added Deadlock Appendix C C
Document legacy interrupt boot requirements F.1
Document legacy PIC multiple ExtInt requirements F.1.3
Document delay from STOP_GRANT to LDTSTOP# F.2
Document A20M ordering requirements F.2.1.1
Updated LDTREQ# requirements F.2.4
Updated differential signal input edge rate requirements 17.10
7.5.13
7.7
B.2
5/30/02
Copyright 2001, 2002, 2003 HyperTransport Technology Consortium
Page 5
HyperTransport Technology Consortium
REVISION CHANGE SECTION DATE
1.05
1.05c Incorporated all errata to date 3.2.3, 4.4.5, 4.8, 4.9.5.2, 5, 6, 6.1, 7.3.1.4,
Added 64-bit addressing 3.2.1, 4.4.6, 5,
7.3.5, 7.4.6,
7.5.4.12, 7.5.8,
7.5.10.5, 10.1.5
Changed Coherent bit from Ignored to Reserved and set 3.2.1.5, 4.4.1
Added Data Error to Responses and Posted Writes,
Changed Error and NXA bits in Responses to Error0/1
Added Isoc bit to Flush and Fence 4.4.3, 4.4.4, D.1
Added UnitID Clumping 4.2, 4.5, 4.6,
Various clarifications 3.2.1, 4.7, 4.9,
Require peer-to-peer reflection of Atomic RMW 4.4.5
Added Extended Configuration Space and Device
Messaging
Restricted ordering within the chain 6, 6.1
Added Interrupt Disable and Status bits 7.3.1.6, 7.3.2.1
Removed VGA Palette Snoop 7.4.1
Added Upstream Configuration Enable bit 7.5.10.8
Added 64-bit Address Remapping 7.8
Added INTA/B/C/D Virtual Wires 8.1, 8.4
Added No Snoop to HyperTransport-to-PCI-X Mapping B.4.2
Added x86 Thermal Management Messages F.2.1
Relaxed x86 SMI and SMIACK Requirements F.2.5
Added Switch Appendix 7.5.3.1, I
Combined First Two Electrical Chapters 14
Corrected Typos 1/22/03
7.3.2.3, 7.3.2.8, 7.4.1, 7.4.9, 7.5.3.1,
7.5.4.1, 7.5.4.6, 7.5.4.12, 7.5.5, 7.5.6,
7.5.7, 7.5.8.3, 7.5.10.6, 7.5.12, 7.5.13,
7.5.14, 7.10.2, 7.12, 8.1, 8.3, 8.5, 10.1.1,
10.1.3, 10.1.5, 10.2.2, 10.2.4, 12.2, B.2.2,
B.4.2, B.5, F.2.1, I.3.3.5, J, 17.7, L.9
4.4.1, 4.5, 7.3,
7.4, 10.2.1,
B.2.2, B.4.2
4.9, 6.4, 7.5.10,
7.10, 12.3
5, 6.1, 7.1, 8.3,
12.2, B.2.2,
B.4.2, C.1.2
4.4.1, 4.9, 5,
7.1, 7.2, 7.5.15,
7.11, 13, B.4.2,
F
11/22/02
8/5/03
Copyright 2001, 2002, 2003 HyperTransport Technology Consortium
Page 6
HyperTransport Technology Consortium
Copyright 2001, 2002, 2003 HyperTransport Technology Consortium
Page 7
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Contents
List of Figures.........................................................................................................................18
List of Tables...........................................................................................................................20
Preface.....................................................................................................................................25
This Document.....................................................................................................................25
Organization.........................................................................................................................25
Section 1 – Protocol...........................................................................25
1 Overview..................................................................................................................25
1.1 Terminology..............................................................................................................26
1.2 HyperTransport™ Technology in x86 Platforms........................................................27
2 Signaling..................................................................................................................28
3 Packet Definition.....................................................................................................30
3.1 Use of the CTL Signal...............................................................................................30
3.2 Packet Structure.........................................................................................................31
3.2.1 Control Packets..................................................................................................31
3.2.2 Data Packet........................................................................................................36
3.2.3 Note about VC Stalls due to Command-Data Separation....................................38
4 Fabric Operation.....................................................................................................39
4.1 Topology...................................................................................................................39
4.1.1 Double-Hosted Chains.......................................................................................41
4.1.2 HyperTransport™ Technology Signals PWROK and RESET#..........................42
4.2 Transactions and UnitID............................................................................................43
4.3 Link Synchronization................................................................................................44
4.4 Requests....................................................................................................................44
4.4.1 Sized Reads and Writes......................................................................................44
4.4.2 Broadcast Message............................................................................................47
4.4.3 Flush..................................................................................................................48
Contents
7
Page 8
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4.4.4 Fence.................................................................................................................49
4.4.5 Atomic Read-Modify-Write...............................................................................50
4.4.6 Address Extension.............................................................................................51
4.5 Responses..................................................................................................................52
4.5.1 Read Response (RdResponse)............................................................................52
4.5.2 Target Done (TgtDone)......................................................................................53
4.6 I/O Streams...............................................................................................................54
4.6.1 UnitID Clumping...............................................................................................55
4.7 Virtual Channels........................................................................................................56
4.8 Flow Control.............................................................................................................57
4.9 Routing.....................................................................................................................60
4.9.1 Acceptance........................................................................................................60
4.9.2 Forwarding........................................................................................................61
4.9.3 Rejection...........................................................................................................61
4.9.4 Host Bridges......................................................................................................62
4.9.5 Fairness and Forward Progress...........................................................................63
5 Addressing...............................................................................................................66
6 I/O Ordering...........................................................................................................68
6.1 Upstream I/O Ordering..............................................................................................69
6.2 Host Ordering Requirements.....................................................................................70
6.2.1 Host Responses to Nonposted Requests.............................................................72
6.3 Downstream I/O Ordering.........................................................................................72
6.4 Ordering in Sharing Double-Hosted Chains...............................................................73
7 Configuration Accesses...........................................................................................74
7.1 Configuration Cycle Types........................................................................................74
7.2 Configuration Space Mapping...................................................................................76
7.2.1 Function and Register Numbering......................................................................76
7.2.2 Device Numbering.............................................................................................76
7.2.3 Bus Numbering..................................................................................................77
7.2.4 Software View of Extended Configuration Space...............................................77
8
Contents
Page 9
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.3 HyperTransport™ Technology Device Header..........................................................77
7.3.1 Command Register: Offset 04h..........................................................................78
7.3.2 Status Register: Offset 06h.................................................................................79
7.3.3 Cache Line Size Register: Offset 0Ch: R/O........................................................81
7.3.4 Latency Timer Register: Offset 0Dh: R/O..........................................................81
7.3.5 Base Address Registers (BARs): Offsets 10-24h: R/W: Warm Reset..................81
7.3.6 CardBus CIS Pointer: Offset 28h: R/O...............................................................82
7.3.7 Capabilities Pointer: Offset 34h: R/O.................................................................82
7.3.8 Interrupt Line Register: Offset 3Ch: R/W: Warm Reset.....................................82
7.3.9 Interrupt Pin Register: Offset 3Dh: R/O............................................................82
7.3.10 Min_Gnt, and Max_Lat Registers: Offsets 3E and 3Fh: R/O..............................82
7.4 HyperTransport™ Technology Bridge Headers.........................................................83
7.4.1 Command Register: Offset 04h..........................................................................84
7.4.2 Status, Primary Latency Timer, Base Address, Interrupt Pin, and Interrupt
Line Registers....................................................................................................85
7.4.3 Cache Line Size: Offset 0Ch: R/O......................................................................85
7.4.4 Secondary Latency Timer Register: Offset 1Bh: R/O.........................................85
7.4.5 Secondary Status Register: Offset 1Eh...............................................................85
7.4.6 Memory and Prefetchable Memory Base and Limit Registers: Offsets 20-2Ch:
R/W: Warm Reset..............................................................................................86
7.4.7 I/O Base and Limit Registers: Offsets 1C, 1D, 30, and 32h: R/W: Warm Reset..87
7.4.8 Capabilities Pointer Register: Offset 34h: R/O...................................................88
7.4.9 Bridge Control Register: Offset 3Eh..................................................................88
7.5 Capability Registers...................................................................................................90
7.5.1 Capability ID: Offset 00h: R/O..........................................................................92
7.5.2 Capabilities Pointer: Offset 01h: R/O.................................................................92
7.5.3 Command Register: Offset 02h..........................................................................92
7.5.4 Link Control Register: Offsets 04h and 08h.......................................................96
7.5.5 Link Configuration Register: Offsets 06h and 0Ah...........................................100
7.5.6 Revision ID Register: Offset 08h or 0Ch: R/O.................................................102
Contents
9
Page 10
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.5.7 Link Frequency Register: Offsets 09h or 0Dh and 11h (Bits 3:0): R/W: Cold
Reset to 0.........................................................................................................103
7.5.8 Link Error Register: Offsets 09h or 0Dh and 11h (Bits 7:4)..............................105
7.5.9 Link Frequency Capability Register: Offsets 0Ah or 0Eh and 12h: R/O...........106
7.5.10 Feature Capability Register: Offset 0Ch or 10h................................................106
7.5.11 Enumeration Scratchpad Register: Offset 10h or 14h: R/W: Cold Reset to 0....108
7.5.12 Error Handling Register: Offset 12h or 16h......................................................108
7.5.13 Memory Base Upper 8 Bits: Offset 14h or 18h: R/W: Warm Reset to 0...........111
7.5.14 Memory Limit Upper 8 Bits: Offset 15h or 19h: R/W: Warm Reset to 0..........111
7.5.15 Bus Number: Offset 1Ah: R/O: Warm Reset to 0.............................................111
7.6 Interrupt Discovery and Configuration Capability Block.........................................111
7.6.1 Last Interrupt: Index 01h: R/O.........................................................................112
7.6.2 Interrupt Definition Registers: Index 10h and Higher: Warm Reset..................112
7.7 40 bit Address Remapping Capability Block............................................................113
7.7.1 Capability Header............................................................................................114
7.7.2 Secondary Bus Window Control Registers: R/W: Warm Reset to 0..................115
7.7.3 Secondary Bus Window Base Registers: R/W: Warm Reset to 0......................115
7.7.4 DMA Window Control Register: R/W: Warm Reset to 0.................................115
7.7.5 DMA Primary Base Register: R/W: Warm Reset to 0......................................116
7.7.6 DMA Secondary Base and Limit Registers: R/W: Warm Reset to 0.................116
7.8 64 bit Address Remapping Capability......................................................................116
7.8.1 Capability Header............................................................................................116
7.8.2 Index and Data Registers.................................................................................117
7.9 Revision ID Capability............................................................................................118
7.10 UnitID Clumping Capability....................................................................................119
7.10.1 UnitID Clumping Support (Offset 4h): R/O.....................................................119
7.10.2 UnitID Clumping Enable (Offset 8h): R/W: Warm Reset to 0..........................120
7.11 Extended Configuration Space Access Capability....................................................120
7.12 MSI Mapping Capability.........................................................................................121
8 System Management.............................................................................................122
8.1 Command Mapping.................................................................................................122
10
Contents
Page 11
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
8.2 Special Cycles.........................................................................................................124
8.3 Disconnecting and Reconnecting HyperTransport™ Links......................................124
8.4 INTx Virtual Wire Messages...................................................................................126
8.5 INT_Pending Message............................................................................................128
9 Interrupts..............................................................................................................129
9.1 Interrupt Requests....................................................................................................129
9.2 End of Interrupt (EOI).............................................................................................130
10 Error Handling.....................................................................................................132
10.1 Error Conditions......................................................................................................132
10.1.1 Transmission Errors: 8-Bit, 16-Bit, and 32-Bit Links.......................................132
10.1.2 Transmission Errors: 2-Bit and 4-Bit Links......................................................134
10.1.3 Protocol Errors.................................................................................................135
10.1.4 Receive Buffer Overflow Errors.......................................................................136
10.1.5 End of Chain Errors.........................................................................................136
10.1.6 Chain Down Errors..........................................................................................137
10.1.7 Response Errors...............................................................................................137
10.2 Error Reporting.......................................................................................................137
10.2.1 Error Responses...............................................................................................138
10.2.2 Data Error in Posted Requests..........................................................................138
10.2.3 Error Interrupts................................................................................................139
10.2.4 Sync Flooding..................................................................................................139
10.2.5 Error Routing CSRs.........................................................................................141
11 Clocking.................................................................................................................142
11.1 Clocking Mode Definitions......................................................................................142
11.2 Receive FIFO..........................................................................................................143
11.3 Async Mode Implementation Example....................................................................144
11.4 Link Frequency Initialization and Selection.............................................................144
12 Reset and Initialization.........................................................................................145
12.1 Definition of Reset..................................................................................................145
12.2 System Powerup, Reset, and Low-Level Link Initialization.....................................145
Contents
11
Page 12
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
12.3 I/O Chain Initialization............................................................................................150
12.3.1 Finding the Firmware ROM.............................................................................152
12.4 Link Width Initialization.........................................................................................153
12.5 Link Frequency Initialization...................................................................................153
13 Device Messaging..................................................................................................154
Protocol Appendices.............................................................................................................157
A Address Remapping Capability............................................................................157
A.1 I/O Space Aliasing...................................................................................................157
A.2 Memory Space Mapping..........................................................................................158
A.3 DMA Window Remapping......................................................................................158
B Ordering Rules and Mapping of Other I/O Protocols.........................................160
B.1 Processors...............................................................................................................160
B.2 PCI..........................................................................................................................160
B.2.1 Ordering..........................................................................................................160
B.2.2 Command Mapping.........................................................................................162
B.3 AGP........................................................................................................................163
B.3.1 Ordering..........................................................................................................163
B.3.2 Command Mapping.........................................................................................164
B.4 PCI-X......................................................................................................................166
B.4.1 Ordering..........................................................................................................166
B.4.2 Command Mapping.........................................................................................167
B.5 Message Signaled Interrupts....................................................................................172
C Summary of Deadlock Scenarios..........................................................................173
C.1 Reflection/Forwarding Loops..................................................................................173
C.1.1 Problem...........................................................................................................173
C.1.2 Solution 1: Avoidance......................................................................................174
C.1.3 Solution 2: Switching Channels.......................................................................175
C.1.4 Solution 3: Modifying Requests.......................................................................175
C.2 Packet Issue and Acceptance...................................................................................176
C.2.1 Control/Data Buffer Dependency.....................................................................176
12
Contents
Page 13
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
C.2.2 Response Buffer Dependency..........................................................................176
C.2.3 Posted Request Acceptance..............................................................................177
C.3 Legacy Buses..........................................................................................................177
C.3.1 Host/DMA Deadlock.......................................................................................177
C.3.2 Peer-to-Peer Deadlock.....................................................................................177
C.4 System Management...............................................................................................178
C.5 PCI Requirements....................................................................................................178
C.5.1 Posted Requests Must Pass Nonposted Requests..............................................178
C.5.2 Responses Must Pass Nonposted Requests.......................................................180
C.5.3 Posted Requests Must Pass Responses.............................................................180
D Considerations for Isochronous Traffic...............................................................181
D.1 Isochronous Flow Control Mode (Optional).............................................................181
D.2 Normal Flow Control Mode.....................................................................................182
E Southbridges and Compatibility Buses................................................................183
E.1 ISA/LPC Deadlock Case.........................................................................................183
E.2 ISA/LPC Write Post Flushing..................................................................................183
E.3 Subtractive Decoding..............................................................................................184
E.3.1 Subtractive Decode in the General Case...........................................................184
E.3.2 Subtractive Decode in x86 Legacy Systems.....................................................184
E.3.3 Subtractive Decode in the Simplest Case.........................................................185
E.3.4 Subtractive Decode Behind a PCI Bridge.........................................................185
E.4 VGA Palette Snooping............................................................................................186
F Required Behavior in x86 Platforms....................................................................187
F.1 Interrupts.................................................................................................................187
F.1.1 Interrupt Request.............................................................................................188
F.1.2 Standard EOI...................................................................................................191
F.1.3 Legacy PIC (8259) Interrupt Request, Acknowledge, and EOI.........................191
F.1.4 Alternate Interrupt Discovery and Configuration Mechanism...........................192
F.2 System Management...............................................................................................193
F.2.1 Command Encoding........................................................................................196
Contents
13
Page 14
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
F.2.2 VID/FID Changes............................................................................................198
F.2.3 Throttling.........................................................................................................199
F.2.4 C3 System State Transitions and LDTREQ#....................................................199
F.2.5 SMI and STPCLK............................................................................................200
F.2.6 Default State of Virtual Wires..........................................................................201
F.3 Initialization Issues..................................................................................................201
F.4 AGP Bridge Issues..................................................................................................201
F.5 Configuration Space Access Mechanism.................................................................202
G CRC Testing Mode................................................................................................203
H Doubleword-Based Data Buffer Flow Control.....................................................204
I Switches.................................................................................................................206
I.1 Overview.................................................................................................................206
I.1.1 Definitions...........................................................................................................207
I.2 Operation................................................................................................................208
I.2.1 Ports....................................................................................................................208
I.2.2 Partitions.............................................................................................................209
I.2.3 Compatibility Accesses........................................................................................210
I.2.4 Configuration Accesses.......................................................................................210
I.2.5 Packet and Event Routing....................................................................................211
I.2.6 LDTSTOP# and LDTREQ#.................................................................................225
I.2.7 Error management...............................................................................................225
I.2.8 Cascading Switches.............................................................................................226
I.2.9 Topology and Ordering Considerations................................................................226
I.2.10 Hot Plug..........................................................................................................227
I.2.11 Virtual Tunnels................................................................................................227
I.2.12 Port Splitting....................................................................................................227
I.3 Switch Configuration...............................................................................................228
I.3.1 Bridge Headers....................................................................................................228
I.3.2 Interface Capability Blocks..................................................................................228
I.3.3 Switch Capability Block......................................................................................229
14
Contents
Page 15
HyperTransport Technology Consortium
I.4 Switch Requirements for x86 Systems.....................................................................235
J Quick Reference for x86 Systems.........................................................................237
HyperTransport™ I/O Link Specification Revision 1.05c
Section 2 – Electrical Interface..................................239
14 HyperTransport™ Link Overview.......................................................................239
15 Supply Characteristics..........................................................................................242
16 Power Requirements.............................................................................................243
17 Input/Output DC Voltage Characteristics...........................................................244
17.1 Impedance Requirements.........................................................................................244
17.2 DC Output Voltage Requirements...........................................................................245
17.2.1 ATE Test Environment....................................................................................245
17.2.2 Reference System Load...................................................................................245
17.2.3 Output Voltage Parameter Descriptions...........................................................246
17.3 DC Input Requirements...........................................................................................246
17.3.1 ATE Test Environment....................................................................................246
17.3.2 Input Parameter Descriptions...........................................................................247
17.4 Differential Signal DC Specifications......................................................................248
17.5 Single-Ended Signal AC/DC Specifications.............................................................249
17.6 Input/Output AC Voltage Characteristics.................................................................250
17.7 Impedance Requirements.........................................................................................250
17.8 AC Output Requirements.........................................................................................251
17.8.1 ATE Test Environment....................................................................................251
17.8.2 Reference System Load...................................................................................252
17.8.3 Output Parameter Descriptions.........................................................................253
17.9 AC Input Requirements...........................................................................................254
17.9.1 ATE Test Environment....................................................................................254
17.9.2 Input Parameter Descriptions...........................................................................254
17.10 Differential Signal AC Specifications......................................................................255
18 Link Transfer Timing Characteristics.................................................................256
18.1 Signal Groups..........................................................................................................256
18.2 Device Output Timing Characteristics.....................................................................258
Contents
15
Page 16
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
18.2.1 Differential Output Skew.................................................................................258
18.2.2 T
CADV (TCADValid
)..............................................................................................258
18.3 Device Input Timing Characteristics........................................................................260
18.3.1 Input Differential Skew....................................................................................260
18.3.2 TSU and THD.....................................................................................................261
18.4 Interconnect Timing Characteristics.........................................................................262
18.4.1 T
CADVRS/RH
.......................................................................................................262
18.5 Transfer Timing Characteristics...............................................................................263
18.6 Reconciling Transfer Timing and Link Frequency...................................................265
19 Phase Recovery Timing Characteristics...............................................................267
19.1 Receiver Modes of Operation..................................................................................267
19.1.1 Synchronous Operation....................................................................................267
19.1.2 Pseudo Synchronous Operation........................................................................267
19.1.3 Asynchronous Operation..................................................................................268
19.2 Phase Recovery Timing Variations..........................................................................268
19.2.1 Uncertainty When Initializing the Pointers.......................................................269
19.2.2 Other Factors Affecting FIFO Size and Read Pointer Separation......................269
19.3 Phase Recovery Timing Characteristics...................................................................270
19.4 Reconciling Phase Recovery Timing to Receiver FIFO Depth and Read Pointer
Initialization............................................................................................................272
19.4.1 Read Pointer Initialization................................................................................272
19.4.2 Minimum FIFO Depth.....................................................................................272
Electrical Interface Appendices............................................................................................273
K DC and AC Characteristics and Relationships....................................................273
K.1 DC Parameters........................................................................................................273
K.2 Relationships Between AC and DC Parameters.......................................................274
K.3 Relationships Between Output and Input Parameters...............................................274
L Detailed Transfer Timing Budget.........................................................................275
L.1 HyperTransport™ Link Transmitter........................................................................275
L.2 Differential Skew....................................................................................................276
L.3 Transmitter Clock Uncertainties..............................................................................277
16
Contents
Page 17
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
L.4 Transmitter PHY Uncertainties................................................................................278
L.5 Transmitter Package Skew.......................................................................................279
L.6 Receiver Package Skew...........................................................................................279
L.7 PCB Skew...............................................................................................................280
L.8 Receiver Setup and Hold times................................................................................280
L.9 Transfer Timing Example........................................................................................281
M Detailed Phase Recovery Timing Budget.............................................................282
M.1 System Reference Clock Uncertainties....................................................................282
M.2 Transmitter CLKOUT Uncertainties........................................................................283
M.2.1 Transmitter PLL Variations.............................................................................283
M.2.2 Transmitter and Link Transfer Variations.........................................................284
M.2.3 Transmitter Cross Byte Lane Variations...........................................................285
M.3 Receiver CLKIN Uncertainties................................................................................286
M.3.1 Receiver PLL Variations..................................................................................286
M.3.2 Receiver Transfer Variations............................................................................286
M.4 CADIN/CTLIN Sampling Error...............................................................................287
N Combining Voltage and Transfer Characteristics...............................................288
Contents
17
Page 18
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
List of Figures
Figure 1. HyperTransport™ I/O Link................................................................................26
Figure 2. Example Device Configurations.........................................................................40
Figure 3. Example Topologies..........................................................................................40
Figure 4. Clumping Configuration....................................................................................56
Figure 5. Example Data Buffer Sizing Calculation............................................................59
Figure 6. Receive FIFO..................................................................................................143
Figure 7. Sync Sequence Timing for Link Initialization..................................................148
Figure 8. Fragment Use for PCI-X Messaging................................................................156
Figure 9. I/O Space Aliasing...........................................................................................157
Figure 10. Memory Window Remapping..........................................................................159
Figure 11. Reflection Example..........................................................................................174
Figure 12. Example PCI System.......................................................................................179
Figure 13. Request/Response Deadlock Loop...................................................................180
Figure 14 External Model of a Switch..............................................................................206
Figure 15 Logical Model of a Switch...............................................................................207
Figure 16. HyperTransport™ Link Interconnect...............................................................240
Figure 17. DC Output Reference System Load.................................................................245
Figure 18. V
Figure 19. V
Figure 20. V
Figure 21. V
DC............................................................................................................246
OD
DC..........................................................................................................246
OCM
DC.............................................................................................................247
ID
DC...........................................................................................................247
ICM
Figure 22. AC Reference System Load for Links up to 800 MT/s.....................................252
Figure 23. AC Reference System Load for Links Above 800 MT/s...................................252
Figure 24. V
Figure 25. V
Figure 26. V
AC............................................................................................................253
OD
AC..........................................................................................................253
OCM
AC.............................................................................................................254
ID
Figure 27. V
Figure 28. T
Figure 29. T
18
AC...........................................................................................................254
ICM
..............................................................................................................258
ODIFF
...............................................................................................................259
CADV
List of Figures
Page 19
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Figure 30. T
...............................................................................................................260
IDIFF
Figure 31. TSU and THD.....................................................................................................261
Figure 32. T
CADVRS / TCADVRH
............................................................................................262
Figure 33. Reconciling Link Transfer Timing...................................................................265
Figure 34. Representative Transmitter and Receiver.........................................................275
Contents
19
Page 20
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
List of Tables
Table 1. Link Signals.......................................................................................................28
Table 2. Reset/Initialization Signals.................................................................................28
Table 3. Power Management Signals...............................................................................29
Table 4. Info Packet Format............................................................................................33
Table 5. Request Packet Format with Address.................................................................33
Table 6. Request Packet Format with Extended Address..................................................34
Table 7. Response Packet Format....................................................................................34
Table 8. Command Field Encoding for All Control Packets.............................................35
Table 9. Eight-Byte Data Packet Format..........................................................................37
Table 10. Sized Byte Write Data Packet Format................................................................38
Table 11. UnitID Field Usage............................................................................................43
Table 12. Sync Pattern Format...........................................................................................44
Table 13. Sized Read or Write Request Format.................................................................45
Table 14. Broadcast Message Format................................................................................47
Table 15. Flush Format......................................................................................................48
Table 16. Fence Format.....................................................................................................49
Table 17. Atomic Read-Modify-Write (RMW) Request Format........................................50
Table 18. Request Packet Format with Extended Address..................................................52
Table 19. Read Response (RdResponse) Packet Format.....................................................52
Table 20. Error bit encodings............................................................................................53
Table 21. Target Done (TgtDone) Format..........................................................................54
Table 22. NOP Packet Format...........................................................................................59
Table 23. HyperTransport™ Technology Address Map.....................................................66
Table 24. Packet Ordering Rules.......................................................................................70
Table 25. Host Ordering Rules..........................................................................................71
Table 26. HyperTransport™ Technology Type 0 Address Format.....................................75
Table 27. HyperTransport™ Technology Type 1 Address Format.....................................75
Table 28. Extended HyperTransport™ Technology Type 0 Address Format......................75
20
List of Tables
Page 21
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 29. Extended HyperTransport™ Technology Type 1 Address Format......................76
Table 30. Extended HyperTransport™ Technology Software Address Format...................77
Table 31. HyperTransport™ Technology Device Header Format.......................................78
Table 32. Memory Space BAR Format..............................................................................81
Table 33. I/O Space BAR Format......................................................................................82
Table 34. HyperTransport™ Technology Bridge Header Format.......................................84
Table 35. Memory and Prefetchable Memory Base and Limit Register Format..................86
Table 36. Prefetchable Memory Upper Register Format....................................................86
Table 37. I/O Base and Limit Register Format...................................................................87
Table 38. I/O Base and Limit Upper Register Format........................................................87
Table 39. Slave/Primary Interface Block Format...............................................................91
Table 40. Host/Secondary Interface Block Format.............................................................91
Table 41. Command Register Format................................................................................92
Table 42. Capability Type Encoding..................................................................................93
Table 43. Packet Forwarding Behavior..............................................................................95
Table 44. Link Control Register........................................................................................96
Table 45. LDTSTOP# Tristate Enable Bit Encoding..........................................................99
Table 46. Link Configuration Register Definition............................................................100
Table 47. Max Link Width In Bit Field Encoding............................................................101
Table 48. Revision ID Register Definition.......................................................................103
Table 49. Link Frequency Bit Field Encoding.................................................................104
Table 50. Link Error Register Definition.........................................................................105
Table 51 Feature Capability Register Layout..................................................................106
Table 52. Error Handling Register Definition..................................................................108
Table 53. Interrupt Discovery and Configuration Capability Block Definition.................112
Table 54. Interrupt Definition Registers...........................................................................112
Table 55. 40 bit Address Remapping Capability Block Definition...................................114
Table 56. Secondary Bus Window Control Register Definition........................................115
Table 57. DMA Window Control Register Definition......................................................115
Table 58. 64 bit Address Remapping Capability Block Definition...................................116
Contents
21
Page 22
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 59. 64-bit Address Remap Indexed Registers.........................................................117
Table 60. Revision ID Capability Block Definition..........................................................118
Table 61. Clumping Capability Block Definition.............................................................119
Table 62. Extended Configuration Space Access Capability Block Definition.................120
Table 63. MSI Mapping Capability.................................................................................121
Table 64. System Management Request WrSized Packet Format.....................................123
Table 65. System Management Request Broadcast Packet Format...................................123
Table 66. System Management Request Type Encoding..................................................123
Table 67. INTx Message Mapping at a Bridge.................................................................127
Table 68. Interrupt Request Packet Format......................................................................129
Table 69. EOI Packet Format..........................................................................................130
Table 70. CRC Window Contents After Link Synchronization........................................133
Table 71. CRC Values for Different Link Widths............................................................135
Table 72. Error Routing Registers...................................................................................141
Table 73. CAD Value Driven by Transmitter Based on Receiver Width..........................146
Table 74. CAD Value Sampled for Transmitter and Receiver Width...............................147
Table 75. Signal States During Reset...............................................................................147
Table 76. Values of CTL and CAD During Link Initialization Sequence.........................149
Table 77. Device Message Header Request Format.........................................................154
Table 78. Device Message Data Request Format.............................................................156
Table 79. PCI Bus Transaction Ordering Rules................................................................161
Table 80. PCI Transaction Mapping to HyperTransport Packets......................................162
Table 81. HyperTransport Packet Mapping to PCI Transactions......................................163
Table 82. HP AGP Transaction Mapping to HyperTransport Packets..............................164
Table 83. Simple LP AGP Transaction Mapping to HyperTransport................................165
Table 84. Alternate LP AGP Transaction Mapping to HyperTransport............................166
Table 85. PCI-X Transaction Ordering Rules..................................................................167
Table 86. PCI-X Transaction Mappings to HyperTransport Packets................................167
Table 87. PCI-X Completion Code Mappings to HyperTransport Encodings...................169
Table 88. HyperTransport Packet Mappings to PCI-X Transactions.................................169
22
List of Tables
Page 23
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 89. HyperTransport Error Mappings to PCI-X Completion Errors..........................171
Table 90. PCI MSI to HyperTransport Packet Mapping...................................................172
Table 91. x86 Interrupt Request Packet Format...............................................................188
Table 92. Destination Mode Bit Field Encoding..............................................................189
Table 93. Trigger Mode Bit Field Encoding....................................................................190
Table 94. Interrupt Request Bit Field Encoding Summary...............................................190
Table 95. Standard End-of-Interrupt (EOI) Format..........................................................191
Table 96. Redirection Table Format................................................................................192
Table 97. System Management Request Command Encoding..........................................196
Table 98. NOP Packet Format for Doubleword-Based Flow Control...............................204
Table 99 Switch Capability Block..................................................................................229
Table 100. HyperTransport™ Link Signal Types..............................................................241
Table 101. HyperTransport™ Link Power Supply Characteristics.....................................242
Table 102. Power Requirements........................................................................................243
Table 103. RTT and RON DC Specifications........................................................................245
Table 104. HyperTransport™ Link Differential Signal DC Specifications.........................248
Table 105. HyperTransport™ Link Single-Ended Signal AC/DC Specifications...............249
Table 106. AC Impedance Specifications..........................................................................251
Table 107. HyperTransport™ Link Differential Signal AC Specifications.........................255
Table 108. Signal Groups for Transfer Timing..................................................................256
Table 109. HyperTransport™ Link Transfer Timing Specifications...................................263
Table 110. Calculated Minimum Link Bit-Times...............................................................266
Table 111. HyperTransport™ Link Phase Recovery Timing Characteristics......................271
Table 112. Relationships Between AC and DC Parameters................................................274
Table 113. Relationships Between Output and Input Parameters........................................274
Table 114. Differential Skew.............................................................................................276
Table 115. Transmitter Clock Uncertainties.......................................................................277
Table 116. Transmitter PHY Uncertainties........................................................................278
Table 117. Transmitter Package Skew...............................................................................279
Table 118. Receiver Package Skew...................................................................................279
Contents
23
Page 24
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 119. PCB Skew.......................................................................................................280
Table 120. Receiver Setup and Hold Time.........................................................................280
Table 121. Transfer Timing Overview...............................................................................281
Table 122. TX to RX Input Reference Clock Distribution Uncertainty..............................282
Table 123. Transmitter PLL Variations.............................................................................283
Table 124. Transmitter and Link Transfer Variations.........................................................284
Table 125. CLKOUT Byte Lane Uncertainty.....................................................................285
Table 126. Receiver PLL Variations..................................................................................286
Table 127. Receiver Transfer Variations............................................................................286
Table 128. CTL Sampling Error........................................................................................287
Table 129. Phase Recovery FIFO Examples......................................................................287
24
List of Tables
Page 25
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Preface
This Document
The HyperTransport™ I/O Link Specification defines and describes the input/output link protocol
and electrical interface for the HyperTransport™ technology link. The document is divided into
two principal parts: Protocol and Electrical. The protocol part includes information on
HyperTransport technology signals, packets, commands, interrupts, configuration accesses,
address map, error handling, clocking, and initialization. The electrical part includes information
on I/O power supply, AC and DC characteristics, transfer timing, and phase recovery timing.
It is intended for system designers, circuit designers, sales and marketing engineers, and other
technology professionals. This document serves as the primary reference for the HyperTransport
protocol.
Organization
The document is divided into two sections, each with appendices.
Section 1 – Protocol
Section 2 – Electrical Interface.
Section 1 – Protocol
1 Overview
This document describes the HyperTransport™ technology I/O link. HyperTransport technology,
formerly code-named Lightning Data Transport (LDT), is a packet-based link implemented on two
unidirectional sets of signals. The link is packet-based, nominally point-to-point, and connects
exactly two devices. Devices can have multiple HyperTransport links, allowing the construction of
larger HyperTransport fabrics.
Chapter 1 Section 1 – Protocol
25
Page 26
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
CLK
CTL
CAD[n:0]
CLK
CTL
CAD[n:0]
Figure 1. HyperTransport™ I/O Link
HyperTransport technology is used as an I/O channel, connecting chains of HyperTransport I/O
devices and bridges to a host system. The interface from the host to the HyperTransport chain(s) is
called the host bridge.
1.1 Terminology
For reference, the following terms are used in the HyperTransport protocol:
• Bit-time—Half of a clock period in duration. Two data bits are transmitted on each signal per
cycle.
• Byte—Eight bits.
• Doubleword—Four bytes.
• Quadword—Eight bytes.
• Packet—A series of bit-times that forms the basis of communication between two nodes.
• Transaction—A sequence of packets that are exchanged between two or more nodes in the
system and that result in a transfer of information.
• Source—The node that initiates a transaction.
• Target—The node that ultimately services the transaction on behalf of the source. Note that
there may be intermediary nodes between the source and the target.
• I/O stream—A collection of transactions that can be treated independently in the fabric with
respect to ordering rules. A given I/O stream always originates from the same node and
terminates at the same destination.
26 Overview
Chapter 1
Page 27
HyperTransport Technology Consortium
• Unit or function—A logical entity within a node that can act as a source or destination of
transactions. Functions are useful for describing the transaction ordering rules of the
HyperTransport protocol.
• Node—A physical entity that connects to one end of a HyperTransport link.
• Chain—A set of HyperTransport technology devices connected in a straight-line daisy-chain
topology with each device connecting to at most two others. Devices with two link interfaces
have no logical bridging function between the two interfaces; the entire chain operates as a
single logical bus.
• Fabric—A HyperTransport I/O fabric is implemented as one or more daisy chains of
HyperTransport technology devices, with a bridge to the host system at one end.
• Bridge—A device that bridges between a logical primary bus (connecting it to the host) and
one or more logical secondary buses. It contains a bridge register set to control mapping of
transactions between the two buses.
• Tunnel—A device that implements two link interfaces and is capable of forwarding traffic
between other devices within a chain.
• Cave—A device that implements a single primary link interface.
HyperTransport™ I/O Link Specification Revision 1.05c
• Slave—A tunnel or cave, implementing HyperTransport link(s) as its primary interface, not
implementing the Host bridge functionality.
• Host—A host can contain multiple bridges, each supporting either a single HyperTransport
chain or a tree of HyperTransport chains.
• Host bridge—The interface from the host to the HyperTransport chain.
• Undefined—Operations or behaviors that are described as undefined in this specification may
result in any outcome from no change in the state of the system to creating an environment in
which the system no longer continues to operate. Note that a cold reset of the HyperTransport
fabric may not be sufficient to restore deterministic operation since the effects of an undefined
action may propagate beyond the HyperTransport fabric.
• CSR—Configuration Space Register
1.2 HyperTransport™ Technology in x86 Platforms
This specification is written as a generic reference suitable for implementation with all CPU
architectures. However, because of the legacy infrastructure associated with x86 platforms, some
additional features must be supported in those platforms. These additional requirements are
specified in Appendix F.
Chapter 1
Overview 27
Page 28
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
2 Signaling
The HyperTransport™ technology signals listed in Table 1 constitute a single unidirectional
connection between two nodes. A full link requires a connection in each direction. However, the
connections need not be the same width in each direction.
Table 1. Link Signals
Signal Width Description
CAD 2, 4, 8,
16, or 32
CTL 1 When asserted, the CAD signals carry a control packet. When
Command, Addresses, and Data (CAD). Carries
HyperTransport™ technology requests, responses, addresses and
data. CAD width can be different in each direction.
deasserted, the CAD signals carry data.
CLK 1, 2, or 4 Clocks for the CAD and CTL signals. Each byte of CAD has a
separate clock signal. Regardless of the width of the link, CTL is
clocked by the same CLK as CAD[0].
HyperTransport links wider than 8 bits are built by ganging multiple 8-bit links in parallel to form
either 16- or 32-bit links. Links wider than 8 bits have one clock per byte, but still only one CTL
bit for the whole link. The forwarded clock for the CTL signal is the clock for the least-significant
byte.
In addition to the link signals, all HyperTransport technology devices require the
reset/initialization input pins listed in Table 2.
Table 2. Reset/Initialization Signals
Signal Width Description
PWROK 1 Power and clocks are stable.
RESET# 1 Reset the HyperTransport™ chain.
All devices in a given HyperTransport chain receive the same PWROK and RESET# signals.
HyperTransport technology devices must sample PWROK and RESET# as inputs, and may
optionally drive these signals low as open-drain outputs. Pullup resistors must be provided by the
system. These signals control the power-up and reset sequence for their HyperTransport links, and
may or may not also control the power-up and reset sequence for other logic within the device—
28 Signaling
Chapter 2
Page 29
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
this is device-specific. See Section 4.1.2 for a description of these signals in the context of systems
with multiple HyperTransport chains. See Chapter 12 for information on reset sequencing.
HyperTransport technology devices deployed in x86 or other systems requiring power
management include the signals listed in Table 3, which are used during the sequencing of system
activities such as power-savings state transitions. These signals are also open-drain wired-OR,
allowing multiple sources to request link disconnection and reconnection, and requiring pullup
resistors.
Table 3. Power Management Signals
Signal Width Description
LDTSTOP# 1 Enables and disables links during system state transitions.
LDTREQ# 1 Indicates link is active or requested by a device.
PWROK, RESET#, LDTSTOP#, and LDTREQ# will likely have long transition times, and
therefore will require hysteresis and/or debounce logic in the input path for correct operation.
Chapter 2
Signaling 29
Page 30
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
3 Packet Definition
This chapter describes the packet definition for the HyperTransport™ link. HyperTransport
technology supports link widths of 2, 4, 8, 16, and 32 bits. All tables later in this chapter assume
an 8-bit wide link.
The packet structure for 16- and 32-bit links can be derived from the 8-bit link packet structure by
combining the fields within adjacent bit-times. Some examples include:
BT116[15:0] = BT28[7:0] || BT18[7:0]
BT132[31:0] = BT48[7:0] || BT38[7:0] || BT28[7:0] || BT18[7:0]
where BTNm represents the Nth bit-time within a packet for a link of width m and “||” represents
concatenation.
Since all packets are multiples of four bytes long, packet boundaries always fall on bit-time
boundaries.
The packet structure for 2- and 4-bit links can be derived from the 8-bit link packet structure by
splitting the 8-bit link bit-times into adjacent bit-times. Some examples:
BT12[1:0] = BT18[1:0]
BT22[1:0] = BT18[3:2]
BT32[1:0] = BT18[5:4]
BT42[1:0] = BT18[7:6]
BT14[3:0] = BT18[3:0]
BT24[3:0] = BT18[7:4]
3.1 Use of the CTL Signal
HyperTransport technology links carry control packets and data packets, distinguished by the use
of the CTL signal. Link transmitters assert CTL during all bit-times of control packets, and
deassert it during data packets. The purpose of the CTL signal is to allow control packets to be
inserted in the middle of long data packets.
30
Packet Definition Chapter 3
Page 31
HyperTransport Technology Consortium
The following rules govern packet transmission.
1. CTL is asserted through all bit-times of a control packet.
2. Control packets larger than four bytes must be transmitted contiguously, without deassertion of
CTL.
3. CTL is deasserted through all bit-times of a data packet.
4. CTL may be asserted on four-byte boundaries within a data packet to insert a control packet.
Control packets inserted into data packets must not themselves have an associated data packet.
When CTL is deasserted at the conclusion of a control packet, data transfer continues from the
point where it left off.
5. Write request and read response packets always have an associated data packet. The data
packet might not immediately follow the last bit-time of its associated control packet, because
other control packets may be inserted before the data packet. However, because inserted
control packets cannot have associated data, there can only be one data transfer outstanding.
6. The order of operations on the link is determined by the order of the control packets. The fact
that data transfer for a control packet may be delayed does not affect how it is ordered.
7. The bit-time immediately following the last bit-time of a data packet is always the start of a
control packet (CTL must be asserted).
8. CTL may only be asserted or deasserted on a four-byte boundary, starting at the point in the
initialization sequence (Section 12.2) where CAD and CTL both transition from 1 to 0. This
alignment must be maintained until RESET# is asserted or a disconnect sequence (Section 8.3)
completes.
9. CTL may only be deasserted when data transfer due to a previously transmitted control packet
is being sent.
HyperTransport™ I/O Link Specification Revision 1.05c
3.2 Packet Structure
This section defines the basic control and data packet types and shows the position of the fields
that are common to all the control packet types. All packets are multiples of four bytes long.
3.2.1 Control Packets
Control packets consist of four or eight bytes. This section shows the basic structure of each of
these control packet forms.
In the tables that follow, the unlabelled packet fields are command-specific. Some common
control packet fields are as follows:
• Cmd[5:0] is the command field that defines the packet type.
• Isoc indicates that this packet may have different priority and ordering requirements from
other packets, as described in Appendix D.
Chapter 3 Packet Definition
31
Page 32
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
• UnitID[4:0] serves to identify one of the participants in a transaction. Since all packets are
transferred either to or from the host bridge at the end of the chain, either the source or
destination node is implied. The value 0 is reserved for the UnitID of the host bridge. See
Section 4.2 for more details on the use of UnitID. Nodes with multiple logical I/O streams can
own multiple UnitID values.
• Bridge indicates that this response packet was placed onto the link by the host bridge, and it is
used to distinguish responses traveling upstream from responses traveling downstream. In the
case of two host bridges sending packets to each other on a double-ended chain, the target host
bridge appears to the requesting host bridge as a HyperTransport technology slave device.
Therefore, the bridge bit will be clear on responses to requests issued from the far host bridge.
• SeqID[3:0] is used to tag groups of requests that were issued as part of an ordered sequence by
a device and must be strongly ordered within a virtual channel. All requests between the same
source and destination and within the same I/O stream and virtual channel that have matching
nonzero SeqID fields must have their ordering maintained. The SeqID value of 0x0 is reserved
to mean that a transaction is not part of a sequence. Transactions with this value have no
sequence-ordering restrictions, although they may be ordered for other reasons as described in
Chapter 6. Tunnels are required to keep requests in the same I/O stream and virtual channel
with matching nonzero SeqID fields in order when forwarding them.
• PassPW indicates that that this packet is allowed to pass packets in the posted request channel
of the same I/O stream. Otherwise, this packet must stay ordered behind them. This bit should
be cleared to maintain the full producer/consumer ordering model of HyperTransport
technology. Systems that do not require this ordering may set PassPW for higher performance.
• SrcTag[4:0] is a transaction tag that is used to uniquely identify all transactions in progress
initiated by a single requester. Each UnitID can have up to 32 transactions in progress at one
time. The concatenation of source UnitID and SrcTag serves to uniquely identify nonposted
requests. The SrcTag field is not relevant for posted requests and is reserved. SrcTag is used to
match responses with their requests.
• Addr[63:2] represents the doubleword address accessed by the request. Not all address bits are
included in all request types. Where finer granularity is required, byte masks are used.
Reserved fields in command packets must always be driven to 0 by transmitters when originating
packets and must be assumed to be undefined by receivers. Reserved fields should be preserved
when forwarding packets through a tunnel or HyperTransport-to-HyperTransport bridge.
3.2.1.1 Info Packet
Info packets (defined in Table 4) are always four bytes long. They are used for nearest neighbor
communication between nodes, and so exist at the lowest level of the protocol. They are not routed
within the fabric, and they require no buffering in the nodes. They are not flow-controlled, and
they can always be accepted by their destination.
32
Packet Definition Chapter 3
Page 33
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 4. Info Packet Format
Bit-Time 7 6 5 4 3 2 1 0
0 Command-Specific Cmd[5:0]
1 Command-Specific
2 Command-Specific
3 Command-Specific
3.2.1.2 Request Packet
Request packets are either four or eight bytes long, depending upon whether the request has an
associated address. Table 5 shows a request packet with an address. Four-byte request packets do
not contain the address field.
Table 5. Request Packet Format with Address
Bit-Time 7 6 5 4 3 2 1 0
0 SeqID[3:2] Cmd[5:0]
1 PassPW SeqID[1:0] UnitID[4:0]
2 Command-Specific
3 Command-Specific
4 Addr[15:8]
5 Addr[23:16]
6 Addr[31:24]
7 Addr[39:32]
3.2.1.3 Address Extension
Request packets that normally carry a 40-bit address can be prepended by an Address Extension
doubleword. This allows 64-bit addressing. Table 6 shows an extended request packet. Creation,
forwarding, and acceptance of 64-bit addresses is optional and enabled on a per-link basis, as
described in Section 7.5.4.12.
Chapter 3 Packet Definition
33
Page 34
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 6. Request Packet Format with Extended Address
Bit-Time 7 6 5 4 3 2 1 0
0 Reserved Cmd[5:0]=111110b
1 Addr[47:40]
2 Addr[55:48]
3 Addr[63:56]
4 SeqID[3:2] Cmd[5:0]
5 PassPW SeqID[1:0] UnitID[4:0]
6 Command-Specific
7 Command-Specific
8 Addr[15:8]
9 Addr[23:16]
10 Addr[31:24]
11 Addr[39:32]
3.2.1.4 Response Packet
Response packets (defined in Table 7) are always four bytes long.
Table 7. Response Packet Format
Bit-Time 7 6 5 4 3 2 1 0
0 Command-Specific Cmd[5:0]
1 PassPW Bridge Rsv UnitID[4:0]
2 Command-Specific Error0 Command-Specific
3 Rsv/RqUID Error1 Rsv Command-Specific
3.2.1.5 Command Field Encoding
The command field, shown in Table 8, is valid for all control packets.
34
Packet Definition Chapter 3
Page 35
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 8. Command Field Encoding for All Control Packets
Code VChan Command Comments/Options Packet Type
000000 - NOP Null packet. Contains flow control information. Info
000001 Reserved-HOST
000010 NPC Flush Flush posted writes Request
000011
0001xx
001xxx
101xxx
Reserved-HOST
NPC
PC
Wr (sized)
Write Request
[5] Defines whether request is posted:
0: Nonposted
1: Posted
[2] Defines the data length:
0: Byte
1: Doubleword
[1] Defines bandwidth/latency requirements:
0: Normal
1: Isochronous
Req/Addr/Data
[0] Indicates whether access requires host cache
coherence (reserved and set if access is not to
host memory):
0: Noncoherent
1: Coherent
01xxxx
100xxx Reserved-I/O
110000 R RdResponse Read Response Resp/Data
NPC Rd (sized)
Read Requests
[3] Defines ordering requirements for response:
0: Response may not pass posted requests
1: Response may pass posted requests
[2] Defines the data length:
0: Byte
1: Doubleword
[1] Defines bandwidth/latency requirements:
0: Normal
1: Isochronous
[0] Indicates whether access requires host cache
coherence (reserved and set if access is not to
host memory):
0: Noncoherent
1: Coherent
Req/Address
110001
110010
110011 R TgtDone Tell source of request that target is done. Response
Chapter 3 Packet Definition
Reserved-HOST
35
Page 36
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Code VChan Command Comments/Options Packet Type
11010x Reserved-HOST
11011x Reserved-I/O
11100x Reserved-HOST
111010 PC Broadcast Broadcast Message Req/Address
111011 Reserved-HOST
111100 PC Fence Fence for posted requests Request
111101 NPC Atomic-RMW Atomic Read-Modify-Write Req/Addr/Data
111110 - AddrExt Address Extension Address
111111 - Sync/Error Link Synchronization and Error Packet Info
Notes:
The fields in Table 8 are defined as follows:
Code is the 6-bit command encoding in each packet.
VChan indicates the virtual channel that the packet travels in. Info packets are only used for single-link
communication and do not use buffer space, and thus are not in a virtual channel. See Section 4.7 and Section 4.8 for
more information.
PC—Posted Command (Request)
NPC—Nonposted Command (Request)
R—Response
Command is the mnemonic used to represent the command.
Comments/Options gives a short description of the command and enumerates any option bits within the Code field.
Packet Type indicates the type of packet(s) used by the command.
Reserved-I/O identifies code points that are reserved for future use.
Reserved-HOST identifies code points that may be used in a host-specific protocol and will not be used to implement
future features in the HyperTransport™ I/O Link Protocol Specification.
Receiving a packet with a reserved command code is a protocol error (see Section 10.1.3) and may
result in undefined operation of devices that do not implement recovery from protocol errors.
3.2.2 Data Packet
Data packets contain the data payload for transactions. Data packets follow write request and read
response packets. Data packets range in length from four to 64 bytes, in multiples of four bytes
(one doubleword), as indicated by the Count field of the most recent payload-bearing command.
Within a doubleword, data bytes appear in their natural byte lanes. For transfers of less than a full
doubleword, the data is padded with undefined bytes to achieve this byte-lane alignment.
Table 9 shows an example of an eight-byte data packet.
36
Packet Definition Chapter 3
Page 37
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 9. Eight-Byte Data Packet Format
Bit-Time 7 6 5 4 3 2 1 0
0 Data[7:0]
1 Data[15:8]
2 Data[23:16]
3 Data[31:24]
4 Data[39:32]
5 Data[47:40]
6 Data[55:48]
7 Data[63:56]
The data packet for a sized read response is arranged with the lowest addressed doubleword
returned first, and the remainder of the addressed data is returned in ascending address order by
doubleword. Sized read responses can contain any number of contiguous doublewords within a
64-byte aligned block. Although, for sized byte reads, not all bytes are guaranteed to be valid. The
data cannot wrap from the most significant doubleword in the aligned 64-byte block to the least
significant doubleword in the block.
Sized doubleword writes work in the same way as sized doubleword read responses and can
contain anywhere from one to 16 doublewords in ascending address order.
Sized byte writes, defined in Section 4.4.1, transmit one doubleword worth of masks first,
followed by from one to eight doublewords of data in ascending address order, as shown in Table
10. Mask[0] corresponds to Data[7:0], Mask[1] to Data[15:8], and so on. Thirty-two mask bits are
always transmitted, regardless of the amount of data. All-zero byte masks are permitted. Interrupt
and system management messages, which are composed of byte write packets to predefined
address ranges, are the only byte write packets that do not require at least one doubleword of data.
Chapter 3 Packet Definition
37
Page 38
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 10. Sized Byte Write Data Packet Format
Bit-Time 7 6 5 4 3 2 1 0
0 Mask[7:0]
1 Mask[15:8]
2 Mask[23:16]
3 Mask[31:24]
4 Data[7:0]
5 Data[15:8]
6 Data[23:16]
7 Data[31:24]
8+ Packet may contain up to eight doublewords of data.
3.2.3 Note about VC Stalls due to Command-Data Separation
Some implementations may issue a command and then not issue the corresponding data packet for
an extended interval due to some internal condition. Because the data following a command must
belong to the command, this can cause all VCs to stall, severely impacting throughput.
Implementations like this are strongly discouraged.
Another reason why VC stalls may occur is when bridging between link speeds. A command
packet from a slower link may appear on the faster link well in advance of the data packet
associated with the command, preventing the insertion of any other command packet which has an
associated data packet. If the rates are mismatched by a factor of 20% or more, the throughput
may be noticeably impacted. Tunnels are urged to address this mismatch case by deferring the
forwarding of a command packet for some number of clocks until the associated data packet can
be sent out back to back.
A third case when this can happen is when a protocol is being bridged which also has the property
that the command and data can be separated or stalled. Bridges are urged to consider this case and
make an attempt to put a command and its associated data packet out back to back.
38
Packet Definition Chapter 3
Page 39
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4 Fabric Operation
The HyperTransport™ link is a pipelined, split-transaction interconnect where transactions are
tagged by the source and responses can return to the source out of order. This chapter outlines the
basic operation of the link.
4.1 Topology
HyperTransport I/O fabrics are implemented as one or more daisy chains of HyperTransport
technology-enabled devices, with a bridge to the host system at one end. Devices can implement
either one or two links.
• A dual-link device that is not a bridge is called a tunnel.
• Single-link devices must always sit on the end of the chain, so only one single-link device is
possible in a chain.
Direct peer-to-peer communication between devices in the chain is not allowed. All packets
(except for info packets) travel between one device and the host bridge. This means that at a high
level, the fabric appears as a group of devices directly connected to the host bridge, but not to each
other. Packets flowing into the fabric from a host bridge are said to be flowing downstream.
Packets flowing to a host bridge from a HyperTransport technology device are said to be flowing
upstream.
A single HyperTransport I/O chain contains no HyperTransport-to-HyperTransport bridge
devices. It can contain native HyperTransport technology peripheral devices (like an Ethernet
controller) and can also contain bridges to other interconnects (like PCI). A chain is terminated at
one or both ends by a bridge. In the simplest topology, a chain connects to the host bridge at one
end and has no connection at the other end.
A HyperTransport tree contains one or more HyperTransport-to-HyperTransport bridge devices. A
HyperTransport technology bridge device has a primary link, being the upstream link in the
direction of the host and one or more secondary links. Each HyperTransport chain that connects to
a bridge’s secondary link is assigned a unique bus number (see Section 7.2.3 for details). The
HyperTransport-to-HyperTransport bridge device operates as a host bridge for devices on its
secondary chain. In addition to its secondary links, a HyperTransport technology bridge device
may have a downstream link that is associated with the same bus number as the bridge’s primary
link. The root of the HyperTransport tree connects to the host.
The host can contain multiple bridges, each supporting either a single HyperTransport I/O chain or
a tree of HyperTransport I/O chains. Some example configurations and topologies are shown in
Figure 2 and Figure 3. In these figures, “P” indicates a primary interface capability block and “S”
Chapter 4 Fabric Operation
39
Page 40
HyperTransport Technology Consortium
Bridge
Bridge
without Tunnel
P S P P S
P P S P P P P P
S S
HyperTransport™ I/O Link Specification Revision 1.05c
indicates a secondary interface capability block. See Section 7.5 for details of these capability
blocks.
P
Single-Link
Tunnel
with Tunnel
Figure 2. Example Device Configurations
Host Bridge
Chain
Host Bridge
Tree
Figure 3. Example Topologies
For convenience in integrating multiple functions onto a single chip, or to allow parallelism
between independent request streams, individual HyperTransport technology devices can use
multiple UnitID values. There is no specific limit on the number of physical devices. However,
40
Fabric Operation Chapter 4
Page 41
HyperTransport Technology Consortium
there are only 31 UnitIDs available to each chain. No combination of devices that exceeds 31
UnitIDs may be connected to a single chain.
HyperTransport™ I/O Link Specification Revision 1.05c
4.1.1 Double-Hosted Chains
Physically, a chain can be connected to a host bridge at each end, as long as the chain contains no
single-link devices. This may be useful to provide another path to I/O devices in the event of a
host bridge or link failure, or to allow sharing of I/O devices between independent hosts to
implement clustering. One bridge is designated the master bridge for the shared chain, while the
other will be the slave bridge. This designation must be made before the chain is reset. (The
method of doing so is beyond the scope of this specification.)
There are two types of double-hosted chains supported by HyperTransport technology: sharing
and non-sharing.
• In a sharing double-hosted chain, traffic is allowed to flow end to end, and both hosts are able
to issue requests to each other and to any device. Generally, all devices in the chain should
belong to the master host to avoid a peer-to-peer transaction deadlock as described in Section
4.7. A device belongs to a host when the Master Host and Default Direction bits (defined in
Section 7.5.3.2) point to that host. If devices need to be accessed from either host, the slave
host may have its Act as Slave bit (defined in Section 7.5.3.3.6) set so that all requests pass
through the master host to maintain ordering, as defined in Section 6.1.
• A non-sharing double-ended chain appears logically as two distinct daisy chains, each
attached to only one host bridge. Software will select a point to break the chain in two and
reconfigure the devices to divide them between the bridges in order to balance traffic. Once the
chain is broken, the hosts will not be able to issue requests to each other until a reset.
The initialization sequence described in Section 12.3 will ensure that all devices are assigned
unique device numbers. In the event of a node or link failure, the sequence will cause the devices
on each side of the break in the chain to belong to the host bridge on that end, forcing a nonsharing chain.
Because devices accept requests from both directions in either double-hosted chain type, they must
keep track of which link incoming request packets were received on and send any responses back
on the same link. An interior node may see the same SrcTag active from the host bridges at both
ends of the link. The node must recognize the two host bridges as having disjoint SrcTag spaces.
Double-hosted chain support for hosts is optional, but recommended. In order to support doublehosted chains, a host must implement the Double Ended and Chain Side fields of the
HyperTransport technology Command register, specified in Section 7.5.3.3, and the host must
properly accept cycles targeted to it, as described in Section 4.9.4. If a host does not support
double-hosted chains, it cannot be connected to the secondary port of a bridge (for clustering).
Chapter 4 Fabric Operation
41
Page 42
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
To support a sharing double-hosted chain, the host must also implement the Device Number and
Host Hide fields of the Command register, specified in Section 7.5.3.3, and deal with the ordering
described in Section 6.4.
4.1.2 HyperTransport™ Technology Signals PWROK and RESET#
This section describes the HyperTransport technology signals PWROK and RESET# in the
context of various system topologies. Section 4.1.2.1 describes requirements that all
HyperTransport technology-enabled devices and systems must meet. Section 4.1.2.2 describes
some host implementations.
4.1.2.1 Requirements
All devices on a HyperTransport I/O chain are expected to share a single logical
PWROK/RESET# signal pair. Due to the potential for devices sampling these signals on different
clocks, copies of the signals coming from different drivers, slow edges being sensed at different
times, or receivers having different thresholds, these signals will not necessarily be observed to
transition simultaneously at all devices. The system must guarantee that all devices see PWROK
and RESET# pulses overlap and that the duration of the overlap meets the minimum requirements
given in Section 12.2. These signals are inputs to each device on the chain and may be driven by
one or more devices on the chain. These signals control the powerup and reset sequence for each
link interface in the chain and may optionally control the powerup and reset sequence for other
logic inside any device along the chain.
A HyperTransport-to-HyperTransport bridge device must have dedicated PWROK/RESET# pin
pairs for its primary chain and for each secondary chain. The bridge must be able to drive
PWROK/RESET# on its secondary chain. In addition, the bridge must pass the assertion and
deassertion of PWROK/RESET# from its primary chain to its secondary chain. A bridge does not
pass the assertion and deassertion of PWROK/RESET# from its secondary chain to its primary
chain. A bridge is required to provide appropriate error responses to any outstanding nonposted
requests when the secondary bus reset is asserted, as described in Section 10.1.6.
4.1.2.2 Host Implementations
In the case of a host with a single HyperTransport chain, the host’s reset signal can be independent
of the HyperTransport link’s PWROK/RESET# signals. This allows software running on the host
to reset the HyperTransport chain without requiring the host to be reset (see Section 7.4.9.7). In
such an implementation, the host bridge must be able to both drive and sample PWROK and
RESET#. In addition, the host bridge must pass the assertion and deassertion of host reset (or
PWROK) to RESET# (or PWROK). Other implementations are possible—for example, host reset
and HyperTransport technology reset functions may be tied to a single host reset pin.
42
Fabric Operation Chapter 4
Page 43
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
In the case of a host with multiple host bridges, there can be independent PWROK/RESET# signal
pairs for each chain connected to the host. As in the previous case, each of these
PWROK/RESET# signals can be independent of host reset. Other implementations are possible—
for example, host reset and all HyperTransport technology reset functions may be tied to a single
host reset pin.
Proper sequencing of the PWROK and RESET#, as described in Section 12.2, must be assured,
even if the host’s own PWROK and reset signals do not follow these sequencing rules.
Devices used in x86 systems have specific mandatory PWROK and RESET# requirements,
described in Appendix F.
4.2 Transactions and UnitID
Since all HyperTransport technology transactions consist of a series of packet transfers between a
device and the host bridge, the use of the UnitID field can be simply summarized in Table 11.
Table 11. UnitID Field Usage
Upstream Downstream
Request
Response
UnitID is source of request:
Device's UnitID
UnitID is target of request:
Device's UnitID
Bridge bit clear
UnitID is source of request:
Host's UnitID: Clumped 0
UnitID is source of request:
Device's UnitID
Bridge bit set
Notes:
1. The Bridge bit in a downstream response must be set and the UnitID must be that of the request, even when
the responder is a slave, not a host, as described in Section 4.9.3.
2. A Clumped 0 UnitID is either 0 itself, or any of the consecutive UnitIDs that have been clumped with 0, as
defined in Section 4.6.1.
Peer to peer communication is implemented as a pair of HyperTransport technology
transactionsa transaction generated by the source device and targeted at the host, and a
transaction generated by the host and directed to the target device. The UnitIDs in the request and
response packets associated with these two transactions follow the rules in Table 11.
Chapter 4 Fabric Operation
43
Page 44
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4.3 Link Synchronization
The Sync pattern is used to indicate that a resynchronization event has occurred in the system,
such as a reset or a chain error, which requires all links to be resynchronized. The Sync pattern is
defined in Table 12.
Table 12. Sync Pattern Format
Bit-Time 7 6 5 4 3 2 1 0
0 11 Cmd[5:0]: 111111
1 11111111
2 11111111
3 11111111
CRC checking on a link is shut down when a Sync packet is received. See Section 10.1 for a
description of CRC.
All fields in a Sync pattern (including the command) are all 1s. Receivers on 8-, 16-, or 32-bit
links may detect a Sync pattern by observing at least 16 bit-times of all 1s on byte lane 0 of the
link (starting with the rising edge of CLK in 8- or 16-bit links), or by decoding the Sync command
via its normal command decode logic. Sync patterns on 4- and 2-bit links require two times and
four times the number of all-1-bit-times, respectively, as 8-bit, 16-bit, and 32-bit links.
Once a transmitter places a Sync pattern onto an active link, it keeps that pattern on the link until
after the link is reset and synchronized. This allows a receiver to detect Sync via either method.
4.4 Requests
4.4.1 Sized Reads and Writes
The Sized Read or Write request is defined in Table 13. Sources use the Sized Read and Write
requests (byte or doubleword) to initiate transactions to either memory or I/O. The data returned
for Sized Reads cannot be coherently cached, as HyperTransport I/O provides no coherence
primitives. Sized requests contain the starting doubleword address of the data and a set of data
elements to be transferred. Bit 2 of the command field indicates whether the data elements to be
transferred are bytes or doublewords, as defined in Table 8. Table 8 also defines the Coherence,
Isochronous, Posted, and Response May Pass Posted Request bits in the command field.
44
Fabric Operation Chapter 4
Page 45
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 13. Sized Read or Write Request Format
Bit-Time 7 6 5 4 3 2 1 0
0 SeqID[3:2] Cmd[5:0]
1 PassPW SeqID[1:0] UnitID[4:0]
2 Mask/Count[1:0] Compat
3 Addr[7:2] Mask/Count[3:2]
4 Addr[15:8]
5 Addr[23:16]
6 Addr[31:24]
7 Addr[39:32]
SrcTag[4]/
Data Error
SrcTag[3]/
Chain
SrcTag[2:0]/Rsv
Doubleword operations can transfer any number of contiguous complete doublewords within a 64byte aligned block. The Count field encodes the number of doubleword data elements that should
be transferred, beginning at the specified address, and going in ascending order. Count codes of 0
through 15 represent 1 through 16 data elements to be transferred, respectively. Requests that
cross a 64-byte boundary must be broken into multiple transactions, issued in ascending address
order.
Byte reads can transfer any combination of bytes within an aligned doubleword. The Mask field is
used to indicate which bytes within the doubleword are being read. Mask[0] corresponds to the
lowest addressed byte, and Mask[3] corresponds to the highest addressed byte. Byte-maskable
reads crossing an aligned doubleword boundary must be broken into multiple requests, each
within a single doubleword. The mask bits can be ignored for reads to regions where reads are
guaranteed not to have side effects. A read where all mask bits are 0 still causes host coherence
action (if to memory and Cmd[0] is asserted) and still returns a RdResponse packet with one
doubleword of (invalid) data.
Byte writes can transfer any combination of bytes within a naturally aligned 32-byte address
region. Transfers that cross an aligned 32-byte boundary must be broken into multiple
HyperTransport technology transactions, issued in ascending address order. Address bits [4:2]
identify the first doubleword of data sent in the data packet within the 32-byte region defined by
address bits [63:5]. The data packet for a byte-write operation contains byte mask information in
the first doubleword of the data packet. The Count field is used to indicate the total size of the data
packet in doublewords, including the byte masks, so it will range from one to eight to indicate two
through nine doublewords to be transferred. In general, it is illegal for a byte-write packet to
contain byte masks and no data, meaning the Count field must contain a nonzero value. The
exceptions to this are interrupt and system management messages—they take the form of byte
writes to predefined address regions and do not require data to be transferred. See Section 3.2.2 for
the format of the data packet. The Count field specifies the length of the data packet independent
of the value of the byte masks. Nonzero byte masks for doublewords that are not sent result in
Chapter 4 Fabric Operation
45
Page 46
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
undefined behavior. Byte masks may be 0 for doublewords that are sent. The entire byte mask
doubleword may be 0, in which case the system performs all activities usually associated with the
request. However, no data is written.
The sized command field contains a bit that indicates whether the access requires coherence action
to be taken by the system for host memory accesses. If this bit is set, the host must take whatever
action is appropriate to ensure that any caching agent remains coherent with system memory.
Writes must cause caches to be updated or invalidated. Reads must return the latest modified copy
of the data, even if main memory is stale. If the bit is clear, reads and writes can happen directly to
and from main memory without polling or modifying cache states. Most devices require host
hardware to maintain coherence between processor caches within the host and host memory. Some
devices may not require coherence to be maintained, or may have alternative application-specific
means of ensuring memory coherence, and may clear the coherent bit to indicate this to the host.
The coherent bit is reserved and must be set for accesses that are not to memory address space.
Transactions also have an Isochronous bit in the command encoding associated with them that
must be maintained by tunnels even when Isochronous flow control mode is disabled. Host
bridges should maintain the bit when forwarding peer-to-peer requests if possible. See Appendix
D for details of how this is used.
Sized Writes have a Posted bit. Besides serving as a virtual channel identifier, a set Posted bit
indicates that the write request will receive no response in the fabric. The requester’s buffer can be
deallocated as soon as the write is transmitted. As such, the SrcTag field is reserved for posted
requests. No assumptions can be made about the uniqueness or meaning of SrcTags for posted
requests, either relative to other posted requests, or to other traffic. Bit 4 of bit-time 2 (Data Error)
is set in posted requests to indicate that a data error has occurred while forwarding the request. If
no error is detected while forwarding the request, the bit is 0. Bit 3 of bit-time 2 (Chain) is set in
posted requests to indicate that this request will be followed by another and they must be
forwarded together without other posted requests interleaved. The Chain bit is currently used in
Device Messages but must be honored for all posted requests. Frequent use of long chains of
requests would have a negative impact on system performance and should be avoided. Requests
that are chained together must have the same destination, or undefined operation will result. All
nonposted requests (such as Reads, Nonposted Writes, Flushes, and Atomic RMW) for a given
UnitID must have a unique SrcTag value for each outstanding request. The last packet in a chain
must have the chain bit clear.
Reads have a Response May Pass Posted Requests bit in the command field. This bit should be
cleared to maintain the full producer/consumer ordering model of HyperTransport technology.
Systems that do not require this ordering may set PassPW for higher performance. This bit is
carried with the request but does not serve any purpose until the response is generated. At that
time, it becomes the PassPW bit in the response.
Unlike read requests, write requests do not contain a Response May Pass Posted Requests bit.
Therefore, the PassPW bit in the TgtDone packet will generally be set. However, this is not strictly
46
Fabric Operation Chapter 4
Page 47
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
required—responders can choose to clear the PassPW bit in the TgtDone packet based on
implementation-specific considerations. See Sections F.2.1.1 and F.2.5 for some examples.
The Compat bit is used to implement the subtractive decode necessary for boot firmware and
legacy devices. When set, it indicates that address decode in the host has found no mapping for the
given access, and therefore the access should be routed to the bus segment containing the
subtractive decode device. As part of the initialization sequence, all HyperTransport technology
devices determine whether they own (or are) the subtractive decode device. Accesses with the
Compat bit set are always accepted by devices that own it and ignored by all other devices,
regardless of address. The Compat bit may only be asserted for downstream requests. It is reserved
for upstream requests and configuration space requests.
4.4.2 Broadcast Message
Broadcast messages (defined in Table 14) are used by the host to communicate information to all
HyperTransport technology-enabled devices.
Table 14. Broadcast Message Format
Bit-Time 7 6 5 4 3 2 1 0
0 SeqID[3:2] Cmd[5:0]: 111010
1 PassPW SeqID[1:0] UnitID[4:0]
2 Reserved
3 Addr[7:2] Rsv
4 Addr[15:8]
5 Addr[23:16]
6 Addr[31:24]
7 Addr[39:32]
Broadcast messages can only be issued by the host bridge, and they travel in the downstream
direction for the entire length of the chain, being both accepted and forwarded by all devices.
Features that are implemented using Broadcast messages have reserved address ranges associated
with them that are recognized by all devices. All information (including potential write data)
necessary to the specific type of operation being performed is carried in the address field.
Broadcasts travel in the posted channel, and the SrcTag field is reserved. No assumptions can be
made about the uniqueness of SrcTags, either relative to other Broadcast messages or other traffic.
Chapter 4 Fabric Operation
47
Page 48
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4.4.3 Flush
Flush is designed to make sure that posted writes have been observed at host memory. It applies
only to requests in the same I/O stream as the flush. The Flush command is defined in Table 15.
Table 15. Flush Format
Bit-Time 7 6 5 4 3 2 1 0
0 SeqID[3:2] Cmd[5:0]: 000010
1 PassPW SeqID[1:0] UnitID[4:0]
2 Rsv Isoc SrcTag[4:0]
3 Reserved
Flush functions very similarly to a Read operation, except that it returns no data. Like Reads,
Flush goes in the nonposted request virtual channel. For a Flush to perform its intended function,
the PassPW bit must be clear, so that the Flush pushes all requests in the posted channel ahead of
it. It is expected that Flushes will never be issued as part of an ordered sequence, so their SeqID
will always be 0. Flush requests with PassPW set or with a nonzero SeqID are legal, but their
effect is unpredictable. The Isoc bit indicates which Virtual Channel set the Flush applies to. A 0
indicates the normal Virtual Channels, and a 1 indicates the Isochronous Virtual Channels. This
bit is only significant when Isochronous Flow Control is enabled. When Isochronous Flow Control
is disabled, Isochronous traffic flows in the normal virtual channels and is affected by all Flushes.
See Appendix D.1 for more on Isochronous Flow Control.
All nonposted requests (such as Reads, Nonposted Writes, Flushes, and Atomic RMW) for a given
UnitID must have a unique SrcTag value for each outstanding request.
Note that Flush only guarantees that posted requests have been flushed to their destination within
the host. If the requests were peer-to-peer, this only means that they reached their destination host
bridge, not the final device.
The Flush response is returned from the host bridge when the requests have become globally
visible in the host. Since there is no data, a TgtDone response with PassPW set is used.
Flush is only issued from a device to a host bridge or from one host bridge to another. Devices are
never the target of a Flush so they do not need to perform the intended function. If a device at the
end of the chain receives a Flush, it must decode it properly to maintain proper operation of the
flow control buffers and should return a TgtDone with a Master Abort indicated.
48
Fabric Operation Chapter 4
Page 49
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4.4.4 Fence
Fence is designed to provide a barrier between posted writes, which applies across all UnitIDs and
therefore across all I/O streams and all virtual channels. The Fence command is defined in Table
16.
Table 16. Fence Format
Bit-Time 7 6 5 4 3 2 1 0
0 SeqID[3:2] Cmd[5:0]: 111100
1 PassPW SeqID[1:0] UnitID[4:0]
2 Rsv Isoc Reserved
3 Reserved
Fence goes in the posted request virtual channel and has no response. There is therefore no SrcTag
field in the request packet. A Fence with PassPW clear will not pass anything in the posted
channel regardless of UnitID. Packets with their PassPW bit clear will not pass a Fence regardless
of UnitID. Packets with their PassPW bit set may pass a Fence.
For a Fence to perform its intended function, the PassPW bit must be clear so that the Fence
pushes all requests in the posted channel ahead of it. Fence requests are never issued as part of an
ordered sequence, so their SeqID will always be 0. Fence requests with PassPW set, or with a
nonzero SeqID, are legal, but may have an unpredictable effect. The Isoc bit indicates which
Virtual Channel set the Fence applies to. A 0 indicates the normal Virtual Channels, and a 1
indicates the Isochronous Virtual Channels. This bit is only significant when Isochronous Flow
Control is enabled. When Isochronous Flow Control is disabled, Isochronous traffic flows in the
normal virtual channels and is affected by all Fences. See Appendix D.1 for more on Isochronous
Flow Control.
Fence is only issued from a device to a host bridge or from one host bridge to another. Devices are
never the target of a fence so they do not need to perform the intended function. If a device at the
end of the chain receives a fence, it must decode it properly to maintain proper operation of the
flow control buffers. The device should then drop it. The node can choose to log this as an end-ofchain error, as described in Section 10.1.5.
Chapter 4 Fabric Operation
49
Page 50
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4.4.5 Atomic Read-Modify-Write
The Atomic Read-Modify-Write (RMW) request is defined in Table 17. The Atomic RMW
request supports two forms of atomic RMW operation on a naturally aligned quadword location:
• Fetch and Add
• Compare and Swap
Table 17. Atomic Read-Modify-Write (RMW) Request Format
Bit-Time 7 6 5 4 3 2 1 0
0 SeqID[3:2] Cmd[5:0]: 111101
1 PassPW SeqID[1:0] UnitID[4:0]
2 Count[1:0] Compat SrcTag[4:0]
3 Addr[7:3] Rsv Count[3:2]
4 Addr[15:8]
5 Addr[23:16]
6 Addr[31:24]
7 Addr[39:32]
The Fetch and Add operation is:
FetchAdd(Out, Addr, In) {
Out = Mem[Addr];
Mem[Addr] = Mem[Addr] + In; // Unsigned add without saturation or carry
}
The Compare and Swap operation is:
CompareSwap(Out, Addr, Compare, In) {
Out = Mem[Addr];
If (Mem[Addr] == Compare) Mem[Addr] = In;
}
These operations must be performed atomically by the target of the request, meaning that no other
agent in the system may access the addressed location between the time that it is read and written
on behalf of the atomic request.
A Fetch and Add request must be accompanied by one quadword of data (the input value) and
have a Count field value of 1. A Compare and Swap request must be accompanied by two
quadwords of data (the compare and input values) and have a Count field value of 3. The Compare
value is first, followed by the input value. The value of the Count field is used to distinguish
between the two request types. While the action taken when the Count field is not 1 or 3 is
50
Fabric Operation Chapter 4
Page 51
HyperTransport Technology Consortium
undefined, all devices must use the value of Count to determine the size of the data payload
accompanying the request in order to forward it properly, regardless of its value.
From a transaction perspective, an Atomic RMW request is a nonposted write that generates a read
response. The read response packet contains a single quadword—that being the original value at
the addressed location. Note that for Compare and Swap, the value of the Count field in the
response packet is different from that in the associated request packet.
It is expected that Atomic RMW requests will be generated by HyperTransport I/O devices or
bridges and directed to system memory (DRAM) that is controlled by the host. Therefore, the
Compat bit will normally be cleared. No targets are required to support atomic operations If a
target receives an unsupported atomic operation, it may either return a one-quadword read
response with Target Abort indicated, or it may perform the RMW in a non-atomic way. If a host
receives an atomic operation that does not target host space, it may either reflect it as a peer-topeer cycle or return a Target Abort.
Unlike the RdSized request packet, the Atomic RMW request packet does not contain the
RespPassPW, Isoc, or Coherent bits in the command field of the packet, as defined in Table 8. The
implied values of these bits are as follows:
HyperTransport™ I/O Link Specification Revision 1.05c
• Coherent: 1—The addressed data may be cached.
• Isochronous: 0—Isochronous Atomic RMW requests are not supported.
• RespPassPW: 0—The response to the Atomic RMW request may not pass posted writes.
All nonposted requests (such as Reads, Nonposted Writes, Flushes, and Atomic RMW) for a given
UnitID must have a unique SrcTag value for each outstanding request.
4.4.6 Address Extension
An address extension control doubleword may be prepended to a Read, Write, Broadcast, or
Atomic RMW request to extend the address of the request to 64 bits, as shown in Table 18. No
control or data packets may be transmitted between the extension doubleword and the request it
affects. (CRC insertion may occur.) An extension doubleword must only be transmitted with an
accompanying request. The extension doubleword and the accompanying request are ordered and
flow controlled together as a single request. A request without an address extension is assumed to
have Address[63:40] 0’s. Address Extension doublewords of all 0’s are illegal and may result in
undefined operation.
Chapter 4 Fabric Operation
51
Page 52
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 18. Request Packet Format with Extended Address
Bit-Time 7 6 5 4 3 2 1 0
0 Reserved Cmd[5:0]=111110b
1 Addr[47:40]
2 Addr[55:48]
3 Addr[63:56]
4 SeqID[3:2] Cmd[5:0]
5 PassPW SeqID[1:0] UnitID[4:0]
6 Command-Specific
7 Command-Specific
8 Addr[15:8]
9 Addr[23:16]
10 Addr[31:24]
11 Addr[39:32]
4.5 Responses
4.5.1 Read Response (RdResponse)
A node that is the target of a request for data (such as Sized Read or Atomic RMW request)
returns a read response packet to the source followed by a data packet that contains the requested
data. The format of the read response packet is shown in Table 19.
Table 19. Read Response (RdResponse) Packet Format
Bit-Time 7 6 5 4 3 2 1 0
0 Isoc Rsv Cmd[5:0]: 110000
1 PassPW Bridge Rsv UnitID[4:0]
2 Count[1:0] Error0 SrcTag[4:0]
3 Rsv/RqUID Error1 Rsv Count[3:2]
The Count field encodes the size minus 1 (in doublewords) of the data packet, so that intermediate
nodes forwarding the response know how much data to expect. For doubleword read requests, the
Count is just taken from the request packet. For byte read requests, the data field always fits within
a single doubleword, so the Count field is always 0 (one doubleword). For Read-Modify-Write
requests, the Count field is always 1 (one quadword).
52
Fabric Operation Chapter 4
Page 53
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
The Error bits are used to indicate that an error occurred during the read. This can be due to the
accessed address being non-existent, an internal error in DRAM or a cache, or other problems. The
requested number of data elements are always driven to the chain, whether they are valid or not,
but the Error bits indicate that the data cannot be used. The different encodings of the error bits are
described in Table 20. A data packet with 1s in all data bit positions must follow a read response
packet with Master Abort indicated. A data packet containing corrupt data will follow a read
response with Data Error indicated. A data packet containing incomplete or invalid data will
follow a read response with Target Abort indicated.
Table 20. Error bit encodings
Error 1 Error 0 Meaning
0 0 Normal Completion
0 1 Target Abort: Request reached target but could not be
completed.
1 0 Data Error: Request completed by the target but data has
been corrupted.
1 1 Master Abort: No agent on the chain accepted the request.
The Isoc bit is set to indicate that this response has special bandwidth and latency requirements,
and must be set if the Isoc bit was set in the request. See Appendix D for details. The Isoc bit is
required to be maintained even when passing through a tunnel with Isochronous flow control
mode disabled. Host bridges should return the value of Isoc from the request when forwarding
peer-to-peer responses.
The SrcTag field is copied from the request that caused the response.
When Bridge=0, RqUID contains the two least significant bits of the requestor’s UnitID.
4.5.2 Target Done (TgtDone)
Target Done (defined in Table 21) signals that a transaction not requiring returned data (such as
Sized Write or Flush) has completed at its target.
Chapter 4 Fabric Operation
53
Page 54
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 21. Target Done (TgtDone) Format
Bit-Time 7 6 5 4 3 2 1 0
0 Isoc Rsv Cmd[5:0]: 110011
1 PassPW Bridge Rsv UnitID[4:0]
2 Rsv Error0 SrcTag[4:0]
3 Rsv/RqUID Error1 Rsv
The target can release its command buffer as soon as it issues TgtDone. A nonposted request will
result in either a RdResponse or a TgtDone, but not both. TgtDone also has the Isoc, Bridge,
PassPW, and Error bits, with the same behavior as the ones in RdResponse. Data Error in a
TgtDone indicates that the target received the request, but detected data corruption on the write
data it received.
The SrcTag field is copied from the request that caused the response.
When Bridge=0, RqUID contains the two least significant bits of the requestor’s UnitID.
4.6 I/O Streams
HyperTransport technology has the concept of I/O streams, which are groupings of traffic that can
be treated independently by the fabric.
Because no peer-to-peer communication exists within the fabric, and all packets travel either to or
from the host bridge, the traffic to or from each node in the fabric could, in theory, be treated
independently by the fabric, leaving the host bridge to manage interactions between streams.
Upstream requests contain the ID of the source node, and upstream responses contain the ID of the
node that generated the response. Therefore, UnitID may be used to identify I/O streams for
upstream packets. Note that Fence requests occupy all UnitIDs (see Section 4.4.4 for details).
Downstream responses contain the ID of the node to which the response is being sent. However,
downstream requests contain the ID of the host bridge, and not the ID of the node that is targeted
by the request. Therefore, it is impossible to determine independent I/O streams in downstream
request traffic, and it must be assumed that all downstream traffic (both requests and responses) is
in the same stream.
The host bridge is responsible for managing interactions between streams. No stream information
is propagated through the host bridge. The host bridge is responsible for maintaining ordering
within the host domain in whatever fashion is appropriate.
54
Fabric Operation Chapter 4
Page 55
HyperTransport Technology Consortium
A single physical node can be allocated multiple UnitIDs if the node generates multiple
independent streams of traffic. This allows more concurrency among the traffic to and from that
device. If allocating multiple UnitIDs is not done, all traffic to and from that device will be
ordered as a single stream, and knowledge of the possible concurrency will be lost.
HyperTransport™ I/O Link Specification Revision 1.05c
4.6.1 UnitID Clumping
A single UnitID only allows 32 outstanding nonposted transactions within the chain, because there
are only 5 bits of SrcTag for tracking them. If a device uses more than one UnitID to achieve
greater concurrency, the transactions in different UnitIDs are not required to be ordered together.
Additionally, the chain host cannot issue requests in UnitIDs other than 0 or nodes on the chain
will not accept them. UnitID Clumping addresses both of these shortcomings by allowing an
arbitrary number of consecutive UnitIDs to be treated as one (clumped) for the purposes of
ordering and routing (acceptance, forwarding, rejection). See sections 7.5.10.6 and 7.10 for the
definition of the registers that control Clumping. HyperTransport 1.05 devices must support
clumping in one of two ways: Passive only or Full support. Passive support is when a device keeps
packets with different UnitIDs ordered together, simply ignoring UnitID for upstream cycles. This
allows non-host devices to achieve greater concurrency without losing ordering when passing
through a Passive device. Full support requires a device to implement the Clumping Capability
block to control which UnitIDs are ordered and routed together. This allows hosts to achieve
greater concurrency by tracking more than 32 outstanding nonposted requests. For any device to
use clumping to obtain greater concurrency, it must implement full clumping support. Note that
use of passive clumping reduces the ability of devices in a chain to reorder packets and could
reduce system performance.
To enable clumping on a chain, after (or during) UnitID assignment, software will check each
node for the presence of a Clumping capability block, indicating full clumping support. If one
does not exist, software will attempt to set the node’s UnitID Reorder Disable feature bit to
determine if a node implements Passive support (bit is set) or no support (bit is clear). If a node
implements Full support, the UnitIDs that that node wants clumped will be indicated by bit 1 and
above in the Clumping Support register of the Clumping capability block. Software will OR
together the results of all the reads to create a combined Clumping “Mask”. If all the nodes on a
chain have indicated Full support, the Clumping Support register of the host can also be read and
Ored into the Clumping Mask. The Mask will then be written into the Clumping Enable registers
of all nodes on the chain.
Hosts may only clump UnitIDs 0 through 3 because only the two least significant bits of the
requestor’s UnitID are preserved in responses. This allows up to 128 outstanding nonposted
requests from the host.
In the example shown in Figure 4, the first node enumerated uses three clumps of UnitIDs: 2, 3 to
4, and 5 to 6. The second node uses a single clump of UnitIDs 7 to 10. The third node uses
UnitIDs 11 and 12 in a single clump, and the host uses UnitIDs 0 and 1 clumped.
Chapter 4 Fabric Operation
55
Page 56
HyperTransport Technology Consortium
11 12
1 1 1 0 0 1 1 0 1 1 1 1 1 0 0 0 1 0
HyperTransport™ I/O Link Specification Revision 1.05c
Blue ovals indicate UnitIDs
that are clumped together.
1st Node, BaseUID = 2
1 0 1 0
0 1 2 3
2nd Node, BaseUID = 7
0 1
3rd Node, BaseUID = 11
Host
0 0
0 1 2 3 4
Clumping Support
0 1
1 0
Clumping Mask
0 1 2 3 4 5 6 7 8 9 10
Figure 4. Clumping Configuration
4.7 Virtual Channels
HyperTransport technology supports three virtual channels of information:
• Posted Requests
• Nonposted Requests (reads, flushes, nonposted writes)
• Responses
Requests may cause responses to be issued by receiving nodes. Requests received by a host bridge
with the ActAsSlave bit clear may also cause downstream requests to be issued (peer-to-peer
reflection). Other nodes may not make accepting a posted request dependent on the ability of that
node to issue an outgoing request. Further, non-host nodes may not make acceptance of a
nonposted request dependent on the ability of that node to issue an outgoing nonposted request.
Additionally, these nodes may not make acceptance of any request dependent upon the receipt of a
response due to a request previously issued by that node. Non-host nodes may not make
acceptance of a response dependent upon the ability to issue a response or request. Finally, they
may not make issuing a response dependent upon the ability to issue a request or dependent upon
receipt of a response due to a previous request.
56
Fabric Operation Chapter 4
Page 57
HyperTransport Technology Consortium
All devices must guarantee that the three virtual channels are not capable of blocking each other
due to buffer management and routing issues, which is why each channel has command and data
buffer space separate from the other two. However, in order to properly maintain I/O ordering,
some rules are added which create dependencies between packets (in the same I/O stream) in
different virtual channels. See Chapter 6.
Hosts may make acceptance of a posted request dependent on their ability to issue a posted
request, but not upon the ability to issue nonposted requests or responses, and not upon receipt of a
response. Hosts may make acceptance of a nonposted request dependent upon issuing a response,
issuing a request, or upon receiving a response. Hosts may make acceptance of a response
dependent upon the ability to issue a posted request or a response, but not upon issuing a
nonposted request. Finally, hosts may make issuing a response dependent upon the ability to issue
a posted request.
Note that in a shared double-hosted chain, if the chain is not partitioned between the two host
bridges, there is the possibility of a deadlock. A deadlocking loop can be formed if peer-to-peer
requests are issued in opposite directions by two different intermediate nodes. Each reflected peerto-peer request coming out of a host bridge can be blocked behind a stack of requests targeting the
other host bridge. The host bridge will only be able to queue a finite number of peer-to-peer
requests in from the link without issuing one. Similarly, for a host bridge connected to two chains,
each of which is terminated by another host, a deadlocking loop can be formed if a device on each
chain is attempting to send peer-to-peer requests to each other in the direction away from the
common host bridge. See Section 4.1.1 for more information on double-hosted chains.
HyperTransport™ I/O Link Specification Revision 1.05c
HyperTransport technology includes support for an optional operating mode in which the number
of virtual channels is doubled to support Isochronous operation. See Appendix D for more
information.
4.8 Flow Control
HyperTransport technology receivers contain the following types of buffers:
• Nonposted Requests
• Posted Requests
• Responses
• Nonposted Request Data
• Posted Request Data
• Response Data
Request and response buffers contain enough storage to store the largest control packet of that
type. All data buffers can hold 64 bytes.
Chapter 4 Fabric Operation
57
Page 58
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 8 defines the virtual channels and the buffers used for each of the control packets.
These buffers are flow-controlled at the link level using a coupon-based scheme in which the
transmitter contains a counter for each type of buffer at the receiver. At link reset, the transmitter
clears its counters, and when reset deasserts, the receiver sends NOP packets to indicate how many
buffers of each type it has available. When the transmitter sends a non-info packet, it decrements
the associated counter, and when a particular counter contains a 0, the transmitter stops sending
packets to the associated buffer. When the receiver frees a buffer, it sends a NOP packet to the
transmitter, and the transmitter increments the associated counter.
A transmitter cannot issue a control packet that has an associated data packet unless the receiver
has both the appropriate control and data buffers available. If this rule is violated, one virtual
channel can block another and lead to deadlock, because commands with associated data packets
cannot be interleaved on the link.
HyperTransport technology supports an optional operating mode in which the number of virtual
channels and associated flow control buffer types is doubled in order to support high-priority
Isochronous communication. See Appendix D for details.
It is the responsibility of nodes generating requests to be able to accept the resulting responses
without other dependencies. Otherwise, the response and/or response data flow control buffers
may become filled with responses that are not yet ready to be accepted. This can be accomplished
by preallocating enough buffer space to hold the response to a request before sending that request.
In a HyperTransport bridge to a protocol that also requires requests to be accepted without
dependencies, preallocation is unnecessary. Due to peer-to-peer requests, host bridges are exempt
from this rule.
It is also required for deadlock avoidance that devices always be able to accept posted requests
without any other dependencies (such as issuing cycles back to the same chain or receiving
responses from the chain). Due to peer-to-peer requests, host bridges are exempt from this rule.
The format of the NOP packet is shown in Table 22. Bit 7 of bit-time 2 within the NOP packet is
used to allow link interface hardware to differentiate a HyperTransport I/O device from a host
device that implements a superset of the HyperTransport I/O protocol. Such a protocol could be
used for the purpose of communication between devices inside the host. The link transmitter of a
HyperTransport I/O device must always place a 0 in this bit position. The link receiver of a
HyperTransport I/O device may ignore the bit completely.
58
Fabric Operation Chapter 4
Page 59
HyperTransport Technology Consortium
Receiver
Latency
80 ns
Time to
send NOP
4 B*BW
Time to
send packet
Wire delay
10 ns*2
Notes:
For a minimal 2
For a 32
HyperTransport™ I/O Link Specification Revision 1.05c
Table 22. NOP Packet Format
Bit-Time 7 6 5 4 3 2 1 0
0 Rsv DisCon Cmd[5:0]: 000000
1 ResponseData[1:0] Response[1:0] PostData[1:0] PostCmd[1:0]
2 0 Diag Isoc Rsv NonPostData[1:0] NonPostCmd[1:0]
3 Rsv
Each 2-bit field in the packet indicates how many buffers of each type have become available.
Hence each 2-bit field can free zero, one, two, or three buffers. Receivers are not limited to having
three buffers of a particular type, and they can free up additional buffers by sending additional
NOP packets.
While the goal is to size each buffer at the receiver to bury the round-trip latency from the
transmitted packet to the returning NOP packet, this is not strictly required by this specification. It
is the responsibility of each device to guarantee that NOP packets cannot be prevented from being
issued due to transmission of other traffic, to avoid starvation of the far transmitter.
Internal Delay
20 ns
TX A
RX B
Internal Delay
RX A
TX B
30 ns
Receiver
Latency
50 ns
68 B*BW
-bit 400-Mbit/s link,
Max Bandwidth: 100 Mbytes/s
Total loop delay: 200 + 720 ns
Data in loop = 100 Mbytes/s * 920 ns, or 92 bytes
So, in this case, at least two buffers are needed.
-bit, 2-Gbit/s link,
Max bandwidth: 8 Gbytes/s
Total loop delay: 200 + 9 ns
Data in loop = 1672 bytes
So, at least 27 buffers would be needed to bury latency in
this case.
Figure 5. Example Data Buffer Sizing Calculation
Figure 5 illustrates how to calculate the size of data buffers that bury latency. For example:
If a transmitter receives more increments than it can keep track of, it must not allow its counter to
wrap, but must discard the extras (saturate). This has the effect that the link will use the maximum
number of buffers that both the transmitter and receiver can support. All transmitter counters must
be a minimum of four bits wide, allowing up to 15 buffers to be tracked without loss.
Chapter 4 Fabric Operation
59
Page 60
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
The Diag bit is used to indicate the beginning of a CRC testing phase, as described in Appendix G.
Everything following the NOP packet, until the conclusion of the current CRC test interval, is
ignored. This test feature is optional—receivers are not required to implement support for this test
mode. Support for this mode is indicated in bit 2 of the Feature Capability register, described in
Section 7.5.10.3.
The DisCon bit is set to indicate that the link transmitter is beginning an LDTSTOP# disconnect
sequence. When this bit is set, all the buffer-release fields in the packet must be 0. See Section 8.3
for details. While support for this bit is optional in non-x86 systems because the LDTSTOP#
feature is optional, it is recommended in all systems for greater interoperability.
The Isoc bit is set to indicate that the flow-control information in the associated packet pertains to
the Isochronous virtual channels. Isochronous flow-control information must only be sent and
utilized when the link has Isochronous flow control mode enabled, as described in Section 7.5.4.9.
4.9 Routing
HyperTransport technology has both directed and broadcast requests. Directed requests may travel
in either the posted or nonposted channel; broadcast requests travel only in the posted channel.
Directed packets are relayed down the chain until they reach their destination, where they are
absorbed. Broadcast packets are relayed down the entire length of the chain, but they are also
accepted at each node they pass through, and they are terminated by the node at the far end of the
chain. Broadcast packets can only be initiated by a host bridge.
An attempt to issue or forward a new packet into the end of the chain will result in one of the
rejection outcomes described in Section 4.9.3.
As described in Section 4.6, clumped UnitIDs are considered to be equivalent for the purposes of
routing and ordering.
4.9.1 Acceptance
A node will accept an incoming packet if any of the following are true:
• The packet is a Broadcast request.
• The packet is a directed downstream request (with a UnitID of 0 or any of the consecutive
UnitIDs clumped with 0,indicating it is from a host bridge) and (for packets with a Compat bit)
the Compat bit clear, to an address owned by this node.
• The packet is a directed downstream request with a set Compat bit, and this node is either the
subtractive decode device or a bridge to it.
60
Fabric Operation Chapter 4
Page 61
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
• The packet is a response with the Bridge bit set (indicating it is from a host bridge) and a
UnitID owned by this node.
• Tunnels must be able to accept downstream packets from either link in a double-hosted chain.
4.9.2 Forwarding
Whenever a node forwards a packet, it always sends the packet along the direction it was
previously traveling.
A node will forward an incoming packet to its outgoing link if any of the following are true:
• The packet is a Broadcast request.
• The packet is a directed downstream request with the Compat bit clear, to an address not
owned by this node.
• The packet is a directed downstream request with a set Compat bit, and this node is neither the
subtractive decode device nor a bridge to it.
• The packet is a directed upstream request (with a UnitID not consecutively clumped with 0,
indicating that it is from an interior node).
• The packet is a response with the Bridge bit set (indicating it is from a host bridge) and a
UnitID that does not match this node.
• The packet is a response with the Bridge bit clear (indicating it is from an interior node).
A HyperTransport technology device may receive a request from one link that should be
forwarded to the other link while its End of Chain and Initialization Complete Configuration
Space Register (CSR) bits are still clear. In this case, the Drop on Uninitialized Link bit defined in
Section 7.5.3.2.5 determines if the device will pend the request until the End of Chain or
Initialization Complete CSR bit becomes set (indicating that the initialization attempt has
completed), or reject the packet. See Section 12.3 for an example of an initialization sequence that
makes use of this requirement.
A packet being forwarded to a link interface that has its End of Chain bit set is rejected. See
sections 7.5.4.5 and 7.5.4.6 for the definitions of the End of Chain and Initialization Complete
CSR bits and more details on how they can affect forwarding.
4.9.3 Rejection
A device at the end of the chain is indicated by the End of Chain CSR bit (or by the Initialization
Complete bit clear when the Drop on Uninitialized Link bit is set). In that case, the device is
unable to forward packets or issue them in the direction of the unusable link. If a packet is
rejected, one of the following actions is taken instead, depending on the type of packet:
Chapter 4 Fabric Operation
61
Page 62
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
• Broadcast requests are silently dropped—they have successfully traversed the whole chain.
• Nonposted downstream-directed requests are responded to with a TgtDone (for Writes) or
Read Response (for Reads) packet with a Master Abort indicated and the Bridge bit clear. The
UnitID of the response may be either that of the requestor or that of the responding device.
Read responses return the requested number of doublewords with a data value of all 1 bits. See
Section 10.1.5 for more details.
• Nonposted upstream-directed requests are responded to with a TgtDone (for Writes) or Read
Response (for Reads) packet with a Master Abort indicated, the Bridge bit set, and a UnitID
matching that of the request, even when the responder is not the host of the chain. Read
responses return the requested number of doublewords with a data value of all 1 bits. See
Section 10.1.5 for more details.
• Response and posted request packets are dropped. See Section 10.1.5 for more details.
4.9.4 Host Bridges
Host bridges are always at the ends of the chain, and therefore never forward packets. However,
the acceptance of a packet by a host bridge will likely result in action within the host.
Host bridges take the following action upon receiving a packet:
1. Downstream directed requests must be coming from another host bridge on the far end of a
double-hosted chain. Type 0 configuration accesses to the device number specified in the
Device Number register (see Section 7.5.3.3.3) are directed to the bridge CSRs if the host
supports use in a double-hosted chain and the Host Hide bit is clear (see Section 7.5.3.3.5).
Optionally, the host bridge can also implement a memory or I/O space region addressable from
the far host bridge to be used for messaging in clustered systems. (A description of how this
would be used and what it would look like is beyond the scope of this specification.) In that
case, the bridge would respond to accesses to this area as if it were an interior node. The
responses would have the Bridge bit clear and the UnitID of the requestor, unless the
responding host has its Act as Slave bit set (see Section 7.5.3.3.6), in which case responses
will carry the value of the responding host's Device Number register, as defined in Section
7.5.3.3.3. All requests to addresses not included above are considered downstream accesses to
nonexistent addresses and rejected as specified in Section 4.9.3
2. Broadcast requests must be coming from another host bridge on the far end of a double-hosted
chain. They have successfully traversed the whole chain and may be silently dropped.
Optionally, the host bridge could also implement a region addressable by broadcasts from the
far host bridge. (A description of how this would be used and what it would look like is
beyond the scope of this specification.) In that case, the bridge would handle accesses to this
area as if it were an interior node, and route the broadcast to the appropriate internal target.
3. Upstream directed requests are from interior nodes, and they are accepted and handled by the
node logic. Address decode within the host determines the proper destination for the request.
This may be an internal destination, the same HyperTransport chain from which the request
62
Fabric Operation Chapter 4
Page 63
HyperTransport Technology Consortium
was received, or a different HyperTransport chain. When the request maps to a
HyperTransport chain, it is issued on that chain with a UnitID of 0 (or a UnitID consecutively
clumped with 0 if so enabled). A SrcTag (if nonposted) and SeqID are issued from the pool of
tags and SeqIDs available for downstream cycles on that chain. Because the cycle is now a
downstream cycle, the Compat bit may become set and the coherence bit cleared. All other
fields are passed through unchanged, except when a system host fabric implementation does
not allow it.
• A sequence of peer-to-peer requests in one I/O stream sent upstream with matching
sequence IDs must be reissued downstream with matching sequence IDs.
• Hosts may implement a compatibility chain, to which requests that map to no other target
may be sent. If no compatibility chain exists and the request maps to no internal target,
then the request has reached the end of chain, and is treated like a rejected cycle, as
described in Section 4.9.3. Note that HyperTransport host bridges that implement bridge
headers always have a target from the HyperTransport chain's point of view—the primary
bus of the bridge. If no target is found on this internal bus, that error occurs internally to
the bridge, not on the HyperTransport chain. In this case, an error response must be
handled according to the rules of Section 10.2.1.
HyperTransport™ I/O Link Specification Revision 1.05c
• HyperTransport technology hosts must maintain information about nonposted peer-to-peer
requests that were forwarded so that when responses are returned from the target chain,
responses with the correct attributes for the original request can be issued on the source
chain. (The implementation details of the structures used to maintain this information are
beyond the scope of this specification.)
• Hosts that support upstream configuration cycles must convert upstream Type 1 requests to
the current bus number to downstream Type 0 requests. Hosts that do not will abort the
cycle. Support for upstream configuration cycles is controlled by the enable bit in Section
7.5.10.8.
4. Responses with the Bridge bit set are silently dropped. This means that a host bridge tried to
respond to an interior node that did not pick up the response. The node can choose to log this
error and report it as a response error, described in Section 10.1.7.
5. Response packets with the Bridge bit clear are responses to requests issued by this bridge. The
host bridge will match this to one of its outstanding requests. If no match exists, the node can
choose to log this error and report it as a response error, described in Section 10.1.7.
4.9.5 Fairness and Forward Progress
In order to issue packets, a node must insert them into the stream of traffic that it is forwarding. A
node must guarantee that forward progress is always made by not allowing forwarded and injected
traffic to starve each another. Tunnels are required to implement the method described in this
section of assuring fair access to the chain for all units, approximating the round-robin behavior of
a fair bus. Some HyperTransport technology devices may be used in applications where the
fairness consideration is not relevant. One such example is a simple Southbridge that is always
placed at the end of a HyperTransport chain.
Chapter 4 Fabric Operation
63
Page 64
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
4.9.5.1 Policy
Each unit is allowed to insert packets into a busy link at a rate matching that of the heaviest unit
inserting traffic through it. In addition, the unit can freely use any idle time on the link. This
property must be met over a window in time small enough to be responsive to the dynamic traffic
patterns, yet large enough to be statistically convergent. In order for a system of units to behave
consistently, each unit must implement this policy using the same algorithm as described below.
Generation of Info packets (NOP and Sync) is not restricted by this algorithm, since they exist on
a per-link basis and are not forwarded.
4.9.5.2 Algorithm
The algorithm consists of two parts.
• The first is the method used to calculate the insertion rate the unit can use.
• The second governs how the unit achieves that insertion rate.
This algorithm must be implemented independently for both the upstream and downstream
direction to support double-hosted chain configurations. The algorithm requires no dedicated
control or status registers and has no configurable parameters.
To calculate the insertion rate, the maximum packet-forwarding rate must be deduced for the
heaviest downstream unit. This is done by implementing 32 3-bit counters, one for each potential
downstream UnitID as well as a single 8-bit counter. Clumped UnitIDs are considered separate for
the purposes of fairness. At reset, all counters are reset to 0. When a packet is forwarded the 3-bit
counter corresponding to the packet’s UnitID is incremented. The 8-bit counter is incremented
once for every forwarded packet. When one of the 3-bit counters overflows, the value of the 8-bit
counter (post increment) is captured (hereafter referred to as the denominator). All counters are
then cleared. The packet rate of the worst-case downstream unit has now been calculated and is
equal to 8/denominator. On average, the unit can insert eight packets for every ‘denominator’
packets forwarded. This insertion should be paced and not inserted as bursts. Packets can always
be inserted when there are no packets waiting to be forwarded. The denominator register is set to 1
on reset.
To insert, the unit has an 8-bit counter referred to as Window, which at reset is set to 1. It also has
a 1-bit register, referred to as Priority, that is cleared to 0 at reset. When a unit has packets ready to
be sent on the outbound links, it decides which to send based on the following cases:
• Forward packet to send and no local packet to send—The forward packet is sent and the
Window register is decremented.
• No forward packet to send and a local packet to send—The local packet is sent and the Priority
register is cleared.
64
Fabric Operation Chapter 4
Page 65
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
• Both forward packet and local packet to send—If the bit in the Priority register is set, the local
packet is sent and the Priority bit is cleared. Otherwise, the forward packet is sent and the
Window register is decremented.
Whenever the Window register is decremented to 0, its next value is recalculated and the Priority
bit is set. In order to achieve non-integral insertion rates, the new value of the Window register
must be loaded probabilistically. Each unit will implement a 9-bit linear feedback shift register
using the polynomial x9+x4+1. It is advanced once every time the Window register value is
recalculated. The Window register is loaded with (denominator+LFSR[2:0]) >> 3.
Packets with the Chain bit set are ignored for counting both forwarded and inserted packets, such
that each chained set of packets are counted as a single item.
4.9.5.3 Implementation Note
Care must be taken in implementing the packet insertion logic in order to avoid a potential
starvation problem. The packet inserter is basically a two-input arbiter between issued packets and
forwarded packets. The packets to this arbiter are generated when there is a packet ready to go
from one of these sources, and there are free buffers (as indicated by buffer release messages) at
the other end of the link to receive the packet. It is possible that there is one packet to be issued
and one to be forwarded, both in the same virtual channel and therefore requiring the same buffer
type(s). If the forwarded packet is chosen and there is only one buffer of the needed type free, the
issued packet cannot be transmitted. When the fairness logic next allows a packet to be inserted, a
packet from a different virtual channel can be chosen, allowing the priority of the packet inserter
to swing back to forwarding. Upon arrival of the buffer release message that would allow the
blocked packet to go, the packet no longer has priority in the inserter, and therefore cannot go. If
another packet in the same channel is forwarded before priority changes back to inserting, this
situation can persist, starving packet insertion in a particular virtual channel.
Chapter 1 Fabric Operation
65
Page 66
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
5 Addressing
The HyperTransport™ technology address map is shown in Table 23. The bulk of the address
space can be used for either memory or memory-mapped I/O. The partitioning of this space into
regions for each is implementation-specific. Unlike PCI, configuration and I/O accesses are
performed with the same read and write commands used to access memory, with the upper address
bits identifying the accesses instead of special command codes.
Table 23. HyperTransport™ Technology Address Map
Base Address Top Address Size Use
0000_0000_0000_0000h 0000_00FC_FFFF_FFFFh 1012 Gbytes System Memory/
Memory-Mapped I/O
0000_00FD_0000_0000h 0000_00FD_F8FF_FFFFh 3984 Mbytes Interrupt/EOI
0000_00FD_F900_0000h 0000_00FD_F90F_FFFFh 1 Mbyte Legacy PIC IACK
0000_00FD_F910_0000h 0000_00FD_F91F_FFFFh 1 Mbyte System Management
0000_00FD_F920_0000h 0000_00FD_F92F_FFFFh 1 Mbyte Reserved – x86
0000_00FD_F930_0000h 0000_00FD_FBFF_FFFFh 45 Mbytes Reserved
0000_00FD_FC00_0000h 0000_00FD_FDFF_FFFFh 32 Mbytes I/O
0000_00FD_FE00_0000h 0000_00FD_FFFF_FFFFh 32 Mbytes Configuration
0000_00FE_0000_0000h 0000_00FE_1FFF_FFFFh 512 Mbytes Extended Config/
Device Message
0000_00FE_2000_0000h 0000_00FF_FFFF_FFFFh 7680 Mbytes Reserved
0000_0100_0000_0000h FFFF_FFFF_FFFF_FFFFh ~16Exabytes System Memory/
Memory-Mapped I/O
While the interrupt, IACK, and system management spaces may not be used by all devices, they
must not be used for any other function, and all devices must forward them properly.
Some hosts only recognize interrupts with Address[31:24]=F8h.
Writes to configuration and I/O space must be nonposted and cannot cross a doubleword
boundary. Posted writes or broadcasts to configuration or I/O space may result in undefined
operation. Accesses of more than one doubleword to configuration or I/O space may either be
handled correctly by a target or target aborted.
66 Addressing
Chapter 5
Page 67
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Similarly, writes to interrupt and system management spaces must be posted. Nonposted writes to
interrupt or system management space may result in undefined operation.
Writes or broadcasts to IACK space may result in undefined operation. Upstream accesses to
IACK space may result in undefined operation.
Nonposted requests to the extended configuration space access configuration registers. Posted
writes to the extended configuration space are device messages. Device messages are routed by
bus and device number just like configuration accesses, and also have type 0 and 1 cycles. The
device and function number determine their final destination within the targeted node.
HyperTransport 1.05 devices must allow access to the bottom 256 bytes of register space per
function through both the standard configuration space and the extended configuration space.
HyperTransport technology devices must have address windows aligned on 64-byte boundaries to
guarantee that a maximum-size request will not cross a device boundary.
Accesses to reserved spaces are not accepted by address targets. Tunnels forward them if possible;
otherwise, they are treated as normal end of chain packets.
Chapter 1
Addressing 67
Page 68
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
6 I/O Ordering
This chapter explains the ordering rules for all three types of I/O traffic—peer-to-peer, DMA, and
PIO. Peer-to-peer traffic is traffic that has both its requester and target on the HyperTransport™
I/O link. Since HyperTransport technology does not allow peer-to-peer traffic directly, all peer-topeer traffic (whether to the same chain or not) goes upstream into the host and then back
downstream. For purposes of ordering, the upstream and downstream legs are considered
independently.
These ordering rules only apply to the order in which operations are seen by targets at the same
level of the fabric hierarchy. Consider two ordered peer-to-peer Write requests issued by a
HyperTransport I/O device to two different targets on different HyperTransport I/O chains. The
ordering rules on the originating HyperTransport chain ensure that the two Writes reach the host
bridge in the appropriate order. The host is responsible for ensuring that the two Writes reach their
target host bridges in the correct order. However, beyond that point, the Writes are in independent
chains, and there is no assurance about the order in which they will reach their final target. If an
I/O device requires assurance of final completion, it must have a way of polling the target device
to determine that the first Write has been observed before issuing the second Write, or it must use
nonposted Writes.
Ordered operations that return responses (Reads or nonposted Writes) are required to complete at
the target in the correct order, but no assurance is made about the order in which the returning
responses will be received. All HyperTransport I/O devices must be able to accept responses out
of order or restrict themselves to one outstanding nonposted request. A bridge that is between a
HyperTransport technology device and an I/O protocol that requires responses to be returned in
order must provide sufficient buffering to be able to reorder as many responses as it may have
outstanding requests. Devices must issue responses to nonposted requests only after the results of
those requests are globally visible.
HyperTransport supports the same producer-consumer ordering model as PCI when the PassPW
and bit is 0. In this model, a producer device anywhere in the system can generate data and modify
a flag to indicate data availability to a consumer of the data anywhere in the system. The flag and
data may each be located at the producer device, consumer device, or host memory. They are not
required to be located in the same device as long as the consumer waits for the flag read response
before issuing the data read. In the case where the consumer issues two ordered reads with nonzero sequence Ids (not waiting for the flag read response before sending the data read) producerconsumer ordering is only supported when the flag and data are co-located in the same device. The
ordering rules described in this section ensure that if the flag is modified after the data has been
made available, a read of the flag by the consumer will ensure that all data can be read.
68
I/O Ordering Chapter 6
Page 69
HyperTransport Technology Consortium
When the PassPW bit is 1, these ordering rules can be relaxed for applications where the flag and
data are restricted in their locations. With the use of nonzero sequence IDs, HyperTransport links
can maintain ordered sequences within otherwise unordered virtual channels.
To maintain compatibility with protocols that allow write combining, the individual data beats of a
posted request must be executed in the transmitted order if other accesses to the destination
addresses spanned by the request can occur between the individual beats of an access. Similarly, if
SeqID is nonzero for a nonposted request, and other accesses to the destination addresses spanned
by the request can occur between its individual beats, the beats of the request must be executed in
order.
HyperTransport ordering semantics ensure that a configuration access followed by a second
ordered transaction are delivered to a device in order. HyperTransport devices should ensure that a
subsequent ordered read to the same configuration location as a previous configuration access
results in the latest written data. However, devices are not required to ensure that all side effects of
the first configuration access are visible before receiving the second. Software can ensure the side
effects of the first configuration access have become visible by waiting for the response to the first
nonposted configuration access before issuing the second transaction.
HyperTransport™ I/O Link Specification Revision 1.05c
6.1 Upstream I/O Ordering
HyperTransport technology recognizes three types of traffic—posted requests, nonposted requests,
and responses—each in a separate virtual channel. These three types of traffic can be distinguished
by their command encoding. Requests have a sequence ID (SeqID) tag. Requests in the same I/O
stream and virtual channel with matching non-zero SeqIDs are considered part of a strongly
ordered sequence. Sequences are designed to support groups of HyperTransport technology
transactions generated by a single request on the source I/O bus. Requests and responses both have
a May Pass Posted Writes (PassPW) bit.
For definitions of I/O streams and virtual channels, see sections 4.6 and 4.7, respectively.
HyperTransport technology has the following upstream ordering rules:
1. Packets from different sources are in independent I/O streams and with the exception of the
Fence requests, have no ordering guarantees. Devices receiving packets in different I/O
streams may reorder them freely. If the UnitID Reorder Disable bit in the Feature register is
set, then all I/O streams must be ordered together.
2. Packets in the same I/O stream and virtual channel that are part of a sequence (having
matching nonzero SeqIDs) are strongly ordered (regardless of PassPW) and may not pass each
other. Devices receiving them must keep them strongly ordered.
3. Packets in the same I/O stream, but not part of the same ordered sequence, use the passing
rules listed in Table 24.
Chapter 6 I/O Ordering
69
Page 70
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 24. Packet Ordering Rules
Row Pass Column?
Posted Request, PPW=0
Posted Request, PPW=1
Nonposted Request, PPW=0
Posted Request Response
PassPW=0 PassPW=1
No No Yes Yes Yes
Yes/No No2 Yes Yes Yes
No No Yes/No Yes/No Yes/No
Nonposted
Request
PassPW=0 PassPW=1
Nonposted Request, PPW=1 Yes/No Yes/No Yes/No Yes/No Yes/No
Response, PPW=0
No No Yes No2 No2
Response, PPW=1 Yes/No1 Yes/No1 Yes Yes/No No2
Notes:
1. HyperTransport technology implementations are strongly encouraged to allow responses with PassPW set to
pass posted requests. However, they cannot rely upon this behavior system-wide to ensure deadlock-free
operation. Allowing responses with PassPW set to pass posted writes creates more deterministic latency on
behalf of Isochronous read traffic. See Appendix D for more details.
2. These “No” cases only enforce ordering of transactions within the chain. Hosts and switches are allowed to
treat them as”Yes/No” when the two packets have different destinations.
3. Ordering is not affected by the setting of the Chain bit in posted requests. The Chain bit only affects the
insertion of posted requests when other posted requests are being forwarded.
No—Indicates the subsequently issued transaction is not allowed to complete before the previous transaction to
preserve ordering in the system. This implies an interaction between the otherwise independent virtual channels within
HyperTransport technology.
Yes—Indicates the subsequently issued transaction must be able to pass the previous transaction, or deadlock may
occur. This means that the packet type given in the column cannot be permitted to block the packet type given in the
row at any point in the HyperTransport fabric or host.
Yes/No—Indicates the subsequently issued transaction may optionally be allowed to complete before the previous
transaction if there is any advantage to doing so. There are no ordering requirements between the two transactions.
However, support for reordering is not required—failure to reorder the packets will not lead to deadlock.
6.2 Host Ordering Requirements
The host bridge and host system are required to preserve the ordering of transactions in the virtual
channels provided in the HyperTransport I/O fabric as defined in Section 6.1, and to guarantee that
transactions that are ordered within the HyperTransport fabric are ordered within the host. This
70
I/O Ordering Chapter 6
Page 71
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
means that, for an ordered pair of transactions, the second transaction cannot take effect in the host
fabric (capturing data for a read to an internal target, exposing new data for writes and reads with
side effects to internal targets, or queuing peer-to-peer I/O for transmission to an external target)
until the first transaction has reached its ordering point. The definition of this ordering point
depends on the type of transactions in the ordered pair and the relationship of their targets. In the
case of peer-to-peer I/O operations, the host only guarantees that the first operation has been
queued for issue on its target link, and ordered with respect to requests from all other sources
(Globally Ordered); it does not indicate whether the operation has reached its final target device.
Read or Write accesses from HyperTransport technology are treated differently depending on the
target space within the host to which they are aimed. Accesses to cacheable system memory within
the host have the strongest set of ordering requirements. Accesses to noncacheable regions
(uncacheable system memory, I/O space, or memory-mapped space on an I/O device) have weaker
requirements. Accesses to the reserved interrupt or system management ranges have their own
special ordering requirements. The rules governing the host’s processing of ordered
HyperTransport I/O transactions are expressed in Table 25.
There are two defined ordering points, Globally Ordered (GO), and Globally Visible (GV). Table
25 defines what ordering point the first request in an ordered pair must reach before the second
request can take effect.
Table 25. Host Ordering Rules
First Command
Second Command
Second Command
Waits for the First
Command to Be:
Cacheable Write Cacheable Write GV
Cacheable Write Cacheable Rd GO
Cacheable Read Cacheable Read or Write GO
Non-Cacheable Non-Cacheable GO
Cacheable Write Non-Cacheable GV
Cacheable Read Non-Cacheable GO
Non-Cacheable Cacheable GO
Cacheable Write Flush/Interrupt/
GV
System Management/Response
Cacheable Read Flush/Interrupt/
No wait requirements
System Management/Response
Non-Cacheable Flush/Interrupt/
GO
System Management/Response
Chapter 6 I/O Ordering
71
Page 72
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
First Command
Second Command
Second Command
Waits for the First
Command to Be:
Flush/Response Any No wait requirements
Interrupt/System Management Fence or Response GV
Interrupt/System Management Any but Fence or Response No wait requirements
Posted Cacheable Fence GV
Posted Non-Cacheable Fence GO
Any Nonposted Fence No wait requirements
Fence Any GV
Notes:
Globally Ordered (GO)—The first transaction has reached a point where it is assured to be observed in the correct
order (relative to the second transaction) from any observer in the HyperTransport fabric. The two transactions are
assured to complete in order, but have not necessarily completed yet, so sideband access mechanisms such as cache
agents will not necessarily receive the correct results.
Globally Visible (GV)—The first transaction is visible to all observers. That is, any access mechanism will return the
new data. This means that in addition to being globally ordered, all side effects (such as cache state transitions)
initiated by the first transaction have completed.
Globally Visible implies Globally Ordered, so a host may use a more restrictive rule in some cases to simplify the
implementation. In the absence of sideband access mechanisms such as caching agents in the host, they are
equivalent. Both ordering points apply only to destinations in the chain host. They do not include effects caused by
the transactions outside the host fabric.
These ordering rules apply only to the distribution of Interrupt and System Management messages to all applicable
devices in the system, not necessarily to completion of all actions implied by them, such as service of an interrupt or
a power-state change.
6.2.1 Host Responses to Nonposted Requests
The host cannot generate a response to a nonposted request until all side effects of the request are
globally visible. For a memory request this means that all cache state transitions initiated by the
request have been completed. For I/O requests, this means that data writes or read side effects
have occurred. A response to a nonposted request implies that all previous ordered requests to
memory are globally visible. It also implies that all previous ordered requests to I/O have been
globally ordered, but it cannot be assumed that they are globally visible.
6.3 Downstream I/O Ordering
The rules for downstream ordering are the same as those for upstream ordering, with the exception
that I/O streams are identified by the target of the transaction, rather than the source. The same
72
I/O Ordering Chapter 6
Page 73
HyperTransport Technology Consortium
three virtual channels exist. However, UnitID may not be used to identify unique I/O request
streams in the downstream direction, so it must be assumed that all downstream traffic is in the
same stream. This asymmetry in the definition of I/O streams for upstream and downstream traffic
is why it is important for a node to be able to differentiate upstream responses from downstream
responses. The Bridge bit is used in the response packet for this purpose.
Once a node has accepted a packet, it has been separated from forwarded traffic and no longer
must be ordered with forwarded traffic.
The host must also guarantee that peer-to-peer traffic that was part of an ordered sequence when
received is also emitted downstream as an ordered sequence.
HyperTransport™ I/O Link Specification Revision 1.05c
6.4 Ordering in Sharing Double-Hosted Chains
In general, upstream traffic and downstream traffic moving in the same direction along a
HyperTransport chain have no ordering dependencies with respect to each other, as they will be in
different I/O streams. The exception is the case of communication directly between host bridges at
opposite ends of a double-hosted chain, as defined in Section 4.1.1. In this case, requests from one
host bridge to the other are always traveling downstream, and responses from that host bridge are
traveling upstream.
In the event that one host bridge (bridge A) issues a posted write to the other (bridge B), and
bridge B issues a read request to A, the read response will be traveling in the same direction as the
posted write. Despite the fact that the request is moving downstream and the response is moving
upstream, both must be treated as being in the same I/O stream (the response must push the
request if PassPW is clear) in order to support producer/consumer communication between the
hosts.
In this case, both the request and response will contain a Clumped UnitID of 0. Therefore, this
requirement can be supported simply by doing ordering checks based solely on Clumped UnitID
for upstream responses, and excluding information about whether the request they are checking
against is moving upstream or downstream. If a host has its Act as Slave bit set (defined in Section
7.5.3.3.6), the Clumped UnitID of requests and responses from it will not be 0. However, the
requirement to maintain ordering still exists, and the use of Clumped UnitID to achieve this can be
maintained.
Chapter 1 I/O Ordering
73
Page 74
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7 Configuration Accesses
HyperTransport™ technology implements configuration space similarly to PCI, as defined in the
PCI Local Bus Specification, Revision 2.3. HyperTransport technology devices and bridges
(including host bridges) must implement appropriate PCI configuration headers. Buses and
devices are numbered in a fashion that maps into PCI bus and device numbers. Configuration
software should be able to accomplish configuration of HyperTransport chains in a way that is
indistinguishable from an equivalent PCI bus hierarchy. Configuration space is mapped to a
predefined region of the HyperTransport technology system address space. See Chapter 5 for
details.
7.1 Configuration Cycle Types
PCI uses two types of configuration cycles, type 0 and type 1. The two types are needed because it
must be possible to access the configuration space of devices on a bus without the devices
knowing on which bus they are located.
Type 0 cycles are used to access devices on the current bus. They contain a function number and
register number. The bus number is implicitly the current bus, although it is transmitted so that
devices can be aware of their location within the system (such as for sending device messages).
The device number is indicated by the IDSEL# pins, which are asserted as appropriate by the
bridge. Therefore, PCI devices do not need to know their bus number or device number in order to
respond to configuration accesses.
Type 1 cycles are used to transmit configuration cycles over intermediate buses. They contain bus
number, device number, function number, and register number fields. Bridges forward type 1
cycles through the bus hierarchy and translate them to type 0 cycles when driving them onto their
final destination bus. Host bridges can optionally implement the capability to transmit PCI special
cycles to remote buses using Device 31, Function 7, Register 0, Type 1 configuration cycles.
HyperTransport technology also requires two types of configuration cycles, for the same reasons
as PCI.
A HyperTransport technology Type 0 access is performed by issuing a RdSized or nonposted
WrSized request with an address of the form shown in Table 26. They are only issued by host
bridges and therefore always travel downstream. Unlike PCI, HyperTransport technology Type 0
accesses contain the device number, because all HyperTransport technology devices know what
their device numbers are. HyperTransport technology has no analog of the IDSEL# signals. Host
bridges that support double-ended links will respond to Type 0 accesses on their secondary
interfaces at the Device Number specified in Host Interface Command register. See Section
74
Configuration Accesses Chapter 7
Page 75
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.5.3.3.3. Note that, in a double-hosted link, this implies that both bridges could be responding to
the same address—which one you are talking to is determined by which direction the packets are
traveling. This function is only intended to be used by system-sizing firmware.
Table 26. HyperTransport™ Technology Type 0 Address Format
63 24 23 16 15 11 10 8 7 2
0000_00FD_FEh Bus Number Device Number
Function
Number
Register Number
A HyperTransport technology Type 1 access is performed by issuing a RdSized or nonposted
WrSized request with an address of the form shown in Table 27. In general, Type 1 accesses are
issued by host bridges. However, Type 1 accesses can also be issued by HT slave devices, to do
peer-to-peer configuration. Support for peer-to-peer configuration is optional in hosts. If
supported, the host accepts the configuration cycle, or reflects it, depending on the bus number. If
the cycle targets the local chain, the host translates it from a Type 1 to a Type 0 cycle. If the cycle
targets the host itself, the host accepts it. If the host does not support upstream configuration
accesses, it responds to the cycle with an abort. Support for upstream configuration access is
controlled by the configuration bit in Section 7.5.10.8.
Table 27. HyperTransport™ Technology Type 1 Address Format
63 24 23 16 15 11 10 8 7 2
0000_00FD_FFh Bus Number Device Number
Function
Number
Register Number
RdSized and WrSized configuration accesses of greater than one doubleword are not supported.
Posted configuration writes are not allowed and their effect is undefined.
Extended Type 0 and 1 accesses can address both regular and extended configuration registers
with nonposted requests and carry device messages with posted requests. See Chapter 13 for more
on Device Messaging. Note that older host hardware and software will be unable to generate
extended configuration accesses so a device should not depend on them for basic functions, such
as the interface capability blocks. A device may provide an access mechanism for older hosts with
the extended configuration space capability block, specified in Section 7.11.
Table 28. Extended HyperTransport™ Technology Type 0 Address Format
63 28 27 24 23 16 15 11 10 8 7 2
0000_00FE_0h
Upper Register
Number
Bus Number Device Number
Function
Number
Lower Register
Number
Chapter 7 Configuration Accesses
75
Page 76
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 29. Extended HyperTransport™ Technology Type 1 Address Format
63 28 27 24 23 16 15 11 10 8 7 2
0000_00FE_1h
Upper Register
Number
Bus Number Device Number
Function
Number
Lower Register
Number
7.2 Configuration Space Mapping
7.2.1 Function and Register Numbering
The numbering of functions and registers within a device is device-specific, except that every
implemented device number must have a function 0 containing a standard configuration header
that identifies the device. Certain other standard Configuration Space Registers (CSRs) are
required by the HyperTransport technology specification.
7.2.2 Device Numbering
HyperTransport technology devices are identified by UnitIDs, which range from 00h to 1Fh. A
single physical device can own multiple UnitID values. Every HyperTransport technology device
owns the device numbers that correspond to its UnitIDs, and it must implement a configuration
space (with configuration header) at the Device Number equal to its Base UnitID value. It may
choose to implement configuration spaces corresponding to any number of its remaining UnitIDs,
including none. Each implemented space must contain an appropriate configuration header.
Unimplemented spaces must not be responded to by the device. Accesses to these device numbers
will not be accepted by any device on the HyperTransport chain and therefore will receive a
response with a Master Abort indicated.
As described in Appendix E.3, some systems require a compatibility chain that is enumerated as
Bus 0. In such a system, any configuration space registers implemented in the Bus 0 space by the
host must appear in the uppermost device numbers on that bus. In such a case, the number of
devices (and therefore UnitIDs) available to the HyperTransport chain implementing Bus 0 is
reduced accordingly. If the host were to occupy Device 0, then the HyperTransport chain could
not be enumerated.
As described in Appendix F.4, some legacy operating systems may require AGP configuration
registers to be implemented in the Bus 0, Device 0 range, in which case the host’s configuration
space registers must appear somewhere other than that range. Even though the AGP CSRs appear
in Device 0, UnitID 0 is still reserved for host use so that devices can distinguish upstream cycles
from downstream cycles, as described in Section 4.9. This requires the AGP bridge to consume at
least one extra UnitID, as it cannot use UnitID 0.
76
Configuration Accesses Chapter 7
Page 77
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
For HyperTransport technology hosts that also implement AGP configuration space and require
legacy operating system compatibility, the host may either:
• Make configuration space relocatable so that HyperTransport bus enumeration may occur.
• Place configuration space in the uppermost device numbers and provide a mechanism for
hiding Device 0 registers to allow HyperTransport bus enumeration.
The means to accomplish either of these two actions, or perhaps other solutions to this problem,
are implementation-specific and beyond the scope of this specification.
7.2.3 Bus Numbering
Each HyperTransport chain in the system is assigned a single bus number. For double-hosted
physical chains, which are logically partitioned between two host bridges, each logical chain has a
separate bus number.
Bus numbers are assigned by system initialization software at reset and follow the conventions
used for PCI bus numbering, which require that the bus tree be numbered in a depth-first fashion.
No distinction is made between PCI buses and HyperTransport chains in the numbering.
7.2.4 Software View of Extended Configuration Space
The format of the extended configuration space request packet addresses in Table 28 and Table 29
was chosen to make extension of existing hardware easy. In order to make efficient use of the new
configuration space, the software view of it should be a flat, contiguous mapping of devices into
the memory map, with the format shown in Table 30. The host bridge is responsible for mapping
software accesses into the correct packet format.
Table 30. Extended HyperTransport™ Technology Software Address Format
27 20 19 15 14 12 11 2
Bus Number Device Number
Function
Number
Register Number
7.3 HyperTransport™ Technology Device Header
Devices that sit on a HyperTransport technology chain and perform non-bridging functions
implement device headers, as shown in Table 31. The fact that this is a device header is contained
in the header type register. Connecting to two links within a HyperTransport chain does not
constitute a bridging function. Fields in a HyperTransport technology device header are the same
as defined in the PCI Local Bus Specification, Revision 2.3, with the exceptions listed in the
following sections.
Chapter 7 Configuration Accesses
77
Page 78
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 31. HyperTransport™ Technology Device Header Format
31 24 23 16 15 8 7 0
Device ID Vendor ID
Status Command
Class Code Revision ID
BIST Header Type Latency Timer Cache Line Size
Base Address Registers
Cardbus CIS Pointer
Subsystem ID Subsystem Vendor ID
00h
04h
08h
0Ch
10h
14h
18h
1Ch
20h
24h
28h
2Ch
Expansion ROM Base Address
Reserved Capabilities Pointer
Reserved
Max_Lat Min_Gnt Interrupt Pin Interrupt Line
Note: Shaded registers contain minimum-required read-write bits. Other registers are read-only or
contain only device-dependent bits.
30h
34h
38h
3Ch
Each field is defined as readable and writeable by software (R/W), readable only (R/O), or
readable and cleared by writing a 1 (R/C). Additionally, each field is affected by cold reset only or
by both cold and warm reset.
7.3.1 Command Register: Offset 04h
The following bits are implemented in the Command register of a HyperTransport technology
device. All other bits are not applicable and must be hardwired to 0.
7.3.1.1 I/O Space Enable (Bit 0): R/W: Warm Reset to 0
This bit must be set for the device to accept any non-compatibility accesses (requests with the
Compat bit clear) to the I/O address space, as given in Chapter 5. If this device is a subtractivedecode device, requests with the Compat bit set will still be accepted, regardless of the state of this
bit.
78
Configuration Accesses Chapter 7
Page 79
HyperTransport Technology Consortium
7.3.1.2 Memory Space Enable (Bit 1): R/W: Warm Reset to 0
This bit must be set for the device to accept any non-compatibility accesses to the memory address
space, as given in Chapter 5. If this device is a subtractive-decode device, requests with the
Compat bit set will still be accepted, regardless of the state of this bit.
7.3.1.3 Bus Master Enable (Bit 2): R/W: Warm Reset to 0
This bit must be set to allow the device to issue memory or I/O requests onto the HyperTransport
chain. Requests from other devices on the chain may still be forwarded, independent of the state of
this bit.
7.3.1.4 Data Error Response (Bit 6): R/W: Warm Reset to 0
This bit must be set to allow the device to set the Master Data Error bit of the status register. In
bridges, this bit allows Data Error to be set on TgtDone responses that are being forwarded from
the secondary bus to the primary bus. It has no effect on forwarding posted requests or
RdResponses. Data Error indications from other devices within a chain may always be forwarded,
independent of the state of this bit. Note that bridges accept a cycle on one bus and issue the cycle
on another bus; unlike a tunnel, which simply forwards cycles it does not accept.
HyperTransport™ I/O Link Specification Revision 1.05c
7.3.1.5 SERR# Enable (Bit 8): R/W: Warm Reset to 0
If this bit is set, the device will flood all its outgoing links with sync packets when it detects an
error that causes a sync flood (see Section 10.2.4). If this bit is clear, the device may not generate
sync packets except as part of initial link synchronization, although it can still propagate them
from one link to the other within a chain.
7.3.1.6 Interrupt Disable (Bit 10): R/W: Warm Reset to 0
If this bit is clear, the device is allowed to assert a legacy INTx pin (if present) or send an INTx
assertion message. If this bit is set, a device is not allowed to assert INTx. If a device has already
sent an INTx assertion message and this bit is set, the device must send an INTx deassertion
message. See Section 8.4 for more on INTx messages.
7.3.2 Status Register: Offset 06h
The following bits are implemented in a HyperTransport technology device’s Status register. All
other bits are not applicable to HyperTransport technology and must be hardwired to 0.
Chapter 7 Configuration Accesses
79
Page 80
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.3.2.1 Interrupt Status (Bit 3): R/O
This bit reflects the state of legacy INTx logic in the device, regardless of the state of the Interrupt
Disable bit in the Command register. A 1 indicates that the interrupt source is active. A 0 indicates
that the interrupt is inactive.
7.3.2.2 Capabilities List (Bit 4): R/O
This read-only bit will always be set to 1, to indicate that the device has a capabilities list
containing (at least) configuration information specific to HyperTransport technology.
7.3.2.3 Master Data Error (Bit 8): R/C: Cold Reset to 0
This bit is set by a device that has the Data Error Response bit set in the Command register and
issues a posted request with the Data Error bit set or accepts a response with a Data Error
indicated. This bit is not set if only forwarding packets with the Data Error bit set. Note that
bridges accept a cycle on one bus and issue the cycle on another bus; unlike a tunnel, which
simply forwards cycles it does not accept.
7.3.2.4 Signaled Target Abort (Bit 11): R/C: Cold Reset to 0
This bit is set by a HyperTransport technology device that returns a Target Abort for a transaction
addressed to it. (As described in Section 10.2.1.)
7.3.2.5 Received Target Abort (Bit 12): R/C: Cold Reset to 0
This bit is set by a HyperTransport technology device that receives a Target Abort for a request it
issued. (As described in Section 10.2.1.)
7.3.2.6 Received Master Abort (Bit 13): R/C: Cold Reset to 0
This bit is set by a HyperTransport technology device that receives a Master Abort for a request it
issued. (As described in Section 10.2.1. )
7.3.2.7 Signaled System Error (Bit 14): R/C: Cold Reset to 0
This bit is set by a HyperTransport technology device that has flooded the link with Sync packets
to signal a system error. See Section 10.2.4. A device that is only forwarding Sync packets from
another device on the same chain should not set this bit, so that the device initiating the sync flood
can be localized. Software will not be able to access the device (including this bit) via the flooded
link until a reset has occurred.
80
Configuration Accesses Chapter 7
Page 81
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.3.2.8 Data Error Detected (Bit 15): R/C: Cold Reset to 0
This bit is set by a device that accepts a read response or posted request with a Data Error
indicated. This bit is not set by TgtDone responses or if only forwarding packets with a Data Error
indicated. Note that bridges accept a cycle on one bus and issue the cycle on another bus; unlike a
tunnel, which simply forwards cycles it does not accept.
7.3.3 Cache Line Size Register: Offset 0Ch: R/O
This register is not implemented by HyperTransport technology devices, and returns 0s if read.
7.3.4 Latency Timer Register: Offset 0Dh: R/O
This register is not implemented by HyperTransport technology devices, and returns 0s if read.
7.3.5 Base Address Registers (BARs): Offsets 10-24h: R/W: Warm Reset
Base Address registers (BARs) for HyperTransport technology devices are implemented as
described in the PCI specification. Bit 0 is read-only and indicates if a BAR is for memory or I/O
space.
Table 32. Memory Space BAR Format
N+1 N 4 3 2 1 0
Base Address Size Bits Prefetchable 64b Rsv 0
Devices hardwire bits 4 through N of a memory BAR to 0 to indicate how much address space
they require. For example, to allocate 4Kbytes of memory space, bits 4 through 11 would be readonly 0. Enumeration software will write 1’s into the register and read it back to determine how
much space to allocate. Bit 3 indicates if the memory space allocated to the BAR is prefetchable
(can be read without side effects). Bit 2 indicates if a BAR is 32 or 64 bits wide. Incoming
addresses in the memory-mapped I/O space address range (as described in Chapter 5) are
compared directly to the writeable bits in memory space BARs. If the memory space BARs are
programmed to support 32-bit addressing, the BAR value is 0 extended to 64 bits before being
compared to the address. HyperTransport technology devices must have memory windows aligned
on 64-byte boundaries to guarantee that a maximum-size request will not cross a device boundary
and should therefore always indicate a size of at least 64 bytes. The reset value of these registers is
implementation-specific, so software must initialize them before setting the Memory Space Enable
bit in the Command register.
Chapter 7 Configuration Accesses
81
Page 82
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 33. I/O Space BAR Format
N+1 N 2 1 0
Base Address Size Bits Rsv 1
Like memory space BARs, I/O space BARs have bits 2 to N hardwired to 0 to indicate the amount
of address space required. Incoming addresses in the I/O space address range (which is only a 25bit space) have only their bottom 25 bits compared to the writeable bits of the I/O space BARs.
Bits 31:26 of an I/O space BAR must be 0 for a match to occur. The reset value of these registers
is implementation-specific, so software must initialize them before setting the I/O Space Enable
bit in the Command register.
7.3.6 CardBus CIS Pointer: Offset 28h: R/O
This register is not implemented by HyperTransport technology devices and returns 0s if read.
7.3.7 Capabilities Pointer: Offset 34h: R/O
Every HyperTransport technology device has a capabilities pointer to a linked capabilities list that
contains (at least) the capability registers specific to HyperTransport technology.
7.3.8 Interrupt Line Register: Offset 3Ch: R/W: Warm Reset
The Interrupt Line register should be readable and writeable and may be used by software as a
scratchpad to track interrupt routing. The reset value is implementation-specific.
7.3.9 Interrupt Pin Register: Offset 3Dh: R/O
The Interrupt Pin register is reserved, since HyperTransport technology delivers interrupts via
messages and does not define any interrupt pins, although devices that generate interrupts may
need to provide this register for compatibility with existing software. (Some operating systems
will not correctly utilize interrupts for a function without a non-zero value in its Interrupt Pin
configuration space register.)
7.3.10 Min_Gnt, and Max_Lat Registers: Offsets 3E and 3Fh: R/O
These registers are not implemented by HyperTransport technology devices, and they return 0s if
read.
82
Configuration Accesses Chapter 7
Page 83
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.4 HyperTransport™ Technology Bridge Headers
Devices that bridge between HyperTransport chains and other bus protocols that implement
configuration mechanisms and bridge headers as described in the PCI-to-PCI Bridge Architecture
Specification Revision 1.2 (including subsidiary HyperTransport chains) implement such a bridge
header, with the exceptions listed below. Note that HyperTransport technology can be the primary
or secondary bus of the device, or both. Some register bits have meanings specific to
HyperTransport technology only when a HyperTransport technology chain is connected to a
specific port of the bridge. If that interface is to a non-HyperTransport bus, the requirements of
that bus protocol determine the meaning of the bit.
Unless otherwise noted, the registers described in this section are reset by the reset mechanism of
the primary bus and not that of the secondary bus. For a HyperTransport-to-HyperTransport
bridge, this is HyperTransport technology RESET# of the primary chain. For a Host-toHyperTransport bridge, this is a host-specific reset signal. Implementations that do not require the
ability to reset the HyperTransport chain independently of the host may choose to combine the
two.
Each field is defined as readable and writeable by software (R/W), readable only (R/O), or
readable and cleared by writing a 1 (R/C). Additionally, each field is affected by only cold reset or
both cold and warm reset.
Chapter 7 Configuration Accesses
83
Page 84
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 34. HyperTransport™ Technology Bridge Header Format
31 24 23 16 15 8 7 0
Device ID Vendor ID
Status Command
Class Code Revision ID
BIST Header Type Primary Latency Timer Cache Line Size
Base Address Register 0
Base Address Register 1
Secondary Latency
Timer
Secondary Status I/O Limit I/O Base
Memory Limit Memory Base
Prefetchable Memory Limit Prefetchable Memory Base
Subordinate Bus
Number
Prefetchable Base Upper 32 Bits
Secondary Bus Number Primary Bus Number
00h
04h
08h
0Ch
10h
14h
18h
1Ch
20h
24h
28h
Prefetchable Limit Upper 32 Bits
I/O Limit Upper 16 Bits I/O Base Upper 16 Bits
Reserved Capabilities Pointer
Expansion ROM Base Address
Bridge Control Interrupt Pin Interrupt Line
Note: Shaded registers contain minimum-required read-write bits. Other registers are read-only or contain
only device-dependent bits.
2Ch
30h
34h
38h
3Ch
7.4.1 Command Register: Offset 04h
All of the Command register bits implemented in the Device Header Command register (Section
7.3.1), are implemented in bridges that have a HyperTransport link on their primary bus, and they
affect operation only on the primary bus. All other bits are not applicable and must be hardwired
to 0. If the Bus Master Enable bit is cleared in a bridge device, preventing forwarding of
transactions to the primary bus, transactions that would be forwarded must be master-aborted on
the secondary bus. If the secondary bus is a HyperTransport link, this is indicated by returning a
response with Master Abort indicated.
Note that if the secondary bus is PCI, the Memory Write and Invalidate Enable (bit 4) may be
implemented as a read/write bit to control generation of Memory Write and Invalidate cycles as
specified by the PCI Local Bus Specification, Revision 2.3.
84
Configuration Accesses Chapter 7
Page 85
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.4.2 Status, Primary Latency Timer, Base Address, Interrupt Pin, and
Interrupt Line Registers
For a bridge with a HyperTransport I/O link as its primary bus, these registers are implemented the
same way as the corresponding registers in a HyperTransport technology device header. They are
not related to the secondary bus.
7.4.3 Cache Line Size: Offset 0Ch: R/O
When both the primary and secondary buses are HyperTransport links, this register is reserved.
7.4.4 Secondary Latency Timer Register: Offset 1Bh: R/O
When the secondary bus is a HyperTransport link, this register is reserved.
7.4.5 Secondary Status Register: Offset 1Eh
When the secondary bus is a HyperTransport link, most of the bits defined in the PCI-to-PCI
Bridge Architecture Specification Revision 1.2 are not relevant and are reserved, being hardwired
to 0. The exceptions are listed in the following sections.
7.4.5.1 Master Data Error (Bit 8): R/C: Cold Reset to 0
When set, this bit indicates that the bridge had the Data Error Response bit (in the Bridge Control
register) set and either issued a posted request with the Data Error bit set or accepted a response
with a Data Error indicated on the secondary chain.
7.4.5.2 Signaled Target-Abort (Bit 11): R/C: Cold Reset to 0
When set, this bit indicates that the bridge has issued a Target Abort on the secondary bus. (As
described in section 10.2.1.)
7.4.5.3 Received Target-Abort (Bit 12): R/C: Cold Reset to 0
If set, this bit indicates that the bridge has received a Target Abort from the secondary bus. (As
described in Section 10.2.1.)
7.4.5.4 Received Master-Abort (Bit 13): R/C: Cold Reset to 0
When set, this bit indicates that the bridge has received a Master Abort from the secondary bus.
(As described in Section 10.2.1.).
Chapter 7 Configuration Accesses
85
Page 86
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.4.5.5 Detected System Error (Bit 14): R/C: Cold Reset to 0
When set, this bit indicates that the bridge detected sync packet flooding on its secondary bus. See
Section 10.2.4.
7.4.5.6 Data Error Detected (Bit 15): R/C: Cold Reset to 0
When set, this bit indicates that the bridge has accepted a read response or posted request with a
Data Error indicated on the secondary bus. This bit is not set when accepting a TgtDone with a
Data Error indicated.
7.4.6 Memory and Prefetchable Memory Base and Limit Registers: Offsets
20-2Ch: R/W: Warm Reset
Table 35. Memory and Prefetchable Memory Base and Limit Register Format
15 4 3 1 0
Address[31:20] Reserved Size
The Size bit exists only in the Prefetchable Memory Base and Limit registers, and is read-only. If
1, it indicates that the Prefetchable Memory Upper registers are implemented. The value of the bit
in the base and limit registers must be the same.
Bits 19:0 of the Base addresses are assumed to be 0. Bits 19:0 of the Limit addresses are assumed
to be all 1’s. To disable decode of a memory space (when there are no devices configured in that
space on the secondary bus, for example), the Base register should be programmed to a higher
value than the Limit register.
Table 36. Prefetchable Memory Upper Register Format
31 0
Address[63:32]
It is strongly recommended that these registers be implemented to allow maximum compatibility
with PCI devices.
For accesses coming in on a HyperTransport link, these registers are compared to addresses in the
memory-mapped I/O range only, as defined in Chapter 5. Matching addresses are forwarded from
the primary to the secondary bus, and ignored on the secondary bus. Non-matching addresses are
forwarded from the secondary to the primary bus, and ignored on the primary bus. Accesses
outside the memory-mapped I/O space (FD_0000_0000 to FF_FFFF_FFFF) on the secondary bus
are ignored. HyperTransport technology devices must have windows aligned on 64-byte
boundaries so that a maximum-size request will not cross a device boundary. The reset value of
86
Configuration Accesses Chapter 7
Page 87
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
these registers is implementation-specific, so firmware must initialize them before setting the
Memory Space or Bus Master Enable bits in the Command register.
7.4.7 I/O Base and Limit Registers: Offsets 1C, 1D, 30, and 32h: R/W:
Warm Reset
Table 37. I/O Base and Limit Register Format
7 4 3 1 0
Address[15:12] Reserved Size
The Size bit of the I/O Base and Limit Registers are read-only and if 1, it indicates the I/O Base
and Limit Upper Registers are implemented. The value of the bit in both registers must be the
same.
Bits 11:0 of the Base address are assumed to be 0. Bits 11:0 of the Limit address are assumed to be
all 1’s. To disable decode of the I/O space (when there are no devices configured in I/O space on
the secondary bus, for example), the Base register should be programmed to a higher value than
the Limit register.
Table 38. I/O Base and Limit Upper Register Format
15 0
Address[31:16]
It is strongly recommended that these registers be implemented to allow maximum compatibility
with PCI devices.
For accesses coming in on the primary (HyperTransport) link, these registers are only compared to
addresses in the 32-Mbyte I/O range, as defined in Chapter 5. Only the low 25 bits of the
incoming byte address are used. All bits above bit 24 are forced to 0 before the comparison.
Matching addresses are forwarded from the primary to the secondary bus and ignored on the
secondary bus. Non-matching addresses are forwarded from the secondary to the primary bus and
ignored on the primary bus. Because HyperTransport technology supports only 25-bit I/O space
addressing, accesses outside the 25-bit space on the secondary bus are ignored. HyperTransport
technology devices must have windows aligned on 64-byte boundaries so that a maximum-size
request will not cross a device boundary. The reset value of these registers is implementationspecific, so firmware must initialize them before setting the I/O Space or Bus Master Enable bits
in the Command register.
Chapter 7 Configuration Accesses
87
Page 88
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.4.8 Capabilities Pointer Register: Offset 34h: R/O
Every bridge with a HyperTransport link on at least one port has a capabilities pointer to a linked
capabilities list that contains (at least) the capability registers specific to HyperTransport
technology.
7.4.9 Bridge Control Register: Offset 3Eh
All unspecified bits are reserved and return 0 if read.
7.4.9.1 Data Error Response Enable (Bit 0)
This bit must be set to allow the bridge to set the Master Data Error bit of the Secondary Status
register. It also enables Data Error to be set in TgtDone responses that are being forwarded from
the primary bus to the secondary bus. It has no effect on forwarding posted requests or
RdResponses.
7.4.9.2 SERR# Enable (Bit 1): R/W: Warm Reset to 0
This bit controls the mapping of system errors from the secondary to the primary bus of the
bridge. If set, and the SERR# Enable in the Command register is set, system errors will propagate.
System errors in HyperTransport technology are indicated by flooding the chain with sync
packets, as described in Section 10.2.4.
7.4.9.3 ISA Enable (Bit 2): R/W: Warm Reset to 0
This bit is implemented similarly to the way it is described in the PCI-to-PCI Bridge Architecture
Specification Revision 1.2. For HyperTransport technology requests in the bottom 64 Kbytes of
I/O space (see Chapter 5), this modifies the response to accesses that hit in the range defined by
the I/O Base and Limit registers. If this bit is set, transacations addressing bytes 256 to 1023 of
each 1Kbyte block on the primary bus are not passed to the secondary bus. Conversely,
transactions addressing bytes 256 to 1023 in each 1Kbyte block on the secondary bus will be
passed to the primary bus.
Implementation of this bit is required for PCI compatibility.
If this bit is set when the address mapping extensions described in Appendix A are in use, the
address decode behavior of the device may be undefined.
7.4.9.4 VGA Enable (Bit 3): R/W: Warm Reset to 0
This bit functions similarly to the way it is described in the PCI-to-PCI Bridge Architecture
Specification Revision 1.2. If enabled, RdSized and WrSized operations within the address range
0_000A_0000–0_000B_FFFFh (inclusive) or within the first 64 Kbytes of the I/O range (as
88
Configuration Accesses Chapter 7
Page 89
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
defined in Chapter 5), with address bits 9:0 in the (inclusive) range 3B0–3BBh or 3C0–3DFh, are
forwarded from the primary to the secondary interface and are ignored on the secondary interface,
overriding the values in the Memory Base and Limit and I/O Base and Limit registers.
Implementation of this bit is required for PCI compatibility.
If this bit is set when the address mapping extensions described in Appendix A are in use, the
address decode behavior of the device may be undefined.
7.4.9.5 VGA 16-bit Decode (Bit 4): R/W: Warm Reset to 0
This bit functions similarly to the way it is described in the PCI-to-PCI Bridge Architecture
Specification Revision 1.2. If this bit is set, it prevents the decoding of aliased VGA I/O accesses
such that only accesses to 3B0-3BBh or 3C0-3DFh are forwarded from the primary interface to
the secondary interface and ignored on the secondary interface. This bit has no affect unless VGA
Enable (bit 3) is set.
7.4.9.6 Master-Abort Mode (Bit 5): R/W: Warm Reset to 0
This bit controls the behavior on the source bus when a transaction forwarded through the bridge
receives a Master Abort on the target bus. (See Section 10.2.1.)
When the Master-Abort Mode bit is set, and a nonposted request forwarded from a
HyperTransport chain receives a Master Abort on the target bus, the source bus request will
receive a Target Abort. (That is, it will be signaled as an internal error on the originating chain.)
When the Master-Abort Mode bit is clear, the request will appear to complete normally on the
source bus. The response will return with the Error bits clear. Writes will receive a TgtDone.
Reads will receive a RdResponse, with the appropriate amount of data, which will be all
hexadecimal Fs.
7.4.9.7 Secondary Bus Reset (Bit 6): R/W: Warm Reset to 0
This bit allows software to reset the secondary bus of the bridge. Writing a 1 to this bit causes the
bridge to force the secondary bus into reset. Writing a 0 to this bit cause the bridge to stop forcing
the secondary bus into reset. The remaining details are dependent on the type of secondary bus.
This bit is required for bridges that have a HyperTransport link on their primary interface. This bit
is also required for host bridges in which HyperTransport technology RESET# is independent of
host reset.
If the secondary bus of the bridge is a HyperTransport I/O chain, writing a 1 to this bit will cause
the RESET# signal for that chain to be asserted. If the Warm Reset bit in the Host/Secondary
Interface Command register (Section 7.5.3.3.1) is clear, the PWROK signal for that chain will also
be deasserted. When, after being initially set, the Secondary Bus Reset bit is cleared, the chain will
Chapter 7 Configuration Accesses
89
Page 90
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
come out of reset. If the Warm Reset bit is set, this simply results in the deassertion of RESET#. It
is the responsibility of software to delay deasserting the reset long enough to satisfy the RESET#
pulse width requirement. If the Warm Reset bit is clear, clearing the Secondary Bus Reset will
cause PWROK to assert. Hardware will then wait for the appropriate amount of time and deassert
the RESET# pin. If the programmer wants to be able to determine that the bus has come out of
reset, software can poll the Initialization Complete bit of the Link Control register (Section
7.5.4.5).
7.4.9.8 Fast Back-to-Back Enable, Primary Discard Timer, Secondary Discard Timer,
Discard Timer Status, Discard Timer SERR# Enable (Bits 11:7): R/O
All of these bits are controls for the secondary interface of the bridge. If the secondary interface is
a HyperTransport link, these bits are reserved and hardwired to 0.
7.5 Capability Registers
Configuration and status information specific to HyperTransport technology is mapped into
configuration space using the capabilities list methodology described in the PCI Local Bus
Specification, Revision 2.3.
• A device with multiple HyperTransport technology interfaces (e.g., a HyperTransport-to-
HyperTransport bridge with HyperTransport links on both the primary and secondary
interface) must implement one capability block for each interface. Therefore, a single-link
device would have a single primary interface block containing one active Link Control
register, and a tunnel device would have a single primary interface block containing two active
Link Control registers.
• A bridge would have one primary interface block containing one or two active Link Control
registers (depending on whether or not the device provides a tunnel to allow a chain to
continue on the primary bus) in addition to one secondary interface block for each
HyperTransport technology bridge.
• A HyperTransport technology bridge header and secondary interface capability block are
required for each chain.
• Only one primary interface capability block is required for a device that uses HyperTransport
technology as its primary interface.
• Primary interface capability blocks always have two Link Control registers. If a link is not
implemented, (such as in a single-link device), the Link Control register for that link will be
read-only and indicate Link Failure and End Of Chain, as described below.
• If a device is a host bridge or has a different bus for its primary interface, only secondary
interface block(s) are required and bridge headers are optional.
• Every function of every device implemented in a HyperTransport node must place a
HyperTransport capability list item in its configuration space, indicating the version of the
90
Configuration Accesses Chapter 7
Page 91
HyperTransport Technology Consortium
31
7
HyperTransport™ I/O Link Specification Revision 1.05c
specification to which it is compliant. This can be the primary or secondary interface
capability if already present, or the HyperTransport Revision ID capability otherwise.
The layout of the capabilities block is determined by the value in the Capability Type field in the
Command register, but the Capability ID register, Capabilities Pointer register, and Capability
Type field are always the same. The offset at which the block begins is implementation-specific.
The layout of a Slave/Primary Interface block is shown in Table 39.
Table 39. Slave/Primary Interface Block Format
24 23 16 15 8
Command Capabilities Pointer Capability ID
Link Config 0 Link Control 0
Link Config 1 Link Control 1
LinkFreqCap0 Link Error 0 Link Freq 0 Revision ID
LinkFreqCap1 Link Error 1 Link Freq 1 Feature
0
+00h
+04h
+08h
+0Ch
+10h
Error Handling Enumeration Scratchpad
Reserved Bus Number Mem Limit Upper Mem Base Upper
Note: Shaded registers contain minimum-required read-write bits. Other registers are read-only or
contain only device-dependent bits.
The layout of a Host/Secondary Interface block is shown in Table 40.
Table 40. Host/Secondary Interface Block Format
31 24 23 16 15 8 7 0
Command Capabilities Pointer Capability ID
Link Config Link Control
LinkFreqCap Link Error Link Freq Revision ID
Reserved Feature
Error Handling Enumeration Scratchpad
Reserved Mem Limit Upper Mem Base Upper
Note: Shaded registers contain minimum-required read-write bits. Other registers are read-only or contain
only device-dependent bits.
+14h
+18h
+00h
+04h
+08h
+0Ch
+10h
+14h
Unless otherwise noted, the registers described in this section are reset by the reset mechanism of
the primary bus and not that of the secondary bus. For a HyperTransport-to-HyperTransport
bridge, the reset mechanism is the HyperTransport technology RESET# signal of the primary
chain. For a Host-to-HyperTransport bridge, the reset mechanism is a host-specific reset signal.
Chapter 7 Configuration Accesses
91
Page 92
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Implementations that do not require the ability to reset the HyperTransport chain independently of
the host may choose to combine the two.
Registers marked “Chain Reset” are reset with which the chain they are associated. For host
interface blocks, this is HyperTransport technology RESET#, not host reset. For device primary
interface blocks, this is HyperTransport technology RESET# on the primary chain. For device
secondary interface blocks, this is HyperTransport technology RESET# on the bridge’s secondary
chain.
Each field is defined as readable and writeable by software (R/W), readable only (R/O), readable
and settable by writing a 1 (R/S), or readable and cleared by writing a 1 (R/C). Additionally each
field is affected by only cold reset or by both cold and warm reset.
7.5.1 Capability ID: Offset 00h: R/O
The capability ID for HyperTransport technology is 08h.
7.5.2 Capabilities Pointer: Offset 01h: R/O
This register contains a pointer to the next capability in the list, or a value of 00h if this is the last
one.
7.5.3 Command Register: Offset 02h
The Command register contains bits used to configure the HyperTransport interface, as shown in
Table 41. All unspecified bits are reserved and are hardwired to 0.
Table 41. Command Register Format
Type 15 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Slave/Pri 000
Host/Sec 001
Drop
on
Uninit
Drop
on
Uninit
Default
Direction
Inbound
EOC Err
Master
Host
Act as
Slave
Rsv
Note: Any write to the slave command register may affect the Master Host bit, as described below.
Unit Count Base UnitID
Host
Hide
Chain
Side
Device Number
Double
Ended
Warm
Reset
92
Configuration Accesses Chapter 7
Page 93
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.5.3.1 Capability Type (Bits 15:13): R/O
This field indicates the type of information present in this capability block. For the primary and
secondary interface capability blocks, there are 3 bits used in the encoding. For all other
HyperTransport capability blocks, 5 bits are used. Currently, these encodings are defined as shown
in Table 42.
Table 42. Capability Type Encoding
Encoding Capability Type
000xx Slave or Primary Interface
001xx Host or Secondary Interface
01000 Switch
01001-01111 Reserved
10000 Interrupt Discovery and Configuration
10001 Revision ID
10010 UnitID Clumping
10011 Extended Configuration Space Access
10100 Address Mapping
10101 MSI Mapping
10110-10111 Reserved
11000 Reserved (Retry Mode)
11001 Reserved (X86 Encoding)
11010-11111 Reserved
Primary and secondary interface encodings indicate that this block is used to configure the primary
(including the interface of a HyperTransport technology slave device) or secondary (including the
interface of a host bridge) interface of a device, respectively.
The layout of the rest of the bits of the Command register depends on the Capability Type field, as
described in the following sections.
7.5.3.2 Slave/Primary Interface Command Bits
7.5.3.2.1 Base UnitID (Bits 4:0): R/W: Warm Reset to 0
This field contains the lowest numbered UnitID belonging to this device. If the device owns
multiple UnitIDs, the additional ones occupy the next consecutive UnitID values above the base.
Chapter 7 Configuration Accesses
93
Page 94
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
The contents of this field are used to generate the UnitID field in request and response packets
issued by this device, to identify responses returning to this device, and to identify configuration
requests directed to this device.
7.5.3.2.2 Unit Count (Bits 9:5): R/O
This field contains the number of UnitIDs that this device requires. Therefore, the highest UnitID
used by this device is given by (BaseUnitID + UnitCount – 1). If the highest UnitID used exceeds
1Fh, the behavior of the device is undefined. Some devices may have the capability to modify
their Unit Count through write-once or backdoor access mechanisms. The method by which
enumeration software determines and sets valid Unit Count values is system and device specific
and beyond the scope of this specification.
7.5.3.2.3 Master Host (Bit 10): R/W: Warm Reset to 0
This bit indicates which link is the path to the master (or only) host bridge of the HyperTransport
chain. It is readable from software, but not directly writeable. Any time the Command register is
written, this bit is loaded with the link number from which the write came. For a device with only
one link interface, this bit may be hardwired to point to the implemented link.
7.5.3.2.4 Default Direction (Bit 11): R/W: Warm Reset to 0
This bit determines the default direction for a HyperTransport technology device to send requests.
A 0 indicates requests should be sent toward the master host bridge, as indicated by the Master
Host bit. A 1 indicates requests should be sent in the opposite direction. For a device with only
one link interface, this bit has no meaning and should be hardwired to 0. An intelligent device can
choose to ignore this if it bases the direction of DMA upon some other criteria.
7.5.3.2.5 Drop on Uninitialized Link (Bit 12): R/W: Cold Reset to 0
This bit determines what will happen to packets issued by a device or forwarded from a receiving
link interface to a transmitting interface whose Initialization Complete and End of Chain bits are
clear. If both are deasserted for a given link, packets to be transmitted on that link will be stalled
until either Initialization Complete sets (in which case they will be transmitted) or End Of Chain
sets (in which case they will be treated as End Of Chain packets; see Section 10.1.5). In the case
where hardware is broken, it is possible that neither of these events occurs, in which case the
packet can hang. If Drop on Uninitialized Link is set, a transmitter with its Initialization Complete
bit clear will always act as if the End of Chain bit were set. Hosts that use the initialization
sequence described in Section 12.3 are encouraged to implement a timeout counter to prevent a
system-wide initialization error due to link-level initialization problems on a non-default chain.
Packet forwarding behavior is described in Table 43.
94
Configuration Accesses Chapter 7
Page 95
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Table 43. Packet Forwarding Behavior
End of Chain Initialization Complete Drop on Uninitialized Link Action
1 - - Reject
0 1 - Forward
0 0 1 Reject
0 0 0 Stall
7.5.3.3 Host/Secondary Interface Command Bits
7.5.3.3.1 Warm Reset (Bit 0): R/W: Warm Reset to 1
This optional bit allows a reset sequence initiated by the Secondary Bus Reset bit of the Bridge
Control register (see Section 7.4.9.7) to be either warm or cold. The contents of this bit have an
effect only when software initiates a reset sequence. If it is 0, PWROK will be driven low as part
of the sequence, causing a cold reset. It is the responsibility of the hardware to sequence PWROK
and RESET# correctly. If not implemented, this bit is read-only and hardwired to 1. Changing the
state of the Warm Reset bit while the Secondary Bus Reset bit is asserted results in undefined
behavior.
7.5.3.3.2 Double-Ended (Bit 1): R/W: Warm Reset to 0
This bit indicates that there is another bridge at the far end of the HyperTransport chain. For
bridges that do not support double-ended chains, this bit must be hardwired to 0. This bit controls
no hardware. It exists as a scratchpad for software during link configuration.
7.5.3.3.3 Device Number (Bits 6:2): R/W: Cold Reset to 0
This optional register contains the device number of configuration accesses that the host bridge
responds to when accessed from the chain attached to the host interface. While this will typically
be 0, in some cases there may be legacy software (Section 7.2.2) or ordering (Section 4.1.1)
considerations that require host configuration space registers to be located somewhere other than
Device 0. If not implemented, this register is read-only and hardwired to 0. When accessed from
the host side, the bridge may locate its configuration space registers at a different location, so that
when enumerated in a double-hosted chain, one host will not relocate the register space of another
This value in this register is used as the UnitID of hosts with the Act as Slave bit set (defined in
Section 7.5.3.3.6). If the Act as Slave bit is implemented, this register must be implemented.
7.5.3.3.4 Chain Side (Bit 7): R/O
This bit indicates which side of the host bridge is being accessed. A 0 indicates that the read is
coming from within the host. A 1 indicates that the read is coming from the chain attached to the
host interface. In a host that does not support double-hosted chains, this bit is always 0, because
there cannot be an access from the chain.
Chapter 7 Configuration Accesses
95
Page 96
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.5.3.3.5 Host Hide (Bit 8): R/W: Warm Reset to 0
This bit, when set, causes the host’s configuration space to be inaccessible from the chain attached
to the host interface (any accesses to configuration space are treated as if they have reached the
end of chain). When clear, the host should respond to configuration cycles from the chain. The
determination of the register set to be presented to the chain is implementation-specific, although a
configuration header including a host capability block must be presented for chain enumeration
purposes. In a host that does not support double-hosted chains, this bit is read-only 1, because the
host is never accessible from the chain.
7.5.3.3.6 Act as Slave (Bit 10): R/W: Cold Reset to 0
This optional bit, when set, causes a host to act as a slave, using the device number defined in
Section 7.5.3.3.3 as the base UnitID for requests and responses that it originates, it does not set the
Bridge bit on responses it generates, and follows the routing rules for ordinary nodes defined in
Section 4.9. When clear, a host acts normally (uses UnitID 0, sets the Bridge bit on responses, and
follows the routing rules for hosts defined in Section 4.9.4). If a host does not support sharing
double-hosted chains, this bit may be read-only 0. This bit takes effect only after a warm reset.
7.5.3.3.7 Host Inbound End of Chain Error (Bit 11): R/C: Cold Reset to 0
This bit indicates that a packet received from a far host has taken an end-of-chain error. (See
Section 10.1.5.) This bit is hardwired to 0 if the device does not check for this error condition.
7.5.3.3.8 Drop on Uninitialized Link (Bit 12): R/W: Cold Reset to 0
This bit is defined in the same way as bit 12 of the Slave/Primary Interface Command register.
7.5.4 Link Control Register: Offsets 04h and 08h
Host/secondary interface blocks implement one copy of the Link Control register (defined in
Table 44). Slave/primary interface blocks implement two copies of this register, one for each link.
For devices that implement only one link in the chain, all bits of the second control register are
reserved and hardwired to 0, except for the Link Failure, End of Chain, and Transmitter Off fields,
which are hardwired to 1.
Table 44. Link Control Register
15 14 13 12 11 8 7 6 5 4 3 2 1 0
64b ExtCTL LSEn IsocEn CRC Error TXO EOC Init LkFail CFE CST CFlE Rsv
7.5.4.1 CRC Flood Enable (Bit 1): R/W: Warm Reset to 0
When this bit is set and (for slave devices) SERR# Enable is set, sync flooding will be initiated
and the LinkFail bit will be set whenever any of the CRC Error bits for this link are asserted. See
96
Configuration Accesses Chapter 7
Page 97
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Section 10.2.4. CRC checking logic runs on all lanes enabled by LinkWidthIn, and detected errors
still set the CRC Error bits, regardless of the state of this bit. Note that this bit is not reset by
secondary chain reset, so it will not be reset in a bridge when that bridge takes a sync flooded link
through warm reset, and may cause sync flooding to be immediately restarted after the reset
sequence is complete. In that case, software should clear this bit before taking the chain out of
reset.
7.5.4.2 CRC Start Test (Bit 2): R/W: Warm Reset to 0
When this optional bit is written to a 1 by software, the hardware initiates a CRC test sequence on
the link, as described in Appendix G. When the test sequence is complete, and CRC has been
checked on all CRC intervals containing test pattern data, hardware clears the bit. Software can
determine that the test has completed by reading the bit and checking the status of the CRC Error
bits. Implementation of CRC test pattern generation is optional. If not implemented, this bit must
be hardwired to 0. Software should not set this bit unless it has checked the CRC Test Mode
Capability bit, as defined in Section 7.5.10.3, of the device on the other side of the link.
7.5.4.3 CRC Force Error (Bit 3): R/W: Warm Reset to 0
When this bit is set, bad CRC is generated on all transmitting lanes, as enabled by LinkWidthOut.
The covered data is not affected.
7.5.4.4 Link Failure (Bit 4): R/C: Evaluated at Cold Reset
The LinkFail bit is set if a failure has been detected on the link. Devices with only one
HyperTransport link will hardwire LinkFail for the nonexistent link to 1. Devices that contain an
interface that is not used by the system will also set LinkFail for the unused link to 1. As described
in Section 12.2, devices can identify unused links because their CAD[0] input is tied to a logical 0.
The LinkFail bit is set by hardware in the event of a link error that results in a sync flood, such as
a CRC, protocol, or overflow error (See Section 10.2.4). It is not set by a device that is merely
forwarding sync packets, only by a device that originates them.
7.5.4.5 Initialization Complete (Bit 5): R/O: Warm Chain Reset to 0
This read-only bit is reset to 0 and set by hardware when the low-level link initialization sequence
(see Section 12.2) is successfully complete in both the receiver and transmitter. If there is no
device on the other end of the link, or if that device is unable to properly perform the low-level
link initialization protocol, the bit never gets set. A device may receive a request before
initialization of all the attached links is complete. If the request needs to be forwarded to an
uninitialized link, the disposition of the request is determined by the Drop On Uninitialized Link
bit (see Section 7.5.3.2.5).
Chapter 7 Configuration Accesses
97
Page 98
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
7.5.4.6 End of Chain (Bit 6): R/S: Evaluated at Warm Chain Reset
The End of Chain bit is set to indicate that the given link is not part of the logical HyperTransport
chain. Packets that are issued or forwarded to this link are either dropped or result in a Master
Abort response, as appropriate (see Section 4.9.2). Packets received from this link are ignored,
CRC is not checked, and sync flooding from this link is ignored and not propagated. If the
transmitter is still enabled (Transmitter Off CSR bit is clear) when the End of Chain bit is set, the
transmitter must drive NOP packets (all CAD bits 0, with CTL asserted) with good CRC. This is
required to prevent the far receiver from seeing garbage when we are no longer sending to it. It is
the responsibility of software to make sure that no traffic is going across the link when End of
Chain is set, so that the switch to NOPs does not occur in the midst of a packet.
Slave devices with only one HyperTransport link will hardwire End of Chain for the nonexistent
link to 1. Devices that contain an interface that is not used by the system will also set End of Chain
for the unused link to 1 at the deasserting edge of RESET#. (As described in Section 12.2, devices
can identify unused links because their CAD[0] input is tied to a logical 0.) End of Chain can be
set by software by writing a 1 to indicate the logical end of the chain, or by partitioning a doublehosted chain into two independent logical chains. This bit cannot be cleared by software. A write
of 0 to this bit position has no effect.
7.5.4.7 Transmitter Off (Bit 7): R/S: Warm Chain Reset to 0
This bit provides a mechanism to shut off a link transmitter for power savings or EMI reduction.
When set, no output signals on the link toggle and are driven to electrical levels that satisfy the DC
specification. This bit resets to 0 and can be set by software writing a 1 to the bit. This bit cannot
be cleared by software. A write of 0 to this bit position has no effect. If End of Chain is set on an
active link, the Transmitter Off bit should not be set until the transmitter has driven enough NOPs
to fill the receiver's receive FIFOs.
When this bit is set, the link receiver should also be disabled if necessary to prevent DC current
paths as a result of the inputs that may be invalid or floating.
7.5.4.8 CRC Error (Bits 11:8): R/C: Cold Reset to 0
These bits are set by hardware when a CRC error is detected on an incoming link. Errors are
detected and reported on a per byte-lane basis where bit 8 corresponds to the least-significant byte
lane. Four bits are required to cover the maximum HyperTransport technology width of 32 bits.
Error bits for unimplemented (as specified by Max Link Width In, Section 7.5.5.1) or unused (as
specified by Link Width In, Section 7.5.5.5) byte lanes return 0 when read.
7.5.4.9 Isochronous Flow Control Enable (Bit 12): R/W: Cold Chain Reset to 0
This optional bit controls whether Isochronous flow control, as described in Appendix D.1, is
enabled for this link. The bit is set to enable Isochronous flow control mode and cleared to disable
98
Configuration Accesses Chapter 7
Page 99
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
Isochronous flow control mode. Note that the Isoc bit in requests and responses is used regardless
of this setting. Only Isoc flow control packets are prevented by clearing it. This bit only takes
effect after a warm reset. This bit is reserved if the Isochronous flow control mode capability bit is
cleared. (See Section 7.5.10.1) It is the responsibility of system-sizing software to ensure that this
bit is set to the same value on both sides of the link and only set if both sides of the link have
Isochronous flow control mode capability.
When a device forwards Isochronous traffic from a link that has Isochronous Flow Control
enabled to a link that does not, the packet is unmodified but must be forwarded in the normal
virtual channel set. Conversely, when traffic is forwarded from a link that does not have
Isochronous Flow Control enabled to one that does, the packets are unmodified but are forwarded
in the Isochronous virtual channel set.
7.5.4.10 LDTSTOP# Tristate Enable (Bit 13): R/W: Cold Reset to 0
This bit controls whether the transmitter tristates the link during the disconnected state of an
LDTSTOP# sequence, as described in Section 8.3. When the bit is set, the transmitter tristates the
link. When the bit is clear, the transmitter continues to drive the link. This bit is reserved if the
LDTSTOP# capability bit is cleared. (See Section 7.5.10.2) The behavior of the link transmitter
and receiver in both the tristate and driven cases is described in Table 45.
Table 45. LDTSTOP# Tristate Enable Bit Encoding
LDTSTOP#
Tristate Enable
0 Driven
1 Tristate
Link State in
LDTSTOP#
Disconnect State
Transmitter Behavior Receiver Behavior
CAD and CTL logically undefined,
but driven to electrical levels that
satisfy DC specification. CLK
running.
CAD, CTL and CLK placed in high
impedance state.
Ignores CAD and CTL logical
values.
Disables DC current paths that
could be created as a result of
CAD, CTL and CLK inputs
being tristated and ignores
logical values.
7.5.4.11 Extended CTL Time (Bit 14): R/W: Cold Reset to 0
If this bit is set, during the link initialization sequence in Section 12.2 following an LDTSTOP#
disconnect sequence, CTL will be asserted for 50 us after the point where both the transmitting
device has asserted CTL and it has sampled CTL asserted from the other side of the link. If this bit
is clear, CTL need only be asserted at least 16 bit-times after both sides assert CTL in 8-bit or
larger links. (32 bit-times for 4-bit links, 64 bit-times for 2-bit links) Software must set this bit if
the device on the other side of the link has its Extended CTL Time Required bit (Section 7.5.10.4)
Chapter 7 Configuration Accesses
99
Page 100
HyperTransport Technology Consortium
HyperTransport™ I/O Link Specification Revision 1.05c
set. The extension allows devices using DLLs in their receivers enough time to lock to the transmit
clock. This is necessary after LDTSTOP# because transmit clocks are only required to be stable at
the time when CTL is asserted.
7.5.4.12 64 Bit Addressing Enable (Bit 15): R/W: Warm Reset to 0
If this bit is set, requests that access addresses above FF_FFFF_FFFFh can be issued or forwarded
by this link interface with the Address Extension command. If this bit is clear, then any access
above FF_FFFF_FFFFh will be master aborted as if the end of chain was reached. Software is
required to check the 64 Bit Address Feature bit for each node (described in Section 7.5.10.5) and
set this bit only if the node on the other side of the corresponding link supports 64 bit addressing.
Devices that support 64-bit addressing should be placed together within a chain to ensure that 64bit requests can be used.
7.5.5 Link Configuration Register: Offsets 06h and 0Ah
As with the Link Control register, there may be either one or two copies of the Link Configuration
register, one for each link. If only one link is implemented by the device, the second register is
reserved. All unspecified bits are reserved. The Link Configuration register is defined in Table 46.
As described in the following subsections, software updates to the upper half of this register take
effect after a warm reset sequence and, depending on the field, also after an LDTSTOP#
disconnect sequence.
Table 46. Link Configuration Register Definition
15 14 12 11 10 8 7 6 4 3 2 0
Dw
Out
En
Fc
LinkWidthOut
Dw
Fc
In
En
LinkWidthIn
Dw
Dc
Out
MaxLinkWidthOut
Dw
Fc
In
MaxLinkWidthIn
7.5.5.1 Max Link Width In (Bits 2:0): R/O
This field contains three bits that indicate the physical width of the incoming side of the
HyperTransport link implemented by this device.
The encodings are as shown in Table 47
100
Configuration Accesses Chapter 7
Loading...