Achronix Speedster22i User Manual Ethernet
1
Speedster22i 10/40/100
Gigabit Ethernet User Guide
UG029 – September 6, 2013
UG029, September 6, 2013
Copyright Info
Copyright © 2013 Achronix Semiconductor Corporation. All rights reserved. Achronix is a
trademark and Speedster is a registered trademark of Achronix Semiconductor Corporation.
All other trademarks are the property of their prospective owners. All specifications subject
to change without notice.
NOTICE of DISCLAIMER: The information given in this document is believed to be accurate
and reliable. However, Achronix Semiconductor Corporation does not give any
representations or warranties as to the completeness or accuracy of such information and
shall have no liability for the use of the information contained herein. Achronix
Semiconductor Corporation reserves the right to make changes to this document and the
information contained herein at any time and without notice. All Achronix trademarks,
registered trademarks, and disclaimers are listed at http://www.achronix.com and use of this
document and the Information contained therein is subject to such terms.
2 UG029, September 6, 2013
3
Table of Contents
Copyright Info .................................................................................................... 2
Overview ............................................................................................................ 8
Functional Description .................................................................................... 10
Interface Signal List ........................................................................................ 12
Interface Signal Descriptions ......................................................................... 13
Global Signals .............................................................................................................................. 13
Receive FIFO Interface ................................................................................................................ 13
Transmit FIFO Interface ............................................................................................................... 15
PMA TX/RX Interface ................................................................................................................... 16
Priority Flow Control Interface ...................................................................................................... 17
Auto-Negotiation Control and Status ............................................................................................ 18
Serial Bus Interface ................................ ...................................................................................... 18
SerDes (off-chip) Interface ........................................................................................................... 19
Transmitted Frame Status ............................................................................................................ 19
Timestamp Timer ......................................................................................................................... 19
MAC/PCS Status Indications ........................................................................................................ 20
Implementation with ACE ............................................................................... 21
Software/Hardware Requirements ....................................................................................... 21
Creating an Ethernet Instance ............................................................................................. 21
Configuring the 10/40/100G Ethernet Core .................................................................................. 21
FPGA Fabric Interface ................................................................................................................. 24
PHY Interface ............................................................................................................................... 24
Interface the Ethernet Core to the FPGA Fabric .................................................................. 24
Data ............................................................................................................................................. 24
Serial Bus Interface ................................ ...................................................................................... 24
Simulation ............................................................................................................................ 24
Clock Distribution ........................................................................................... 26
Reset Considerations ...................................................................................... 28
MAC Soft Reset ................................................................................................................... 28
FIFO / Credit Counter Reset ................................................................ ................................ 28
PCS Reset ........................................................................................................................... 28
UG029, September 6, 2013
Fabric FIFO Interface....................................................................................... 29
Overview .............................................................................................................................. 29
Transmit FIFO Interface Block Diagram ....................................................................................... 30
Receive FIFO Interface Block Diagram ........................................................................................ 31
Credit Handling .................................................................................................................... 32
Receive Direction: FIFO to Application ......................................................................................... 32
Transmit Direction: Application to FIFO ........................................................................................ 33
Data Structure ...................................................................................................................... 33
FIFO Interface Transmit Operation ...................................................................................... 37
FIFO Interface Receive Operation ....................................................................................... 39
Frame Status ....................................................................................................................... 42
FIFO Thresholds .................................................................................................................. 43
FIFO Sections Behavior (Watermarks) ......................................................................................... 43
Flow Control Interface ..................................................................................... 45
Link Pause Flow Control Frames ......................................................................................... 45
Priority Flow Control (PFC) Frames ..................................................................................... 46
Transmit Pause/PFC Operation ........................................................................................... 47
Receive Pause/PFC Operation ............................................................................................ 47
PFC Mode .................................................................................................................................... 47
Link Pause Mode ......................................................................................................................... 47
Serial Bus Interface ......................................................................................... 49
Overview .............................................................................................................................. 49
Port List ................................ ................................................................ ................................ 49
Read Operation .................................................................................................................... 49
Write Operation .................................................................................................................... 50
PMA Management Interface ............................................................................ 51
Features .................................................................................. Error! Bookmark not defined.
Power State Descriptions ..................................................................................................... 51
Power State Sequencing ..................................................................................................... 53
Auto-Negotiation ............................................................................................. 54
Overview .............................................................................................................................. 54
Usage .................................................................................................................................. 55
IEEE 1588 Timestamping ................................................................................ 58
Receive Timestamping ........................................................................................................ 58
4 UG029, September 6, 2013
5
Transmit Timestamping ....................................................................................................... 58
MAC & PCS Configuration Registers ............................................................. 59
MAC & PCS Register Overview ........................................................................................... 59
Channelized MAC Registers ................................................................................................ 61
COMMAND_CONFIG Register Bit Definitions .............................................................................. 67
STATUS Register Bit Definitions .................................................................................................. 70
10G MAC SGMII PCS Register Map ................................................................................... 71
1000Base-X / SGMII PCS .................................................................................................... 71
1000Base-X/SGMII PCS Registers Description ........................................................................... 73
Global Registers .................................................................................................................. 77
Core Configuration Registers ................................................................ ....................................... 77
VLAN Tag Configuration Registers .............................................................................................. 78
Channelized PCS Registers................................................................................................. 79
Auto-Negotiation Registers .................................................................................................. 87
Control Register Bits (KXAN_CONTROL) .................................................................................... 89
Status Register Bits (KXAN_STATUS) ......................................................................................... 89
Ability Register Bits (KXAN_ABILITY / KXAN_REM_ABILITY) ..................................................... 90
Next page Ability Register Bits (AN_XNP / LP_AN_XNP) ............................................................ 91
PMA Registers ................................................................................................. 93
PMA State Control ............................................................................................................... 93
Memory Bus (Membus) Interface ................................................................................................. 93
Memory Address Decoding .......................................................................................................... 93
PMA Receive Equalization Registers ................................................................................... 93
TX/RX Lane Receive Equalization Registers................................................................................ 94
Common/Synth Lane Receive Equalization Registers .................................................................. 96
PMA Transmit Control Registers .......................................................................................... 96
TX/RX Lane Transmit Control Registers ...................................................................................... 96
Common/Synth Lane Transmit Control Registers ........................................................................ 97
PMA Adaptive Equalizer Registers ...................................................................................... 98
TX/RX Lane Adaptive Equalizer Registers ................................................................................... 98
Common/Synth Lane Adaptive Equalizer Registers ..................................................................... 99
Statistics Data Registers .............................................................................. 102
Overview ............................................................................................................................ 102
IEEE 802.3 Management Package .................................................................................... 102
IETF Management Information Base (MIB, MIB-II) Objects ............................................... 102
IETF Remote Network Monitoring ...................................................................................... 102
Receive Statistics Vector ................................................................................................... 103
Transmit Statistics Vector .................................................................................................. 105
UG029, September 6, 2013
References ..................................................................................................... 107
6 UG029, September 6, 2013
7
Table of Figures
Figure 1: 10/40/100 Gigabit Ethernet MAC Block Diagram ...................................................... 10
Figure 2: Interface Signal List .................................................................................................. 12
Figure 3: 10/40/100G Ethernet MAC IP Wizard ....................................................................... 22
Figure 4: Generate IP Design Files dialog box ........................................................................ 23
Figure 5: Simulation Flow ................................ ................................ ................................ ........ 25
Figure 6: System clock distribution for the 20-Bit SerDes interface ......................................... 26
Figure 7: Example implementation for the FIFO clock and reset multiplexers ......................... 27
Figure 8: Transmit FIFO Interface Block Diagram ................................................................... 30
Figure 9: Receive FIFO Interface Block Diagram .................................................................... 31
Figure 10: Credit based application interface .......................................................................... 32
Figure 11: FIFO Transmit Interface – Single Frame Transfer .................................................. 37
Figure 12: FIFO Transmit Interface – Frame Transfer with User Application Pause ............... 38
Figure 13: FIFO Transmit Interface – Back-to-Back Frame Transfer ....................................... 38
Figure 14: FIFO Transmit Interface – Frame Transfer with Error ............................................. 38
Figure 15: FIFO Receive Interface – Single Frame Transfer ................................................... 40
Figure 16: FIFO Receive Interface – Frame Transfer with data valid signal not continuously
high .................................................................................................................................... 40
Figure 17: FIFO Receive Interface – Frame Transfer with Error .............................................. 41
Figure 18: FIFO Receive Interface – Frame Transfer with User Pause ................................... 42
Figure 19: FIFO Sections Configuration and Signals ............................................................... 44
Figure 20: FIFO Sections Related Signaling ............................................................................ 44
Figure 21: Read in 32-bit Data Bus Mode................................................................................ 50
Figure 22: Read in 8-bit Data Bus Mode.................................................................................. 50
Figure 23: Write in 32-bit Data Bus Mode ................................................................................ 50
Figure 24: Write in 8-bit Data Bus Mode .................................................................................. 50
Figure 25: Power State Transitioning Diagram ........................................................................ 53
Figure 26: Auto negotiation Use Flow ...................................................................................... 55
UG029, September 6, 2013
Overview
The hardened 10/40/100 Gigabit Ethernet controller available in Achronix Speedster22i
FPGAs provides a flexible, high-performance, and power efficient networking interface.
The features include:
Fully integrated 10/40/100 Gigabit Ethernet MAC
Designed to the IEEE Std 802.3ba-2010 specification
Configurable full-duplex for 10/40/100 Gigabit Ethernet operation
5 – configurable modes of operation
o 1-12 x 10 Gigabit Ethernet Channels
o 1 x 100 Gigabit, 1-2 x 10 Gigabit Ethernet Channels
o 1-3 x 40 Gigabit Ethernet Channels
o 1-4 x 10 Gigabit, 1-2 x 40 Gigabit Ethernet Channels
o 1-8 x 10 Gigabit, 1 x 40 Gigabit Ethernet Channels
User-accessible raw statistic vector outputs (IEEE 802.3 basic, mandatory and
recommended Management Information packages (clause 30, MIB, MIB-II, IETF
RFC 2665, SNMP, RMON in accordance with IETF RFC 2819))
Provides counters to generate the applicable objects of the Management
Information Base (MIB, MIB-II) according to IETF RFC 2665 (including its update
to 10 Gbps) for SNMP (Simple Network Management Protocol) managed
environments.
Support for VLAN 802.1q VLAN Tag (VLAN Type and VLAN Info fields) frames
Configurable in-band Frame Check Sequence (FCS) field passing on both transmit
and receive paths
Auto padding on transmit and stripping on receive paths
Configured and monitored through a host interface
Each PCS Layer implements a X/XL/CGMII side loopback to the MAC, which
returns all data from the MAC transmit back to the MAC receive side without
passing through any the PCS blocks.
o 10G Base-R: The PCS transmits the constant pattern of 0x00ff
to the SerDes line interface.
o 40G/100G Base-R: The PCS transmits the MAC transmit data
unchanged to the SerDes line interface (as defined by
IEEE802.3ba).
Hardware-selectable Device Control Register (DCR) bus or generic host bus
interface
Configurable flow control through Ethernet MAC Control PAUSE frames;
symmetrically or asymmetrically enabled
Configurable support for jumbo frames of any length
8 UG029, September 6, 2013
9
Each PCS layer implements auto-negotiation, but does not include Parallel
Detection. Parallel Detection must be implemented in user logic when the remote
device does not support auto-negotiation or when auto-negotiation is disabled.
When operating in 10G mode of operation, the 10G MAC can implement a
configurable 10/100/1000 SGMII/1000Base-X PCS layer instead of the normal
XGMII/10GBase-R PCS layer to allow operations below 10Gbps.
Fully configurable Inter-Packet Gap (IPG) supports LAN and WAN with support
for Deficit Idle Count (DIC) to reduce bandwidth loss. Configurable in full-duplex
operation
Optional Frame Check Sequence (FCS) checking (add and delete)
PCS Lane Marker Insertion and deletion..
MAC support for Link Pause Flow Control and Priority Flow Control (PFC)
enables prioritization of up to 8 traffic classes.
PCS layer timestamp support enables IEEE 1588 precision time protocol.
UG029, September 6, 2013
Functional Description
FPGA Fabric
Interface
10/40/100
Gigabit Ethernet
Hard IP Core
sbus
12 RX
SerDes
Lanes
12 TX
SerDes
Lanes
Media
Access
Controller
(MAC)
Physical
Coding
Sublayer
(PCS)
RX FIFO
TX FIFO
Configuration / Control / Statistics
Physical
Media
Attachment
(PMA)
JTAG
Physical
Interface
(PHY)
10 UG029, September 6, 2013
Figure 1: 10/40/100 Gigabit Ethernet MAC Block Diagram
On the FPGA Fabric interface side, the 10/40/100 Gigabit Ethernet MAC and PCS Core
implements a flexible FIFO interface that can be connected to a custom user application.
On the Ethernet line side, the Core implements a 12 x 20-Bit line interface to the Physical
Media Attachment (PMA) module which consists of 12 x 10G SerDes lanes directly connected
to the FPGA I/O pins. The 12 SerDes lanes in the PMA module can be utilized independently
of the 10/40/100G Ethernet MAC. The physical interface for this module in configured via the
10/40/100G Ethernet MAC IP configuration wizard. Details of the Achronix SerDes I/O are
beyond the scope of the user guide. A separate user guide is available for the SerDes I/O
functionality.
X/XL/CGMII Loopbacks
Each PCS Layer implements a X/XL/CGMII side loopback to the MAC, which returns all data
from the MAC transmit back to the MAC receive side without passing through any of the
PCS blocks.
When the loopback is enabled the transmitted data is treated depending on the mode of
operation as follows:
10G Base-R: The PCS transmits the constant pattern of 0x00ff (8x'1' bits alternating
with 8x'0' bits) to the SerDes line interface.
40G/100G Base-R: The PCS transmits the MAC transmit data unchanged to the SerDes
line interface (as defined by IEEE802.3ba).
11
UG029, September 6, 2013
Interface Signal List
ref_clk
reset_ff_rx_clk_n[2:0]
reset_ff_tx_clk_n[2:0]
reset_n
reset_ref_clk_n
align_done[2:0]
block_lock[11:0]
hi_ber[11:0]
loc_fault[11:0]
rem_fault[11:0]
pma_{11:0}_pd[1:0]
pma_rx_cdr_lck2dat[11:0]
pma_rx_iddq_n[11:0]
pma_rxready[11:0]
pma_rxstat[11:0]
pma_sig_detect[11:0]
pma_synth_iddq_n[11:0]
pma_synthready[11:0]
pma_synthestat[11:0]
pma_tx_iddq_n[11:0]
pma_txready[11:0]
pma_txstat[11:0]
serdes_aprobe[11:0]
serdes_ck_ref_m[11:0]
serdes_ck_ref_p[11:0]
serdes_rx_m[11:0]
serdes_rx_p[11:0]
serdes_tx_m[11:0]
serdes_tx_p[11:0]
i_sbus_sw_rst
sbus_clk
reg_ts_avail[11:0]
reset_sbus_clk_n
i_sbus_req
i_sbus_data[1:0]
o_sbus_data[1:0]
o_sbus_ack
pma_{11:0}_i_sbus_req
pma_{11:0}_i_sbus_data[1:0]
pma_{11:0}_o_sbus_data[1:0]
pma_{11:0}_o_sbus_ack
pma{11:0}_pd
Fabric Transmit FIFO Interface
Fabric Receive FIFO Interface
Priority Flow Control
Transmitted Frame Status
Auto-Negotiation Control & Status
Global Signals
MAC/PCS Status
PMA TX/RX Interface
FPGA SerDes Off-Chip I/O pins
MAC & PMA Serial Control Bus
Time Stamp Timer
sys_clk
ff_clk[2:0]
ff_tx_data[767:0]
ff_tx_wren[11:0]
ff_tx_sop[11:0]
ff_tx_eop[11:0]
ff_tx_mod[71:0]
ff_tx_err[11:0]
ff_tx_crc[11:0]
ff_tx_rdy[11:0]
ff_tx_ovr[11:0]
ff_tx_id[3:0]
ff_tx_ts_frm
ff_tx_preamble_val
ff_tx_preamble[55:0]
ff_rx_rdy[11:0]
ff_rx_data[767:0]
ff_rx_dval[11:0]
ff_rx_sop[11:0]
ff_rx_eop[11:0]
ff_rx_mod[71:0]
ff_rx_err[11:0]
ff_rx_vlan[23:0]
ff_rx_afull[11:0]
ff_rx_err_stat[23:0]
ff_rx_ts[31:0]
ff_rx_preamble_val
ff_rx_preamble[55:0]
pfc_mode[11:0]
ff_tx_pfc_xoff{11:0}[7:0]
ff_tx_pfc_ack[11:0]
ff_rx_pfc_xoff{11:0}[7:0]
tx_ts_val
tx_ts_id[3:0]
tx_ts[31:0]
an_ena[11:0]
an_int[11:0]
an_done[11:0]
frc_in[31:0]
ts_clk
reset_ts_clk_n
12 UG029, September 6, 2013
Figure 2: Interface Signal List
13
Interface Signal Descriptions
Signal Name
Mode
Description
ref_clk
In
Reference Clock. Must be at least 652 MHz +/- 100ppm.
reset_n
In
Active low hard reset for all SerDes channels.
reset_ref_clk_n
In
Active low reset signal for ref_clk clock domain.
reset_ts_clk_n
In
Active low reset signal for ts_clk clock domain (if ts_clk is
used, see below).
reset_ff_tx_clk_n[2:0]
In
Active low reset signal for ff_tx_clk[2:0] clock domains.
reset_ff_rx_clk_n[2:0]
In
Active low reset signal for ff_rx_clk[2:0] clock domains.
Signal Name
Mode
Description
sys_clk
In
FPGA fabric System Clock. All the FIFO signals are
synchronized on sys_clk rising edge. The minimum
frequency for the system clock is a function of the interface
rate:
10G: at least 155 MHz
40G: at least 177 MHz
100G: at least 295 MHz
ff_clk[2:0]
In
FIFO Reference Clocks per FIFO group. Can be set to any
value required to get the required bandwidth on the FIFO
768-Bit interface. Can be independent from the System
clock, however the FIFO clock has to be at least 357.15 MHz
to allow for the start of frame to be always aligned on lane 0
for 40G mode and may be relaxed to at least 320.51 MHz for
the 10G or 100G modes.
ff_rx_data
[767:0]
Out
Receive Data. Refer to the ‘Fabic FIFO Interface’ for the
details of how to map this 768-bit bus to the individual
10/40/100G channels.
ff_rx_dval[11:0]
Out
Receive Data Valid per segment. Asserted (set to 1) by the
MAC to indicate that data on ff_rx_data, ff_rx_sop,
ff_rx_eop, ff_rx_mod, ff_rx_err, ff_rx_vlan,
ff_rx_err_stat and ff_rx_ts is valid.
ff_rx_sop[11:0]
Out
Receive Start of Frame per segment. Set to 1 when the first
data word of a frame is driven on ff_rx_data.
ff_rx_eop[11:0]
Out
Receive End of Frame per segment. Set to 1 when the final data
word of a frame is driven on ff_rx_data.
ff_rx_mod
[(12*6)-1:0]
Out
Receive Word Modulo per segment. Indicates which portion of the
final frame word is valid:
Bit 543210
000000 : ff_rx_data[63:0]/[255:0]/[511:0] is valid (for
10/40/100G)
000001 : ff_rx_data[7:0] is valid
000010 : ff_rx_data[15:0] is valid
000011 : ff_rx_data[23:0] is valid
000100 : ff_rx_data[31:0] is valid
000101: ff_rx_data[39:0] is valid
Global Signals
Receive FIFO Interface
Table 2 – Receive FIFO Interface (All syncrounous to sys_clk at user interface)
Table 1 – Global Signals
UG029, September 6, 2013
000110: ff_rx_data[47:0] is valid
000111: ff_rx_data[55:0] is valid
001000: ff_rx_data[63:0] is valid (40/100G only)
001001: ff_rx_data[71:0] is valid (40/100G only)
.....
.....
011110: ff_rx_data[239:0] is valid (40/100G only)
011111: ff_rx_data[247:0] is valid (40/100G only)
100000: ff_rx_data[255:0] is valid (100G only)
100001: ff_rx_data[263:0] is valid (100G only)
.....
.....
111110: ff_rx_data[495:0] is valid (100G only)
111111: ff_rx_data[503:0] is valid (100G only)
ff_rx_err[11:0]
Out
Receive Frame Error per segment. Asserted with the frame's final
data word to indicate that an error was detected when receiving the
frame. The type of error is coded on the status word
ff_rx_err_stat[23:0].
ff_rx_rdy[11:0]
In
Receive Ready per segment. The ff_rx_rdy signal is asserted high
to indicate to the Receive FIFO that it may transmit ff_rx_data.
Deasserting the ff_rx_rdy signal allows the user to pause the
reception of ff_rx_data, but the Receive FIFO Almost Full flag,
ff_rx_afull, must be monitored to prevent the Receive FIFO from
overflowing, resulting is a loss of data.
ff_rx_afull[11:0]
Out
Receive FIFO Almost Full flag per segment. The ff_rx_afull flag is
asserted high when there are 15 or fewer empty locations
remaining in the Receive FIFO.
ff_rx_vlan
[(12*2)-1:0]
Out
Receive Frame VLAN Indication per segment. Asserted with the
frame’s final data word to indicate that the current frame
implements a VLAN Tag (bit 0 asserted) or a Stacked VLAN Tag
(bit 1 asserted).
ff_rx_err_stat[23:0]
Out
Receive Frame Status and Error Indications. A status word is
available for each received frame with the final word (ff_rx_eop =
1). The receive frame status ff_rx_err_stat[23:0] can be
mapped to any segment of FIFO group 0 (10G: SEG0-3, 40G:
SEG0, 100G: SEG0).
_stat[0]: Set to 1 when the current frame has an invalid length,
i.e. less than 64 octets or more than the maximum value defined in
register FRM_LENGTH, or a mismatch between the payload
received and the payload length given within the frame was
detected.
_stat[1]: Set to 1 to indicate that the current frame was received
with a CRC-32 error.
_stat[2]: Set to 1 to indicate that the current frame was received
with a wrong or unexpected code during frame reception reported
by the reconciliation sub-layer function.
_stat[3]: Set to 1 to indicate that the current frame was truncated
because of a FIFO exception (Overflow).
_stat[4]: Set to 1 to indicate that a Sequence Error (Local or
Remote) was received from the PHY device during frame reception.
_stat[5]: Set to 1 to indicate that the current Frame implements a
Stacked VLAN Tag.
_stat[6]: Set to 1 to indicate that the current frame was received
with an Error control character on the XL/CGMII interface.
_stat[7]: Set to 1 to indicate that the current Frame implements a
VLAN Tag.
_stat[23:8]: Payload length of the frame. This is a copy of the
length/type field as it is found within the frame. For VLAN frames it
is a copy of the length/type field following the 4-octet VLAN tag.
ff_rx_ts[31:0]
Out
Receive Timestamp Value. Time when the MAC detected the
14 UG029, September 6, 2013
15
SFD of the frame.
Valid with ff_rx_sop. The receive timestamp
ff_rx_ts[31:0] can be mapped to any segment of FIFO
group 0 (10G: SEG0-3, 40G: SEG0, 100G: SEG0).
ff_rx_preamble_val
Out
Receive Frame Preamble Valid Indication. Asserted (set to
1) to indicate that a valid preamble is available on pin
ff_rx_preamble[55:0].
Note: Since the signal ff_rx_preamble_val is not a
pulse, the application should sample
ff_rx_preamble[55:0] when ff_rx_sop is set to 1.
ff_rx_preamble[55:0]
Out
Receive Frame Preamble. 56-Bit preamble of the current
frame, valid when ff_rx_preamble_val is set to 1. The
receive frame preamble ff_rx_preamble[55:0] can be
mapped to any segment of FIFO group 0 (10G: SEG0-3,
40G: SEG0, 100G: SEG0).
Signal Name
Mode
Description
ff_tx_data
[767:0]
In
Transmit Data. Refer to the ‘Fabic FIFO Interface’ for the
details of how to map this 768-bit bus to the individual
10/40/100G channels.
ff_tx_wren[11:0]
In
Transmit Data Write Enable per segment. Asserted by the
Transmit application to write data into the MAC Core FIFO.
ff_tx_sop[11:0]
In
Transmit Start of Frame per segment. Set to 1 when the
first data word of a frame is driven on ff_tx_data.
ff_tx_eop[11:0]
In
Transmit End of Frame per segment. Set to 1 when the
final data word of a frame is driven on ff_tx_data.
ff_tx_mod
[(12*6)-1:0]
In
Transmit Word Modulo per segment. Indicates which
portion of the final frame word is valid:
Bit 543210
000000 : ff_tx_data[63:0]/[255:0]/[511:0] is
valid (for 10/40/100G)
000001 : ff_tx_data[7:0] is valid
000010 : ff_tx_data[15:0] is valid
000011 : ff_tx_data[23:0] is valid
000100 : ff_tx_data[31:0] is valid
000101 : ff_tx_data[39:0] is valid
000110 : ff_tx_data[47:0] is valid
000111 : ff_tx_data[55:0] is valid
001000 : ff_tx_data[63:0] is valid (40/100G only)
001001: ff_tx_data[71:0] is valid (40/100G only)
.....
.....
011110: ff_tx_data[239:0] is valid (40/100G only)
011111: ff_tx_data[247:0] is valid (40/100G only)
100000: ff_tx_data[255:0] is valid (100G only)
100001: ff_tx_data[263:0] is valid (100G only)
.....
.....
111110: ff_tx_data[495:0] is valid (100G only)
Transmit FIFO Interface
Table 3 – Transmit FIFO Interface (all synchronous to ff_tx_clk[2:0])
UG029, September 6, 2013
111111: ff_tx_data[503:0] is valid (100G only)
ff_tx_err[11:0]
In
Transmit Frame Error per segment. Asserted with the
frame’s final data word to indicate that the transmitted
frame is invalid. When ff_tx_err is asserted, the frame
is transmitted to the XL/CGMII interface with a transmit
error.
ff_tx_crc[11:0]
In
Transmit CRC Append per segment. If set, a CRC field will
be appended to the frame. If cleared, the MAC does not
append a FCS to the frame. This signal must be valid
during ff_tx_sop assertion.
ff_tx_rdy[11:0]
Out
Transmit FIFO Ready per segment. When the ff_tx_rdy
signal is high, the user may send ff_tx_data to the transmit
FIFO of the addressed segment. When the ff_tx_rdy signal
is low, the transmit interface FIFO is almost full and the
user must stop sending data to the ff_tx_data port.
ff_tx_ovr[11:0]
Out
Transmit Overflow Error per segment. Asserted (set to 1)
as long as an overflow condition persists on the application
FIFO per segment. This signal can be used to trigger an
application interrupt.
ff_tx_id[3:0]
In
Frame Identifier. An arbitrary value that must be valid
during ff_tx_eop assertion that can be used to mark
specific frames. The frame identifier ff_tx_id[3:0] can
be mapped to any segment of FIFO group 0 (10G: SEG0-3,
40G: SEG0, 100G: SEG0).
The value is available at the transmit status pins
tx_ts_id[3:0] when the frame has been transmitted to
the PHY.
Has no further meaning inside the MAC besides the
forwarding to the transmit status.
ff_tx_ts_frm
In
IEEE 1588 Timing Frame Indication that must be valid
during ff_tx_eop assertion. The frame indication
ff_tx_ts_frm can be mapped to any segment of FIFO
group 0 (10G: SEG0-3, 40G: SEG0, 100G: SEG0). Allows
the application to mark specific 1588 event frames. When
set for a frame, its transmit timestamp will be returned on
tx_ts[31:0]
ff_tx_preamble_val
In
Transmit Frame Preamble Valid Indication. Should be
asserted with ff_tx_sop to indicate that the current frame
should be sent with the preamble provided on
ff_tx_preamble[55:0].
ff_tx_preamble[55:0]
In
Transmit Frame Preamble. 56-Bit preamble inserted in the
current frame, must be valid when ff_tx_preamble_val
is set to 1. The transmit frame preamble
ff_tx_preamble[55:0] can be mapped to any segment
of FIFO group 0 (10G: SEG0-3, 40G: SEG0, 100G: SEG0).
Signal Name
Mode
Description
pma_{11:0}_pd{1:0]
Input
Individual Lane power down state control:
11 – Coma Power State (P2) - Everything but receiver
detection + signal detect is disabled. Minimum power
consumption
PMA TX/RX Interface
16 UG029, September 6, 2013
Table 4 – PMA TX/RX Interface
17
10 – Slumber Power State (P1) - PLL is enabled. CDR
and Driver are disabled. Increased power consumption
01 – Doze Power State (P0s) - Everything but transmit
driver is enabled. Apprx. 20-30mW saved from the Wake
state.
00 – Wake Power State (P0) - Everything is Asserted.
Maximum power consumption.
pma_rx_cdr_lck2dat
[11:0]
Output
CDR Lock to Data status indicator
0 – CDR is locked to reference clock
1 – CDR is locked to data
pma_rx_iddq_n[11:0
]
Input
Individual Receive Lane disable/power-down control 1 –
Non-PD State - all analog circuits are enabled 0 – PD
State - all analog circuits are disabled. Analog Receiver
impedance is placed into High Impedance mode.
pma_rxready[11;0]
Output
Receive Lane Ready Status Signal:
0 – RX Lane is not ready for data transmission
1 – RX Lane is ready for data transmission
pma_rxstat[11:0]
Output
Receive Lane State Transition Status. Indicates when the
PMA has completed a requested state transition:
0 – RX Lane has not completed its state change
1 – RX Lane has completed its state change
pma_sig_detect[11:
0]
Output
Receiver Data Detection Status Signal.
0 – Indicates no/invalid data on receive pins
1 – Indicates valid data on receive pins
pma_synth_iddq_n[
11:0]
Input
Individual Synthesizer disable/power-down control
1 – Non-PD State - all analog circuits are enabled
0 – PD State - all analog circuits are disabled
pma_synthready[11:
0]
Output
SYNTH Ready Status Signal:
0 – SYNTH is not ready for data transmission
1 – SYNTH is ready for data transmission
pma_synthstat[11:0]
Output
SYNTH state transition status. Indicates when the PMA
has completed a requested state transition:
0 – SYNTH has not completed its state change
1 – SYNTH has completed its state change
pma_tx_iddq_n[11:0
]
Input
Individual Transmit Lane disable/power-down control:
1 – Non-PD State - all analog circuits are enabled
0 – PD State - all analog circuits are disabled
Signal Name
Mode
Description
pfc_mode[11:0]
Out
Per segment Priority Flow Control Mode. For each of the
12 segments, this signal represents the setting of the
PFC_MODE configuration register bit.
See COMMAND_CONFIG Register Bit Definitions on
page 67 for more details.
ff_tx_pfc_xoff{11:0}[
7:0]
In
Per segment transmit flow control generate. When PFC
Pause mode is enabled, for each of the 12 segments, an
8-bit input vector is used to signal the creation of PFC
control frames.
When Link Pause mode is enabled, Bit 0 of each segment
is used only.
Priority Flow Control Interface
UG029, September 6, 2013
Table 5 – Priority Flow Control Interface
ff_tx_pfc_ack[11:0]
Out
Per segment Transmit Flow Control Acknowledge. Each
segment provides an ACK back to the application when it
samples the ff_tx_pfc_xoff inputs to indicate that a
PFC/Pause control frame is about to be sent according to
the provided status.
ff_rx_pfc_xoff
[11:0][ 7:0]
Out
12 – 8bit bus interfaces on a per segment Receive Flow
Control Status. For each of the 12 segments, an 8-bit
vector indicating the current pause status for the 8
priorities based on the internal pause quanta counters that
were set when a PFC control frame was received. When
asserted, it indicates that a PFC pause condition is in
place for that priority and the upstream core logic should
not schedule further traffic for this class. When zero, this
indicates the pause condition is no longer present and
traffic can be scheduled for this class.
In Link Pause Frame mode, Bit 0 is asserted (set to 1) to
indicate that the transmit path is paused as a result of a
received XOFF Pause frame. The signal deasserts, when
the pause timer has expired and the transmitter is allowed
to transmit frames again.
Signal Name
Mode
Description
an_ena[11:0]
In
Per segment Default Auto-Negotiation Enable. If ‘1’, the autonegotiation process will start after reset de-assertion for the
respective segment. The application can also start the autonegotiation process by writing the
KXAN_CONTROL.an_enable bit with ‘1’.
an_int[11:0]
Out
Per segment Auto-Negotiation Page Received Interrupt.
Asserted when a new page is received. Active only when the
Page Received Interrupt pin is enabled by writing the
KXAN_CONTROL.page_rcv_int_en bit with ‘1’.
See
Control Register Bits (KXAN_CONTROL) page 89 for details.
an_done[11:0]
Out
Per segment Auto-Negotiation Done. If ‘1’, the autonegotiation process has completed.
Signal Name
Mode
Description
sbus_clk
In
Register Access Clock.
reset_sbus_clk_n
In
Active low reset signal for the Ethernet MAC register
interface controlled by the sbus_clk clock domain.
i_sbus_data[1:0]
In
Carries read/write indication, address and data to write
i_sbus_req
In
Asserted for 9-cycles in case of read and for 11-cycles in
case of write
i_sbus_sw_rst
In
Active high reset signal for the Serial Bus Interface
controlled by the sbus_clk clock domain.
o_sbus_ack
Out
Acknowledgment from register i/f once read or write is
Auto-Negotiation Control and Status
Table 6 – Auto-Negotiation Control and Status (all synchronous to sbus_clk)
Serial Bus Interface
18 UG029, September 6, 2013
Table 7 – Serial bus Interface (all synchronous to sbus_clk)
19
completed thru SBUS. During write it is valid for one cycle
to indicate the end of the transfer. This is asserted for 4cycles to validate 8-bit data at the end of read.
o_sbus_data[1:0]
Out
Contains read data for 4-cycles when o_sbus_ack is
asserted.
pma_[11:0]_i_sbus_
data[1:0]
In
Input serial data interface for PHY PMA internal registers.
pma_0_i_sbus_req
In
Request signal for starting a read or write transaction on the
serial interface for PHY PMA internal registers.
pma_0_o_sbus_ack
Out
Acknowledge signal for a complete read or write operation
on the serial interface for PHY PMA internal registers.
pma_0_o_sbus_dat
a[1:0]
Out
Output serial data interface for PHY PMA internal registers.
Signal Name
Mode
Description
serdes_ck_ref_m[11:0]
In
Management Data Clock.
serdes_ck_ref_p[11:0]
In
Management Data Input.
serdes_rx_m[11:0]
In
Management Data Output.
serdes_rx_p[11:0]
In
Management Data Output Enable (active low).
serdes_tx_m[11:0]
Out
Management Data transaction is ongoing
serdes_tx_p[11:0]
Out
Management Data transaction is ongoing
Signal Name
Mode
Description
tx_ts_val
Out
Timestamp Valid. Asserted for one ref_clk clock
cycle to indicate that tx_ts_id and tx_ts are valid.
The timestamp can be mapped to any segment of
FIFO group 0 (10G: SEG0-3, 40G: SEG0, 100G:
SEG0).
The signal is not asserted for internally generated
Pause frames.
tx_ts_id[3:0]
Out
Frame Identifier. The value that was provided by the
application at ff_tx_id[3:0] for the frame.
tx_ts[31:0]
Out
Frame Timestamp Value. Transmit timestamp value
for the frame sent with the sequence number set on
tx_ts_id.
Signal Name
Mode
Description
ts_clk
In
Clock for the timestamp timer.
Maximum frequency is 1/4 of the ref_clk to allow for
proper clock domain synchronization.
frc_in[31:0]
In
Current value of an externally provided free running
SerDes (off-chip) Interface
Table 8 – FPGA SerDes Off-Chip I/O pins (all synchronous to serdes_ck_ref_*)
Transmitted Frame Status
Table 9 – Transmitted Frame Status (all synchronous to ff_tx_clk[0])
Timestamp Timer
UG029, September 6, 2013
Table 10 – Timestamp Timer (all synchronous to ts_clk)
counter (FRC). Used for timestamping.
The value typically expresses nanoseconds within the
current one second interval, hence ranging from 0 to
10^9-1.
reg_ts_avail[11:0]
Out
Per segment Register TS_TIMESTAMP contains new
data. The pin is the direct representation of the
STATUS.ts_avail register bit: It asserts when a new
timestamp is stored and it becomes deasserted when
writing the STATUS.ts_avail bit with ‘1’.
Signal Name
Mode
Description
loc_fault[11:0]
Out
Local fault state indication from MAC RS layer per
segment.
rem_fault[11:0]
Out
Remote fault state indication from MAC RS layer per
segment.
block_lock[11:0]
Out
Lane block lock indication (if 1). In 10G Mode and
SGMII PCS active it indicates proper sync to 10B
comma characters.
align_done[2:0]
Out
Multi-lane alignment done indication. Relevant in
40G/100G mode only. Uses bits 0,1,2 when operating
in 40G mode for segments 0,4,8 respectively. Uses
bit 0 only when operating in 100G mode.
hi_ber[11:0]
Out
High Bit Error rate indication from HIBER monitor for
each lane.
MAC/PCS Status Indications
Table 11 – MAC/PCS Status Indications (all synchronous to ref_clk)
20 UG029, September 6, 2013
21
Implementation with ACE
Mode
10G
Channels
40G
Channels
100G
Channels
1
12 0 0 2 2 0 1 3 0 3 0 4 4 2 0 5 8 1 0
Software/Hardware Requirements
The ACE software suite has the following system requirements:
Platform:
Memory Requirements by design size:
Creating an Ethernet Instance
The ACE design suite documentation outlines how to install the software, launch it, and
setup your first project. Refer to ACE documentation to learn how to setup your first project.
Configuring the 10/40/100G Ethernet Core
o 64 bit Linux (RHEL/Centos)
o 64bit Windows 7
o Minimum: 12GB
o Recommended for < 100k LUTs: 16GB
o Recommended for 100k – 400k LUTs: 24GB
o Recommended for > 400k LUTs: 32GB
The 10/40/100G Ethernet core is automatically generated from a design wizard in the ACE
design tool suite. Simply launch the IP wizard, select the 10/40/100G Ethernet core from the
list of available IP and a configuration wizard will prompt the user for configuration options.
The options that are presented will be based on the number of lanes you chose and the speed
of each lane. Only certain combinations are available and the wizard will restrict the user to
only those 5 modes available:
Table 12 – Five modes of operation
The Speedster22i 10/40/100G Ethernet MAC IP wizard configuration menu is shown below.
The “Target Device” is selected by the user from a series of drop-down menus that will limit
the options in successive option choices. If we were to select zero 100G lanes we could select
two 40G and four 10G lanes. These options are bound by the five modes of operation defined
above.
Once the lane configuration are chosen, the user will need to define the clock speeds for the
reference clock, serial bus interface clock; and the transmit and receive clock for the lane
groups. Groups of four lanes share a common clock source.
UG029, September 6, 2013
Figure 3: 10/40/100G Ethernet MAC IP Wizard
Additionally, the user will need to select the placement of the core. The MAC cores will be
located at the bottom of the device. The individual device datasheets designate the location
and number of each core.
Lastly, the user will chose the SerDes lane configuration that determines the positions of the
chosen channels. Once the IP configuration options are entered, the user can select the
“Generate” button at the bottom of the dialog box and a new screen will appear with options
regarding the type and location of the files it will generate. First the user selects the
hierarchical instance
22 UG029, September 6, 2013
23
Figure 4: Generate IP Design Files dialog box
UG029, September 6, 2013
FPGA Fabric Interface
The fabric interface is the primary interface for the user to connect his design to the 10/40/100
Gigabit Ethernet core. The other side of the core is the dedicated PHY SerDes interface.
The user accesses the Ethernet core via asynchronous transmit and receive FIFO’s. These
FIFO’s have programmable watermarks that are configured by the user. All transfers to/from
the user application are handled independently of the Core operation, and the Core provides
a simple interface to user applications based on a FIFO almost-full flag.
PHY Interface
The Physical Interface (PHY) side of the core is hardwired to specific external SerDes I/O
pins. These pin locations will vary with device and package options, so refer to the datasheet
for your device for these locations. The PHY interface is highly configurable with complex
interactions between configuration registers. The Achronix IP wizard will manage these
configuration options for you. However a small subset of these configuration registers will
be exposed to the user. Refer to the detailed specification on internal PHY PMA registers
later in this document. Each of the twelve PHY channels has a dedicated serial bus for
configuring registers.
Interfacing the Ethernet Core to the FPGA Fabric
Data
The data connections are labeled (ff_tx_* & ff_rx_*). In order to support any packet size, the
transmit and receive interface clocks have to run faster than the nominal required clock
frequency (100Gbps/512b = 195.31 MHz). In 100G mode of operation, worst case is 65-byte
packets, which require two 64-byte words at the user interface to the Ethernet block.
Therefore, for 100G mode, the minimum required transmit/receive interface clock rate is 295
MHz. In the 10G and 40G modes that have narrower interfaces per lane, the data packs more
efficiently, so the required transmit/receive clock rates are lower at 155 MHz. and 177 MHz.
each. There is a simple flow control interface to user application based on a FIFO flag
scheme.
Serial Bus Interface
All internal registers for in the Ethernet core are configurable though a series of 13 serial bus
interfaces. There is one for each for the 12 SERDES channels and one for the MAC/PCS core
itself.
Simulation
In addition to synthesis and place and route functions, the Achronix software flow also
supports various stages of simulation.
24 UG029, September 6, 2013
25
.vp or .ve file extenstion
.vma file extenstion
Figure 5: Simulation Flow
Software simulation can be done pre-tool chain at the functional RTL level, post-synthesis at
the gate level, and post-route at the Achronix technology specific level.
Throughout the flow, various checkpoints can be done to insure that the design functionality
is kept intact. Figure 5 shows what files are generated at each step and how they are used in
the simulation framework.
At the RTL Design Description level, the FPGA designer’s behavioral RTL description is
compiled by the simulator.
At the Mapped Netlist level, the output of the synthesis tools (Synplify Pro
TM)
) is used. This
is the synchronous gate-level constructs that Achronix Speedster22i understands. It is a
Verilog netlist file that has have a *.vma extension.
At the Post P&R Netlist level, the output of the Achronix CAD Environment (ACE) will
generate a *_routed.vp or *_routed.ve netlist for simulation. This will exist in the project and
active implementation directory under “output”. This file is encrypted using the IEEE STD
1364-2005 Verilog encryption standard, but this file can be decrypted correctly by supported
simulators.
UG029, September 6, 2013
Clock Distribution
The clock frequency of the SerDes interface depends on the selected SerDes datapath width
(synthesis option). The ACE GUI allows the user to pick one of several frequencies.
The Figure below shows the system clock distribution for the 10/40/100 Gigabit Ethernet
MAC and PCS Core for the 20-Bit SerDes interface.
Figure 6: System clock distribution for the 20-Bit SerDes interface
On the FIFO interface, 3 individual clock signals are provided for both transmit
(ff_tx_clk[2:0]) and receive (ff_rx_clk[2:0]). When 100G mode of operation is selected, the
clock signals ff_tx_clk[2:0] and ff_rx_clk[2:0] and their respective reset lines have to be driven
from the same clock and reset sources. The system clock distribution diagram below shows
an example implementation for the external clock and reset multiplexers.
26 UG029, September 6, 2013
27
Figure 7: Example implementation for the FIFO clock and reset multiplexers
UG029, September 6, 2013
Reset Considerations
MAC Soft Reset
When the MAC control register (COMMAND_CONFIG) reset bit is written, the following
functions are executed:
Ongoing receive is terminated when next possible (graceful stop). A currently
received frame may be written truncated to the FIFO.
Transmit is disabled when next possible (graceful stop). This may lead to outgoing
frame corruption (frame not terminated but transmit switches to idle
immediately).
Transmit and Receive are disabled (COMMAND_CONFIG bits 0,1 reset to 0)
Pause timers are all reset and pause conditions are cleared
Receive Credit value is cleared (set 0)
RX and TX FIFOs are reset
Reset bit clears itself
FIFO / Credit Counter Reset
The receive FIFO credit counter can be initialized by writing the credit value to the MAC
register INIT_CREDIT followed by a write to register CREDIT_TRIGGER. This will enable
the MAC to begin writing received frame data into the FIFO. As long as the
CREDIT_TRIGGER has not been written, all incoming frames will be discarded.
The credit counter can be re-initialized any time during operation. This will abort any
ongoing receive activity and flush the receive FIFO (discarding frame data if any). Once the
FIFO has been flushed normal receive resumes. An ongoing transaction on the application
interface will be terminated cleanly by producing a final word with EOP and error being
asserted to the application.
PCS Reset
When the PCS control register (CONTROL1) reset bit is written, the following functions are
executed:
Alignment is lost (for 40G and 100G PCS layers) which eventually leads to local
fault indication to the MAC on XL/CGMII
Alignment FIFOs are flushed (for 40G and 100G PCS layers)
All error counters are reset to 0
Reset bit clears itself
When the reset is issued in 40G mode, the reset causes the 4 lanes of the segment to be reset.
When the reset is issued in 100G mode, the reset causes the 10 lanes used to be reset.
28 UG029, September 6, 2013
29
Fabric FIFO Interface
Configuration
Segment ID
Datapath Bits
Clocks
1-12 x 10G
SEG0 –
SEG11
[63:0] = SEG0
[127:64] = SEG1
[191:128] = SEG2
[255:192] = SEG3
[319:256] = SEG4
[383:320] = SEG5
[447:384] = SEG6
[511:448] = SEG7
[575:512] = SEG8
[639:576] = SEG9
[703:640] = SEG10
[767:704] = SEG11
ff_clk[0] = SEG0-3
ff_clk[1] = SEG4-7
ff_clk[2] = SEG8-11
1 x 100G,
1-2 x 10G
SEG0,
SEG10-11
[511:0] = SEG0
[703:640] = SEG10
[767:704 ] = SEG11
ff_ clk[2:0] = SEG0
ff_ clk[2] = SEG10-11
1-3 x 40G
SEG0, SEG4,
SEG8
[255:0] = SEG0
[ 511:256] = SEG4
[767:512] = SEG8
ff_ clk[0] = SEG0
ff_ clk[1] = SEG4
ff_ clk[2] = SEG8
1-4 x 10G,
1-2 x 40G
SEG0-3,
SEG4, SEG8
[63:0] = SEG0
[127:64] = SEG1
[191:128] = SEG2
[255:192] = SEG3
[511:256] = SEG4
[767:512] = SEG8
ff_ clk[0] = SEG0-3
ff_ clk[1] = SEG4
ff_ clk[2] = SEG8
1-8 x 10G,
1 x 40G
SEG0-7,
SEG8
[63:0] = SEG0
[127:64] = SEG1
[191:128] = SEG2
[255:192] = SEG3
[319:256] = SEG4
[383:320] = SEG5
[447:384] = SEG6
[511:448] = SEG7
[767:512] = SEG8
ff_ clk[0] = SEG0-3
ff_ clk[1] = SEG4-7
ff_ clk[2] = SEG8
Overview
The following table shows the segment definition and datapath bit assignments for each
configuration.
Table 13 – Modes of Operation
UG029, September 6, 2013
Transmit FIFO Interface Block Diagram
Figure 8: Transmit FIFO Interface Block Diagram
30 UG029, September 6, 2013
Loading...