Altera Arria 10 Avalon-MM DMA User Manual

Arria 10 Avalon-MM DMA Interface for

PCIe Solutions

User Guide

Last updated for Altera Complete Design Suite: 15.0

Subscribe Send Feedback

UG-01145_avm

m_dma

2015.05.14

101 Innovation Drive San Jose, CA 95134

www.altera.com

TOC-2

Datasheet............................................................................................................. 1-1

Getting Started with the Avalon-MM DMA ......................................................2-1

Arria 10 Avalon-MM DMA Interface for PCIe Datasheet.....................................................................1-1

Features ........................................................................................................................................................ 1-2

Release Information ....................................................................................................................................1-6

Device Family Support ...............................................................................................................................1-7

Example Designs..........................................................................................................................................1-7

Debug Features ............................................................................................................................................1-7

IP Core Verification ....................................................................................................................................1-7

Compatibility Testing Environment ............................................................................................1-8

Performance and Resource Utilization ....................................................................................................1-8

Recommended Speed Grades ....................................................................................................................1-8

Steps in Creating a Design for PCI Express............................................................................................. 1-9

Generating the Testbench ..........................................................................................................................2-2

Understanding the Simulation Generated Files ......................................................................... 2-4

Understanding Simulation Log File Generation......................................................................... 2-4

Simulating the Example Design in ModelSim.........................................................................................2-4

Running a Gate-Level Simulation..............................................................................................................2-5

Generating Quartus II Synthesis Files.......................................................................................................2-5

Creating a Quartus II Project .................................................................................................................... 2-5

Adding Virtual Pin Assignment to the Quartus II Settings File (.qsf)................................................. 2-6

Compiling the Design .................................................................................................................................2-6

Descriptor Controller Connectivity when Instantiated Separately.......................................................2-7

Parameter Settings.............................................................................................. 3-1

Physical Layout of Hard IP In Arria 10 Devices.................................................4-1

Altera Corporation

System Settings.............................................................................................................................................3-1

Interface System Settings ........................................................................................................................... 3-5

Base Address Register (BAR) Settings ......................................................................................................3-7

Device Identification Registers ..................................................................................................................3-8

PCI Express and PCI Capabilities Parameters ........................................................................................3-9

Device Capabilities ..........................................................................................................................3-9

Error Reporting .............................................................................................................................3-10

Link Capabilities ........................................................................................................................... 3-11

MSI and MSI-X Capabilities ........................................................................................................3-11

Power Management ......................................................................................................................3-12

PCIe Address Space Settings.................................................................................................................... 3-13

Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates..............................................4-4

TOC-3

Channel Placement and fPLL Usage for the Gen1 and Gen2 Data Rates............................................4-5

Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate...................................... 4-7

IP Core Interfaces ...............................................................................................5-1

Arria 10 DMA Avalon-MM DMA Interface to the Application Layer................................................5-1

Read DMA Avalon-MM Master Port .......................................................................................... 5-3

Write DMA Avalon-MM Master Port .........................................................................................5-5

RX Master Module ..........................................................................................................................5-5

TX Slave Module .............................................................................................................................5-7

32-Bit Non-Bursting Avalon-MM Control Register Access (CRA) Slave Signals .................5-8

Avalon-ST Descriptor Control Interface when Instantiated Separately .................................5-9

Descriptor Controller Interfaces when Instantiated Internally ..............................................5-11

Clock Signals ..............................................................................................................................................5-14

Reset, Status, and Link Training Signals.................................................................................................5-14

MSI Interrupts for Endpoints ................................................................................................................. 5-19

Hard IP Reconfiguration Interface .........................................................................................................5-20

Physical Layer Interface Signals ..............................................................................................................5-22

Serial Data Signals .........................................................................................................................5-22

PIPE Interface Signals .................................................................................................................. 5-22

Test Signals .................................................................................................................................... 5-27

Registers...............................................................................................................6-1

Correspondence between Configuration Space Registers and the PCIe Specification .....................6-1

Type 0 Configuration Space Registers ..................................................................................................... 6-5

PCI Express Capability Structures.............................................................................................................6-6

Altera-Defined VSEC Registers................................................................................................................. 6-8

CvP Registers................................................................................................................................................6-9

Uncorrectable Internal Error Mask Register ........................................................................................ 6-12

Uncorrectable Internal Error Status Register ....................................................................................... 6-13

Correctable Internal Error Mask Register .............................................................................................6-14

Correctable Internal Error Status Register ............................................................................................6-14

DMA Descriptor Controller Registers ...................................................................................................6-15

Read DMA and Write DMA Descriptor Format ..................................................................... 6-21

Read DMA Example .....................................................................................................................6-22

Software Program for Simultaneous Read and Write DMA .................................................. 6-25

Control Register Access (CRA) Avalon-MM Slave Port .....................................................................6-26

Arria 10 Reset and Clocks................................................................................... 7-1

Reset Sequence for Hard IP for PCI Express IP Core and Application Layer ....................................7-2

Clocks ........................................................................................................................................................... 7-4

Clock Domains ................................................................................................................................7-4

Clock Summary ...............................................................................................................................7-5

Error Handling ................................................................................................... 8-1

Physical Layer Errors ..................................................................................................................................8-2

Altera Corporation

TOC-4

Data Link Layer Errors ...............................................................................................................................8-2

Transaction Layer Errors ...........................................................................................................................8-3

Error Reporting and Data Poisoning ....................................................................................................... 8-5

Uncorrectable and Correctable Error Status Bits ...................................................................................8-6

IP Core Architecture........................................................................................... 9-1

Top-Level Interfaces ...................................................................................................................................9-3

Avalon-MM DMA Interface.......................................................................................................... 9-3

Clocks and Reset ............................................................................................................................. 9-3

Interrupts ......................................................................................................................................... 9-3

PIPE .................................................................................................................................................. 9-3

Data Link Layer ...........................................................................................................................................9-4

Physical Layer ..............................................................................................................................................9-6

Arria 10 Avalon-MM DMA for PCI Express ..........................................................................................9-8

Design Implementation.................................................................................... 10-1

Making Pin Assignments to Assign I/O Standard to Serial Data Pins ..............................................10-1

Recommended Reset Sequence to Avoid Link Training Issues ......................................................... 10-1

SDC Timing Constraints.......................................................................................................................... 10-2

Frequently Asked Questions.............................................................................. A-1

Additional Information......................................................................................B-1

Revision History for the Avalon-MM Interface with DMA..................................................................B-1

How to Contact Altera................................................................................................................................B-4

Typographic Conventions..........................................................................................................................B-5

Altera Corporation

2015.05.14

Bridge

and DMA

Engine

PCIe Hard IP

Block

PIPE

Interface

PHY IP Core

for PCIe

(PCS/PMA)

Serial Data

Transmission

Application

Layer

(User Logic)

Avalon-MM with

DMA Interface

www.altera.com

101 Innovation Drive, San Jose, CA 95134

Datasheet

UG-01145_avmm_dma

Send Feedback

Arria 10 Avalon-MM DMA Interface for PCIe Datasheet

Altera® Arria 10 FPGAs include a configurable, hardened protocol stack for PCI Express compliant with PCI Express Base Specification 3.0.

The Arria® 10 Hard IP for PCI Express with the Avalon ® Memory-Mapped (Avalon-MM) DMA interface removes some of the complexities associated with the PCIe protocol. For example, the IP core handles TLP encoding and decoding. In addition, the IP core includes Read DMA and Write DMA engines. If you have already architected your own DMA system with the Avalon-MM interface, you may want to continue to use that system. However, you may benefit from the simplicity of having the included DMA engines. New users of this protocol should use this IP core. This variant is available in Qsys for 128and 256-bit interfaces to the Application Layer. The Avalon-MM interface and DMA engines are implemented in FPGA soft logic.

Figure 1-1: Arria 10 PCIe Variant with Avalon-MM DMA Interface

The following figure shows the high-level modules and connecting interfaces for this variant.

that is

2015 Altera Corporation. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are trademarks of Altera Corporation and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services.

ISO 9001:2008 Registered

1-2

Features

UG-01145_avmm_dma

2015.05.14

Table 1-1: PCI Express Data Throughput

The following table shows the aggregate bandwidth of a PCI Express link for Gen1, Gen2, and Gen3 for 2, 4, and 8 lanes. The protocol specifies 2.5 giga-transfers per second for Gen1, 5.0 giga-transfers per second for Gen2, and

8.0 giga-transfers per second for Gen3. This table provides bandwidths for a single transmit (TX) or receive (RX) channel. The numbers double for duplex operation. Gen1 and Gen2 use 8B/10B encoding which introduces a 20% overhead. In contrast, Gen3 uses 128b/130b encoding which reduces the data throughput lost to encoding to less than 1%.

Units are Gigabits per second (Gbps).

Link Width

×2 ×4 ×8

PCI Express Gen1 (2.5 Gbps)

N/A N/A

PCI Express Gen2 (5.0 Gbps) N/A 16 Gbps 32 Gbps

PCI Express Gen3 (8.0 Gbps) 15.75 Gbps 31.51 Gbps 63Gbps

Related Information

• PCI Express Base Specification 3.0

• AN 456: PCI Express High Performance Reference Design

• Creating a System with Qsys

Features

New features in the Quartus® II 15.0 software release:

• Added Enable Altera Debug Master Endpoint (ADME) parameter to support optional Native PHY

• Added support for downstream burst read with a payload of size up to 4 KBytes, if Enable burst capability for RXM BAR2 port is turned on in the Parameter Editor. Previous maximum downstream read request payload size was 512 bytes.

16 Gbps

The Arria 10 Avalon-MM DMA for PCI Express supports the following features:

• Complete protocol stack including the Transaction, Data Link, and Physical Layers implemented as

• Native support for Gen1 x8, Gen2 x4, Gen2 x8, Gen3 x4, Gen3 x8 for Endpoints. The variant

• Dedicated 16 KByte receive buffer.

• Support for 128- or 256-bit Avalon-MM interface to Application Layer with embedded DMA up to

• Support for 32- or 64-bit addressing for the Avalon-MM interface to the Application Layer.

• Qsys example designs demonstrating parameterization, design modules, and connectivity.

• Extended credit allocation settings to better optimize the RX buffer space based on application type.

Altera Corporation

hard IP.

downtrains when plugged into a lesser link width or changes to a different maximum link rate.

Gen3 ×8 data rate.

Datasheet

Send Feedback

UG-01145_avmm_dma

2015.05.14

• Optional end-to-end cyclic redundancy code (ECRC) generation and checking and advanced error reporting (AER) for high reliability applications.

• Support for Arria 10 Avalon-MM DMA for PCI Express with either a 128- or 256-bit interface to the Application Layer. This variant includes an embedded DMA controller for data transfers. The following table shows the available configurations.

• Easy to use:

• Flexible configuration.

• No license requirement.

• Example designs to get started.

Table 1-2: Feature Comparison for all Hard IP for PCI Express IP Cores

The table compares the features of the four Hard IP for PCI Express IP Cores.

Features

1-3

Feature Avalon-ST Interface Avalon-MM

Interface

Avalon-MM DMA Avalon-ST Interface with SR-

IP Core License Free Free Free Free

Native

Supported Supported Supported Supported

Endpoint

Legacy Endpoint

(1)

Supported Not Supported Not Supported Not Supported

Root port Supported Supported Not Supported Not Supported

Gen1 ×1, ×2, ×4, ×8 ×1, ×2, ×4, ×8 Not Supported

Gen2 ×1, ×2, ×4, ×8 ×1, ×2, ×4, ×8 ×4, ×8

Gen3 ×1, ×2, ×4, ×8 ×1, ×2, ×4 ×2, ×4, ×8

64-bit Applica‐

Supported Supported Not supported Not supported

×8

×4, ×8

×2, ×4, ×8

tion Layer interface

IOV

(1)

Datasheet

128-bit

Supported Supported Supported Supported Application Layer interface

256-bit

Supported Not Supported Supported Supported Application Layer interface

Not recommended for new designs.

Send Feedback

Altera Corporation

1-4

Features

UG-01145_avmm_dma

2015.05.14

Feature Avalon-ST Interface Avalon-MM

Interface

Maximum payload size

Number of tags

128, 256, 512,

128, 256 bytes 128, 256 bytes 128, 256 bytes

1024, 2048 bytes

256 8 16 256 supported for non-posted requests

Automatically

Not supported Supported Supported Not supported handle out-oforder completions (transparent to the Application Layer)

Automatically

Not supported Supported Supported Not Supported handle requests that cross 4 KByte address boundary (transparent to the Application Layer)

Avalon-MM DMA Avalon-ST Interface with SR-

IOV

Polarity

Supported Supported Supported Supported Inversion of PIPE interface signals

Number of MSI requests

1, 2, 4, 8, 16, or 32 1, 2, 4, 8, 16, or 32 1, 2, 4, 8, 16, or 32 1, 2, 4, 8, 16, or 32 (for

Physical Functions)

MSI-X Supported Supported Supported Supported

Legacy

Supported Supported Supported Supported interrupts

Expansion

Supported Not supported Not supported Not supported ROM

Altera Corporation

Datasheet

Send Feedback

UG-01145_avmm_dma

2015.05.14

Features

Table 1-3: TLP Support Comparison for all Hard IP for PCI Express IP Cores

The table compares the TLP types that the four Hard IP for PCI Express IP Cores can transmit. Each entry indicates whether this TLP type is supported (for transmit) by endpoints (EP), Root Ports (RP), or both (EP/RP).

1-5

Transaction Layer

Packet type (TLP)

(transmit support)

Memory Read Request (Mrd)

Memory Read Lock Request (MRdLk)

Memory Write Request (MWr)

I/O Read Request (IORd)

I/O Write Request (IOWr)

Config Type 0 Read Request (CfgRd0)

Config Type 0 Write Request (CfgWr0)

Avalon-ST Interface Avalon-MM

Interface

Avalon-MM DMA Avalon-ST Interface with SR-

EP/RP EP/RP EP EP

EP/RP EP EP

EP/RP EP/RP EP EP

EP/RP EP/RP EP

RP RP EP

IOV

Datasheet

Config Type 1 Read Request (CfgRd1)

Config Type 1 Write Request (CfgWr1)

Message Request (Msg)

Message Request with Data (MsgD)

Completion (Cpl)

Completion with Data (CplD)

RP RP EP

EP/RP EP/RP EP

EP/RP EP/RP EP EP

EP/RP EP EP

Altera Corporation

Send Feedback

1-6

Release Information

UG-01145_avmm_dma

2015.05.14

Transaction Layer

Packet type (TLP)

(transmit support)

Completion-

Avalon-ST Interface Avalon-MM

Interface

Avalon-MM DMA Avalon-ST Interface with SR-

EP/RP EP Locked (CplLk)

Completion

EP/RP EP Lock with Data (CplDLk)

Fetch and Add

EP AtomicOp Request (FetchAdd)

The Arria 10 Avalon-MM DMA Interface for PCIe Solutions User Guide explains how to use this IP core and not the PCI Express protocol. Although there is inevitable overlap between these two purposes, use this document only in conjunction with an understanding of the PCI Express Base Specification.

Related Information

• Arria 10 Avalon-MM Interface for PCIe Solutions User Guide

• Arria 10 Avalon-ST Interface for PCIe Solutions User Guide

• Arria 10 Avalon-ST Interface with SR-IOV for PCIe Solutions User Guide

IOV

Release Information

Table 1-4: Hard IP for PCI Express Release Information

Item Description

Version 15.0

Release Date May 2015

Ordering Codes No ordering code is required

Product IDs

Vendor ID

The Product ID and Vendor ID are not required because this IP core does not require a license.

Altera Corporation

Datasheet

Send Feedback

UG-01145_avmm_dma

2015.05.14

Device Family Support

Table 1-5: Device Family Support

Device Family Support

1-7

Arria 10

Other device families Refer to the Altera's PCI Express IP Solutions web

Related Information

• Altera's PCI Express IP Solutions web page

Preliminary. The IP core is verified with prelimi‐ nary timing models for this device family. The IP core meets all functional requirements, but might still be undergoing timing analysis for the device family. It can be used in production designs with caution.

page for support information on other device families.

Example Designs

Qsys example designs are available for the Arria 10 Avalon-MM DMA for PCI Express IP Core. You can download them from the <install_dir>/ip/altera/altera_pcie/altera_pcie_a10_ed/example_design/a10 directory.

Related Information

Getting Started with the Avalon-MM DMA on page 2-1

Debug Features

Debug features allow observation and control of the Hard IP for faster debugging of system-level problems.

IP Core Verification

To ensure compliance with the PCI Express specification, Altera performs extensive verification. The simulation environment uses multiple testbenches that consist of industry-standard bus functional

Datasheet

Send Feedback

Altera Corporation

1-8

Compatibility Testing Environment

models (BFMs) driving the PCI Express link interface. Altera performs the following tests in the simulation environment:

• Directed and pseudorandom stimuli are applied to test the Application Layer interface, Configuration Space, and all types and sizes of TLPs

• Error injection tests that inject errors in the link, TLPs, and Data Link Layer Packets (DLLPs), and check for the proper responses

• PCI-SIG® Compliance Checklist tests that specifically test the items in the checklist

• Random tests that test a wide range of traffic patterns

Altera provides example designs that you can leverage to test your PCBs and complete compliance base board testing (CBB testing) at PCI-SIG, upon request.

Compatibility Testing Environment

Altera has performed significant hardware testing to ensure a reliable solution. In addition, Altera internally tests every release with motherboards and PCI Express switches from a variety of manufac‐ turers. All PCI-SIG compliance tests are run with each IP core release.

Performance and Resource Utilization

UG-01145_avmm_dma

2015.05.14

Because the PCIe protocol stack is implemented in hardened logic, it uses no core device resources (no ALMs and no embedded memory).

The Arria 10 variants include a soft logic bridge that functions as a front end to the hardened protocol stack. The following table shows the typical expected device resource utilization for selected configura‐ tions using the current version of the Quartus II software targeting a Arria 10 device. With the exception of M20K memory blocks, the numbers of ALMs and logic registers are rounded up to the nearest 50.

Table 1-6: Performance and Resource Utilization Arria 10 Avalon-MM DMA for PCI Express

Interface Width ALMs M20K Memory Blocks Logic Registers

128 1100 14 1650

256 1750 19 2600

Related Information

Fitter Resources Reports

Recommended Speed Grades

Recommended speed grades are pending characterization of production Arria 10 devices.

Related Information

• Area and Timing Optimization

• Altera Software Installation and Licensing Manual

Altera Corporation

Datasheet

Send Feedback

UG-01145_avmm_dma

2015.05.14

• Setting up and Running Analysis and Synthesis

Steps in Creating a Design for PCI Express

Before you begin

Select the PCIe variant that best meets your design requirements.

• Is your design an Endpoint or Root Port?

• What Generation do you intend to implement?

• What link width do you intend to implement?

• What bandwidth does your application require?

• Does your design require CvP?

1. Select parameters for that variant.

2. Simulate using an Altera-provided example design. All of Altera's static PCI Express example designs

are available under <install_dir>/ip/altera/altera_pcie/. Alternatively, generate an example design that matches your parameter settings, or create a simulation model and use your own custom or thirdparty BFM. The Qsys Generate menu generates simulation models. Altera supports ModelSim-Altera for all IP. The PCIe cores support the Aldec RivieraPro, Cadence NCsim, Mentor Graphics ModelSim, and Synopsys VCS and VCS-MX simulators.

3. Compile your design using the Quartus II software. If the versions of your design and the Quartus II software you are running do not match, regenerate your PCIe design.

4. Download your design to an Altera development board or your own PCB. Click on the All Develop‐ ment Kits link below for a list of Altera's development boards.

5. Test the hardware. You can use Altera's SignalTap® II Logic Analyzer or a third-party protocol analyzer to observe behavior.

6. Substitute your Application Layer logic for the Application Layer logic in Altera's testbench. Then repeat Steps 3–6. In Altera's testbenches, the PCIe core is typically called the DUT (device under test). The Application Layer logic is typically called APPS.

Steps in Creating a Design for PCI Express

1-9

Datasheet

Related Information

• Parameter Settings on page 3-1

• Getting Started with the Avalon-MM DMA on page 2-1

• All Development Kits

Altera Corporation

Send Feedback

2015.05.14

www.altera.com

101 Innovation Drive, San Jose, CA 95134

Getting Started with the Avalon-MM DMA

UG-01145_avmm_dma

Send Feedback

You can download this Qsys design example, ep_g3x8_avmm256_integrated.qsys, from the <install_dir>/

ip/altera/altera_pcie/altera_pcie_a10_ed/example_design/example_design/a10 directory.

The design example includes the following components:

Avalon-MM DMA for PCI Express

This IP core includes highly efficient DMA Read and DMA Write modules. The DMA Read and Write modules effectively move large blocks of data between the PCI Express address domain and the AvalonMM address domain using burst data transfers. Depending on the configuration you select, the DMA Read and DMA Write modules use either a 128- or 256-bit Avalon-MM datapath.

In addition to high performance data transfer, the DMA Read and DMA Write modules ensure that the requests on the PCI link adhere to the PCI Express Base Specification, 3.0. The DMA Read and Write engines also perform the following functions:

• Divide the original request into multiple requests to avoid crossing 4KByte boundaries.

• Divide the original request into multiple requests to ensure that the maximum payload size is equal to or smaller than the maximum payload size for write requests and maximum read request size for read requests.

• Supports out-of-order completions when the original request is divided into multiple requests to adhere to the read request size.

Using the DMA Read and DMA Write modules, you can specify descriptor entry table entries with large payloads.

On-Chip Memory IP core

This IP core stores the DMA data. This 32-KByte memory has a 256-bit data width.

Descriptor Controller

The Descriptor Controller manages the Read DMA and Write DMA modules. Host software programs the Descriptor Controller internal registers with the location of the descriptor table. The Descriptor Controller instructs the Read DMA module to copy the entire table to its internal FIFO. It then pushes the table entries to DMA Read or DMA Write modules to transfer data. The Descriptor Controller also sends DMA status upstream via an Avalon-MM TX slave port.

In this example design the Descriptor Controller parameter, Instantiate internal descriptor controller, is on. Consequently, the Descriptor Controller is integrated into the Avalon-MM bridge as shown in the figure below. Embedding the Descriptor Controller in Avalon-MM bridge simplifies the design. If you

ISO 9001:2008 Registered

Transaction,

Hard IP for PCIe

Data Link,

and

Physical

Layers

On-Chip Memory

DMA Data

Qsys System Design Arria 10 Hard IP for PCI Express

PCI Express

Link

Gen3 x8

Descriptor Controller

DMA Engine

Avalon-MM to

PCIe TLP

Bridge

Arria 10 Hard IP for PCI Express Using Avalon-MM Inteface and DMA

Interconnect

2-2

Generating the Testbench

plan to replace the Descriptor Controller IP core with your own implementation, do not turn on the Instantiate internal descriptor controller in the parameter editor when parameterizing the IP core.

Figure 2-1: Block Diagram of Arria 10 Avalon-MM DMA for PCI Express

Related Information

UG-01145_avmm_dma

2015.05.14

• Arria 10 Avalon-MM DMA for PCI Express on page 9-8

• DMA Descriptor Controller Registers on page 6-15

Generating the Testbench

1. Copy the example design, ep_g3x8_avmm256_integrated.qsys, from the installation directory:

<install_dir>/ip/altera/altera_pcie/altera_pcie_a10_ed/example_design/a10/ to your working directory.

2. Start Qsys, by typing the following command:

qsys-edit

3. Open ep_g3x8_avmm256_integrated.qsys.

Altera Corporation

Getting Started with the Avalon-MM DMA

Send Feedback

UG-01145_avmm_dma

2015.05.14

Figure 2-2: Arria 10 Avalon-MM DMA for PCI Express Qsys System Design

Generating the Testbench

2-3

4. Click Generate > Generate Testbench System. The Generation dialog box appears.

5. Specify the following parameters:

Table 2-1: Parameters to Specify in the Generation Dialog Box

Parameter Value

Testbench System

Create testbench Qsys system Standard, BFMs for standard Qsys interfaces

Create testbench simulation model Verilog

Allow mixed-language simulation You can leave this option off.

Output Directory

Testbench

<working_dir>/ep_g3x8_avmm256_integrated_tb

6. Click Generate. Qsys generates the testbench.

Getting Started with the Avalon-MM DMA

Send Feedback

Altera Corporation

2-4

Understanding the Simulation Generated Files

Table 2-2: Qsys Generation Output Files

Directory Description

UG-01145_avmm_dma

2015.05.14

<working_dir>/ep_g3x8_avmm256_integrated_tb/ep_ g3x8_avmm256_integrated_tb/

Includes directories for all components of the testbench. Also includes the following files:

• Simulation Package Descriptor File (.spd) which lists the required simulation files

• Comma-Separated Value File (.csv) describing the files in the testbench

<working_dir>/ep_g3x8_avmm256_integrated_tb/ep_ g3x8_avmm256_integrated_tb/sim/<cad_vendor>/

Includes testbench subdirectories for the Aldec, Cadence, Mentor, and Synopsys simulation tools with the required libraries and simulation scripts.

Understanding Simulation Log File Generation

Starting with the Quartus II 14.0 software release, simulation automatically creates a log file, altpcie_

monitor_<dev>_dlhip_tlp_file_log.log in your simulation directory.

Table 2-3: Sample Simulation Log File Entries

Time TLP Type Payload

(Bytes)

17989 RX CfgRd0 0004 04000001_0000000F_01080008 17989 RX MRd 0000 00000000_00000000_01080000

TLP Header

18021 RX CfgRd0 0004 04000001_0000010F_0108002C 18053 RX CfgRd0 0004 04000001_0000030F_0108003C 18085 RX MRd 0000 00000000_00000000_0108000C

Simulating the Example Design in ModelSim

1. In a terminal, change directory to <workingdir>/pcie_g3x8_integrated_tb/ep_g3x8_avmm256_integrated_tb/

sim/mentor.

2. Start the ModelSim® simulator.

3. To run the simulation, type the following commands in a terminal window: a. do msim_setup.tcl

b. ld_debug

The ld_debug command compiles all design files and elaborates the top-level design without any optimization.

c. run -all

The simulation performs the following operations:

Altera Corporation

Getting Started with the Avalon-MM DMA

Send Feedback

UG-01145_avmm_dma

2015.05.14

• Various configuration accesses after the link is initialized

• Setup of the DMA controller to read data from the Transaction Layer Direct BFM’s shared memory

• Setup of the DMA controller to write the same data back to the Transaction Layer Direct BFM’s shared memory

• Data comparison and report of any mismatch

Running a Gate-Level Simulation

The PCI Express testbenches run simulations at the register transfer level (RTL). However, it is possible to create you own gate-level simulations. Contact your Altera Sales Representative for instructions and an example that illustrate how to create a gate-level simulation from the RTL testbench.

Generating Quartus II Synthesis Files

1. On the Generate menu, select Generate HDL.

2. For Create HDL design files for synthesis, select Verilog.

You can leave the default settings for all other items.

3. Click Generate to generate files for Quartus II synthesis.

4. Click Finish when the generation completes.

Running a Gate-Level Simulation

2-5

Creating a Quartus II Project

You can create a new Quartus II project with the New Project Wizard, which helps you specify the working directory for the project, assign the project name, and designate the name of the top-level design entity.

1. On the Quartus II File menu, click then New Project Wizard, then Next.

2. Click Next in the New Project Wizard: Introduction (The introduction does not appear if you

previously turned it off.)

3. On the Directory, Name, Top-Level Entity page, enter the following information: a. For What is the working directory for this project, browse to <project_dir>/ep_g3x8_avmm256_

integrated/.

b. For What is the name of this project? browse to the <project_dir>/ep_g3x8_avmm256_integrated/

synth directory and select ep_g3x8_avmm256_integrated.v.

c. Click Next.

4. For Project Type select Empty project.

5. Click Next.

6. On the Add Files page, add <project_dir>/ep_g3x8_avmm256_integrated/synth/ep_g3x8_avmm256_

integrated.qip to your Quartus II project.Click

7. Click Next to display the Family & Device Settings page.

8. On the Device page, choose the following target device family and options:

Getting Started with the Avalon-MM DMA

Send Feedback

Altera Corporation

2-6

Adding Virtual Pin Assignment to the Quartus II Settings File (.qsf)

UG-01145_avmm_dma

a. In the Family list, select Arria 10 (GX/SX/GT). b. In the Devices list, select All. c. In the Available devices list, select the appropriate device. For Arria 10 ES2 development kits, select

10AX115S1F45I3SGE2.

9. Click Next to close this page and display the EDA Tool Settings page.

10.From the Simulation list, select ModelSim. From the Format list, select the HDL language you intend

to use for simulation.

11.Click Next to display the Summary page.

12.Check the Summary page to ensure that you have entered all the information correctly.

13.Click Finish.

14.Save your project.

Adding Virtual Pin Assignment to the Quartus II Settings File (.qsf)

To compile successfully you must add a virtual pin assignment statement for the PIPE interface to your .qsf file. The PIPE interface is useful for debugging, but is not a top-level interface of the IP core.

1. Browse to the synthesis directory that includes the .qsf for your project, <project_dir>/ep_g3x8_

avmm256_integrated/synth

2. Open ep_g3x8_avmm256_integrated.qsf.

3. Add the following assignment statement:

set_instance_assignment -name VIRTUAL_PIN ON -to pcie_a10_hip_0_hip_pipe_*

4. Save the .qsf file.

2015.05.14

Compiling the Design

1. Before compiling, you need to make a few changes to your top-level Verilog HDL file to create a design that you can successfully download to a PCB.

a. In the <project_dir>/ep_g3x8_avmm256/synth/, open ep_g3x8_avmm256_integrated.v. b. Comment out the declaration for pcie_a10_hip_0_hip_ctrl_test_in. c. Add a wire [31:0] pcie_a10_hip_0_hip_ctrl_test_in declaration to the same the same file. d. Assign pcie_a10_hip_0_hip_ctrl_test_in = 0x000000A8. e. Connect pcie_a10_hip_0_hip_ctrl_test_in to the test_in port on the Arria 10 Hard IP for

PCI Express instance.

2. On the Quartus II Processing menu, click Start Compilation.

3. After compilation, expand the TimeQuest Timing Analyzer folder in the Compilation Report. Note

whether the timing constraints are achieved in the Compilation Report.

If your design does not initially meet the timing constraints, you can find the optimal Fitter settings for your design by using the Design Space Explorer. To use the Design Space Explorer, click Launch Design Space Explorer on the Tools menu.

Altera Corporation

Getting Started with the Avalon-MM DMA

Send Feedback

UG-01145_avmm_dma

2015.05.14

Descriptor Controller Connectivity when Instantiated Separately

This Qsys design example block diagram shows how to connect the external Descriptor Controller to the Hard IP for PCI Expess with Avalon-MM DMA interface. This design example is available in <install_dir>/

ip/altera/altera_pcie/altera_pcie_hip_256_avmm/example_design/<dev>.

Figure 2-3: External Descriptor Controller Connectivity

2-7

Getting Started with the Avalon-MM DMA

Send Feedback

Altera Corporation

2015.05.14

www.altera.com

101 Innovation Drive, San Jose, CA 95134

Parameter Settings

UG-01145_avmm_dma

Send Feedback

System Settings

Table 3-1: System Settings for PCI Express

Parameter Value Description

Number of Lanes x1, x2, ×4, ×8 Specifies the maximum number of lanes supported. Avalon-

MM Interface with DMA does not support x1 configurations.

Lane Rate Gen1 (2.5 Gbps)

Gen2 (2.5/5.0 Gbps)

Gen3 (2.5/5.0/8.0

Gbps)

Port type Native Endpoint

Root Port

Specifies the maximum data rate at which the link can operate.

Specifies the port type. Altera recommends Native Endpoint for all new Endpoint designs. The Arria 10 Hard IP for PCI Express for the Avalon-MM Interface with DMA currently does not support Root Ports.

The Endpoint stores parameters in the Type 0 Configuration Space. The Root Port stores parameters in the Type 1 Configu‐ ration Space.

Application interface type

Avalon-ST

Avalon-MM

Selects either the Avalon-ST, Avalon-MM interface, or Avalon-MM interface with embedded DMA.

Avalon-MM DMA

Avalon-ST with SR-

IOV

RX Buffer credit allocation -

Minimum

Low

Determines the allocation of posted header credits, posted data credits, non-posted header credits, completion header credits, and completion data credits in the 16 KByte RX buffer.

ISO 9001:2008 Registered

3-2

System Settings

Parameter Value Description

UG-01145_avmm_dma

2015.05.14

performance for received requests

Balanced The 5 settings allow you to adjust the credit allocation to

optimize your system. The credit allocation for the selected setting displays in the message pane.

The Message window dynamically updates the number of credits for Posted, Non-Posted Headers and Data, and Completion Headers and Data as you change this selection.

Altera Corporation

Parameter Settings

Send Feedback

UG-01145_avmm_dma

2015.05.14

Parameter Value Description

System Settings

• Minimum RX Buffer credit allocation -performance for received requests )—configures the minimum PCIe

specification allowed for non-posted and posted request credits, leaving most of the RX Buffer space for received completion header and data. Select this option for variations where application logic generates many read requests and only infrequently receives single requests from the PCIe link.

• Low—configures a slightly larger amount of RX Buffer space for non-posted and posted request credits, but still dedicates most of the space for received completion header and data. Select this option for variations for which application logic generates many read requests and infrequently receives small bursts of requests from the PCIe link. This option is recommended for typical endpoint applications in which most of the PCIe traffic is generated by a DMA engine that is located in the endpoint application layer logic.

• Balanced—configures approximately half the RX Buffer space to received requests and the other half of the RX Buffer space to received completions. Select this option for applications in which the received requests and received completions are roughly equal.

• High—configures most of the RX Buffer space for received requests and allocates a slightly larger than minimum amount of space for received completions. Select this option if most of the PCIe requests are generated by the other end of the PCIe link and the local application layer logic only infrequently generates a small burst of read requests. This option is recommended for typical Root Port applications in which most of the PCIe traffic is generated by DMA engines located in the endpoints.

• Maximum—configures the minimum PCIe specification allowed amount of completion space, leaving most of the RX Buffer space for received requests. Select this option when most of the PCIe requests are generated by the other end of the PCIe link and the local application layer logic never or only infrequently generates single read requests. This option is recommended for control and status Endpoint applications that don't generate any PCIe requests of their own and are the target only of write and read requests from the root complex.

3-3

Use 62.5 MHz application clock

Parameter Settings

Send Feedback

On/Off

This mode is only available only for Gen1 ×1. It saves power.

Altera Corporation

3-4

System Settings

Parameter Value Description

UG-01145_avmm_dma

2015.05.14

Enable byte parity ports on Avalon-ST Interface

Enable multiple packets per cycle for the 256-bit interface

Enable configu‐ ration via Protocol (CvP)

Enable credit consumed selection port

Enable Configu‐ ration Bypass (CfgBP)

On/Off When On, the RX and TX datapaths are parity protected.

Parity is odd. This parameter is only available for the AvalonST interface.

On/Off When On, the 256-bit Avalon-ST interface supports the

transmission of TLPs starting at any 128-bit address boundary, allowing support for multiple packets in a single cycle. To support multiple packets per cycle, the Avalon-ST interface includes 2 start of packet and end of packet signals for the 256-bit Avalon-ST interfaces. This feature is only supported for Gen3 ×8.

On/Off When On, the Quartus II software places the Endpoint in the

location required for configuration via protocol (CvP). For more information about CvP, click the Configuration via Protocol (CvP) link below.

On/Off When you turn on this option, the core includes the tx_cons_

cred_sel port. This parameter is not available for the Avalon-

MM with DMA interface.

On/Off When you turn on this option, you can substitute a custom

Configuration Space implemented in soft logic for the Configuration Space included in the Hard IP for PCI Express. This option is only available for the Avalon-ST interface.

Enable dynamic reconfiguration of PCIe read-only registers

Enable Altera Debug Master Endpoint (ADME)

Related Information

Arria 10 Transceiver PHY User Guide

Provides information about the ADME feature for Arria 10 devices.

Altera Corporation

On/Off When On, you can use the Hard IP reconfiguration bus to

dynamically reconfigure Hard IP read-only registers. For more information refer to Hard IP Reconfiguration

Interface on page 5-20.

On/Off

When On, you can use the Altera System Console to read and write the embedded Arria 10 Native PHY registers.

Parameter Settings

Send Feedback

UG-01145_avmm_dma

2015.05.14

Interface System Settings

Table 3-2: Interface System Settings

Parameter Value Description

Interface System Settings

3-5

Application Interface width

Avalon-MM address width

Enable completer-only Endpoint

64-bit 128-bit 256-bit

32, 64

On/Off

Specifies the data width for the Application Layer to Transaction Layer interface. Refer to Application Layer

Clock Frequency for All Combinations of Link Width, Data Rate and Application Layer Interface Widths for all

legal combinations of data width, number of lanes, Application Layer clock frequency, and data rate. The Avalon-MM with DMA interface does not support the 64-bit interface width.

Specifies the address width for Avalon-MM RX master ports that access Avalon-MM slaves in the Avalon address domain. When you select 32-bit addresses, the PCI Express Avalon-MM bridge performs address translation. When you specify 64-bits addresses, no address translation is performed in either direction. The destination address specified is forwarded to the Avalon-MM interface without any changes.

For the Avalon-MM interface with DMA, this value must be set to 64.

In this mode, the Hard IP can receive requests, but cannot initiate upstream requests. However, it can transmit completion packets on the PCI Express TX link. This mode removes the Avalon-MM TX slave port and thereby reduces logic utilization.

Enable completer-only Endpoint with 4-byte payload

Parameter Settings

Send Feedback

On/Off

This is a non-pipelined version of Completer Only mode. At any time, only a single request can be outstanding. Single dword completer uses fewer resources than completer only Endpoint. This variant is targeted for systems that require simple read and write register accesses from a host CPU. If you select this option, the width of the data for RXM BAR masters is always 32 bits, regardless of the Avalon-MM width.

For the Avalon-MM interface with DMA, this value must be Off .

Altera Corporation

3-6

Interface System Settings

Parameter Value Description

UG-01145_avmm_dma

2015.05.14

Enable control register access (CRA) Avalon-MM slave port

Export MSI/MSI-X conduit interfaces

Enable PCIe interrupt at power-on

On/Off

Allows read and write access to bridge registers from the interconnect fabric using a specialized slave port. This option is required for Requester/Completer variants and optional for Completer Only variants. Enabling this option allows read and write access to bridge registers, except in the Completer-Only single dword variations.

When you turn this option on, the core exports top-level MSI and MSI-X interfaces that you can use to implement a Custom Interrupt Handler for MSI and MSI-X interrupts. For more information about the Custom Interrupt Handler, refer to Interrupts for End

Points Using the Avalon-MM Interface with Multiple

MSI/MSI

X Support. If you turn this option Off, the

core handles interrupts internally. If you select this option, you must design your own

external descriptor controller. The embedded controller does not support MSI-X.

When you turn this option on, the Avalon-MM Arria 10 Hard IP for PCI Express enables the interrupt register at power-up. Turning off this option disables the interrupt register at power-up. The setting does not affect runtime configuration of the interrupt enable register.

Enable Hard IP Status Bus when using the AVMM interface

Instantiate internal descriptor controller

For the Avalon-MM interface with DMA, this value must be off.

On/Off When you turn this option on, your top-level variant

includes the following top-level signals:

• Link status signals

• ECC error signals

• TX and RX parity error signals

• Completion header and data signals, indicating the total number of Completion TLPs currently stored in the RX buffer.

On/Off When you turn this option on, the descriptor controller

is included in the Avalon-MM bridge. When you turn this option off, the descriptor controller should be included as a separate external component. Turn this option on, if you plan to use the Altera-provided descriptor controller in your design. Turn this option off if you plan to modify or replace the descriptor controller logic in your design.

Altera Corporation

Parameter Settings

Send Feedback

UG-01145_avmm_dma

2015.05.14

Base Address Register (BAR) Settings

Parameter Value Description

3-7

Enable burst capabilities for RXM BAR2 ports

On/Off When you turn on this option, the BAR2 RX Avalon-

MM master is burst capable. If BAR2 is 32 bits and Burst capable, then BAR3 is not available for other use. If BAR2 is 64 bits, the BAR3 register holds the upper 32 bits of the address.

Address width of accessible PCIe memory space

Number of address pages 2, 4, 8, 16, 32, 64,

20–64 Specifies the size of the PCIe memory space. The value

you specify sets the width of the TX slave address, txs_

address for 64-bit addresses.

Specifies the number of consecutive address pages in the

128, 256, 512

PCI Express address domain. This parameter is only necessary for 32-bit addresses.

Size of address pages 4 KByte–4GByte Sets the size of the PCI Express system pages. All pages

must be the same size. This parameter is only necessary for 32-bit addresses.

Related Information

coreclkout_hip on page 7-5

Base Address Register (BAR) Settings

The type and size of BARs available depend on port type.

Table 3-3: BAR Registers

Parameter Value Description

Type Disabled

64-bit prefetchable memory 32-bit non-prefetchable memory 32-bit prefetchable memory I/O address space

If you select 64-bit prefetchable memory, 2 contiguous BARs are combined to form a 64-bit prefetchable BAR; you must set the higher numbered BAR to Disabled.

Defining memory as prefetchable allows contiguous data to be fetched ahead. Prefetching memory is advantageous when the requestor may require more data from the same region than was originally requested. If you specify that a memory is prefetch‐ able, it must have the following 2 attributes:

• Reads do not have side effects

• Write merging is allowed The 32-bit prefetchable memory and I/O address

space BARs are only available for the Legacy Endpoint.

Parameter Settings

Send Feedback

Altera Corporation

3-8

Device Identification Registers

Parameter Value Description

UG-01145_avmm_dma

2015.05.14

Size

N/A Qsys automatically calculates the required size after

you connect your components.

Device Identification Registers

Table 3-4: Device ID Registers

The following table lists the default values of the read-only Device ID registers. You can use the parameter editor to change the values of these registers. Refer to Type 0 Configuration Space Registers for the layout of the Device Identification registers.

Vendor ID 16 bits 0x00000000

Device ID 16 bits 0x00000001 Sets the read-only value of the Device ID register.

Sets the read-only value of the Vendor ID register. This parameter can not be set to 0xFFFF per the PCI Express Specification.

Address offset: 0x000.

Device ID

16 bits

0x00000000

Sets the read-only value of the Device ID register. Address offset: 0x000.

Revision ID 8 bits 0x00000000 Sets the read-only value of the Revision ID register.

Address offset: 0x008.

Class code 24 bits 0x00000000 Sets the read-only value of the Class Code register.

Address offset: 0x008.

Subsystem Vendor ID

16 bits 0x00000000 Sets the read-only value of the Subsystem Vendor ID

register in the PCI Type 0 Configuration Space. This parameter cannot be set to 0xFFFF per the PCI Express Base Specification. This value is assigned by PCI-SIG to the device manufacturer.

Address offset: 0x02C.

Subsystem Device ID

16 bits 0x00000000 Sets the read-only value of the Subsystem Device ID

Altera Corporation

Parameter Settings

Send Feedback

UG-01145_avmm_dma

2015.05.14

Related Information

PCI Express Base Specification 2.1 or 3.0

PCI Express and PCI Capabilities Parameters

This group of parameters defines various capability properties of the IP core. Some of these parameters are stored in the PCI Configuration Space - PCI Compatible Configuration Space. The byte offset indicates the parameter address.

Device Capabilities

Table 3-5: Capabilities Registers

Parameter Possible Values Default Value Description

PCI Express and PCI Capabilities Parameters

3-9

Maximum payload size

Completion timeout range

128 bytes 256 bytes

512 bytes 1024 bytes 2048 bytes

ABCD

BCD ABC

B A

None

128 bytes Specifies the maximum payload size supported. This

parameter sets the read-only value of the max payload size supported field of the Device Capabilities register (0x084[2:0]). Address: 0x084.

The Maximum payload size is 256 Bytes for the

Avalon-MM interface and for the Avalon-MM with DMA interface.

ABCD Indicates device function support for the optional

completion timeout programmability mechanism. This mechanism allows system software to modify the completion timeout value. This field is applicable only to Root Ports and Endpoints that issue requests on their own behalf. This parameter must be set to NONE for the Avalon-MM with DMA interface. Completion timeouts are specified and enabled in the Device Control 2 register (0x0A8) of the PCI Express Capability Structure Version. For all other functions this field is reserved and must be hardwired to 0x0000b. Four time value ranges are defined:

• Range A: 50 us to 10 ms

• Range B: 10 ms to 250 ms

• Range C: 250 ms to 4 s

• Range D: 4 s to 64 s

Parameter Settings

Send Feedback

Altera Corporation

3-10

Error Reporting

Parameter Possible Values Default Value Description

Bits are set to show timeout value ranges supported. The function must implement a timeout value in the range 50 s to 50 ms. The following values specify the range:

• None—Completion timeout programming is not supported

• 0001 Range A

• 0010 Range B

• 0011 Ranges A and B

• 0110 Ranges B and C

• 0111 Ranges A, B, and C

• 1110 Ranges B, C and D

• 1111 Ranges A, B, C, and D

All other values are reserved. Altera recommends that the completion timeout mechanism expire in no less than 10 ms.

UG-01145_avmm_dma

2015.05.14

Disable completion timeout

On/Off On Disables the completion timeout mechanism. When On,

the core supports the completion timeout disable mechanism via the PCI Express Device Control

implement the actual completion timeout mechanism for the required ranges.

Error Reporting

Table 3-6: Error Reporting

Parameter Value Default Value Description

Advanced error reporting (AER)

ECRC checking

On/Off Off When On, enables the Advanced Error Reporting (AER)

capability.

On/Off Off When On, enables ECRC checking. Sets the read-only

value of the ECRC check capable bit in the Advanced

Error Capabilities and Control Register. This

parameter requires you to enable the AER capability.

ECRC generation

Altera Corporation

On/Off Off

When On, enables ECRC generation capability. Sets the read-only value of the ECRC generation capable bit in the Advanced Error Capabilities and Control

AER capability.

Parameter Settings

Send Feedback

+ 103 hidden pages

Altera Arria 10 Avalon-MM DMA User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Contents

Datasheet

Arria 10 Avalon-MM DMA Interface for PCIe Datasheet

Features

Release Information

Device Family Support

Example Designs

Debug Features

IP Core Verification

Compatibility Testing Environment

Performance and Resource Utilization

Recommended Speed Grades

Steps in Creating a Design for PCI Express

Getting Started with the Avalon-MM DMA

Generating the Testbench

Understanding the Simulation Generated Files

Understanding Simulation Log File Generation

Simulating the Example Design in ModelSim

Running a Gate-Level Simulation

Generating Quartus II Synthesis Files

Creating a Quartus II Project

Adding Virtual Pin Assignment to the Quartus II Settings File (.qsf)

Compiling the Design

Descriptor Controller Connectivity when Instantiated Separately

Parameter Settings

System Settings

Interface System Settings

Base Address Register (BAR) Settings

Device Identification Registers

PCI Express and PCI Capabilities Parameters

Device Capabilities

Error Reporting