KCU105 PCI Express
Control Plane TRD
User Guide
KUCon-TRD01
Vivado Design Suite
UG918 (v2017.2) July 18, 2017
Revision History
Send Feedback
The following table shows the revision history for this document.
Date Version Revision
07/18/2017 2017.2 Released with Vivado Design Suite 2017.2 with no changes from previous
version.
07/17/2017 2017.1 Released with Vivado Design Suite 2017.1. Updated Figure 3-2 through
Figure 3-5 and Figure 4-1 through Figure 4-6. Updated commands in Run the
Design in Chapter 3, step 2.
02/14/2017 2016.4 Released with Vivado Design Suite 2016.4 with no changes from previous
version.
10/05/2016 2016.3 Released with Vivado Design Suite 2016.3 with no changes from previous
version.
06/08/2016 2016.2 Released with Vivado Design Suite 2016.2 with no changes from previous
version.
04/14/2016 2016.1 Released with Vivado Design Suite 2016.1 with no changes from previous
version.
11/24/2015 2015.4 Released with Vivado Design Suite 2015.4 with no changes from previous
version.
10/05/2015 2015.3 Released with Vivado Design Suite 2015.3 with minor textual edits.
06/30/2015 2015.2 Released with Vivado Design Suite 2015.2 with no changes from previous
version.
05/05/2015 2015.1 Updated for Vivado Design Suite 2015.1. TRD ZIP file changed to
rdf0305-kcu105-trd01-2015-1.zip. Updated Information about resource
utilization for the base design and the user extension design in Ta bl e 1-1 and
Tabl e 1- 2. Added information about Windows 7 driver support of the reference
design, updating these: sections: Features, Computers, Software, and
Appendix A, Directory Structure. Updated Figure 5-3, Figure A-1, and Tab le A-1
to include Windows information. The section Install TRD Drivers on the Host
Computer (Windows 7) was added to Chapter 2, Setup. The section Using the
QuestaSim/ModelSim Simulator was removed from Chapter 4, Implementing
and Simulating the Design, because QuestaSim simulation is not supported in
Vivado tool release 2015.1.
02/26/2015 2014.4.1 Initial Xilinx release.
PCI Express Control Plane TRD www.xilinx.com 2
UG918 (v2017.2) July 18, 2017
Table of Contents
Send Feedback
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Chapter 1: Introduction
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Resource Utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Chapter 2: Setup
Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Preliminary Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 3: Bringing Up the Design
Set the Host System to Boot from the LiveDVD (Linux) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Configure the FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Run the Design on the Host Computer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Test the Reference Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Remove Drivers from the Host Computer
(Windows Only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Chapter 4: Implementing and Simulating the Design
Implementing the Base Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Implementing the User Extension Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Simulating the Base Design Using Vivado Simulator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Chapter 5: Targeted Reference Design Details and Modifications
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Reference Design Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
PCI Express Control Plane TRD www.xilinx.com 3
UG918 (v2017.2) July 18, 2017
Appendix A: Directory Structure
Send Feedback
Appendix B: Recommended Practices and Troubleshooting in Windows
Recommended Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Appendix C: Additional Resources and Legal Notices
Xilinx Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Solution Centers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Please Read: Important Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
PCI Express Control Plane TRD www.xilinx.com 4
UG918 (v2017.2) July 18, 2017
Introduction
352&(6625
5RRW
&RPSOH[
$;,3&,H
%ULGJH
$;,6ODYH
$;,0DVWHU
3&,H
'ULYHU
.HUQHO
VSDFH
3&,H
[*HQ
/LQN
397021
5HJLVWHUV
3)%$5
6<6021
ELWVDW
0+]
*8,
*
7
+
,QWHJUDWHG3&,H%ORFN
3&,H,3
:UDSSHU
,QWHJUDWHGEORFNRQ)3*$
;LOLQ[LQKRXVH,3
&XVWRPORJLFLQ)3*$
6RIWZDUHUXQQLQJRQ
KRVWFRPSXWHU
&RPSRQHQWRQKRVWFRPSXWHU
7KLUGSDUW\,3DQG)3*$
$;,,QWHUFRQQHFW
UG918_c1_01_021315
$;,%ORFN5$0
3)%$5
XCKU040-2FFVA1156E FPGA
KCU105 Evaluation Board
+RVW&RPSXWHU
Send Feedback
This document describes the features and functions of the PCI Express® Control Plane
targeted reference design (TRD). The TRD comprises a base design and a user extension
design. The user extension design adds custom logic on top of the base design. The
pre-built user extension design in this TRD adds another AXI block RAM controller to the
design and sets up ingress translations through BAR4 to access this memory space.
Overview
The TRD targets the Kintex® UltraScale™ XCKU040-2FFVA1156E FPGA running on the
KCU105 evaluation board. It demonstrates a control plane application using a PCI Express
Endpoint block in a x1 Gen1 configuration. Simple base address register (BAR)-mapped
read and write transactions are demonstrated using a kernel mode software driver
controlled by the Control & Monitoring graphical user interface (GUI). The top-level block
diagram of the TRD is shown in Figure 1-1.
Chapter 1
X-Ref Target - Figure 1-1
Figure 1-1: KCU105 PCI Express Control Plane Targeted Reference Design
PCI Express Control Plane TRD www.xilinx.com 5
UG918 (v2017.2) July 18, 2017
Chapter 1: Introduction
Send Feedback
An Expresso DMA Bridge Core from Northwest Logic (NWL) [Ref 1] is used to demonstrate
PCIe-to-AXI conversion of transactions. The downstream slaves include a power, voltage,
and temperature (PVT) module monitoring parameters from the FPGA system monitor and
AXI block RAM IP targeted to BARs.
Features
The TRD includes these features:
• PCIe x1 Gen1 Endpoint operating at 2.5 GigaTransfers per second (GT/s) per
lane/direction
Single physical function with support for three 64-bit BARs
°
• DMA Bridge IP Core
Ingress address translation capability
°
AXI3 interface
°
• 64-bit kernel space drivers for Linux and Windows 7, which run on the host computer
• Control and monitoring graphical user interface (GUI)
Resource Utilization
Tab le 1- 1 and Ta ble 1-2 list the resources used by the base and user extension designs after
synthesis. Place and route can alter these numbers based on placements and routing paths.
These numbers are to be used as a rough estimate of resource utilization. These numbers
might vary based on the version of the TRD and the tools used to regenerate the design.
Table 1-1: Base Design Resource Utilization
Resource Type Available Used Usage (%)
CLB registers 484,800 43,896 9.05
CLB LUT 242,400 27,431 11.32
Block RAM 600 22 3.66
MMCME3_ADV 10 1 10
Global Clock Buffers 240 3 1.25
BUFG_GT 120 5 4.17
SYSMONE1 1 1 100
IOB 520 16 3.08
PCI Express Control Plane TRD www.xilinx.com 6
UG918 (v2017.2) July 18, 2017
Chapter 1: Introduction
Send Feedback
Table 1-1: Base Design Resource Utilization (Cont’d)
Resource Type Available Used Usage (%)
GTHE3_CHANNEL 20 1 5
GTHE3_COMMON 5 0 0
Table 1-2: User Extension Design Resource Utilization
Resource Type Available Used Usage (%)
CLB Registers 484,800 44,395 9.16
CLB LUTs 242,400 27,817 11.48
Block RAM 600 24 4
MMCME3_ADV 10 1 10
Global Clock Buffers 240 3 1.25
BUFG_GT 120 5 4.17
SYSMONE1 1 1 100
IOB 520 16 3.08
GTHE3_CHANNEL 20 1 5
GTHE3_COMMON 5 0 0
PCI Express Control Plane TRD www.xilinx.com 7
UG918 (v2017.2) July 18, 2017
Setup
Send Feedback
This chapter identifies the hardware and software requirements, and the preliminary setup
procedures required prior to bringing up the targeted reference design.
Requirements
Hardware
Board and Peripherals
• KCU105 board with the Kintex® UltraScale™ XCKU040-2FFVA1156E FPGA
Chapter 2
• USB cable, standard-A plug to micro-B plug (Digilent cable)
• Power supply: 100 VAC–240 VAC input, 12 VDC 5.0A output
• ATX power supply
• ATX power supply adapter
Computers
A control computer is required to run the Vivado® Design Suite and confige the on-board
FPGA. It can be a laptop or desktop computer with any operating system supported by
Vivado tools, such as Redhat Linux or Microsoft® Windows 7.
The reference design test configuration requires a host computer comprised of the chassis,
containing a motherboard with a PCI Express slot, monitor, keyboard, and mouse. A DVD
drive is also required if a Linux operating system is used. If a Windows 7 operating system
is used, the 64-bit Windows 7 OS and the Java SE Development Kit 7 must be installed.
Software
Vivado Design Suite 2017.2 is required. The Fedora 20 LiveDVD, on which the TRD software
and GUI run, is only required if a Linux operating system is used.
PCI Express Control Plane TRD www.xilinx.com 8
UG918 (v2017.2) July 18, 2017
Chapter 2: Setup
Send Feedback
Preliminary Setup
Complete these tasks before bringing up the design.
Install the Vivado Design Suite
Install Vivado Design Suite 2017.2 on the control computer. Follow the installation
instructions provided in Vivado Design Suite User Guide Release Notes, Installation, and
Licensing (UG973) [Ref 2].
Download the Targeted Reference Design Files
1. Download rdf0 305-kcu105-trd01 -2017-2.zip from the Xilinx Kintex UltraScale
FPGA KCU105 Evaluation Kit - Documentation & Designs website
the hardware design, software drivers, and application GUI executables.
2. Extract the contents of the file to a working directory.
. This ZIP file contains
3. The extracted contents are located at <working_dir>/kcu105_control_plane.
The TRD directory structure is described in Appendix A, Directory Structure.
Install TRD Drivers on the Host Computer (Windows 7)
Note: This section provides steps to install KUCon-TRD drivers and is only applicable to a host
computer running Windows 7 64-bit OS. If running Linux, proceed to Set DIP Switches, page 11.
This section includes steps to set up and install KUCon-TRD drivers in a host computer
running the Windows 7 64-bit OS.
Disable Driver Signature Enforcement
Note: Windows only allows drivers with valid signatures obtained from trusted certificate
authorities to load in Windows 7 64-bit OS. Windows drivers provided for this reference design do
not have a valid signature. Therefore, you have to disable driver signature enforcement on the host
computer, as follows:
1. Power up the host system. Press F8 to go to the Advanced Boot Options menu.
2. Select the Disable Driver Signature Enforcement option as shown in Figure 2-1, and
press Enter.
PCI Express Control Plane TRD www.xilinx.com 9
UG918 (v2017.2) July 18, 2017
X-Ref Target - Figure 2-1
UG918_c2_01_040315
Send Feedback
Chapter 2: Setup
Figure 2-1: Disable Driver Signature Enforcement
PCI Express Control Plane TRD www.xilinx.com 10
UG918 (v2017.2) July 18, 2017
Chapter 2: Setup
UG918_c2_02_040715
Send Feedback
Install Drivers
1. From the Windows explorer, navigate to the folder in which the reference design is
downloaded (<dir>\kcu105_control_plane\software\wind ows\) and run the
setup file with Administrator privileges, as shown in Figure 2-2.
X-Ref Target - Figure 2-2
Figure 2-2: Run the Setup File with Administrator Privileges
2. Click Next after the InstallShield Wizard opens.
3. Click Next to install to the default folder; or click Change to install to a different folder.
4. Click Install to begin driver installation.
5. A warning screen displays as the drivers are installed, because the drivers ar e not signed
by a trusted certificate authority yet. To install the drivers, ignore the warning message
and click Install this driver software anyway. This warning message pops up two
times. Repeat this step.
6. After installation is complete, click Finish to exit the InstallShield Wizard.
Set DIP Switches
Ensure that the DIP switches and jumpers on the KCU105 board are set to the factory
default settings as identified in the Kintex UltraScale FPGA KCU105 Evaluation Board User
Guide (UG917) [Ref 3].
PCI Express Control Plane TRD www.xilinx.com 11
UG918 (v2017.2) July 18, 2017
X-Ref Target - Figure 2-3
UG918_c3_01_091314
Send Feedback
Chapter 2: Setup
Install the KCU105 Board
1. Remove all rubber feet and standoffs from the KCU105 board.
2. Power down the host chassis and disconnect the power cord.
CAUTION! Remove the power cord to prevent electrical shock or damage to the KCU105 board or other
components.
3. Ensure that the host computer is powered off.
4. Open the chassis. Select a vacant PCIe Gen3-capable expansion slot and remove the
expansion cover at the back of the chassis.
5. Plug the KCU105 board into the PCIe connector slot, as shown in Figure 2-3.
PCI Express Control Plane TRD www.xilinx.com 12
UG918 (v2017.2) July 18, 2017
Figure 2-3: PCIe Connector Slot
Chapter 2: Setup
UG918_c3_02_040715
Send Feedback
6. Connect the ATX power supply to the KCU105 board using the ATX power supply
adapter cable as shown in Figure 2-4.
Note:
A 100 VAC–240 VAC input, 12 VDC 5.0A output external power supply can be substituted for
the ATX power supply.
X-Ref Target - Figure 2-4
Figure 2-4: Power Supply Connection to the KCU105 Board
7. Slide the KCU105 board power switch SW1 to the ON position (ON/OFF is marked on the
board).
PCI Express Control Plane TRD www.xilinx.com 13
UG918 (v2017.2) July 18, 2017
Bringing Up the Design
Send Feedback
This chapter describes how to bring up and test the targeted reference design.
Set the Host System to Boot from the LiveDVD
(Linux)
Note: This section is only applicable to host computers running Linux. If running Windows 7,
proceed to Configure the FPGA.
1. Power on the host system and stop it in BIOS to select options to boot from the DVD
drive. BIOS options are entered by pressing DEL, F12, or F2 keys on most computers.
Chapter 3
Note:
first. Then power on the host system.
2. Place the Fedora 20 LiveDVD into the DVD drive.
3. Select the option to boot from DVD.
If an external power supply is used instead of the ATX power, the FPGA can be configured
PCI Express Control Plane TRD www.xilinx.com 14
UG918 (v2017.2) July 18, 2017
Chapter 3: Bringing Up the Design
UG918_c3_03_042115
Send Feedback
Configure the FPGA
While in BIOS, program the FPGA with the BIT file:
1. Connect the standard-A plu g to micro-B plug USB cable to the JTAG port on the KCU105
board and to the control computer laptop as shown in Figure 3-1.
X-Ref Target - Figure 3-1
Note:
The host system can remain powered on.
Figure 3-1: Connect the USB Cable to the KCU105 Board and Control Computer
Note: Figure 3-1 shows a Rev C board. The USB JTAG connector is on the PCIe panel for
production boards.
PCI Express Control Plane TRD www.xilinx.com 15
UG918 (v2017.2) July 18, 2017
Loading...