Zynq UltraScale+ MPSoC
ZCU106 Video Codec Unit
Targeted Reference Design
User Guide
UG1250 (v2019.1) May 29, 2019
Revision History
Send Feedback
The following table shows the revision history for this document.
Date Version Revision
05/29/2019 2019.1 Updated hardware and software tools for Vivado Design Suite 2019.1 and
Petalinux-2019.1. This release has all the designs supported in 2018.3 and the
following new designs:
• SCD feature in multi-stream design
• Multi-stream audio support
• PCIe based file transcoding
• PLDDR in HDMI pipeline
• SDI-RX and SDI-TX designs with 4:2:2 10 bit support
• SDI RX/TX design with audio support
12/05/2018 2018.3 Updated for hardware and software tools for Vivado Design Suite 2018.3. Updated for
HDMI video display, HDMI video capture and HDMI display with audio, 10G HDMI
video capture and HDMI display, 10G HDMI video capture and HDMI display with
SDSoC support, and SDI video display designs. Updated with complete VCU TRD
design details and with design components for the audio and streaming feature.
Added 1080p30 multi-stream support.
07/27/2018 2018.2 Updated for hardware and software tools for Vivado Design Suite 2018.2. Updated
Figure 1-3, Figure 3-2, Figure 3-3, Figure 3-9, Figure 3-11, Figure 5-1, and Figure 5-6.
06/29/2018 2018.1 Updated for hardware and software tools for Vivado Design Suite 2018.1.
02/15/2018 2017.4 Updated for hardware and software tools for Vivado Design Suite 2017.4. Updated
Figure 3-16 and Figure 3-17. Removed GStreamer Interface Library Description.
Limited release.
12/20/2017 2017.3 Updated for Vivado Design Suite 2017.3. Video support is upgraded from 4KP30 to
4KP60. Added multi-stream encode/decode support, pipelined MIPI video input, and
the HDMI TX video display pipeline. Updated Exported APIs. Limited release.
12/01/2017 2017.2 Initial Xilinx release. Limited release.
Zynq UltraScale+ VCU TRD User Guide 2
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Table of Contents
Send Feedback
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Chapter 1: Introduction
About this TRD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Zynq UltraScale+ MPSoC Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Chapter 2: Targeted Reference Design Details
Design Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Design Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Chapter 3: APU Software Platform
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
GUI Application (vcu_qt) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
GStreamer Application (vcu_gst_app) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
GStreamer Interface Library (vcu_gst_lib) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
AXI Performance Monitor (APM) Library (vcu_apm_lib) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Video Library (vcu_video_lib) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Zynq UltraScale+ VCU TRD User Guide 3
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Chapter 4: System Considerations
Boot Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Chapter 5: Hardware Platform
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Clocking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Video Pipelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Address Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Interrupt Map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Appendix A: Input Configuration File
Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Appendix B: Additional Resources and Legal Notices
Send Feedback
Xilinx Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Solution Centers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Documentation Navigator and Design Hubs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Please Read: Important Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Zynq UltraScale+ VCU TRD User Guide 4
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Introduction
Send Feedback
About this TRD
This document describes the features and functions of the Zynq® UltraScale+™ MPSoC
Video Codec Unit (VCU) targeted reference design (TRD). The VCU TRD is an embedded
video encoding/decoding application partitioned between the SoC processing system (PS),
VCU, and programmable logic (PL) for optimal performance. The design demonstrates the
capabilities and performance throughput of the VCU embedded macro block available in
Zynq UltraScale+ MPSoC devices.
The TRD serves as a platform to tune the performance parameters of the VCU to arrive at
optimal configurations for encoder and decoder blocks.
Chapter 1
The TRD demonstrates the following hard block features in the PS and PL:
• VCU hard block capable of performing up to 4K (3840 x 2160) @60 Hz
• Simultaneous encoding and decoding of single and multiple streams
• PS DisplayPort controller for 4K (3840 x 2160) @ 30 Hz
•PL-based HDMI-TX/SDI-TX for 4K (3840 x2160) @ 60 Hz
• GPU used for rendering a graphical user interface (GUI)
• Extensible platform uses:
GStreamer v1.14.4 pipeline architecture to construct a multimedia pipeline [Ref 1]
°
Standard Linux software frameworks
°
OpenMAX™ v1.1.2 based client interface for the VCU
°
Modular and hierarchical architecture (enables partner modules)
°
Configurable IP Subsystems
°
• System software configuration:
Linux symmetric multi-processing (SMP) on the application processing unit (APU)
°
Zynq UltraScale+ VCU TRD User Guide 5
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Chapter 1: Introduction
Send Feedback
This user guide describes the architecture of the reference design and provides a functional
description of its components. It is organized as follows:
• Chapter 1, Introduction (this chapter) provides a high-level overview of the Zynq
UltraScale+ MPSoC architecture, the reference design architecture, and a summary of
key features.
• Chapter 2, Targeted Reference Design Details gives an overview of the design modules
and design components that make up this reference design.
• Chapter 3, APU Software Platform describes the APU software platform covering the
middleware and operating system layers of the Linux software stack and the Linux
GStreamer application running on the APU.
• Chapter 4, System Considerations describes system architecture considerations
including boot flow, system address map, video buffer formats, and performance
analysis.
• Chapter 5, Hardware Platform describes the hardware platform of the design including
key PS and PL peripherals.
• Appendix A, Input Configuration File lists additional resources and references.
Zynq UltraScale+ VCU TRD User Guide 6
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
X-Ref Target - Figure 1-1
Zynq UltraScale+ MPSoC Processing System
Application Processing Unit
ARM®
Cortex™A53
NEON™
Floating PointUnit
32KB
D-Cache
w/ECC
32KB I-
Cache w/
Parity
Memory
Mgmt Unit
Trace
Macro Cell
GIC-400 SCU CCI/SMMU 1MB L2 w/ECC
1
2
3
4
Real-Time Processing Unit
ARM®
Cortex-R5
Vector Floating Point Unit
32KB
D-Cache
w/ECC
12KB
TCM
w/ECC
32KB
D-Cache
w/ECC
Trace
Macro
Cell
1
Memory Protection Unit
2
GIC
Memory
DDR4/3/3L, LPDDR4/3
ECC Support
256KB OCM
With ECC
System Control
DMA, Timers,
WDT, Resets,
Clocking, and Debug
General
Connectivity
GigE
CAM
UART
SPI
Quad SPI NOR
NAND
SD/EMMC
Zynq UltraScale+ MPSoC Programmable Logic
Storage & Signal Processing
Block RAM
UltraRAM
DSP
General-Purpose I/O
High-Performance HP I/O
High-Density HD I/O
High-Speed Connectivity
GTH
GTY
Inerlaken
100G EMAC
PCIe Gen4
Video Codec
H.265/H.264
System Monitor
High-Speed
Connectivity
Display Port
USB 3.0
SATA 3.1
PCIe Gen2
PS-GTR
Platform
Management Unit
Power
System
Management
Configuration &
Security Unit
Config AES
Decryption,
Authentication,
Secure Boot
TrustZone
Voltage/Temp Monitor
Graphics Processing Unit
ARM Mali™-400 MP2
Geometry
Processor
Two Pixel
Processors
Memory Management Unit
64KB L2 Cache
X20051-112718
Send Feedback
Chapter 1: Introduction
Zynq UltraScale+ MPSoC Overview
The Zynq device is a heterogeneous, multi-processing SoC built on the 16-nm FinFET
technology. Figure 1-1 shows a high-level block diagram of the device architecture and key
building blocks inside the processing system (PS) and the programmable logic (PL).
The MPSoC key features include:
Figure 1-1: Zynq UltraScale+ MPSoC Block Diagram
• Application processing unit (APU) with a 64-bit quad-core Arm® Cortex™-A53
processor
• Real-time processing unit (RPU) with a 32-bit dual-core Arm Cortex-R5 processor
•Multimedia blocks
Graphics processing unit (GPU) Arm Mali-400MP2
°
Video codec (encoder/decoder) unit up to 4K (3840 x 2160) 60 frames per second
°
(FPS)
Zynq UltraScale+ VCU TRD User Guide 7
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Chapter 1: Introduction
Send Feedback
DisplayPort controller interface up to 4K (3840 x 2160) 30 FPS
°
• High-speed peripherals
PCIe root complex and Endpoint (Gen1 or Gen2 x1, x2, and x4 lanes)
°
USB 3.0/2.0 with host, device and on-the-go (OTG) modes
°
SATA 3.1 host
°
• Low-speed peripherals
Gigabit Ethernet, controller area network (CAN), universal asynchronous
°
receiver-transmitter (UART), Serial Peripheral Interface (SPI), Quad SPI, NAND flash
memory, Secure Digital embedded Multimedia Card (SD/eMMC), inter IC (I2C), and
general purpose I/O (GPIO)
• Platform management unit (PMU)
• Configuration security unit (CSU)
• 6-port DDR controller with error correction code (ECC), supporting x32 and x64
DDR4/3/3L and LPDDR4/3
Reference Design Overview
The MPSoC has a heterogeneous processor architecture. The TRD makes use of multiple
processing units available inside the PS using this software configuration:
The APU consists of quad Arm Cortex-A53 cores configured to run in SMP Linux mode. The
main task of the ap plication is to configur e and control the video pipelines using a Qt v5.9.4
based graphical user application. See Figure 1-2.
Zynq UltraScale+ VCU TRD User Guide 8
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
X-Ref Target - Figure 1-2
ARM Cortex-A53-0
ApplicationOSProcessorPL
VCU_APM_LIB
VCU_GST_LIB
VCU_VIDEO_LIB
PCIe_LIB
VCU_GST_APP
VCU_QT
PCIe_TRANSCODE
ALSA
V4L2
DRM
pcie_ep_client
MEDIA
DMABUF
U10
ARM Cortex-A53-1
ARM Cortex-A53-2
ARM Cortex-A53-3
DisplayPort
GPU
USB
SATA
SD
TPG
HDMI-Tx
HDMI-Rx
CSI-Rx
VCU
PL DDR
SDI-Rx
SDI-Tx
PCIe
XDMA
I2S-Rx
I2S-Tx
SCD
X22060-041719
Send Feedback
Chapter 1: Introduction
Figure 1-2 shows the software state after the boot process has completed and the
individual applications have been started on the target processing units. The TRD does not
use virtualization and therefore does not run a hypervisor on the APU.
The APU application controls the following video data paths implemented in the PS and PL
(see Figure 1-3, page 11):
• Capture pipeline capturing video frames into DDR memory from a high definition
• Processing (memory-to-memory) pipeline includes VCU encode/decode. Video frames
• Display pipeline reading video frames from memory and sending them to a monitor via
• Audio Capture pipeline to capture audio frames from HDMI-RX, SDI-RX and I2S-RX
Figure 1-2: Key Reference Design Components
media interface (HDMI source connected through the PL, an image sensor on an FMC
daughter card connected via MIPI CSI-2 RX Subsystem through the PL, serial digital
interface (SDI) source connected through the PL, and a Test Pattern Generator (TPG)
implemented inside the PL. Additionally, video can be sourced from a SATA drive, USB
3.0 device, or an SD card, which is also used as a boot device.
are read from DDR memory, processed by the VCU, and written back to memory.
the DisplayPort TX Controller inside the PS, SDI Transmitter Subsystem through the PL
or the HDMI Transmitter Subsystem through the PL. The DisplayPort TX Controller
supports two layers—one for video, the other for graphics and the SDI Transmitter
Subsystem with mixer IP support up to four layers and HDMI Transmitter Subsystem
with mixer IP supports up to eight such layers.
interfaces.
The graphics layer is rendered by the GPU
.
Zynq UltraScale+ VCU TRD User Guide 9
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Chapter 1: Introduction
Send Feedback
• Audio Renderer pipeline to playback the audio frames through HDMI-TX, SDI-TX, DP,
and I2S-TX interfaces.
• Transcode pipeline to transfer the file from the HOST machine to the client board
(zcu106) through PCIe XDMA bridge interface in the PL. The file is passed to the VCU
encoder and decoder block for transcoding. The transcoded file is written back to
HOST machine using the PCIe XDMA bridge interface read channel.
Zynq UltraScale+ VCU TRD User Guide 10
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Full-fledged VCU TRD
+ + + + +
+ + + + + +
+ + + + +
+ + +
+ + +
+ +
+ +
+ +
+ +
SDx Design
HDMI 10G Design
HDMI Video Capture and HDMI
Display with Audio (HDMI/I2S)
SDI Main Design
SDI Rx Design
SDI Tx Design
HDMI Tx Design
HDMI Rx Design
Capture Pipeline
FB Write
VPSS
(Scaler Only)
SDI RX
Source
FB
Write
TPG
AXI
DMA
10G
Ethernet SS
FB Write
Encode
SDX
Accelerator
File
System
SATA/
USB/SD
Audio
Formatter
HDMI TX SS
AXI
DMA
10G
Ethernet SS
Video
Mixer
SDI TX
Video
Mixer
Source
VPSS
(Scaler Only)
VPSS (CSC)
Gamma
Demosaic
MIPI CSI-2
SS
Source
Sink
Sink
Processing Pipeline Output Pipeline GPU
Audio
Formatter
HDMI RX SS
FB
Write
VPSS
(Scaler Only)
Sink
APU
DDR Memory
Video Buffers Encoded Frame Buffers Decoded Frame Buffers Graphics Buffers
Source
Overall Diagram with All Pipelines
+
Audio
Formatter
I2S
Sink
Audio
Formatter
PL DDR
Decode
PS DDR
FB Write
Audio
Formatter
I2S
Source
Audio
Formatter
SCD
X20053-051719
Send Feedback
X-Ref Target - Figure 1-3
Chapter 1: Introduction
The TRD consists of nine designs which are highlighted in four colors as shown in
Figure 1-3.
Zynq UltraScale+ VCU TRD User Guide 11
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Figure 1-3: VCU TRD Block Diagram
Note:
In Figure 1-3, except for the VCU Audio design, HDMI pipelines in all other designs exclude
Audio Formatter IP and thus do not have audio.
Chapter 1: Introduction
Live video source
(HDMI/TPG/SDI/MIPI)
File/streaming source
PS CLK
(Si5341)
33.33 MHz
PS DDR Memory
Capture
Pipeline
(Audio/Video
Processing
Pipeline
Render
Pipeline
(Audio/Video
Video Sink
(HDMI/SDI/DP)
SFP_SI5328_out
Si5328
156.25 MHz
Audio Sink
(HDMI/SDI/DP/I2S
PL DDR Memory
Live audio source
(HDMI/SDI/I2S)
X19301-041719
Send Feedback
The remaining blocks are common to all designs. See Chapter 2, Targeted Reference Design
Details for more details.
The reference design targets the ZCU106 evaluation board. The board has an onboard
HDMI transmitter and receiver connector, SDI transmitter and receiver connector, and a
DisplayPort connector interface. The evaluation board provides the HDMI reference clock,
data recovery unit (DRU) clock, and the reference clock for the design. The PS_REF_CLK is
sourced from another dedicated clock generator present on the evaluation board.
Figure 1-4 shows the block diagram of the TRD along with the board components.
X-Ref Target - Figure 1-4
Zynq UltraScale+ VCU TRD User Guide 12
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Figure 1-4: High-Level Block Diagram of ZCU106 Device Architecture
Key Features
Target platforms and extensions:
• ZCU106 evaluation board (see ZCU106 Evaluation Board User Guide (UG1244)) [Ref 2]
• Optional: Leopard Imaging LI-IMX274MIPI-FMC image sensor daughter card [Ref 3]
• SDI Receiver - Blackmagic Design Teranex Mini HDMI to 12G converter
• SDI Transmitter - Blackmagic Design Teranex Mini 12G to HDMI converter
Xilinx tools:
Send Feedback
• Vivado® Design Suite 2019.1
• Xilinx® Software Development Kit (XSDK) 2019.1 [Ref 4]
• PetaLinux tools 2019.1
Hardware interfaces and IP:
•GPU
•Video inputs
TPG
°
HDMI RX
°
MIPI CSI-2 RX
°
File source (SD card, SATA and USB 3.0 drives)
°
SDI RX
°
Chapter 1: Introduction
Stream In
°
•Video outputs
DisplayPort TX controller
°
HDMI TX
°
SDI TX
°
• Audio Inputs
HDMI RX
°
SDI RX
°
I2S RX
°
• Audio Outputs:
HDMI TX
°
SDI TX
°
I2S TX
°
DP
°
Zynq UltraScale+ VCU TRD User Guide 13
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
• Video compression/decompression
VCU hard block
°
• Auxiliary peripherals
SD
°
I2C
Send Feedback
°
GPIO
°
1G/10G Ethernet
°
UART
°
USB 2.0/USB 3.0
°
AXI Performance Monitor (APM)
°
PCIe
°
Digilent PMOD audio card [I2S2]
°
3.5mm auxiliary cables
°
Speakers
°
Software components:
• Operating systems
Chapter 1: Introduction
APU: SMP Linux
°
• Linux frameworks/libraries
Video: Video4Linux (V4L2), Media controller
°
Audio: libalsa
°
Display: Direct Rendering Manager/Kernel Mode Setting (DRM/KMS), X-Server
°
(X.Org)
Graphics: Qt5, OpenGL ES2
°
• User application:
APU: GStreamer-based command line application, QT GUI application
°
Supported video formats:
• Input resolution
4Kp60 (3840 x 2160)
°
4Kp30 (3840 x 2160)
°
1080p60 (1920 x 1080)
°
Zynq UltraScale+ VCU TRD User Guide 14
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
1080p30 (1920 x 1080)
°
•Output resolution
4Kp60 (3840 x 2160) — HDMI only
°
4Kp30 (3840 x 2160) — HDMI and DisplayPort
°
Native 1080p60 on both DisplayPort and HDMI
°
• Pixel formats
Send Feedback
NV12
°
NV16
°
XV15
°
XV20
°
Chapter 1: Introduction
Zynq UltraScale+ VCU TRD User Guide 15
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Targeted Reference Design Details
Send Feedback
Design Modules
The VCU TRD consists of nine design modules (DMs). A short summary of each design
follows.
PL HDMI Video Capture
This module enables capture from the HDMI RX Subsystem implemented in the
programmable logic (PL) into a file or to Stream-out video. The video captured from the
HDMI RX S ubsystem is enc oded and stored in SD cards or USB/SATA drive s. Th e module can
Stream-out encoded data through an Ethernet interface.
Chapter 2
PL HDMI Video Display
This module enables video display to HDMI TX implemented in the PL. The video stored in
SD cards or USB/SATA drives is decoded and displayed on HDMI TX. The module can
Stream-in encoded data through an Ethernet interface and decode and display it on HDMI
TX.
Multi-Stream Audio Design
This module enables capture of audio data from I2S RX /HDMI RX and Video data from the
HDMI RX/MIPI RX Subsystem. The audio/video data can be played through HDMI TX in the
PL and recorded in SD cards or USB/SATA drives. This module can Stream-in/out the
audio/video data through an Ethernet interface. This design supports the following
streams:
•Stream 1
Input Source: Video and audio are captured from HDMI RX
°
Output Sink: Video and audio are played on HDMI TX
°
•Stream 2
Input Source: Video is captured from the MIPI RX Subsystem and audio is captured
°
from the I2S RX Subsystem
Zynq UltraScale+ VCU TRD User Guide 16
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Chapter 2: Targeted Reference Design Details
Send Feedback
Output Sink: Video is played on HDMI TX and audio is played on I2S TX
°
PL 10G HDMI Video Capture and HDMI Display
This module enables capture of video from an HDMI RX Subsystem implemented in the PL.
The video can be displayed through HDMI TX through the PL, and recorded in SD cards or
USB/SATA drives. The module can Stream-in or Stream-out encoded data through the 10G
Ethernet interface.
PL HDMI Video Capture and HDMI Display with SDSoC Support
The design has SDSoC™ tool support along with PL 10G HDMI Video Capture and HDMI
Display. The video stored in SD cards or USB/SATA drives is decoded and displayed on
HDMI TX. This module can Stream-in or Stream-out encoded data through an Ethernet
interface. It also supports raw pipeline (v4l2src > accelerator > display) playback.
The SDSoC tool allows estimating the performance increase, using high-level synthesis
(HLS) to create RTL from a C algorithm, and automatically inserts data movers along with
the required drivers.
PL SDI Video Capture and SDI Display with Audio
This module captures audio/video from the SDI RX Subsystem and playback of audio video
through the SDI TX Subsystem implemented in the PL. It can record encoded audio/video
streams in SD cards or USB/SATA drives. This module can Stream-in/out the audio video
data through an Ethernet interface.
PL SDI Video Display
This module enables the video display to the SDI TX Subsystem implemented in the PL. The
video stored in SD cards or USB/SATA drives is decoded and displayed via SDI TX. This
module can Stream-in encoded data through an Ethernet interface and decode and display
it on SDI TX.
PL SDI Video Capture
This m odule ena b les captu r e of video f rom an SDI R X Subsyst em implemented in the PL into
a file or to Stream-out video. The video captured from the SDI RX Subsystem is encoded
and stored in SD cards or USB/SATA drives. The module can Stream-out encoded data
through an Ethernet interface.
Zynq UltraScale+ VCU TRD User Guide 17
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Chapter 2: Targeted Reference Design Details
Send Feedback
Full-fledged VCU TRD
Note: PL 10G Ethernet is supported only in 10G HDMI Video Capture and HDMI Display and HDMI
Video Capture and HDMI Display with SDSoC Support designs. All other designs support PS 1G
Ethernet.
This module enables video capture from an HDMI source, an image sensor connected
through CSI-2 RX, or a Test Pattern Generator (TPG) implemented in the PL. This module
also enables support for Scene Change Detection IP (SCD IP). SCD is supported in
memory-based mode. The video can be displayed via DP TX through the processing system
(PS) using HDMI TX through the PL, and can be recorded in SD cards or USB/SATA drives.
The module can Stream-in or Stream-out encoded data through an Ethernet interface.
PL DDR HDMI Video Capture and HDMI Display
This module enables capture of video from an HDMI RX Subsystem implemented in the PL.
The video can be displayed through HDMI TX through the PL and recorded in SD cards or
USB/SATA drives. The module can Stream-in or Stream-out encoded data through an
Ethernet interface. This module supports NV12, NV16, XV15, and XV20 pixel format.
This is the new design approach proposed to use PL_DDR for decoding and PS_DDR for
encoding so that DDR bandwidth would be enough to support high bandwidth VCU
applications requiring simultaneous encoder and decoder operations and transcoding at
4k@60fps. This approach makes most effective use of limited AXI4 read/write issuance
capability in minimizing latency for the decoder. DMA buffer sharing requirements
determine how capture, display, and intermediate processing stages should be mapped to
the PS or PL DDR.
VCU TRD PCIe
This module is used for transcoding MP4 files from the HOST machine to the client board
(zcu106) through the PCIe XDMA bridge interface in the PL. The file is passed to the VCU
decoder and encoder block for transcoding. The transcoded file is written back to the HOST
machine using the PCIe XDMA bridge interface read channel.
The Zynq UltraScale+ MPSoC VCU TRD wiki for 2019.1 provides additional content
including:
• Prerequisites for building and running the reference designs.
• Instructions for running the pre-built SD card image on the evaluation board.
Zynq UltraScale+ VCU TRD User Guide 18
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
• Detailed step-by-step design and tool flow tutorials for each design module.
The rdf0428-zcu106-vcu-trd-2019-1.zip targeted reference design ZIP file is
associated with this user guide and available from the Zynq UltraScale+ MPSoC ZCU106
Evaluation Kit Documentation website.
Chapter 2: Targeted Reference Design Details
Send Feedback
Design Components
Download the targeted reference design ZIP file. The file contains the following
components grouped by APU or PL.
APU
• vcu_apm_lib: Library that provides the interface to query read and write throughput of
the VCU encoder/decoder.
• vcu_gst_lib: Interface library that manages the video/audio-video capture, processing,
and display pipelines using the GStreamer, V4L2, Advanced Linux Sound Architecture
(ALSA) [Ref 6], and DRM frameworks.
• petalinux_bsp: PetaLinux board support package (BSP) to build a pre-configured SMP
Linux image for the APU. The BSP includes the following components:
First stage boot loader (FSBL)
°
Arm trusted firmware (ATF)
°
U-Boot
°
Linux kernel
°
Device tree
°
PMU firmware
°
Root file system (rootfs).
°
• vcu_qt: Application that uses the vcu_gst_lib, vcu_apm_lib, and vcu_video_lib libraries
and provides a GUI to control and visualize various parameters of this design. The GUI
is supported only on DP.
• vcu_video_lib: Library that configures various video pipelines in the design
• vcu_gst_app: Command line application that uses the vcu_gst_lib, vcu_apm_lib, and
vcu_video_lib libraries. It allows you to configure and run the capture, display, record,
stream in, and stream out pipelines through the command line.
• pcie_transcode: Command-line application that uses the pcie_lib library. It allows you
to transcode the MP4 file into ts.
• pcie_lib: This library provides abstract APIs for pcie_transcode applications that interact
with PCIe user space configuration.
Zynq UltraScale+ VCU TRD User Guide 19
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
• host_package: The host package installs the PCIe XDMA driver on the host machine. It
identifies the PCIe endpoint ZCU106 Board connected to the host machine. This
package has the application for sending files from the host machine along with the
encoder parameters for transcoding the file on the ZCU106 PCIe endpoint, and writes
back the transcoded file to the host machine.
Chapter 2: Targeted Reference Design Details
Send Feedback
PL
• Vivado: Vivado® IP integrator design that integrates the capture, processing
(encode/decode), and display pipeline.
Zynq UltraScale+ VCU TRD User Guide 20
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
APU Software Platform
Send Feedback
Introduction
This chapt er describes the application processing unit (APU) Linux software platform, which
is further subdivided into a middleware layer, an operating system (OS) layer, and an
application stack (see Figure 3-1). The two layers are examined in conjunction because they
interact closely for most Linux subsystems. These layers are further grouped by vertical
domains which reflect the organization of this chapter:
•Video
•Audio
•Display
Chapter 3
•Graphics
• Accelerator
•PCIe
Zynq UltraScale+ VCU TRD User Guide 21
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Software Architecture
Mali-400
vcu_gst_app vcu_qt
Graphics Display Video
Accelerators
(Codec)
vcu_gst_lib
vcu_apm_lib
vcu_video_lib
libQt5*
X.Org lib4lsubdev
omx_il
libMali
libdrm
libmediactl
CtlSW
Xilinx DRM
V4L
subdev
VCU
(al5c, al5e,
and al5r)
Xilinx
VIPP
GPU
VCU (Encoder
and Decoder)
DP
HDMI Tx
SDI Tx
SDI Rx TPG
HDMI Rx MIPI CSI
Application
(user)
Middleware
(user)
OS
(Kernel)
HW
libalsa
Alsa
Framework
Audio
Formatter
IP
Audio
gstreamer
pcie_ transcode
pcie_lib
Xilinx
PCIe
PCIe
PL DDR
SCD
10 G
X19929-041719
Send Feedback
Figure 3-1 shows the APU Linux software platform.
X-Ref Target - Figure 3-1
Chapter 3: APU Software Platform
Zynq UltraScale+ VCU TRD User Guide 22
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
The middleware layer is a horizontal layer implemented in the user-space. It provides the
following functionality:
• Interfaces with the application layer
Figure 3-1: APU Linux Software Platform
• Provides access to kernel frameworks
The OS layer i s a ho rizontal layer implemen ted in the kernel-sp ace. It provides the followin g
functionality:
• Provides a stable, well-defined API to user-space
• Includes device drivers and kernel frameworks (subsystems)
• Accesses the hardware
Chapter 3: APU Software Platform
Send Feedback
Video
To model and control video capture pipelines such as the ones used in this TRD on Linux
systems, multiple kernel frameworks and APIs must work in concert. For simplicity, the
overall solution is referred to as Video4Linux (V4L2), although the framework only provides
part of the required functionality. Individual components are discussed in the following
sections.
Driver Architecture
Figure 3-2 shows the VL42 driver stack (a generic V4L2 driver model of a video pipeline).
The video pipeline driver loads the necessary subdevice drivers and registers the device
nodes it needs, based on the video pipeline configuration specified in the device tree. The
framework exposes the following device node types to user space to control certain aspects
of the pipeline:
• Media device node: /dev/media*
• Video device node: /dev/video*
• V4L subdevice node: /dev/v4l-subdev*
Note:
These steps describe the data flow within software:
1. The V4L2 source driver allocates frame buffer for the capture device.
2. The V4L2 framework imports/exports the DMA_BUF file descriptor (FD) to the next
3. The encoder reads the source buffer from the capture device, encodes it, and writes the
4. The decoder allocates a decoded frame buffer, reads the bitstream buffer, a n d writes t he
5. The decoder shares the decoded frame buffer using the DMA_BUF framework with the
The * means [0 . . .n], e.g., /dev/media1, /dev/media2, and so on.
GStreamer element.
encoded bi tstrea m to a bi tstream buff er. The encode d bits tream d oes n ot us e DMA _BUF
framework for sharing the buffer.
decoded frame buffer into memory.
DRM display device.
Zynq UltraScale+ VCU TRD User Guide 23
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
X-Ref Target - Figure 3-2
vcu_qt/vcu_gst_app
vcu_gst_lib
libv4lsubdev
libv4I2 libmediactl
User Space
Kernel Space
/dev/v4I-subdev* /dev/video* /dev/media*
DMA Engine
Channel
DMA
V4L2 subdev
XVIPP Driver
TPG
Fmbuf Wr
VPSS
(Scaler Only)
Fmbuf Wr
TPG Capture Pipeline
HDMI Rx Capture Pipeline
CSK-2 Rx Capture Pipeline
SDI Rx Capture Pipeline
SDI Rx
Fmbuf Wr
IMX274
MIPI
CSI-2
RX
Demosaic Gamma
VPSS
CSC
VPSS
Scaler
Fmbuf Wr
VTC
HDMI Rx
X19930-120118
HW
VPSS
(Scaler Only)
vcu_apm_lib
vcu_video_lib
SCD
SCD Pipeline
Send Feedback
Chapter 3: APU Software Platform
Zynq UltraScale+ VCU TRD User Guide 24
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Figure 3-2: VL42 Driver Stack
Media Framework
The main go al of t he media fram ework i s to di scover the de vice topology of a video pipeline
and to configure it at run time. To achieve this, pipelines are modeled as an oriented graph
of building blocks called entities connected through pads.
Chapter 3: APU Software Platform
Send Feedback
A pad is a connection endpoint through which an entity can interact with other entities.
Data produced by an entity flows from the entity's output to one or more entity inputs. A
link is a point-to-point oriented connection between two pads, either on the same entity or
on different entities. Data flows from a source pad to a sink pad.
An entity is a basic media hardware building block. It can correspond to a large variety of
blocks such as physical hardware devices (e.g., image sensors), logical hardware devices
(e.g., soft IP cores inside the PL), DMA channels, or physical connectors. Physical or logical
devices are modeled as subdevice nodes and DMA channels as video nodes.
A media device node is created that allows the user space application to configure the
video pipeline and its subdevices through the libmediactl and libv4l2subdev libraries. The
media controller API provides this functionality:
• Enumerates entities, pads, and links
•Configures pads
Sets media bus format
°
Sets dimensions (width/height)
°
• Configures links
Enable/disable
°
Validates formats
°
Figure 3-3 shows the media graph for the SDI-RX, TPG, HDMI RX, and CSI RX video capture
pipelines as generated by the media-ctl utility. The TPG subdevice is shown in white with its
corresponding control interface address and subdevice node in the center. The numbers on
the edges are pads and the solid arrows represent active links. The grey boxes are video
nodes that correspond to Frame Buffer Write channels, in this case write channels (outputs).
Zynq UltraScale+ VCU TRD User Guide 25
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
X-Ref Target - Figure 3-3
vcap_tp0 output 0
/dev/video1
vcap_hdmi output 0
/dev/video0
vcap_csi output 0
/dev/video2
0
1
0
0
1
0
1
0
0
0
0
1
0
1
1
a00e0000.tpg
/dev/v4l-subdev0
a0000000.v_hdmi_rx_ss
/dev/v4l-subdev8
a0080000.scaler
/dev/v4l-subdev7
a0250000.v_demosaic
/dev/v4l-subdev3
IMX274
/dev/v4l-subdev1
a00f0000.csiss
/dev/v4l-subdev2
a0200000.scaler
/dev/v4l-subdev6
a0270000.v_gamma
/dev/v4l-subdev4
a0240000.csc
/dev/v4l-subdev5
vcap_sdirx output 0
(/dev/video0)
1
a0080000.scaler
/dev/v4l-subdev1
0
1
80000000.vcap_uhds
di_rx_ss
(/dev/v4l-subdev0)
0
video_cap input 0
(/dev/video9)
0
xlnx-scdchan 0
/dev/v4l-subdev20
X19447-050819
Send Feedback
Chapter 3: APU Software Platform
Figure 3-3: Video Capture Media Pipelines from Left: SDI, TPG, HDMI RX, and CSI RX
Graphics
Qt is a full development framework with tools designed to streamline the creation of
applications and user interfaces for desktop, embedded, and mobile platforms. Qt uses
standard C++ with extensions including signals and slots that simplify handling of events.
This helps in the development of both the GUI and server applications which receive their
own set of event information and should process them accordingly.
Display
Linux kernel and user-space frameworks for display and graphics are intertwined and the
software stack can be quite complex with many layers and different standards and APIs. On
the kernel side, the display and graphics portions are split with each having their own APIs.
However, both are commonly referred to as a single framework, namely DRM/KMS. This
split is advantageous, especially for SoCs that often have dedicated hardware blocks for
display and graphics. The display pipeline driver responsible for interfacing with the display
Zynq UltraScale+ VCU TRD User Guide 26
UG1250 (v2019.1) May 29, 2019 www.xilinx.com
Loading...