Altera Video and Image Processing Suite User Manual
Video and Image Processing Suite User
Guide
Subscribe
Send Feedback
UG-VIPSUITE
2015.05.04
101 Innovation Drive
San Jose, CA 95134
www.altera.com
TOC-2
Contents
Video and Image Processing Suite Overview..................................................... 1-1
Interfaces............................................................................................................. 2-1
Release Information.....................................................................................................................................1-3
Device Family Support................................................................................................................................1-4
Latency...........................................................................................................................................................1-5
In-System Performance and Resource Guidance....................................................................................1-8
Stall Behavior and Error Recovery.......................................................................................................... 1-12
Video Formats..............................................................................................................................................2-2
Avalon-ST Video Protocol..........................................................................................................................2-7
Video Data Packets........................................................................................................................2-11
Static Parameters of Video Data Packets....................................................................................2-11
Control Data Packets.....................................................................................................................2-15
Ancillary Data Packets.................................................................................................................. 2-19
User-Defined and Altera-Reserved Packets...............................................................................2-19
Packet Propagation........................................................................................................................2-19
Transmission of Avalon-ST Video Over Avalon-ST Interfaces..............................................2-20
Packet Transfer Examples.............................................................................................................2-21
Avalon-MM Slave Interfaces....................................................................................................................2-25
Specification of the Type of Avalon-MM Slave Interfaces.......................................................2-27
Avalon-MM Master Interfaces.................................................................................................................2-28
Specification of the Type of Avalon-MM Master Interfaces....................................................2-28
Buffering of Non-Image Data Packets in Memory............................................................................... 2-29
Getting Started.................................................................................................... 3-1
Clocked Video Interface IP Cores.......................................................................4-1
Altera Corporation
IP Catalog and Parameter Editor...............................................................................................................3-1
Specifying IP Core Parameters and Options................................................................................3-2
Installing and Licensing IP Cores..............................................................................................................3-2
OpenCore Plus IP Evaluation........................................................................................................ 3-3
Control Port..................................................................................................................................................4-1
Clocked Video Input Format Detection...................................................................................................4-2
Interrupts.......................................................................................................................................... 4-5
Clocked Video Output Video Modes........................................................................................................4-5
Interrupts.......................................................................................................................................... 4-9
Generator Lock.............................................................................................................................................4-9
Underflow and Overflow..........................................................................................................................4-11
Timing Constraints....................................................................................................................................4-12
Handling Ancillary Packets......................................................................................................................4-12
TOC-3
Modules for Clocked Video Input II IP Core........................................................................................4-15
Clocked Video Interface Parameter Settings......................................................................................... 4-18
Clocked Video Interface Signals..............................................................................................................4-26
Clocked Video Interface Control Registers............................................................................................4-36
2D FIR Filter IP Core.......................................................................................... 5-1
Calculation Precision...................................................................................................................................5-1
Coefficient Precision....................................................................................................................................5-2
Result to Output Data Type Conversion..................................................................................................5-2
2D FIR IP Core Parameter Settings...........................................................................................................5-2
2D FIR Filter Signals....................................................................................................................................5-4
2D FIR Filter Control Registers................................................................................................................. 5-5
Video Mixing IP Cores........................................................................................6-1
Alpha Blending.............................................................................................................................................6-2
Video Mixing Parameter Settings..............................................................................................................6-3
Video Mixing Signals...................................................................................................................................6-5
Video Mixing Control Registers................................................................................................................6-8
Chroma Resampler IP Core................................................................................ 7-1
Horizontal Resampling (4:2:2)...................................................................................................................7-1
4:4:4 to 4:2:2......................................................................................................................................7-2
4:2:2 to 4:4:4......................................................................................................................................7-2
Vertical Resampling (4:2:0)........................................................................................................................ 7-3
Chroma Resampler Parameter Settings....................................................................................................7-4
Chroma Resampler Signals.........................................................................................................................7-5
Video Clipping IP Cores..................................................................................... 8-1
Video Clipping Parameter Settings...........................................................................................................8-1
Video Clipping Signals................................................................................................................................8-4
Video Clipping Control Registers..............................................................................................................8-6
Color Plane Sequencer IP Core...........................................................................9-1
Combining Color Patterns..........................................................................................................................9-1
Rearranging Color Patterns........................................................................................................................9-2
Splitting and Duplicating............................................................................................................................9-3
Subsampled Data..........................................................................................................................................9-4
Color Plane Sequencer Parameter Settings.............................................................................................. 9-4
Color Plane Sequencer Signals...................................................................................................................9-5
Color Space Conversion IP Cores.....................................................................10-1
Input and Output Data Types..................................................................................................................10-2
Color Space Conversion............................................................................................................................10-2
Result of Output Data Type Conversion................................................................................................10-3
Altera Corporation
TOC-4
Color Space Conversion Parameter Settings..........................................................................................10-4
Color Space Conversion Signals.............................................................................................................. 10-8
Color Space Conversion Control Registers..........................................................................................10-10
Control Synchronizer IP Core.......................................................................... 11-1
Using the Control Synchronizer IP Core............................................................................................... 11-2
Control Synchronizer Parameter Settings..............................................................................................11-4
Control Synchronizer Signals...................................................................................................................11-5
Control Synchronizer Control Registers................................................................................................11-6
Deinterlacing IP Cores......................................................................................12-1
Deinterlacing Methods..............................................................................................................................12-2
Bob with Scanline Duplication.....................................................................................................12-3
Bob with Scanline Interpolation..................................................................................................12-3
Weave.............................................................................................................................................. 12-3
Motion-Adaptive........................................................................................................................... 12-3
Pass-Through Mode for Progressive Frames.............................................................................12-6
Frame Buffering......................................................................................................................................... 12-6
Frame Rate Conversion.............................................................................................................................12-8
Bandwidth Requirement Calculations for 10-bit YCbCr Video.........................................................12-8
Behavior When Unexpected Fields are Received..................................................................................12-9
Handling of Avalon-ST Video Control Packets..................................................................................12-10
Deinterlacing Parameter Settings..........................................................................................................12-10
Deinterlacing Signals...............................................................................................................................12-17
Deinterlacing Control Registers............................................................................................................ 12-23
Design Guidelines for Broadcast Deinterlacer IP Core......................................................................12-30
Tuning Motion Shift....................................................................................................................12-32
Active Video Threshold Adjustment........................................................................................12-32
Frame Reader IP Core.......................................................................................13-1
Frame Buffer IP Cores.......................................................................................14-1
Altera Corporation
Single-Cycle Color Patterns......................................................................................................................13-1
Frame Reader Output Pattern and Memory Organization..................................................................13-2
Frame Reader Parameter Settings............................................................................................................13-3
Frame Reader Signals................................................................................................................................ 13-3
Frame Reader Control Registers..............................................................................................................13-5
Double Buffering........................................................................................................................................14-2
Triple Buffering..........................................................................................................................................14-2
Locked Frame Rate Conversion...............................................................................................................14-3
Handling of Avalon-ST Video Control Packets....................................................................................14-3
Color Format..............................................................................................................................................14-4
Frame Buffer Parameter Settings.............................................................................................................14-5
Frame Buffer Signals..................................................................................................................................14-9
Frame Buffer Control Registers.............................................................................................................14-13
TOC-5
Gamma Corrector IP Core................................................................................15-1
Gamma Corrector Parameter Settings....................................................................................................15-1
Gamma Corrector Signals........................................................................................................................ 15-2
Gamma Corrector Control Registers......................................................................................................15-3
Interlacer IP Core..............................................................................................16-1
Interlacer Parameter Settings...................................................................................................................16-2
Interlacer Signals........................................................................................................................................16-2
Interlacer Control Registers..................................................................................................................... 16-4
Scaler II IP Core.................................................................................................17-1
Nearest Neighbor Algorithm................................................................................................................... 17-1
Bilinear Algorithm.....................................................................................................................................17-2
Bilinear Algorithmic Description................................................................................................17-2
Polyphase and Bicubic Algorithm...........................................................................................................17-3
Double-Buffering...........................................................................................................................17-5
Polyphase Algorithmic Description............................................................................................17-6
Choosing and Loading Coefficients............................................................................................17-6
Edge-Adaptive Scaling Algorithm...........................................................................................................17-8
Scaler II Parameter Settings......................................................................................................................17-9
Scaler II Signals........................................................................................................................................ 17-12
Scaler II Control Registers......................................................................................................................17-14
Video Switching IP Cores..................................................................................18-1
Mixer Layer Switching.............................................................................................................................. 18-2
Video Switching Parameter Settings.......................................................................................................18-3
Video Switching Signals............................................................................................................................18-3
Video Switching Control Registers......................................................................................................... 18-5
Test Pattern Generator IP Cores.......................................................................19-1
Test Pattern.................................................................................................................................................19-1
Generation of Avalon-ST Video Control Packets and Run-Time Control.......................................19-3
Test Pattern Generator Parameter Settings............................................................................................19-4
Test Pattern Generator Signals................................................................................................................ 19-6
Test Pattern Generator Control Registers..............................................................................................19-8
Trace System IP Core........................................................................................ 20-1
Trace System Parameter Settings.............................................................................................................20-2
Trace System Signals................................................................................................................................. 20-3
Operating the Trace System from System Console...............................................................................20-4
Loading the Project and Connecting to the Hardware.............................................................20-5
Trace Within System Console......................................................................................................20-6
TCL Shell Commands................................................................................................................... 20-7
Altera Corporation
TOC-6
Avalon-ST Video Monitor IP Core...................................................................21-1
Packet Visualization.................................................................................................................................. 21-2
Monitor Settings........................................................................................................................................ 21-3
Avalon-ST Video Monitor Parameter Settings......................................................................................21-3
Avalon-ST Video Monitor Signals.......................................................................................................... 21-4
Avalon-ST Video Monitor Control Registers........................................................................................21-6
Avalon-ST Video Verification IP Suite..............................................................A-1
Avalon-ST Video Class Library.................................................................................................................A-2
Running the Tests....................................................................................................................................... A-7
Video File Reader Test................................................................................................................. A-10
Constrained Random Test...........................................................................................................A-15
Complete Class Reference........................................................................................................................A-19
c_av_st_video_control ................................................................................................................ A-19
c_av_st_video_data ......................................................................................................................A-21
c_av_st_video_file_io ..................................................................................................................A-21
c_av_st_video_item ..................................................................................................................... A-26
c_av_st_video_source_sink_base .............................................................................................. A-27
c_av_st_video_sink_bfm_’SINK ................................................................................................A-28
c_av_st_video_source_bfm_’SOURCE .....................................................................................A-29
c_av_st_video_user_packet ........................................................................................................A-30
c_pixel ............................................................................................................................................A-31
Raw Video Data Format...........................................................................................................................A-31
Choosing the Correct Deinterlacer.................................................................... B-1
Cadence Detection and Reverse Pulldown in the Deinterlacer II IP Core..........................................B-3
Additional Information......................................................................................C-1
Document Revision History......................................................................................................................C-1
How to Contact Altera............................................................................................................................... C-5
Altera Corporation
2015.05.04
www.altera.com
101 Innovation Drive, San Jose, CA 95134
Video and Image Processing Suite Overview
1
UG-VIPSUITE
Subscribe
Send Feedback
The Altera® Video and Image Processing Suite collection of IP cores ease the development of video and
image processing designs.
You can use these IP cores in a wide variety of image processing and display applications.
Attention: Altera has scheduled the following IP cores for product obsolescence and will discontinue
support for it.
• Clipper
• Test Pattern Generator
Altera recommends that you do not use these IP cores in new designs. For more information
about Altera’s current IP offering, visit the Altera Intellectual Property web page.
Table 1-1: Video and Image Processing Suite IP Core Features
The table lists the IP cores in the Video and Image Processing Suite.
IP Core
Pixels in Parallel 4:2:2 Support Interlaced
Feature Support
2D FIR Filter No No No
Alpha Blending
No Yes Yes
Mixer
(1)
Chroma Resampler No Yes No
Clipper No Yes Yes
Clipper II Yes Yes Yes
Clocked Video
No Yes Yes
(2)
(2)
Input (CVI)
Clocked Video
Yes Yes Yes
Input II (CVI II)
(1)
The IP core accepts interlaced input streams but they are treated as progressive inputs. Consequently, you
require external logic to synchronize the input fields and prevent the mixing of F0 fields with F1 fields.
(2)
The IP core accepts interlaced inputs but they are treated as progressive inputs.
©
2015 Altera Corporation. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are
trademarks of Altera Corporation and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as
trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance
of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any
products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information,
product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device
specifications before relying on any published information and before placing orders for products or services.
ISO
9001:2008
Registered
1-2
Video and Image Processing Suite Overview
UG-VIPSUITE
2015.05.04
IP Core
Clocked Video
Pixels in Parallel 4:2:2 Support Interlaced
No Yes Yes
Feature Support
Output (CVO)
Clocked Video
Yes Yes Yes
Output II (CVO II)
Color Plane
No Yes Yes
Sequencer
Color Space
No No Yes
Converter (CSC)
Color Space
Yes No Yes
Converter II (CSC
II)
Control Synchron‐
No Yes Yes
izer
Deinterlacer No Yes Yes
Deinterlacer II No Yes Yes
Broadcast Deinter‐
No Yes Yes
lacer
Frame Buffer No Yes Yes
Frame Buffer II Yes Yes Yes
Frame Reader No Yes Yes
Gamma Corrector No Yes Yes
Interlacer No Yes Yes
Mixer II Yes Yes Yes
Scaler II No Yes Yes
(3)
(1)
(2)
Switch No Yes Yes
Switch II Yes Yes Yes
Test Pattern
No Yes Yes
(4)
Generator
Test Pattern
Yes Yes Yes
(4)
Generator II
(3)
The IP core either discards or propagates without change the interlaced data if you select Pass-through
mode in the parameter editor.
(4)
For interlaced data NTSC, mismatched line counts of F0 and F1 are not supported.
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Release Information
The following table lists information about this release of the Video and Image Processing Suite.
Table 1-2: Release Information
Item Description
Version 15.0
Release Date May 2015
Release Information
1-3
Ordering
Code
Product ID
IPS-VIDEO (Video and Image Processing Suite)
00B3 (2D FIR Filter)
00B5 (Alpha Blending
Mixer)
0115 (Mixer II)
00D1 (Avalon-ST Video
Monitor)
00B1 (Chroma Resampler)
00C8 (Clipper)
0115 (Clipper II)
00C4 (Clocked Video
Input)
00F1 (Clocked Video
Input II)
00C5 (Clocked Video
Output)
00EF (Clocked Video
Output II)
00C9 (Color Plane
Sequencer)
0003 (Color Space
Converter)
00F2 (Color Space
Converter II)
00D0 (Control Synchron‐
izer)
00B6 (Deinterlacer)
00EE (Deinterlacer II)
0114 (Broadcast Deinter‐
lacer)
0112 (Frame Reader)
00C3 (Frame Buffer)
0115 (Frame Buffer II)
00B2 (Gamma Corrector)
00DC (Interlacer)
00E9 (Scaler II)
00CF (Switch)
00F4 (Switch II)
00CA (Test Pattern Generator)
00F3 (Test Pattern Generator II)
00ED (Trace System)
Vendor ID 6AF7
Altera verifies that the current version of the Quartus II software compiles the previous version of each IP
core, if this IP core was included in the previous release. Any exceptions to this verification are reported in
the MegaCore IP Library Release Notes and Errata. Altera does not verify compilation with IP core
versions older than the previous release.
Related Information
• Altera IP Library Release Notes
• Errata for VIP Suite in the Knowledge Base
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
1-4
Device Family Support
Device Family Support
The table below lists the device support information for the Video and Image Processing Suite IP cores.
Table 1-3: Device Family Support
Device Family Support
Arria II GX / Arria II GZ Final
Arria V Final
Arria 10 Final—Supports only the following IP cores:
• Avalon-ST Video Monitor
• Broadcast Deinterlacer
• Clipper II
• Clocked Video Input
• Clocked Video Input II
• Clocked Video Output
• Clocked Video Output II
• Color Space Converter II
• Deinterlacer II
• Frame Buffer II
• Mixer II
• Scaler II
• Switch II
• Test Pattern Generator II
UG-VIPSUITE
2015.05.04
Cyclone IV ES / Cyclone IV GX Final
Cyclone V Final
MAX 10 Final
Stratix IV Final
Stratix V Final
Other device families No support
Related Information
What's New for IP in Quartus II Software
Provides more information about the support levels and current status.
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Latency
Latency
You can use the latency information to predict the approximate latency between the input and the output
of your video processing pipeline.
The latency is described using one or more of the following measures:
• the number of progressive frames
• the number of interlaced fields
• the number of lines when less than a field of latency
• a small number of cycles O (cycles)
Note: O refers to a small number of clock cycles, and is not of zero value.
The latency is measured with the assumption that the IP core is not being stalled by other functions on the
data path; (the output ready signal is high).
Table 1-4: Video and Image Processing Suite Latency
The table below lists the approximate latency from the video data input to the video data output for typical usage
modes of the Video and Image Processing Suite IP cores.
IP Core Mode Latency
1-5
2D FIR Filter Latency Filter size: N × N (N–1) lines + O (cycles)
Alpha Blending Mixer/ Mixer II All modes O (cycles)
Input format: 4:2:2; Output
O (cycles)
format: 4:4:4
Chroma Resampler
Input format: 4:2:0; Output
1 line + O (cycles)
format: 4:4:4 or 4:2:2
Clipper/ Clipper II All modes O (cycles)
Clocked Video Input
Note: Add 1 cycle if you turned
on the Allow color planes
in sequence input
• Synchronization signals:
Embedded in video
• Video in and out use the
same clock: On
8 cycles
parameter.
• Synchronization signals: On
5 cycles
separate wires
• Video in and out use the
same clock: On
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
1-6
Latency
UG-VIPSUITE
2015.05.04
IP Core Mode Latency
• Synchronization signals:
10 cycles
Embedded in video
• Video in and out use the
same clock: On
Clocked Video Input II
• Synchronization signals: On
6 cycles
separate wires
• Video in and out use the
same clock: On
Clocked Video Output/
Clocked Video Output II
Note: Add 1 cycle if you turned
on the Allow color planes
in sequence input
All modes with video in and out
use the same clock: On
3 cycles
Note: Note: Minimum latency
case when video input
and output rates are
synchronized.
parameter.
Color Plane Sequencer All modes O (cycles)
Color Space Converter (CSC)/
All modes O (cycles)
Color Space Converter II
Control Synchronizer All modes O (cycles)
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Latency
IP Core Mode Latency
1-7
Deinterlacer
• Method: Bob
• Frame buffering: None
• Method: Motion-adaptive or
Weave
• Frame buffering: Double or
triple buffering with rate
conversion
• Output frame rate: As input
frame rate
• Method: Motion-adaptive or
Weave
• Frame buffering: Double or
triple buffering with rate
conversion
• Output frame rate: As input
field rate
• Method: All
• Frame buffering: Double or
triple buffering with rate
conversion
• Passthrough mode
(propagate progressive
frames unchanged): On.
O (cycles)
1 frame + O (lines)
1 field + O (lines)
1 frame + O (lines)
Deinterlacer II
Video and Image Processing Suite Overview
• Method: Motion-adaptive
• Frame buffering: None
• Output frame rate: As input
field rate
2 lines
Altera Corporation
Send Feedback
1-8
In-System Performance and Resource Guidance
IP Core Mode Latency
UG-VIPSUITE
2015.05.04
• Method: Motion-adaptive
1 field + 2 lines
• Frame buffering: None
• Output frame rate: As input
field rate
• Method: Motion-adaptive,
1 field + 2 lines, or 2 lines
video-over-film mode
Broadcast Deinterlacer
• Frame buffering: 3 input
fields are buffered
• Output frame rate: As input
field rate
40% to 60% (depending on
phasing) of the time, the core
performs a weave forward so
there is no initial field of latency.
Frame Buffer/ Frame Buffer II All modes 1 frame + O (lines)
Frame Reader No latency issues.
Gamma Corrector All modes O (cycles)
Interlacer All modes O (cycles)
Scaler II
• Scaling algorithm: Polyphase
• Number of vertical taps: N
Switch/ Switch II All modes 2 cycles
Test Pattern Generator/
No latency issues.
Test Pattern Generator II
In-System Performance and Resource Guidance
The performance and resource data provided for your guidance.
Run your own synthesis and f
Note:
ments.
trials to confirm the listed IP cores meet your system require‐
MAX
(N–1) lines + O (cycles)
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
In-System Performance and Resource Guidance
Table 1-5: Performance and Resource Data Using Arria V Devices
The following data are obtained through a 4K test design example using an Arria V device (5AGXFB3H4F35C4).
1-9
The general settings for the design is 8 bits per color plane; 2 pixels in parallel. The target f
IP Core Configuration ALM RAM DSP
Mixer II
• Number of color planes in parallel = 3
1,591 0 0
• Inputs = 4
• Output = 1
• Internal Test Pattern Generator
Clocked Video
Input II
• Number of color planes in parallel = 3
• Sync signals = On separate wires
540 26 0
• Pixel FIFO size = 4096 pixels
• Use control port = On
Clocked Video
Output II
• Number of color planes in parallel = 3
• Sync signals = On separate wires
2,504 49 0
• Pixel FIFO size = 4096 pixels
• Use control port = On
• Run-time configurable video modes = 4
Color Space
Converter II
• Run-time control = On
• Color model conversion = RGb to YCbCr
1,515 0 18
is 148.5 MHz.
MAX
Broadcast Deinter‐
lacer
Frame Buffer II
Test Pattern
Generator II
• Number of color planes in parallel = 2
• Avalon-MM master local ports width =
256
• FIFO depths = 512
• Run-time control = Off
• Number of color planes in parallel = 2
• Avalon-MM master ports width = 256
• Read/write FIFO depth = 128
• Frame dropping = On
• Frame repeating = On
• Color space = RGB
• Run-time control of image size = On
11,516 145 34
1,472 19 0
135 0 0
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
1-10
In-System Performance and Resource Guidance
UG-VIPSUITE
2015.05.04
Table 1-6: Performance and Resource Data Using Cyclone V Devices
The following data are obtained through a video design example using a Cyclone V device (5CGTFD9E5F35C7).
The general setting for the design is 8 bits per color plane. The target f
IP Core Configuration ALM RAM DSP
2D FIR Filter
• Number of color planes in sequence = 3
• Filter size = 3×3
• Runtime control = On
• Integer bits = 4
• Fractional bits = 3
Avalon-ST Video
Monitor
Avalon-ST Video
Monitor
Alpha Blending
Mixer
• Number of color planes in parallel = 3
• Capture video pixel data = On
• Number of color planes in parallel = 3
• Capture video pixel data = Off
• Number of color planes in parallel = 3
• Number of layers being mixed = 5
• Alpha blending = On
Chroma
Resampler
• 4:2:2, number of color planes in parallel
(din) = 2
• 4:4:4, number of color planes in parallel
(dout) = 3
• Horizontal filtering algorithm = Filtered
• Luma adaptive = On
is 100 MHz.
MAX
581 10 3
1,035 10 0
479 9 0
1,324 1 24
591 0 0
Clipper II
Clocked Video
Input
Clocked Video
Input
Altera Corporation
• Number of pixels (color planes) in parallel
= 3
• Clipping method = Rectangle
• Enable runtime control of clipping
parameters = On
• Number of color planes in parallel = 3
• Sync signals = On separate wires
• Pixel FIFO size = 2048 pixels
• Use control port = On
• Number of color planes in sequence = 2
• Sync signals = Embedded
• Pixel FIFO size = 2048 pixels
• Use control port = On
402 0 0
257 13 0
317 9 0
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
In-System Performance and Resource Guidance
IP Core Configuration ALM RAM DSP
1-11
Clocked Video
Output
Color Plane
Sequencer
Color Space
Converter
Deinterlacer II
• Number of color planes in parallel = 3
• Sync signals = On separate wires
• Pixel FIFO size = 1024 pixels
• Use control port = On
• Run-time configurable video modes = 1
• din0: Color planes in parallel = 4
• dout0: Color planes in parallel = 3
• dout1: Color planes in parallel = 1
• Color plane configuration = Three color
planes in parallel
• Run-time control = Off
• Color model conversion = CbCrY': SDTV
to Computer B'G'R'
• Coefficients integer bits = 2
• Summands integer bits = 9
• Coefficient and summand fractional bits =
8
• Number of color planes in parallel = 3
• Deinterlace algorithm = Motion adaptive
• Cadence detection algorithm = 3:2 detector
• Avalon-MM master local ports width =
128
• FIFO depths = 64
512 5 0
104 0 0
284 0 9
3,655 67 3
Frame Buffer
• Number of color planes in parallel = 3
• Avalon-MM master ports width = 128
• Read/write FIFO depth = 64
• Frame dropping = On
• Frame repetition = On
Frame Reader
• Number of color planes in parallel = 4
• Avalon-MM master port width = 128
• Read master FIFO depth = 64
• Use separate clocks for the Avalon-MM
master interfaces = On
Gamma Corrector Number of color planes in parallel = 3 142 3 0
Video and Image Processing Suite Overview
Send Feedback
1,084 19 0
756 6 0
Altera Corporation
1-12
Stall Behavior and Error Recovery
IP Core Configuration ALM RAM DSP
UG-VIPSUITE
2015.05.04
Scaler II
• Symbols in parallel = 3
• Scaling algorithm = Polyphase
• Enable run-time control of input/output
frame size = On
• Vertical/horizontal filter taps = 4
• Vertical/horizontal filter phases = 14
Test Pattern
Generator
Trace System
• Color space = RGB
• Run-time control of image size = Off
• Buffer size = 8192
• Bit width of capture interface(s) = 32
• Number of inputs = 2
Stall Behavior and Error Recovery
The Video and Image Processing Suite IP cores do not continuously process data. Instead, they use flowcontrolled Avalon-ST interfaces, which allow them to stall the data while they perform internal
calculations.
During control packet processing, the IP cores might stall frequently and read or write less than once per
clock cycle. During data processing, the IP cores generally process one input or output per clock cycle.
There are, however, some stalling cycles. Typically, these are for internal calculations between rows of
image data and between frames/fields.
1,010 12 12
65 0 0
1,224 12 0
When stalled, an IP core indicates that it is not ready to receive or produce data. The time spent in the
stalled state varies between IP cores and their parameterizations. In general, it is a few cycles between rows
and a few more between frames.
If data is not available at the input when required, all of the IP cores stall and do not output data. With the
exceptions of the Deinterlacer and Frame Buffer in double or triple-buffering mode, none of the IP cores
overlap the processing of consecutive frames. The first sample of frame F + 1 is not input until after the IP
cores produce the last sample of frame F.
When the IP cores receive an endofpacket signal unexpectedly (early or late), the IP cores recover from
the error and prepare for the next valid packet (control or data).
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
1-13
2D FIR Filter
• Has a delay of a little more than N–1
lines between data input and output in
the case of a N×N 2D FIR Filter.
• Delay caused by line buffering internal
to the IP core.
• Resolution is not configurable at run
time.
• Does not read the control packets
passed through it.
An error condition occurs if an
endofpacket signal is received too early
or too late for the compile time
configured frame size. In either case, the
2D FIR Filter always creates output
video packets of the configured size.
• If an input video packet has a late
endofpacket signal, then the extra
data is discarded.
• If an input video packet has an early
endofpacket signal, then the video
frame is padded with an undefined
combination of the last input pixels.
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
1-14
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
UG-VIPSUITE
2015.05.04
Alpha Blending
Mixer/
Mixer II
All modes stall for a few cycles after each
output frame and between output lines.
Between frames, the IP core processes nonimage data packets from its input layers in
sequential order. The core may exert
backpressure during the process until the
image data header has been received for all
its input.
During the mixing of a frame, the IP core:
• Reads from the background input for
each non-stalled cycle.
• Reads from the input ports associated
with layers that currently cover the
background image.
Because of pipelining, the foreground pixel
of layer N is read approximately N active
cycles after the corresponding background
pixel has been read.
• If the output is applying backpressure or
if one input is stalling, the pipeline stalls
and the backpressure propagates to all
active inputs.
• When alpha blending is enabled, one
data sample is read from each alpha port
once each time that a whole pixel of data
is read from the corresponding input
port.
There is no internal buffering in the IP core,
so the delay from input to output is just a
few clock cycles and increases linearly with
the number of inputs.
The Alpha Blending Mixer IP core
processes video packets from the
background layer until the end of packet
is received.
• Receiving an endofpacket signal too
early for the background layer—the
IP core enters error mode and
continues writing data until it has
reached the end of the current line.
The endofpacket signal is then set
with the last pixel sent.
• Receiving an endofpacket signal
early for one of the foreground layers
or for one of the alpha layers—the IP
core stops pulling data out of the
corresponding input and pads the
incomplete frame with undefined
samples.
• Receiving an endofpacket signal late
for the background layer, one or
more foreground layers, or one or
more alpha layers—the IP core enters
error mode.
This error recovery process maintains
the synchronization between all the
inputs and is started once the output
frame is completed. A large number of
samples may have to be discarded
during the operation and backpressure
can be applied for a long time on most
input layers. Consequently, this error
recovery mechanism could trigger an
overflow at the input of the system.
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
1-15
Chroma
Resampler
All modes stall for a few cycles between
frames and between lines.
Latency from input to output varies
depending on the operation mode of the IP
core.
• The only modes with latency of more
than a few cycles are 4:2:0 to 4:2:2 and
4:2:0 to 4:4:4—corresponding to one line
of 4:2:0 data
• The quantities of data input and output
are not equal because this is a ratechanging function.
• Always produces the same number of
lines that it accepts—but the number of
samples in each line varies according to
the subsampling pattern used.
When not stalled, always processes one
sample from the more fully sampled side on
each clock cycle. For example, the
subsampled side pauses for one third of the
clock cycles in the 4:2:2 case or half of the
clock cycles in the 4:2:0 case.
• Receiving an early endofpacket
signal—the IP core stalls its input but
continues writing data until it has
sent an entire frame.
• Not receiving an endofpacket signal
at the end of a frame—the IP core
discards data until it finds end-ofpacket.
Clipper/
Clipper II
Clocked Video
Input/
Clocked Video
Input II
• Stalls for a few cycles between lines and
between frames.
• Internal latency is less than 10 cycles.
• During the processing of a line, it reads
continuously but only writes when
inside the active picture area as defined
by the clipping window.
• Dictated by incoming video.
• If its output FIFO is empty, during
horizontal and vertical blanking periods
the IP core does not produce any video
data.
• Receiving an early endofpacket
signal—the IP core stalls its input but
continues writing data until it has
sent an entire frame.
• Not receiving an endofpacket signal
at the end of a frame—the IP core
discards data until it finds end of
packet.
If an overflow is caused by a
downstream core failing to receive data
at the rate of the incoming video, the
Clocked Video Input sends an
endofpacket signal and restart sending
video data at the start of the next frame
or field.
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
1-16
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
UG-VIPSUITE
2015.05.04
Clocked Video
Output/
Clocked Video
Output II
Color Plane
Sequencer
• Dictated by outgoing video.
• If its input FIFO is empty, during
horizontal and vertical blanking periods
the IP core stalls and does not take in
any more video data.
• Stalls for approximately 10 cycles after
processing each line of a video frame.
• Between frames the IP core stalls for
approximately 30 cycles
• Receiving an early endofpacket
signal— the IP core resynchronizes
the outgoing video data to the
incoming video data on the next start
of packet it receives.
• Receiving a late endofpacket— the
IP core resynchronizes the outgoing
video data to the incoming video
immediately.
• If Genlock functionality is enabled—
the IP core does not resynchronize to
the incoming video.
• Processes video packets per line until
the IP core receives an endofpacket
signal on din0—the line width is
taken from the control packets on
din0.
• Receiving an endofpacket signal on
either din0 or din1— the IP core
ceases to produce output.
Color Space
Converter/
Color Space
Converter II
• Only stalls between frames and not
between rows.
• It has no internal buffering apart from
the registers of its processing pipeline—
only a few clock cycles of latency.
For the number of cycles left to finish
the line, the IP core continues to drain
the inputs that have not indicated end of
packet.
• Drains din0 until it receives an
endofpacket signal on this port
(unless it has already indicated end of
packet), and stalls for up to one line
after this endofpacket signal.
• Signals end of packet on its outputs
and continue to drain its inputs that
have not indicated end of packet.
• Processes video packets until the IP
core receives an endofpacket signal
—the control packets are not used.
• Any mismatch of the endofpacket
signal and the frame size is
propagated unchanged to the next IP
core.
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
1-17
Control
Synchronizer
Deinterlacer
• Stalls for several cycles between packets.
• Stalls when it enters a triggered state
while it writes to the Avalon-MM Slave
ports of other IP cores.
• If the slaves do not provide a wait
request signal, the stall lasts for no more
than 50 clock cycles. Otherwise the stall
is of unknown length.
• Bob algorithm
• While the bob algorithm (with no
buffering) is producing an output
frame, it alternates between simulta‐
neously between
• receiving a row on the input port
and producing a row of data on
the output port
• just producing a row of data on
the output port without reading
any data from the input port
The delay from input to output is just
a few clock cycles.
• While a field is being discarded, input
is read at the maximum rate and no
output is generated.
• Weave algorithm
• The IP core may stall for longer than
the usual periods between each
output row of the image.
• The delays may possibly stretch up to
45 clock cycles due to the time taken
for internal processing in between
lines.
• Motion-adaptive algorithm
• Processes video packets until the IP
core receives an endofpacket signal
—the image width, height and
interlaced fields of the control data
packets are not compared against the
following video data packet.
• Any mismatch of the endofpacket
signal and the frame size of video
data packet is propagated unchanged
to the next IP core.
• Receiving an endofpacket signal too
early or too late is relative to the field
dimensions contained in the last
control packet processed.
• Receiving an endofpacket signal too
late—discards extra data in all
configurations.
• Receiving an early endofpacket
signal when it is configured for no
buffering—the IP core interrupts its
processing within one or two lines
sending undefined pixels, before
propagating the endofpacket signal.
• Receiving an early endofpacket
signal when it is configured to buffer
data in external memory—the input
side of the IP core stops processing
input pixels. It is then ready to
process the next frame after writing
undefined pixels for the remainder of
the current line into external RAM.
The output side of the IP core
assumes that incomplete fields have
been fully received and pads the
incomplete fields to build a frame,
using the undefined content of the
memory.
• The IP core may stall up to 90 clock
cycles.
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
1-18
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
UG-VIPSUITE
2015.05.04
Deinterlacer II/
Broadcast
Deinterlacer
Stores input video fields in the external
memory and concurrently uses these input
video fields to construct deinterlaced
frames.
• Stalls up to 50 clock cycles for the first
output frame.
• Additional delay of one line for second
output frame because the IP core
generates the last line of the output
frame before accepting the first line of
the next input field.
• Delay of two lines for the following
output frames, which includes the one
line delay from the second output frame.
• For all subsequent fields, the delay
alternates between one and two lines.
• Receiving an endofpacket signal too
early :
• The IP core generates a line with
the correct length.
• The video data in the output
frame is valid up to the point
where the IP core receives the
endofpacket signal.
• The IP core then stops generating
output until it receives the next
startofpacket signal.
• Receiving a late endofpacket signal:
• The IP core completes generating
the current output frame with the
correct number of lines as
indicated by the last control
packet.
• The IP core discards the
subsequent input lines.
• Once it receives a startofpacket
signal, the IP core performs a soft
reset and it loses the stored
cadence or motion values.
• The IP core resumes deinterlacing
when it receives the next
startofpacket signal.
Frame Reader
Altera Corporation
• Stalls the output for several tens of cycles
before producing each video data packet.
• Stalls the output where there is
contention for access to external
memory.
The IP core can be stalled due to
backpressure, without consequences and
it does not require error recovery.
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
1-19
Frame Buffer/
Frame Buffer II
Gamma
Corrector
• May stall frequently and read or write
less than once per clock cycle during
control packet processing.
• During data processing at the input or at
the output, the stall behavior of the IP
core is largely decided by contention on
the memory bus.
• Stalls only between frames and not
between rows.
• Has no internal buffering aside from the
registers of its processing pipeline— only
a few clock cycles of latency
• Does not rely on the content of the
control packets to determine the size
of the image data packets.
• Any early or late endofpacket signal
and any mismatch between the size
of the image data packet and the
content of the control packet are
propagated unchanged to the next IP
core.
• Does not write outside the memory
allocated for each non-image and
image Avalon-ST video packet—
packets are truncated if they are
larger than the maximum size
defined at compile time.
• Processes video packets until the IP
core receives an endofpacket signal
—non-image packets are propagated
but the content of control packets is
ignored.
• Any mismatch of the endofpacket
signal and the frame size is
propagated unchanged to the next IP
core.
Interlacer
• Alternates between propagating and
discarding a row from the input port
while producing an interlaced output
field—the output port is inactive every
other row.
• The delay from input to output is a few
clock cycles when pixels are propagated.
Video and Image Processing Suite Overview
• Receiving endofpacket signal later
than expected—discards extra data.
• Receiving an early endofpacket
signal—the current output field is
interrupted as soon as possible and
may be padded with a single
undefined pixel.
Altera Corporation
Send Feedback
1-20
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
UG-VIPSUITE
2015.05.04
Scaler II
• The ratio of reads to writes is
proportional to the scaling ratio and
occurs on both a per-pixel and a per-line
basis.
• The frequency of lines where reads and
writes occur is proportional to the
vertical scaling ratio.
• For example scaling up vertically by a
factor of 2 results in the input being
stalled every other line for the length of
time it takes to write one line of output;
scaling down vertically by a factor of 2
results in the output being stalled every
other line for the length of time it takes
to read one line of input.
• In a line that has both input and output
active, the ratio of reads and writes is
proportional to the horizontal scaling
ratio. For example, scaling from 64×64
to 128×128 causes 128 lines of output,
where only 64 of these lines have any
reads in them. For each of these 64 lines,
there are two writes to every read.
• Receiving an early
endofpacket
signal at the end of an input line—the
IP core stalls its input but continues
writing data until it has sent on
further output line.
• Receiving an early endofpacket
signal part way through an input line
—the IP core stalls its input for as
long as it would take for the open
input line to complete; completing
any output line that may accompany
that input line. Then continues to
stall the input, and writes one further
output line.
• Not receiving an endofpacket signal
at the end of a frame—the IP core
discards extra data until it finds an
end of packet.
The internal latency of the IP core depends
on the scaling algorithm and whether any
run time control is enabled. The scaling
algorithm impacts stalling as follows:
• Bilinear mode: a complete line of input
is read into a buffer before any output is
produced. At the end of a frame there
are no reads as this buffer is drained.
The exact number of possible writes
during this time depends on the scaling
ratio.
• Polyphase mode with Nv vertical taps: N
– 1 lines of input are read into line
buffers before any output is ready. The
scaling ratio depends on the time at the
end of a frame where no reads are
required as the buffers are drained.
v
Altera Corporation
Video and Image Processing Suite Overview
Send Feedback
UG-VIPSUITE
2015.05.04
Stall Behavior and Error Recovery
IP Core Stall Behavior Error Recovery
Enabling run-time control of resolutions
affects stalling between frames:
• With no run-time control: about 10
cycles of delay before the stall behavior
begins, and about 20 cycles of further
stalling between each output line.
• With run-time control of resolutions:
about additional 25 cycles of delay
between frames.
1-21
Switch/
Switch II
Test Pattern
Generator/
Test Pattern
Generator II
• Only stalls its inputs when performing
an output switch.
• Before switching its outputs, the IP core
synchronizes all its inputs and the inputs
may be stalled during this synchroniza‐
tion.
• All modes stall for a few cycles after a
field control packet, and between lines.
• When producing a line of image data,
the IP core produces one sample output
on every clock cycle, but it can be stalled
without consequences if other functions
down the data path are not ready and
exert backpressure.
—
—
Video and Image Processing Suite Overview
Send Feedback
Altera Corporation
2015.05.04
Clocked Video Input
IP Core
Deinterlacer
IP Core
DDR 2 SDRAM
Controller with UniPHY
IP Core
Nios II
Processor
Avalon ST Connection
Avalon MM Master to Slave Connection
Scaler II
IP Core
Clocked Video Output
IP Core
www.altera.com
101 Innovation Drive, San Jose, CA 95134
Interfaces
2
UG-VIPSUITE
Subscribe
Send Feedback
The IP cores in the Video and Image Processing Suite use standard interfaces for data input and output,
control input, and access to external memory. These standard interfaces ensure that video systems can be
quickly and easily assembled by connecting IP cores together.
The IP cores use the following types of interface:
• Avalon-ST interface—a streaming interface that supports backpressure. The Avalon-ST Video
protocol transmits video and configuration data. This interface type allows the simple creation of video
processing data paths, where IP cores can be connected together to perform a series of video
processing functions.
• Avalon-MM slave interface—provides a means to monitor and control the properties of the IP cores.
• Avalon-MM master interface—when the IP cores require access to a slave interface, for example an
external memory controller.
Figure 2-1: Abstracted Block Diagram Showing Avalon-ST and Avalon-MM Connections
The figure below shows an example of video processing data paths using the Avalon-ST and Avalon-MM
interfaces.
Note:
©
2015 Altera Corporation. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are
trademarks of Altera Corporation and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as
trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance
of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any
products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information,
product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device
specifications before relying on any published information and before placing orders for products or services.
This abstracted view is similar to that provided in the Qsys tool, where interface wires are grouped
together as single connections.
ISO
9001:2008
Registered
2-2
Video Formats
The Clocked Video Input and Clocked Video Output IP cores also have external interfaces that support
clocked video standards. These IP cores can connect between the function’s Avalon-ST interfaces and
functions using clocked video standards such as BT.656.
Related Information
Avalon Interface Specifications
Provides more information about these interface types.
Video Formats
The Clocked Video Output IP cores create clocked video formats, and Clocked Video Input IP cores
accept clocked video formats.
The IP cores create and accept the following formats:
• Video with synchronization information embedded in the data (in BT656 or BT1120 format)
• Video with separate synchronization (H sync, V sync) signals
The CVO IP cores create a video frame consisting of horizontal and vertical blanking (containing syncs)
and areas of active picture (taken from the Avalon-ST Video input).
• Video with synchronization information embedded in the data (in BT656 or BT1120 format)
• Video with separate synchronization (H sync, V sync) signals
UG-VIPSUITE
2015.05.04
Altera Corporation
Interfaces
Send Feedback
Vertical Blanking
F0 Active Picture
Horizontal Blanking
Horizontal Sync
Vertical Sync
Width
Height
UG-VIPSUITE
2015.05.04
Figure 2-2: Progressive Frame Format
Video Formats
2-3
Interfaces
Altera Corporation
Send Feedback
Loading...