Fast,Accurate,Repeatable,and ObjectivePicture Quality Measurement
Predicts DMOS(Differential Mean Opinion Score) based on Human
VisionSystem Model
Picture QualityMeasurements can bemade on aVarietyof HD Video
Formats ( 1080i, 720p) andSD VideoFormats (525ior 625i)
Makes PictureQuality Comparison acrossDifferent Resolutions fro m
HD toSD, or SD/HDto CIF
User-configurable ViewingCondition andDisplay Modelsfor Reference
and Comparison
Attention/Artifact WeightedMeasurement
RegionOf Interest (ROI) on MeasurementExecution andReview
Automatic Temporal andSpatial Alignment
Easy Regression Testing and Automation using XML Scriptingwith
2-channel Capture and 2-channel Gene ration with Swap-channel
Capability
Preinstalled Sample Reference andTestSequences
CODEC Design
Conformance Testing,Transmission Equipme nt, and System Evaluation
Digital VideoMastering
Video CompressionServices
DigitalConsumer Product Developmentand Manufacturing
, Optimization,and Verification
Picture Quality Analysis System
The PQA600is the latest-generation Picture QualityAnalyzer built onthe
EmmyAward winningTektronixPQA200/300. Basedonthe conceptsof the
human vision system, thePQA600 providesa suiteof repeatable,objective
quality mea
visual assessment. These measurements provide valuable info rmation
to engineersworking to optimize videocompression and recovery,and
maintaining a level ofcommon carrierand distributiontransmission service
to clientsand viewers.
Compressed Video Requires New Test Methods
The truemeasure of anytelevision system isviewer satisfaction. While
the qualityof analog andfull-bandwidth digital videocan becharacterized
indirectly bymeasuring the distortionsof static testsignals, compressed
television systemspose afar more difficultchallenge. Picturequality in a
compressed systemcan change dynamicallybased on acombination of
data rate
staticnature of te st signals doesnot providetrue characterizationof picture
quality.
Human viewertesting has been traditionallyconducted as described in
ITU-R Rec. BT.500-11. Atest scene with natural content and motion
is displayed ina tightly controlled environment, with human viewers
expressing theiropinion of picture quality to create a DifferentialMean
Opinion Score, orDMOS. Extensive testing using this method can be
refined to
evaluatingthe capabilities of a compressed video system canbe inefficient,
taking severalwee ks to months to perform the experiments. This test
methodology can be extremelyexpensive tocomplete, andoften theresults
are notrepeatable. Thus, subjectiveDMOS testing withhuman viewers
is impracticalfor the CODEC designphase, and inefficient forongoing
operati
repeatable, andobjective measurement alternative to subjective DMOS
evaluation ofpicture quality.
surements that closelycorrespond with subjective human
, picturecomplexity,and the encodingalgorithm employed. The
yield aconsistent subjective rating. However, this method of
onal qualityevaluation. The PQA600 provides afast, practical,
Page 2
Data Sheet
UserInterface of PQA600. Showing reference, testsequences, with differencemap and
statisticalgraph.
System Eva
luation
The PQA600 can beused for installation, verification, andtroublesh ooting
ofeach blo
ckof the videosystem becauseit isvideo technology agnostic:
any visibledifferences between videoinput and output fromprocessing
componentsin the systemchain canbe quantifiedand assessedfor video
quality degradation. Not only canCODEC technologies beassessed in a
system, butany process that has potentialfor visible differencescan also
be assessed. Forexample, digital transmissionerrors, format co nversion
0i to480p in set-top box conversions), 3-2pull-down, analog
(i.e. 108
transmission degradation, da ta errors, slowdisplay responsetimes, frame
rate reduction(for mobile transmissionand videophone teleconferencing),
andmore can allbe evaluated,separately orin anycombination.
How It Works
The P QA600 takes twovideo files asinputs: areference videosequence
and a compressed, impaired,or processed versionof thereference. First,
the PQA600performs a spatialand temporal alignmentbetween the two
sequences, withoutthe need fora calibration stripeembedded within the
quence. Then the PQA600 analyzes the qualityof the testvideo,
videose
using measurementsbased on the humanvision system and attention
models, an d then outputsquality measurements thatare highlycorrelated
with subjectiveassessments. Theresults include overallquality summary
metrics,frame-by-frame measurement metrics,and animpairment map for
eachframe. ThePQA600 alsoprovides traditionalpicture qualitymeasures
such asP
SNR (PeakSignal-to-Noise Ratio) as anindustry benchmark
impairment diagnosistool for measuring typicalvideo impairments and
detecting artifacts.
Eachreference video sequenceand testclip canhave different resolution s
and framerates. The PQA600 can provide picturequality measurement
betweenHD vs SD, SD vsCIF,or anycombination. Thiscapability sup ports
Picture Quality Analysis System
a varietyof repurposing applications such as format conversion, DVD
authoring, IPbroadcasting, and semiconductordesign. ThePQA600 can
also supportmeasurement clips withlong sequence duration,allowing a
video clipto be quantified for picture quality throughvarious conversion
processes.
Prediction of Human Vision Perception
PQA600 measurementsare developed from thehuman vision system
model andadditional algorithms have beenadded to improve uponthe
model usedin the PQA200/300. Thisnew extended technology allows
legacy PQR measurements forSD whileenabling predictions ofsubjective
quality ratingof video for a varietyof video formats(HD, SD, CIF, etc.). It
takes into consideration different display typesused to view the video(for
example, interlacedor progressiveand CRTor LCD) anddifferent viewing
conditions (for example, roomlighting and viewin g distance).
A modelof the human visionsystem has beendeveloped to predict the
response tolight stimuluswith respectto thefollowing parameters:
Contrast includingSupra-threshold
Mean Luminance
Spatial Frequency
Temporal Frequency
Angular Extent
Temporal Extent
Surround
Eccentricity
Orientation
Adaptation Effects
2 www.tektronix.com
Page 3
A: Modulation Sensitivity vs. TemporalFrequency
Picture Quality Analysis System— PQA600
C: Reference Picture
B: Modulation Sensitivity vs. SpatialFrequency
This modelhas been calibrated, over theappropriate combinations of
s for these parameters, with reference stimulus-response data from
range
visionscience research. As aresult ofthis calibration,the modelprovides a
highly accurateprediction.
ptual Contrast Map
D:Perce
The graphsabove areexam ples of scientificdata regarding humanvision
characteristics usedto calibrate the humanvision system model in the
PQA600. Graph (A) showsmodulation sensitivity vs. temporal frequency,
and graph (B ) shows modulation sensitivity vs. spatial frequency. The
over 1400calibration pointssupports high-accuracy measurement
use of
results.
re (C)is a single frame fro m the reference sequence ofa moving
Pictu
sequence, andpicture (D)is theperceptual contrast mapcalculated bythe
PQA600. The perceptualcontrast mapshows howthe viewerperceives the
reference s equence. The blurringon thebackground iscaused by temporal
masking due to camera panning and theblack area around the jogger
showsthe masking effectdue to the highcontrast betweenthe background
ejogger. The PQA600creates the perceptualmap forboth reference
andth
and testsequences, then createsa perceptual differencemap for usein
making percep tually based, full-referencepicture quality measurements.
www.tektronix.com 3
Page 4
Data Sheet
E: Reference
F:Test
Comparison of Predicted DMOS with PSNR
In theexample above, Reference (E)is a scenefrom oneof the VClips
libraryfiles. Theimage Test (F),has been passedthrough a compression
system which has degraded the resultant image. In this case the
background ofthe joggerin Test (F) is blurred comparedto the Reference
image (E). A PSNR measurement ismade onthe PQA600of thedifference
en the Reference and Testclip and the highlighted whiteareas
betwe
of PSNR Map (G) shows the areasof greatest difference betweenthe
original and deg raded image. Anothermeasurement is then made bythe
PQA600, thistime usingthe Predicted DMOSalgorithm and theresultant
Perceptual D ifference Map forDMOS (H) imageis shown. Whiter regions
G: PSNR Map
ptual Difference Map forDMOS
H:Perce
in thisPerceptual Contrast Difference mapindicate greater perceptual
contrast differencesbetween the referenceand test images. In creating
the Perceptual Contrast Differencemap, thePQA600 usesa human vision
system model to determinethe differencesa viewer wouldperceive when
ing the video.
watch
The Predicted DMOS measurement uses the Perceptual Contrast
rence Map(H) to measure picture quality. ThisDMOS measurement
Diffe
wouldcorrectly recognize theviewers perceivethe joggeras lessdegraded
than the treesin the background. The PSNR measurement uses the
differencemap (G) andwould incorrectly includedifferences that viewers
do not see.
4 www.tektronix.com
Page 5
AttentionMap Example: The joggeris highlighted
Attention Model
ThePQA600 alsoincorporates an AttentionModel thatpredicts focusof
attention. Thismodel considers:
Motion of Objects
Skin Coloration(to identify people)
Location
Contrast
Shape
Size
ViewerDistraction due toNoticeable Quality Artifacts
These attention parameters canbe customized to give greateror less
importanceto each characteristic. Thisallows each measurementusing an
on modelto beuser-configurable. Themodel is especiallyuseful to
attenti
evaluatethe video process tuned tothe specificapplication. For example, if
the content issports programming, theviewer is expected to have higher
attentionin limited reg ional areas of the scene. Highlightedareas within th e
attention imagemap will showthe areas of th e image drawing theeye’s
attention.
Artifact Detection
Artifact Detection reports avariety of differentchanges tothe edges of the
image:
Loss ofEdges or Blurring
Addition ofEdges orRinging/Mosquito Noise
Rotationof Edges to Verticaland Horizontalor EdgeBlockiness
Loss of Edges withinan Image Block or DCBlockiness
They work as weighting parameters for subjective and objective
measurements with any combination. The resultsof these different
rement combinationscan help toimprove picture qualitythrough
measu
the system.
Picture Quality Analysis System— PQA600
ArtifactDetection Settings
For example,artifact detection canhelp answer questionssuch as: “Will
theDMOSbeimprovedwithmorede-blockingfiltering?” or, “Should less
prefiltering be used ?”
If edge-blockingweighted DMOS ismuch greater thanblurring-weighted
DMOS, theedge-b loc king isthe dominant artifact, and perhaps more
de-blocking filteringshould be considered.
In someapplications, itma y be knownthat added edges,such as ringing
and m osquito noise, aremore objectionablethan the otherartifacts. These
weightingscan becustomized bythe userand configured forthe application
toreflectthis viewerpreference, thus improvingDMOS prediction.
Likewise, PSNRcan be measured with these artifact w
determine howmuch of theerror contributing tothe PSNR m easurement
comes from each artifact.
The Att ention Model andArtifact Detectioncan also be used inconjunction
with anycombination of picture quality measurements. This allows,for
le, e valuation of howmuch of aparticular noticeable artifactwill be
examp
seen wherea viewer ismost likely tolook.
eightings to
www.tektronix.com 5
Page 6
Data Sheet
Configure Measure Dialog
Comprehensive Picture Quality Analysis
The P QA600 provides FullReference (FR) picturequality measurements
that comparethe luminance signal ofreference and test videos. It also
ome NoReference (NR) measurementson theluminance signal of
offerss
thetest video only. ReducedReference (RR)measurements canbe made
manually fromdifferences in NoReference measurements. Thesuite of
measurements includes:
Casual Viewing (Attention Weighted,Full Reference,or No Reference)
Picture Quality
Peak Signal-to-Noise Ratio (PSNR,Full Reference)
Focus ofAttention (Applied toboth Full Referenceand No Reference
Measurements)
ArtifactDetection (Full Reference, except forDC Blockiness)
DC Blockiness(Full Referenceand No Reference)
Edit Measure Dialog
The PQA600 supports these measurements through preset and
user-defined combinations ofdisplay type, viewing conditions, human
vision response(demographic), focus ofattention, and artifactdetection,
in additionto the defaultITU BT-500conditions. The abilityto configure
measurement conditions helps CODEC designers evaluate design
trade-offs as they optimize for different applications, and helps any
user investigate h ow different viewing conditions affectpicture quality
measurement results. Auser-defined measurementis createdby modifying
apreconfigured measurementor creating a new one,then saving and
recalling theuser-defined measurementfrom theConfig ure Measure dialog
menu.
Easy-to-Use Interface
The PQA600 has two modes: Measurement and Review. The
urement modeis used toexecute the measurementselected in the
Meas
Configure Dialog. During measurementexecution, the summarydata and
map resultsare displayed on-screenand saved tothe system harddisk.
The R eview mode isused to viewpreviously saved summaryresults and
maps creat ed either withthe measurement modeor XML scriptexecution.
The user can choosemultiple results in this modeand compare each
ltside by sideusing thesynchronous displayin Tilemode. Comparing
resu
multiple resultsmaps made with the differentCODEC parameters and/or
differentmeasurement configurations enableseasy investigationof theroot
cause ofany difference.
6 www.tektronix.com
Page 7
Integrated Graph
Picture Quality Analysis System— PQA600
Overlay display,Reference and Map
Six-tiled display
Multiple Result Display
Resultantmaps can be displayed synchronouslywith thereference andtest
video in a Summary,Six-tiled, orOverlaid display.
In Summarydisplay, the user can seethe multiple measurementgraphs
with abarchart along withthe reference video,test video, anddifference
map during videoplayback. Summarymeasures of st andard parameters
erceptual summation metrics for each frame and overall video
and p
sequence areprovided.
x-tiled display, theuser candisplay the2 measurement resultsside by
InSi
side. Each consists of a referencevideo, testvideo, and differencemap to
compare toeach other.
Autospatial alignment execution with spatialregion of interestselected
In Overlay display,the usercan control themixing ratio with the faderbar,
enabling co-locationof differencemap, reference, andimpairments intest
videos.
Error log ging and alarmsare available tohelp users efficiently trackdown
the causeof video quality problems.
The PQA600supports automatic temporaland spatial alignmen t, as well
as manualalignm ent.
The automaticspatial alignment withspatial regionof interest inMeasure
modeselected independently ofthe spatialalignment functioncan measure
the cropping, scale,and shift in eachdimension, even across different
utionsand aspect ratios(for example,when aligning SD to HDvideo).
resol
Ifextra blanking ispresent withinthe standardactive region, itis measured
as croppingwhen this functionis enabled.
The automatic spatial andtemporal alignment allows picture quality
measurement betweenreference and testvideos of differentresolutions
and frame rates.
www.tektronix.com 7
Page 8
Data Sheet
SpatialROI on Reviewmode forin-depth investigation
Import/ExportScript in Configure MeasureDialog
Script Sample
Region of Interest
Onthe Re
viewmode, the PQA600allows usersto set thetemporal/spatial
Region OfInterest (ROI) area for detailedanalysis. Spatial ROI inthe
Review modeis instantly selectedby mouse operation and givesa score
just forthe selected spatial area. It’s an effectiveway to investigate the
specific spatial region in thedifference map showingcertain impairments.
Temporal ROIin theReview modeis setby marker operationon thegraph.
alROI allowsusers toget aresult fora scenewhen thevideo stream
Tempor
has multiplescenes. Temporal ROIalso allows usersto provide aresult
withoutthe influence frominitial transientsin thehuman visionmodel.
AutomatedTestingwithXMLScripting
In theCODEC debugging/optimizing process,the designer may wantto
repeat several measurement routinesas CODEC parameters a re revised.
Result File Sample
Automatedregression testingusing XMLscripting can ease therestrictions
of manualoperation by allowingthe userto write aseries ofmeasurement
sequences withinan XML script. The script filecanbeexportedfromor
importedto the measurementconfiguration menu tocreate andmanage the
scriptfiles easily. Measurementresults ofthe scriptoperation canbe viewed
g either the PQA600 userinterface or anyspreadsheet application
by usin
thatcan read thecreated .csvfile formatas asummary. Multiple scriptscan
be executed simultaneously forfaster measurement resu lts.
8 www.tektronix.com
Page 9
Picture Quality Analysis System— PQA600
Generation
/Capture
Optional SD/HD SDI Interface
An optionalSD/HD SDI interface en ables both generation and capture of
SDI video with threemodes of simultaneousoperation.
Simultaneous generationand capture letsthe user playoutthe reference
video clips directly from thePQA600 into the device under test. The
test outputfrom the devicecan then be simultaneouslycaptured by the
PQA600. This saves the user fromhaving touse an externalvideo source
to apply any required video input to the device under test. With this
generatio
ncapability, filescreated byvideo editingsoftware can bedirectly
usedas referenceand testsequences for picture qualityme asureme nts.
Simultan
asreference andtest videosin evaluatingthe deviceunder testin operation.
To accomm
odate equipmentprocessing delay thatmay be present in the
system, theuser can usethe Delay Startfunction when capturingvideo.
Using D elayed Start minimizesthe number ofunused overheadframes in
the testfile and enablesfaster execution of the auto temporal alignment
in themeasurement.
Simultaneous 2-channel generation capabilitysupports two types of
subjectivetesting. Withtwo displays,the usercan check the reference and
test videocontent on side-by-side monitors.
2-channel Generation
2-channel Capture
With one display and swap-channel capability,the user can che ck the
reference and test videocontent withoutmoving theeye’s focuspoint.
www.tektronix.com 9
Page 10
Data Sheet
Supported File Formats for SD/HD SDI Interface
The SD/HDSDI video option can generateSDI video from files inthe
following formats (8 bit unlessotherwise stated):
Display Technology: CRT/LCD/DMD each with preset
and user-configurable parameters(Interlace/Progressive,
Gamma, Response Time,etc). Reference Displayand Test
Displaycan beset independently
Viewing distance, Ambient Luminance for Reference
and Testindependently, imagecropping and registration:
automaticor manualcontrol of image cropping andtest
imagecontrast (ac gain),brightness (dc offset),horizontal
and vertical scale and shift
average brightness, response speed tothe movingobject,
sensitivity to photosensitiveepilepsy triggers, etc)
(Ringing/Mosquito Noise), Rotated Edges (Edge
Blockiness), and DCBlockiness (Removed detail within a
block)
or % Perceptual Contrast. Objective: Mean Abs LSB,
dB)., Map type: Signed on gray or unsigned on black.
Worst-caseTrainingSequence for ITU-RBT.500Training
(Defaultor User-application Tuned: Determinedby Worst
Case Video % Perceptual Contrast), ErrorLog Threshold,
Save Mode
*If the Europeanphone number aboveis not accessible,please call: +41 52 675 3777
rope, Ukraine, and the Baltics +41 52 675 3777
Central Europe & Greece +41 52 675 3777
Middle E ast,
Asia, and North Africa +41 52 675 3777
The Netherlands* 00800 2255 4835
People’s Rep
Republic of
United Kingdom & Ireland* 00800 2255 4835
sia (65) 63563900
Austria* 00800 2255 4835
Belgium* 00800 2
Brazil +55(11)37597600
Canada 1 800833 9200
Denmark +4580881401
Finland +41526
France* 00800 2255 4835
Germany* 00800 2255 4835
Hong Kong 400 8
India 000 800 650 1835
Italy* 00800 2255 4835
Japan 81 (3) 67
Luxembourg +41526753777
ublic of China 400 820 5835
Poland +41 52 675 3777
Korea 001 800 8255 2835
Russia & CIS +7 (495) 7484900
South Africa +41526753777
Spain* 0080
Sweden* 00800 2255 4835
Switzerland* 00800 2255 4835
Tai wa n 886 (
255 4835
75 3777
20 5835
14 3010
Norway 800 16098
Portugal 80 08 12370
022554835
2) 2722 9622
USA 1 800 833 9200
Updated 21 April 2010
www.tektronix.com
For Further Information. Tektronix maintains a comprehensive, constantly expanding
collection of application notes, technical briefs and other resources to help engineers working
on the cutting edge of technology. Please visit www.tektronix.com