Mathworks SYSTEMTEST 2 user guide

Download

SystemTest™ 2

User’s Guide

How to Contact The MathWorks

www.mathworks. comp.soft-sys.matlab Newsgroup www.mathworks.com/contact_TS.html Technical Support

suggest@mathworks.com Product enhancement suggestions

bugs@mathwo doc@mathworks.com Documentation error reports service@mathworks.com Order status, license renewals, passcodes

info@mathwo

com

rks.com

Web

Bug reports

Sales, prici

ng, and general information

508-647-7000 (Phone)

508-647-7001 (Fax)

The MathWorks, Inc. 3 Apple Hill Drive Natick, MA 01760-2098

For contact information about worldwide offices, see the MathWorks Web site.

SystemTest™ User’s Guid e

The software described in this document is furnished under a license agreement. The software may be used or copied only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written consent from The MathW orks, Inc.

FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern theuse,modification,reproduction,release,performance,display,anddisclosureoftheProgramand Documentation by the federal government (or other entity acquiring for or through the federal government) and shall supersede any conflicting contractual terms or conditions. If this License fails to meet the government’s needs or is inconsistent in any respect with federal procurement law, the government agrees to return the Program and Docu mentation, unused, to The MathWorks, Inc.

Trademarks

MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See

www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand

names may be trademarks or registered trademarks of their respective holders.

Patents

The MathWorks products are protected by one or more U.S. patents. Please see

www.mathworks.com/patents for more information.

Revision History

May 2006 Online only New for Version 1.0 (Release 2006a+) September 2006 First printing Revised for Version 1.0.1 (Release 2006b) March 2007 Online only Revised for Version 1.1 (Release 2007a) September 2007 Second printing Revised for Version 2.0 (Release 2007b) March 2008 Online only Revised for Version 2.1 (Release 2008a) October 2008 Online only Revised for Version 2.2 (Release 2008b) March 2009 Online only Revised for Version 2.3 (Release 2009a) September 2009 O nline only Revised for Version 2.4 (Release 2009b) March 2010 Online only Revised for Version 2.5 (Release 2010a)

Getting Started

Product Overview ................................. 1-2

Contents

Quick Tour of the SystemTest Software

Getting Familiar with the Desktop General Desktop Features Setting SystemTest Preferences Viewing Test Results

Running Tests from the MATLAB Command Line

Example: Building a Test

Overview Planning Your Test Building Your Test Running Your Test Analyzing Your Test Results

........................................ 1-11

................................ 1-11

................................ 1-12

................................ 1-37

.......................... 1-5

...................... 1-7

.............................. 1-9

........................... 1-11

........................ 1-40

.............. 1-3

................... 1-3

Working with Test Vectors

Creating MATLAB Express io n Test Vectors .......... 2-2

.... 1-10

Creating Grouped Test Vectors

About Test Vectors and the MATLAB Workspace

Creating MA T-F ile Test Vectors

..................... 2-5

..................... 2-13

..... 2-12

Creating Randomized Test Vectors with Probability

Distributions

Using Probability Distributions in Test Vectors Creating a Test Vector with Probability Distributions Viewing Data While Configuring the Test Vector The Probability Distributions Example: Creating Test Vectors with Probability

Distributions

.................................... 2-18

......... 2-18

........ 2-23

........................ 2-26

................................... 2-34

.... 2-18

Creating Spreadsh eet Data Test Vectors

Introduction Creating a Spreadsheet Data Test Vector Configuring the Spreadsheet Data Test Vector Replacing Strings

Creating Simulink Design Verifier Data File Test

Vectors

Prerequisites Automatically Creating a SystemTest Test Harness from

Simulink

Creating a Simul ink Design Verifier Data File Test

Vector

Important Usage Notes

Creating Signal Builder Block Test Vectors

Creating a Test Case Data Test Vector

Using a MATLAB Element to Access Test Case Data

Test Vector Information

...................................... 2-44

................................. 2-51

......................................... 2-53

..................................... 2-53

Design Verifier ........................ 2-53

......................................... 2-55

............................. 2-65

.......................... 2-76

............. 2-44

.............. 2-44

.......... 2-48

.......... 2-67

............... 2-73

vi Contents

Editing a Test Vector from within an Element

........ 2-77

Working with the Basic Elements

Working with the Sections of a Test ................. 3-2

Overview

........................................ 3-2

Pre Test ......................................... 3-2

Main Test Post Test

........................................ 3-3

Basic Elements

Introduction MATLAB Element Limit Check Element — General Check Limit Check Element — Tolerance Check IF Element General Plot Element Vector Plot Element Scalar Plot Element Stop Element Subsection Element

Deprecated Elements

Converting Elements Scalar Plot Conversion Details Vector Plo t Conversion Details

.................................... 3-5

...................................... 3-5

................................. 3-6

............... 3-7

.............. 3-11

....................................... 3-14

.............................. 3-15

............................... 3-20

............................... 3-23

..................................... 3-26

................................ 3-27

.............................. 3-29

....................... 3-31

....................... 3-32

Using the Simulink Element

Before You Begin .................................. 4-3

Mapping Test Vectors and Test Variables to a Simulink

Model

Introduction Adding a Simulink Element Specifying the Simulink Model Overriding Simulink Model Inputs Mapping Simulink Model Outputs to Test Variables Using the Model Output M appings Assistant Editing a Test Vector or Test Variable from within the

Overriding Inport Block Signals

Introduction

.......................................... 4-5

...................................... 4-5

......................... 4-6

...................... 4-7

................... 4-7

........... 4-20

Element

....................................... 4-21

.................... 4-22

...................................... 4-22

..... 4-13

vii

Overriding Inport Block Signals in a Simulink Element .. 4-23 Using the Inport Block Mappings Assistant Example: Overriding Simulink Inport Blocks Using a

Spreadsheet Data Test Vector Mapping Logged Signals from a Model to Inport Blocks Editing a Test Vector or Test Variable from within the

Element

....................................... 4-37

..................... 4-28

............ 4-27

.. 4-36

Using Simulink Model Coverage

Using Simulink

Using Signal Builder Block Tes t Cases in a Test

Using Test Cases and Signals from the Test Case Editor

in a Simulink Element

Design Verifier Data Files in a Test .. 4-45

........................... 4-47

.................... 4-38

...... 4-46

Authoring Signals in the Test Case Editor

Introduction to the Test Case Editor ................ 5-2

Definitions

Workflow of Authoring and Using Signals

Creating a Test Case Data Test Vector

....................................... 5-2

........... 5-4

............... 5-6

viii Contents

Working in the Test Case Editor

Navigating in the Edit View and Test Case View Creating Test Cases Adding Signals to Test Cases The Signal Types

Linking to Requirements in Telelogic

Introduction and Setup Adding Requirements Requirements Tab

............................... 5-13

.................................. 5-23

............................. 5-31

.............................. 5-31

................................. 5-34

.................... 5-9

........................ 5-18

DOORS ....... 5-31

........ 5-9

Test Case Report .................................. 5-37

Using Test Cases and Signals in SystemTest Test

Elements

Introduction Simulink Element MATLAB Element General Plot Element

....................................... 5-40

...................................... 5-40

................................. 5-40

................................. 5-41

.............................. 5-41

Using the Instrument Control Toolbox Elements

Introduction ...................................... 6-2

Instrument Control Toolbox Elements Accessing Resources

............................... 6-2

................ 6-2

Example: Measuring a Generator’s F requency

Introduction Setting Up the Signal Generator Setting Up the Oscilloscope Taking the Measurement Saving Test Results Running the Test and Viewing Test Results

...................................... 6-4

..................... 6-5

......................... 6-9

........................... 6-11

................................ 6-12

....... 6-4

............ 6-13

Using the Data Acquisition Toolbox Elements

Introduction ...................................... 7-2

Overview Data Acquisition Toolbox Test Elements

Example: Testing a Voltage Regulator

Introduction Sending Analog Stimulus Data to the DUT Enabling the DUT with Digital Data

........................................ 7-2

............... 7-2

............... 7-3

...................................... 7-3

............ 7-4

.................. 7-7

Receiving Analog Response Data from the DUT ......... 7-9

Disabling the DUT with Digital Data Performing Data Analysis Defining Post Test Elements Saving and Viewing Test Results

.......................... 7-12

........................ 7-13

................. 7-10

..................... 7-14

Using the Image Acquisition Toolbox Element

Introduction ...................................... 8-2

Example: Acquiring Video Data in a Test

Adding the Video Input Element to a Test Saving and Viewing Test Results Running the Test

.................................. 8-9

..................... 8-8

............ 8-3

............. 8-3

Distributing Tests Using Parallel Computing

Toolbox Integration

SystemTest Software and Parallel Computing Toolbox

Integration

Enabling Distributed Testing

Selecting a User Configuration

Setting Up File Dependencies

Setting Up Path Dependencies

..................................... 9-2

....................... 9-3

..................... 9-5

....................... 9-7

...................... 9-9

x Contents

Distributing Iterations Across Tasks

................ 9-12

Running a Distributed Test ......................... 9-14

Example: Distributing a Test

....................... 9-17

Using the Test Results Viewer

Viewing Test Results ............................... 10-2

Before You Begin

A Quick Tour of the Test Results Viewer

Viewing Your Test Results

Reserved K ey w ords Browsing Results Generating Plots Exploring Plots

Refining Your Test Results

Creating and Applying Constraints Plotting Sin gle Iterations

.................................. 10-3

............. 10-6

.......................... 10-8

................................ 10-8

.................................. 10-8

.................................. 10-9

................................... 10-16

......................... 10-29

................... 10-29

........................... 10-36

Viewing Simulink Time Series Data

Overview Creating a Time Series Plot

Saving and Reloading Test Results

Saving Test Results Loading Test Results

........................................ 10-38

......................... 10-38

................................ 10-43

.............................. 10-44

................. 10-38

.................. 10-43

Accessing Test Results from the MATLAB

Command Line

Viewing Test Results at the C omm and Line .......... 11-2

Introduction Accessing the Results Summary Accessing the dataset Array

...................................... 11-2

..................... 11-2

......................... 11-5

Working with Test Results

Introduction Managing Test Results Data in its Native Format Managing Test Results as a Dataset Array Plotting Results Data

Accessing Test Results While a Test Is Running

...................................... 11-8

.......................... 11-8

............. 11-9

.............................. 11-10

Function Reference

SystemTest Hot Keys

The dataset Array

....... 11-8

...... 11-15

xii Contents

Dataset Arrays .................................... B-2

Overview Test Results Data Looking at Data

Dataset Array Operations

........................................ B-2

................................. B-3

................................... B-3

.......................... B-5

Index

xiii

xiv Contents

Getting Started

This se ction explains w hat the SystemTest™ software is and shows you how to use it. It contains the following topics:

• “Product Overview” on page 1-2

• “Quick Tour of the SystemTest Software” on page 1-3

• “Running Tests f rom the MATLAB Command Line” on page 1-10

• “Example: Building a Test” on page 1-11

1 Getting Started

Product Overview

The SystemTest software provides MATLAB®and Simulink®users with a framework that integrates software, hardwa re, simulation, and other types of testing in one environment. You use predefined elements to build test sections that simplify the development and maintenance of standard test routines. You can save and share tests throughout a development project to ensure standard and repeatable test verification. The SystemTest sof twa r e offers integrated data management and analysis capabilities for creating and executing tests, and saving test results to facilitate continuous testing across the development process.

The SystemTest software automates testing in MATLAB and Simulink products. With the SystemTest software you get:

• Graphical test e diting — Quickly edit your test within a graphical test

• Repeatable test execution — All tests developed with the SystemTest

development environment.

software share the same execution flow, which provides a consistent test framework among tests.

1-2

• Parameterized testing — Create test vectors over which your test iterates.

• Reusability — After you design a test, you can save it for later use b y you

or others.

• Maintainability — Because you design and execute tests from the

SystemTest desktop, you do not need to understand unfamiliar code or concepts.

• Integration — The SystemTest software integrates with MATLAB,

Simulink, and other products based on MATLAB and Simulink.

Quick Tour of the SystemTest Software

In this section...

“Getting Familiar with the Desktop” on page 1-3

“General Desktop Features” on page 1-5

“Setting SystemTest Preferences” on page 1-7

“Viewing Test Results” on page 1-9

Getting Familiar with the Desktop

The SystemTest desktop is an integrated deve lo pme nt environment that lets you perform all of your testing activities from one centraliz ed location. This section provides an overview of the SystemTest environment. For more informationabouthowtousetheSystemTestsoftwaretobuildtestsandrun them, see “Example: Building a Test” on p age 1-11.

To get familiar with the SystemTest environment, open the SystemTest desktop from MAT LAB by selecting Start > MATLAB > SystemTest > SystemTest Desktop or typing

systemtest at the M ATLAB command line.

Quick Tour of the SystemTest™ Software

1-3

1 Getting Started

1-4

The desktop has a number of different panes that help you to build and run your test.

• Test Browser — Shows the overall structure of a test. A test is made up of

Pre Test, Main Test, Save Results, and Post Test. Use the Test Browser to add elements to your test. These elements determine what actions your test performs.

• Test Vectors — Lets you define the parameters or test cases of your test.

The test vectors you define determine the number of iterations performed by your test. Test vectors are automatically indexed during test execution.

Quick Tour of the SystemTest™ Software

• Test Variables — Lets you define variables used in the scope of your test.

Variables can serve both input and output functions in your test. You can define variables that are declared in the Pre Test section of your test or in the Main T est section of your test.

• Properties — Shows the properties of the test or the element you are

editing. The contents of this pane change when you select a section or element in your test.

• Elemen ts — If open, this undocked Elements pane allows you to add

elements to your test. If not open, you can add elements using the New buttonintheTest Browser.

• Resources — Lists the instrument or other external device resources

associated with the current test. This is only used if you have a license for the Instrument Control Toolbox™ software.

• Getting Started — Shows information to help you start using the

SystemTest software. If the Getting Started page is closed, select Help > SystemTest Getting Started to open it.

• Desktop Help — Shows help about the element or aspect of the test that

is currently selected. For the full product Help, select Help > SystemTest User’s Guide.

• Run Status — Shows a summary of the test’s execution status.

General Desktop Features

The SystemTest deskto p has a variety of features to make navigation easier.

Context Menus

Many areas of the user interface have context menus. For example, if you right-click in the Test Vectors, Test Variables, Resources, Run Status, Getting Started,orDesktop Help panes, you can access these context menus.

If you have the Elements pane open, you can add elements to your test using the context menus. If you right-click any element there, you can insert it directly into Pre Test, Main Test, or Post Tes t using the Elements pane context menus. If that section of the test already contains elements, the inserted element will be placed below the currently selected element in that

1-5

1 Getting Started

section. You can change the order of elements in the test by using the arrow buttons in the Test Browser,orbydragginganddropping.

Hot Keys

The SystemTest software offers various keyboard shortcuts, or hot keys, to access certain commands via the keyboard. For example, pressing F5 is an alternative way to run a test, and pressing Ctrl+N creates a new untitled test.

See the full list of SystemTest hot keys in Appendix A, “SystemTest H ot Keys”.

Undo/Redo Support

Undo and redo support is available through the Edit menu or on the SystemTest toolbar. This feature allowsyoutoundoactionsyouhavedone throughout the desktop. The undo queue is global to the entire desktop. For example, if you add a test vector and then perform an action in the Properties pane, those two actions will be the last two items in the queue. The undo order applies across all the panes in the desktop.

1-6

To use this feature, select the Edit > Undo action command, where action is the last action you performed. Use the Undo command repeatedly to undo multiple actions. The Edit > Redo action command will redo the last undo you performed.

Most actions in the desktop are undoable. Some actions pertaining to the elements that are part of the hardware toolboxes, Data Acquisition Toolbox™, Instrument Control Toolbox, and Image Acquisition Toolbox™, cannot be undone since they involve connections to hardware.

Thefollowingactionswillclearthelistofactionsintheundoqueue:

• Closing a test

• Opening a test

• Creating a new test

• Refreshing a Simulink model in the Simulink element

Quick Tour of the SystemTest™ Software

Setting SystemTest Preferences

You can set SystemTest preferences by selecting File > Preferences on the SystemTest desktop. This opens the MATLAB Preferences dialog box. Click SystemTest in the left tree if SystemTest Preferences are not showing in the right pane.

Most Recently Used Test List

This option determines how many tests will appear on the SystemTest File menu’s most recent files list. The default is 4 tests. If you change it to 0, no recent tests will appear on the list. The maximum number is 9.

1-7

1 Getting Started

Test Run Options

Select Minimize SystemTest when starting a test if you want the SystemTest desktop to minimize when a test starts running. This check box is cleared by default.

Select Save test before running if you want the SystemTest software to save your test before it runs. If this option is selected and you run a test thatisnotyetsaved,youwillbepromptedtonameandsavethetest. This check box is selected by default.

Note You can save a test any time, before or after running it, by selecting File > Save.

Confirmation Dialog Boxes

You can also turn off confirmation dialog boxes used in the SystemTest software in a diffe rent area of the P re fere nces dialog box by selecting General > Confirmation Dialogs. Three SystemTest confirmation dialog boxes are listed there, as shown in the figure that follows.

1-8

• Warn about launching Test Results Viewer without any saved

results — Occurs if you attempt to open the Test Results Viewer when the latest test that ran does not contain any mapped results under Saved Results.Tosaveresults,clickSaved Results in the Test Browser.

• Warn about using a Simulink model with an infinite simulation

stop time — Occurs if you attempt to run a test containing a Simulink

element that uses a model with an infinite simulation stop time.

• Warn about using a Simulink model with unnamed logged signals

— Occurs if you have a model that has logging enabled but has logged signals with no name, and you use that model in a Simulink element in the SystemTest software.

• Warn each time a new signal is added in the test case editor —

Occurs if you add a signal in the Test Case Editor.

• Warn each time a signal is deleted in the test case editor — Occurs if

you delete a signal in the Test Case Editor.

Quick Tour of the SystemTest™ Software

Viewing Test Results

The SystemTest software allows you to view the results you have chosen to save for your test using a workspace variable called access to the test results object, which is useful for comparing the results of separate test runs and for postprocessing test results.

For more information, see Chapter 11, “Accessing Test Results from the MATLAB Command Line”.

stresults.Itprovides

1-9

1 Getting Started

Running Tests from the MATLAB Command Line

You can run one or more SystemTest tests from the MATLAB command line, using the batch or calling a test file as part of a MATLAB file .

Note If you use this feature, it is a good idea to first run the test from the SystemTest desktop to verify that elements are not in an error state, and that the test will run successfully, before running it via the MATLAB command line using the

The function takes the name of your test file as a string. The test file must be on the MATLAB path, or you can specify the full path in the string.

strun function. This is useful for running multiple test files as a

strun function.

For example, to run a test called this syntax:

strun('mytest')

To run a test called mytest that is not on the MATLAB path, but is in a local directory called

strun('c:\work\mytest.test')

To run multiple tests, use a cell array of strings, as follows:

strun({'mytest' 'mytest2'})

Note MATLAB will remain busy while tests are executing via the strun command. Control is returned to the MATLAB command line once all tests execute.

For more information about using strun, see the f unction page.

c:\work, use this syntax:

mytest that is on the MATLAB path, use

1-10

Example: Building a Test

In this section...

“Overview” on page 1-11

“Planning Your Test” on page 1-11

“Building Your Test” on page 1-12

“Running Your Test” on page 1-37

“Analyzing Your Test Results” on page 1-40

Overview

This simple example illustrates the four primary stages of testing: planning, building, running the test, and viewing test results.

The example uses a simple MATL AB expression to emulate a scalar measurement during each iteration of the test. The example uses an arbitrary formula d ependent on the test vector named The example tests each measurement to determine if it falls within certain specified limits. If a measurement exceeds these limits, that particular iteration of the test fails. By default, the t est fails if any iteration fails, but you can configure other pass/fail criteria.

Example: Bui lding a Test

signal to generate the Y data.

Thefollowingsectionsprovidemoreinformation about each stage, building the example test along the way. If you prefer, instead of working through the following sections to b uild the example, you can load it into the SystemTest software by running the Getting Started with SystemTest demo from the

Demos page in the MATLAB H elp browser (under MATLAB > SystemTest > MATLAB)orbyentering

command prompt.

systemtest Simple_Demo at the MATLAB

Planning Your Test

In this first stage, you must identify what it is you want to test. The SystemTest so ftware lets you specify input data, such as measurements from a model or device, and compare this input data to some predefined limits. Based on this comparison, the SystemTest software can declare whether a test passes or fails.

1-11

1 Getting Started

Keep the following in mind as you plan tests:

• Identify your test data and test vectors.

• Specify test limits and determine if these limits can be expressed as scalar

or matrix values. (The Limit Check element supports both scalar and matrix data.)

• Determinewhatoperationsyourtestmustperform. Mustcertain

operations happen before others?

• Determine pass/fail criteria for your test.

• Decidewhichtestvariablesyouwanttosaveastestresults.

After this planning, you can begin to construct your test, which is described in “Building Your Test” on page 1-12.

Building Your Test

The SystemTest interface provides a graphical integrated environment that you can use to create and edit tests. Tests consist of elements, test v ectors, and test variables. You can use each of theseentitiestocreateavariety of test scenarios ranging from a simple test that runs a series of elements once to a full parameter sweep that iterates over the values of test vectors that you define.

1-12

The following sections show how to construct a test:

• “Starting the SystemTest Software” on page 1-13

• “Structuring Your Test” on page 1-13

• “How Test Vectors and Test Variables Relate to the MATLAB Workspace”

on page 1-15

• “Creating a Test Vector” on page 1-15

• “Defining Test Variables” on page 1-18

• “Adding Elements” on page 1-20

• “Defining Pass/Fail Criteria” on page 1-30

• “SavingTestResults”onpage1-31

Example: Bui lding a Test

• “Generating a Test Report” on page 1-34

• “Saving Your Test” on page 1-36

Starting the SystemTest Software

Start by opening the SystemTest desktop using the MATLAB Start button. To open the SystemTest software, select Start > MATLAB > SystemTest > SystemTest Desktop.

Alternatively, you can execute the command line.

The SystemTest software displays the desktop on your screen. See “Quick Tour of the SystemTest Software” on page 1-3 for an overview.

systemtest command from the MATLAB

Structuring Your Test

The SystemTest software divides tests into three sections.

• Pre Test — This sectio n is used to execute test elements in order to

perform any test set-up operations, such as initializing variables, loading data from a file, and initializing system resources. Using Pre Test variables, you can assign an initial value to a test variable that persists between M ain Test section iterations (unless another element in Main Test modifies the value). Pre Test is not mandatory, but it can be used if your test requires set-up operations to be performed.

• Main Test —MainTestdefinesthetestelementsthatneedtobe

performed across the parameter space d efined by your test vectors. In this section Main Test variables are initialized before each Main Test iteration, which lets you assign an initial value to a test variable each time the Main Test runs. This is u sefu l if your test variable has a derived valu e such as being indexed by a test vector or is the result of a MATLAB expression.

The number of iterations performed in the Main Test is indicated in the Test Browser in parentheses after Main Test. Iterations specifies the number of times the Main Test section will be run. This is determined from the test vectors you define. The SystemTest desktop also offers a Save Results area for you to specify which test variables you want to save as test results at the end of each Main Test iteration.

1-13

1 Getting Started

• Post Test — In this section you can perform any cleanup work necessary

at the completion of the Main Test section, such as clearing workspace variables, closing a file, or freeing system resources.

For details about the sections of the test, see “Working with the Sections of a Test” on page 3-2.

The following figure illustrates the structure of a test.

1-14

Example: Bui lding a Test

How Test Vectors and Test Variables Relate to the MATLAB Workspace

The SystemTest software has its own internal workspace that it uses to manage test variables and test vectors independently. However it does leverage the MATLAB workspace during test execution, and when using a MATLAB element.

During test execution, SystemTest test variables and test vectors are evaluated in the MATLAB base workspace. Then at the end of test execution, they are cleared out and the MATLAB base workspace is restored to what it was before the test execution.

When using a MATLAB element in the SystemTest software, you can reference a variable in the base workspace without having to create a test vector or test variable in the SystemTest software. However the SystemTest software will not be aware of this data, so you could not make use of it in any other element type or in saved results. You can only access it from a MATLAB element. If you need to use it in other elements, you can create test variables or test vectors in the SystemTest software.

Creating a Test Vector

Test ve cto rs are composed of values derived from a MATLAB expression. You can use any MATLAB expression that evaluates to a 1-by-n matrix or cell array to define your test vector. Using test vectors, you can iterate through a range of values to see how a system performs. Test vectors constitute parameterized testing in the SystemTest software. They are the test cases foryourtest.

For tests with multiple test vectors, the product of the lengths of the test vectors defines the number of iterations the test performs. For example , if you define the test vector 10 for the first run, 20 for the second, and 30 for the final run. If you add a second test vector w ith three other values, the total number of test runs would be nine. The SystemTest software iterates through each vector in combination with the other vector as though the test were a group of nested FOR loops—the outermost loop being the first test vector in your table and the innermost loop being the last test vector. The Main Test sectionintheTest Browser shows the total number of test iterations defined by your test v ectors .

[102030], the test runs three times, using a value of

1-15

1 Getting Started

For the example, use the vector [pi/15:pi/15:4*pi] which defines 60 values for our test vector ranging from

pi/15 to 4*pi in pi/15 increments. To specify

this test vector:

1 Click the New Vector button in the Test Vectors pane.

The Insert Test Vector dialog box opens.

1-16

2 Keep t

3 Assign a value to the test vector by clicking the Expression field. Enter

he default test vector type of M ATL AB Expression. Assign a name test vector by clicking the Name field. For this example, name the

to the

ector

test v

signal.

the test vector specified above for the pi values. Click OK.

Example: Bui lding a Test

After you create the test vector, in the Test Browser pane, the Main Test section label updates to include the number of iterations defined by the test vector. It should say

Main Test (60 Iterations).

1-17

1 Getting Started

Note Grouping test vectors determines how they will be iterated through

when the test runs. For information on grouping vectors, see “Creating Grouped Test V ectors” on page 2-5.

Note You can also use probability distributions when you create a test vector. For information, see “Creating Randomized Test Vectors with Probability Distributions” on page 2-18.

Defining Test Variables

The SystemTest software u ses test variables to define temporary storage variables that a test acts on or generates. You assign test variables in the Pre Test or Main Test sections of your test.

You can define Pre Test variables or Main Test variables. Using Pre Test variables, you can assign an initial value to a test variable that persists between Main Test section ite rations (unless another element in M ain Test modifies the value). Pre Test is not m an d atory, bu t it c a n be used if your test requires set-up operations to be performed.

1-18

Main Test defines the test elements that need to be performed across the parameter space defined by your test vectors. Main Test variables are initialized before each Main Test iteration, which allows you to assign an initial value to a test variable each time the Main Test runs. This is useful if your test variable has a derived value such as being indexed by a te st vector or is the resu l t of a MATLAB expression. You add elements in this section.

The example test requires three test variables:

•

Y — Contains a value that will be calculated from the signal test vector

at each iteration.

•

HiLimit — Contains the upper limit for Y that you do not want the signal

to exceed.

•

LowLimit — Contains the lower limit for Y that you do not want the signal

to go below.

Example: Bui lding a Test

To create thes e test variables:

1 Click the Test Variables tab in the middle pane of the SystemTest desktop.

2 Click the New button to create a Pre Test or Main Test variable. The

Insert Test Variable dialog box opens. Leave the default value o f

in the Assigned in field, to create a new Main Test variable.

Test

Main

3 Assign a name to the test variable by clicking the Name field and entering

the test variable name. For this example, enter

4 Set the test variable’s initial value by clicking the Initial Value field and

entering a value. For the example test variable

Y,enter0.ClickOK.

Note If you do not provide an initial value, it will default to empty, that is,

Var1 = []; in MATLAB code.

1-19

1 Getting Started

Note Test variables are re-initialized at the start of each test iteration.

The Initial value field is blank by default when you create a test variable. If you leave it blank, it will initialize to (which can be any valid MATLAB expression), that value gets assigned in every iteration.

[]. If you enter an initial value

1-20

5 Repeat st

setting

Variable Name Initial Value

HiLimit 1

LowLimit -1

Addin

Elements are the actions that a test performs. The SystemTest software includes the following set of elements, listed in alphabetical order.

• General Plot — Used to plot any type of data over multiple iterations.

• IF — Implements a logic control operator.

• Limit Check — Specifies the comparison to be performed of the value(s)

under test and their e xpected value(s), or limit(s).

• MATLAB — Executes any MATLAB s tatements.

eps 2 to 4 to create the remaining two test variables, using the

s listed in the following table:

Assign in

Main Test

Main Te

gElements

Example: Bui lding a Test

• Simulink — Runs a Simulink model. Note that you need to have a license

for Simulink to use this element.

• Stop — Implements a logic control operator.

• Subsection — Creates a new section in a test that you can use to group

elements within.

Note Some MathWorks products, such as the Image Acquisition Toolbox™ software, the Data Acquisition Toolbox™ software, and the Instrument Control Toolbox™ software, provide their own elements that integrate those products’ capabilities within the SystemTest software. If you have licenses for those pro ducts, those elements will also appear in the elements list.

For more information about using the basic elements, see Chapter 3, “Working with the Basic Elements”.

You add elements to a section in your test; however, not all elements can be added to all sections. For example, you can use a MATLAB element anywhere within a tes t, but you can o nly use the Limit Check element in the Main Test section.

To illustrate using elements, let’s continue with this example. This test uses three elements in the Main Test section.

Element

Description

MATLAB Use a MATLAB expression to assign data to Y that is

dependent on the test vector

signal.

Limit Check Compare the value generated in the MATLAB element to

the specified limit and see if the Y test variable exceeds

General Plo t

the upper or lower limit you defined in your

LowLimit test variables.

Plot the current test variable values and see whether the

HiLimit and

test variable exceeds the upper and lower limits.

To add these eleme n ts :

1-21

1 Getting Started

1 Select the section of the test in which you want to add the element. For this

example, click Main Test in the Test Browser.

1-22

2 Specify the element you want to add to the test section. For this example,

click the New > Test Element button and select MATLAB.AMATLAB element appears in the M ain Test section of your test and the MATLAB element property page opens in the Properties pane of the SystemTest desktop.

Example: Bui lding a Test

3 In the Properties pane, type the following code in the MATLAB Script

edit box. This MATLAB code calculates a value for on the test vector

Y = sin(signal)+ rand -.5 ;

signal.

Y that is dependent

During each iteration, the SystemTest software evaluates the MATLAB expression and assign s a value to

1-23

1 Getting Started

4 Add the Limit Check element to the Main Test section of the test. With

the MATLAB element selected, click the New > Test Element button, and click Limit Check. A Limit Check element appears in the Main

Test section of the test and the Limit Check properties page opens in the Properties pane. For this example, the Limit Check element must follow

the MATLAB element in the test.

Note You can reposition an element in a test by selecting the element and then clicking the up and down arrows in the Test Browser toolbar. Y ou can also drag and drop elements within Main Test. You cannot move elements between test sections.

1-24

Example: Bui lding a Test

In the General Check tab, click the New button to add a limit check. Notice that the Limit Check element icon in the Test Browser shows a red x, which indicates that inform a t ion is missing. The corresponding red outlining in the Properties pane highlights any fields that require configuration. A test cannot run unless everything is properly configured.

1-25

1 Getting Started

5 Specify the limit comparison operations in the Limit Check element.

a In the Test Variable column, click the drop-down list and select a test

variable you created in step 4. For this example, select

b In the Operator column, click the drop-down list and select the

comparison you want to perform. For this example, pick the less-than-or-equal-to operator,

c In the Limit column, click the drop-down list and select the test variable

you w ant to compare to. For this example, select

<=.

HiLimit,whichisthe

test variable you created earlier.

The following figure shows the configuration of this limit.

1-26

6 To add another limit comparison operation, click the New button again in

the General Check Properties pane. A new row appears below the last limit you specified. In this new row, set Test Variable to to

>=,andsetLimit to LowLimit.

Y,setOperator

Example: Bui lding a Test

The following figure shows the configuration of this second limit.

For e ach iteration of the Main Test, the MATLAB element’s expression is evaluated and a new value assigned to runs, it determines whether the value of

LowLimit values. If Y is outside this range, the test iteration fails. The

Y. When the Limit Check element

Y falls between the HiLimit and

default pass/fail criteria for the overall test passes the test only if both expressions in the limit check evaluate to true.

7 To view the test variables as the test runs, plot the data. To add a Plot

element to the test, click the New > Test Element button, and select General Plot. A General Plot element appears in the Main Test section, and the properties page for the element opens in the Properties pane.

1-27

1 Getting Started

1-28

With each Main Test iteration of the test, the General Plot element updates a figure window with data you selected.

8 Click the Add Plot button, and choose plot fromthelisttocreatealine

plot.

9 We will

create

set up three axes. For the first axes, use the one automatically

d. Configure it as follows:

Click the arrow in YDataSourceand select Keep t

of sol

he default Line color of blue, and keep the default Line style

id.

Change Line marker to point (the first selectio n in t he list that shows one dot).

On the Options tab, select Keep any existing data on the figure.

10 Add t

11 Configure the second and third axes to match the following table, and using

he second and third axes by clicking the Add Plot button again twice

choosing plot fromthelisttocreatealineplot.

and

<Iteration Number> as the XDataSourcefor each one and selecting

Example: Bui lding a Test

the Keepanyexistingdataonthefigureoption for each one. The configured element looks like the figure following the table.

YDataSorce

HiLimit

LowLimit

Line Color Line Style Li ne Marker

Blue

Solid

Point

Red Dashed No Marker

Black Dashed No Marker

1-29

1 Getting Started

To see the resulting plot, see “Tracking Output” on page 1-37.

Defining Pass/Fail Criteria

You can define whether your test passes or fails by monitoring the outcome of any or all Limit Check elements during any or all Main Test iterations. Your test’s threshold of success can range from the passing of any Limit Check in any single test iteration to the passing of all Limit Check elements in all test iterations. If your test contains no Limit Check elements, there is no notion of pass/fail and no pass/fail information is displayed. (Testing of this type is useful for experimenting with a system or to e xplore its behavior rather than validate its performance.)

You can set any of the following conditions to define when your test passes:

• All Limit Check eleme nts pass in a ll test iterations.

• All Limit Check elements pass in any test iteration.

• Any Limit Check element passes in all test iterations.

1-30

• Any Limit Check element passes in any test iteration.

You can configure this behavior within the test’s Properties pane. Click the test name in the Test Browser (named Untitled by default) to open the test’s properties and look for the section labe led This Test Passes If.

Using the signal test example that you constructed in this section, set the test to pass if all Limit Check elements pass in all test iterations.

Example: Bui lding a Test

Saving Te

You can save the results from the iterations of your test in a MAT-file. You must explicitly specify which test variables to save as test results.

Note Test variables that are not s aved as a test result will be lost at the end of the test execution.

The SystemTest software lets you save results at the end of each iteration. Before running your test, select the Save Results section in your test and specify w hich test variables to save as test results. Click the New Mapping button and then select from the drop-down list the name of the test variable to map to a result. You can optionally specify a name for the results that you want to save. By default, the name of the saved result is the same as the test variable. The following figure shows the mapping of test variables to test results.

st Results

1-31

1 Getting Started

After specifying which test variables to save as test results, specify the name of the MAT-file to use. Using this MAT-file you can reload the test results into the base workspace. By default for a test that is untitled, the SystemTest software names the file folderastheTEST-File. TochangethenameorlocationoftheMAT-file, click the test name in the Test Browser,thenclicktheOutput Files tab in the Properties pan e .

Untitled_results.mat and puts the file in the same

1-32

Use the Select File Names sectiontonametheresultsfile.UsetheOutput Folder Numbering section to select overwrite behavior. By default, each time you run the test you overwrite this file unless you select the Create new folder for each run option. I f you sele ct that option, a ne w folder is created

for each run and a new results file is created and put inside the folder. In this case, the test run. For example, if the folder name is create

MyResults\Outputs(1),followedbyMyResults\Outputs(2),etc.

Outputs subfolder name is appended by successive numbers for each

MyResults, the first test run will

MyResults\Outputs and the next test run results folder will be called

Example: Bui lding a Test

Use the Select Output Folder sectiontodesignatethelocationtosavethe results file, as follows:

• Same folder as TEST-file – This default setting saves any output files to

the same location as the TEST-F ile. In this case your test and any output files it generates will be in the same location. You set this location when you select File > Save to save your test, or if prompted to save when you close the SystemTest software.

• Same folder as the MATLAB current folder (PWD) – Save any output

files to the current working directory in MATLAB. You can see the current working directory when you open SystemTest from MATLAB.

• Browse – Select the third option and then click the Browse button to

choose an absolute directory location for the output files. This location should be stable and not read-only.

1-33

1 Getting Started

Note that the location you select here is also where the Test Report will be saved if you generate one by selecting the Generate report check box.

Note If a file or folder location is read-only, you will get an error when the test runs. For results files and Test Reports to be generated, the files and folder locations must be writable.

Generating a Test Report

When you run your test, the test status appears in the Run Status pane. This display contains basic information about your test:

• Time elapsed since your test started running.

• Which section your test is in.

• How many test iterations have passed or failed as defined by any limit

checks.

1-34

• Whether your test completed successfully.

• Any errors that cause your test to stop.

You can generate an d save mo re detail about the running test by enabling the Test R eport, which is a test execution log file in HTML format. This report is useful when you use limit checks in your test and you want to see specific test iterations that passed or failed. For example, instead of just finding that a test iteration failed, the report helps you determine how far a test variable varied from the upper or lower limit you defined in a Limit Check element. It also displays any plots that were generated. This report is also useful for documenting and sharing your test results.

Example: Bui lding a Test

To enable the Test Report:

1 SelectthetestnameintheTest Browser,thenclicktheOutput Files

tab on the Properties pane.

2 In the Select File Names section, select the Generate report check box.

3 Use the default name or type a new name in the edit field next to the check

box.

4 Use the Select Output Folder sectiontodesignatethelocationtosave

the Test Report:

• Same folder as TEST-file – This default setting saves any output

filestothesamelocationastheTEST-File. Inthiscaseyourtestand any output files it generates will be in the same location. You set this location wh en you select File > Save to save your test, or if prompted to save when you close the SystemTest software.

• Same folder as the M ATL AB current folder (PWD) –Saveany

output files to the current working directory in MATLAB. You can see the current working directory when youopenSystemTestfromMATLAB.

• Browse – Select the third option a nd then click the Browse button to

choose an absolute directory location for the output files. This location should be stable and not read-only.

The Test Report is stored in an

Outputs subfolder in this folder, along with

all dependent files, such as plot or Simulink model snapshots. The overwrite options y ou set for your test results MAT-file also apply to the file name and folder of your report file. To learn how to change these options, see “Saving Test Results” on page 1-31 .

Note that the locati o n you select here is also where the test results will be saved.

Note If a file or folder location is read-only, you will get an error when the test runs. For results files and Test Reports to be generated, the files and folder locations must be writable.

1-35

1 Getting Started

The Test Report contains the following information about the test run, organized by iteration in the report:

• The test description, if you entered one in the Description field of the

Properties pane of the test.

• A test summary, including start and stop times, number of iterations

completed, number of iterations that passed and failed, and final status of the test.

• Pass/fail results of Limit Check elements, by iteration.

• Values for any saved results you captured by setting up mappings in Saved

Results,byiteration.

• Test vector values, by i te ration.

• A snapshot of your model if you use a Simulink element in the test.

• AsnapshotofyourplotifyouuseaVector Plot, Scalar Plot, or General Plot

element in y our test, by iteration.

• A summary of generated files, with links to them. These can include a

Simulink model coverage report and test results.

1-36

Note Because the Test Report generates while the test is running, this option results in the test taking longer to execute.

To see what information the report generates, see “Viewing the Test Report” on page 1-40 .

Saving Your Test

You can save tests so that you can reu se them later. For example, to save the signal test:

1 Select File > Save As to open the Save file as dialog box.

2 Select a directory location and enter mySavedTest in the File name field.

3 Click Save.

Example: Bui lding a Test

The SystemTest software saves the test as mySavedTest.test and renames your test as it appears in the Test Browser. This does not rename the test results MAT-file or the Test Report file. Their names are controlled separately from the name of the test, as explained in “Saving T est Results” on page 1-31.

Running Your Test

After you build a test, you are ready to run it. At run time, the SystemTest softwareassignsvaluestotestvectors and test variables in the order they appear in the Test Vectors and Test Variables panes. Each test section runs elements in the order that they appear in the Test Browser.

To execute your test, do one of the following:

• Click the Run button.

• Select Run > Run.

• Press the F5 key.

Note While a test is running, you can stop its execution by pressing Ctrl+C

or clicking the Stop button on the toolbar.

Tracking Output

While the test runs, the Run Status pane shows summary test output, including start and stop times, number of iterations completed, number of iterations that passed and failed, and final status of the test. It will also display any error messages if the test has an error.

1-37

1 Getting Started

1-38

Example: Bui lding a Test

If your test includes a Plot element, the SystemTest software creates the plot and updates the plot during each iteration. Since Limit Check elements evaluate whether an iteration passed or failed, they directly affect the data that appears in the Test Report and the Run Status pa ne.

In the example test, the plot includes the high and low limits defined in the Limit Check element, to show w hich test iterations exceed the limits.

When the test is done running, the Run Status pane provides links to generated output. The Generated Files section contains a summary of

1-39

1 Getting Started

generated files, with links to them. These can include the Test Report; the test results, opened in the Test Results Viewer; a nd a Simulink model coverage report, if your test uses the model cove rag e feature.

Analyzing Your Test Results

After the SystemTest software runs your test, you can explore the results that are generated. This section shows how to:

• View and interpret the Test Report.

• Inspect your test results.

Viewing the Test Report

When you enable the Test Report, the SystemTest software saves information abouteachtestiterationinanHTMLfile. ToenabletheTestReport,check the Generate report option on the Output Files tab of the Properties pane before running your test. The report contains summary information about the test run, snapshots of any plots you used, snapshots of any models you used, pass/fail results of Limit Check elemen ts, and other in formation. See Test Report for a full description of what the report contains.

1-40

Example: Bui lding a Test

After a test runs, you can see the contents of this file by clicking Tools > Test Report or using the Test Report link in the Run Status pane. The

generated output resembles the following.

1-41

1 Getting Started

The M ain Test section of the report shows each iteration. Y ou see the value of the test vector in evaluating whether the test passed. For the first several iterations, the value of You can also see this in the scalar plot drawn w h ile the test ran. For other iterations that failed, you can scroll through the report to find the values of

signal and determine the values the Limit C heck element used

Y did not exceed either the high or low limits so the ite rations passed.

Viewing Test Results

For more information, see Chapter 11, “Accessing Test Results from the MATLAB Command Line”.

To continue the example of Simple Demo, after you run the test, return to MATLAB and type returned.

stresults. The following summary of the results is

stresults.Itprovides

1-42

Example: Bui lding a Test

You can see that the test contains 60 iterations, that it contains a test vector

signal, and the names of the three saved results are HiLimit, LowLimit,

and

To see a more complete list of properties, type:

get(stresults)

That displays the following.

The ResultsDataSet property contains the test results data in the form of a

dataset array. This is what you set up using the Saved Results node in the

Test Browser. See “Saving Test Results” on page 1-31 for more information

on setting up saved results.

To access the

stresults.ResultsDataSet

ResultsDataSet property, type :

This returns the test results data in the form of a dataset array.

In the Simple Demo example, a portion of the test results data looks like this:

1-43

1 Getting Started

In the dataset array, each row represents a test iteration, labeled using the convention of iterations. Test vector values are listed first, in alphabetical order, followed by test results, listed in alphabetical order, as shown in the above figure. This is a simple way to view the results you set up in Saved Results.Thetest results for all iterations are displayed at the command line, even though only the first ten are only shown here.

['I' + Iteration_Number]. Thisexampleshowsthefirst10

1-44

You can now plot the results. See “Plotting Results Data” on page 11-10 to see plots created from these results.

Working with Test Vectors

• “Creating MATLAB Express ion Test Vectors” on page 2-2

• “Creating Grouped Test Vectors” on page 2-5

• “About Test Vectors and the MATLAB Workspace” on page 2-12

• “Creating MAT-File Test Vectors” on page 2-13

• “Creating Randomized Test Vectors with Probability Di s trib u t ion s” on

page 2-18

• “Creating Spreadsheet D ata Test Vectors” on page 2-44

• “Creating Simulink Design Verifier Data File Test Vectors” on page 2-53

• “Creating Signal Builder Block Test Vectors” on page 2-67

• “Creating a Test Case Data Test Vector” on page 2-73

• “Using a MATLAB Element to Access Test Case Data Test Vector

Information” on page 2-76

• “Editing a Test Vector from within an Element” on page 2-77

2 Working with Test Vectors

Creating MATLAB Expression Test Vectors

Test v ectors define the parameter space or set of test cases you want to run. Test vectors are composed of values that can be d eriv ed from a MATLAB expression. You can use any M A TLAB expression that evaluates to a 1-by-N matrix or cell array to define your test vector. You must have at least one test vector defined to run a test.

The total number of Main Test iterations is determined by permuting all test vector values. For ex ample, if one test vector is a 1-by-3 array and another is 1-by-2, it wou ld result in a total of six iterations covering all the test vector value combinations.

To add a test vector:

1 Click the New Vector button in the Test Vectors pane.

In the Insert New Test Vector dialog box, keep the default test vector type of MATLAB Expression.

2-2

Creating MATLAB Expression Test Vectors

2 Assign

3 EnterthevaluebytypinginvaluesoraMATLABexpressioninthe

anametothevectorintheName field.

Expression field.

The Size field fills in automatically based on what you entered if you press Enter or click outside of the Size field. For example, if you entered

:10

in the Expression field, the Size would be a 1 x 10 double, which

1: 1

means 10 iterations.

ct the Evaluate Test Vector each time the test is run option if

4 Sele

want to use new values every time the test is run. For example, if

you

r expression included a

you

ld be calculated each time. Leave it unselected if you want to use the

wou

e values each time the test is run.

sam

rand function, a new set of random numbers

2-3

2 Working with Test Vectors

5 Click OK in the Insert Test Vector dialog box.

The new vector appears in the Test Vectors pane.

Note Grouping test vectors determines how they will be iterated through when the test runs. For information on grouping vectors, see “Creating Grouped Test V ectors” on page 2-5.

For an example o f creating test vectors in a test, see “Creating a Test Vector” on page 1-15.

2-4

Creating Grouped Test Vectors

When you create a test vector, it is an ungrouped vector by default, except for Probability Distribution test vectors. You can also create grouped vectors, in order to affect the way iterations are run. By grouping test vectors, they will be indexed simultaneously with the other vectors in their group. Each set of grouped values are then permuted with all the ungrouped test vectors. This gives more contro l over the flow of tests and is useful for Design of Experiments (DOE) or Monte Carlo-based testing as well as defining signal groups, similar to those defined in the Simulink Signal Bui lder block.

For example, if you are testing a throttle body controller, you may want to sweep across a range of input level or gain values, while simultaneously selecting different throttle body types, each defined by their mass and damping characteristics.

An example of the vectors in this scenario could look like this:

gain = [1 10 100] mass = [a b c d] damping = [w x y z]

Creating Grouped Test Vectors

If the gain vector is ungrouped, and the mass and damping vectors are grouped, it will result in each value of

Run 1: (1, a, w) Run 2: (1, b, x) Run 3: (1, c, y) Run 4: (1, d, z) Run 5: (10, a, w) Run 6: (10, b, x) Run 7: (10, c, y) Run 8: (10, d, z) Run 9: (100, a, w) Run 10: (100, b, x) Run 11: (100, c, y) Run 12: (100, d, z)

gain. The test runs would look like this:

mass and damping being indexed simultaneously for

2-5

2 Working with Test Vectors

Note Grouped test vectors must be the same length.

You create a grouped test vector as you do any other vector, by clicking the New Vector button in the Test Vectors pane. To group a vector, change the selection using the Grouping tab in the Insert Test Vector dialog bo x. Y ou can group any type of test vector, and you can create multiple test vector groups. You can also group or ungroup test vectors after you create them.

In general, it doesn’t usually make sense to group Signal Builder Block test vectors or Simulink Design Verifier Data File test vectors. There are advantages to grouping MATLAB Expression, Probability Distribution, and Spreadsheet Data test vectors at times, depending on your test goals. One ofthemainadvantagestogroupingisforMonteCarlo-basedtesting,as described by the example above.

To group a test vector:

2-6

1 Create a test v ector and configure it in the General tab of the Insert Test

Vector dialog box.

2 Click the Grouping tab in the Insert Test Vector dialog box.

3 Select the Assign test vector to a group option.

A group is created and given the default name of

Group1, as shown here.

Creating Grouped Test Vectors

4 To chan

ge the name, type the new name over the default name and press

Enter.

5 Click OK in the Insert Test Vector dialog box.

In the Test Vectors pane, the name of the group is displayed in the table.

6 Now i

7 Select the test vector type and configure it in the General tab.

8 Click the Grouping tab, and select the Assign test vector to a group

f you create another test vector, you can add it to the same group as

irst one. To do this, click the New Vector button a gain.

the f

option.

te that test vectors in a group must all be the same length.

2-7

2 Working with Test Vectors

If you already have one test vector group, the new vector is placed in that group by default.

2-8

k OK in the Insert Test Vec tor dialog box.

9 Clic

Creating Grouped Test Vectors

You can create multiple test vector groups. Onceyouhavemultiplegroups, when you create new test vectors, you can select which group to put them in as you create them. The following figure shows

Group1 containing TestVector1

and TestVect or2,andGroup2 containing TestVector3 and TestVector4.

2-9

2 Working with Test Vectors

2-10

Creating Grouped Test Vectors

You can also create groups after test vectors are already created by editing a test vector in the Test Vectors pane. Select a test vector in the table to edit its properties in the editor area below the table. There you can add it to a group using the Grouping tab. You can also add it to a group in the table by clicking in the Group Name column.

Managing Test Vector Groups

You can modify g roups to ungroup a test vector, move a test vector to another group, rename a group, or d elete a group.

• Ungroup a test vector — To remove a test vector from a group, select

it in the test vectors table, then click the Group Name column. Use the down-arrow to select the first entry, which is a blank space. The Group Name co lumn will then be empty for that test vector, indicating it is no longer in a group.

• Move a test vector to another group — To move a test vector from o ne

group to another, select it in the test vectors table, then click the Group

Name column. Use the down arrow to select the group to move it to. The Group Name column will then show the new group name.

• Rename a group — You can change the name of a tes t vector group either

in the table or in the editor area. Renaming a group in the table results in the group name for a single test vector being changed. Renaming a group in the editor area results in the name being changed for all vectors in the group.

To rename a group for a single test vector, select that vector in the table, then click in the Group Name column. Typeanewnameandpress Enter.

To rename a group for a ll test vectors in the group, select one of the test vectors in the table. Then in the Grouping tab in the editor area, select that group name in the upper section and type a new name. Press Enter. You then see all of the test vectors in that group change to the new name in the table.

• Delete a group — To delete a test vector group, select one of the test

vectors in the table that is in that group. Then in the editor area, under the Grouping tab, that group name will be selected. Click the Delete button on the Grouping tab. The group is deleted and all test vectors belonging to that group become ungrouped.

2-11

2 Working with Test Vectors

About Test Vectors and the MATLAB Workspace

2-12

Creating MAT-File Test Vectors

The MAT-File test vector offers an easy way for you to use data from a MAT-file in the SystemTest software.

To add a test vector:

1 Click the New Vector button in the Test Vectors pane.

In the Insert New Test Vector dialog box, select the test vector type of MAT-File.

Creating MAT-File Test Vectors

2-13

2 Working with Test Vectors

2-14

The red border indicating that the element is in an error state is normal, andwillgoawayonceyouaddfile(s)instep4.

gnanametothevectorintheName field.

2 Assi

3 Click the Add File(s) button.

4 In the Select MAT-File dialog box, browse for your MAT-file(s). You can

select multiple files at the same time. Only MAT-files can be added. Other file types produce an error. After selecting the file(s), click the Open button to bring them into the test vector.

Creating MAT-File Test Vectors

In the MAT-Files to read table on the General tab, MAT-files that are checked will be used in the test. Unchecking a file means it will not be included in the test.

5 Click the Variables tab. All the common variables contained in all the

selected MAT-file(s) you added appear in the table.

Note that the variables are sorted in alphabetical order. If you have multiple MAT-files, only variables that are common across all files appear in the table.

ables that are checked will be used in the test. Unchecking a variable

Vari

ns it will not be included in the test. In the example above, all variables

mea exc

ept for

ans will be used in the test.

2-15

2 Working with Test Vectors

Checking or unchecking the checkbox in the table header will select or unselect all variables. It is a Select All/Unselect All toggle option.

6 MAT-File test vectors are ungrouped by default. On the Grouping tab,

you can select the Assign test vector to a group option if you want to group the test vector.

Grouping test vectors is useful for reducing the number of iterations to execute. It means that the SystemTest software will sequentially combine values for all grouped test vectors, instead of permuting their values. See “Creating Grouped T est Vectors” on page 2-5 for more information on grouped test vectors.

7 Click OK in the Insert T est Vector dialog box.

The new vector appears in the Test Vectors pane.

Important Usage Notes

• If you use multiple MAT-Files in a test vector, only commonly named

variables included in all of the files will be read and used. For example, if you have variables in MAT-file A called and in MAT-file B you have variables only

Speed and Pressure will be shown since they are included in both

MAT-Files.

Distance and Torque will not be used since they do not exist

Speed, Distance,andPressure,

Speed, Pressure,andTorque,

in both files.

2-16

• If the order of execution of the MAT-files is important, then use the up and

down arrows to order the files accordingly in the test vector table. Each MAT-file is one iteration of the test vector, and they are executed in the order they appear in the table.

• The test vector is evaluated every time the test is run – that means the

data is read from the MAT-File(s) every time the test is run.

• If a MAT-File test vector is mapped to the inport blocks in a Simulink

element using the All Inport blocks are mapped option, the model is simulated using all the variables that are selected in the Variables table in the test vector. If it is mapped to the inport blocks using the Individual Inport blocks are mapped option, the model is simulated with individually selected variables from the M AT-file.

Creating MAT-File Test Vectors

• Checking or unchecking the checkbox in the Variables table header will

select or unselect a ll variables. It is a Select All/Unselect All toggle option. This option affects the variables selection behavior when you add or remove or select or unselect MAT-files in the MA T -file list on the General tab.

For example, if the checkbox is selected (to Select All variables) and then a MAT-file is added/removed or selected/unselected, all common variables will be selected by default.

If the checkbox is unselected (to Unselect All variables) and then a MAT-file is added/removed or selected/unselected, all common variables will be unselected by default.

2-17

2 Working with Test Vectors

Creating Randomized Test Vectors with Probability Distributions

In this section...

“Using Probability Distributions in Test Vectors” on page 2-18

“Creating a Test Vector with Probability Distributions” on page 2-18

“Viewing Data While Configuring the Test Vector” on page 2-23

“The Probability Distributions” on page 2-26

“Example: Creating Test Vectors with Probability D istributions” on page 2-34

Using Pro

The Syste vector v set up te

If you h

ates with it to provide use of some of its probability distribution

integr

ons, such as exponential, gamma, lognormal, T (Student’s t), and

functi Weibul MATLA

Crea

You c To us

l. If you do not have the S tatistics Toolbox software, you can use the

ting a Test Vector with Probability Distributions

an use a probability distribution when you create or edit a test vector. e a probability distribution:

he Test Vectors pane, click the New Vector button.

1 In t

2 In the Insert Test Vector dialog box, select Probability Distribution as

the test vector type.

3 EnteranameforthenewvectorintheName field.

lect a distribution function from the Distribution list.

4 Se

bability Distributions in Test Vectors

mTest software provides an easy way to generate randomized test

alues for your test. You can use probability distribution functions to

st vectors, which i s useful for performing Monte Carlo analyses.

ave the Statistics Toolbox™ software, the SystemTest software

B probability distribution functions normal (Gaussian) and uniform.

2-18

Creating Randomized Test Vectors with Probability Distributions

If you have the Statistics Toolbox s oftware, all of the functions shown in the figure appear in the list. If you do not have this toolbox, you can use normal (Gaussian) and uniform.

For inf Distri

5 Once you select a distribution, the relevant options appear. Fill in the

ormation on the distribution functions,see“TheProbability

butions” on page 2-26.

parameters for your distribution.

2-19

2 Working with Test Vectors

For example, normal (Gaussian) allows you to set Mean and Standard deviation.

2-20

6 After setting the relevant probability parameters, type in the Number of

values you want to use. That is the number of values you would like to

generate for the test vector.

The Number of values must be a positive integer. It must also be the same value for all of y our probability distributions because the vector is grouped.

7 If you want to see the data you have configured before running the test,

click the View Data button. This displays a histogram visualization of the probability distribution data. If you are not satisfied with the data as it is configured, you can adjust one or more of the parameters and hit Enter to see the ch a n ges in the figure window.

For more information on viewing the data, see “Viewing Data While Configuring the Test Vector” on page 2-23.

8 Select the Evaluate Test Vector each time the test is run option if you

want to use new values every time the test is run. For example, for the

Creating Randomized Test Vectors with Probability Distributions

probability distribution, a new set of values for the parameters (such as Mean) would b e calculated each time. Le av e it unselected if you want to use the same values each time the test is run.

If you are doing Monte Carlo testing and you want repeatability of the data,donotusethisoption.

9 On the Grouping tab, keep the default of Grouped,orselectUngrouped.

Randomized test vectors with probability distributions are grouped by default, as indicated by Grouped being selected.

Grouping test vectors is useful for reducing the number of iterations to execute. It means that the SystemTest software will sequentially combine values for all grouped test vectors, instead of permuting their values. In thecaseofrandomizedtestvectors,grouping avoids introducing additional variation into your test. See Creating Grouped Test Vectors for more information on grouped test vectors.

10 Click OK in the Insert Test Vector dialog box.

The new vector then appears in the Test Vectors pane.

2-21

2 Working with Test Vectors

2-22

Creating Randomized Test Vectors with Probability Distributions

Viewing Data While Configuring the Test Vector

You can view your probability distribution data while configuring the test vector, without having to run the test. You can quickly inspect the test vector data for outliers, data range coverage, or correctness of the test function before running the test. This allows you to make necessary adjustments until you have data you are satisfied with, which saves time.

To view data while configuring a test vector:

1 Create the test vector by clicking the New button in the Test Vectors pane.

2 In the Insert Test Vector dialog box, select Probability Distribution as

the test vector type.

3 Select a distribution function from the Distribution list.

4 Once you select a distribution, the relevant parameters appear. Fill in the

parameters for your distribution.

In this example, Normal (Gaussian) is shown, with a mean of 1.0, standard deviation of 3, and 40 values.

ick the View Data button on the General tab.

5 Cl

2-23

2 Working with Test Vectors

6 The data viewer window displays the data you configured in a histogram

visualization. The values are displayed on the x-axis, and in this case they range from approximately

-6 to 9 . The parameters are also displayed

textually in the figure window in the upper right corner. For comparison purposes, a light orange line showing the “ideal” probability distribution is also displayed on top of your data.

2-24

7 When

8 If you are dissatisfied with the data, change one or mo re parameters and

satisfied with the data that is shown, click OK to finish creating

test vector.

the

redisplay it. In this case, change the standard deviation from 3 to 2. To change a value, type a new value in the parameter you want to change

Creating Randomized Test Vectors with Probability Distributions

and either press Enter or click outside of the field. The figure window automatically updates to display the new data.

You can also view and modify the test vector data any time after creating a test vector. Access the data viewer by clicking the Test Vectors tab in the Properties pane, then selecting a test vector from the list. That test vector then becomes editable, and you can click the View Data button on the General tab.

2-25

2 Working with Test Vectors

The Probability

If you have the St integrates with functions, suc Weibull. If you to the MATLAB p uniform.

The SystemTe following se the Insert Te

The Insert T pick in the l the functi want to gen

ctions. Select the Probability Distribution test vector type in

ist, as shown in the sections below. In each case, enter values for

on-specific parameters, and then enter the Number of values you

erate for the test vector.

Normal (Gaussian)

The norma paramete is often

l distribution is a two-parameter family of curves. The first

r is the mean. The second parameter is standard deviation. Normal

used for data that is symmetrical about the mean.

atistics Toolbox software, the SystemTest software

it to provide use of some of its probability distribution

h as exponential, gamma, lognormal, T (Student’s t), and

do not have the Statistics Toolbox software, you have access

robability distribution functions normal (Gaussian) and

st software supports the distribution functions shown in the

st Vector dialog box to access the functions.

est Vectors dialog box shows fields specific to the distribution you

Distributions

2-26

Creating Randomized Test Vectors with Probability Distributions

Normal uses the function randn and takes parameters for Mean and Standard deviation. The SystemTest software us es the following

calculation for norm al:

mean + St

For more information, see randn in the MATLAB documentation.

Unifor

The uniform distribution (also called rectangular) has a constant probability density function between its two parameters, the minimum and the maximum.

The uniform distribution is appropriate for representing the distribution of round-off errors in values tabulated to a particular number of decimal places.

d_Dev * randn(1, #values)

2-27

2 Working with Test Vectors

2-28

Uniform u value an calcula

min + (max-min) * rand(1, #values)

For mor

ses the function

d Maximum value. The SystemTest software uses the follow ing

tion for uniform:

e informatio n, see

rand and takes parameters for Minimum

rand in the MATLAB documentation.

Exponential

The ex expon that

Expo happ tim sen

ponential distribution is a special case of the gamma distribution. The

ential distribution is special because of its utility in modeling eve nts

occurrandomlyovertime.

nential is often used to model the time between independent events that

en at a constant average rate. For example, y ou could use it for the e it takes a radioactive particle decays, or the time between messages toveranetwork.

Creating Randomized Test Vectors with Probability Distributions

Exponent

For more documen

ial uses the function

exprnd and takes o ne parameter for Mean.

information, see Exponential Distribution in the Statistics Toolbox

tation.

2-29

2 Working with Test Vectors

Gamma

The gamma distribution models sums of exponentially distributed random variables.

2-30

Gamma uses the function gamrnd and takes parameters for A and B.

For more information, see Gamma Distribution in the Statistics Toolbox documentation.

Creating Randomized Test Vectors with Probability Distributions

Lognormal

The normal and lognormal distributions are close ly related. The lognormal distribution is applicable when the quantity of interest must be positive, since log(X) exists only when X is positive .

Lognormal can be used to model something that can be thought of as the multiplicative product of many small independent factors. A common example is the long-term return rate on a stock investment, because it can be considered as the product of daily return rates.

rmal uses the

Logno

ard deviation.

Stand

ore information, see Lognormal Distribution in the Statis tics Toolbox

For m

mentation.

docu

lognrnd function and takes parameters for Mean and

2-31

2 Working with Test Vectors

The T (Student’s t) distribution is a family of curves that depend on a single parameter v (the degrees of freedom). As v goes to infinity, the T distribution approaches the standard normal d istribution.

T is often used to estimate properties when the sample size is small.

2-32

Tusesthetrnd function and takes one parameter for Degrees of freedom.

For more information, see Student’s t Distribution in the Statistics Toolbox documentation.

Creating Randomized Test Vectors with Probability Distributions

Weibull

The Weibull distribution is an appropriate analytical tool for modeling the breaking strength of materials. Current usage also includes reliability and lifetime modeling. The Weibull distribution is m ore flexible than the exponential distribution for these p urposes.

Weibull uses the function wblrnd and takes parameters for A and B.

For more information, see Weibull Distribution in the Statistics Toolbox documentation.

2-33

2 Working with Test Vectors

Example: Creati

ng Test Vectors with Probability

Distributions

Many models must model paramete in electric mo tester define as single fixe running the mo to account fo

In this case, The test var and shaft in

To create t

1 In the Test

2 In the Insert Test Vector dialog box, select Probability Distribution as

the test vector type.

3 Enter ArmatureResistance in the Name field.

4 In the In

(Gauss

s the model’s parameters as distributions of values, rather than

d values. The tester then performs a Monte Carlo simulation,

r variability in manufacturing.

the tester defines the uncertain motor parameters as test vectors. ies parameters for armature resistance, armature inductance, ertia.

he first vector, for a rmature resistance:

sert Test Vector dialog box, use the default distribution, normal

ian).

take into account the effect of evaluating uncertainty in

rs. In this example the tester needs to account for uncertainty

tor characteristics that come off the production line so the

del repeatedly with random combinations of parameter values

Vectors pane, click the New Vector button.

2-34

You do not need to have the Statistics Toolbox software installed to use normal (Gaussian) since it is included with MATLAB.

5 In the Mean field, enter 1.71.

6 In th

e Standard deviation field, enter

.056.

Creating Randomized Test Vectors with Probability Distributions

7 In the Number of values field, enter 1000.

For this vector, the test is varying armature resistance up to a standard deviation of .056, around a mean of 1.71, and using 1000 values.

8 Click the View Data button to see a visualization of the test vector

data that you configured. This displays a histogram visualization of the probability distribution data that will be used when the test is run. If you are not sati sf ied with the data as it is configured, you can adjust one or more of the parameters and hit Enter to see the changes in the figure window. In this case, we keep the data, as shown here.

2-35

2 Working with Test Vectors

2-36

For more information on viewing the data, see “Viewing Data While Configuring the Test Vector” on page 2-23.

9 Click

OK in the Insert Test Vector dialog box.

The new vector appears in the Test Vectors pane.

Creating Randomized Test Vectors with Probability Distributions

create the second vector, for armature inductance:

2-37

2 Working with Test Vectors

1 In the Test Vectors pa ne, click the New Vector button.

2 In the Insert Test Vector dialog box, select Probability Distribution as

the test vector type.

3 Enter ArmatureInductance in the Name field.

4 Use the default distribution, normal (Gaussian).

5 In the Mean field, enter .3.

6 In the Standard deviation field, enter .01.

2-38

Creating Randomized Test Vectors with Probability Distributions

7 In the Number of values field, enter 1000.

For this vector, the test is varying armature inductance up to a standard deviation of .01, around a mean of .3, and using 1000 values.

8 You can optionally click the View Data button to see a visualization of the

test vector data that you configured.

9 Click OK in the Insert T est Vector dialog box.

The new vector appears in the Test Vectors pane.

2-39

2 Working with Test Vectors

2-40

To create the third vector, for shaft inertia:

Creating Randomized Test Vectors with Probability Distributions

1 In the Test Vectors pa ne, click the New Vector button.

2 In the Insert Test Vector dialog box, select Probability Distribution as

the test vector type.

3 Enter ShaftInertia in the Name field.

4 Use the default distribution, normal (Gaussian).

5 In the Mean field, enter 44.5.

6 In the Standard deviation field, enter .443.

2-41

2 Working with Test Vectors

7 In the Number of values field, enter 1000.

2-42

For this vector, the test is varying shaft inertia up to a standard deviation of .443, around a mean of 44.5, and using 1000 values.

8 You can optionally click the View Data button to see a visualization of the

test vector data that you configured.

9 Click OK in the Insert T est Vector dialog box.

The new vector appears in the Test Vectors pane.

Mathworks SYSTEMTEST 2 user guide

Specifications and Main Features

Frequently Asked Questions

User Manual