Mathworks MODEL-BASED CALIBRATION TOOLBOX 4 CAGE user guide

Download

Model-Based Calib

Model Browser User’s Guide

ration Toolbox™ 4

How to Contact The MathWorks

www.mathworks. comp.soft-sys.matlab Newsgroup www.mathworks.com/contact_TS.html Technical Support

suggest@mathworks.com Product enhancement suggestions

bugs@mathwo doc@mathworks.com Documentation error reports service@mathworks.com Order status, license renewals, passcodes

info@mathwo

com

rks.com

Web

Bug reports

Sales, prici

ng, and general information

508-647-7000 (Phone)

508-647-7001 (Fax)

The MathWorks, Inc. 3 Apple Hill Drive Natick, MA 01760-2098

For contact information about worldwide offices, see the MathWorks Web site.

Model-Based Calibration Toolbox™ Model Browser User’s Guide

The software described in this document is furnished under a license agreement. The software may be used or copied only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written consent from The MathW orks, Inc.

FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern theuse,modification,reproduction,release,performance,display,anddisclosureoftheProgramand Documentation by the federal government (or other entity acquiring for or through the federal government) and shall supersede any conflicting contractual terms or conditions. If this License fails to meet the government’s needs or is inconsistent in any respect with federal procurement law, the government agrees to return the Program and Docu mentation, unused, to The MathWorks, Inc.

Trademarks

MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See

www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand

names may be trademarks or registered trademarks of their respective holders.

Patents

The MathWorks products are protected by one or more U.S. patents. Please see

www.mathworks.com/patents for more information.

Revision History

December 2001 Online only New for Version 1.0 (Release 12.1) August 2002 Online only Revised for Version 1.1 (Release 13) May 2003 Online only Revised for Version 2.0 (Release 13+) June 2004 Online only Revised for Version 2.1 (Release 14) June 2004 Online only Revised for Version 2.1.1 (Release 14+) November 2005 Online only Revised for Version 3.0 (Release 14SP3+) September 2006 Online only Version 3.1 (Release 2006b) March 2007 Online only Version 3.2 (Release 2007a) September 2007 Online only Revised for Version 3.3 (Release 2007b) March 2008 Online only Revised for Version 3.4 (Release 2008a) October 2008 Online only Revised for Version 3.4.1 (Release 2008a+) October 2008 Online only Revised for Version 3.5 (Release 2008b) March 2009 Online only Revised for Version 3.6 (Release 2009a) September 2009 Online only Revised for Version 3.7 (Release 2009b) March 2010 Online only Revised for Version 4.0 (Release 2010a)

Workflows For Modeling

What Is the Model Browser? ........................ 1-2

Contents

How to Use This Manual

Overview of Modeling with the Model-Based Calibration

Toolbox Product

How to Set Up a One-Stage Model

How to Set Up a Two-Stage Model

How to Set Up a Point-by-Point Model

Point-by-Point Modeling Pr ocess Use Cases for Point-by-Point Models Analyzing Point-by -P oint Models Exporting Point-by-Point Models to CAGE

Creating Alternative Models to Compare

................................. 1-5

........................... 1-3

................... 1-7

................... 1-9

............... 1-11

..................... 1-11

.................. 1-13

..................... 1-14

............. 1-14

............ 1-17

Projects and Test Plans

Project Level: Startup View ........................ 2-2

Project View All Models Pane Data Sets Pane Notes Pane Test Plans List Pane Project Level: Toolbar

...................................... 2-2

................................... 2-3

....................................... 2-4

............................... 2-5

.............................. 2-7

Project Lev el: Menus .............................. 2-8

Model Tree

Navigating the Model Tree Tree Structure Icons: Curves, Worlds, and Houses Icons: Blue Backgrounds and Purple Worlds

Test Plans

Creating a New Test Plan Creating New Test Plan Templates

Test Plan Level

Test Plan View Test Plan Level: Toolbar and Menus Designing Experiments

........................................ 2-11

.......................... 2-11

.................................... 2-12

................... 2-13

........... 2-14

......................................... 2-17

........................... 2-17

................... 2-18

.................................... 2-21

................................... 2-21

.................. 2-25

............................. 2-27

The Design Editor ................................. 3-2

Introducing the Design Editor Opening the Design Editor Design Styles Design Editor Displays Design Editor Toolbar and Menus

..................................... 3-4

............................. 3-5

....................... 3-2

.......................... 3-2

.................... 3-9

Designs

vi Contents

Creating a Classical Design

Creating a Space-Filling Design

Introducing Space-Filling Designs Setting Up a Space-Filling Design Halton Sequence Sobol Sequence Latin Hypercube Sampling Lattice Stratified Latin Hypercube Augmenting Space-Filling Designs

.......................................... 3-24

.................................. 3-21

................................... 3-22

......................... 3-16

.......................... 3-23

.......................... 3-25

..................... 3-19

.................... 3-19

.................... 3-20

................... 3-26

Creating an Optimal Design ........................ 3-31

Introducing Optimal Designs Optimal Design: Initial Design Tab Optimal Design: Candidate Set Tab Optimal Design: Algorithm Tab Averaging Optimality Across Multiple Models

........................ 3-31

................... 3-33

.................. 3-35

...................... 3-38

.......... 3-40

Adding and Editing Design Points

Adding Design Points Editing Design Points

Merging Designs

Fixing, Deleting, and Sorting Design Points

Exporting and Importing Designs

Applying Constraints

How to Apply Constraints Constraint Types Importing Constraints

Prediction Error Variance Viewer

Introducing the Prediction Error Variance Viewer Display Options Prediction Error Variance Prediction Erro r Variance for Two-Stage Models

Design Evaluation Tool

Introducing the Design Evaluation Tool Table Options Design Matrix Full FX Matrix Model Terms Z2 Matrix Alias Matrix Z2.1 Matrix Regression Matrix Coefficient I nformatio n Standard Error Hat Matrix

.................................... 3-73

..................................... 3-74

........................................ 3-74

...................................... 3-74

...................................... 3-75

....................................... 3-78

.............................. 3-42

.............................. 3-44

................................... 3-45

.............................. 3-51

.......................... 3-51

.................................. 3-54

............................. 3-60

................................... 3-65

........................... 3-66

............................ 3-71

................................... 3-74

................................. 3-75

............................. 3-75

................................... 3-77

.................. 3-42

................... 3-50

.................. 3-63

............... 3-71

.......... 3-47

....... 3-63

........ 3-68

vii

|X’X| ........................................... 3-78

Raw R esidual Statistic Degrees of Freedom Table Design Evaluation Graphical Displays Export of Design Evaluation Information

............................. 3-79

.......................... 3-79

................ 3-80

.............. 3-81

Using Data ........................................ 4-2

Data

The Data Editor

Introducing the Data Editor Data Editor Views Data Editor Toolbar and Menus

Data Loading and M e rging

Introducing Data Loading Loading Data from File Loading Data from the Workspace Tailor-Made Excel Sheets

Creating Variables

How to Create Variables New Variables

Creating Filters

HowtoCreateFilters Test Filters and Test Notes Filter Editor

Storage

........................................... 4-30

................................... 4-5

................................. 4-7

............................. 4-18

........................... 4-22

................................. 4-24

............................ 4-24

.................................... 4-24

................................... 4-27

.............................. 4-27

...................................... 4-28

......................... 4-5

...................... 4-14

......................... 4-18

.......................... 4-18

.................... 4-20

......................... 4-28

viii Contents

Test Groupings

Data Selection Wizard

Opening the Data Selection Wizard

.................................... 4-34

............................. 4-37

................... 4-37

Step 1: Select Data Set ............................. 4-37

Step 2: Select Input Signals Step 3: Select Response Models Step 4: Set Tolerances

......................... 4-38

...................... 4-40

............................. 4-41

Matching Data to Designs

Introducing the Cluster Plot View HowtoUsetheClusterPlotView The Tolerance Editor What Will Happen to My Data and Design?

Data Loading Application Programming Interface

.......................... 4-44

.................... 4-44

.................... 4-46

.............................. 4-49

............ 4-50

Setting Up Models

What Models Are A vailable? ........................ 5-2

Input Factor Setup

Local Model Setup

Introducing Local Model Setup Local Model Class: Polynomials and Polynomial Splines Local Model Class: Truncated Power Series Local Model Class: Free Knot Spline Local Model Class: Growth Models Local Model Class: Linear Models Local Model Class: Average Fit Local Model Class: Multiple Models Local Model Class: User-Defined Models Local Model Class: Transient Models Covariance Modeling Correlation Models Transforms

...................................... 5-46

................................ 5-4

................................. 5-6

...................... 5-6

............ 5-11

.................. 5-13

................... 5-14

.................... 5-21

...................... 5-22

.................. 5-24

.............. 5-28

................. 5-37

............................... 5-44

................................ 5-46

.... 4-52

.. 5-7

Boundary Model Setup

Introducing Boundary Models Creating Boundary Models

............................. 5-48

....................... 5-48

.......................... 5-49

Combining Best Boundary Models .................... 5-56

Boundary Model Toolbar Boundary Model Menus Boundary Model Fit Options

........................... 5-59

............................ 5-61

........................ 5-63

Global Model Setup

Introducing Global Model Setup Global Linear Models: Polynomials and Hybrid Splines Global Mode l Class: Radial Basis Function Global Model Class: Hybrid RBF Global Mode l Class: Interpolating RBF Global Model Class: Multiple Linear Models Global Mode l Class: Free Knot Spline Global Model Class: Ne ural Network Global Model Class: User-Defined and Transient

Models

New R esponse Models a nd Datum Models

Adding New Response Models Datum Models

Build Models Dialog Box

Introducing the Build Models Dialog Box New Template User-Defined Template Polynomial Template RBF Template Hybrid RBF Template Free Knot Spline Template Model Browser Template Parallel Model Building

........................................ 5-87

................................ 5-67

..................... 5-67

............ 5-74

..................... 5-79

................ 5-80

........... 5-81

................. 5-83

................. 5-85

............ 5-88

....................... 5-88

.................................... 5-89

........................... 5-91

.............. 5-91

.................................... 5-92

............................. 5-93

.............................. 5-93

.................................... 5-94

.............................. 5-94

.......................... 5-95

........................... 5-95

............................ 5-96

.. 5-67

x Contents

Selecting Models

Local Level ........................................ 6-2

What Is Local L ev el? Local Special Plots Local Scatter Plots

............................... 6-2

................................ 6-4

................................ 6-5

Response Features List ............................. 6-6

Diagnostic Statistics Pane Pooled Statistics Test Notes Pane Data Tab Using the RMSE Explorer with Local Models Local Level: Toolbar Local Level: Menus Outlier Selection Criteria

........................................ 6-10

.................................. 6-8

.................................. 6-10

................................ 6-13

.......................... 6-7

........... 6-10

............................... 6-12

........................... 6-20

Global Level

What Is Global Level? Global Special Plots Global Scatter Plots Removed Data Pane Models List Global Level: Toolbar Global Level: Model-Specific Tools Global Level: Menus

Selecting Models

Select Button Model Selection Guide Summary Statistics Using Information Criteria to Compare Models Model Selection Window

MLE

Response Level

.............................................. 6-72

Calculating MLE MLE Settings

....................................... 6-24

.............................. 6-24

............................... 6-26

............................... 6-27

............................... 6-29

...................................... 6-30

.............................. 6-31

................... 6-32

............................... 6-35

.................................. 6-40

..................................... 6-40

............................. 6-42

................................ 6-47

............................ 6-52

.................................. 6-72

..................................... 6-74

.................................... 6-76

......... 6-50

Model E valuation Window

About the Model Evaluation Window Using Validation Data

Stepwise

What Is Stepwise? Using the Stepwise Regression Window Automatic Stepwise

.......................................... 6-84

................................. 6-84

.......................... 6-79

............................. 6-81

............................... 6-89

................. 6-79

............... 6-84

Stepwise in the Model Building Process ............... 6-90

PRESS statistic

................................... 6-93

Box-Cox Transformation

Two-Stage Models for Engines

Overview of the Mathematics of Two-Stage Models Local Models Local Covariance Modeling Response Features Global Models Two-Stage Models Global Mo de l Selection Initial Values for Covariances Quasi-Newton Algorithm Expectation Maximization Algorithm References Linear Regression Toolbox T erms and Statistics Definitions

Exporting Models

How to Export Models What Is Exported? Evaluating Models in the Workspace

..................................... 6-102

.................................... 6-104

................................. 6-105

....................................... 6-108

................................. 6-109

.................................. 6-113

................................. 6-118

........................... 6-96

...................... 6-100

.......................... 6-102

................................ 6-104

............................. 6-106

....................... 6-107

........................... 6-107

................. 6-107

.............. 6-110

.............................. 6-113

................. 6-119

...... 6-100

xii Contents

Radial Basis Functions

Guide to Radial Basis Functions for Model Building .. 7-2

Types of Radial Basis Func tions

HowtoChooseaKernel Gaussian Thin-Plate Spline Logistic Basis Function Wendland’s Compactly Supported Function Multiquadrics Reciprocal Multiquadrics Linear

........................................ 7-4

.................................... 7-9

........................................... 7-11

............................ 7-4

................................. 7-5

............................. 7-7

........................... 7-10

.................... 7-4

............ 7-7

Cubic ........................................... 7-12

Fitting Routines

Center Selection Algorithms

Rols

............................................. 7-14

RedErr WiggleCenters CenterExchange

Lambda Selection Algorithms

IterateRidge IterateRols StepItRols

Width Selection Algorithms

TrialWidths WidPerDim Tree Reg ression

Prune F unctionality

Statistics

Overview o f Radial Basis Function Statistics GCV Criterion GCV for Ridge Regression GCV for Rols References

.......................................... 7-15

.......................................... 7-30

................................... 7-13

........................ 7-14

.................................... 7-15

.................................. 7-16

....................... 7-17

...................................... 7-17

....................................... 7-18

....................................... 7-20

......................... 7-21

...................................... 7-21

...................................... 7-22

................................... 7-23

............................... 7-27

.................................... 7-31

.......................... 7-32

..................................... 7-32

....................................... 7-33

........... 7-30

Hybrid Radial Basis Functions

Introducing Hybrid Radial Basis Functions Width Selection Algorithm: TrialWidths Lambda and Term Selection Algorithms: Interlace Lambda and Term Selection Algorithms: Two-Step

Tips for Modeling with Radial Basis Functions

Plan of Attack How Many RBFs to Use Width Selection Algorithms Which RBF to Use

.................................... 7-38

............................ 7-40

................................. 7-42

...................... 7-35

............ 7-35

............... 7-35

......................... 7-41

...... 7-35

...... 7-36

....... 7-38

xiii

Lambda Selection Algorithms ....................... 7-42

Center Selection Algorithms General Parameter Fine-Tuning Hybrid RBFs How to Find RBF Model Formula

..................................... 7-43

......................... 7-43

..................... 7-43

.................... 7-43

Index

xiv Contents

Workflows For Modeling

The following sections introduce the Model Browser part of the Model-Based Calibration Toolbox™ product.

• “What Is the Model Browser?” on page 1-2

• “How to Use This Manual” on page 1-3

• “Overview of Modeling with the Model-Based Calibration Toolbox Product”

on page 1-5

• “How to Set Up a One -Stage Model” on page 1-7

• “How to Set Up a Two-Stage Model” on page 1-9

• “How to Set Up a Point-by-Point Model” on page 1-11

• “Creating Alternative Models to Compare” on page 1-17

1 Workflows For Modeling

What Is the Model Browser?

The Model-Based Calibration Toolbox product contains tools for design of experiment, statistical modeling, and calibration of complex systems. See “Product Overview”. The toolbox has two main user interfaces:

• Model Browser for design of experiment and statistical modeling

• CAGE Browser for analytical calibration

The Model Browser is a flexible, powerful, intuitive graphical interface for building and evaluating experimental designs and statistical models:

• Design of experiment tools can drastically reduce expensive data collection

time.

• You can create and evaluate optimal, space filling, and classical designs,

and constraints can be designed or imported.

• Hierarchical statistical models can capture the nature of variability

inherent in engine data, accounting for variation both within and between tests.

1-2

• The Model Browser has powerful, flexible tools for building, comparing,

and evaluating statistical models and experimental designs.

• There is an extensive library of prebuilt model types and the capability to

build user-defined models.

• You can export models to CAGE or to MATLAB

Starting the Model Browser

To start the application, type

mbcmodel

at the MATLAB command prompt.

, or Simulink®software.

How to Use This Manual

This manual is the Model Browser User’s Guide. See also the CAGE User’s Guide for information on the other main inte rface of the Model-Based Calibration Toolbox product.

Learning the Model Browser

For tutorials and case studies, see Getting Started in the Model-Based Calibration Toolbox Getting Started Guide.

Using the Model Browser

• “Overview of Modeling with the Model-Based Calibration Toolbox Product”

on page 1-5 provides an overview of how to model with the Model Browser and where to find information.

• Chapter 2, “Projects and Test Plans” describes how to set up and use

projects and test plans, and navigate using the Model Tree.

How to Use This Manual

• Chapter 3, “Designs” is a guide to constructing experimental designs

using the Design Editor. You can set up, view, and compare design types: classical, space-filling, and optimal. You can define and apply constraints and export designs.

• Chapter 4, “Da ta” describes how to load, merge, filter, and view data using

the Data Editor, including matching data to experimental designs. You can load data from files, or the workspace, or custom Excel view plots of the data and define new variables and filters. You can store and import user-defined variables and filters, and define test groupings. You can customize the data loading interface with your own functions.

• Chapter 5, “Setting Up Models” is a gu ide to setting up models using the

Model Browser, including descriptions and illustrations of model types. This section describes how to set up local, global, response and boundary models; and how to use the Build Models dialog box to add multiple model types to compare, after you have an initial model.

• Chapter 6, “Selecting Models” is a complete guide to viewing, evaluating,

and verifying models using the Model Browser. This explains all the functionality available in the different model views of the Model Browser, the Model Selection and Model Evaluation windows, the Stepwise an d

sheets. You can

1-3

1 Workflows For Modeling

Box-Cox Transformation tools; how to use validation data; the mathematics of two-stage models; and how to export models.

• Chapter 7, “Radial Basis Functions” is a guide to all aspects of using radial

basis functions in m odeling, from setup to the mathematical basis.

1-4

Overview of M odeling with the Model-Based Calibration Toolbox™ Product

Overview of Modeling with the Model-Based Calibration Toolbox Product

Functionality is described in the order you see it during the process of model building. For a quick guide to setting up models, then searching for the best fit, see these overview pages:

• “How to Set Up a One -Stage Model” on page 1-7

• “How to Set Up a Two-Stage Model” on page 1-9

• “How to Set Up a Point-by-Point Model” on page 1-11

• “Creating Alternative Models to Compare” on page 1-17

The different views of the Model Browser are described in these sections:

• “Project Level: Startup View” on page 2-2

• “Test Plan Level” on page 2-21

• “Local Level” on page 6-2

• “Global Level” on page 6-24

• “Response Level” on page 6-76

To construct models, you must navigate using the model tree, use test plans, andloadandmanipulatedataanddesigns. These topics are covered in these sections:

• “Model Tree” on page 2-11

• “TestPlans”onpage2-17

• Chapter 4, “Data”

• Chapter 3, “Designs”

Model construction, evaluation and export are covered in these s ections:

• “Selecting Models” on page 6-40

• “Model Evaluation Window” on page 6-79

1-5

1 Workflows For Modeling

• “Exporting Models” on page 6-113

1-6

HowtoSetUpaOne-StageModel

The following steps are necessary to set up a one-stage model:

1 From the project node, create a new one-stage test plan. See “Creating a

New Test Plan” on page 2-17. The view chang es to select the new node in the model tree and show you the test plan level.

2 Set up the inputs and model type by double-clicking the Inputs block and

the Model block in the test plan diagram. See “Input Factor Setup” on page 5-4 and “Global Model Setup” on page 5-67.

HowtoSetUpaOne-StageModel

3 At this poin

Editor” on p

4 From the test plan node, lo ad a new data set to use. Choose

t, you might want to design an experiment. See “The Design

age 3-2.

TestPlan > Select Data which opens the Data Wizard. Click Load new data set and the Data Import Wizard appears. See “Loading Data from

File” on page 4-18. Use the Data Wizard to match up data signals with model inputs and outputs.

5 Dismissing the Data Wizard opens the Data Editor, where you can select

data for modeling and match data to designs. See “The Data Editor” on page 4-5.

On clos node ap

6 View the model fit.

ing the Data Editor the model fit is calculated and the new model

pears in the model tree.

Functionality available for viewing and refining the model fit i s described in “Glo bal Level” on page 6-24 and “Selecting Models” on page 6-40.

7 You c

an create a boundary model at the test plan node. A boundary model

ribing the limits of the operating envelope can be useful when you are

desc

ting and evaluating designs, optimization results and global models. It

crea

e useful to create the boundary m odel before viewing global models, so

can b

can see the model areas inside the boundary on plots. See “Boundary

you

el Setup” on page 5-48.

Mod

1-7

1 Workflows For Modeling

Onceyouhavebuiltasinglemodel,youshouldcreatemoremodelsfor comparison, to search for the best fit. You can follow the guidelines in “Creating Alternative Models to Compare” on page 1-17.

1-8

HowtoSetUpaTwo-StageModel

The following steps are necessary to se t up a two-stage model:

1 From the project node, create a new two-stage test plan. See “Creating a

New Test Plan” on page 2-17.

2 From the test plan node, set up the inputs and models at the local and

global stages. See “Input Factor Setup” on page 5-4, “Local Model Setup” on page 5-6 and “Global Model Setup” on page 5-67.

HowtoSetUpaTwo-StageModel

3 At this poin

Editor” on p

4 From the test plan node, load the data set you want to use. Select

t, you might want to design an experiment. See “The Design

age 3-2.

TestPlan > Select Data. See “Loading Data from File” on page 4-18.

This opens the Data Wizard, where you can load a data set, match data signals to model variables and then set up the response model.

5 On comple

data for m 4-5. Clos

Note On

6 At the l

you ca

7 You can create a boundary model at the test plan node. Boundary models

ting the Data Wizard, the Data Editor opens. Here you can se lect

odeling and match data to designs. See “The Data Editor” on page

e the Data E dit or and click Yes to accept the data for modeling.

closing the Data Editor, the local and global models are calculated.

ocal node , you can view the fit of the local models to each test, and

nalsoviewtheglobalmodelsattheresponsefeaturenodes.

describe the limits of the operating envelope and can be useful when you are creating and evaluating designs, optimization results and global models. It can be useful to create the boundary model before viewing global models, s o you can see the model areas inside the boundary on plots. See “Boundary Model Setup” on page 5-48.

8 The two-stage model is not calculated until you use the Select button (from

the local node, in the Response Features pane) and choose a model as best (evenifitistheonlyonesofar),unless you go straight to MLE. See below.

1-9

1 Workflows For Modeling

See “Selecting Models” on page 6-40.

Note At this point, the two-stage model is calculated, and the icon changes at the local node to reflect this. See “Icons: Curves, Worlds, and Houses” on page 2-13.

9 You are prompted to calculate the maximum likelihood estimate (MLE) at

this point. You can do this now, or later by selecting Model > Calculate MLE. See “MLE” on page 6-72 for a detailed explanation.

Note If there are exactly enough response features for the model, you can go straight to MLE calculation without going through the Select process. The MLE toolbar button and the Model > Calculate MLE menu item are b oth active in this case. If you add new response features, you cannot calculate MLE until you go through model selection to choose the response features to use.

1-10

See “Two-Stag e Models f or Engines” on page 6-100 for a detailed explanation of two-stage models.

Double-click the Model blocks of the block diagram or select the TestPlan >SetUpModelmenu item. The toolbox supports a wide range of models.

For model descriptions, see “Global Mo de l Setup” on page 5-67 and “Local Model Setup” on page 5-6.

Onceyouhavebuiltasinglemodel,youshouldcreatemoremodelsfor comparison, to search for the best fit. You can follow the guidelines in the next section, “Creating Alternative Models to Compare” on page 1-17.

How to Set Up a Point-by-Point Model

In this section...

“Point-by-Point Modeling Process” on page 1-11

“Use Cases for Point-by-Point Models” on page 1-13

“Analyzing Point-by-Point Models” on page 1-14

“Exporting Point-by-Point Models to CAGE” on page 1-14

Point-by-Point Modeling Process

Use the following process to set up a point-by-point model:

1 Create a new test plan.

a From the project node, click New to create a new point-by-point te st plan.

b Select the Point-by-Point template and click OK.

This template allows you to create point-by-point test plans with local models at each engine operating point, which is useful when testing is done at fixed operating point settings. See“UseCasesforPoint-by-Point Models” on page 1-13.

How to Set Up a Point-by-Point Model

The new test plan sets up two local and two global inputs, and the local model type is set to Multiple Models. This local model type allows you to choose a variety of models to try for each test.

2 Set up

diag

3 Set up the multiple model types. By default, point-by-point test plan

templates include four model types: a quadratic, a cubic, an RBF and a hybrid RBF. If you want to change or add to these, double-click the Local Model block in the test plan diagram, and then click Edit in the Local Model Setup dialog box. The toolbox fits all the models y ou choose and then indicates the best one chosen for each test in your data. For example, for some tests a radial basis function may fit best, while for others a quadratic would be acceptable. You can use any model available as one-stage models (including radial basis functions (RBF) and hybrid RBF).

the inputs by double-clicking the Inputs blocks in the test plan

ram. See“InputFactorSetup”onpage5-4.

1-11

1 Workflows For Modeling

The Automatic input ranges check box defaults to selecte d. This selection uses the range of the data for each test, instead of a single range for all. You can choose any of the summary statistics as the selection criteria for deciding which model fits best to each test.

For more information see “Local Model Class: Multiple Models” on page 5-24.

4 Select data to use. From the test plan node, select TestPlan > Select Data

which op e ns the Data Selection Wizard. Load a data file (see “Loading Data from File” on page 4-18) then match up data signals with model inputs and outputs (see “Step 2: Select Input Signals” on page 4-38).

You can click Finish on the Input Names pane: there is no need to build response models before building boundary models. Alternatively, you can continue with the Data Selection Wizard to specify any responses you want to model.

5 Create boundary models in the Boundary Editor. Select

TestPlan > Boundary Models. To build point-by-point boundary models for each operating point, create a local boundary model, and select

Point-by-Point for the Global evaluation type. See “Creating Boundary

Models” on page 5-49.

1-12

6 If you h av e not already built response models, from the test plan node,

double-click the Response block to choose a response and fit models.

The view switches to the new

Multiple Models node in the m odel tree.

View and edit the local model fit for each operating point.

Tools specific to local multiple models are described in “Analyzing Point-by-Point Models” on page 1-14.

For details about other functionality available for viewing and refining the model fit, see “Local Level” on page 6-2 and “Selecting Models” o n page 6-40.

7 Export your point-by-point models to CAGE for optimized calibration. From

the test plan node, select Test Plan > Export Point-by-Point Models.

If CAGE is open, you can also use the dialog box options to automatically create the following objects from your point-by-point models:

How to Set Up a Point-by-Point Model

• Data set

• Tradeoff

• Optimization

See “Exporting Point-by-Point Models to CAGE” on page 1-14.

Note This export option is only available if you have exactly two global inputs; otherwise you can import your point-by-point models into CAGE using the CAGE Import Tool.

Use Cases for Point-by-Point Models

Thepoint-by-pointtestplantemplate and the local multiple models type provides a convenient mechanism to model a number of tests at different operating points using the same set of models. Using the test plan has several advantages, including:

• You can divide the data into tests and model it within a single test plan

rather than having a separate one-stage test plan for each operating point. The toolbox does not co n struct two-stage models or response feature models becauseitisimpossibletochooseresponsefeaturesthatapplytoalltests, when there are different model types for different tests. You must have at least one global variable (e.g. speed, injection timing, load) and you cannot use covariance modeling.

• The local multiple models type provides a smooth interface with the CAGE

browser p art of Model-Based Calibration toolbox software. To make use of this, you must specify two global inputs (often speed and load) which can form the axes of tradeoff tables. This useful application for multiple models allows you to calibrate from local maps.

• You can also use point-by-point models in CAGE optimization, by creating

an optimization from your models, or you can use the models in an existing optimization provided the global variable values are exactly the same as the global variables used for the local models in the Model Brows er.

• You can export point-by-point models to file or directly into CAGE, and

automatically create an optimization, a tradeoff, and a data set from your point-by-point models.

1-13

1 Workflows For Modeling

Analyzing Point

In the local mode point-by-point

• You can select M

Selection win only the best m Selection win data, and sel

In the Model S boundaries

input range

data ranges

• You can sel

Setup dial you click O selects i alternat new best m

• You can s

Summary to displ Summar page 6in this curren fit RM

t as best if it is better (by your selection criteria) than any of the

ives for a test. A dialog box informs you which tests (if any) have a

ay in the Model Selection window and in the Diagnostic Statistics

47. If you are using validation data, the validation RMSE appears Summary Table for the test, if there is v alidation data for the

t test (glob al variables must match), for com pa rison with the model

SE. See “Using Validation Data” on page 6-81.

l view, you find controls and menu items specific to

models (using the

odel > Utilities > Select L ocal Model to open the Model

dow. All the alternative models are refitted at this stage (as

odel is stored) so this refitting can take time. In the Model

dow y ou can compare the fit of your local model types to the

ect which of those model types to use for the selected test.

election and Model Evaluation window, you can view local

on surface and cross-section plots. If you selecte d Automatic

s during model setup, plots for point-by-point models use the

per test (unlike typical two-stage models).

ect M odel > Utilities > Add Local Model to open the Model

og box. In this dialog box, you can add a model type. When

K the toolbox fits the new model type to all tests, and then

odel.

elect Model > Utilities > Summary Statistics to open the

Statistics dialog box. In this dialog box, you can select statistics

y Table (in the local model view). See “Summary Statistics” on

-by-Point Models

Multiple models local model type).

1-14

• In the

the dr (e.g You s loca

Exp

If y mod

Diagnostic Statistics pane you can select

op-down list. This pane displays the value of your selection criteria

., AICc) for each model type, with the best model highlighted in red.

elect criteria in the Local Model Setup dialog box when you create

l multiple models.

orting Point-by-Point Models to CAGE

ou have exactly two global inputs, you can export point-by-point

els from the Test Plan node in the Model Browser by selecting

stPlan > Export Point-by-Point Models.

Model Selection from

How to Set Up a Point-by-Point Model

• If the CAGE Browser is closed, the toolbox creates a point-by-point model

tradeoff file for use with the CAGE import dialog box Import Point-by-Point Model Tradeoff. Choose a location and enter a file name in the dia log box.

• If the CAGE Browser is open, you see the Export Point-by-Point Models

dialog box, as shown in the following figure.

You can create the following items in CAGE (depending on your check box selections in the dialog box):

• Point-by-point models, for all responses in the test plan.

The toolbox creates a single CAGE model for each response and defines the models only at operating points corresponding to the global inputs—for example, at the values of speed and load where you performed tests.

The toolbox uses the response name for each CAGE model name, and replaces any existing CAGE model of that name. The model inputs are connected to variables matching the symbols. The set points are the same

1-15

1 Workflows For Modeling

as the first row of the dataset, corresponding to the first operating point. You can view the model surface in the Model View and the Surface Viewer.

• Local boundary mo dels for each point-by-point model.

If you have cre ated local boundary models (and selected them as best), the toolbox includes them in the export.

If you have not created local boundary models, the toolbox creates them automatically, and you see a notation of “ describing the boundary model. The toolbox builds a in all local inputs for each test.

• Data set for model operating points (optional).

The dataset contains the midpoints of the local input ranges for all tests and the global operating points.

• A point-by-point model tradeoff (optional).

The toolbox creates a point-by-point model tradeoff in the same way as using the Import Point-by-Point Model Tradeoff dialog box. These tradeoff tables are initialized with the midpoints of the local input ranges.

(created)”afterthetext

range boundary model

1-16

• An optimization (optional).

The toolbox creates an

Ifyouchoosetocreatetheoptimization, use the drop-down menus to specify the response to be optimized and whether the objective should be minimized or maximized. The optimization includes the boundary model as a constraint and uses the same values as the dataset to specify a run per operating point. You can use the new optimization with Automated Tradeoff.

You can also import the point-by-point models directly into CAGE from the Model Browser using the CAGE Import Tool. If you have more than two global inputs you must use the Import Tool.

foptcon optimization.

Creating Alternative Models to Compare

Once you have fitted and examined a single model (eithe r one- or two-stage), youwillnormallywanttocreatemoremodelstosearchforthebestfit. You can:

• Create individual new models.

- You can create new child nodes by clicking the New button from any

modeling node. The Model Setup dialog box appears where you can change the type and settings. You can repeat this for multiple child nodes to create a selection of different model types fitted to the same data.

• Create a template to save a varie ty of model settings for reuse.

- From any global or one-stage model with child nodes, select

Model > Make Template. You can save the child node model types of your currently selected modeling node as a template. You can then use the Build Models dialog box to find your user-defined templates and quickly build all those model types again for any global model you choose (see below).

Creating Alternative Models to Compare

- From any global model node (before calculating MLE), click Build

Models in the toolbar. You can save a template containing whatever

models you choose by selecting

• Use the Build Models function to create a selection of models at once.

You can create templates, use predefined templates, and use models in the current project as a template. All the child node model types in the template y ou select are built as child nodes of the currently selected global model. See “Boundary Model Setup” on page 5-48.

Creating a template containing a list of all the models you want is a very efficient way to quickly build a selection of alternative model child nodes for many global models. Use these techniques to find models that fit well to the data for each of your global models.

We provide a detailed tutorial example to guide you through using these techniques to make a number of models to compare. See the modeling tutorial section “Creating Multiple Models to Compare” in the Getting Started documentation. This tutorial section requires you to have completed the

New and adding the model types you want.

1-17

1 Workflows For Modeling

previous steps in the Quick Start tutorial, so that you have an initial model as astartingpoint.

Note that you can click New from any modeling node to create new child nodes.

• For all global model nodes (one- and two-stage) a new child node appears

that is a copy of the parent, and the Model Setup dialog box appears where you can change the type and settings.

• For the other two-stage modeling nodes (local and response models) you

also get a new child node but the procedure is slightly different — see the tutorial example for details.

Once you have a variety of models to compare , you should use the diagnostic statistics and powerful visual aid plotting capabilities of the Model Browser to help you decide which models are best. For a description of the views and statistics available in each modeling view, see:

• “Selecting Models” on page 6-40

1-18

• “Model Evaluation Window” on page 6-79

• “Local Level” on page 6-2

• “Global Level” on page 6-24

• “Response Level” on page 6-76

For linear models, make use of the Stepwise functions (open the Stepwise window for existing models, and/or choose a Stepwise option during model setup) to refine your models and remove less useful model terms. Make sure you examine outliers but do not automatically remove them without good reason. Pay attention to the diagnostic statistics to help you find the best models. The following sections describe how to u se Stepwise to make better models and how to understand the diagnostic statistics:

• “Model Selection Guide” on page 6-42 for guidelines.

• “Stepwise” on page 6-84 for an introduction to Stepwise.

• “PRESS statistic” on page 6-93 — See this section for guidelines on what

to look for in t he statistics to indicate a good fit.

Projects and Test Plans

• “Project Level: Startup View” on page 2-2

• “Model Tree” on page 2-11

• “TestPlans”onpage2-17

• “Test Plan Level” on page 2-21

2 Projects and Test Plans

Project Level: Startup View

In this section...

“Project View” o n page 2-2

“All Models Pane” on page 2-3

“Data Sets Pane” on page 2-3

“Notes Pane” o n page 2-4

“Test Plans List Pane” on page 2-5

“Project Level: To olbar” on page 2-7

“Project Level: Menus” on page 2-8

Project View

When you open the Model Browser part of the Model-Based Calibration Toolbox product, there is a single node, the project (named Untitled), in the model tree. This node is automatically selected.

2-2

When the project node in the model tre e is selected at any time, the following functionality is available. This state is called project level.Whenyoustart you are automatically at p roject level, as there are not yet any other nodes to select.

See “Overview of Modeling with the Model-B ase d Calibration Toolbox Product” on page 1-5 to find information about using the Model Browser.

Note The node selected in the model tree determines what appears in the menus and panes of the rest of the Model Browser.

Project Level: Startup View

All Models pane Data Sets pane

Tip of the day

Notes pane Test Plan list pane

All Models Pane

This pane contains a hierarchical structure showing all the models created within the current project. See “Model Tree” on page 2-11 for a detailed description of the information contained in this pane.

Data Sets Pane

All data sets loaded in the current project are displayed in the Data Sets pane (whether in use for modeling or not).

2-3

2 Projects and Test Plans

You can select a data set (by clicking it) and then

• Delete it by pressing the Delete key.

• Rename it, by clicking again or pressing F2 (as when selecting to rename

in Windows Explorer), then editing the name.

• Open it by double-clicking. Double-clicking a data set opens the Data

Editor; unless it is already associated with a test plan, see below. See “The Data Editor” on page 4-5.

Note All data sets loaded are visible at the project node and appear in the Data Sets pane. However, they are not necessarily used by any test plan child nodes of that project until you select them within a particular test plan. For example, with a data set loaded at the project node, when you switch to a new test plan node, the Data Sets pane at top right displays ’

Data is selected

test plan. See “Data Selection Wizard” on page 4-37.

’ until you use the Data Wizard to attach data to that

2-4

Thesamedatasetcanbeusedbymanytest plans within a project, although each individual test plan ca n only ever use one data set (and one validation data set).

When you h av e associated a data set with a test plan, a new data set icon (with a name specific to that test plan) appears here in the Data S ets pane. The same data set may be used by several test plans, at which you may have applied different filters, groupings or edits, and so each time you associate a data set with a n ew test plan a new icon appears here. You cannot edit these test plan-specific data sets at project level, you must edit them from the associated test plan.

Notes Pane

The Notes pane contains a list box showing all previous notes on the current project. You use notes to record changes within a project, perhaps by different users, over time.

Project Level: Startup View

• You add new notes

or by pressing th Notes automatic

• You edit notes (

must match) by s selecting to r time and date f

• You remove not

same user tha

• Notes are au

example, th user “info”

Test Plans

You genera New butto

This pane the level the chil Models p

te new test plans from the Test Plans list pane by clicking the

n. See “Test Plans” on page 2-17.

is the Test Plans list pane at startup but change s depending on

in the model tree that is selected. The list box always displays all

d n odes of whichever node is curre ntly selected in the tree in the All

ane, and always contains three buttons: New, Delete,andSelect.

e Insert keyafterselectingtheNotes pane by clicking.

ally have the u ser login name and the date.

only the user that created them can edit them; user names

ename in Windows Explorer). Edited notes have updated

ields.

es by selecting them and pressing Delete (but only the

t created them can delete them).

tomatically added to the project when it is modified (for

e initial “Created by <username>” note). These notes (listed as

) cannot be deleted or edite d.

List Pane

by clicking the Add new note button

elect-clicking or by pressing F2 when s ele cted (as when

in the toolbar,

Note Do that no can als (sele

• The T

Mode

sele pane can

• The

For an

uble-clicking any item within this pane changes the view directly to

de. (This is equivalent to selecting that node in the model tree.) You

ousetheDelete and Insert keys to remove or add new test plans

ct a test pla n first).

est Plans list becomes the Response Models list, the Local ls list, the Response Features list, and the Models list as you

ct the nodes at subsequent levels of the model tree. In each case this

displays the immediate child nodes of the current node selected. You

use the buttons to delete selected nodes or create new nodes.

feature added by clicking New always corresponds to the list items.

example, clicking New whenthepaneshowsalistoftestplansadds

ew test plan . Clicking New whenthepaneshowsalistofresponse

2-5

2 Projects and Test Plans

features opens the Response Features dialog box, as shown in the following example. The response features you can add are model-specific. This example shows the response features available for a polyspline model.

2-6

For example, if you choose f(x + datum) and enter 10 in the Value edit box , the new response feature tracks the datum +10. For a torque/spark polyspline model, the datum is MBT (maximum brake torque); so the new response feature is MBT + 10 degrees of spark angle. This allows you to create response features that have engineering interest.

The response features available depend on the model type. For more details on which response features are available, see each model type under “Local Model Setup” on page 5-6.

• You can use the Select button to select the best child node, but only

when the child nodes are local m o de ls, response features, or submodels of response features. In each case clicking the Select buttontakesyoutoa selection window where you can compare the child nodes. See “Selecting Models” on page 6-40.

Project Level: Startup View

Tip of the Day

Hints about using the Model Browser appear here. Y ou can scroll through more tips using the buttons at the bottom,andyoucansnapthispaneclosed or open by clicking the “snapper point” where the curs or changes if you roll the mouse over it.

Project Level: Toolbar

New Data Object

Edit Data Object

Help

Copy Data Object

New Project Open Project Save Project

This is how the toolbar appears when you first start the toolbox. The last two Data buttons are grayed out; the Edit data object and Copy data object buttons are not enabled until a data set has been loaded.

• All the toolbar items are duplicated under the menus except New Note.

• For the Project buttons, see the File menu.

• For the Data buttons, see the Data menu.

• New Note adds a note to the Notes pane.

• The New and Delete node buttons are duplicated in the File menu. In

both cases, their function depends on the node selected in the model tree. In every case, New generates a new child node of the one currently selected, and Delete removes the current node (and all its children).

• The Up One Level button moves the current selection (and h ence all the

views) one level up the model tree. For example, if a test plan node is selected, clicking this butto n moves one level up to the project node.

Up One Level Delete current node New child node

Add New Note

2-7

2 Projects and Test Plans

Two buttons, Delete and Up One Level, are grayed out at startup because the default selection in the model tree is the project node, so there are no levels above, and you cannot delete the project node (although you can replace it with a n ew one).

• The print icon is only enabled in views with plots, for example, the local

node, response feature nodes, and response nodes after selection of a best two-stage models (response nodes are blank until then).

• The Help button opens the Help Browser and displays the appropriate help

documentation for the current view in your Model Browser.

Project Level: Menus

File Menu

Note The File menu remains constant in each Model Browser view. The New

child n ode function always creates a new child node, and the Delete current node function always deletes the current node. These change according to

which node in the model tree is currently selected.

2-8

• New Project opens a new project file. You are prompted to save or lose

the current project.

• Open Project opens a file browser to select the project to open.

• Save Project and Save Project As save the project with all the models it

contains as a .

• New Test Plan opens a dialog box with the choice of

Two-Stage test plans, or you can browse for other test plans. The New

(child node) menu option always creates a new child node of whichever node is selected i n the model tree.

At startup, the project node is automatically selected, so the appropriate child node is a new test plan.

mat file.

One-Stage or

Project Level: Startup View

Note File > New changes depending on which node in the model tree is selected. In each case the option offers a new child node immediately below the one currently selected, that is, a New Test Plan (if a project node is selected), a New Response Model (from a test plan) or a New Model child n ode (from a one-stage response). For two-stage models you can add a New Local Model (from a response node), a New Response Feature (from a local node) and a New Model from a response feature node.

• Export Models brings up the Export Models dialog box. This allows you

to export any models selected in the tree (along with their child nodes, in some cases) to the MATLAB workspace, to file for importing into CAGE, or to Simulink software. See “Exporting Models ” on page 6-113.

• Delete “Untitled” Like the New item in this menu, this option changes

depending on which nod e in the model tree is selected. This menu item deletes w hichever node is currently selected in the model tree (alo ng with any child nodes), and the a ppropriate name appears here.

• Clean Up Tree From any modeling node where a be st model has been

selected (from the child nodes), you can use this to delete all other child nodes. Only the child nodes selected as best remain.

• Preferences brings up the MBC File Preferences dialog box, in which you

can specify default locations for projects, data, models, and templates. You can also edit and save user information: name, company, department and contact details. This inform ation is saved with each project level note, and you can use this to trace the history of a project.

• Print is only enabled in view s with plo ts — no t test plan or project level.

• 1,2,3,4: A list of the four most recent project files, including their

pathnames.

• Close Exits from the Model Browser part of the toolbox (CAGE and

MATLAB are unaffected).

Data Menu

• New Data —OpenstheDataEditor. See“TheDataEditor”onpage4-5.

• Copy Data — Copies the selected data set.

2-9

2 Projects and Test Plans

• Edit Data — Opens the Data Editor to enable data editing.

• Delete Data — Deletes the selected data set.

View Menu

• Tip of the Day —Choosetodisplayorhidethetipspane.

• Note Information — Opens a dialog box where you can decide which

categories of information to display for each project note. Y ou can specify user information for display w ith notes by using File > Preferences.

Window Menu

Depending on which toolbox windows are open, a list appears under this menu and whichever windo w is selected is brought to the front. The Window menu remains throughout the Model Browser.

Help Menu

The Help menu remains consistent throughout the Model Browser.

2-10

• MBC Help — Opens the Model-Based Calibration Toolbox Roadmap with

links to the help tutorials and the indexed help pages.

• Context Help — Depending on what part of the Model Browser is

currently active, Context Help links to different places in the Help files.

• About MBC — Displays version notes.

Model Tree

In this section...

“Navigating the Model Tree” on page 2-11

“Tree Structure” on page 2-12

“Icons: Curves, Worlds, and Houses” on p age 2-13

“Icons: Blue Backgrounds and Purple Worlds” on page 2-14

Navigating the Model Tree

ThetreeintheAll Models pane displays the hierarchical structure of the models you have built. Views and functionality within the browser differ according to which node is selected.

The following is an example of a model tree.

Project node

Test plan node

Response node

Local node

Global nodes

The elements of the tree consist of the following:

1 Project node

tplannode

2 Tes

3 Response node

4 Local node

2-11

2 Projects and Test Plans

5 Global nodes — All one-stage m odel nod es are global models. For two-stage

models, global models are fitted to response features of the local models. Each step down in the tree is a child node of the node above. Global models are child nodes of local nodes and so on.

Note The selected node in the tree governs the model that is displayed in the various other panes of the browser and which menu items are available. The selected node governs the level displayed: project level, test plan level, and so on. The functionality of each level is described in the Help.

You can renam e all nodes, as in Windows Explorer, b y clicking again or by pressing F2 when a node is selec ted.

There is a context menu available. When you right-click any node in the model tree, you can choose to delete or rename that node, or create a new child node.

Tree Structure

2-12

Project node

Test plan node

Response node

Two-stage models

Global nodes

Model Tree

The preceding example shows a more extensive model tree, w ith two two-stage models as child nodes of a single response model.

There can be many models within (or under, as child nodes in the tree) each two-stage global node, or a ny one-stage model node.

There can also be many different response nodes within a single test plan, and each project can contain several different test plans. However, there is only one project node in the tree at any time.

Note You can only have one project open at any one time; if you open another, you are prompted to save or discard your current project.

You can add child nodes to all global models — several candidate models can be tried at each global node and the best selected. There is an example showing this at the end of the section on “Icons: Blue Backgrounds and Purple Worlds” on page 2-14 and the process is illustrated in the “Tutorial: Model Quickstart ” in the Getting Started documentation.

Icons: Curves, Worlds, and Houses

The icons are designed to give visual reminders of their function.

• Test plan icons have a tiny representation of the test plan diagram. You

can see the one-stage and two-stage icons in the following example.

• The local model icon shows a curve over a house.

• Global model icons show a curve over a globe. All one-stage models are

global models and for two stage models, all nodes below the local node are global models.

2-13

2 Projects and Test Plans

Test plan nodes

Response node

Local node

Global nodes

• Theresponsenode(emptyuntilatwo-stagemodeliscalculated)hasan

icon that combines the local and global motifs — a curve over a house and a globe — to symbolize the two-stage process.

• When a two-stage model has been calculated, the icon at the local no de

changes to show the combination of local and global motifs.

Icons: Blue Backgrounds and Purple Worlds

2-14

Project node

Test plan node

Response node

Two-stage models

Global nodes

Icon changes convey information about modeling processes.

• When a model is selected as the best model, its icon changes color and gains

a blue background, like the

BSPLINE1 model in the preceding example.

Model Tree

• When the maximum likelihood estimate (MLE) is calculated and chosen as

the best model, the associated model icon and all its child nodes (along with the plots of that model) become purple.

Youcanseethisintheprecedingexample: the B Spline model and all its response features have purple curves, globes, and house, indicating that they are MLE models. The Poly3 model and its children have blue curves and globes and a red house, indicating that they are univariate models.

• Observe the other difference between the B Spline and the Poly3 icons: the

B Spline has a blue background. This indicates that this is selected as best model, and is used to calculate the two-stage model at the response node, sotheresponsenodeisalsopurple.IfanMLEmodel(withpurpleworlds) is selected as best model and is used to create the two-stage model, the response node always reflects this and is also purple.

• Notice also that the response features all have blue backgrounds. This

shows they are se le cted as best and are all being used to calculate the two-stage model. In this case they a re all needed. That is, a B Spline model needs six response features, and a Poly3 model requires fo ur. If more response features are added, however, some combination must be selected as best, and the response features not in use do not have a blue background. Anexampleisshownin“Tutorial: ModelQuickstart”inthe Getting Started documentation.

In the following example you can see child nodes of a global m odel. You can try different models within a global model, and you must select one of the attempts as best. In this example you can see that

Cubic is selected as

best, because it has a blue background, so it is the model being used for the

Blow_2 global model.

2-15

2 Projects and Test Plans

When a model is selected as best it is copied up a level in the tree together with the outliers for that model fit.

When a new global or lo cal model is created the parent model and outliers are copied from the current level to the new child node. This gives a mechanism for copying outliers around the model tree.

2-16

A tree node is automatically selected as best if it is the only child, except two-stage models which are never automatically selected - you must use the Model Selection window.

If a best model node is changed the parent node loses best model status (but the automatic selection process will reselect that best model if it is the only child node).

Note Try the Quick Start tutorial in the Getting Started documentation to understand how to use the model tree. The last section, “Creating Multiple Models to Compare”, guides you through the process of creating a variety of models of different types and how to understand the information in the model tree. You need to complete the previous sections of the tutorial first, which guides you through setting up a single two-stage model to g et started.

Test Plans

Test P l ans

In this section...

“Creating a New Test Plan” on page 2-17

“Creating New Test Plan Templates” on page 2-18

Creating a New Test Plan

You need to select a test plan to construct models.

You can select templates for one-stage, two-stage or point-by-point test plans, as described next. You can also use these to create your own test plan templatesoyoucanreusethesetupforonetestplanwithanothersetofdata. See “Creating New Test Plan Templates” on page 2-18.

To create a new test plan:

• Click New in the Test Plans pane (visible at startup and whenever the

project node is selected in the model tree).

Alternatively, make sure the project node is selected first, and then do one of the following:

• Click the New Test Plan icon (

• Select File > New Test Plan.

• Press Insert immediately after clicking the tree.

These steps all open a dialog box with the choice of or

Point-by-Point test plans, or you can browse for other test plans (as new

templates can be created and saved). See “Local Level” on page 6-2.

A new test plan node appears in the model tree. To view the new test plan:

Change to test plan level:

lect the node

• Se

inthetreebyclickingit.

)inthetoolbar.

One Stage , Two Stage,

2-17

2 Projects and Test Plans

Alternatively, double-click the n ew test plan in the Test Plans pa ne, as in Windows Explorer.

The Model Browser changes to tes t plan level, showing the block diagram representations of models in the main display, and the Test Plans pane changes to the Response Models pane (empty until models are set up).

For the next steps in model construction, see

• Chapter 5, “Setting Up Models”

• “Loading Data from File” on page 4-18

• “Selecting Models” on page 6-40

Creating New Test Plan Templates

You build user-defined templates from existing test plans using the Make

Template toolbar button

The procedures for m odeling engines for calibrations are usually repeated for a number of different engine programs. The test plan template facility allows youtoreusethesetupforonetestplanwith another set of data. You can alter the loaded test plan settings without restriction.

A list of test plan templates is displayed when you build a new test plan. There are built-in templates for one- and two-stage models.

Test plan templates store the following information:

• Hierarchical model — Whether the model is one- or two-stage and the

default models for each level.

• Designs — If they were saved with the template (check box in the Test Plan

Template Setup dialog box)

Thedesignforonetypeofenginemightormightnotbeappropriatefora similar type of engine. You can redefine or modify the design using the Design Editor.

or TestPlan > Make Template .

2-18

• All response models (for example, torque, exhaust temperature, emissions)

— If they were saved with the template (check box in the Test Plan Template Setup dialog box)

• Numbers and names of input factors to m odels

• Model types (local and global)

• Summary Statistics for display (see “Summary Statistics” on page 6-47)

• No model child nodes are included, just the top level of the test plan

(response mo dels, and local and global models for two-stage mo dels).

Theresponsemodelsareautomaticallybuilt after you assign data to the test plan; see “Using Stored Templates” on page 2-19.

Saving a New Template

From the test plan node that you want to make into a template:

• Click the Make Template toolbar icon or choose TestPlan > Make

Template.

Test P l ans

The templates are stored in the directory specified in the File > Preferences dialog box.

The Test Plan Template Setup dialog box appears, in which you can change the nam e of the new template and choose whether to include designs and/or response models.

Using Stored Templates

• When you load a new test plan from the project node, any stored templates

appear in the New Test Plan dialog box.

From the project node, select File > New Test Plan,orusethetoolbar button, or click New in the Test Plans pane.

• Selecting templates in the New Test Plan dialog box displays all templates

found in the directory specified in the Preferences dialog box (File menu). Select by clicking to see the informationonaparticulartemplate;the number of stages and factors is displayed in the Information pane. You

2-19

2 Projects and Test Plans

can use the Browse button if the required template is not in the directory specified in the File > Preferences dialog box.

• Click OK to use the selected test plan template. The new test plan node

appears in the model tree.

Use stored templates in exactly the same way as the default blank one- and two-stage templates. Models and input factors are already selected (including theresponseifthatwassavedwiththetemplate)soyoucangostraightto selecting new d a ta to model. You can still change any settings and design experiments.

Double-click the Responses block to launch the Data Wizard and select data for the test pl an . The response models are automatically built after selection of data.

Note The data selection process takes you through the Data Wizard. If any signal names in the new data do not match the template input factors, you must select them here, including the responses. If signal names match the factor names stored in the template, they are automatically selected by the Data Wizard, and you just click Next all the way to the end of the wizard. When you click Finish the response models are built autom atically .

2-20

Test Plan Level

In this section...

“Test Plan View” on page 2-21

“Test Plan Level: Toolbar and Me nus” on page 2-25

“Designing Experiments” on page 2-27

Test Plan View

When you select a test plan node (with the icon )inthemodeltree,this view appears.

Test P lan L eve l

2-21

2 Projects and Test Plans

This example is a two-stage model. All test plan nodes (one- and two-stage) show this view with a block diagram of the test plan. The diagram provides a graphical interface so you can set up inputs and set up models by double-clicking the blocks in the test plan diagram. These functions can also be reached using the TestPlan menu.

The diagram has the following functionality for setting up the stages in hierarchical modeling. A t present MBC only supports one- and two-stage models. You can reach these functions via the right-click context menu (on each block) or the menus:

1 “Input Factor Setup” on page 5-4 — Setting the number of inputs for each

stage of the model hierarchy.

2 “Local Model Setup” on page 5-6 and “Global Model Setup” on page 5-67 —

Setting up the new default models for each stage in the model hierarchy.

3 “Designing Experiments” on page 2-27 — Using the Design Editor.

You can access the Design Edi tor via the right-click menus on the model blocks or the TestPlan menu (for a particular model—you must select a model or input block before you can design an experiment). View Design Data also opens the Design Editor where you can investigate the statistical design properties of the data.

2-22

If the test plan already has a design, the design name is displa yed in the right pane.

4 “Loading Data from File” on page 4-18 and “New Response Models and

Datum Models” on pag e 5-88.

You can attach data to a new test plan by choosing TestPlan > Select Data or by double-clicking the Responses block in the diagram, which launches the Data Wizard (if the p roject already has data loaded).

If a test plan already has data attached to it, you can reach the Data Selection views in the Data Editor using the Select Data toolbar button

or the TestPlan m enu item. In the Da ta Editor you can select data

for modeling and match data to a design. Fo r example, after the design

Test P lan L eve l

changes, new data matching might be necessary. See “Matching Data to Designs” on page 4-44 for details.

If a test plan already has data attached to it, details of the data set (such as name, number of records) are displayed in the right pane.

5 “Summary Statistics” on page 6-47 — Right-click on the global model block

and select Summary Statistics to reach the Summary Statistics dialog box. In this dialog box you can choose which summary statistics you want displayed to help you evaluate models.

Other test plan level functionality includes:

• From the test plan level you can access the Boundary Constraint Modeling

functionality from the toolbar or TestPlan menu. See “Boundary Model Setup” on page 5-48.

If the test plan already has a boundary model, the right pane displays which boundary models are combined in the best boundary model, as shownintheexample.

• You can save the current test plan as a template using the

TestPlan > Make Template command or the toolbar button

.This capability can be useful for speeding up creation of subsequent projects. See “Creating New Test Plan Templates” on page 2-18.

• You can attach validation data to your test plan using the TestPlan menu.

You can use validation data with any model except response features. When you attach validation data to your test plan, Validation RMSE is automatically added to the summary statistics for comparison in the bottom list view of response models in the test plan. See “Using Validation Data” on p age 6-81.

If the test plan already has validation data attached to it, the name is displayed in the right pane.

The selected Model block is highlighted in yellow if a Setup dialog box is open; otherwise it is indicated by blocks at the corners. The selected Model block indicates the stage of the model hierarchythatisusedbythefollowing menu choices:

• Set Up Model

2-23

2 Projects and Test Plans

• Design Experiment

• View Design Data

• View Model

• Summary Statistics

The block diagram in the test plan view represents the hierarchical structure of models. Following is an example of a two-stage test plan block diagram.

2-24

See also “Test Plan Level: Toolbar and Menus” on page 2-25

Test P lan L eve l

Test Plan Level:

The eight buttons on the left (project and node management, plus the Print and Help buttons) appear in every view level. See “Project Level: Toolbar” on page 2-7 for details.

The right buttons change at different levels.

In the test plan level view, the right buttons are as follows:

• Design Experiment opens the Design Editor. Only available when a

model or input has been selected in the test plan block diagram. You must specify the stage (local or global) you are designing for. See .

• Select Data opens the Data Wizard, or opens the Data Selection views in

the Data Editor if data sets have already been selected. See Chapter 4, “Data”.

Toolbar and Menus

• Edit Boundary Constraint opens the Boundary Constraint Editor. See

“Boundary Model Setup” on page 5-48.

• Make Template opens a dialog box to save the current test plan as a

template, including any designs and response models. See “Local Level” on page 6-2.

Test Plan Level: Menus

File Menu. Only the New (child node) and Delete (current node) functions

change according to the node level currently selected. Otherwise the File menu remains constant. See “File Menu” on page 2-8.

2-25

2 Projects and Test Plans

Window Menu. The Window menu remains throughout the Model

Browser. Itallowsyoutoswitchwindowsifthereismorethanonetoolbox window open. S ee “Window Menu” on page 2-10.

Help Menu. The Help menu remains the same throughout the Model Browser. You can always reach the MBC Toolbox Help Roadmap by selecting

Help > MBC Help. The context help takes you to relevant Help pages, and Help > About MBCshows the version notes. See “Help Menu” on page 2-10.

Test Plan Menu.

• Set Up Inputs — See “Input Factor Setup” on page 5-4.

• Set Up Model — See “Local Model Setup” on page 5-6 and “Global Model

Setup” on page 5-67.

You can also reach these functions by double-clicking the input and model blocks in the test p lan diagram, and both can only be used when a Model block is first selected in the diagram. You must specify the model to set up, local or global.

2-26

• Design Experiment —See“TheDesignEditor”onpage3-2.

This is also available in the toolbar and in the right-click context menu on the blocks in the test plan diagram.

• Boundary Constraints — Opens the Constraint Modeling window. Also

available in the toolbar. See “Boundary Model Setup” on page 5-48.

• New Data — Opens the Data Editor to load new data.

• Select Data — Opens the Data Selection views of the Data Editor.

YoucanreachboththesefunctionswiththetoolbarSelect Data button. If no data is selected, this button opens the Data Wizard, and if a data set is already selected, it takes you straight to the Data Selection views. See Chapter 4, “Data”.

• Validation Data Opens a wizard to select data for validation. See “Using

Validation Data” on page 6-81.

• Make Template — Opens a dialog box for saving the current test plan as

a new template, with or without designs and response models. Same as the toolbar button. See “Creating New Test Plan Templates” on page 2-18.

Test P lan L eve l

• Export Multimodels —Thisprovidesasmoothinterfacewiththe

Multimodel Tradeoff in the CAGE browser part of Model-Based Calibration toolbox. Two global inputs are required (normally speed and load). This item is only enabled if you have set up a two-stage model with the correct number of inputs. This useful application for multiple models allows you to ca l ibr ate from local maps. See “Local Model Class: Mu l tiple Models” on page 5-24 for details.

View Menu (Test Plan Level).

• Design Data — Opens the Design Editor. The view design facility enables

you to investigate the statistical properties of the collected data. This provides access to all the Design Editor and design evaluation utility functions with the current design rather than the prespec ified design (after data matching, the data points are used as the new design points). See “The Design Editor” on page 3-2.

For two-stage models, viewing level 1 designs creates a separate design for each test.

• Model — Opens a dialog box showing the terms in the current model.

• Both of these are only available when a model or input block is selected in

the test plan block diagram.

Designing Experiments

You can design experiments after setting up models. You can design experiments for both stages, local and global. You invoke the Design Editor in several ways from the test plan level:

• Right-clic k a Model block in the test plan diagram and select

Experiment

You must select (by clicking) a stage to design for (first or second stage) or the following two options are not possible.

• Click the Design Experiment toolbar button

• Select TestPlan > Design Experiment.

For an existing design, View > Design Data also launches the Design Editor (alsointheright-clickmenuoneachModelblock). Inthiscaseyoucanonly

Design

2-27

2 Projects and Test Plans

view the current data being used as a design at this stage. If you enter the Design Editor b y the other routes, you can view a ll alternative designs for that stage.

See Chapter 3, “Designs”.

Viewing Designs

The view design facility enables the user to investigate the statistical properties of the current data.

From the test plan node, select the model stage you are interested in by clicking, then choose View > Design Data. Alternatively, use the right-click menu on a Model block.

This provides access to all the Design Editor and design evaluation utility functions w ith the current data rather than the prespecified design. If you have done some data-matching to a design, each data point is used as a design point. You can now investigate the statistical properties of this design.

2-28

For two-stage models, viewing stage one (local model) designs creates a separate design for each test.

See“TheDesignEditor”onpage3-2orthestep-by-stepguidein“Tutorial: Design of Experiment” in the Getting Started d ocumentation.

Designs

This section discusses the following topics:

• “The Design Editor” on page 3-2

• “Creating a Classical Design” on page 3-16

• “Creating a Space-Filling Design” on page 3-19

• “Creating an O ptimal Design” on page 3-31

• “Adding and Editing D esign Points” on page 3-42

• “Merging Designs” on page 3-45

• “Fixing, Deleting, and Sorting Design Points” on page 3-47

• “Exporting and Importing Designs” on page 3-50

• “Applying Constraints” on page 3-51

• “Prediction Error Variance Viewer” on page 3-63

• “Design Evaluation Tool” on page 3-71

3 Designs

The Design Editor

In this section...

“Introducing the Design Editor” on page 3-2

“Opening the Design Editor” on page 3-2

“Design Styles” on page 3-4

“Design Editor Displays” on page 3-5

“Design Editor Toolbar and Menus” on page 3-9

Introducing the Design Editor

The Design Editor provides prebuilt standard designs to allow a user with a minimal knowledge of the subject to quickly create experiments. You can applyengineeringknowledgetodefinevariablerangesandapplyconstraints to exclude impractical points. You can increase modeling sophistication by altering optimality criteria, forcing or removing specific design points, a nd optimally augmenting existing designs with additional points.

3-2

There is a step-by-step guide to using the Design Editor in “Tutorial: Design of Experiment” in the Getting Started documentation.

Opening the Design Editor

YoumustfirsthaveatestplanbeforeyoucanopentheDesignEditor.

1 From the startup (project) view of the Model Browser, click New and select

a one or two-stage test plan. See “Project Level: Startup View” on page 2-2 and “Test Plans” on page 2-17 in the Modeling section.

You can design experiments at both stages, for local models and global models; f or most two-stage models the global model is most appropriate for design of experiment.

fore you design an experiment we recommend that you set up your input

2 Be

riables, by double-clicking the Inputs blocks on the test plan diagram.

e“InputFactorSetup”onpage5-4.

The Design Editor

You can choose the number of inputs for your model and set up their names and definitions, then you can design an experiment to collect data. It is much easier to understand your design points if they are labeled with the factor names. Also, if you do not set up model inputs first, then you can only create designs for the default number of variables (one).

3 If you want to use optimal designs, then the type of model you are going to

use to fit the data is important, and you should choose a model type before opening the Desig n Editor. Double-click a model block in the test plan diagram to set up model types. Optimal designs are best for cases with high system knowledge, where previous studies have given confidence on the best type of model to be fitted, so in these cases you should pick your model type before designing an experiment. See Chapter 6, “Selecting M ode ls” to find out about model types in the Model-Based Calibration Toolbox product.

Ifyouhavenoideawhatmodelyouaregoingtofit,chooseaspace-filling design. Model type has no effect on designs that are space-filling or classical, so if you want to create these designs you can leave the model type at the default and open the Design Editor.

You can invoke the Design Editor in several ways from the “Test Plan Level” on page 2-21:

1 First you must select the stage (first/local or second/global) for which you

want to design an experime n t. Click to select the appropriate model block in the test plan diagram.

2 Right-click the model block and select Design Experiment.

Alternatively, click the Design Experiment toolbar icon

You can also select TestPlan > Design Experiment.

For an existing design, View > Design Data also launches the Design Editor (also in the right-click menu on each Model block). This shows the selected data as a design.

3-3

3 Designs

Design Styles

The Design Edito You can make thre optimal.

r provides the interface for building expe rim ental designs.

e different styles of design: classical, space-filling, and

Classical des suitable for s Design” on pag

Space-filli cases where y constraint as to maximi “Creating a

Optimal de previous s and the co Optimal D

You can a allows n second a Design P

igns (including full factorial) are very well researched and are imple reg ions (hypercube or sphere). See “Creating a Classical

ng designs are better when there is low system knowledge. In

s are uncertain, space-filling designs collect data in s uch as a way

ze coverage of the factors’ ranges as quickly as possible. See

Space-Filling Design” on page 3-19.

signs are best for cases with high system knowledge, where

tudies have given confidence in the best type of model to be fitted,

nstraints of the system are well understood. See “Creating an

esign” on page 3-31.

ugment any design by optimally adding points. Working in this way

ew experiments to enhance the original, rather than simply being a

ttempt to gain the necessary kno wledge. See “Adding and Editing

oints” on page 3-42.

e3-16.

ou are not sure what type of model is appropriate, and the

3-4

The Design Editor

Design Editor Di

The follo wing ex

ample shows the display after creating an optimal design.

splays

When you first create or open a design, the main display area shows the default Design Table view of the design (see example above ). All the views on the right show the design selected in the left tree (see “The Design Tree” on page 3-7). There is a context menu for the views on the right, available by right-clicking the title bars, in which you can change the view of the design to 1-D, 2-D, 3-D, 4-D,andPairwise Projections, 2-D,and3-D Constraint views, and the Table view (also under View menu). This menu also allows you to split the display either horizontally or vertically so that you simultaneously have two different views on the current design. The split can also be merged again. You can also use the toolbar buttons. After splitting, each view has the same functionality; that is, you can continue

3-5

3 Designs

to split views until you ha v e as many as you want. When you click a view, its title bar becomes blue to show it is the current active view. See “Design Editor Toolbar and Menus” on page 3-9 for more information about how to change your display options.

The information pane, bottom left, displays pieces of information for the current design selected in the tree. The amount of information in this pane can change depending on what the design is capable of; for example, only certain models can support the optimal designs and only these can show current optimal values. You can also see this information and more by selecting File > Properties or using the context menu in the tree.

The Design Editor can display multiple design views at once, so while working on a design you can keep a table of design points open in one corner of the window, a 3-D projection of the constraints below it, and a 2-D, 3-D, or pairwise plot of the current design points as the main plot.

3-6

The following example shows several views in use at once.

The Design Editor

The Design Tree

The currently available designs are displayed on the left in a tree structure.

The tree displays three pieces of information:

• The name of the design, which you can edit by clicking it

• The state of the design

3-7

3 Designs

- The icon changes from if it is empty, to the appropriate icon for the

design type when it has design points (for ex ample, the toolbar buttons for Optimal, Classical, and Space-Filling designs).

optimized, as in

- The icon changes to when design points have been added using a

different method (for exam ple, augmenting a classical design with optimally chosen points). It becomes a custom de sign sty le . You can mix and match all design options in this way.

- A padlock appears ( ) if the design is locked. This happens when it

has child nodes (to maintain the relationship between designs, so you can retreat back up the design tree to reverse changes).

• The design that is selected as best. This is the default design that is used

for matching against experimental data. The icon for the selected design is the normal icon turned blue. When you have created more than one design, you should select as best the design to be used in modeling, using the Edit menu. Blue icons are also locked designs, and do not acquire padlocks when they have child nodes.

3-8

• You can reach a context menu by right-clicking in the design tree pane.

Here you can delete or rename designs and add new designs. Choose Evaluate Design to open the Design Eva lu ation window. Properties opens the Design Properties dialog box, which displays information about the size, constraints, properties (such as optimality values), and modification dates of the selected design.

The Design Editor

Design Editor To

Sort Points Delete Point Add Point

New Design Delete Design Print

• New Design, Delete Design, Print — See the “File Menu” on page 3-9.

• Add, Delete and Sort Point — See the “Edit Menu” on page 3-11.

• Classical, Space Filling and Optimal Design —Seethe“DesignMenu”

on page 3-14.

• Split View Horizontally and Vertically—Seethe“ViewMenu”onpage

3-12.

File Me

olbar and Menus

Split View Vertically

Split View Horizontally

Help

Optimal Design Space Filling Design Classical Design

• New De

the fi is a co the de

• Dele

Also

• Rena

You rig

• Pro

e information about your current design, such as the number of factors,

se po

sign — Creates a new design node in the tree. This is blank if it is

rst design you create, or, if you have an existing design, the child node

py of the p arent design. A lso in the toolbar and the context menu in sign tree. Use the Design menu or toolbar buttons to set up designs.

te Design — Deletes the currently selected design and its subdesigns. in the toolbar and the context menu in the design tree.

me Design — Enables you to edit the name of the current design.

can also do thi s by clicking again on the selected design name, or

ht-click to use the context menu in the design tree.

perties — Opens the Design Propertiesdialogbox,whereyoucan

ints, and constraints; the design style; when the design was last m odified;

3-9

3 Designs

optimality values, space filling, and classical settings. Also in the context menu in the design tree.

• Import Design — Opens the Import Design dialog box, where you can

import designs from Design Editor files ( files (

*.csv), or from the workspace. You can browse to the required file or

*.mvd), comma-separated-values

specify the source variable if importing from the workspace. If it is not a Design Editor file you can choose to convert the design points from [-1, 1] range. See “Exporting and Importing Designs” on page 3-50.

• Export Design —OpenstheExportDesigndialog box, where you can

export designs to Design Editor files ( files (

*.csv), or from the workspace. You can specify the name of the

destination file or variable. For choose whether to include factor names, and for the workspace and

*.mvd), comma-separated-values

.csv files you can use the check box to

.csv

files you can choose whether to convert design points to [-1, 1] range. See “Exporting and Importing Designs” on page 3-50. Note that you do not have to export your designs to save them — they are saved when you save your project in the Model Browser.

• Merge Designs — Open s the Merge Designs dialog box, where you can

choose which designs to merge and a b ase design. See “Merging Designs” on page 3-45.

3-10

• Import Constraints — Opens the Import Constraints dialog box. Here

you can import any suitable constraints for the currently selected design. Youcanimportanyexistingconstraints in the design tree, or from a design file, or you can import boundary constraints from file or the current pro ject. See “Importing Constraints” on page 3-60.

• Print — Prints the current view (plots only). You can also use Print

Preview.SeealsoPrint to Figure in the View menu. If you want to

print information from the Table view you can copy the information to the clipboard by using Edit > Copy Design Data, or save the design as a

.csv file. Also in the toolbar.

• Close — Closes the Des ign Editor. You return to the Model Browser

window. Note that you do not lose your designs, you simply close the Design Editor. The designs reappear when you reopen the Design Editor. When you save your project in the Model Browser your designs remain part of that project.

The Design Editor

Edit Menu

• Copy View — Copies the current view to the clipboard.

• Copy Design Data — Copies the design data to the clipboard. This can be

useful if you want to print the contents of the Table view.

• Clear — D eletes all points in the current design.

• Add Point — Opens the Add Design Points dialog box. Here you can

choose how many points to add optimally, randomly, or at specified points. Alsointhetoolbar. See“AddingandEditing D esign Points” on page 3-42.

• Delete Point — Opens the Delete Desig n Points dialog box. Here you can

choose the points to delete. Also in the toolbar. See “Fixing, Deleting, and Sorting Design P oints” on page 3-47.

• Sort Points — Opens the Sort dialog box. Here you can choose to sort by

any or all of your factors, by custom expression , or at random. Also in the toolbar. See “Fixing, Deleting, and Sorting Design P oints” on page 3-47.

• Fix/Free Points — Opens the Fix Design Points dialo g box. You can fix

design points so they are not mov ed by design optimization processes. See “Fixing, Deleting, and Sorting Design Points” on page 3-47.

• Randomize — Select this option as a quick way of randomly resorting the

points in the current design. This is a shortcut to the same functionality provided by the Random optio n in the Sort dialog box .

• Round Factor — Opens the Round Design dialog box. Here you can select

a factor to round, and limit it to:

- A fixed interval, with optional offset (e.g., an in t erva l of 5 and an offse t

of2roundsto[... -8-3271217...])

- A specified number of significant figures

- Specified levels

• Constraints — Opens the Constraints Manager dialog box. Here you can

add, edit, duplicate, and delete constraints on your designs. See “Applying Constraints” on page 3-51.

• Model — Opens the Global or Local Model Setup dialog box (depending on

whichstageyouaredesigninganexperimentfor). Hereyoucanchange

3-11

3 Designs

the model for which you are designing an experiment. The model type is important for optim al designs.

• Select As Best — Selects the current design as best. This changes the

icon in the tree blue. This is the default design that is used for matching against experimental data.

View Menu

• Current View — Changes the current view to your selection from the

submenu:

- Design Table

- 1-D Design Projection

- 2-D Design Projection

- 3-D Design Projection

- 4-D Design Projection

3-12

- Pairwise Design Projections

- 2-D Constraints

- 3-D Constraints

- Model Description

• View Options — these items depend on the currently selected view:

- Plot Properties— For 1-D, 2-D and 3-D Design Projections. Opens

dialog boxes for configuring details of the current display. You can change basic properties such as color on the projections (1-D, 2-D, 3-D, and 4-D). You can rotate all 3-D views as usual.

- Edit Colormap For the 3-D and 4-D Design Projections. You can also

double-click the color bar to edit the colormap.

- Graph Size For the Pairwise Projections, you can choose graph size or

to display all graphs.

- Value Filter — For the table view, you can set up a filter to selectively

display certain ranges of values.

The Design Editor

• Display Design Point Numbers — You can select this option to toggle

the display of design point numbers in views that support the feature. A design point number is the index of a particular point in the design: this value is permanently displayed in the table view. Views that support the display of design point numbers are

- 2-D Design Projection

- 3-D Design Projection

- 4-D Design Projection

- Pairwise Design Projections

Becausealltheseviewsareprojectionsthatuseasubsetofthedesign’s input factors, it is often the case that the resulting view contains points that have been plotted on top of each other. In this case, the design point numbers will stack up in a column above the common point to aid readability. You can use Display Design Point Count to see at a glance how many points are overlapping in any stack. You can select point count orpointnumbersbutnotboth.

Note Displaying multiple views with design point numbers for large designs can significantly slow down the display. Yo u might wan t to turn off the design poin t number display in these cases.

• Display Design Point Count — If points are plotted on top of each other

(in 2-D, 3-D, 4-D, or pairwise plots) this option allows you to see how many points are overlapping in each cluster. A number next to a point indicates that more than one point is plotted there.

• Split View — splits the current view and adds your selected n ew view

from the submenu.

• Split View Vertically — Splits the current view vertically to produce a

new view in addition to the currently selected view. Also in the toolbar and the buttons in the title bar of each view. New plot types are produced for each new view. T his is a quick way to produce a variety of different plots. Remember that you can change any existing view type by selecting View > Current View or, alternatively, using the context menu to select from the Current View submenu.

3-13

3 Designs

• Split View Horizontally — Splits the current view horizontally to

produce new views, as for Split Vertically. Also in the toolbar and the buttons in the title bar of each view.

• Close View — Deletes the current view.

• Print to Figure — This option copies the current view into its own figure,

allowingyoutousethestandardMATLAB plotting tools to annotate and print the display.

Design Menu

• Optimal — Opens the Optimal Design dialog box. Also in the toolbar. See

“Creating an Optimal Design” on page 3-31.

• Classical — You can use the submenu here to go directly to the type of

classical design you want, or select Design Browser toseealltheoptions. See “Creating a Classical Design” on page 3-16. The toolbar button opens the Design Browser.

• Space Filling — You can use this subm enu to go directly to the type of

space filling design you want, or select Design Browser to see all the options. See “Creating a Space-Filling Design” on page 3-19. The toolbar button opens the Design Browser.

3-14

Tools Menu

• Prediction Error Variance Viewer — Opens the Prediction Error

Variance Viewer where you can evaluate the predictive power of your designs. See “Prediction Error Variance Viewer” on page 3-63.

• Evaluate Designs — Opens the Design Evaluation window where you

can e xamine detailed mathematical properties of your design. Also in the context menu in the design tree. See “Design Evaluation Tool” on page 3-71.

Window Menu

This allows yo u to switch between the Model Browser and Design Editor windows.

The Design Editor

Help Menu

As w ith everywhere in the toolbox, the Help menu provides access to general toolbox help and help specific to the current view. Here you can select MBC Help to browse all the toolbox help, Design Editor Help to go straight to the Designs documentation, or About MBC to see the current version number.

3-15

3 Designs

Creating a Classical Design

1 Add a new design by clicking the button in the toolbar or select

File > New.

2 Select the new design node in the tree. An empty Design Table appears

if you have not yet chosen a design. Otherwise if this is a new child node the display remain s the same, because child nodes inherit all the parent design’s properties. All the points from the previous design remain, to be deleted or added to as necessary. The new design inherits all its initial settings from the currently selected design and becomes a child node of that design.

3 Click the button in the toolbar or select Design > Classical > Design

Browser.

Note In cases where the preferred type of classical design is known, you can go straight to one of the five options under Design > Classical. Choosing the Design Browser option allows you to see graphical previews of these same five options before making a choice.

3-16

4 A dialog box appears if there are alread y points from the previous design.

You must choose between replacing and adding to those points or keeping only fixed points from the design. The default is replacement of the current points with a new design. Click OK to proceed, or Cancel to change your mind.

The Classical Design Browser appears.

Creating a Classical Design

In the Design Style drop-down menu there are five classical design options:

•

Central Composite

Generates a design that has a center point, a point at each of the design volume corners, and a point at the center of each of the design volume faces. The options are Face-center cube, Spherical, Rotatable,orCustom. If you choose Custom, you can then choose a ratio value (

) between the corner points and the face points for each factor and the number of center points to add. Five levels for each factor are used. You can set the ranges for each factor. Inscribe star points scales all points within the coded values of 1 and -1 (instead of plus or minus

outside that range). When

this box is not selected, the points are circumscribed.

•

Box-Behnken

Similar to Central Composite designs, but only three levels per factor are required, and the design is always spherical in shape. All the design points (except the center point) lie on the same sphere, so you should choose at least three to five runs at the center point. There are no face points. These

3-17

3 Designs

designs are particularly suited to spherical regions, when p rediction at the corners i s not required. You can set the rang es of each factor.

•

Full Factorial

Generates an n-dimensional g rid of points. You can choose the number of levels for each factor, the number of additional center points to add, and the ranges for each factor.

•

Plackett Burman

These are “screening” designs. They are tw o-level designs that are designed to allow you to work out which factors are contributing any effect to the model while using the minimum number of runs. For example, for a 30-factor problem this can be done with 32 runs. They are constructed from Hadamard m atrices and are a class of two-level orthogonal array.

•

Regular Simplex

These designs are generated by taking the vertices of a k-dimensional regularsimplex(k=numberoffactors). Fortwofactorsasimplexisa triangle; for three it is a tetrahedron. Above that are hyperdimensional simplices. These are economical first-order designs that are a possible alternative to Plackett Burman or full factorials.

3-18

Creating a Space-Filling Design

In this section...

“Introducing Space-Filling Designs” on page 3-19

“Setting Up a Space-Filling Design” on page 3-20

“Halton Sequence” on page 3-21

“Sobol Sequence” on page 3-22

“Latin Hypercube S amp lin g” on page 3-23

“Lattice” on page 3-24

“Stratified Latin Hypercube” on page 3-25

“Augmenting Space-Filling Designs” on page 3-26

Introducing Space-Filling Designs

Space-filling designs should be used when there is little or no information about the underlying effects of factors on responses. For example, they are most useful when you are faced with a new type of engine, with little knowledge of the operating envelope. These designs do not assume a particular model form. The aim is to spread the points as evenly as possible around the operating space. These designs literall y fill out the n-dimensional space with points that are in some way regularly spaced. These designs can be especially useful in conjunction with nonparametric models such as radial basis function (a type of neural network).

Creating a Space-Filling Design

1 Add a new design by clicking the button in the toolbar or select

File > New.

2 Selectthenodeinthetreebyclicking. An empty Design Table appears if

you have not yet chosen a design. Otherwise, if this is a new child node the display remain s the same, because child nodes inherit all the parent design’s properties.

3 Select Design > Space Filling > Design Browser,orclicktheSpace

Filling Design button

on the toolbar.

3-19

3 Designs

4 A dialog box appears if there are alread y points from the previous design.

The Space Filling Desig n Browser appears.

Note As with the Classical Design Browser, you can select the types of design you can preview in the Space Filling Design Browser from the Design > Space Filling menu in situations when you already know the type of space-filling design you want.

3-20

ting Up a Space-Filling Design

Set

all design types, you can edit these settings:

For

Creating a Space-Filling Design

• Select from the Design type drop-down menu to choose a space-filling

design style.

The default Design type is

• You can set the Number of points by typing in the edit box or using the

controls.

Observe the information displayed above the preview to see how many points are excluded by constraints.

Alter the number of points until the Size of constrained design displays the number of points you want.

• You can use the tabs under the display to view 2-D, 3-D, and 4-D previews.

The preview is identical to the final design.

When you edit settings for very large designs, you can clear the check box Automatically update preview to avoid waiting for the preview calculation. This check box is cleared automatically if the current design is large enough to cause preview calculation to be very slow. You can click the Generate button when you want to create a preview.

• You can set the ranges for each factor.

• For settings for specific design types, see the sections for each type:

Halton Sequence.

- “Halton Sequence” on page 3-21

- “Sobol Sequence” on page 3-22

- “Latin Hypercube Sampling” on page 3-23

- “Lattice” on page 3-24

- “Stratified Latin Hypercube” on page 3-25

• Click OK to calculate the design and return to the main Design Editor.

Halton Sequence

Halton Sequence designs are generated from the haltonset class in the Statistics Toolbox™ software. The Halton sequence is a low-discrepancy point set w here the coo rdinate values for each dimension are generated by forming the radical inverse of the point’s index, using a different prime base for each dimension. For more information see the Statistics Toolbox documentation.

3-21

3 Designs

Settings

For Halton sequence designs, you can choose the following options:

• Leap sequence points using prime number —Usesonlyeveryk-th

point in the Halton sequence. k is the next prime number after those used as bases in the radical inverse; i.e., this value is the (NFactors+1) prime number.

This property at the command line is

• Skip zero point — Causes the first point of the sequence, which is always

at the lower limit of each input factor, to be skipped. This point is often seen as unbalancing because the upper limits of each input factor can never be produced by the algorithm.

This property at the com mand line is

• Apply RR2 Scramble — Sets the scramble to

permutation of the radical inverse coefficients using the RR2 a lgorithm.

This property at the com mand line is

PrimeLeap.

SkipZero.

'RR2', which performs a

Scramble.

Sobol Sequence

Sobol sequence designs are generated from the sobolse t class in the Statistics Toolbox software. The Sobol sequence is a low-discrepancy (t,s)-sequence in base 2. For more information see the Statistics Toolbox documentation.

Settings

For Sobol sequence designs, you can choose the following options:

• Use the radio buttons to specify whether and how to skip initial points

from the sequence:

- No skip — Do not skip any points.

3-22

This property at the command line is

SkipMode 'None'.

- Skip initial 2^k points — Automatically chooses the smallest value

for k so that 2^k is larger than the number of points requested, and then skip 2^k points.

This property at the command line is

SkipMode '2^k'.

Creating a Space-Filling Design

- Custom skip — Enter a value to be used as the number of initial points

to miss out from the sequence.

This property at the command line is

Numberofpoints

• Apply Matousek Affine Owen scramble — Performs a linear scramble

of the generator matrices for the sequence using random lower-triangular matrices in base 2 and also applies a random digital shift to the points.

This property at the com mand line is

SkipMode, 'Custom', Skip,

Scramble.

Latin Hypercube Sampling

Latin Hypercube Sampling (LHS) designs are sets of design points that, for an N point design, project onto N different levels in each factor. In this design, the points are generated randomly. You choose a particular Latin Hypercube by trying several such sets of randomly generated points and choosing the one that best satisfies user-specified criteria.

Settings

For both Latin H ypercube Sampling and Stratified Latin Hypercube, you can choose the following options:

• The Selection criteria drop-down menu has these options:

- Maximize minimum distance (between points).

- Minimize maximum distance (between points)

- Minimize discrepancy — Minimizes the deviation from the average

point density

- Minimize RMS variation from CDF — This default o ption minimizes

theRootMeanSquare(RMS)variation of the Cumulative Distribution Function (CDF) from the ideal CDF.

- Minimize maximum variation from CDF — Minimizes the maxim um

variation of the CDF from the ideal CDF

The final two (CDF variation) options scale best with the number of points and it is advisable to choose one of these options for large designs.

3-23

3 Designs

• The Enforce Symmetrical Points check b ox is selected by default. T his

creates a design in which every design point has a mirror design point on the opposite side of the center of the design volume and an equal distance away. Restricting the design in this way tends to produce better Latin Hypercubes.

Lattice

Lattice designs project onto N different levels per factor for N points. The points are not randomly generated but are produced by an algorithm that uses a prime number per factor. If good prime numbers are chosen, the lattice spreads points evenly throughout the design volume. A poor choice of prime numbers results in highly visible lines or plan es in the design projections. If all the design points are clustered into one or tw o planes, it is likely that you cannotestimatealltheeffectsinamorecomplexmodel. Whendesignpoints are projected onto any axes, there are a large number of factor levels.

For a small number of trials (relative to the number of factors) LHS designs are preferred to Lattice designs. This is because of the way Lattice designs are generated. Lattice designs use prime numbers to generate each successive sampling for each factor in a different place. No two factors can have the same generator, because in such cases the lattice points all fall on the main diagonal of that particular pairw ise projection, creating the visible lines or planes described above. When the numb er of points is small relative to the number of factors, the choice of generators is restricted and this can lead to Lattice designs with poor projection properties in some pairwise dimensions, in which the points lie on diagonals or double or triple diagonals. This means that Latin Hypercube designs are a better choice for these cases.

3-24

See the illustrations in the following section comparing the properties of good and poor lattices and a hypercube design.

Settings

For a Lattice space-filling design, you can choose:

• The Lattice size by using the buttons or typing in the edit box.

• The prime number generator by using the up/down buttons on the Prime

number for X edit box.

• The range for each factor.

Creating a Space-Filling Design

Stratified Latin Hypercube

Stratified Latin Hypercubes separate the normal hypercube into N different levels on user-specified factors. This can be useful for situations where the preferred number of levels for certainfactorsmightbeknown;moredetail mightberequiredtomodelthebehaviorofsomefactorsthanothers. They can als o be useful when certain factors can only be run at given levels.

The preceding example shows the different properties of a poor lattice (left) and a good lattice (right), with a similar number of points. The poorly chosen prime number produces highly visible planes and does not cover the space well.

3-25

3 Designs

AnexampleofanLHSdesignofthesamesizeisshownforcomparisonwith the preceding lattice examples.

3-26

Settings

See“LatinHypercubeSampling”onpage3-23,theoptionsarethesame.

Comparing Latin Hypercube and Stratified Latin Hypercube

Latin Hypercube Sampling and Stratified Latin Hypercube Sampling d i ffer only in that with Stratified Latin Hypercube Sampling, you can restrict the number of levels available to each factor. If the number of stratifications equals the number of points in the design, then both Latin Hypercube Sampling and Stratified Latin Hypercube Sampling give the same results. However, if the number of stratifications in a given factor is less than the number of points in the design, then some points will be projected onto the same values in that factor. You can see this change by using the one-dimensional design projection view in the Design Editor.

Augmenting Space-Filling Designs

You can progressively augment Halton and Sobol sequence space-filling designstoaddpointswiththesamesequenceparameters. Thefollowing

Creating a Space-Filling Design

method allows you to add points to your original space-filling sequence, preserving the original points and adding new ones with the same sequence parameters.

To augment a space-filling design without losing existing points or space-filling properties:

1 Create a Halton or Sobol sequence space-filling design.

Warning If you want to add constraints, you must first c reate

a child design and constrain that. If you do not create a child design, when you add constraints you lose the original space-filling settings and then you cannot augment the sequence. With constraints, the design type changes to

Custom and you cannot

access the original sequence settings to add new points.

2 To preserve your original design when you add new points, create a child

design of your original unco nstrained design. Select your design, and click

the New Design

The new c

3 Select File > Properties.

hild design, identical to the parent, is selected in the tree.

button in the toolbar, or select File > New.

The Design Properties dialog box opens.

4 Select

the Space-Filling tab.

3-27

3 Designs

3-28

5 Enter the desired new total number of points in the Number of points

edit box.

Leave the other settings unchanged.

6 Click OK.

The Design Editor augments your original design by adding points up to the new total number of points, with the same space-filling sequence parameters as your original Halton or Sobol sequence.

Creating a Space-Filling Design

7 Use the pairwise view, switching between the parent and child designs to

visually verify that the new points have been added while preserving your original points.

8 Create a copy child design to add constraints. Preserving your

unconstrained parent design allows you to perform another augmentation iteration later if you need to.

9 New points are added to the end of the list of existing points. If you want to

extract only the augmented points for testing, select Edit > Delete Point to open a dialog box in which you can choos e the points to delete. See “Fixing, Deleting, and Sorting Design Points” on page 3-47.

10 Round and sort your data before sending it out for testing. Select

Edit > Round Factor to limit decimal places of factor s. Select Edit > Sort to sort the points for test efficiency, because operators often test in order of speed f ollowe d by load. See “Edit Menu” on page 3-11 and “Fixing, Deleting, and Sorting Design Points” on page 3-47.

Augmentation Restrictions

You can only augment Halton and Sobol sequence space-filling designs with this method that uses the original sequence settings.

You can augment any Halton sequence, but for Sobol sequences, you must use the default No skip setting.

You cannot achieve the same result by selecting any Design > Space Filling menu option and selecting the Augment option, because doing so will keep your existing points, but will generate the additional points with a new space-filling sequence.

You cannot augment a constrained design in this way. When you add constraints, the design type changes to the original sequence settings on the Space-Filling tab. If you wan t to add constraints, you must create a child design and constrain it. This approach preserves the original space-filling sequence design.

You cannot use this method with Latin H ypercube or Lattice designs as they always create a completely new design.

Custom. You can no longer access

3-29

3 Designs

You cannot achieve the same result by using the Add Design Points dialog box because that dialog box only allows you to add optimal, custom, or random points.

3-30

Creating an Optimal Design

In this section...

“Introducing Optimal Designs” on page 3-31

“Optimal Design: Initial Design Tab” on page 3-33

“Optimal Design: Candidate Set Tab” on page 3-35

“Optimal Design: Algorithm Tab” on page 3-38

“Averaging O ptimality Across Multiple Models” on page 3-40

Introducing Optimal Designs

Optimal designs are best for cases with high system knowledge, where previous studies have given confidence on the best type of model to be fitted, and the constraints of the system are well understood. Optimal designs require linear models.

Creating an Optimal Design

1 Click the button in the toolbar or select File > New Design.Anew

node appears in the design tree. It is named according to the model for which you are designing, for example,

2 Selectthenodeinthetreebyclicking. An empty Design Table appears if

you have not yet chosen a design. Otherwise, if this is a new child node the display remain s the same, because child nodes inherit all the parent design’s properties.

3 Set up any constraints at this point. See “Applying Constraints” on page

3-51.

4 Choose an Optimal design by clicking the button in the toolbar, or

choose Design > Optimal.

The optimal designs in the Design Editor are formed using the following process:

• An initial starting design is chosen at random from a set of defined

candidate points.

Linear Model Design.

3-31

3 Designs

• p additional points are added to the design, either optimally o r at random.

These points are chosen from the candidate set.

• p points are deleted from the design, either optimally or at random.

• If the resulting design is better than the original, it is kept.

This process is repeated until either (a) the maximum number of iterations is exceeded or (b) a certain number of iterations has occurred without an appreciable change in the optimality value for the design.

The O p tima l Design d ia log box con sists of three tabs that contain the settings for three main aspects of the design:

• Initial Design tab: Starting point and number of points in the design

• Candidate Set tab: Candidate set of points from which the design points

are chosen

• Algorithm tab: Options for the algorithm that is used to generate the points

3-32

Creating an Optimal Design

Optimal Design:

Initial Design Tab

The Initial Design tab allows you to define the composition of the initial design: how many points to keep from the current design and how many total or additional points to choose from the candidate set.

1 Choose the type of the optimal design, using the Optimality criteria

drop-down menu:

•

D-Optimal designs — Aims to reduce the volume of the confidence

ellipsoid to obtain accurate coefficients. This is set up as a maximization problem, so the progress graph should go up w ith time.

3-33

3 Designs

The D-optimality value used is calculated using the formula

whereXistheregressionmatrixandkisthe

number of terms in the regression matrix.

•

V-Optimal designs — Minimizes the average prediction error variance,

to obtain accurate predictions. This is better for calibration modeling problems. This is a minimization process, so the progress graph should go down with time.

The V-optimality value is calculated using the formula

where xjare rows in the regression matrix, XCis the regression matrix for al l candidate set points, and n

A-Optimal designs — Minimizes the average variance of the parameters

•

is the number of candidate set points.

and reduces the asphericity of the confidence ellipsoid. The progress graph also goes down with this style of optimal design.

3-34

The A-optimality value is calculated using the formula

where X is the regression matrix.

2 You might already have points in the design (if the new design node is a

child no de, it inherits all the properties of the parent design). If so, choose from the radio buttons:

• Replace the current points with a new initial design

• Augment the current design with additional points

• Keep only the fixed points from the current design

For information on fixed design points, see “Fixing, Deleting, and Sorting Design Points” on page 3-47.

Creating an Optimal Design

3 You can choose the total number of points and/or the number of additional

points to add by clicking the up/down buttons or by typing directly into the edit boxes for Optional additional points or Total number of points.

Optimal Design: Candidate Set Tab

The Candidate Set tab allows you to set up a candidate set of points for your optimal design. Candidate sets are a set of potential test points. This typically ranges from a few hundred points to several hundred thousand.

3-35

3 Designs

Select variables in this list

Choose algorithm type from this list

3-36

Open display windows with these buttons

Change the number of levels of the selected variable here

Creating an Optimal Design

The set generation schemes are as follows:

•

Grid — Full factorial grids of points, with fully customizable levels.

Grid/Lattice — A hybrid set where the m ain factors are used to generate

•

a lattice, which is then replicated over a small number of levels for the remaining factors.

•

Halton Sequence — H alto n Sequence designs are generated from the haltonset class in the Statistics Toolbox software. See “Halton Sequence ”

on page 3-21 for more information

•

Lattice — These have the same definition as the space-filling design

lattices, but are typically used with about 10,000 points. The advantage of a lattice is that the number of points does not increase as the number of factors increases; however, you do have to try different prime number generators to achieve a good lattice. See “Lattice” on page 3-24.

•

Sobol Sequence — Sobol sequence designs are generated from the sobolset class in the Statistics Toolbox software. See “Sobol Sequence”

on page 3-22 for more information.

•

Stratified Lattice — Another method of using a lattice when some

factors cannot be set to arbitrary values. Stratified lattices ensure that the required number of levels is present for the chosen facto r. Note that you cannot set more than one factor to stratify to the same N levels. This is becauseforcingthesamenumberoflevels would also force the factors to have the same generator. As for a lattice space-filling design, no two factors can have the same generator, because in such cases the lattice points all fall on the main diagonal of that particular pairwise projection, creating highly visible planes in the points and poor coverage of the space. For illustrations of this effect, see “Lattice” on page 3-24.

•

User-defined — Import custom matrices of points from MATLAB software

or MAT-files.

Foreachfactoryoucandefinetherange and number of different levels within that range to select points.

1 Choose a ty pe of generation algorithm from the drop-down menu. N ote

that you could choose different parameters for different factors (within an overall scheme such as

Grid).

3-37

3 Designs

2 This tab also has buttons for creating plots of the candidate sets. T ry

them to preview your candidate set settings. If you have created a custom candidate set you can check it here. The edit box sets the maximum number of points that will be plotted in the preview windows. Candidate sets w ith many factors can quickly become very large, and attempting to display the entire set will take too long. If the candidate set has more points than you set as a maximum, only every

N is chosen such that (a) the total displayed is less than the maximum and

(b)

N is prime. If you think that the candidate set preview is not displaying

Nth point is displayed, where

an adequate representation of your settings, try increasing the maximum number of points displayed.

3 Notice that you can see 1-D, 2-D, 3-D, and 4-D displays (fourth factor is

color) all at the sam e time as they appear in separate windows (see the example following). Move the display windows (click and drag the title bars) so you can see them while changing the number of levels for the different factors.

4 You can change the factor ranges and the number of levels using the edit

boxes or buttons.

3-38

Optimal Design: Algorithm Tab

The Algorithm tab h as the following algorithm details:

• Augmentation method —

slow (searches the entire candidate set for points) but converges using fewer iterations. Random is much faster per iteration, but requires a larger number of iterations. The Random setting does also have the ability to lower the optimal criteria further when the Optimal setting has found a local minimum.

• Deletion method —

Random or Optimal— Optimal deletion is much faster

than augmentation, because only the design points are searched.

• p — number of points to alter per iteration — The number of points

added/removed per iteration. For optimal augmentation this is best kept smaller (~5); for optimal deletion only it is best to set it larger.

• Delta — value below which the change in optimality criteria

triggers an increment in q — This is the size of change below which

changes in the optimality criteria are considered to be not significant.

Random or Opt imal —Optimalcanbevery

Creating an Optimal Design

• q — number of consecutive non-productive iterations which

trigger a stop — Number of consecutive iterations to allow that do not

increase the optimality of the design. This only has an effect if random augmentation or deletion is chosen.

• Maximum number of iterations to p erform — Overall maximum

number of iterations.

1 Choose the augmentation and deletion methods from the drop-down menus

(or leave at the defaults).

3-39

3 Designs

2 You can alter the other parameters by using the buttons or typing directly

in the edit boxes.

3 Click OK to start optimizing the design.

When you click the OK buttonontheOptimalDesigndialogbox,another window appears that contains a grap h. This window shows the progress of the optimization and has two b uttons: Accept and Cancel. Accept stops the optimization early and takes the current design from it. Cancel stops the optimization and reverts to the original design.

4 You can click Accept at any time, but it is most useful to wait until

iterations are not producing noticeable improvements; that is, the graph becomes very flat.

3-40

You can always return to the Optimal Design dialog box (following the same steps) and choose to keep the current points while adding more.

Averaging Optimality Across Multiple Models

The Design Editor can a verage optimality across several linear models. This is a flexible way to design experiments using optimal designs. If you have no idea wh at model you are going to fit, you would choose a space-filling design. However, if you have some idea what to expect, but are not sure which model

Mathworks MODEL-BASED CALIBRATION TOOLBOX 4 CAGE user guide

Specifications and Main Features

Frequently Asked Questions

User Manual

Workflows For Modeling

What Is the Model Browser?

How to Use This Manual

HowtoSetUpaOne-StageModel

HowtoSetUpaTwo-StageModel

How to Set Up a Point-by-Point Model

Point-by-Point Modeling Process

Use Cases for Point-by-Point Models

Creating Alternative Models to Compare

Projects and Test Plans

Project Level: Startup View

Project View

All Models Pane

Data Sets Pane

Notes Pane

Tip of the Day

Project Level: Toolbar

Project Level: Menus

File Menu

Data Menu

View Menu

Window Menu

Help Menu

Model Tree

Navigating the Model Tree

Tree Structure

Icons: Curves, Worlds, and Houses

Icons: Blue Backgrounds and Purple Worlds

Test Plans

Creating a New Test Plan

Creating New Test Plan Templates

Saving a New Template

Using Stored Templates

Test Plan Level

Test Plan View

Test Plan Level: Menus

Designing Experiments

Viewing Designs

Designs

The Design Editor

Introducing the Design Editor

Opening the Design Editor

Design Styles

The Design Tree

Edit Menu

View Menu

Design Menu

Tools Menu

Window Menu

Help Menu

Creating a Classical Design

Creating a Space-Filling Design

Introducing Space-Filling Designs

Halton Sequence

Settings

Sobol Sequence

Settings

Latin Hypercube Sampling

Settings

Lattice

Settings

Stratified Latin Hypercube

Settings

Comparing Latin Hypercube and Stratified Latin Hypercube

Augmenting Space-Filling Designs

Augmentation Restrictions

Creating an Optimal Design

Introducing Optimal Designs

Optimal Design: Candidate Set Tab

Optimal Design: Algorithm Tab

Averaging Optimality Across Multiple Models