SAP Match/Consolidate 8.00c Job-File Reference

Match/Consolidate

Job-File Reference

Match/Consolidate 8.00c

April 2009

BusinessObjects, Crystal Reports®, SAP BusinessObjects Rapid Mart™, SAP BusinessObjects Data Insight™, SAP BusinessObjects Desktop Intelligence™, SAP BusinessObjects Rapid Marts®, SAP BusinessObjects Watchlist Security™, SAP BusinessObjects Web Intelligence®, and Xcelsius® are trademarks or registered trademarks of Business Objects, an SAP company and/or affiliated companies in the United States and/or other countries. SAP® is a registered trademark of SAP AG in Germany and/or other countries. All other names mentioned herein may be trademarks of their respective owners.

Match/Consolidate Job-File Reference

Preface .............................................................................................................5

Chapter 1:

Introduction to Match/Consolidate Job ...................................................... 7

Install Match/Consolidate ................................................................................8

Run a Match/Consolidate job...........................................................................9

Steps in Match/Consolidate processing..........................................................11

Run the Match/Consolidate job......................................................................13

Create Match/Consolidate jobs ......................................................................15

Match/Consolidate command line..................................................................17

Messages during verification and processing.................................................18

Template job files...........................................................................................19

Chapter 2:

Job-file blocks and parameters .................................................................. 21

General ...........................................................................................................22

Execution........................................................................................................23

Auxiliary Files................................................................................................28

Unicode Conversion.......................................................................................31

Input File ........................................................................................................33

Input List Defaults..........................................................................................37

Input List Description.....................................................................................45

Super List Defaults.........................................................................................46

Super List Description....................................................................................47

Post to Input File ............................................................................................48

Custom Purge Input File(s) ............................................................................50

Match Qualification........................................................................................52

Match Criteria ................................................................................................54

Match Options................................................................................................59

Match Set Defaults.........................................................................................69

Match Set........................................................................................................70

Combine Match Sets ......................................................................................72

Group Posting.................................................................................................75

Create File for Output ....................................................................................78

Match/Consolidate Output File ......................................................................80

All-Duplicates Output File.............................................................................85

Multi-Occurrence Output File........................................................................90

Custom Match/Consolidate Output File.........................................................95

Custom Output Sorting.................................................................................100

Report Defaults ............................................................................................101

Report: Executive Summary Report: Job Summary Report: Match Results Report: Input File Summary Report: Input List Summary Report: List Quality

Contents

Report: List-By-List Match Report: Multi-List

Report: Posted Dupe Groups ....................................................................... 105

Report: List Match

Report: List Duplicates ................................................................................ 106

Report: Output File ......................................................................................107

Report: Purge by List...................................................................................109

Report: Duplicate Records...........................................................................110

Report: Sorted Records................................................................................ 111

Report: Unparsed Records...........................................................................114

Statistics Files ..............................................................................................115

Appendix A:

Master job file (master.mpg) ....................................................................117

Appendix B:

Tips and troubleshooting...........................................................................133

Error-messages.............................................................................................134

Fine-tune your processing............................................................................136

Create output files........................................................................................ 137

Index............................................................................................................ 139

Match/Consolidate Job-File Reference

Preface

Match/Consolidate Job

Conventions

This manual provides detailed information about Match/Consolidate (MCD) job. Use this as a reference as you set up and run jobs.

For conceptual information about matching records, see the User’s Guide to Record Matching. This guide will acquaint you with the concepts of matching records and your matching records options and possibilities.

This document follows these conventions:

Convention Description

Bold We use bold type for file names, paths, emphasis, and text that you

should type exactly as shown. For example, “Type

Italics We use italics for emphasis and text for which you should substitute

your own data or values. For example, “Type a name for your file,

Menu commands

and the

We indicate commands that you choose from menus in the following format: Menu Name > Command Name. For example, “Choose File > New.”

We use this symbol to alert you to important information and potential problems.

We use this symbol to point out special cases that you should know about.

.txt

extension (

testfile

.txt

).”

cd\dirs

.”

We use this symbol to draw your attention to tips that may be useful to you.

Preface

Documentation

Documents related to this manual include the following:

Document Description

Access the latest documentation

System Administrator’s

Explains how to install your software.

Guide

Database Prep

Explains how to prepare input files for processing, including how to create DEF, FMT, and DMT files.

Match/Consolidate User’s Guide to Record Matching

Explains the concepts behind name and address matching software and provides examples of how to implement, analyze, and fine-tune match detection strategies for the best results.

Match/Consolidate Extended Matching

Contains the operational how-to instructions for setting up extended matching.

Reference

Quick Reference

Contains descriptions of the input and output fields, and the command line for the Match/Consolidate job file.

You can access documentation in several places:

 On your computer. Release notes, manuals, and other documents for

each product that you’ve installed are available in the Documentation folder. Choose Start > Programs > Business Objects Applications > Documentation.

 On the SAP Service Market Place. Go to http://help.sap.com, and then

click the Business Objects tab. Here, you can search for your products’ documentation.

Match/Consolidate Job-File Reference

Chapter 1: Introduction to Match/Consolidate Job

This chapter is a synopsis of the most important facts that you will need to get started with Match/Consolidate (MCD). If you have never used MCD software, consider reading the User’s Guide to Record Matching before using this reference and before running sample jobs.

Chapter 1: Introduction to Match/Consolidate Job

Install Match/Consolidate

You must install MCD and the sample and template files before you can work with them. To install MCD, see the System Administrator’s Guide, which explains how to set up the computer and how to install the software. Follow those instructions thoroughly and carefully. Remember to set PATH and PW_PATH. See the Database Prep manual for information about databases and ASCII text files and how to prepare them for MCD.

Windows When you run install setup for Windows operating systems, the program

prompts you to select a drive, and to name a directory where you want MCD located. The default directory location is c:\pw; we recommend that you accept this default.

The installation program automatically creates subdirectories under \PW. If you’ve used another drive or directory name, change the path names accordingly. For details, see the System Administrator’s Guide.

dirsadm mpg mpgc

Edjob and other

utilities

UNIX The UNIX directory structure is shown below. You must create the postware and

ZCF and other

directories

work

Files for the MCD jobs MCD templates

MCD executables

and dictionaries

template

job files

MCD Custom

(optional)

samples

Files for Quick

Start jobs

adm directories. For details, see the System Administrator’s Guide.

postware

Edjob and other

utilities

dirsadm mpgc

ZCF and other

directories

merge

MCD executables

and dictionaries

MCD Custom

(optional)

work

Files for the MCD jobs MCD templates

Match/Consolidate Job-File Reference

template

job files

samples

Files for Quick

Start jobs

Run a Match/Consolidate job

Match/Consolidate is a batch program. You prepare several files for input, and enter a command line at the operating-system prompt. Match/Consolidate begins operation, requiring little or no further input.

The illustration below shows what goes into, and comes out of, a MCD job. When you start the program, it first scans the job file for errors. If any are found, you must edit the job file to correct them, then enter the MCD command line again. When all errors have been found and corrected, processing begins. Match/ Consolidate then presents messages about its progress.

Job file

Input

database(s)

Supporting files

Output

database(s)

Match/Consolidate

Reports

Work files

Supporting files

Input Here are the files that MCD requires:

File Description

Job file A job file tells MCD everything it needs to run: Where to find the

input file(s), what sort of processing to do, which reports and outputs to create, and where to place them.

Input file(s) The input file contains name, address, and other data used in the

matching process.

FMT, DMT file For some types of input files, you’ll need a format file. This

describes the input file in a physical way: the sequence of fields, their names, their lengths, and type of data.

DEF file Think of this file as a dictionary. The fields in the input file may have

any names you like. The DEF file translates those names into particular names that MCD can recognize. For details about input files and their support files, see

Database Prep

Chapter 1: Introduction to Match/Consolidate Job

Results Here are the results that you may expect from MCD processing:

Result Description

Reports MCD prepares plenty of reports documenting the duplicate detection

process. Check the reports to verify good results. Most users instruct MCD to save reports in files, so they can be read on-screen or sent to a printer.

Statistics files MCD can generate statistics files containing information on the job.

These files can be brought into a database, spreadsheet, or word processing program so you can create the job-related reports.

Output file(s) MCD can create output files, such as databases of processed names

and addresses. You control the format and the content of output files.

Input posting MCD can place information back in the input records.

Input purge MCD can delete unwanted records from the input file(s), or mark

them for later removal.

Consolidation MCD can consolidate information from matched records to form a

best

record for the database.

Work files While processing, MCD stores internal information in work files. These files are

not directly useful to you, but you do need to know a few things about work files:

 The MCD installation program creates a work subdirectory. Set MCD

(through the Work File Directory parameter in the Execution block) to place its work files in this subdirectory.

 MCD gives you the option to save work files; otherwise, they are

automatically deleted when the job completes.

Match/Consolidate Job-File Reference

Steps in Match/Consolidate processing

The simplest kind of MCD job—finding duplicate records and eliminating them—is a three-step process that automatically happens within the MCD operation.

1. Read records and create key file.

2. Find duplicate records.

3. Either delete unwanted records from the input file(s), or copy good records to an output file.

Read records and create key file

The first step is to read all of the input records. Match/Consolidate does two things to each input record. First, MCD decides whether the record should be included in the job. Perhaps the record has already been marked for deletion, for example. Or, you may set conditions that an input record must meet (called a filter) to be included in the job.

If the record is to be included, MCD then extracts data from the record and copies it into another file, called the key file. The key file is an internal work file. When searching for dupes, MCD will look at the data in the key file, not the data in the input file(s).

As data is copied from the input file, it may be modified. Names, firms, and city/ state/ZIP data may be standardized, and address data is parsed. When MCD prepares data for the key file, this is only for internal use, to improve dupe detection. It does not change the data in the input file, nor (if you choose to create one) the output file.

For more information about setting up input files and the key file, see the User’s Guide to Record Matching.

Find duplicates The second step is to find duplicates. You will choose the logic that MCD uses to

decide whether records match.

When the program finds two or more records that match, it assembles them into a dupe group (duplicate record group). Once the search is complete and all dupe groups have been formed, MCD sorts each group to prioritize the records. Again, you will choose from one of the following prioritizing methods.

Prioritizing method Description

Unique records Records that did not match any other record

Master dupes Records that ranked at the top of their dupe group

Subordinate dupes Records that ranked second or lower in their dupe group

Chapter 1: Introduction to Match/Consolidate Job

For most users, the unique records and the master dupes are the good records and the subordinate dupes are unwanted. The following table may help you to picture these groups. Normally, records above the line are kept, and those below it are dropped.

Purge dupes or create output

Master dupe

Unique Unique Master

dupe

Unique Master

dupe

Unique

Subordinate Subordinate Subordinate

Subordinate Subordinate

Subordinate

Note that all of step 2 is carried out in MCD work files. So far, MCD has not output or eliminated any records.

The third and final step of most MCD jobs is to prepare the output. You have a choice of two methods.

Method Description

Purge Delete subordinate dupes from the input file, leaving the unique records

and master dupes intact.

Output Copy unique records and master dupes from the input file(s) to an output

file, omitting the subordinates, and leaving the input files intact.

Of course, there are plenty of ways to make a MCD job more complex to meet your particular needs. The User’s Guide to Record Matching explains how you can use the program’s features to get the results you want.

Match/Consolidate Job-File Reference

Run the Match/Consolidate job

Sample job We’ve sent you a sample MCD job that is ready to run. Running this job:

 Verifies that the system is set up correctly.

 Verifies that MCD is installed correctly.

 Increases the confidence in working with MCD.

 Gives you a starting point from which to develop jobs of your own.

There are two versions of the sample job file:

 quikwin.mpg for Windows

 quikunix.mpg for UNIX

You’ll find commands for running these jobs on the next page. If the software has been installed and the system set up correctly, MCD will display messages as it goes through these steps:

1. Starts the program and verifies the job file.

2. Reads input records.

3. Searches for dupes.

4. Creates an output file of desirable records.

5. Generates reports.

After you run the job, look at the files in the samples subdirectory.

Job description It’s impossible for one sample to apply perfectly to all users. However, we’ve

designed the sample job to be as simple, yet typical, as possible:

 Small mailing list, all one state.

 One input file of 1000 records, fixed-length ASCII text.

 Family mailing (match on last name and address).

 MCD output file (unique records and master dupes).

 Basic set of reports.

Create jobs from our sample

You can use the Quick Start job as the basis for your own jobs. Copy the Quick Start job file and edit your copy. Look for the parameter entries in UPPER CASE; those are the ones you’ll probably want to change, especially path and file names.

Chapter 1: Introduction to Match/Consolidate Job

Windows commands Complete the following steps to run the Quick Start sample job. If you installed

MCD to a drive or directory other than c:\pw, change the path name.

Open Windows Explorer before and after, so that you can compare what is input to MCD, and the files that MCD creates.

1. Choose Start > Run

2. In the Run box, enter pwmpg c:\pw\mpg\samples\quikwin.mpg (Processing messages appear, and then processing is complete.)

Note that as an alternative to steps 1 and 2, you can enter the following command from a DOS prompt in the c:\pw\mpg directory: pwmpg samples\quikwin.mpg

UNIX commands To run the sample job, type the commands shown below. We’re assuming that

you’ve installed MCD in /usr/postware. If you’ve used another location, change the path name accordingly.

Run ls before and after, so that you may compare what’s input to MCD, and the files that MCD creates.

$ cd /usr/postware/merge/samples

$ ls

$ pwmpg quikunix.mpg (Processing messages appear. Then processing is complete.)

$ ls

Match/Consolidate Job-File Reference

Create Match/Consolidate jobs

To help you prepare your own MCD jobs, we provide the following samples. You can copy any of these files as a starting point. You will find them in the template directory (see “Install Match/Consolidate” on page 8).

Before editing any of the sample jobs discussed below, make a copy with a

different file name. When we ship software updates, we always ship new copies of the sample jobs. Ensure that the new version does not overwrite a file on which you’ve been working.

Quick Start Now that you’ve run the Quick Start job and you know that it runs, you can adapt

it to your own needs.

Templates Templates are job files that are nearly ready to use; they require just a few

minutes of editing from you. Each template is set up for a particular type of job.

Master The master job file is called master.mpg. It contains one of each type of block

that a MCD job file might contain. When you need to add new blocks to a job, you can copy them from master.mpg.

Resources The resource file match.mpg is not a complete job file. Instead, it contains

samples of the blocks that control matching. These samples will help you understand and select a strategy for matching. You can copy sample blocks into any other job file.

Similarly, the resource file group.mpg contains sample blocks for Group Posting. See the User’s Guide to Record Matching and Chapter 2, “Job-file blocks and

parameters” on page 21 for descriptions and setup instructions.

Extended matching files

There are six extended matching files that you may refer to from the MCD job file. These files are named auto.mpg, family.mpg, firm.mpg, firmindv.mpg, hhold.mpg, and indiv.mpg. See the Extended Matching Reference manual for details about using these files.

Editing tips To edit job files, you will need working knowledge of a good text editor or word-

processing program. If you use a word processor, be sure to save job files as simple ASCII text.

Chapter 1: Introduction to Match/Consolidate Job

Guidelines for editing job files

Consider the following points when editing job files:

 Copy and edit the job files provided with MCD, or copy blocks between files

as necessary.

 Use the file name extension .mpg.

 Some blocks are required, but most are optional. You can place blocks in

any sequence.

 Do not edit the BEGIN or END lines, block titles, or parameter names

(anything to the left of the equal sign). There is only one exception to this rule: To make MCD ignore a block, insert an asterisk (*) before the word BEGIN.

Never delete parameters or rearrange them within a block. There are a few

blocks in which you can copy and repeat the last parameter as many times as you need. Only when the manual says it is okay should you change the number of parameter lines within a block.

Never press the Enter key while typing a long parameter entry; let the entry

wrap onto an additional line. If you press the Enter key, MCD counts the extra end-of-line marks as separate lines.

 You can add comments at the beginning or end of the job file, and between

blocks, but not within a block. Notes or comments might make it easier for others to understand and use the job file.

We recommend that you start all comment lines with an asterisk (*). We also recommend that you do not use the keywords BEGIN or END in comments. A comment can be anywhere in a job file after an END or before the next BEGIN.

 Many parameters require some sort of entry. In the sample job files that we

provide, many parameters have suggested entries already in place. There are some optional parameters that may be left blank.

 Where space allows, parameter names are followed by clues or options in

parentheses. Clues are shown in lower case; options are in upper case. Case doesn’t matter in the entry you type, but be sure to spell options exactly as shown, and do not abbreviate. Exception: At Y/N parameters, you may spell out Yes or No.

 As programs are updated to new versions, parameters and blocks may be

added or changed. Do not manually update the job files to the new version; instead, use the Edjob utility. For instructions about how to do this, see the Edjob User’s Guide.

Match/Consolidate Job-File Reference

Match/Consolidate command line

Checklist Before running MCD, make sure that you have finished the following prepwork:

 Create an FMT file for each fixed-length ASCII input file.

 Create a DMT file for each delimited ASCII input file.

 Create a DEF file for each input file.

 Complete the job file.

 Verify that adequate disk and memory space are available.

Basic command line To run the job, type the MCD command, followed by the path and name of the

job file: pwmpg [path] jobfile.mpg

Options See the Quick Reference for a complete list of options for the MCD command

line.

Chapter 1: Introduction to Match/Consolidate Job

Messages during verification and processing

Verifier messages The MCD job-file verifier is a part of the pwmpg program. It checks the job and

input files for the following types of mistakes and/or omissions:

 Missing PW field from DEF file.

 Input file is defined as fixed-length ASCII, but FMT file cannot be found.

 Missing execution block from job file.

 Entry at Filter parameter is invalid because the syntax is wrong.

A verifier message may be either a warning or an error. When you receive a warning, you may choose to continue or to stop processing. An error is more serious, and processing must stop.

You may control the verifier by adding options to the command line. See the a, nos, and v options in the Quick Reference.

Correct errors For command-line processing, the verifier catches only one error per run. Note

that MCD Views lists all warnings and errors at once; when it stops, you will be back at the operating system prompt. Start the text editor, open the job file, and correct the error, then start pwmpg again. Continue this cycle until the job makes it all the way through verification without errors.

Process messages Match/Consolidate reports progress by printing messages on the screen. The

processing messages will refer to tasks that MCD performs: Reading input records, finding dupes, sorting dupes, creating reports and output files, and so on.

During each step, MCD reports progress as a percentage. When all the processing you requested is finished, the last message is as follows:

Processing completed

You might want to capture messages in a file for later reading; to do so, add something like this to the command line:

UNIX:

Windows: /lmessagefile

messagefile

.log

This redirects the standard output to a file.

Match/Consolidate Job-File Reference

Template job files

Match/Consolidate template job files are nearly ready to use; they require a few minutes of editing from you. You’ll find them on the system in the template subdirectory. Below is a list and description of each template.

Before editing any of the templates, make a copy with a different file name.

When we ship software updates, we always ship new copies of the templates. Ensure that a new version does not overwrite a file on which you’ve been working.

Most of the templates do not contain the crucial pair of blocks in which matching logic is set (Match Criteria and Match Options); they are not needed for extended matching. Note that the Match Options block and the Match Criteria block are required for standard matching. For more information, see “Matching Method

(STD/EXT/ADV)” on page 24.

Select the matching logic from the options offered in the resource file match.mpg. Then, copy the appropriate pair of blocks into the job file. This way, you can adapt the template for the following mailing types:

 Residential (one per address)

 Family (one per last name)

 Individual (one per person)

dedupe.mpg We’ve received a client’s mailing list, to prepare it for a direct-mail campaign.

The client has asked us to de-dupe the list. We run the file through MCD and delete dupes from it.

good_out.mpg This job is just like the first, with one change: Instead of deleting dupes from the

input file, we’re going to copy good records to a separate output file. As above, this template may be adapted for individual, household, residential, or businessto-business mailings.

suppress.mpg Our company is a member of the Direct Mail Association. Four times per year we

receive from the DMA an updated copy of their Mail Preference File. This is a list of almost three million people who have asked not to receive unsolicited mail advertising. We want to compare this list to our mailing list to make sure we don’t mail our catalogs and flyers to people who probably wouldn’t buy anyway.

fishing.mpg A common variation on the suppression job is to use the house file as a

suppression list, to prevent mailing to current customers. For example, a charitable foundation mails a special solicitation to rented lists. They want to make sure that no one on their current-donor list receives this mailing. Another example would be a cruise line with berths to fill, mailing a special discount offer that must not be received by anyone already booked.

Chapter 1: Introduction to Match/Consolidate Job

multibuy.mpg Our company mails special offers and coupons to frequent buyers; that is, people

who show a pattern of repeat business. Today, we have to mail 10,000 copies of a rather expensive catalog, so we want to select our 10,000 best prospects for this mailing. To prepare this mailing, we’re going to merge several files: Our own customer list, plus lists that we’ve rented.

The total number of input records is about a dozen times the number of catalogs we have to mail, so we can afford to be choosy. We’re going to stipulate that, to receive this mailing, a person’s name must appear several times in our input files. And of those repeat buyers, we are going to take only the 10,000 highest incomes.

firm_pkg.mpg We’re a magazine publisher, mailing technical and engineering titles at second

class. We’re anxious to reduce our postage by using Presort to form firm packages whenever possible. However, Presort looks only for firm names that match exactly. Any variation in spelling or punctuation prevents Presort from finding a match. This is a problem for us, because our subscription-entry system does nothing to standardize company names. But we can use MCD to add a field that will work reliably in Presort.

Match/Consolidate Job-File Reference

Chapter 2: Job-file blocks and parameters

This chapter describes each parameter in the job file, listing parameters by block. We list the blocks in the order in which you are most likely to set up a job; see the table of contents for an alphabetical listing of each block.

Chapter 2: Job-file blocks and parameters

General

When you open a new job, the optional General Information block appears. The purpose of the this block is to stamp each job with the current version of Match/ Consolidate (MCD). It also provides a location to label the job with a description and a job owner. The information included in this block appears in some reports.

Job description Enter a name up to 80 characters for the job. This information is printed in the Job

Summary Report. Use the following shortcuts (macros) in the description.

 $job is converted to the base name (without path or extension) of the job file.

 $date and $time are taken from the computer’s clock at the time you start the

job. The date is nine characters long, in the format dd-mmm-yyyy. The time is ten characters long, in the hh:mm:ss format, with am or pm.

Job owner Enter a name up to 20 characters of the job owner. Match/Consolidate prints this

information in the Job Summary Report.

Match/Consolidate Job-File Reference

Execution

This block controls how MCD executes. Decisions made in this block determine the matching method, type of output file(s) created, and processing options.

Read Records & Create Match Sets (Y/N)

Reading records and creating the match sets is the first step in MCD processing. If you change the selection of match sets or their lengths, you must repeat this step (set this parameter to Yes again).

Option Description

Ye s En t e r

No If you want to rerun the job using saved work files, set this parameter to

Find Duplicates (Y/N/Predict)

Once you have created the key file (the Read Records & Create Match Sets parameter process), or if you are working from an existing valid reference file rather than raw input data, you can perform this process to compare records, find duplicates, and form dupe groups.

Option Description

Yes To find duplicate records, set this parameter to Yes and then run

No Once dupes have been found correctly, change this parameter to No to run

Ye s

to read all input records and create match sets. This is a

mandatory first step for all the execution processes that follow. If you intend to create, update, or use a reference file, set this option to

Yes. On the first run, set this parameter to Yes and the other processes to No, and then run ence file status), and generates any reports.

No. If you leave it set at Yes, MCD creates the match set again.

MCD searches for dupes and creates any reports that are set up. You can check the reports before going on to the next execution process.

from saved work files for subsequent output processes. If you leave it set at Yes, MCD continues searching for dupes.

pwmpg

. MCD creates the match set (depending on refer-

pwmpg

Execute in Background

Predict You can enter Predict to save processing time and perhaps find more

matches. When Find Duplicates is set to Predict, and you run the job, MCD forms break groups but does not perform the full matching process. Then you can check the Job Summary and adjust the break-group setup as needed.

If you select this option, MCD programs operate in a background window. If not selected, the program operates in the active window, with messages, and so on, on top.

Chapter 2: Job-file blocks and parameters

Matching Method (STD/EXT/ADV)

Use this parameter to choose a standard, extended, or advanced matching method.

Standard matching This method lets you match on any fields within the database and set the

appropriate match level per component. Standard matching also has default matching strategies for quick and easy setup.

Extended matching This method is rule-based.

MCD uses extended matching rules for settings that tell it how to search for duplicates. You can enter the path and name of an external extended matching file that you want to use at the EXT Match Blocks parameter in the Auxiliary File block. For more information about extended matching, see the

Extended Matching Reference

Advanced matching You can implement this method with standard and/or extended matching. MCD

manual.

has the ability to perform multi level matching and multi criteria matching and lets you use the Match Set blocks and parameters. Advanced matching parameters start with the letters ADV and are active only when you set the Matching Method to ADV.

Use advanced matching if you are using multiple match strategies within one pass of a job or when combining more than one match set. If you want to perform advanced matching, you still need to choose a matching strategy (STD or EXT).

To help you decide whether to perform standard matching or extended

Do I have existing jobs set up that I don't want to change right now?

Yes

Set the matching method to Std to use standard matching, with the Match Criteria and Match Options blocks (as you've done in the past).

matching, answer the questions in the flowchart at right. Keep in mind that there are two different kinds of

Do I want the quickest possible setup time, with slower processing time and less control over how Match/Consolidate performs matching?

Yes

Set the matching method to Ext, and use automatic extended matching. See the

Extended Matching Reference

for details.

extended matching: automatic and rulebased.

Do I want to:

control the order in which

•

fields are compared and the relative importance of each field?

•

compare fields such as social security number and gender, in addition to the usual name and address fields?

•

control how Match/Consolidate handles compar isons with blank fields and abbreviations?

Yes

Set the matching method to Ext, and use rule-based extended matching. See the

Extended Matching Reference

for details.

Match/Consolidate Job-File Reference

Create Match/Consolidate File (Y/N) Create Multi-Occurrence File (Y/N) Create All-Duplicates File (Y/N) Create Custom M/C File (Y/N)

These parameters control what kind of output file you might create.

Option Description

Yes If you want to create an output file, enter Yes at one of the parameters that

controls the output you want.

No If you don’t want to create an output file, enter No

Post to Input File (Y/N) Group Post to Purged Files (Y/N) Purge (Y/N/PREDICT) Custom Purge (Y/N/PREDICT)

These parameters control the purging of data from the input file(s) and the posting of data to input file(s).

Option Description

Yes If you want to post data to, or purge dupes from, the input file(s), enter Yes

No If you don’t want to post or purge the input files, enter No at all

Predict If you want to create a Purge by List prediction report without performing

At the end of any batch run in which you post or purge the input file(s),

MCD deletes its work files. This means that you cannot come back later and create any reports or output files.

at all four parameters.

at the appropriate parameter(s). You may choose either the ter or the Custom Purge parameter, but not both.

four parameters.

an actual Purge or Custom Purge, set the Purge or Custom Purge parameter to Predict

Purge parame-

If you want any reports or output files, set them up now before the purge. Work files are not lost when you set the Purge parameter to Predict.

You cannot set both the Purge and Custom Purge parameters to Yes. If you

do, MCD purges dupes from the input files; however, it ignores the Custom Purge request.

Chapter 2: Job-file blocks and parameters

Create Reports (Y/N) Create Report Statistics Files (Y/N) Save Work Files (Y/N)

Unless

you don’t want to save work files to be used in subsequent runs, set

Reports and Save Work Files parameters to Yes.

Option Description

Yes Set the Create Report Statistics Files parameter to Yes if you intend to use

No Set this option to No if you don’t want to save work files to be used in

Warn Before Overwrite (Y/N)

When MCD attempts to create a file, it might find that a file with the same name already exists. This parameter applies to all types of files that MCD might create, except work files. Note that MCD never warns before overwriting a work file.

Option Description

Yes MCD halts, sends a warning message, and waits for a response. You can

the Create

the statistics generated from the job to create the reports. You can then create the statistics reports by bringing the statistics files into a database, spreadsheet, or word processing program. Be sure to also set up the Statistics Files block.

subsequent runs.

stop the job, change the file name, and resume. Consider how setting this parameter to Yes will affect the processing of unattended batch jobs.

No MCD deletes the existing file and overwrites it with a new one without

Work File Directory (path) Sort Work File Directory (path)

You can enter a path for the work file directory and the sort work directory at these parameters. This lets you store the two different kinds of files in separate directories, which you will find useful if there is not enough disk space to store both work files and sort files in the same directory.

The sort work directory holds temporary sort work files. Match/Consolidate uses work files for temporary data storage during MCD processes.

You may leave these parameters blank. If they are left blank, MCD writes the work files and sort work files to the current directory.

warning.

Match/Consolidate Job-File Reference

Create Backup File (Y/N)

Backup Directory (path)

Match/Consolidate can make a backup copy of the input file(s) before purging or posting to them.

Option Description

Yes If you would like to create a backup copy of the input file(s), enter Yes.

MCD creates the backup just before posting or purging the input file(s). The backup will have the same base name as the input file and the extension

.bak

If you want the backup placed in a particular directory, enter that path and directory name at the Backup Directory parameter. Otherwise, MCD creates the backup copy in the same directory as the input file.

No If you don’t have enough free disk space, or if you don’t feel that a backup

is necessary, enter No at the Create Backup File parameter and leave the Backup Directory parameter blank.

If you elect to create a backup file, it will have the same base name as the input file, with the extension BAK, and will be placed in the same directory as the input file. This directory will not automatically appear in the box. If you want the backup placed in a particular directory, enter the path and directory name in that box.

Maximum Work Buffer Size (kilobytes)

Match/Consolidate runs faster when it has a large memory space. However, if you share memory space with other users, you can set this parameter to restrict how much memory MCD uses for work buffers. Remember that the size you set applies only to work buffers; this is only a portion of the total memory space occupied by the program.

This parameter controls the maximum amount of memory that MCD uses for buffers. For Windows users, the suggested setting for this parameter is 4096 kilobytes (four megabytes). If less memory is available, MCD uses less.

For UNIX users, use 1,024 kilobytes (one megabyte) as a starting point. With both operating systems, if the maximum size is set too large, disk swapping may occur, which decreases speed.

Sort Optimization (SPACE/SPEED)

Match/Consolidate does a lot of sorting: keys, break groups, dupe groups, and so on. Like any sorting program, MCD runs faster when it has adequate disk space. For information about estimating the disk space required for work files, see Appendix A of the User’s Guide to Record Matching.

Option Description

Space MCD uses less disk space but runs slower.

Speed MCD uses more disk space but runs faster.

Chapter 2: Job-file blocks and parameters

Auxiliary Files

Auxiliary Files tell MCD where to find the necessary dictionaries, rule files, and directories for parsing and standardizing input data for matching purposes. Match/Consolidate performs standardization on the record’s key for matching purposes only. Data is never changed in MCD.

Match/Consolidate has default options that allow the user to set the auxiliary files. Once they are set in the defaults, they are available for all subsequent jobs. To set defaults, go to Options > Match Consolidate Defaults.

Standard processing When you enable the standard option for Name, Title, and firm Parsing or

Address and Last-line Parsing in the Match Options block, MCD uses the following dictionaries and directories for parsing.

Dictionaries Listed below are the six parsing dictionaries used for standard processing.

Match/Consolidate uses the dictionaries to parse and identify specific data components.

Dictionary Function

Address line (

Addrln.dct

Last line (

Lastln.dct

Standard Prename

Prename.dct

(

Standard Name

Name.dct

(

Standard Pre-Lastname (

Prelname.dct

Standard Postname (

Postname.dct

Match Level Dictionary The matchpct.dct file is a table that sets, at three levels (loose, medium, and

)

Parses address line components and standardizes the spelling of suffix and directional information.

Parses city, state, and ZIP components.

Parses prename words such as Mr., Mrs., Dr.

Parses name words.

Parses a prefix, such as Von or Van, appearing as the first word in a compound last name.

Parses postname words such as Sr., Jr., or PhD.

tight), the percentage two components must be alike to be considered matches. Use this parameter to customize the dictionary and then enter the re-named file.

Dictionary Function

Standard Match Percent (

Matchpct.dct

)

Defines the values and sets match score adjustments.

Directories The following table lists the two directories that MCD uses to verify and

standardize city, state, and ZIP information.

Directory Function

City (

City09.dir

ZIP-City (

Match/Consolidate Job-File Reference

) Lists cities and the ZIP codes associated with them.

ZCF09.dir

) Lists ZIP codes and the cities and states associated with them.

Defaults If input files are in the same format, it is not necessary to create separate format or

definition files for each input file. Create one set of format and definition files as a default and enter its path and file name.

File Name Function

ASCII Format File (

.fmt

) Provides the layout for the input file when no individual

format file exists.

Definition File (

.def

) Defines the file when no individual definition file exists.

Extended processing When you enable the extended option for Name, Title, and firm Parsing and

Address and Last-line Parsing in the Match Options block, MCD uses the following dictionaries and directories for parsing.

Dictionaries Listed below are the three dictionaries MCD uses to parse and standardize name

and firm data for extended processing. These are the same dictionaries that DataRight uses for parsing.

Dictionary Function

Capitalization (

Pwcap.dct

) Standardizes special cases for capitalization such as

VanBuren and McDonald’s.

Parsing (

Parsing.dct

) Parses prename, name designators, name specials, last

name prefixes, first and last names, postnames, and titles.

Firm (

Firmln.dct

Rule Files The following table lists the rule files that MCD uses to take into account data

) Parses and standardizes firm data.

sequences and patterns for the identification of firm data and the components of multiline information.

Rules File Function

Firm Rules (

Fprules.gcf

) Contains information on sequences of firm data and

typical patterns that firms use.

Multiline Rules (

Mlrules.gcf

) Contains information on data sequences and typical

data patterns.

Directories The following table lists the directories MCD uses to standardize address and

lastline data.These are the same directories that ACE uses to standardize data.

Directory Function

ZIP4us.dir (ZIP+4 (1)) A national directory of addresses that is used to verify

and standardize address and lastline data.

(ZIP+4 (2)) A user defined custom directory that holds a smaller

portion of the national ZIP+4 directory.

Reverse ZIP+4 (RevZIP+4) A national directory of addresses with unique ZIP codes.

Chapter 2: Job-file blocks and parameters

Matching file The matching file identifies the path to the extended matching file that will be

used for this job. Match/Consolidate, in its extended matching process, uses this file instead of the Matching Criteria and Matching Options windows. If you are performing standard matching, as opposed to extended matching, leave this setting blank. If you have set a Match Blocks default (at the Auxiliary Files tab), MCD enters that default here for each new job you create.

To set or change the entry, type the information into this field. Or, click the folder icon at the end of the field to browse to the file location. Once you have found and selected a file name, MCD automatically enters its path and name into the field.

To use extended matching, you must also select extended matching at the Matching Method setting of the Execution window.

Match/Consolidate Job-File Reference

Unicode Conversion

This block lets you specify the character encoding of the data, specify address conversion errors, and specify an external Unicode-to-ASCII (Latin1) conversion table.

Match/Consolidate can read Unicode data, convert it to Latin1, and then match on the data. Match/Consolidate automatically reads data if the numeric value of the Unicode character does not exceed 255. If Unicode character values of data exceed 255, you must use this block to convert the data, and you must also provide the Unicode to Latin1 conversion table.

Input files can be a combination of Latin1 or Unicode data. Match/Consolidate will read in any form of Unicode data, translate that data to Latin1, and then return any form of Unicode data to you.

When processing Unicode records, you can create either Latin1 or commadelimited output files, and then append Unicode records to it. (DBASE3 and EBCDIC are not valid database types for Unicode records). Match/Consolidate does not alter data that passes to an output file that is not in Unicode format.

Unicode Conversion Name

This parameter assigns a name or label, which lets parameters in other blocks, such as the Unicode Conversion parameter in the Input File block, reference this block.

This is an optional parameter unless you plan to output Unicode data to an ASCII or Delimited database type. If you do not use this parameter, MCD uses a set of defaults in place of the conversion block. Conversion defaults include: No conversion table is used, and Delete, which MCD uses for the unassigned action.

Unicode to ASCII Table

This optional parameter indicates the name of an external file, which defines a table to convert an unassigned Unicode value to an ASCII value.

Unassigned Character Action

This required parameter provides the following three options for addressing unassigned or illegal characters in a job.

Option Description

Delete Delete unassigned and illegal characters.

Abort Discontinue processing the job if Match/Consolidate detects an

Default Convert all unassigned characters to the Default character setting. Set this

Note:

MCD appends one trailing blank space for every character that is

deleted in order to preserve the fixed field length.

unassigned character.

character in the Unassigned Character Default parameter (Job) or Default for Unassigned Characters option (Views).

Chapter 2: Job-file blocks and parameters

Unassigned Character Default

This parameter indicates what character to use to replace unassigned or illegal characters encountered by an MCD job. This is a required parameter only if you set the Unassigned Character Action parameter to Default.

Match/Consolidate Job-File Reference

Input File

Input File Name (path & file name)

Enter the path and name of the input file to be used in this job. If you have based the job-file name on the input file name, you may use the $job macro.

Reference File (path & file name)

Use this parameter if you intend to create or update a reference file associated with this input file, or to use an existing reference file rather than the raw input data for this job.

To use this parameter, enter the path and file name of the reference file. If you enter a file name, be sure to complete the other three Ref File parameters in this block. The Ref File always has to have .ref as the extension master.ref.

To create, update, or use this reference file, enable the Read Records & Create Match Sets processing option of the Execution block. In addition, set the matching options and matching criteria to capture or use relevant key data, and then run the MCD job.

If you do not want to use an existing reference file or create a new one, leave this parameter blank.

If the reference file is valid and the Input File Name parameter is blank, then the Starting Record Number, Maximum Number of Records to Input, and the Input Filter parameters must be blank.

Ref File Invalid (CREATE/ERROR)

If MCD cannot find the reference file named in the Reference File parameter, or if MCD finds that changes have been made to the input file from which the reference file was generated, the reference file is considered invalid.

Any of these changes to its input file would make a reference file invalid: number of records, record length, number of fields, or name of the database.

Option Description

Create MCD creates a new reference file using the relevant settings from this job.

Error MCD issues a verification error when it determines that the reference file

This parameter is typically set to Create.

is invalid.

Chapter 2: Job-file blocks and parameters

Ref File Old (CREATE/UPDATE/USE)

If the reference file is valid, MCD checks to see if you changed the input file date since its creation. If you changed the input file date, MCD considers the reference file to be old. You will usually set this parameter to Create.

Match/Consolidate also considers the reference file old if you are updating or appending to an input file while generating an output file, posting to an input file, and/or purging an input file.

Option Description

Create Creates a new reference file using the relevant settings from this job.

Update Regenerates the reference file using the same key layout as before but

Use Uses the reference file as is.

Ref File OK (CREATE/UPDATE/USE)

If the reference file is neither invalid or old, MCD considers it to be OK (valid). You will usually set this parameter to Use.

Match/Consolidate also considers the reference file to be OK if you changed a setting for the Starting Record Number, Maximum Number of Records to Input, or Input Filter parameters since creating a reference file. In this case, you will set the parameter to Create or Update.

using the new input file.

Option Description

Create Creates a new reference file using the relevant settings from this job.

Update Regenerates the reference file using the same key layout as before but

using the new input file.

Use Uses the reference file as is.

Starting Record Number Maximum Number of Records to Input

By setting these parameters, you may limit input from this input file based on record numbers. Limiting input by record numbers is faster than using a filter, which is useful when you want to run a quick sample or test run. To process all records, leave both lines blank.

Only records that pass the Input Filter count toward the Maximum Number of Records to Input parameter.

Match/Consolidate Job-File Reference

Input Filter (to 1024 chars)

With a filter, you can select certain records from the input file for processing. Match/Consolidate lets you set the criteria that each record must meet to be selected. For example, you might want to limit input to those records within a certain state.

When using an Input Filter, you may want to create an output file rather than purge dupes from the input files. For example, to prepare a list of records whose state is Missouri, copy the good Missouri records to a separate file.

If you are using a reference file for input, the Input Filter setting must be the same as any that was set when the reference file was created. You may leave this filter blank so that MCD uses the same filter that it used when it created the reference file.

Bypass Filter (to 1024 chars)

This optional parameter lets you determine which input records should bypass the Find Matches job.

Keys for Break Groups (ALL/AS_NEEDED)

This parameter controls which records MCD stores in the key file.

Option Description

All

As_Needed

In the key file, MCD stores all eligible records (that is, all records that pass the Starting Record Number parameter, the Maximum Number of Records to Input parameter, and the Input Filter). At least one of the input files must be set to All.

Match/Consolidate stores only the keys that belong to an existing break group (a break group formed by records from the file(s) for which you set this option to All). This saves time and disk space.

For example, assume that you are comparing a large database with a small database and you store all keys for the small database. To save time and space, you choose to store keys from the large database only as needed.

During the read step, MCD keeps track of the break strings stored for the small database. Then, MCD reads records from the large database and stores only the keys for those records whose break strings match the break string of an existing break group. Records that are not stored are called “sample drops”; the Input File Summary report shows the number of records in the sample-drop group.

This option runs faster when it has a large memory space; the memory used is controlled with the parameter “Maximum Work Buffer Size

(kilobytes).” Depending on the amount of resources available, to

maximize memory usage, set the maximum work buffer size as high as possible.

When this parameter is set to As_Needed, records from the As_Needed file that do not match the break key criteria of records from a smaller file are dropped from input. They will not be processed and will not be passed through as unique records.

Chapter 2: Job-file blocks and parameters

Protect Input File From Purge (Y/N)

This parameter lets you protect this file from any purge operation.

Option Description

Yes No records of this input file are deleted even though a purge operation may

be properly set up and executed in this job.

No Any purge operation defined and executed acts upon this input file as well

as any others not protected from the purge.

Character Encoding

This optional parameter indicates the character encoding in which MCD stores data. If you leave this parameter blank, MCD defaults the character encoding to ASCII.

Unicode Conversion Name (to 20 chars)

This parameter assigns a name or label, which lets parameters in other blocks, such as the Unicode Conversion parameter in the Input File block, reference this block.

Override Default Format (path & file) Override Default DEF (path & file)

Use these parameters to specify the format and definition files that you want to use for all of your input files. For example, if you typically have multiple input files specified in your Auxiliary Files block, you can use the override options to specify one format and definition file for all of your input files without having to remove the default options in the Auxiliary Files block.

Match/Consolidate looks for the definition and format files in the following order:

 the Override options listed in the Input File block  the files with the same name in the same folder as the input file  the files listed in the Auxiliary Files block

Match/Consolidate Job-File Reference

Input List Defaults

The Input List Defaults block controls the automatic generation of lists. It also contains parameters for setting the default list values, so MCD knows what list values to assign when a record doesn’t belong to any of the other lists.

You may have a combination of automatically generated lists and lists defined in Input List Description blocks. There are three ways that you can set up input lists:

1. You may set them up individually, each with its own Input List Description

2. You can have MCD generate one list for every input file by not setting up

3. You may set up automatic generation of input lists in the Input List Defaults

Count Special List in MultiList

This new functionality lets you determine whether MCD includes records from a special list in the multi-list count of a match group. Select this option if you want records from special lists to be included in a multi-list count.

For example, assume that you have the following two identical records:

block.

Input List Defaults or Input List Descriptions blocks.

block.

John Smith (record 1) John Smith (record 2)

Also assume that the first record is from a normal list and the second record is from a special list.

If you select this option, or if you set this option to Yes in the job file, MCD considers records 1 and 2 to be a multi-list dupe group because both lists are represented in the dupe group.

If you do not select this option, or if you set this option to No in the job file, MCD considers this example to be a single-list dupe group because only one list is represented in the dupe group; MCD does not count the special list record.

Chapter 2: Job-file blocks and parameters

Auto Generate List From List_ID (Y/N)

This parameter controls whether MCD creates a new list for every unique value in the field in the database associated with PW.List_ID.

Option Description

Yes If you want MCD to create a new list for every unique value in the List_ID

field, enter Yes at this parameter. This can save you time because you won’t have to set up an Input List Description block for each list. Note that the name of an automatically generated list will be the same as its List_ID value.

See the following table for details about how MCD handles other aspects of automatically generated lists.

No If you want to set up individual Input List Description blocks for every list

that you create, enter No at this parameter. If there are no Input List Description blocks set up, then MCD creates one list for every input file.

When you choose to automatically generate lists, MCD looks to the input DEF file(s) for the following PW fields. These fields define characteristics of the list, and they correspond to job-file parameters.

Field Description Corresponding parameter

List_Type This field defines the list type for this list. MCD assigns the list

type according to the first character in the field: N —

Normal

P —

Suppress (or Purge)

S —

Special

If the List_Type field is not defined, or if the first character in the field is not N, P, or S, MCD uses the default List Type in the Input List Defaults block.

List_Srch MCD searches for duplicate records within the list according to

the first character in the field: Y —

Search for dupes within this list.

N —

Do not search for dupes within this list.

If the List_Srch field is not defined, or if the first character in the field is not Y or N, MCD uses the default value at the Dupe Search Within This List parameter in the Input List Defaults block.

Driv_Prior This field determines which record is the driver record in a break

group. MCD expects to find a value between 0 and 255 in this field, 0 being the highest priority.

If the Driv_Prior field is not defined, or if the value in that field is not numeric or not between 0 and 255, MCD uses the default value at the List Break Priority parameter in the Input List Defaults block.

List Type (See the List Type parameter in this

block for details.)

Dupe Search Within This List (See the Dupe Search Within This List

parameter in this block for details.)

List Break Priority (See the List Break Priority parameter in

this block for details.)

Match/Consolidate Job-File Reference

Field Description Corresponding parameter

List_Prior This field determines the list’s priority, which sorts records

within dupe groups to determine which record is the master duplicate. MCD expects to find a value between 0 and field, 0 being the highest priority.

If the List_Prior field is not defined, or if the value in that field is not numeric or not between 0 and 999, MCD uses the default value at the List Match Priority parameter in the Input List Defaults block.

List_NewID This field determines if an auto-generated list generates an

AP.ID_INC_NO number. Selecting Yes generates the AP.ID_INC_NO for all dupe groups

assigned to that list. Selecting No sets the increment number to zero for each dupe

group in that list.

AppBlnkPri This field is for suppression lists only. It determines whether

MCD should apply blank priority to the suppression list’s match priority.

Y —

Apply blank priority to this list’s match priority.

N —

Do not apply blank priority to this list’s match priority.

If the AppBlnkPri field is not defined, or if the first character is not Y or N, MCD uses the default value at the Suppress Apply Blank Priority parameter in the Input List Defaults block.

Driv_Salvg This field determines whether MCD should perform data

salvaging on a list. Y —

Perform data salvaging.

N —

Do not perform data salvaging.

If the Driv_Salvg field is not defined, or if the first character is not Y or N, MCD uses the default value at the Perform Data Salvage parameter in the Input List Defaults block.

999 in this

List Match Priority (See the List Match Priority parameter

in this block for details.)

Use List to Assign New ID

Suppress Apply Blank Priority (See the Suppress Apply Blank Priority

parameter in this block for details.)

Perform Data Salvage (see the Perform Data Salvage parameter in this block for details.)

List Redefinition Action (USE/ABORT)

This parameter controls how MCD handles situations in which an automatically generated list’s definition changes while MCD is reading the records.

Each records’ PW fields store the characteristics that define a list. So, if one record defines a list with a certain set of characteristics, it is possible for a subsequent record with the same List_ID value (meaning that they are assigned to the same list) to contain a different set of list characteristics.

The two example records below contain a different definition for the list to which they belong. The circled fields show the differences.

Record List_ID List_Type List_Srch Driv_Prior List_Prior AppBlnkPri Driv_Salvg

# 15 list3 Suppress Y 10 0 Y Y

# 78 list3 Normal Y 10 10 Y Y

Chapter 2: Job-file blocks and parameters

With the List Redefinition Action parameter, you can decide to stop processing the job when lists are redefined or you can decide to allow the list to be redefined. You choose which list definition to use with the next parameter, List Definition to Use.

Option Description

Use Allow automatically generated lists to be redefined while reading

Abort Stop processing the job when MCD encounters a list redefinition.

List Definition to Use (FIRST/LAST)

Setting the List Redefinition Action parameter to Use lets you choose which record defines the list—the first record to define the list or the last one.

Option Description

First Use the first record’s list definition. Do not allow the list to be redefined by

Last Use the last record’s list definition. This means that, for every subsequent

the records.

different list characteristics presented by a subsequent record from that same list. With the example record pair shown on the previous page, record #15 defines the list. Choose First for faster processing.

record (from that list) that contains different list values, MCD keeps redefining the list. With the example record pair shown on the previous page, record #78 defines the list.

List Type (NORMAL/SUPPRESS/SPECIAL)

This parameter sets the default list type for automatically generated lists and for lists whose Input List Description block is blank at this parameter.

Option Description

Normal A Normal list contains good or eligible records.

Suppress A suppression list contains records that would often disqualify a record

from use. For example, if you’re using MCD to refine a mailing list, a Suppress list removes records from the mailing. Examples:

• DMA Mail Preference File

• American Correctional Association prisons/jails lists

• No pandering or non-responder lists

• Credit card or bad-check suppression lists

Special A Special list is treated like a Normal list, with one exception. A Special

list is not counted in when determining whether a dupe group is single-list or multi-list. A Special list can contribute records, but it’s not counted toward multi-buyer status.

For example, some companies use a list of seed names. These are names of people who report when they receive advertising mail, so that the mailer can measure mail delivery. Appearance on the seed list should not be counted toward multi-buyer status.

Match/Consolidate Job-File Reference

Perform Duplicate Search within this List (Y/N)

This parameter controls whether MCD compares records within the list with each other. This parameter sets the default for automatically generated lists and for lists whose Input List Description block is blank at this parameter.

Option Description

Yes MCD compares records within the list to each other.

No If you have already de-duped the list, enter No to save processing time.

When using more than one input file, the results of the duplicate records search can be compromised by the order of the input files, when you choose not to dupe search within a list. If one of the input files is a suppress-type list, that means the output could include records that you wanted (and expected) to be suppressed.

In general, if you use a suppress list, you should dupe search within the other lists. This ensures that all the dupes of those lists are suppressed when any are found to duplicate a record on the suppress list, regardless of which record is the driver record.

If you don’t want to search within the other lists, ensure that the suppression list is the first input file or has the highest list break priority.

Match/Consolidate ignores the setting at this parameter if you fill out the Match Qualification block.

Use List to Assign New ID (Y/N)

This parameter lets you generate a value for AP.ID_INC_NO on a per-list basis. You might want to enable/disable generating a value for AP.ID_INC_NO if some incoming records already have a valid ID and you do not want to assign them a new one.

Option Description

Yes AP.ID_INC_NO has a positive, unique value assigned to it.

No AP.ID_INC_NO is set to zero.

List Break Priority (0 to 255)

This parameter gives you control over the order of records in break groups. That means that you control, on a list-by-list basis, which record becomes the driver record, the record to which the other records in the break group are compared.

You can set this parameter to any number from 0 to 255, with 0 being the highest priority. If two records being compared have the same list break priority, then MCD uses input file order to determine the driver record. Otherwise, input file order doesn’t matter when determining the driver record.

Blank priority is not applied to the list’s break priority. For example, no matter how much more complete the second record is, a record with a list break priority of 10 will always come before a record with a list break priority of 11.

This parameter sets the default list break priority for automatically generated lists and for lists whose Input List Description block is blank at this parameter.

Chapter 2: Job-file blocks and parameters

List Match Priority (0 to 999)

This parameter is the primary way that MCD sorts records within dupe groups. The smaller the priority number, the higher the priority, and the more likely that records from that list will rise to the top of their dupe groups to become master duplicates.

Assign a priority of 0 to the best list, and larger numbers to other lists. For example:

 House file, customer list, or donor list: List Match Priority = 0

 Outside list rented at $45/M: List Match Priority = 11

 Outside list rented at $195/M: List Match Priority = 21

You can use any combination of numbers you want. For the example above, we could just as easily use values of 0/2/3 or 0/200/300. You may also wish to check out the Blank Priority parameter of the Match Criteria block, which can also affect how the List Match Priority parameter works.

You can sort output files according to list match priority plus blank priority with the sorting option Lb_Prior, available at the Sort By parameter in each of the four Output File blocks. The Lb_Prior option is associated with the application field AP.LB_Prior.

This parameter sets the default list match priority for automatically generated lists and for lists whose Input List Description block is blank at this parameter.

Suppress Apply Blank Priority (Y/N)

This parameter determines whether MCD applies blank priority to the suppression list’s match priority. This parameter applies to suppression lists only; if the list’s type is not Suppress, MCD ignores this parameter.

This parameter sets the default for automatically generated lists and for lists whose Input List Description block is blank at this parameter.

See BlnkP in the Match Criteria block, starting on page 54, for more details about blank priority. Or, see the Blank Priority parameter in the Prioritize Matches block in the Extended Matching Reference manual.

Option Description

Yes Apply blank priority to this list’s match priority.

No Do not apply blank priority to this list’s match priority.

Data Salvage (Y/N)

Data salvaging temporarily copies data from a passenger record to the driver record after comparing the two records. The data that’s copied is data that is found in the passenger record but is missing or incomplete in the driver record. Data salvaging prevents blank matching or initials matching from matching records that you may not want to match.

This parameter sets the default for automatically generated lists and for lists whose Input List Description block is blank at this parameter.

Match/Consolidate Job-File Reference

If you did not perform data

salvaging, the records at right would all belong to the same match group because the driver record, which contains a blank Name field, matches both of the other records.

drive record

Record Name Address ZIP

# 1 123 Main St 54601

# 2 John Smith 123 Main St 54601

# 3 Jack Hill 123 Main St 54601

If you performed data salvaging, MCD would temporarily copy John Smith from the second record into the driver record. The result: Record #1 matches Record #2, but Record

Record Name Address ZIP

driver

# 1 John Smith 123 Main St 54601

record

# 2 John Smith 123 Main St 54601

# 3 Jack Hill 123 Main St 54601

#1 does not match Record #3 (because John Smith doesn’t match Jack Hill)

The following example shows how this is used for a suppression list. Assume that the suppression list is a list of no-pandering addresses. In that case, you would set the suppression list to have the highest break priority, and you would set Perform Data Salvaging to No. That way, MCD suppresses all records that match the suppression list records.

For example, a suppress record of 123 Main St would match 123 Main St #2 and 123 Main St Apt C; both of these would be suppressed.

Option Description

Yes Perform data salvaging for records in this list.

No Do not perform data salvaging for records in this list.

Action for Unmatched Records (ASSIGN DEFAULT/IGNORE/ABORT)

Match/Consolidate performs the action you select in the following situations:

 You have not chosen to automatically generate lists, and MCD encounters a

record that doesn’t belong to any of the lists defined in the Input List Description blocks.

 You have chosen to automatically generate lists according to the List_ID

value, and MCD encounters a record that contains a blank List_ID field. Or, the List_ID field is not blank, but the maximum number (2000) of automatically defined lists has been reached.

Option Description

Assign Default The record is assigned to the default list, the list whose name you enter

at the Default List Name parameter below.

Ignore The record is left out of the job.

Abort The MCD batch run halts the first time that the program finds a record

that cannot be assigned to any list.

Chapter 2: Job-file blocks and parameters

Default List Name

If you select Assign Default at the Action parameter, enter the name of the list that you want to use as the default list. Be sure to enter the name exactly as you entered it at the List Name parameter of the list’s Input List Description block. Copy the name from the List Name parameter to prevent data-entry errors.

If you select Ignore or Abort at the Action parameter, leave this line blank.

Match/Consolidate Job-File Reference

Input List Description

If you decide to manually set up the input lists, as opposed to having MCD automatically generate them, use the Input List Description block(s) to define the list(s).

Even if you choose to automatically generate the lists, you may need to set up at least one Input List Description block to use as the default list. The default list is the one that is assigned when no other list can be or is assigned. If you set the Action parameter to Assign Default, then you must set up an Input List Description block to use as the default list.

At the Input List Defaults block, you can set default values for some of the parameters in this block. For details, see “Input List Defaults” on page 37.

List Name (to 20 chars)

Enter a descriptive name for the list. This name is printed on reports.

Value in PW Field List_ID List Filter (to 1024 chars) List Type (NORMAL/SUPPRESS/SPECIAL) Dupe Search Within This List (Y/N/Use Default) Use List to Assign New ID (Y/N/Use Default) List Break Priority (0 to 255) List Match Priority (0 to 999) Suppress Apply Blank Priority (Y/N) Perform Data Salvage (Y/N)

Match/Consolidate needs a way to decide in which list each record belongs. There are two ways to link records with lists; use one or the other, but not both.

Method of linking records to list

List_ID In the DEF files, define the PW field List_ID. At this parame-

List Filter The list filter may be based on any field(s), not necessarily

Description

ter, enter the value (data) that will identify a record as belonging to the list. The entry on this line must match exactly the data in the List_ID field. To indicate a blank field, enter two consecutive apostrophes (single-quote marks).

List_ID. Records that pass the filter (return True) will be included in the list. The sequence of Input List Description blocks matters, because a record will be placed in the first list for which it passes the filter.

For complete descriptions of the remaining parameters for this block, see “Input

List Defaults” on page 37.

If there are no values entered for these parameters in the Input List Description block, MCD uses the values entered in the Input List Defaults block.

Chapter 2: Job-file blocks and parameters

Super List Defaults

Default Super List Name

Use this block to set up a default super list.

If the list doesn’t appear in any of the Super List Description blocks, enter the name of the super list to which you want a list to belong.

Match/Consolidate Job-File Reference

Super List Description

Super List Name (to 20 chars)

Enter a descriptive name for the list, which is printed on some MCD reports.

List Name (to 20 chars)

Repeat the name of a regular list from the Input List Description block by using the Copy and Paste commands. Repeat this line as many times as necessary, with one line for each list to be included in the super list.

Chapter 2: Job-file blocks and parameters

Post to Input File

Input File Name (path & file name)

Enter the path and file name to identify the input file to which you are posting. If that file is not in the current directory, add a full or relative path name. Ensure that the file name is a file that is defined in one of the input file blocks.

ADV Select Match Set(s) (name[,name…])

This is a required parameter only if MCD performs advanced matching. This parameter lets you specify a comma-separated list of match sets to which this block applies. For example: MS1, MS2, MS4.

In this example, if a job contains four match sets, this block reports on match sets MS1, MS2, and MS4, but does not report on MS 3.

ADV Match Level (FINEST/SELECT)

This parameter determines to which level the AP prefix points. This is a required parameter only if MCD performs advanced matching. This is also a required p a r a m e t e r o n l y i f t h e b l o c k s p e c i f i e s a m a t c h s et t h a t h a s m o r e t h a n o n e l e v e l .

Option Description

Finest Uses the match set’s finest level.

Select Uses the match level specified in the ADV Select Match Level to Post

parameter.

ADV Select Match Level to Post

This parameter lets you specify the match level that you want to use. This is a required parameter only if MCD performs advanced matching. This is also a required parameter if you set the ADV Match Level parameter to Select. Otherwise, MCD ignores this parameter.

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results are used. This is a required parameter only if MCD performs advanced matching. This is also a required parameter if the block specifies a match set that has more than one level.

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Copy (source, destination)

Match/Consolidate Job-File Reference

Use the Copy parameter for input posting, such as to copy information into the fields of the input file. You may post AP fields or constants, and you may use filter functions. You may not post DB or PW fields, or use them in filter expressions.

Enter first the information that you want to post, then a comma, then the name of the field in the input file. You may repeat this line as many times as necessary.

Remember to think about field lengths. If the information that you post is

longer than the destination field, it will be truncated to fit. Also, be sure the spelling of field names is correct. Case is ignored, so it doesn’t matter if you type in uppercase, lowercase, or mixed case.

Chapter 2: Job-file blocks and parameters

Custom Purge Input File(s)

ADV Select Match Set(s) (name[,name…])

This is a required parameter only if MCD performs advanced matching. This parameter lets you specify a comma-separated list of match sets to which this block applies. For example: MS1, MS2, MS4.

In this example, if a job contains four match sets, this block reports on match sets MS1, MS2, and MS4, but does not report on MS3.

ADV Match Level (FINEST/SELECT)

This parameter controls what match level results to use to determine whether MCD purges a record; it is a required parameter only if MCD performs advanced matching. This parameter also determines to which level the AP prefix points. For more information, see Chapter 12 of the User’s Guide to Record Matching.

Option Description

Finest Uses the match set’s finest match level.

Select Uses the match level specified in the ADV Select Match Level to

Output parameter.

ADV Select Match Level to Purge

This parameter lets you specify the match level results that you want to use. Enter in a name that corresponds to the level in the match set.

This is a required parameter only if MCD performs advanced matching. This is also a required parameter if you set the ADV Match Level parameter to Select. Otherwise, MCD ignores this parameter.

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results MCD uses. This is a required parameter only if MCD performs advanced matching. This is also a required parameter if the block specifies a match set that has more than one level.

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Match/Consolidate Job-File Reference

Delete Unique Records (Y/N) … Delete Suppress List Subord (Y/N)

These parameters refer to nine categories of records as listed in the following table. On each line, enter No to save records; enter Yes to delete them. A record may be deleted if it is in one of the categories in which you have set Yes, or it may pass the Delete Filter parameter, if there is one.

In master.mpg, we have preset these parameters to deliver the same results that you would get with a conventional Purge (using the Purge parameter in the Execution block). You may change these as needed. The default values shown below correspond to the conventional Purge.

Category Description Default

Unique Records Not members of any dupe group. They come from Normal and Special input lists. N

Single List Masters May come from Normal or Special input lists. They are the masters of single-list

dupe groups—that is, groups whose members all came from the same input list.

Single List Dupes May come from Normal or Special input lists. They are the subordinate members of

single-list dupe groups.

Multiple List Masters May come from Normal or Special input lists. They are the masters of multi-list

dupe groups—that is, groups whose members came from more than one input list.

Multiple List Dupes May come from Normal or Special input lists. They are the subordinate members of

multi-list dupe groups.

Suppress Dupes Records from a Normal or Special list that belong to a dupe group that contains a

record from a Suppress list with a higher priority.

Suppression List Uniques Unique records from an input Suppress list. N

Suppression List Masters Records that came originally from an input Suppress list. They are masters of sup-

press dupe groups.

Suppression List Subordinates

Delete Filter (to 1024 chars)

Records that came originally from an input Suppress list. They are subordinate members of suppress dupe groups.

You can use a database filter to select records for deletion. If you don’t want to use a filter, leave this line blank. The filter expression must result in a true or false decision. Generally, MCD deletes records that return true, and does not delete records that return false. However, this is subject to the Delete… parameters.

Unless the database is set up for non-destructive delete marking, MCD

deletes records that pass the filter.

Remember, it’s better to err on the side of making the filter criteria too restrictive. If the filter is too permeable, records that you might truly wish to save are deleted. Your only means of recovery will be to restore the input file(s) and rerun the job from the beginning. We recommend you use the “Custom Purge Predict” feature (see “Execution” on page 23). You should also create a backup of the input file before deleting it.

Chapter 2: Job-file blocks and parameters

Match Qualification

In this optional block, you can choose to compare or not compare records from one list or super list with records in another list or super list. If you don’t include it in the job file, MCD compares all qualifying records except those from the same list, if you turned off the Dupe Search Within This List parameter.

When populating the grid, for best performance, maximize the Match/ Consolidate Views window.

Because this block is optional, you probably won’t need to set it up unless there are particular lists in the job that you don’t want compared to each other.

Consider the following points:

 If you include the Match Qualification block in the job, it overrides the Input

List Description block’s Dupe Search Within This List parameter.

 To use the Match Qualification block, you must have Input List Description

blocks set up in the job. If you are using automatically generated lists, you cannot use the Match Qualification block.

 For Views users only, if you exceed 200 input lists in a job and you want to

use the Match Qualification block, you will receive the following warning:

To resolve this issue, setup the block within a text editor, such as WordPad or Notepad. After you finish setting up the block in the text editor, save the file. You can perform further job set up in Views; however, any future edits to this block requires changes to be made in the text editor.

Qualification Name (to 20 chars)

Enter a name for this list-by-list comparison. Match/Consolidate requires an entry if you include this block in the job file.

Qualification Result (INCLUDE/EXCLUDE)

Indicate whether you want the list(s) or super list(s) entered at the Qualification Lists parameter(s) to be compared. If you include this block in the job file, MCD requires an entry at this parameter.

Option Description

Include Compare the records in the list(s) or super list(s).

Exclude Do not compare the records in the list(s) or super list(s).

Match/Consolidate Job-File Reference

Qualification Lists (list1[,list2])

At this repeatable parameter, you enter the name(s) of the list(s) or super list(s) whose records you want to include or exclude from the search (depending on the setting of the Qualification Result parameter).

You may enter one or two lists or super lists per parameter. If you enter just one, then MCD compares (or does not compare, depending on the setting of the Qualification Result parameter) two records only if at least one of the records in the pair is from that list or super list. If you enter two lists or super lists, MCD compares (or does not compare) records in the first list or super list with records in the second list or super list.

Any lists or super lists that you don’t name in a Qualification Lists parameter is treated the opposite of the setting at the Qualification Result parameter.

For instance, if you had a job with 10 lists (list1, list2,

BEGIN Match Qualification =============================

Qualification Name (to 20 chars)........ = qual1

Qualification Result (INCLUDE/EXCLUDE).. = exclude

Qualification Lists (list1[,list2])..... = list1, list2

and so on), and you wanted them all compared except for list1 with list2, you could set up this block as shown. Then, MCD compares records in all of the other lists and will compare records from list1 and list2 with records in other lists (for example, MCD will compare list1 records with list3 records, list4 records, and so on).

Chapter 2: Job-file blocks and parameters

Match Criteria

Match/Consolidate uses the Match Criteria block when performing standard matching. To perform standard matching, set the Execution Options block’s Matching Method parameter to Standard or Advanced.

Note that if you set the Execution Options block’s Matching Method parameter to Advanced, and if you plan to use standard matching in a match set, then you must set up the ADV Match Criteria Name and the ADV Match Option Name parameters.

ADV Match Criteria Name (to 20 chars)

This is a required parameter only if MCD performs advanced matching. Use this parameter to assign a logical name for this block. Any block that needs a match criteria or a key layout defined references this name. Do not use the specified name in any other Match Criteria or Extended Match Criteria blocks.

ADV Match Options Name (to 20 chars)

This is a required parameter only if MCD performs advanced matching. Use this parameter to assign a logical name for this block; the Match Criteria block references this name. Match/Consolidate ignores this parameter if you set the matching method to STD or if the ADV matching job uses extended matching.

The following table shows PW fields that MCD accesses for record data. Use this table with the table on the next page to determine what fields you need for implementing the match strategy that’s best for the job.

PW field name Name/firm parsing

mode

Line1 – Line12 Extended (EXT) The Line fields may contain name, firm, and/or address data.

Standard (STD) Names, titles, and firms are not parsed.

Name_Line1 - Name_Line6 Extended (EXT) Name_Line1-6 may contain name and/or title data.

Standard (STD) Only Name_Line1 (or the Name_Line field) is supported.

Title1

Title6 Extended (EXT) Title1–6 are supported, and may contain titles only.

Standard (STD) Only Title1 (or the Title field) is supported.

Firm1, Firm2 Extended (EXT) Fields Firm1 and Firm2 are supported, and may contain firm

Standard (STD) Only the Firm field is supported.

Firm_Line1, Firm_Line2 Extended (EXT) only May contain firm-name and firm location data, like depart-

Name_Firm1

Name_Firm6 Extended (EXT) only May contain any non-address data, such as name, title, firm

Notes

Does not parse titles and firm names.

names only.

ment, mail stop, or building number.

name and firm location data.

First_Part1, First_Part2 Extended (EXT) only May contain the first part of names one and two, where the

first part may include prename, first name and middle name.

Match/Consolidate Job-File Reference

PW field name Name/firm parsing

mode

Notes

Last_Part1, Last_Part2 Extended (EXT) only May contain the last part of names one and two, where the last

part may include last name, maturity postname, other postname and title.

See the following table for a list of possible key fields. To activate a key field, enter a length in its KLen column. If a key field’s KLen column is blank, that key field is not included in the key data, so it can’t be used for breaking or matching.

Field name Recommended

key-field length

Pre_Name First_Name Mid_Name Last_Name Post_Name

SSN Birthdate

Title Firm

4 8 1 10 4

9 6 to 8 Dependant upon if you use a 4-digit year or 2-digit year.

20 30

Phone 3

Street_Range Street Pre-directional Street Primary Name Street Suffix Street Post-directional Street Secondary Range PMB

PO Box Rural Route Number Rural Route Box

10 2 22 4 2 6 14

6 2 6

Comments

MCD takes data from the PW field of the same name, or parsed from PW fields consistent with the selected name, title, and firm parsing routine. For more information, see the previous page.

MCD takes data from the PW field of the same name, or, when using extended (EXT) name/firm parsing, parsed from PW fields. For more information, see the previous page.

MCD parses data from the PW field Address, Unit, or Line1-12, or, when using extended (EXT) address/last line parsing, parsed from additional appropriate PW fields. For more information, see the previous page.

City State ZIP ZIP4

13 2 5 4

MCD takes data from the PW field of the same name, or parsed from PW fields consistent with the selected address and last line parsing routine (EXT or STD). For more information, see the previous page.

Country MCD takes data from the PW field of the same name.

Merg_Purg1

MCD takes data from the PW field of the same name. Merg_Purg2 Merg_Purg3 Merg_Purg4 Merg_Purg5 Merg_Purg6 Merg_Purg7 Merg_Purg8 Merg_Purg9 Merg_Purg0 User_Field (Name,len,...)

Chapter 2: Job-file blocks and parameters

The resource file match.mpg contains samples of Match Criteria blocks already set up. These samples will help you understand and select a strategy for matching. You can copy sample blocks from match.mpg into the job file.

KLen (key field length)

On each key field that you want to use, enter a length in numbers. Every field you want to use for breaking or matching must have an entry in this column.

BPos (break field starting position) BLen (break field length)

On each key field that you want to use for breaking, enter a number for the starting position. To begin at the left end of the field, enter 1. Then in the next column (BLen), enter a length in numbers. For example, to break on the 5-digit ZIP Code, set KLen to 5, BPos to 1 and BLen to 5 on the ZIP row. If you are not going to break on the field, you must enter 1 in the BPos column. The maximum number of fields on which you may break is 16.

MPos (match field starting position) MLen (match field length)

On each line to be used for matching, enter a number for the starting character position. To begin at the left end of the field, enter 1. Then, in the next column (MLen), enter a length in characters. For example, to match on the first 10 characters of Last_Name, set MPos to 1 and MLen to 10 on the Last_Name row.

Match (match level)

You might wish to break on part of a field, and match on the whole. For example, you might break on the first three characters of a street name, and then match on the whole street name (up to 15 characters). On the Street Name row, set KLen to 15, BPos to 1, BLen to 3, MPos to 1, and MLen to 15. You must set a minimum value of 1 in the MPos column.

Specify how closely two items must match to be called dupes, by entering one of the following letters: E for Exact, T for Tight, M for Medium, L for Loose, or N for None (or Not used). For a finer level of control, you may enter a numerical score between 0 and 100.

To see the required match scores for all components of the Match Criteria, use a text editor to view the matchpct.dct file, which is located in the mpg or merge directory.

Match/Consolidate Job-File Reference

BlnkP (blank priority)

Enter a numerical value. Whenever you turn on the Blank Matching criteria for a key field (see “BlnkM (blank matching)” on page 57), you can also set a value for its Blank Priority. The value you set will be added to the List Match Priority parameter of the record that contains the blank field. The lowest numeric score is the highest priority. Generally, this means that records with more complete data will be more likely to be output or survive a purge.

For example, assume the List Match Priority parameter value of the following records is equal, and a Blank Priority criteria value of 5 is entered for Middle Name. The record that lacks middle name data would end up with a larger number and lower overall priority.

first middle last range street suffix ZIP

John Q Smith 100 Main St 54601

John Smith 100 Main St 54601

BlnkM (blank matching)

ListP BlnkP Overall

10 0 =10

10 5 =15

Enter Yes or No. If you enter Yes, MCD allows a match between a blank field and one containing data. For example, if you set BlnkM = yes for Middle Name, the records below would be dupes.

first middle last range street suffix ZIP Code

John Smith 100 Main St 54601

John Q Smith 100 Main St 54601

Blank matching might be appropriate if you are matching on first or middle name, title, street directionals or suffix, secondary range, or rural box. Blank matching is not recommended on street name, PO box, or rural route number.

See the “Data Salvage (Y/N)” on page 42 for information related to blank matching in suppression lists.

Chapter 2: Job-file blocks and parameters

Maximum field lengths

When setting the lengths of key, break, and match fields, observe the following maximum standard and extended field lengths. If you set the length of a key field longer than the maximum, MCD issues a warning message and automatically shortens the key field to its maximum length.

Standard field lengths Extended field lengths

Pre_Name 255 Pre_Name 63 First_Name 255 First_Name 63 Mid_Name 255 Mid_Name 63 Last_Name 255 Last_Name 63 Post_Name 255 Post_Name 63 SSN 255 Title 63 Birthdate 255 Firm 255 Title 255 Street Range 10 Firm 255 Street Pre-directional 2 Phone 255 Street Primary Name 28 Street Range 10 Street Suffix 4 Street Pre-directional 2 Street Post-directional 2 Street Primary Name 28 Street Secondary Range 8 Street Suffix 4 Street Post-directional 2 PO Box 9 Street Secondary Range 8 Rural Route Number 9 PMB PO Box 9 City 28 Rural Route Number 9 State 2 Rural Route Box 9 ZIP 5 City 28 ZIP+4 4 State 2 Country 255 ZIP 5 All other fields 255 ZIP+4 4 Country 255 All other fields 255

14 Rural Route Box 9

PMB

Match/Consolidate Job-File Reference

Match Options

This block lets you control the various matching options available for the matching criteria that you set up. Match/Consolidate uses this block to perform standard matching. To perform standard matching, set the Execution Options block’s Matching Method parameter to Standard or Advanced.

Note that if you set the Advanced, and if you plan to use standard matching in a match set, then you must set up the ADV Match Option Name parameter.

ADV Match Option Name (to 20 chars)

Use this parameter to assign a logical name for this block; the Match Criteria block references this name. Match/Consolidate ignores this parameter if you set the matching method to Standard.

Household Match Type (IND/RESIDENT) Firm Match Type (IND/FIRM)

These parameters let you evaluate business and household records differently. To use these parameters, you must set up matching on personal name fields in the Match Criteria block (First_Name, Last_Name, and so on).

Match/Consolidate categorizes a record as business or household based on the PW field Firm. If Firm is empty (or, when using standard name parsing, if Firm is not defined in the DEF file) the record is a household record.

The Household Match Type parameter applies when MCD compares two household records; the Firm Match Type parameter applies when at least one of the two records is a business record.

Execution Option block’s Matching Method parameter to

Option Description

Ind MCD uses personal name fields for matching. In this case, when

Resident or Firm MCD ignores the personal name fields. If you are matching on

If you use the Household Match Type parameter and choose the Resident option, be sure to include Last_Name in the Match Criteria settings.

Match/Consolidate ignores personal name fields unless the address line consists only of a rural route, such as RR5. Then, MCD uses Last_Name so that all households in a rural route do not become members of the same dupe group.

Check for Transposed Letters (Y/N)

Transposed characters are two consecutive characters that have been switched within the word. Match/Consolidate deducts only half as much from the match score for transposed characters as is deducted for an invalid character. The

matching on address fields, two different names at the same address would not be members of the same dupe group.

address fields, two records at the same address will be members of the same dupe group, even though the names differ.

Chapter 2: Job-file blocks and parameters

transposition check lets MCD detect more matching records; however, the transposition check takes additional processing time.

Option Description

Ye s En t er

No Enter

This parameter does not affect calculations performed for the Adjust for

Unmatched Firm Words and Adjust for Abbrev. Firm Words parameters.

Ye s

if you want MCD to make a smaller deduction from the match score for letters that are transposed (as described above). This process adds to the processing time.

if you want MCD to deduct as usual for all incorrect letters without

giving special treatment to transposed letters.

If you set the Check for Transposed Letters parameter to No, but set either of the Adjust for... parameters to Yes, the transposition check is incorporated into the firm word calculations performed to satisfy the Adjust for... match option.

Name, Title, and Firm Parsing (EXT/STD/NONE)

This parameter controls whether MCD uses its standard parsing routine or uses its extended routine. You can expect better matching results from the extended parsing routine, but with an increase in processing time.

Option Description

STD MCD uses the name-related dictionaries listed in the four STD parameters

in the Auxiliary Files block. Fill out the four parameters when setting this parameter to STD.

EXT MCD uses the extended parsing dictionaries listed in the Auxiliary Files

None MCD does not look at the parsing dictionaries. This is best used when the

Standardize Name Keys (Y/N)

This parameter lets you standardize name keys.

Option Description

Yes Entering Yes standardizes name data when copying it from the input file to

No Enter No if you don’t want to standardize name keys.

block. Note that e acquire it, contact your Business Objects account representative.

input consists of all non-U.S. names and firms. A setting of None reduces the number of required more information, see “Auxiliary Files” on page 28.

the key file. This process affects only the data used for breaking and matching, not input or output data.

MCD converts abbreviations and alternate spellings of first and middle names to most common spellings and nicknames. For example,

iam

Dick to Richard

and the standardized form in the key file and uses them in the comparison. Name key standardization makes the key file a larger, and it takes slightly

more processing time, but the improved matching usually is well worth it.

xtended parsing is available for an additional fee. To

.dct

and

.dir

files in the Auxiliary file block. For

Will-

, and

Nikolai

Nicholas

. MCD stores both the input

Match/Consolidate Job-File Reference

Include Name of Second Person (Y/N)

This parameter lets you include names for a second person.

Option Description

Yes If you enter Yes, MCD attempts to parse out a second separate set of first

and middle names (often husband and wife names). When comparing to other records, only one name needs to match for records to be duplicates. For example,

If you decide to compare second names, the duplicate search takes longer because the number of comparisons is greater and the key file will also be larger. This parameter has no effect unless you are matching on first name.

This feature handles only one last name. It does not separate two completely distinct names, such as “John Doe and Mary Smith.”

No Enter No if you don’t want to include names for a second person.

Match First Name to Initial (Y/N) Match Middle Name to Initial (Y/N) Match Last Name to Initial (Y/N)

Most users set first and middle names to Yes and last name to No. These parameters have no effect unless you are matching on name fields.

Option Description

Jack & Jill Smith

matches

Jill Smith

Yes MCD matches initials to the first letter of the name. For example, “M” and

“Mark” would be a match.

No “M” and “Mark” would not meet any practical match criteria.

Match on Hyphenated Last Name (Y/N)

This option controls whether MCD performs a special comparison process when the last name of one key is hyphenated and the other is not. For example, consider the comparison of Jones to Jones-Smith.

Option Description

Yes With this option set to Yes, MCD does the following:

Compares Jones to Jones-Smith Compares Jones to Jones (the first hyphenated word) Compares Jones to Smith (the next hyphenated word) In each additional comparison, MCD uses the higher match score. Here,

MCD uses the second comparison score (Jones to Jones; score 100).

No With this option set to No, only the first comparison is made (Jones to

Jones-Smith). For a match, that score must meet the last-name match threshold.

Chapter 2: Job-file blocks and parameters

Match on Gender (Y/N)

This option may reduce processing time and prevent false matches.

Option Description

Yes Here’s how the process works if you enter Yes:

No Enter No if you don’t want to match on gender.

Standardize Title Keys (Y/N)

This setting is valid only with extended name parsing.

While generating the key file, MCD assigns a gender based on the input prename, first name, and middle name, if available.

Later, when comparing pairs of records, MCD first checks the genders. If the genders differ, then the records are not matched. All other fields are ignored, so you save the time that would otherwise be spent on comparisons and scoring. Here are some details of which to be aware:

To match on gender, you must turn on extended name/firm parsing. If the matching setup does not involve first or middle names, then MCD

ignores the Match on Gender setting. For example, gender is irrelevant when you are running a family or business match.

Gender is used only if it can be assigned with confidence (“strong” gender type) — for example, names such as is gender-ambiguous, such as

Hank

and

Chris

, then gender is ignored.

Margaret

. If the input name

Option Description

Yes If you enter Yes, MCD standardizes title data when copying it from the

No Enter No if you don’t want to standardize title keys.

Standardize Firm Keys (Y/N)

This parameter lets you standardize firm data when copying the data from an input file to a key file.

Option Description

Yes Match/Consolidate standardizes firm data when copying it from the input

input file to the key file. This standardization affects only the data used for breaking and matching; it does not affect the input or output file.

file to the key file. There are two standardization routines; the setting of name parsing (standard or extended) determines which routine MCD uses.

When using standard name parsing, MCD first strips

noise

words from firm

names. The cleaned-up version is then used as input for firm keys. When using extended name parsing, MCD removes noise words, and then

looks in

firmln.dct

for alternate spellings, abbreviations that could be

expanded, and acronym forms such as IBM and DEC. Firm standardization increases processing time slightly (more so with

extended name parsing), but the improved matching usually is well worth it. The firm standardization process affects only the data used for breaking and matching; it does not affect the input or output file.

No Match/Consolidate copies firm data to the key file as is from the database.

Match/Consolidate Job-File Reference

Match Firm Name to Initials (Y/N)

This parameter lets you match firm name to initials.

Option Description

Yes If this parameter is set to Yes, MCD compares an “initialized” version of the

firm name (the first letter of each significant word of the firm name) before concluding that two firm names do not match. In this case, “IBM” would match “International Business Machines,” because the three letters IBM match the first letters of the three words that make up

ness

No If this parameter is set to No, then “IBM” and “International Business

Machines” will match only in the unrealistic event that the comparison meets the threshold for firm name match.

Adjust for Unmatched Firm Words (Y/N)

This parameter matches firm data when all the words of the shorter of the two firm names exactly match part of the longer firm name.

For example, consider the comparison of Midtown Power and Light and Midtown Power. In this case, the shorter firm name (Midtown Power) exactly matches part of the longer firm name (Midtown Power and Light).

Option Description

achines

nternational Busi-

Yes If you set this parameter to Yes, MCD assigns extra credit to this comparison,

making it more likely that the two firm names will be considered a match. The preset value for the “extra credit” is in the mula for the calculation:

(

# of matching letters

# of total letters in comparison

With this parameter set to yes, and with the adjustment score set to 71, the match score is 93. (24 x 100) + (8 x 71) / 32 = 93

No With the Adjust for Unmatched Firm Words parameter set to No, this compar-

ison generates a match score of 75. (24 x 100) + (8 x 0) / 32 = 75

Adjust for Abbrev. Firm Words (Y/N)

This parameter controls how MCD treats firm data in cases where the letters of a shorter firm name word are contained in a longer word of the second firm name. In each comparison, the first letter of the shorter word must match the first letter of the longer word. In addition, all the letters in the shorter word must appear in the longer word, and must appear in the same order in both.

For example, consider the comparison of the words International Bus Machines and Intl Business Mach. In this example, the first letter of the shorter word (I) matches the first letter of the longer word (International).

In addition, all the letters of Intl are in International and they are in the same order. The same is true of each of the other two words as well. Notice that this

x 100) + (

matchpct.dct

# of nonmatching letters

file. Here is the for-

adjustment score

)

Chapter 2: Job-file blocks and parameters

adjustment is performed word-by-word. As shown in this example, it is not necessary that all the shorter words be in the same record.

Option Description

Yes If this option is set to Yes, MCD assigns extra credit to the comparison,

increasing the chance that the two firm names will be considered a match. The value for the “extra credit” is maintained in the formula for the calculation:

# of matching letters

( # of total letters in comparison

With this parameter set to Yes, and with the adjustment score set to 71 (its default setting), then this comparison generates a match score of 87.

(22 x 100) + (18 x 71) / 40 = 87

No With the Adjust for Unmatched Firm Words parameter set to No, the compar-

ison in the example above generates a match score of 55. (22 x 100) + (18 x 0) / 40 = 55

Address and Last Line Parsing (EXT/STD/NONE)

This parameter determines if address data is standardized prior to being stored in the key.

Option Description

x 100) + (

matchpct.dct

# of nonmatching letters

file. Here is the

adjustment score

)

EXT If you set this parameter to EXT, MCD performs extended address parsing,

STD If you set the parameter to STD, MCD uses a smaller and faster process to

None MCD does not look at the parsing dictionaries. This is best used when the

Standardize Last Line Keys (Y/N)

This parameter lets you standardize city, state, and ZIP Code data when copying it from an input file to a key file.

Option Description

Yes If you enter Yes, MCD standardizes city, state, and ZIP Code data when copy-

and the address key data is standardized to be consistent with the national directory (

zip4us.dir).

Extended parsing significantly increases processing

time to produce higher quality results. Note that

Extended address parsing is available for an additional fee. To

acquire it, contact your Business Objects account representative.

parse address information. With standard parsing, only the suffix and directionals of the address key data are standardized.

input consists of all non-U.S. addresses. A setting of None reduces the number of required

.dct

and

.dir

files in the Auxiliary file block. For more infor-

mation, see “Auxiliary Files” on page 28.

ing it from the input file to the key file. MCD compares the city, state, and ZIP Code data with the USPS directories.

If any one of the three elements is missing or wrong, it is assigned based on the other two. The ZIP Code alone is enough to assign city and state. More importantly, place names (unincorporated towns, vanity addresses, etc.) are converted to USPS preference to improve matching.

This standardization affects only the data used for breaking and matching; it does not affect the input or output file.

Match/Consolidate Job-File Reference

Option Description

No If this parameter is set to No, MCD uses last line data (city, state, ZIP Code)

as is from the database.

If this job uses extended address parsing (see “Address and Last Line Parsing

(EXT/STD/NONE)” on page 64), any ZIP Code data may be affected. For

example, the extended parsing process can generate the ZIP+4 code for the address, even if the record does not include it.

In addition, the extended parsing process might substitute a more accurate ZIP Code, based on the comprehensive information in the national ZIP+4 directory file used by the extended process.

Match on Resident, RR#, No Box (Y/N)

This parameter applies only when MCD finds an input record that contains a resident-type name (Current Resident, Occupant, blank, or name not defined) and a rural route address, with no box number. Selecting Yes places all records with this type of name data and the same rural route address into the same dupe group.

Match on Street, RR, or PO Box (Y/N)

This parameter affects business and household records matching on address. Consider the following example.

firm range street suffix PO box ZIP Code

Acme Hardware 100 Elm Ave 200 02961

Acme Hardware 123 Main St 200 02961

Option Description

Yes Records are considered a match if the PO Boxes match. If the PO Boxes do

No All forms of the address (street, rural route, and PO Box) must match for

Ignore Firm if Names Match (Y/N)

This parameter lets you work with odd abbreviations or spellings of firm names. This assumes that you are matching on two real names, firm, and address line, and breaking or matching on city or ZIP Code.

not match, then the address and rural route address must pass the match criteria settings.

In the previous example, if set to Yes, even though the street address does not match, MCD considers the records a match because the PO Box matches.

these records to be placed in the same dupe group. In the previous example, if set to No, MCD does not consider these records a match because the street address does not match.

Chapter 2: Job-file blocks and parameters

This parameter has no effect unless the Firm Match Type parameter is set to Ind. Also, it does not take effect when either name is blank, even if you have turned on blank matching on the name fields.

Option Description

Yes If you enter Yes, you’re indicating “if you find matching names at the same

address, call them dupes, even if the firms don’t match.” This lets you catch the following dupes, which might otherwise have been missed:

Rita Terranova ETI Rita Terranova Eco Technologies

No If you insist on a good match on firm name, enter No.

Addr Blank Match if Firms Match (Y/N)

Use this parameter to control blank matching when comparing firm records within the same break group.

This will only affect records when one has Street information and the other has PO Box information. If both records have Street information that do not match, or if both have PO Box information that do not match, the records will not be found as dupes. Also, any fields being used to break on will affect whether the records will be compared to each other (for example, when breaking on the Primary Name, the records will not be found as dupes).

100 Bren Rd 55343

Consider the following two records: For each of the address-line components, one or the other record is blank. This parameter affects records when one contains a street address and the other contains a post-office box address. This assumes that you are matching on firm and address line, and breaking or matching on city or ZIP Code.

This parameter has no effect when both records contain a street address, or both records contain a PO Box.

firm range street suffix PO box ZIP Code

Acme Hardware 100 Elm Ave 02961

Acme Hardware

Option Description

Yes If firm data matches and neither firm field is blank, MCD allows blank match-

ing for all address components. Records, such as those listed above, would be considered a match.

No If firm data matches, but address data in one of the records is blank, the

records will not be considered dupes (unless blank matching is turned on for those address components in the Match Criteria block).

300

02961

Match/Consolidate Job-File Reference

Upper Case M/P & UserDef Fields (Y/N)

This parameter gives you the option of storing special key fields in upper case letters. Select this option if you want MCD to match on words even if their case is different in the input data. If you do not check this option, MCD stores special key fields as they appear in the input file.

If you check this option or set this option to Yes in the job file, consider the following these examples: If you have a text string JONES, MCD leaves it all upper cased as JONES. If you have a text string jones, MCD converts it to JONES. If you have a mixed-case text string, such as JoNeS, MCD converts it to JONES.

Assuming that you want an exact match, MCD considers all three of these records to be a dupe group because the final result is that the text strings are all upper cased. If you do not check this option, or if you set this option to No in the job file, MCD considers these three examples to be unique records because each text string has different casing.

Option Description

Yes MCD stores special key fields in uppercase letters. Select this option if you

want MCD to match on words even if their case is different in the input data.

No MCD stores special key fields as they appear in the input file. Do not select

this option if case differences in the input data should cause fields to be considered unique.

Standardize Diacritical Chars (Y/N)

This parameter controls whether MCD standardizes diacritical characters. Note that the standardization of diacritical characters happens only in the key data; the input and output data is not changed.

Option Description

Y MCD standardizes diacritical characters.

N MCD does not standardize diacritical characters.

The following table shows some extended ASCII characters that MCD converts if you enter Yes for this option. The first character of a pair is the original value; the second character in the pair is the replacement value.

For example, the following table shows that MCD converts the character À to A, and the character Ä to A.

Š S Œ O Ž Z š s œ o ž z Ÿ Y À A Á A Â A Ä A Å A Æ A

Ç CÈ EÉ EÊ EË EÌ IÍ IÎ IÏ IÐ DÑ NÒ OÓ O

Ô OÕ OÖ OØ OÙ UÚ UÛ UÜ UÝ YÞ Pß Sà aá a

â aã aä aå aæ aç cè eé eê eë eì Ií Ií I

î Iï ið dñ nò oó oô oõ oö oø où uú uû u

ü uý yþ pÿ y

Chapter 2: Job-file blocks and parameters

Priority Field (OFF/DESCEND/ASCEND)

Sometimes, you may wish to prioritize duplicate records based on some piece of data within those records. For example, given two matching records, you might prefer the record with the larger donation or the earlier date of membership.

In this situation, use the PW field Priority. In the DEF file(s), define PW.Priority based on (in this example) the amount or date field, such as PW.Priority = mbrdate. Then, set this parameter to Ascend or Descend.

Option Description

Off Do not prioritize based on a field.

Descend The sequence is: z–a, Z–A, 9–0. If you set Priority to a date field, select

Descend when you prefer the later or most current date.

Ascend The sort sequence is: a-Z, A-Z, 0–9. If you set Priority to a date field, select

Ascend to give priority to the record bearing the earlier or least current date.

Match/Consolidate sorts characters according to their numeric equivalents in the ASCII character set. To be sorted correctly, numbers have to be prepadded with zeroes. For example, when sorting in ascending order, “02” will come before “10,” but “2” or “2 ” will not.

Random Priority (Y/N)

This parameter controls the final action MCD takes in attempting to break ties in priority.

Option Description

Yes MCD’s final action in sorting dupe groups to prioritize records is to use ran-

dom sortation. This means that if you run the same job twice, you may get a different set of surviving records each time.

No MCD breaks ties in favor of the driver record, and then by input file and

record number.

Match/Consolidate Job-File Reference

Match Set Defaults

This block indicates what MCD does with records that do not qualify for any match set. This is a required block only if MCD performs advanced matching.

Action (ASSIGN DEFAULT/IGNORE/ABORT)

This required parameter indicates what to do with a record that does not belong to any of the defined match sets.

Option Description

Assign Default MCD assigns the record to the default match set, which is the match

set whose name you enter at the Default Match Set Name parameter.

Ignore MCD leaves the record out of the job.

Abort The MCD job halts the first time that the program finds a record that

MCD cannot assign to any match set.

Default Match Set Name

Enter the name of the match set that MCD should assign a record to if that record does not belong to any other defined match set. This is a required parameter only if you set the Action parameter to Assign Default; otherwise, MCD ignores this parameter.

Chapter 2: Job-file blocks and parameters

Match Set

The Match Set block lets you control how MCD matches on certain records, let you segregate records, and let you match on records independently. For example, you could use one match set to point U.S records to one match criteria, and another match set to point foreign records to a different match criteria, all within the same job.

Or, you can match on name, address, and ZIP Code with one match set, and match on name and SSN in another match set so that records for the same person with multiple addresses can be brought together or associated. For more information, see “Combine Match Sets” on page 72. You can also use multiple levels within a match set to perform N-per matching, such as N-per-firm or N-per-family.

The Match Set block defines a match set and its matching strategy. The match set holds a subset of the keys formed from the records taken from the input file(s). Each match set may have its own break, match, and prioritization settings. You can also use a match set to write reports, output files, post to input files, and purge. This is a required block only if MCD performs advanced matching.

For more information about match sets, see the User’s Guide to Record Matching.

Match Set Name (to 20 chars)

This required parameter lets you assign a logical name for the block. This name is referenced by the Match Set Name parameter in other MCD blocks. Do not use this name in any other Match Set blocks or Combined Match Set blocks.

Match Set Filter (to 1024 chars)

This is an optional parameter that controls which records MCD allows to be stored in the match set defined by this block. If you leave this parameter blank, assume that MCD stores all records in this match set.

Note that the following parameters can eliminate a record from the entire job and may therefore prevent records from being stored in a match set:

 In the Input File block: Starting Record Number, Maximum Number of

Records to Input, and Input Filter. Each of these parameters can limit the number of records read into the job. For more information, see “Input File”

on page 33.

 In the Input List Defaults block: Action parameter. If you have a record that

is associated with an undefined list, the action you choose may prevent the record from being read into the job. For more information, see “Input List

Defaults” on page 37.

Match/Consolidate Job-File Reference

Exclusive Use of Records (Y/N)

This parameter controls whether this match set gets exclusive use of a record. Exclusive use of a record means that MCD does not allow any other match set to read in and process the records once they are exclusively read into a match set.

Option Description

Yes Provided the record belongs to this match set, MCD does not attempt to deter-

No The order in which you set the match sets within a job is important.

Generate Adjusted AP Fields (Y/N)

This parameter controls whether MCD generates the adjusted AP fields. To save processing time, you may choose not to generate the AP fields. For more information about adjusted records, see Chapter 12 of the User’s Guide to Record Matching.

Work File Directory (path)

This is an optional parameter that controls the directory in which MCD creates the work files and sort work files for this match set. If you leave this parameter blank, MCD uses the Work File Directory and Sort Work File Directory parameters in the Execution block.

mine if the record qualifies for any other subsequently defined match sets.

MCD issues a verification error if a Match Set block with this parameter set to Yes follows a Match Set block with this parameter set to No. A record may be assigned to a subsequent match set.

Match (Level, Criteria)

This required parameter defines the match level(s) and what matching MCD performs in each level. You may be repeat this parameter up to a total of three times. This parameter has the following arguments:

Argument Description

Match Level This argument assigns a label to a match level. The label should

have alphanumeric characters and should be no longer than 20 characters. For example: Level 1

Match Criteria This argument specifies the name of a match criteria to use to com-

pare the two records. The match criteria can either be a Match Criteria block or an Extended Match Criteria block. For example: Family

Chapter 2: Job-file blocks and parameters

Combine Match Sets

The Combine Match Sets block combines two or more match sets together. Match/Consolidate performs no matching on the combined match set; however, it associates records from individual match sets. Match/Consolidate may optionally perform relationship and adjustment operations on the input match sets’ match results after combining them into one set of match results. For information about relationship and adjustment operations, see Chapter 12 of the User’s Guide to

Record Matching.

Use the combined match set as you would a normal match set to write reports and output files, post to input files, and purge. This is an optional block, which may be repeated.

Combine Match Set Name (to 20 chars)

This required parameter lets you assign a logical name for the combined match set. The Match Set Name parameter in other MCD blocks references this name. Do not use the name specified in any other Match Set blocks or Combined Match Set blocks.

Match Fields to Store (ALL/NONE)

This parameter indicates which keys to store in the combined match set.

Option Description

All Uses all key field lengths defined in all the match sets.

None No key data is stored (except for match results). Choosing None saves disk

Prioritize Matches (ALL/NONE)

This parameter controls how MCD prioritizes matches within each match group. As described in the following table, there are two valid values for this parameter: All and None.

Option Description

All Uses all prioritization settings from all match sets appearing in the

None MCD does not perform prioritization on the combined match set.

When you select All, MCD uses the prioritization settings from all of the match sets that appear in the Combine Match Sets block. If two or more match sets contribute to a particular level in the combine match set, MCD performs a series of checks looking for conflicting prioritization settings. These checks only occur when you select All, and if any of the checks fail, MCD sends an error message during verification.

space but removes the option of generating record reports on keys and sorted output files on key data.

Combine Match Sets block.

Match/Consolidate Job-File Reference

First, MCD reviews the prioritization type for consistency. Valid prioritization types include: LIST, FLD, LIST_FLD, and FLD_LIST. Any combination of priority type is valid except for prioritization of LIST_FLD with FLD_LIST, or FLD_LIST with LIST_FLD.

The following table shows how MCD sorts the combined results if you use a combination of priority types.

Priority type of the first match set in the combined match set

Priority type contained within the remaining match sets

LIST LIST

FLD LIST_FLD FLD_LIST

FLD LIST

FLD LIST_FLD FLD_LIST

LIST_FLD LIST

FLD LIST_FLD FLD_LIST

FLD_LIST LIST

FLD LIST_FLD FLD_LIST

Sort order of combined results

LIST LIST_FLD LIST_FLD FLD_LIST

FLD_LIST FLD LIST_FLD

FLD_LIST

LIST_FLD LIST_FLD LIST_FLD Invalid

FLD_LIST FLD_LIST Invalid FLD_LIST

When using extended matching, set the prioritization type directly in the Prioritize Matches block. When using standard matching, you do not set the prioritization type directly through a parameter. Instead, based on the entry made in the Priority Field parameter of the Matching Options block, MCD indirectly sets the prioritization setting to either LIST or LIST_FLD.

When using standard matching, in the Matching Options block, a setting of Off equates to a LIST prioritization setting; a setting of Ascend/Descend equates to a LIST_FLD prioritization setting.

If the priority type is consistent and any of the prioritization settings include FLD, the priority field for each match set in the combined match set must sort in the same direction (either descend or ascend).

Once again, if the priority type is consistent and the sort includes LIST, then any common key fields from the match sets involved in the combining process must have the same blank priority score. For example, if you define Sec_Range in more than one match set, then the blank priority score must be the same in each match set that defines Sec_Range (but only if LIST prioritization is active in each match set).

Chapter 2: Job-file blocks and parameters

Random sort the combined results

You can randomly sort the combine match sets by setting the coarsest level of one of the match sets being combined to On. Coarsest level is defined as the level defined in the first Match parameter of a Match Set block.

To set a coarsest level to On:

 For standard (STD) matching, set the Random Priority parameter of the

Matching Options block to Y (Yes).

 For extended (EXT) matching, set the Break Priority Ties Randomly

parameter of the Prioritize Matches block to Y (Yes).

Build Extra AP Fields (NONE/REL/REL_ADJ)

This parameter controls which extra AP fields MCD builds. The extra AP fields include the Relationship AP fields and the Adjusted AP fields.

Option Description

None Does not build the Relationship and Adjusted AP fields.

Rel Builds the Relationship AP fields. For information about Relationship AP

fields, see Chapter 12 of the

Rel_Adj Builds both the Relationship AP fields and the Adjusted AP fields. For

information about Relationship and Adjusted AP fields, see Chapter 12 of the

User’s Guide to Record Matching

Work File Directory (path)

Match Set to Combine

This parameter, which is required and can be repeated, indicates a Match Set to combine. The name specified in this parameter must match a name specified in the Match Set Name parameter of a Match Set block defined in the job file.

Match/Consolidate Job-File Reference

Group Posting

In this block, you set up the group posting operation. Group posting salvages data from matching records before discarding them, or posts data from one or more records to specified matching records. For more information about group posting, see the User’s Guide to Record Matching.

Group Post Name (to 20 chars)

Enter a clear, descriptive name for the group posting operation. Match/ Consolidate uses this name in reports to identify this particular group posting operation. You might include the names of fields involved.

You will find examples of Group Posting blocks in the resource file group.mpg. That file is located in the MCD template subdirectory. You may copy blocks from group.mpg into the own job file.

ADV Match Level to Group Post

This parameter determines the match level on which MCD performs group posting. For example, suppose you have two match levels: level 1 and level 2. Level 1 is business match groups, and level 2 is individual match groups. You could group post a phone number at the business level (match level 1). Or, you could post a social security number at the individual level (match level 2).

This is a required parameter only if MCD performs advanced matching.

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results MCD uses. This is a required parameter only if MCD performs advanced matching.

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Posting Destination (MASTER/SUBS/ALL)

This parameter identifies the member(s) of each dupe group to which data should be posted.

Option Description

Master MCD posts only to the master record of a dupe group.

Subs Posts data to all the subordinates of the dupe group (unless filtered out),

but not to the master record. Be careful when using this option in a job that involves purging input files.

When group posting to subordinates as part of input purging, you may be group posting to records that MCD will purge.

Chapter 2: Job-file blocks and parameters

Option Description

All MCD posts to all the records (master and subordinates) when the filter is

passed. Be careful when using this option in a job that involves purging input files.

When group posting to subordinates as part of input purging, you may be group posting to records that MCD will purge.

Post Only Once Per Destination (Y/N)

This parameter controls how many times MCD posts for each destination record. No matter how you set this parameter, MCD always sorts the dupe group members in priority order.

Option Description

Yes MCD performs this group posting action to each destination record only

once the next group posting operation (if more than one is defined) or to the next destination record.

No MCD may perform this group posting action more than once to the destina-

tion record. It works through each member of the destination record’s dupe group in priority order, performing this posting action each time the filter is passed. Only then does it move on to the next group posting operation (if more than one is defined) or to the next destination record.

Once performed for that record, MCD then moves on to either perform

Group Post to Suppress Records (Y/N)

This parameter controls whether MCD posts to suppress-type records. This includes all records from suppression and non-suppression lists that have a lower priority than the suppression records in the match group. Usually, you will not want to post to suppress-type records.

Option Description

Yes Set this option to yes if you are creating a file of records that matched a

suppression file, use group posting to enhance the data in that file.

No Most often, this parameter should be set to No. When any member of a

dupe group comes from a suppress-type input list, all members of that group will usually be discarded. There is no sense wasting time on a salvage operation when you are going to discard the entire group anyway.

Group Posting Filter (to 1024 chars)

You can use a filter to determine whether MCD performs the posting operation. When the filter returns True, MCD performs the operation. When the filter returns False, MCD cancels the operation.

If you are deciding what to post, do not use the Group Posting Filter. Instead, include a filter in the Copy parameter—in particular, consider the iif function. Using a group posting filter slows the MCD process; use a filter only if it is needed.

The group posting filter performs based on the results and level identified in the output file block and not based on the results and level identified in the group posting block.

Match/Consolidate Job-File Reference

Copy (source,destination)

Enter the name of the source field, a comma, and the name of the destination field; they may be different fields. You may repeat this line as many times as necessary.

In group posting, MCD uses special prefixes on field names. As usual, the prefix identifies a field as an AP (application), PW (from the DEF file), or DB (database) field. It’s also necessary to identify whether you are referring to the source record (SRC), destination (DST), or output (OUT). The combinations are as follows:

Field Description

APSRC.

APDST.

DBSRC.

DBDST.

DBOUT.

PWSRC.

PWDST.

field

An application field pertaining to the source record.

An application field pertaining to the destination record.

A database field in the source record.

A database field in the destination record.

A database field in the output record.

A PW field defined in the source record.

A PW field defined in the destination record.

As the source, use the special prefixes on all field names, as shown above. You may include database functions and operators to modify the source data before it is posted.

The destination field may be a DB field or a singular PW field; use the prefix DBDST, PWDST, or DBOUT. Match/Consolidate issues an error if the destination PW field is based on more than one DB field. When using DBDST or PWDST, if the destination field does not exist in the format of the input record, the operation is canceled. The output field is a field in the output file.

These prefixes apply only to group posting because only here do we work with two records at a time. In all other situations where we refer to field names in a job file, such as in all PW products, we work with only one record at a time.

If the Group Posting block contains more than one Copy parameter, all of the Copy parameters must copy to DBDST or PWDST, or they must all copy to DBOUT. You cannot combine copying to DBDST or PWDST and copying to DBOUT in the same Group Posting block.

Chapter 2: Job-file blocks and parameters

Create File for Output

File Name (path & file name)

Enter a name for the output file. By default, MCD places the file in the current directory; if you want it placed somewhere else, enter a path in the entry.

You can use the shortcut $job. For example, the entry here might be $job.dbf; then, if the job file is named example.mpg, the output file would be named example.dbf. However, if you also use the $job shortcut when naming the input file, be sure to set a different location for the output file, or you might have a file name conflict. Similarly, do not use $job.out.

If you’re creating multiple output files, use $job and a number or letter to make each name unique. For example, suppose in the region.mpg job, you’re creating four output files. You could create them as shown at right.

File Type (DBASE3, ASCII, DELIMITED, EBCDIC)

Enter the type of output file you want to create. For more information about file types, see Database Prep.

Depending on the type you select, MCD may create supporting files (such as FMT) to go with the output file. For more information about creating supporting files, see Database Prep.

Create DEF File (Y/N)

This parameter controls whether MCD should automatically create a DEF file. If there is an existing output DEF file that matches the output file name, MCD doesn’t change it regardless of the setting at this parameter.

Option Description

Yes Enter Yes if you want MCD to automatically create a DEF file when it cre-

ates the output file. Use this if you plan to use the MCD output file as input to another Business Objects product. Because the only information placed in the DEF file is the DATABASE TYPE line, you will need to edit the DEF file to add field definitions.

Job-file entry Actual result

$jobN.dbf regionN.dbf

$jobW.dbf regionW.dbf

$jobE.dbf regionE.dbf $jobS.dbf regionS.dbf

No Enter No if you want MCD to use an existing DEF file. Enter No if you

plan to use the output file as input to another Business Objects product and you use a default DEF file with that product.

Match/Consolidate Job-File Reference

Rec Format to Clone (path & file name)

You may specify the physical format of the new output file by cloning or borrowing the format of another existing database. Enter the full path name of that file here. The pathname need not be one of the input files for this job.

If you are cloning a Unicode-encoded file, the file may contain a 2-byte binary field (that represents the end of a record). If it does, you must use AP.Newline in the Copy parameter of the output file block(s).

For example, assume that you are cloning a Unicode-encoded file for output that contains EOR,2,b to produce a carriage return, line feed (CR LF). In this case, you must override the EOR by using AP.Newline in the Copy parameter.

Match/Consolidate duplicates only the physical format of that file, such as field lengths and names, not its data. You may append more fields to the end of each record by using the Field parameter as explained below.

Leave this parameter blank if you want to define a new physical format, not based on any other file.

Field (name,length,type[,misc])

Specify the name, length, data type, and format of one field. Separate these items by placing a comma between them. The entries at this parameter will look just like entries in an FMT or DMT file; for guidelines, see Database Prep.

Repeat the Field parameter as many times as necessary to create all the fields that the output records should include.

Chapter 2: Job-file blocks and parameters

Match/Consolidate Output File

In this block, you choose settings for a MCD Output File. The following kinds of records qualify for the MCD output file:

 Unique Records: Not members of any dupe group. They come from Normal

and Special input lists.

 Single List Masters: May come from Normal or Special input lists. They are

the masters of single-list dupe groups; these are groups whose members all came from the same input list.

 Multiple List Masters: May come from Normal or Special input lists. They

are the masters of multi-list dupe groups, such as groups whose members came from more than one input list.

File Name (path & file name)

Enter a name for the output file. By default, MCD places the file in the current directory; if you want it placed somewhere else, include a path in the entry. Unless you are appending records to a file that already exists, you will also need a Create File for Output block.

If you are using a Create File for Output block, the same path and file name must be entered in both blocks. The Create File for Output block makes the structure of the file, and the MCD Output File block puts data into that structure. Both blocks must point to the same file.

Existing File (APPEND/REPLACE)

This parameter takes effect only if the output file (named above) already exists. If you are making a new file using a Create File for Output block, MCD ignores this parameter.

Option Description

Append Add the output records to the end of the file.

Replace Overwrite the file and replace its contents with the new output.

ADV Select Match Set(s) (name[,name…])

This is a required parameter only if MCD performs advanced matching. This parameter lets you specify a comma-separated list of match sets to which this block applies. For example: Match Set 1, Match Set 2, Match Set 4.

In this example, if a job contains four match sets, this block reports on Match Sets 1, 2, and 4, but does not report on Match Set 3.

Match/Consolidate Job-File Reference

ADV Match Level (FINEST/SELECT)

This parameter controls what match level to use to determine whether a record is output. It also determines to which level the AP prefix points. This is a required parameter only if MCD performs advanced matching.

Option Description

Finest Uses the match set’s finest level. For example, if you have three match level

criteria parameters defined, the last parameter listed is considered to be “finest.” For more information, see the Match parameter in the Match Set block on page 71.

Select Uses the match level specified in the ADV Select Match Level to

Output parameter.

ADV Select Match Level to Output

This parameter lets you specify the match level that you want to use. This is a required parameter only if MCD performs advanced matching. This is also a required parameter if the ADV Match Level parameter is set to Select. Otherwise, MCD ignores this parameter.

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results MCD uses. This is a required parameter only if MCD performs advanced matching.

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Group Posting (NONE/ALL/SELECT)

Group posting salvages data from matching records before discarding them, or posts data from one record to all matching records. For more information about group posting, see the User’s Guide to Record Matching.

Option Description

None Does not perform group posting.

All Uses all Group Posting blocks defined in the job file.

Select Uses Group Posting blocks specified in the Select Group Posting parameter.

Select Group Posting (name[,name…])

This parameter lets you specify a comma-separated list of Group Posting blocks to which this block applies. This is a required parameter if the Group Posting parameter is set to Select.

Chapter 2: Job-file blocks and parameters

Sort By

Choose the sequence of records in the output file.

Option Description

File Outputs records in the same sequence as the input file(s). Choose this

option if speed is most important.

Random Assigns records a random number and sorts by that number. Choose this

option when you want to produce a test mailing.

Dup Sorts records by dupe group number (AP.Group_No). Keeps dupe

groups together in the All-Duplicates Output File.

MP1 Sorts records by their value in the key-field Merg_Purg1.

Geo Sorts records geographically by state, city, ZIP, street name, street range,

PO Box, rural route number, last name, first name, and firm.

LB_Prior Sorts records on the application field AP.LB_Prior. Choose this option

to output the highest-priority records.

List_Cnt Sorts records on the application field AP.List_Cnt. Useful when creating

a multi-buyer output file.

Group_Cnt Sorts records on the application field AP.Group_Cnt. Useful when creat-

ing a multi-occurrence output file.

Custom Sorts records as defined in the User Custom Sort block (see the next

parameter).

User Custom Sort

If you have set up a Custom Output Sorting block, you can implement that sort by entering its name at this parameter. Regardless of how many Custom Output Sorting blocks you may have set up in this job, only the one named in this block is activated for this output file.

Sort Order (ASCEND/DESCEND)

Once you have chosen a method of sorting, you can choose the direction. This parameter has no effect if a Custom Output Sorting block has been selected in the User Custom Sort parameter.

Option Description

Ascend Sort 0–9, A–Z, a–z

Descend Sort 9-0, Z-A, z–a

Output Records Previously Output (Y/N)

This parameter controls whether MCD writes to the output file records that have already been written to another output file, earlier in the job.

Option Description

Yes Enter Yes if you want to output records that have already been written to

another output file, earlier in this job.

Match/Consolidate Job-File Reference

Option Description

No Enter No if you don’t want to output any records that have already been out-

put in this job. Note that when using ADV matching with multiple match sets, each match

set works independently of each other. Selecting No excludes only records that have been output from the match set listed in the appropriate output block.

For example, if you have two match sets and you want to output record 1 from match set 1, you can still output record 1 from match set 2.

Output Filter (to 1024 chars)

You can use output filters to decide whether a record is included in the output file. Leave this line blank if you don’t want to use a filter.

The filter expression must result in a True/False decision. Records that pass the filter (return True) are included in the output file. Records that do not pass the filter (return False) are omitted.

The filter acts only on those records that already qualify for placement into the output file: unique records, single-list masters, and multiple-list masters.

Nth Select Type (USER/AUTO/RANDOM) User Nth Select (1.0-???)

To limit the number of records in the output file (to set up test mailings, split output into multiple segments, etc.), many users would prefer to output records at intervals — typically every third record, every fifth record, etc. We call this “Nth select.”

Option Description

User MCD selects every “Nth” record, using as N the number you enter in the

User Nth Select parameter. (for example, 5 for every 5th record).

Auto MCD calculates that increment based on the setting at the Maximum

Number of Records to Output parameter. Leave the User Nth Select parameter blank.

Random MCD selects at random which records to output, up to the setting at the

Maximum Number of Records to Output parameter.

Match/Consolidate applies “Nth select” after applying sorting and filters. Any record that does not pass the filter is not included in any computations. If you enter either Auto or Random and there is an output filter, MCD takes more processing time to count the records that pass that filter before outputting the records.

Regardless of which type of “Nth select” you use, MCD respects the Maximum Number of Records to Output setting.

If you specify decimals for the User Nth Select parameter, MCD rounds the decimal down to the nearest whole number. For example, assume that you want to output four of ten records, and you specify the decimal 2.5 as the User Nth Select. In this example, MCD outputs record 2 (2.5 rounded down to 2), record 5, record 7 (rounded down from 7.5), and record 10.

Chapter 2: Job-file blocks and parameters

Maximum Number of Records to Output

You might enter a number here if, for example, you’re preparing a test mailing to 500 names. Leave this parameter blank to output all eligible records. Do not enter commas; for example, enter 10000, not 10,000.

Character Encoding

This optional parameter indicates the character encoding in which MCD stores data. If you leave this parameter blank, MCD defaults the character encoding to ASCII.

Unicode Conversion Name

This parameter assigns a name or label, which lets parameters in other blocks, such as the Unicode Conversion parameter in the Input File block, reference this block.

Copy Input Data to Output File (Y/N)

The following options exist for this parameter.

Option Description

Yes Enter Yes to automatically copy data from input records to output

No If you would rather post everything yourself, enter No and use the Copy

Copy (source, destination)

Use the Copy parameter for output posting—that is, to copy information into the fields of the output file. You may post DB, PW, or AP fields, or constants.

You may use filter functions to modify data before posting it. Enter first the information that you want to post, then a comma, then the name of the field in the output file. Repeat this line as many times as necessary.

records. MCD looks for field names that appear both in the input and the output file; this data is copied over to the output file.

If a field exists in the input file but not in the output file, its data is not copied over to the output file. If a field exists in the output file but not in the input file, the output field will be blank unless you use the copy parameter.

If you enter Yes, you may also use the Copy parameter to post additional information to the output file, over and above the automatic copying.

parameter for all postings.

Match/Consolidate Job-File Reference

All-Duplicates Output File

This output file includes all duplicate records (masters and subordinates) from all list types (suppress, normal, or special). Match/Consolidate does not include unique records in the output file.

File Name (path & file name)

Enter a name for the output file. By default, MCD places the file in the current directory; if you want it placed somewhere else, indicate a path in this entry.

If you are using a Create File for Output block, enter the same path and file name in both blocks. The Create block creates the structure of the file, and the AllDuplicates block puts data into that structure.

Existing File (APPEND/REPLACE)

This parameter takes effect only if the output file (named above) already exists. If you are creating a new file (using a Create File block), this parameter has no effect.

Option Description

Append Use this option to add the output records to the end of the file.

Replace Use this option to overwrite the file and replace its contents with the

new output.

ADV Select Match Set(s) (name[,name…])

In this example, if a job contains four match sets, this block reports on Match Sets 1, 2, and 4, but does not report on Match Set 3.

ADV Match Level (FINEST/SELECT)

Option Description

Finest Uses the match set’s finest match level. For example, if you have three

match levels defined, the last level listed is considered to be the “finest.” For more information, see “Match (Level, Criteria)” on page 71.

Select Uses the match level specified in the ADV Select Match Level to

Output parameter.

ADV Select Match Level to Output

This parameter lets you specify the match level that you want to use. This is a required parameter only if MCD performs advanced matching, and if the ADV Match Level parameter is set to Select. Otherwise, MCD ignores this parameter.

Chapter 2: Job-file blocks and parameters

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results MCD uses. This is a required parameter only if MCD performs advanced matching. In addition, this is a r e q u i r e d p a r a m e t e r i f t h e b l o c k s p e c i f i e s a m a t c h s e t t h a t h a s m o r e t h a n o n e l e v e l .

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Group Posting (NONE/ALL/SELECT)

Group posting salvages data from matching records before discarding them or posts data from one record to all matching records. For more information about group posting, see the User’s Guide to Record Matching.

Option Description

None Does not perform group posting.

All Uses all Group Posting blocks defined in the job.

Select Uses Group Posting blocks specified in the Select Group Posting parameter.

Select Group Posting (name[,name…])

This parameter lets you specify a comma-separated list of Group Posting blocks to which this block applies. This is a required parameter if you set the Group Posting parameter to Select.

Sort By

Use this parameter to choose the sequence of records in the output file. Users most often use the first three options for an All-Duplicates file; the others may be less appropriate.

Option Description

File Outputs records in the same sequence as the input file(s). Choose this

option if speed is most important.

Random Assigns records a random number and sorts by that number. Choose this

option when you want to produce a test mailing.

Dup Sorts records by dupe group number (AP.Group_No). Keeps dupe groups

together in the All-Duplicates Output File.

MP1 Sorts records by their value in the key-field Merg_Purg1.

Geo Sorts records geographically by state, city, ZIP, street name, street range,

PO Box, rural route number, last name, first name, and firm.

LB_Prior Sorts records on the application field AP.LB_Prior. Choose this option to

output the highest-priority records.

List_Cnt Sorts records on the application field AP.List_Cnt. Useful when creating a

multi-buyer output file.

Match/Consolidate Job-File Reference

Option Description

Group_Cnt Sorts records on the application field AP.Group_Cnt. Useful when creating

a multi-occurrence output file.

Custom Sorts records as defined in the User Custom Sort block (see the User Cus-

tom Sort parameter).

User Custom Sort

Sort Order (ASCEND/DESCEND)

After choosing a method of sorting, you can choose the direction to sort. This parameter has no effect if you select a Custom Output Sorting block.

Option Description

Ascend Sort 0–9, A–Z, a–z

Descend Sort 9-0, Z-A, z–a

Output Filter (to 1024 chars)

You can use output filters to decide whether a record is included in the output file. Leave this line blank if you don’t want to use a filter.

The filter expression must result in a True/False decision. Match/Consolidate includes records that pass the filter (return True) in the output file. Match/ Consolidate omits records that do not pass the filter (return False).

Nth Select Type (USER/AUTO/RANDOM) User Nth Select (1.0-???)

To limit the number of records in the output file (to set up test mailings, split output into multiple segments, and so on), many users prefer to output records at intervals—typically every third record, every fifth record, etc. We call this “Nth select.”

Option Description

User MCD selects every “Nth” record, using N as the number you enter in the

User Nth Select parameter.

Auto MCD calculates that increment based on the setting at the Maximum Num-

ber of Records to Output parameter. Leave the User Nth Select parameter blank.

Random MCD selects at random which records to output, up to the setting at the

Maximum Number of Records to Output parameter.

Chapter 2: Job-file blocks and parameters

processing time to count the records that pass that filter before outputting the records.

Regardless of which type of “Nth select” you use, MCD adheres to the Maximum Number of Records to Output setting.

If you specify decimals for the User Nth Select parameter, MCD rounds the decimal down to the nearest whole number.

For example, assume that you want to output four of ten records, and you specify the decimal 2.5 as the User Nth Select. In this example, MCD outputs record 2 (2.5 rounded down to 2), record 5, record 7 (rounded down from

7.5), and record 10.

Maximum Number of Records to Output

To output all duplicate records, leave this parameter blank. If you would like to limit the size of the output file, you may set a Maximum Number of Records. Do not enter commas; for example, enter 10000, not 10,000.

Character Encoding

This optional parameter indicates the character encoding in which MCD stores data. If you leave this parameter blank, MCD defaults the character encoding to ASCII.

Unicode Conversion Name

This parameter assigns a name or label, which lets parameters in other blocks, such as the Unicode Conversion parameter in the Input File block, reference this block.

Copy Input Data to Output File (Y/N)

This parameter controls the process of copying input records to output records.

Option Description

Yes Enter Yes to automatically copy data from input records to output records.

MCD looks for field names that appear both in the input and the output file; this data is copied over to the output file.

If you enter Yes, you may also use the Copy parameter to post additional information to the output file, over and above the automatic copying.

No If you would rather post everything yourself, enter No and use the Copy

parameter (explained below) for all posting.

Match/Consolidate Job-File Reference

Copy (source, destination)

Use the Copy parameter for output posting—that is, to copy information into the fields of the output file. You may post DB, PW, or AP fields, or constants.

Chapter 2: Job-file blocks and parameters

Multi-Occurrence Output File

In this block, you choose settings for a multi-occurrence output file. The following kinds of records qualify for the multi-occurrence output file:

 Single List Masters: May come from Normal or Special input lists. They are

the masters of single-list dupe groups, such as groups whose members all came from the same input list.

 Multiple List Masters: May come from Normal or Special input lists. They

are the masters of multi-list dupe groups, such as groups whose members came from more than one input list.

File Name (path & file name)

Enter a name for the output file. By default, MCD places the file in the current directory; if you want it placed somewhere else, indicate a path in the entry. If the output file is a new file, or if you want to replace a file that already exists, you will also need a Create File for Output block.

If you are using a Create File block, enter the same path and file name in both blocks. The Create File block makes the structure of the file, and the MultiOccurrence Output File block puts data into that structure.

Existing File (APPEND/REPLACE)

This parameter takes effect only if the output file (named above) already exists. If you are making a new file (using a Create File block), this parameter has no affect.

Option Description

Append Use this option to add the output records to the end of the file.

Replace Use this option to overwrite the file and replace its contents with the

new output.

ADV Select Match Set(s) (name[,name…])

In this example, if a job contains four match sets, this block reports on Match Sets 1, 2, and 4, but does not report on Match Set 3.

ADV Match Level (FINEST/SELECT)

Option Description

Finest Uses the match set’s finest level (largest level number).

Select Uses the match level specified in the ADV Select Match Level to

Output parameter.

Match/Consolidate Job-File Reference

ADV Select Match Level to Output

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results MCD uses. This is a required parameter only if MCD performs advanced matching.

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Minimum Dupe Group Size (2-??)

Enter a number to specify the minimum dupe group size to be included in the output file. The lowest valid number is 2.

Group Posting (NONE/ALL/SELECT)

Group posting is a way to salvage data from matching records before discarding them, or a way to post data from one record to all matching records. For more information about group posting, see the User’s Guide to Record Matching.

Option Description

None Does not perform group posting.

All Uses all Group Posting blocks defined in the job file.

Select Uses Group Posting blocks specified in the Select Group Posting parameter.

Select Group Posting (name[,name…])

This parameter lets you specify a comma-separated list of Group Posting blocks to which this block applies. This is a required parameter if the Group Posting parameter is set to Select.

Sort By

Choose the sequence of records in the output file.

Option Description

File Outputs records in the same sequence as the input file(s). Choose this

option if speed is most important.

Random Assigns records a random number and sorts by that number. Choose this

option when you want to produce a test mailing.

Dup Sorts records by dupe group number (AP.Group_No). Keeps dupe

groups together in the All-Duplicates Output File.

Chapter 2: Job-file blocks and parameters

Option Description

MP1 Sorts records by dupe group number (AP.Group_No). Keeps dupe

Geo Sorts records geographically by state, city, ZIP, street name, street range,

LB_Prior Sorts records on the application field AP.LB_Prior. Choose this option

List_Cnt Sorts records on the application field AP.List_Cnt. Useful when creating

Group_Cnt Sorts records on the application field AP.Group_Cnt. Useful when

Custom Sorts records as defined in the User Custom Sort block (see the User

User Custom Sort

groups together in the All-Duplicates Output File.

PO Box, rural route number, last name, first name, and firm.

to output the highest-priority records.

a multi-buyer output file.

creating a multi-occurrence output file.

Custom Sort parameter).

Sort Order (ASCEND/DESCEND)

Choose the direction to sort. This parameter has no effect if a Custom Output Sorting block is named (above).

Option Description

Ascend Sort 0–9, A–Z, a–z

Descend Sort 9-0, Z-A, z–a

Output Records Previously Output (Y/N)

This parameter controls whether MCD writes to the output file records that have already been written to another output file, earlier in the job.

Option Description

Yes Enter Yes if you want to output records that have already been written to

another output file, earlier in this job.

No Enter No if you don’t want to output any records that have already been out-

put in this job. Note that when using ADV matching with multiple match sets, each match

set works independently of each other. Selecting No excludes only records that have been output from the match set listed in the appropriate output block.

For example, if you have two match sets and you want to output record 1 from match set 1, you can still output record 1 from match set 2.

Match/Consolidate Job-File Reference

Output Filter (to 1024 chars)

You can use output filters to decide whether a record will be included in the output file. Leave this line blank if you don’t want to use a filter.

The filter expression must result in a True/False decision. Records that pass the filter (return True) are included in the output file. Records that do not pass the filter (return False) are omitted.

Nth Select Type (USER/AUTO/RANDOM) User Nth Select (1.0-???)

Option Description

User MCD selects every “Nth” record, using as N the number you enter in the

User Nth Select parameter.

Auto MCD calculates that increment based on the setting at the Maximum

Number of Records to Output parameter. Leave the User Nth Select parameter blank.

Random MCD selects at random which records to output, up to the setting at the

Maximum Number of Records to Output parameter.

If you specify decimals for the User Nth Select parameter, MCD rounds the decimal down to the nearest whole number.

7.5), and record 10.

Maximum Number of Records to Output

You might enter a number here if, for example, you’re preparing a test mailing to 500 names. To output all eligible records, leave this parameter blank. Do not enter commas; for example, enter 10000, not 10,000.

Character Encoding

This optional parameter indicates the character encoding in which MCD stores data. If you leave this parameter blank, MCD defaults the character encoding to ASCII.

Chapter 2: Job-file blocks and parameters

Unicode Conversion Name

This parameter assigns a name or label, which lets parameters in other blocks, such as the Unicode Conversion parameter in the Input File block, reference this block.

Copy Input Data to Output File (Y/N)

The following table lists the Copy Input Data to Output File options.

Option Description

Yes Enter Yes to automatically copy data from input records to output records.

MCD looks for field names that appear both in the input and the output file; this data is copied over to the output file.

If you enter Yes, you may also use the Copy parameter to post additional information to the output file, over and above the automatic copying.

No If you would rather post everything yourself, enter No, and use the Copy

parameter for all posting.

Copy (source, destination)

Use the Copy parameter for output posting—that is, to copy information into the fields of the output file. You may post DB, PW, or AP fields, or constants.

Match/Consolidate Job-File Reference

Custom Match/Consolidate Output File

This output file lets you customize the output by selecting which record types to include in the file.

File Name (path & file name)

Enter a name for the output file. By default, MCD places the file in the current directory; if you want it placed somewhere else, indicate a path in this entry.

If you are using a Create File for Output block, enter the same path and file name in both blocks. The Create block creates the structure of the file, and the Custom MCD Output File block puts data into that structure.

Existing File (APPEND/REPLACE)

This parameter takes effect only if the output file (named above) already exists. If you are making a new file (using a Create File block), this parameter has no effect.

Option Description

Append Adds the output records to the end of the file.

Replace Overwrites the file and replace its contents with the new output.

ADV Select Match Set(s) (name[,name…])

In this example, if a job contains four match sets, this block reports on Match Sets 1, 2, and 4, but does not report on Match Set 3.

ADV Match Level (FINEST/SELECT)

Option Description

Finest Uses the match set’s finest match level. For example, if you have three

match level criteria parameters defined, the last parameter listed is considered to be finest.

For more information, see the Match parameter in the Match Set block on

page 71.

Select Uses the match level specified in the ADV Select Match Level toOutput

parameter.

ADV Select Match Level to Output

This parameter lets you specify the match level that you want to use. This is a required parameter if you set the ADV Match Level parameter to Select; otherwise, MCD ignores this parameter.

Chapter 2: Job-file blocks and parameters

ADV Match Results to Use (STD/REL/ADJ)

This parameter controls which match results MCD uses. This is a required parameter only if MCD performs advanced matching.

Option Description

STD Uses the standard match results.

REL Uses the relationship match results.

ADJ Uses the adjusted match results.

Group Posting (NONE/ALL/SELECT)

Option Description

None Does not perform group posting.

All Uses all Group Posting blocks defined in the job file.

Select Uses Group Posting blocks specified in the Select Group Posting parameter.

Select Group Posting (name[,name…])

This parameter lets you specify a comma-separated list of Group Posting blocks to which this block applies. This is a required parameter if the Group Posting parameter is set to Select.

Sort By

Use this parameter to choose the sequence of records in the output file.

Option Description

File Outputs records in the same sequence as the input file(s). Choose this

option if speed is most important.

Random Assigns records a random number and sorts by that number. Choose this

option when you want to produce a test mailing.

Dup Sorts records by dupe group number (AP.Group_No). Keeps dupe groups

together in the All-Duplicates Output File.

MP1 Sorts records by their value in the key-field Merg_Purg1.

Geo Sorts records geographically by state, city, ZIP, street name, street range,

PO Box, rural route number, last name, first name, and firm.

LB_Prior Sorts records on the application field AP.LB_Prior. Choose this option to

output the highest-priority records.

List_Cnt Sorts records on the application field AP.List_Cnt. Useful when creating a

multi-buyer output file.

Group_Cnt Sorts records on the application field AP.Group_Cnt. Useful when creating

a multi-occurrence output file.

Match/Consolidate Job-File Reference

Option Description

Custom Sorts records as defined in the User Custom Sort block (see the User Cus-

tom Sort parameter).

User Custom Sort

Sort Order (ASCEND/DESCEND)

Once you have chosen a method of sorting, you can choose the direction to sort. This parameter has no effect if you select a Custom Output Sorting block.

Option Description

Ascend Sort 0–9, A–Z, a–z

Descend Sort 9-0, Z-A, z–a

Output Records Previously Output (Y/N) ... Output Suppress List Subord (Y/N)

These parameters refer to ten categories of records as listed in the following table. On each line, enter Yes to include records in the output file, or No to omit them. To be included in the output file, a record must be in one of the categories for which you have set Yes. (It also must pass other controls such as output filter or Nth select.)

In the master.mpg file, we have preset these parameters to deliver the same results that you would get with a conventional MCD Output File. You may change our presets as the needs dictate.

Option Description

Records Previously Output Records that have already been written to another output

file, earlier in this job.

Unique Records Not members of any dupe group. They come from Nor-

mal and Special input lists.

Single List Masters May come from Normal or Special input lists. They are

the masters of single-list dupe groups—that is, groups whose members all came from the same input list.

Single List Dupes May come from Normal or Special input lists. They are

the subordinate members of single-list dupe groups.

Multiple List Masters May come from Normal or Special input lists. They are

the masters of multi-list dupe groups—that is, groups whose members came from more than one input list.

Multiple List Dupes May come from Normal or Special input lists. They are

the subordinate members of multi-list dupe groups.

Chapter 2: Job-file blocks and parameters

Option Description

Suppress Dupes Records from a Normal or Special list that belong to a

Suppress List Uniques Unique records from an input Suppress list.

Suppress List Masters Records that came originally from an input Suppress list.

Suppress List Subord Records that came originally from an input Suppress list.

Output Filter (to 1024 chars)

You can use output filters to decide whether MCD includes a record in the output file. Leave this line blank if you don’t want to use a filter.

The filter expression must result in a True/False decision. Records that pass the filter (return True) are included in the output file. Records that do not pass the filter (return False) are omitted.

Nth Select Type (USER/AUTO/RANDOM) User Nth Select (1.0 - ???)

To limit the number of records in the output file (to set up test mailings, split output into multiple segments, and so on), many users would prefer to output records at intervals—typically every third record, every fifth record, and so on. We call this “Nth select.”

dupe group that contains a record from a Suppress list with a higher priority.

They are masters of suppress dupe groups.

They are subordinate members of suppress dupe groups.

Option Description

User MCD selects every “Nth” record, using N as the number you enter in the

User Nth Select parameter.

Auto MCD calculates that increment based on the setting at the Maximum Num-

ber of Records to Output parameter. Leave the User Nth Select parameter blank.

Random MCD selects at random which records to output, up to the setting at the

Maximum Number of Records to Output parameter.

If you specify decimals for the User Nth Select parameter, MCD rounds the decimal down to the nearest whole number.

7.5), and record 10.

Match/Consolidate Job-File Reference

Maximum Number of Records to Output

Character Encoding

This optional parameter indicates the character encoding in which MCD stores data. If you leave this parameter blank, MCD defaults the character encoding to ASCII.

Unicode Conversion Name

This parameter assigns a name or label, which lets parameters in other blocks, such as the Unicode Conversion parameter in the Input File block, reference this block.

Copy Input Data to Output File (Y/N)

This parameter controls copying input records to output records.

Copy (source, destination)

Option Description

Yes Enter Yes to automatically copy data from input records to output records.

MCD looks for field names that appear both in the input and the output file; this data is copied over to the output file. If a field exists in the input file but not in the output file, its data is not copied over to the output file.

If you enter Yes, you may also use the Copy parameter (explained below) to post additional information to the output file, over and above the automatic copying.

No If you would rather post everything yourself, enter No and use the Copy

parameter (explained below) for all posting.

Use the Copy parameter for output posting—that is, to copy information into the fields of the output file. You can post DB, PW, or AP fields, or constants.

Chapter 2: Job-file blocks and parameters

Custom Output Sorting

Custom Sort Name (20 chars)

Enter a name (up to 20 characters) for this sorting setup. This must be the name you entered in the User Custom Sort parameter of any output file block upon which you want to apply this custom sort. The name you enter here is also used on the Sorted Records report.

Custom Sort Field (name,order)

You may use key fields (from Match Criteria or Parsing and Key Field Options) or AP fields for sorting. The list below includes all possible key fields and those AP fields on which you can sort. You can sort only on those key fields you are using in the job.

Enter the field name, a comma, and Ascend or Descend to define the order you want to sort the output based on this field. Repeat this parameter for each additional sort field; you can have up to sixteen. Match/Consolidate performs the multilevel sort in the order that fields are listed in this block.

When you use key fields, MCD sorts based on the data in key fields, rather than the data in the database fields. Therefore, the parameters that control standardizing of key data can affect the sorted results.

Key Fields AP Fields

birthdate oth_post file_no

city phone group_cnt

country po_box group_no

error_code postdir group_ord

firm pre_name lb_prior

firmloc predir list_cnt

first_name prim_name list_no

gender prim_range parse

last_name rec_type record_no

mat_post rr_box super_cnt

merg_purg1 rr_number unique_no

merg_purg2 sec_range

merg_purg3 pmb

merg_purg4 ssn

merg_purg5 state

100

merg_purg6 suffix

merg_purg7 title

merg_purg8 unp_addr

merg_purg9 unp_lline

merg_purg0 zip

mid_name zip4

Match/Consolidate Job-File Reference

SAP Match/Consolidate 8.00c Job-File Reference

Specifications and Main Features

Frequently Asked Questions

User Manual

Match/Consolidate

Job-File Reference

Contents

Preface

Match/Consolidate Job

Conventions

Install Match/Consolidate

Windows When you run install setup for Windows operating systems, the program

UNIX The UNIX directory structure is shown below. You must create the postware and

Run a Match/Consolidate job

Input

Results Here are the results that you may expect from MCD processing:

Work files While processing, MCD stores internal information in work files. These files are

Steps in Match/Consolidate processing

Read records and create key file

Find duplicates The second step is to find duplicates. You will choose the logic that MCD uses to

Purge dupes or create output

Run the Match/Consolidate job

Sample job We’ve sent you a sample MCD job that is ready to run. Running this job:

Job description It’s impossible for one sample to apply perfectly to all users. However, we’ve

Create jobs from our sample

Windows commands Complete the following steps to run the Quick Start sample job. If you installed

UNIX commands To run the sample job, type the commands shown below. We’re assuming that

Create Match/Consolidate jobs

Quick Start Now that you’ve run the Quick Start job and you know that it runs, you can adapt

Templates Templates are job files that are nearly ready to use; they require just a few

Master The master job file is called master.mpg. It contains one of each type of block

Resources The resource file match.mpg is not a complete job file. Instead, it contains

Extended matching files

Editing tips To edit job files, you will need working knowledge of a good text editor or word-

Guidelines for editing job files

Match/Consolidate command line

Checklist Before running MCD, make sure that you have finished the following prepwork:

Basic command line To run the job, type the MCD command, followed by the path and name of the

Options See the Quick Reference for a complete list of options for the MCD command

Messages during verification and processing

Verifier messages The MCD job-file verifier is a part of the pwmpg program. It checks the job and

Correct errors For command-line processing, the verifier catches only one error per run. Note

Process messages Match/Consolidate reports progress by printing messages on the screen. The

Template job files

dedupe.mpg We’ve received a client’s mailing list, to prepare it for a direct-mail campaign.

good_out.mpg This job is just like the first, with one change: Instead of deleting dupes from the

suppress.mpg Our company is a member of the Direct Mail Association. Four times per year we

fishing.mpg A common variation on the suppression job is to use the house file as a

multibuy.mpg Our company mails special offers and coupons to frequent buyers; that is, people

firm_pkg.mpg We’re a magazine publisher, mailing technical and engineering titles at second

General

Job description Enter a name up to 80 characters for the job. This information is printed in the Job

Job owner Enter a name up to 20 characters of the job owner. Match/Consolidate prints this

Execution

Auxiliary Files

Standard processing When you enable the standard option for Name, Title, and firm Parsing or

Extended processing When you enable the extended option for Name, Title, and firm Parsing and

Unicode Conversion

Input File

Input List Defaults

Input List Description

Super List Defaults

Super List Description

Post to Input File

Custom Purge Input File(s)

Match Qualification

Match Criteria

Match Options

Match Set Defaults

Match Set

Combine Match Sets

Group Posting

Create File for Output

Match/Consolidate Output File

All-Duplicates Output File

Multi-Occurrence Output File

Custom Match/Consolidate Output File

Custom Output Sorting