SAP Business objects DATA INTEGRATOR Getting Started Guide
Data Integrator Getting Started Guide
Data Integrator Getting Started Guide
Data Integrator 11.7.2
for Windows and UNIX
Copyright
© 2007 Business Objects. All rights reserved.
Trademarks
Third-party
contributors
Patents
Date
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal
Xcelsius, Crystal Decisions, Intelligent Question, Desktop Intelligence, Crystal Enterprise,
Crystal Analysis, Web Intelligence, RapidMarts, and BusinessQuery are trademarks or
registered trademarks of Business Objects in the United States and/or other countries. All
other names mentioned herein may be trademarks of their respective owners.
Business Objects products in this release may contain redistributions of software licensed
from third-party contributors. Some of these individual components may also be available
under alternative licenses. A partial listing of third-party contributors that have requested or
permitted acknowledgments, as well as required notices, can be found at:
http://www.businessobjects.com/thirdparty
Business Objects owns the following U.S. patents, which may cover products that are offered
and licensed by Business Objects: 5,555,403; 6,247,008; 6,289,352; 6,490,593; 6,578,027;
6,768,986; 6,772,409; 6,831,668; 6,882,998 and 7,139,766.
April 26, 2007
If you find any problems with this documentation, please report them to Business Objects S.A.
in writing at documentation@businessobjects.com.
2 Data Integrator Getting Started Guide
Contents
Chapter 1 Introduction 11
About this guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Audience and assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Data Integrator product documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Suggested reading path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Chapter 2 Introducing Data Integrator 17
About this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Data Integrator and the Business Objects Product Suite . . . . . . . . . . . . . . 18
Data Integrator product benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Unification with the platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Ease of use and high productivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
High availability and performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Data Integrator associated products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Composer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Metadata Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Data Integrator interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Chapter 3 Data Integrator Architecture 25
About this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Standard Data Integrator components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Data Integrator Designer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Data Integrator repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Data Integrator Job Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Data Integrator engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Data Integrator Access Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Data Integrator Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Data Integrator Metadata Reports applications . . . . . . . . . . . . . . . . . . 29
Impact and Lineage Analysis reports . . . . . . . . . . . . . . . . . . . . . . 30
Data Integrator Getting Started Guide 3
Contents
Operational Dashboard reports . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Auto Documentation reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Data Validation dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Data Integrator Metadata Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Data Integrator Web Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Tomcat servlet engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Data Integrator Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Data Integrator SNMP Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Data Integrator Adapter SDK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Optional Data Integrator components . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Data Integrator Multi-user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Data Integrator management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
License Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Repository Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Server Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Data Integrator operating system platforms . . . . . . . . . . . . . . . . . . . . . . . . 35
Data Integrator distributed architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Host names and port numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Chapter 4 Preparing to Install Data Integrator 39
About this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
System installation overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
System requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Data Integrator repository requirements . . . . . . . . . . . . . . . . . . . . . . . .44
Database requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
Data Integrator Designer requirements . . . . . . . . . . . . . . . . . . . . . . . . .47
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Data Integrator Job Server requirements . . . . . . . . . . . . . . . . . . . . . . . 47
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Data Integrator Access Server requirements . . . . . . . . . . . . . . . . . . . .48
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4 Data Integrator Getting Started Guide
Contents
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Data Integrator Management Console requirements . . . . . . . . . . . . . . 49
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Web applications communicating with Data Integrator . . . . . . . . . . . . 49
Data Integrator Profiler requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Pre-installation tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Determine component distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Obtain license keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Check port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Create a database for each repository . . . . . . . . . . . . . . . . . . . . . . . . . 54
Check network connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Create a Windows cluster (optional) . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Data Integrator licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
License types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Restricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Emergency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Permanent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Optional license-controlled features . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Managing your licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Obtaining your license files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Determining the Host ID for your platform . . . . . . . . . . . . . . . . 60
Viewing previously-generated license files . . . . . . . . . . . . . . . . . . 61
Re-hosting previously-generated license files . . . . . . . . . . . . . . . . 61
Repository database information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
For DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
For Microsoft SQL Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
For MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
For Oracle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
For Sybase ASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Data Integrator Getting Started Guide 5
Contents
Chapter 5 Upgrading Data Integrator 65
About this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66
Upgrade paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66
Data Integrator component upgrades . . . . . . . . . . . . . . . . . . . . . . . . . .66
Repository upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
Upgrading to a multi-user development environment . . . . . . . . . . . . . .67
Upgrading your Data Integrator system . . . . . . . . . . . . . . . . . . . . . . . . . . .68
Using configurations from a previous version . . . . . . . . . . . . . . . . . . . . 68
Uninstall existing product version(s) . . . . . . . . . . . . . . . . . . . . . . . . . . .69
Install a new version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70
Upgrading repositories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
After upgrading Data Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Unsuccessful upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
Successful upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
Chapter 6 Installing Data Integrator on Windows Systems 75
Installing on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76
Running the installation program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76
Installing the Designer and Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
JMS and SalesForce Interface Integration . . . . . . . . . . . . . . . . . . .79
Creating or upgrading repositories . . . . . . . . . . . . . . . . . . . . . . . . .81
Entering locale information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Configuring Job Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Configuring Job Server run-time resources . . . . . . . . . . . . . . . . . .86
Configuring Access Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Restarting Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Verifying that Job and Access Servers are running . . . . . . . . . . . . . . . 89
Starting Data Integrator services automatically . . . . . . . . . . . . . . . . . . .90
Logging in to the Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90
Connecting the Data Profiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Installing and Configuring the Metadata Integrator . . . . . . . . . . . . . . . . . . .91
Installing Metadata Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
6 Data Integrator Getting Started Guide
Contents
Installing on the same computer as Data Integrator . . . . . . . . . . . 91
Installing on a different computer than Data Integrator . . . . . . . . . 91
Installation tips and troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Configuration fails during installation . . . . . . . . . . . . . . . . . . . . . . 92
Changing Data Integrator repository connection information . . . . 93
Metadata Collector fails to start with java version error . . . . . . . . 93
Metadata Collector fails to start . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Metadata Integrator runs but report information is missing . . . . . . 93
Configuring the Metadata Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Configuring repositories after installation . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Creating or upgrading repositories in batch mode . . . . . . . . . . . . . . . . 97
Configuring Job Servers or Access Servers after installation . . . . . . . . . . 98
Recovering from installation errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Repository problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Administrator problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Verifying connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Distributing the test files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Testing a job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Testing the path from client to service . . . . . . . . . . . . . . . . . . . . . . . . 103
Configuring the Access Server . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Web client to Access Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Further connectivity tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Updating licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Installing Message Client libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Using the Message Client library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Interface components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Creating the connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Sending messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Closing the connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Pseudo code example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
C++ language interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Java language interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
COM interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Data Integrator Getting Started Guide 7
Contents
Connect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113
Invoke . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Disconnect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
ReturnValue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
ErrorMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117
ErrorStatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Chapter 7 Installing Data Integrator on UNIX Systems 119
About this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120
Additional system requirements for UNIX . . . . . . . . . . . . . . . . . . . . . . . . .120
Hardware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120
Software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120
Cron service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121
User IDs and permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121
High-availability support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .122
About HACMP software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .122
Using Data Integrator with HACMP software . . . . . . . . . . . . . . . . 123
Kernel parameters and user resource limits . . . . . . . . . . . . . . . . . . . . . . . 126
HP-UX for PA-RISC 32-bit kernel configuration . . . . . . . . . . . . . . . . .126
HP-UX for Itanium 64-bit kernel configuration . . . . . . . . . . . . . . . . . . . 127
AIX user resource limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Solaris user resource limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .128
Linux user resource limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .128
Installing Job Servers and Access Servers . . . . . . . . . . . . . . . . . . . . . . . .128
Locales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .134
Setting Data Integrator environment variables . . . . . . . . . . . . . . . . . .134
Configuring the Data Integrator Web Server . . . . . . . . . . . . . . . . . . . .135
Configuring Job Servers and Access Servers . . . . . . . . . . . . . . . . . . . 136
Starting AL_JobService at restart . . . . . . . . . . . . . . . . . . . . . . . . .145
Job Servers, Access Servers, SNMP . . . . . . . . . . . . . . . . . . . . . . . . .146
Updating licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146
Uninstalling Data Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147
8 Data Integrator Getting Started Guide
Contents
Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Designer-Job Server connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Server Manager problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Administrator connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Memory issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Threading issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Index 151
Data Integrator Getting Started Guide 9
Contents
10 Data Integrator Getting Started Guide
Data Integrator Getting Started Guide
Introduction
chapter
Introduction
1
About this guide
About this guide
This guide provides an overview of the BusinessObjects™ Data Integrator
Platform and architecture. It also describes how to upgrade and install Data
Integrator.
With Data Integrator you can:
• Easily build and execute batch processing applications that create and
update a data warehouse.
• Combine batch capabilities with request-response processing logic and
message handling to support e-commerce and its data integration
requirements.
This chapter covers the following topics:
• Audience and assumptions
• Data Integrator product documentation
Audience and assumptions
This and other Data Integrator product documentation assumes that:
• You are an application developer, consultant, or database administrator
working on data extraction, data warehousing, or data integration.
• You understand your source data systems, RDBMS, business
intelligence, and e-business messaging concepts.
• You understand your organization’s data needs.
• You are familiar with SQL (Structured Query Language).
• You are familiar enough with Microsoft Windows or UNIX platforms to
effectively install Data Integrator.
Data Integrator product documentation
Data Integrator documentation is provided in PDF format on the Data
Integrator CD. You can read PDF files using the latest version of Adobe
Acrobat Reader (download instructions at the Adobe website).
After you install Data Integrator, you can view technical documentation from
many locations. To view documentation in PDF format, you can:
12 Data Integrator Getting Started Guide
Introduction
Data Integrator product documentation
• If you accepted the default installation, select Start > Programs >
Business Objects > Data Integrator > Data Integrator Document ation
and select:
• Release Notes—Opens the Release Notes PDF. This document lists
supported and unsupported products, details migration
considerations, includes known and fixed bugs, and provides lastminute documentation corrections associated with the release.
• Release Summary—Opens the Release Summary PDF. this
document describes new Data Integrator features in the release.
• Technical Manuals—Opens the combined Technical Manuals PDF,
which contains Data Integrator core documentation and provides
searchable cross-book master Contents and master Index.
• Technical Manuals—Opens a “master” PDF document that has
been compiled so you can search across the Data Integrator
documentation suite
• Tutorial—Opens the Data Integrator Tutorial PDF, which you can
use for basic stand-alone training purposes
• Select one of the following from the Designer Help menu:
• Release Notes
• Release Summary
• Technical Manuals
• Technical Manuals
• Tutorial
Other links from the Designer’s Help menu include:
• DIZone—Opens a browser window to the DI Zone, an online resource for
the Data Integrator user community)
• Knowledge Base—Opens a browser window to Business Objects’
Technical Support Knowledge Exchange forum (access requires
registration)
1
Data Integrator Getting Started Guide 13
Introduction
1
Data Integrator product documentation
You can also access Data Integrator documentation from the Start Page that
opens automatically when you open the Designer. To open the Designer,
choose Start > Programs > Business Objects > Data Integrator > Data
Integrator Designer:
You can also view and download PDF documentation, including Data
Integrator documentation for previous releases (including Release
Summaries and Release Notes), by visiting the Business Objects
documentation Web site at http://support.businessobjects.com/
documentation/.
You can also open Help, using one of the following methods:
• Choose Contents from the Designer Help menu.
• Click objects in the object library or workspace and press F1.
Online Help opens to the subject you selected.
Use Online Help links and tool bar to navigate.
14 Data Integrator Getting Started Guide
Introduction
Data Integrator product documentation
The following technical documentation is currently available:
• Data Integrator Release Summary— Provides the most up-to-date
information about the product including information that was not available
when the other documents were published.
• Data Integrator Release Summary— Provides summarized information
highlighting all the newest features in the release.
• Data Integrator Technical Manuals includes the following books with a
master Table of Contents and master Index for powerful cross-book
search capabilities:
• Data Integrator Getting S ta rted Guide — Introduces the features and
benefits of using Data Integrator. This guide explains Data Integrator
architecture and how to install the product including system
requirements and pre-installation preparation that will help you
understand dependencies between Data Integrator and your
computing environment.
• Data Integrator Designer Guide — Provides conceptual and
procedural information for designing, developing, and implementing
applications using the Data Integrator Designer.
• Data Integrator Management Console: Administrator Guide —
Provides conceptual and procedural information for scheduling and
monitoring batch jobs as well as configuring and administering realtime jobs. Explains the Data Integrator Administrator features.
• Data Integrator Reference Guide — Provides detailed information
about Data Integrator Designer objects, data types and expressions,
transforms, functions, and user exits.
• Data Integrator Advanced Development and Migration Guide —
Discusses how to migrate Data Integrator applications through
development phases and includes multi-user development concepts
and procedures.
• Data Integrator Performance Optimization Guide — Provides
information about improving the performance of your Data Integrator
environment and the Designer options you can use to reduce job
run-times by measuring and tuning performance. Includes
documentation on bulk loading and parallel execution.
• Data Integrator Supplement for J.D. Edwards — Provides
information about using Data Integrator to extract data from J.D.
Edwards World and J.D. Edwards OneWorld.
• Data Integrator Supplement for Oracle Applications — Provides
information about using Data Integrator to extract data from Oracle
Applications.
1
Data Integrator Getting Started Guide 15
Introduction
1
Suggested reading path
• Data Integrator Supplement for PeopleSoft — Provides information
about using Data Integrator to extract data from PeopleSoft ERP and
HRMS software.
• Data Integrator Supplement for SAP — Provides information about
using Data Integrator to extract data from SAP R/3 and SAP BW.
This supplement includes conceptual, procedural, and reference
information.
• Data Integrator Supplement for Siebel — Provides information about
using Data Integrator to extract data from Siebel applications.
If you install the documentation with Data Integrator, you can also go to Start
> Programs > Business Objects > Data Integrator > Data Integrator
Documentation > Tutorial to view:
• Data Integrator Core Tutorial — Provides exercises to help you learn how
to extract, transform, and load data with the Data Integrator Designer.
Exercises include using design logic to extract data from SAP R/3 and
non-SAP sources.
Suggested reading path
Begin by reading the Data Integrator Getting St arted Guide which includes an
introduction to functionality and architecture as well as installation
instructions.
Understand how to use Data Integrator Designer by reading the Data
Integrator Designer Guide and doing the exercises in the Data Integrator
Core Tutorial. Use the Data Integrator Reference Guide to answer specific
questions about objects and window options in the Designer.
The Data Integrator Management Console: Administrator Guide provides
information about how to use the Administrator which is a browser-based
graphical user interface used to schedule and monitor batch jobs and to
configure real-time jobs and adapter interfaces available with Data Integrator.
Use the Data Integrator Performance Optimization Guide for tips about using
system and Designer options related to performance optimization. These
include measuring and tuning techniques, bulk loading, and parallel
processing objects within a data flow.
To install and configure Data Integrator’s pre-packaged adapters, read the
individual adapter guides on the adapter product CDs.
16 Data Integrator Getting Started Guide
Data Integrator Getting Started Guide
Introducing Data Integrator
chapter
Introducing Data Integrator
2
About this chapter
About this chapter
This chapter introduces Data Integrator and explains its place in the Business
Objects product suite. Topics include:
• Data Integrator and the Business Objects Product Suite
• Data Integrator product benefits
• Data Integrator interfaces
Data Integrator and the Business Objects Product Suite
The Business Objects product suite delivers extreme insight through
specialized end-user tools on a single, trusted business intelligence platform.
This entire platform is supported by BusinessObjects™ Data Integrator. On
top of Data Integrator, Business Objects layers the most reliable, scalable,
flexible, and manageable BI platform which supports the industry’s best
integrated end-user interfaces: reporting, query and analysis, and
performance management dashboards, scorecards, and applications.
True data integration blends batch extraction, transformation, and loading
(ETL) technology with real-time bi-directional data flow across multiple
applications for the extended enterprise.
By building a relational datastore and intelligently blending direct real-time
and batch data-access methods to access data from enterprise resource
planning (ERP) systems and other sources, Business Objects has created a
powerful, high-performance data integration product that allows you to fully
leverage your ERP and enterprise application infrastructure for multiple uses.
Business Objects provides a batch and real-time data integration system to
drive today’s new generation of analytic and supply-chain management
applications. Using the highly scalable data integration solution provided by
Business Objects, your enterprise can maintain a real-time, on-line dialogue
with customers, suppliers, employees, and partners, providing them with the
critical information they need for transactions and business analysis.
18 Data Integrator Getting Started Guide
Data Integrator product benefits
Use Data Integrator to develop enterprise data integration for batch and realtime uses. With Data Integrator:
• You can share data and metadata with BusinessObjects Enterprise
platform.
• You can create a single infrastructure for batch and real-time data
movement to enable faster and lower cost implementation.
• Your enterprise can manage data as a corporate asset independent of
any single system. Integrate data across many systems and reuse that
data for many purposes.
• You have the option of using pre-packaged data solutions for fast
deployment and quick ROI. These solutions extract historical and daily
data from operational systems and cache this data in open relational
databases.
Data Integrator customizes and manages data access and uniquely combines
industry-leading, patent-pending technologies for delivering data to analytic,
supply-chain management, customer relationship management, and Web
applications.
Introducing Data Integrator
Data Integrator product benefits
2
Unification with the platform
Data Integrator provides several points of unification with BusinessObjects
Enterprise platform:
• Get end-to-end data lineage and impact analysis
• Create the semantic layer (universe) and manage change within the ETL
design environment
Business Objects deeply integrates the entire ETL process with the business
intelligence platform so you benefit from:
• Easy metadata management
• Simplified and unified administration
• Lifecycle management
• Trusted information
Data Integrator Getting Started Guide 19
Introducing Data Integrator
2
Data Integrator associated products
Ease of use and high productivity
Data Integrator combines both batch and real-time data movement and
management to provide a single data integration platform for information
management from any information source, for any information use.
Using Data Integrator, you can:
• Stage data in an operational datastore, data warehouse, or data mart.
• Update staged data in batch or real-time modes.
• Create a single graphical development environment for developing,
testing, and deploying the entire data integration platform.
• Manage a single metadata repository to capture the relationships
between different extraction and access methods and provide integrated
lineage and impact analysis.
High availability and performance
Data Integrator’s high-performance engine and proven data movement and
management capabilities include:
• Scalable, multi-instance data-movement for fast execution
• Load balancing
• Changed-data capture
• Parallel processing
Data Integrator associated products
Choose from several Business Objects product options to further support and
enhance the power of your Data Integrator product.
Composer
Even before you start a data warehouse implementation, Business Objects
knows that you need to collect and organize critical data from information
users to “compose” a blueprint from which you can start to build your Data
Integrator jobs.
The BusinessObjects ™Composer is a stand-alone, web-based application
for designing extraction, transformation, and loading (ETL) projects.
20 Data Integrator Getting Started Guide
Developing higher-level designs for your ETL jobs can make the
implementation process more efficient and improve the quality of resulting
jobs. Composer complements other ETL tools, providing a platform for
creating your design even before you start to build your data warehouse. Use
Composer to identify sources of data, implement transformations, and
document your designs.
For more information on BusinessObjects™ Composer, contact your
Business Objects sales representative.
Metadata Manager
Business Objects™ Metadata Manager provides an integrated view of
metadata and its multiple relationships for an complete Business Intelligence
project spanning some or all of the Business Objects stack of products. Use
Metadata Manager to:
• View metadata about Business Objects Reports, Documents, and data
sources from a single repository.
• Analyze lineage to determine data sources of Business Objects
Documents and Reports.
• Analyze the impact of changing a source table, column, element, or field
on existing Business Objects Documents and Reports.
• Track different versions (changes) to each object over time.
• View operational metadata (such as the number of rows processed and
CPU utilization) as historical data with a datetime.
• View metadata in different languages.
For more information on Business Objects™ Metadata Manager , contact your
Business Objects sales representative.
Introducing Data Integrator
Data Integrator interfaces
2
Data Integrator interfaces
Data Integrator provides many types of interface components. Interfaces
include:
• Data-level read/write interfaces using SQL:
Attunity
BusinessObjects Data Federator
DB2/UDB
DB2/UDB via DB2Connect to MVS
Data Integrator Getting Started Guide 21
Introducing Data Integrator
2
Data Integrator interfaces
DB2/UDB via DB2Connect to AS/400
Informix
MS SQL Server
MySQL
Netezza
Oracle
ODBC to generic databases
Sybase ASE
Sybase IQ
Teradata
• Application-level interfaces (read/write and batch/real-time unless
otherwise specified):
J.D. Edwards One World or World (batch)
Oracle Application (batch)
PeopleSoft interface (batch)
Salesforce.com
SAP R/3 ABAP (batch, read only)
SAP R/3 Hierarchy
SAP R/3 IDoc
SAP R/3 RFC/BAPI
SAP BW (batch)
Siebel Application (batch)
• Technology read/write real-time interfaces:
HTTP/HTTPS
Java Message Service (JMS)
SNMP
Web Services
• Files read/write connectivity interfaces:
ASCII
COBOL (read only)
Excel (read only)
XML
22 Data Integrator Getting Started Guide
Introducing Data Integrator
Data Integrator interfaces
You can use the Data Integrator Interface Development Kit to develop
adapters that read from and/or write to other applications.
In addition to the interfaces listed above, the Data Integrator Nested
Relational Data Model (NRDM) allows you to apply the full power of SQL
transforms to manipulate, process, and enrich hierarchical business
documents. For more information, see Chapter, “Nested Data,” in the Data
Integrator Designer Guidethe.
To learn more about which products and components are supported by your
version of Data Integrator, see your Data Integrator Release Notes and
Supported Platform documents.
2
Data Integrator Getting Started Guide 23
Introducing Data Integrator
2
Data Integrator interfaces
24 Data Integrator Getting Started Guide
Data Integrator Getting Started Guide
Data Integrator Architecture
chapter
Data Integrator Architecture
3
About this chapter
About this chapter
This chapter describes Data Integrator components and their distribution on
your network.
This chapter contains the following topics:
• Standard Data Integrator components
• Optional Data Integrator components
• Data Integrator management tools
• Data Integrator operating system platforms
• Data Integrator distributed architecture
The Data Integrator architecture is layered to allow data integration to occur
over a variety of open, industry-standard APIs for optimal data and metadata
management.
Standard Data Integrator components
Standard Data Integrator components include:
• Data Integrator Designer
• Data Integrator repository
• Data Integrator Job Server
• Data Integrator engine
• Data Integrator Access Server
• Data Integrator Administrator
• Data Integrator Metadata Reports applications
• Data Integrator Web Server
• Data Integrator Service
• Data Integrator SNMP Agent
• Data Integrator Adapter SDK
The following diagram summarizes the relationships among Data Integrator
components.
26 Data Integrator Getting Started Guide
Data Integrator Architecture
Standard Data Integrator components
3
Data Integrator Designer
The Designer is a development tool with an easy-to-use graphical user
interface. It enables developers to define data management applications that
consist of data mappings, transformations, and control logic.
Use the Designer to create applications containing work flows (job execution
definitions) and data flows (data transformation definitions).
To use the Designer, create objects, then drag, drop, and configure them by
selecting icons in flow diagrams, table layouts, and nested worksp ace pages.
The objects in the Designer represent metadata. The Designer interface
allows you to manage metadata stored in a Data Integrator repository. From
the Designer, you can also trigger the Data Integrator Job Server to run your
jobs for initial application testing.
Data Integrator Getting Started Guide 27
Data Integrator Architecture
3
Standard Data Integrator components
Data Integrator repository
The Data Integrator repository is a set of tables that hold user-created and
predefined system objects, source and target metadata, and transformation
rules. Set up repositories on an open client/server platform to facilitate
sharing metadata with other enterprise tools. Store each repository on an
existing RDBMS.
Each repository is associated with one or more Data Integrator Job Servers
which run the jobs you create. There are two types of repositories:
• A local repository is used by an application designer to store definitions of
Data Integrator objects (like projects, jobs, work flows, and data flows)
and source/target metadata.
• A central repository is an optional component that can be used to support
multi-user development. The central repository provides a shared object
library allowing developers to check objects in and out of their local
repositories.
Data Integrator Job Server
The Data Integrator Job Server starts the data movement engine that
integrates data from multiple heterogeneous sources, performs complex data
transformations, and manages extractions and transactions from ERP
systems and other sources. The Data Integrator Job Server can move data in
either batch or real-time mode and uses distributed query optimization, multithreading, in-memory caching, in-memory data transformations, and parallel
processing to deliver high data throughput and scalability.
While designing a job, you can run it from the Designer which tells the Job
Server to run the job. The Job Server gets the job from its associated repository,
then starts a Data Integrator engine to process the job. In your production
environment, the Job Server runs jobs triggered by a scheduler or by a realtime service managed by the Data Integrator Access Server. In production
environments, you can balance job loads by creating a Job Server Group
(multiple Job Servers) which executes jobs according to overall system load.
Data Integrator engine
When Data Integrator jobs are executed, the Job Server starts Data Integrator
engine processes to perform data extraction, transformation, and movement.
Data Integrator engine processes use parallel processing and in-memory
data transformations to deliver high data throughput and scalability.
28 Data Integrator Getting Started Guide
Data Integrator Access Server
The Access Server is a real-time, request-reply message broker that collects
message requests, routes them to a real-time service, and delivers a
message reply within a user-specified time frame. The Access Server queues
messages and sends them to the next available real-time service across any
number of computing resources. This approach provides automatic scalability
because the Access Server can initiate additional real-time services on
additional computing resources if traffic for a given real-time service is high.
You can configure multiple Access Servers.
Data Integrator Administrator
The Administrator provides browser-based administration of Data Integrator
resources including:
• Scheduling, monitoring, and executing batch jobs
• Configuring, starting, and stopping real-time services
• Configuring Job Server, Access Server, and repository usage
• Configuring and managing adapters
• Managing users
• Publishing batch jobs and real-time services via Web services
Data Integrator Architecture
Standard Data Integrator components
3
Data Integrator Metadata Reports applications
The Metadata Reports applications provides browser-based analysis and
reporting capabilities on metadata that is associated with:
• your Data Integrator jobs
• other Business Objects applications associated with Data Integrator
Metadata Reports provide four applications for exploring your metadata:
• Impact and lineage analysis
• Operational dashboards
• Auto documentation
• Data validation
Data Integrator Getting Started Guide 29
Data Integrator Architecture
3
Standard Data Integrator components
Impact and Lineage Analysis reports
Impact and Lineage Analysis reports include:
• Datastore Analysis — For each datastore connection, view overview,
table, function, and hierarchy reports. Data Integrator users can
determine:
• What data sources populate their tables
• What target tables their tables populate
• Whether one or more of the following Business Objects reports uses
data from their tables:
• Business Views
• Crystal Reports
• Universes
• Web Intelligence documents
• Desktop Intelligence documents
• Universe analysis — View Universe, class, and object lineage. Universe
users can determine what data sources populate their Universes and
what reports use their Universes.
• Business View analysis — View the dat a sources for Business Views in
the Central Management Server (CMS). You can view business element
and business field lineage reports for each Business View. Crystal
Business View users can determine what data sources populate their
Business Views and what reports use their views.
• Report analysis — View data sources for reports in the Central
Management Server (CMS). You can view table and column lineage
reports for each Crystal Report and Web Intelligence Document
managed by CMS. Report writers can determine what data sources
populate their reports.nic
• Dependency analysis — Search for specific objects in your repository
and understand how those objects impact or are impacted by other Data
Integrator or Business Objects Universe objects and reports. Metadata
search results provide links back into associated reports.
To view impact and lineage analysis for Business Objects applications, you
must configure the Metadata Integrator. To learn more about Metadata
Integrator, see “Installing and Configuring the Metadata Integrator” on
page 91.
30 Data Integrator Getting Started Guide
Loading...