Business Objects products in this release may contain redistributions of software
licensed from third-party contributors. Some of these individual components may
also be available under alternative licenses. A partial listing of third-party
contributors that have requested or permitted acknowledgments, as well as required
notices, can be found at: http://www.businessobjects.com/thirdparty
2008-11-28
Contents
Introduction7Chapter 1
Welcome to Data Services..........................................................................8
Overview of this guide...............................................................................14
Migration Basics17Chapter 2
Development phases.................................................................................18
Migration mechanisms and tools...............................................................20
Central repository migration.....................................................................101
Index103
6Data Services Advanced Development Guide
Introduction
1
Introduction
1
Welcome to Data Services
Welcome to Data Services
Welcome
Data Services XI Release 3 provides data integration and data quality
processes in one runtime environment, delivering enterprise performance
and scalability.
The data integration processes of Data Services allow organizations to easily
explore, extract, transform, and deliver any type of data anywhere across
the enterprise.
The data quality processes of Data Services allow organizations to easily
standardize, cleanse, and consolidate data anywhere, ensuring that end-users
are always working with information that's readily available, accurate, and
trusted.
Documentation set for Data Services
You should become familiar with all the pieces of documentation that relate
to your Data Services product.
What this document providesDocument
Documentation Map
Release Summary
Release Notes
Getting Started Guide
Installation Guide for Windows
8Data Services Advanced Development Guide
Information about available Data Services books,
languages, and locations
Highlights of key features in this Data Services release
Important information you need before installing and
deploying this version of Data Services
An introduction to Data Services
Information about and procedures for installing Data
Services in a Windows environment.
Introduction
Welcome to Data Services
What this document providesDocument
1
Installation Guide for UNIX
Advanced Development Guide
Designer Guide
Integrator's Guide
Management Console: Administrator
Guide
Management Console: Metadata Reports Guide
Migration Considerations Guide
Information about and procedures for installing Data
Services in a UNIX environment.
Guidelines and options for migrating applications including information on multi-user functionality and
the use of the central repository for version control
Information about how to use Data Services Designer
Information for third-party developers to access Data
Services functionality. Also provides information about
how to install, configure, and use the Data Services
Adapter for JMS.
Information about how to use Data Services Administrator
Information about how to use Data Services Metadata
Reports
Information about:
•Release-specific product behavior changes from
earlier versions of Data Services to the latest release
•How to migrate from Data Quality to Data Services
Performance Optimization Guide
Reference Guide
Information about how to improve the performance
of Data Services
Detailed reference material for Data Services Designer
Data Services Advanced Development Guide9
Introduction
1
Welcome to Data Services
Technical Manuals
What this document providesDocument
A compiled “master” PDF of core Data Services books
containing a searchable master table of contents and
index:
•
Getting Started Guide
•
Installation Guide for Windows
•
Installation Guide for UNIX
•
Designer Guide
•
Reference Guide
•
Management Console: Metadata Reports Guide
•
Management Console: Administrator Guide
•
Performance Optimization Guide
•
Advanced Development Guide
•
Supplement for J.D. Edwards
•
Supplement for Oracle Applications
•
Supplement for PeopleSoft
•
Supplement for Siebel
•
Supplement for SAP
Tutorial
A step-by-step introduction to using Data Services
In addition, you may need to refer to several Adapter Guides and
Supplemental Guides.
What this document providesDocument
Salesforce.com Adapter
Interface
Supplement for J.D. Edwards
Supplement for Oracle Applications
Supplement for PeopleSoft
10Data Services Advanced Development Guide
Information about how to install, configure, and use the Data
Services Salesforce.com Adapter Interface
Information about license-controlled interfaces between Data
Services and J.D. Edwards World and J.D. Edwards OneWorld
Information about the license-controlled interface between Data
Services and Oracle Applications
Information about license-controlled interfaces between Data
Services and PeopleSoft
Introduction
Welcome to Data Services
What this document providesDocument
1
Supplement for SAP
Supplement for Siebel
Information about license-controlled interfaces between Data
Services, SAP ERP, and SAP BI/BW
Information about the license-controlled interface between Data
Services and Siebel
Accessing documentation
You can access the complete documentation set for Data Services in several
places.
Accessing documentation on Windows
After you install Data Services, you can access the documentation from the
Start menu.
1. Choose Start > Programs > BusinessObjects XI 3.1 >
BusinessObjects Data Services > Data Services Documentation.
Note:
Only a subset of the documentation is available from the Start menu. The
documentation set for this release is available in LINK_DIR\Doc\Books\en.
2. Click the appropriate shortcut for the document that you want to view.
Accessing documentation on UNIX
After you install Data Services, you can access the online documentation by
going to the directory where the printable PDF files were installed.
1. Go to LINK_DIR/doc/book/en/.
2. Using Adobe Reader, open the PDF file of the document that you want
to view.
Data Services Advanced Development Guide11
Introduction
1
Welcome to Data Services
Accessing documentation from the Web
You can access the complete documentation set for Data Services from the
Business Objects Customer Support site.
1.
Go to http://help.sap.com.
2. Cick Business Objects at the top of the page.
You can view the PDFs online or save them to your computer.
Business Objects information resources
A global network of Business Objects technology experts provides customer
support, education, and consulting to ensure maximum business intelligence
benefit to your business.
Useful addresses at a glance:
ContentAddress
12Data Services Advanced Development Guide
Introduction
Welcome to Data Services
ContentAddress
1
Customer Support, Consulting, and Education
services
Information about Customer Support programs,
as well as links to technical articles, downloads,
and online forums. Consulting services can
provide you with information about how Business Objects can help maximize your business
intelligence investment. Education services can
provide information about training options and
modules. From traditional classroom learning
to targeted e-learning seminars, Business Objects can offer a training package to suit your
learning needs and preferred learning style.
Get online and timely information about Data
Services, including tips and tricks, additional
downloads, samples, and much more. All content is to and from the community, so feel free
to join in and contact us if you have a submission.
Search the Business Objects forums on the
SAP Community Network to learn from other
Data Services users and start posting questions
or share your knowledge with the community.
Blueprints for you to download and modify to fit
your needs. Each blueprint contains the necessary Data Services project, jobs, data flows, file
formats, sample data, template tables, and
custom functions to run the data flows in your
environment with only a few modifications.
Data Services Advanced Development Guide13
Introduction
1
Overview of this guide
http://help.sap.com/
ContentAddress
Business Objects product documentation.Product documentation
Documentation mailbox
documentation@businessobjects.com
Supported platforms documentation
https://service.sap.com/bosap-support
Send us feedback or questions about your
Business Objects documentation. Do you have
a suggestion on how we can improve our documentation? Is there something that you particularly like or have found useful? Let us know,
and we will do our best to ensure that your
suggestion is considered for the next release
of our documentation.
Note:
If your issue concerns a Business Objects
product and not the documentation, please
contact our Customer Support experts.
Get information about supported platforms for
Data Services.
In the left panel of the window, navigate to
Documentation > Supported Platforms >
BusinessObjects XI 3.1. Click the BusinessObjects Data Services link in the main window.
Overview of this guide
About this guide
The guide contains advanced development information. Topics include:
•Migration—How to move your projects to different development
environments
•Multi-user development—How to manage a project developed by multiple
users
14Data Services Advanced Development Guide
You will find this guide most useful:
•After you have learned product basics
•While planning the design, test, and production phases of your
data-movement projects
•As an advanced source of information during any phase of your projects
Who should read this guide
This and other Data Services product documentation assumes the following:
•You are an application developer, consultant, or database administrator
working on data extraction, data warehousing, or data integration.
•You understand your source data systems, RDBMS, business intelligence,
and messaging concepts.
•You understand your organization's data needs.
•You are familiar with SQL (Structured Query Language).
Introduction
Overview of this guide
1
•If you are interested in using this product to design real-time processing,
you are familiar with:
•DTD and XML Schema formats for XML files
•Publishing Web Services (WSDL, HTTP, and SOAP protocols, etc.)
•You are familiar Data Services installation environments—Microsoft
Windows or UNIX.
Further, Business Objects recommends that you review both the DataServices Getting Started Guide and the Data Services Designer Guide before
using advanced concepts in this document.
About environment migration
One of the most powerful aspects of Data Services is its architectural flexibility
through development, test, and production environments. Data Services is
designed to support various configurations including large enterprises with
many developers working on multiple projects. Data Services supports
multi-site architectures whether centralized or not.
Data Services Advanced Development Guide15
Introduction
1
Overview of this guide
This guide discusses architectural options for implementing Data Services
in development, test, and production environments.
Related Topics
•Migration Basics on page 17
•Preparing for Migration on page 25
•Export/Import on page 37
•Multi-user Development on page 51
•Multi-user Environment Setup on page 57
•Working in a Multi-user Environment on page 75
•Migrating Multi-user Jobs on page 97
16Data Services Advanced Development Guide
Migration Basics
2
Migration Basics
2
Development phases
About this section
Migration as it relates to Data Services is the process of moving applications
through multiple development phases into production. Data Services supports
simple and complex application migration through all phases into production.
Related Topics
•Development phases on page 18
•Migration mechanisms and tools on page 20
Development phases
The ETL application development process typically involves three distinct
phases:
•Design phase
•Test phase
•Production phase
You can use Data Services in all three phases. Because each phase might
require a different repository to control environment differences, Data Services
provides controlled mechanisms for moving objects from phase to phase.
Each phase could involve a different computer in a different environment
with different security settings. For example, design and initial test may only
require limited sample data and low security, while final testing may require
a full emulation of the production environment including strict security.
18Data Services Advanced Development Guide
Design phase
In this phase, you define objects and build diagrams that instruct Data
Services in your data movement requirements. Data Services stores these
specifications so you can reuse them or modify them as your system evolves.
Design your project with migration to testing and final production in mind.
Consider these basic guidelines as you design your project:
•Construct design steps as independent, testable modules.
•Use meaningful names for each step you construct.
•Make independent modules that can be used repeatedly to handle
common operations.
•Use test data that reflects all the variations in your production data.
Test phase
Migration Basics
Development phases
2
In this phase, you use Data Services to test the execution of your application.
At this point, you can test for errors and trace the flow of execution without
exposing production data to any risk. If you discover errors during this phase,
return the application to the design phase for correction, then test the
corrected application.
Testing has two parts:
•The first part includes designing the data movement using your local
repository.
•The second part includes fully emulating your production environment,
including data volume.
Data Services provides feedback through trace, error, and monitor logs during
both parts of this phase.
The testing repository should emulate your production environment as closely
as possible, including scheduling jobs rather than manually starting them.
Data Services Advanced Development Guide19
Migration Basics
2
Migration mechanisms and tools
Production phase
In this phase, you set up a schedule in Data Services to run your application
as a job. Evaluate results from production runs and when necessary, return
to the design phase to optimize performance and refine your target
requirements.
After moving a Data Services application into production, monitor it in the
Administrator for performance and results. During production:
•Monitor your jobs and the time it takes for them to complete.
The trace and monitoring logs provide information about each job as well
as the work flows and data flows contained within the job.
You can customize the log details. However, the more information you
request in the logs, the longer the job runs. Balance job run-time against
the information necessary to analyze job performance.
•Check the accuracy of your data.
To enhance or correct your jobs:
•Make changes in your design environment.
•Repeat the object testing.
•Move changed objects back into production.
Migration mechanisms and tools
Data Services provides two migration mechanisms:
•Export/import migration works best with small to medium-sized projects
where a small number of developers work on somewhat independent
Data Services applications through all phases of development.
•Multi-user development works best in larger projects where two or more
developers or multiple teams are working on interdependent parts of Data
Services applications through all phases of development.
Regardless of which migration mechanism you choose, Business Objects
recommends you prepare for migration using one or more tools that best fit
20Data Services Advanced Development Guide
your development environment for more information). The mechanism and
tools you use will depend on the needs of your development environment.
If your source data will come from multiple, homogeneous systems, Business
Objects recommends you use Datastore and system configurations tools.
When migrating applications in a multi-user environment, Business Objects
strongly recommends you use Naming conventions for migration.
Related Topics
•Export/import migration on page 23
•Multi-user Development on page 51
•Preparing for Migration on page 25
•Datastore and system configurations on page 30
•Designer Guide: Datastores, Creating and managing multiple datastore
configurations
•Naming conventions for migration on page 26
Which mechanism is best?
Migration Basics
Migration mechanisms and tools
2
Although Data Services supports a multi-user environment, you may not
need to implement this architecture on all projects. If your project is small to
medium in size and only consists of one or two developers, then a Central
Repository may not be a necessary solution to integrating the work of those
developers.
For example, only two consultants worked on a certain HR data mart
application. The Development system was designed so that while Consultant
1 managed the Master Repository, Consultant 2 worked on a new section
within a complete copy of the Master Repository.
Consultant 2 then exported this new section back into the Master Repository
using the export utility that allows objects to be 'Created', 'Replaced', or
'Ignored'. After updating the Master Repository, Consultant 2 took a new
complete copy of the Master Repository, overwriting the previous copy.
Use the following matrix to help you determine which mechanism and tools
would work best in your environment.
Data Services Advanced Development Guide21
Migration Basics
2
Migration mechanisms and tools
Situation/re
quirements
Small to
medium-sized
project
Export/im
port
Multi-user
ToolsMigration Mechanisms
Naming conventions
Configura
tions
OOX
Multiple-team
project
Source data
from multiple,
homogeneous systems
Different
source or target database
among environments
Need a "fast
and easy" migration solution
Optimal solution: X Compatible solution: O
OXX
XX
XX
XO
22Data Services Advanced Development Guide
Export/import migration
Export/import is the basic mechanism for migrating Data Services applications
between phases. First, you export jobs from the local repository to another
local repository or to an intermediate file which you can then import into
another local repository. For example, when moving from design repository
to test repository, you export from the design repository to a file, then import
the file to your test repository.
Migration Basics
Migration mechanisms and tools
2
If you find application errors during testing, you can correct them in the
development environment, then export the corrected version and import it
back into the test repository for retesting.
Related Topics
•Export/Import on page 37
Multi-user migration
You can also migrate Data Services applications between phases in more
complex development environments. Instead of exporting and importing
applications, multi-user development provides a more secure check-in,
check-out, and get mechanism, using a central repository to store the master
copies of your application elements. Multi-user development includes other
advanced features like labeling and filtering to provide you more flexibility
and control in managing application objects.
Data Services Advanced Development Guide23
Migration Basics
2
Migration mechanisms and tools
Related Topics
•Migrating Multi-user Jobs on page 97
24Data Services Advanced Development Guide
Preparing for Migration
3
Preparing for Migration
3
Naming conventions for migration
About this section
Before you develop Data Services applications, Business Objects
recommends that you first set up a comprehensive structure to facilitate the
migration process between development phases.
This section discusses tools that can help you build your migration structure.
Business Objects recommends that you implement standardized naming
conventions for connectivity between computer systems. Add datastore and
system configurations to more easily work with multiple homogeneous
systems.
Related Topics
•Naming conventions for migration on page 26
•Datastore and system configurations on page 30
Naming conventions for migration
The best way to ensure fast and seamless migration is to use common
naming conventions across all systems and phases of all your development
environments.
Just as Business Objects recommends you standardize object prefixes,
suffixes, and path name identifiers to simplify your projects internally, we
also recommend the use of naming conventions externally for migration
purposes.
To ease migration, use common naming conventions for:
•Connections to external datastores
•Directory locations
•Schema structures and owners
You want to make it as quick and easy as possible to migrate applications
between users and between phases. This translates to significantly reducing
or eliminating time spent reconfiguring your jobs to work in each specific
environment.
While the actual data you are extracting, transforming, and loading usually
differs by database, the essential structure of the data should be the same
26Data Services Advanced Development Guide
on every database with which you want the same applications to work.
Therefore, it makes the most sense to standardize your database naming
and structuring before starting the development process.
Related Topics
•Designer Guide: Projects and Jobs, Naming conventions for objects in
jobs
•Connections to external datastores on page 27
•Directory locations on page 29
•Schema structures and owners on page 29
Connections to external datastores
In Data Services, migration is the process of moving objects between local
repositories, whether directly using the Export/Import method or indirectly
using the Multi-user development method. Regardless of method, you must
consider how the migration will impact connection configurations associated
with your jobs.
Preparing for Migration
Naming conventions for migration
3
Using generic naming for similar external datastore connections reduces the
time you spend on reconfiguring the connections to the same database type.
For example, you should choose the same logical name for all your Oracle
datastore connections to the same type of database structure regardless of
migration phase environment.
You can make connection names meaningful to a certain phase and specific
computer system names (Test_DW, Dev_DW, Prod_DW), however if you
choose this naming structure, Business Objects recommends that you use
datastore configurations for migration purposes.
Test phaseDevelopment phase
User name: Test_DWUser name: Dev_DW
Password: Test_DWPassword: Dev_DW
Host String: Test_DWHost String: Dev_DW
Data Services Advanced Development Guide27
Preparing for Migration
3
Naming conventions for migration
For a job to run against Test and Development, it would have to use Test_DW
and Dev_DW and this would require you to create different datastore
configurations for when the job runs against the Test or the Dev instance,
respectively.
Alternatively, you could call the connection string DW and regardless of what
instance you ran the job against, it would run without users having to create
multiple datastore configurations.
Test PhaseDevelopment Phase
Database A
Examples:
•There is one Oracle source system in your company that processes order
entry data. Multiple instances of this system exist for development, test,
and production purposes. Therefore, you name the connection string to
your Oracle source system "ORDER_SYSTEM". Then in all phases, you
configure that name to point to the correct (phase-specific) instance of
the system.
•Name the connection string to your target data warehouse "DW" then
point it to different databases depending on whether you are in the
development, test, or production environment.
Datastore Connection
Database B
Datastore Connection
User name: DWUser name: DWUser name: DWUser name: DW
When you use this generic, cross-phase naming method, you cannot access
both dev and test from the same computer (since the connection string maps
only to one instance). If you require access to both, use multiple datastore
configurations.
28Data Services Advanced Development Guide
Related Topics
•Export/Import on page 37
•Multi-user Development on page 51
Directory locations
Business Objects recommends you use logical directory names (for example,
X:\) or point to common local drives to standardize directory location. For
example, since every computer has a C:\ drive, pointing to the directory
location, C:\TEMP would be a safe, reproducible standard.
Schema structures and owners
To further facilitate a seamless structure between development phases, give
all your database instances the same owner name for the same schema
structures from which you are reading and to which you are loading.
Regardless of name, the owner of each schema structure can vary and Data
Services will reconcile them.
Preparing for Migration
Naming conventions for migration
3
Data Services Advanced Development Guide29
Preparing for Migration
3
Datastore and system configurations
Datastore and system configurations
Datastore and system configurations are powerful tools for reducing the
configurations required to execute the same logic against different datastore
environments. With configurations, migration between development phases
becomes faster and more simplified.
Related Topics
•Datastore configurations and migration on page 31
•Multiple configurations in multi-user environments on page 33
30Data Services Advanced Development Guide
Loading...
+ 76 hidden pages
You need points to download manuals.
1 point = 1 manual.
You can buy points or you can get point for every manual you upload.