IBM Problem Determination User Manual
T
ivoli
®
Version 5.1
Intelligent Orchestrator
Problem
Determination and Troubleshooting Guide
SC32-2216-00
T
ivoli
®
Version 5.1
Intelligent Orchestrator
Problem
Determination and Troubleshooting Guide
SC32-2216-00
Note:
Before using this information and the product it supports, read the information in Appendix C, “Notices,” on page 229.
First Edition, July 2006
© Copyright International Business Machines Corporation 2003, 2006. All rights reserved.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
Contents
Preface . . . . . . . . . . . . . . vii
Who should read this book . . . . . . . . . vii
Publications . . . . . . . . . . . . . . vii
Tivoli Intelligent Orchestrator library . . . . . vii
Prerequisite publications . . . . . . . . . viii
Related publications and resources . . . . . viii
Accessing publications online . . . . . . .ix
Ordering publications . . . . . . . . . . .ix
Accessibility . . . . . . . . . . . . . .ix
Tivoli software training . . . . . . . . . . .x
Support information . . . . . . . . . . . .x
Conventions used in this book . . . . . . . .x
Typeface conventions . . . . . . . . . .x
Chapter 1. Introduction to
troubleshooting . . . . . . . . . . .1
Product and system overview . . . . . . . .1
Product comparison . . . . . . . . . . .2
Product architecture . . . . . . . . . . . .3
Tivoli Intelligent Orchestrator components . . .3
Prerequisites installed with Tivoli Intelligent
Orchestrator . . . . . . . . . . . . .4
Built-in troubleshooting features . . . . . . . .5
Problem classification . . . . . . . . . . .5
Product maintenance . . . . . . . . . . .5
Backing up the system . . . . . . . . . .5
Performing periodic checks and maintenance . .6
Problem resolution . . . . . . . . . . . .6
Chapter 2. Problem determination
essentials for Tivoli Intelligent
Orchestrator . . . . . . . . . . . . .7
Problem determination methodologies . . . . . .7
Defining the problem . . . . . . . . . .7
Identifying the cause of the problem . . . . .8
Fixing the problem . . . . . . . . . . .9
Troubleshooting methods for Tivoli Intelligent
Orchestrator and prerequisites . . . . . . . .9
Troubleshooting the prerequisite installation . .10
Tivoli Intelligent Orchestrator installation and
subcomponent configuration verification . . .17
Tivoli Intelligent Orchestrator runtime problem
determination . . . . . . . . . . . .21
Accessing the knowledge bases . . . . . . .26
Using the available command line tools . . . .26
Chapter 3. Log file types . . . . . . .35
Message logs . . . . . . . . . . . . . .35
Message ID format . . . . . . . . . . .35
Message elements . . . . . . . . . . .36
Trace logs . . . . . . . . . . . . . . .37
Common Base Event logs . . . . . . . . . .37
Chapter 4. Locating and configuring
log files . . . . . . . . . . . . . .39
Log locations . . . . . . . . . . . . . .39
Log directory files . . . . . . . . . . .39
Logging levels . . . . . . . . . . . . .40
Configuring logs with log4j . . . . . . . . .40
log4j.prop . . . . . . . . . . . . . .40
Log level defaults . . . . . . . . . . .42
RollingFileAppender defaults . . . . . . .42
Configuring log4j dynamically . . . . . . .42
Preparing log files for review by IBM Tivoli
Support . . . . . . . . . . . . . .43
Chapter 5. Tivoli Intelligent
Orchestrator log files . . . . . . . .45
Log files for product setup . . . . . . . . .45
Log files for the uninstall process . . . . . . .45
Log files for starting and stopping the server . . .46
Log files for the We b interface . . . . . . . .47
Deployment engine log files . . . . . . . . .47
Workflow logs . . . . . . . . . . . . .48
Logs for data collection and decision making . . .49
Automation package logs . . . . . . . . . .49
Chapter 6. Middleware prerequisite log
files . . . . . . . . . . . . . . . .51
Installation log files for prerequisites . . . . . .51
Installation logs for individual prerequisites . .51
Runtime logs for prerequisites . . . . . . . .52
WebSphere Application Server . . . . . . .52
DB2 Universal Database . . . . . . . . .53
Directory Server . . . . . . . . . . . .53
Tivoli Agent Manager . . . . . . . . . .54
Chapter 7. Common problems and
known limitations in Tivoli Intelligent
Orchestrator . . . . . . . . . . . .55
Problems with installation . . . . . . . . .55
In Linux, you cannot create the user tioadmin . .55
The system cannot connect to the IBM Tivoli
Directory Server during the Tivoli Intelligent
Orchestrator installation . . . . . . . . .56
The installation of the DB2 Universal Database
Client on Windows 2003 system fails . . . . .56
Installing the common agent and the agent
manager is not supported . . . . . . . .57
The system cannot connect to the database server
during the Tivoli Intelligent Orchestrator
installation . . . . . . . . . . . . .57
The Microsoft Active Directory installation fails 57
The Microsoft Active Directory installation fails
with invalid certificate value . . . . . . . .58
© Copyright IBM Corp. 2003, 2006 iii
The silent installation program for Tivoli
Intelligent Orchestrator exits before the
installation is completed . . . . . . . . .58
Environment variables not set for user tioadmin 58
The installation of IBM Tivoli NetView on
Windows fails if the password for creating a user
account does not meet the system requirements .59
The agent manager installation fails during the
Tivoli Intelligent Orchestrator installation . . .59
An error occurs when running the rpm -qa
command after the DB2 Universal Database, IBM
Tivoli Directory Server, and WebSphere
Application Server prerequisites are installed on
Linux . . . . . . . . . . . . . . .62
Problems with logging in . . . . . . . . . .62
You cannot log in . . . . . . . . . . .62
You cannot change your password . . . . .63
Problems with workflows . . . . . . . . .64
DB2 Universal Database errors occur when you
deploy resources . . . . . . . . . . . .64
DB2 Universal Database creates a database state
error . . . . . . . . . . . . . . . .64
DB2 Universal Database deadlocks occur when
the system runs a logical operation . . . . .65
Shell command error: Resource temporarily
unavailable . . . . . . . . . . . . .65
Shell command error: Exit value=1, Error
stream=", Result stream="no bash in ..." . . . .65
IBM Tivoli Software Support personnel have
requested a copy of a workflow . . . . . .65
A workflow does not install . . . . . . . .67
Cannot run workflows if the PS1 environment
variable is altered . . . . . . . . . . .68
Other common problems . . . . . . . . . .68
On Windows, Tivoli Intelligent Orchestrator does
not start . . . . . . . . . . . . . .69
Problems occur when administering Tivoli
Intelligent Orchestrator from the Web interface
while a local firewall is enabled . . . . . .69
Incorrect SOAP parameters generate detailed
Java exception messages . . . . . . . . .70
XML import does not work properly . . . . .70
DB2 Universal Database does not work properly
with terminal server . . . . . . . . . .71
On UNIX, Tivoli Intelligent Orchestrator logs fill
up the file system . . . . . . . . . . .71
On Windows 2000, the embedded messaging
feature does not work . . . . . . . . . .71
WebSphere Application Server on Linux remains
paused when you try to stop the JMS pub sub
broker . . . . . . . . . . . . . . .72
On Windows, the remote connection to the
database hangs when the database server is on a
multiprocessor machine . . . . . . . . .72
For languages other than English, the text in CSV
reports imported in Microsoft Excel is garbled . .73
An error occurs if you use special characters to
create a new user or security role . . . . . .74
Common agents unable to communicate with the
agent manager . . . . . . . . . . . .75
Limitations when trying to manually associate a
discovered software resource with a software
definition . . . . . . . . . . . . . .75
Some patch installations might fail when trying
to perform a multiple patch installation . . . .76
There is a maximum size for returnResult
parameter storage . . . . . . . . . . .76
Device Manager limitations . . . . . . . . .76
Running lightweight management server with
DB2 Universal Database generates exceptions . .76
Agent receives an HTTP Unauthorized (401)
response code . . . . . . . . . . . .77
Known report limitations . . . . . . . . . .77
Running any report with too many objects
selected fails . . . . . . . . . . . . .77
Chapter 8. Troubleshooting the agent
manager . . . . . . . . . . . . . .79
Log files for the agent manager . . . . . . . .79
Packaging the log files . . . . . . . . . .79
Locating the installation log files . . . . . .79
Locating the uninstallation log files . . . . .81
Locating the remote registry logs . . . . . .81
Locating the runtime log files . . . . . . .82
Locating the DB2 Universal Database runtime
logs . . . . . . . . . . . . . . . .82
Determining the version of agent manager . . . .83
Installation and uninstallation verifications . . . .83
Verifying that the agent manager service is
running . . . . . . . . . . . . . . .83
Verifying the installation of WebSphere
Application Server . . . . . . . . . . .84
Verifying the installation of DB2 Universal
Database Enterprise Server Edition (ESE) . . .84
Errors starting the agent manager application
server . . . . . . . . . . . . . . .84
Errors starting the agent manager application . .84
Verifying the connection to the registry . . . .84
Enabling and disabling tracing . . . . . . .85
Recovering from an incomplete uninstallation of
the agent manager . . . . . . . . . . .86
Common problems . . . . . . . . . . . .87
Registration and communication failures . . .87
Problems using the WebSphere Administrative
Console . . . . . . . . . . . . . . .88
Chapter 9. Contacting IBM Tivoli
Software Support . . . . . . . . . .91
General data to collect for Tivoli Support . . . .91
packageLog ZIP tool . . . . . . . . . .92
Level Reporting tool for the WebSphere
Application Server platform . . . . . . . .92
Chapter 10. Tivoli Intelligent
Orchestrator messages . . . . . . .95
Common subsystem messages . . . . . . . .95
Deployment engine subsystem messages . . . . 124
Job Distribution Service (JDS) subsystem messages 135
J2ee subsystem messages . . . . . . . . . 135
Policy engines subsystem messages . . . . . . 150
iv Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
Common agent subsystem messages . . . . . 153
TC driver manager subsystem messages . . . . 154
Utilities subsystem messages . . . . . . . . 164
Compliance Messages . . . . . . . . . . 166
Group Messages . . . . . . . . . . . . 168
Networking Messages . . . . . . . . . . 169
RXA Messages . . . . . . . . . . . . . 171
Software Distribution Messages . . . . . . . 173
Chapter 11. Tivoli Common Agent
Services messages . . . . . . . . . 179
Common Agent messages . . . . . . . . . 179
Agent Manager messages . . . . . . . . . 202
Appendix A. Common Agent
serviceability tool . . . . . . . . . 221
Components . . . . . . . . . . . . . . 221
Running the serviceability tool . . . . . . . 221
Contents of the CASservice.zip file . . . . . . 221
Appendix B. Support information . . . 225
Searching knowledge bases . . . . . . . . . 225
Search the information center on your local
system or network . . . . . . . . . . . 225
Search the Internet . . . . . . . . . . 225
Obtaining fixes . . . . . . . . . . . . . 225
Contacting IBM Software Support . . . . . . 226
Determine the business impact of your problem 227
Describe your problem and gather background
information . . . . . . . . . . . . . 227
Submit your problem to IBM Software Support 227
Appendix C. Notices . . . . . . . . 229
Trademarks . . . . . . . . . . . . . . 230
Index . . . . . . . . . . . . . . . 231
Contents v
vi Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
Preface
This guide describes how to identify and resolve problems that might occur when
you use Tivoli® Intelligent Orchestrator or Tivoli Provisioning Manager.
The instructions in this book apply to both Tivoli Intelligent Orchestrator and
Tivoli Provisioning Manager and will clearly identify any information or tasks that
apply only to one application or the other.
Who should read this book
This book should be read by system administrators, operators, or anyone else
responsible for performing orchestration and provisioning tasks with the
application. The book is also addressed to all Tivoli Customer Support service
engineers or developers who are responsible for providing customer support and
for troubleshooting Tivoli Intelligent Orchestrator.
People reading this book should have knowledge in the following areas:
v Windows® 2000 Server or Windows Server 2003
v Linux® on Intel
v IBM® AIX® operating system
v Sun Solaris operating system
v SUSE Linux Enterprise Server 8 for Linux on iSeries
v Networking concepts
v Basic operating system commands
v DB2 Universal Database™ Enterprise Server Edition or Oracle9i Database
operation, configuration, and maintenance
v WebSphere® Application Server
v Basic SQL commands
v XML
v The Internet
®
™
to the online help for more information on configuring and administering
Refer
Tivoli Intelligent Orchestrator.
Publications
This section lists publications in the Tivoli Intelligent Orchestrator library and
related documents. It also describes how to access Tivoli publications online and
how to order Tivoli publications.
Read the descriptions in the Tivoli Intelligent Orchestrator library, the prerequisite
publications, and the related publications to determine which publications you
might find helpful. After you determine the publications you need, refer to the
instructions for accessing publications online.
Tivoli Intelligent Orchestrator library
The publications in the Tivoli Intelligent Orchestrator library are:
v Tivoli Intelligent Orchestrator Installation Guide
© Copyright IBM Corp. 2003, 2006 vii
Explains how to install and upgrade Tivoli Intelligent Orchestrator software.
This guide also provides information about troubleshooting installation
problems.
v Tivoli Intelligent Orchestrator Migration Guide
This book describes the steps to migrate from a previous version of Tivoli
Intelligent Orchestrator.
v Tivoli Intelligent Orchestrator Release Notes
These release notes describe last minute product changes and known problems
in the Tivoli Intelligent Orchestrator product release.
v Tivoli Intelligent Orchestrator Problem Determination Guide
This guide describes how to identify and resolve problems that occur when you
use Tivoli Intelligent Orchestrator. The instructions in this book apply to both
Tivoli Intelligent Orchestrator and Tivoli Provisioning Manager and will clearly
identify any information or tasks that apply only to one application or the other.
remaining product documentation is available in the online help, which can be
The
launched from the We b interface for the product.
Prerequisite publications
To use the information in this book effectively, you must have some prerequisite
knowledge, which you can obtain from the following publications:
v WebSphere Application Server Information Center , available from
www.ibm.com/websphere.
You can also download the WebSphere Application Server documentation
plug-in and install it in the Tivoli Intelligent Orchestrator. This plug-in enables
you to view the WebSphere Application Server documentation in the same
Information Center as the Tivoli Intelligent Orchestrator help.
v DB2 Universal Database Information Center, available from www.ibm.com/db2.
v IBM Directory Server documentation., available from http://
publib.boulder.ibm.com/tividd/td/tdprodlist.html.
®
Related publications and resources
Information related to Tivoli Intelligent Orchestrator is available in the following
locations:
v The IBM Orchestration and Provisioning Automation Library is available online
at: http://catalog.lotus.com/wps/portal/tpm. The IBM Orchestration and
Provisioning Automation Library delivers the tools that you need to build your
business around on demand automation. New workflows and automation
packages and updated readme files for automation packages that are included
with Tivoli Intelligent Orchestrator will be available in the IBM Orchestration
and Provisioning Automation Library. This site will also contain the most
up-to-date information about the full list of available workflows.
Note: Many workflows, for example the network devices and storage
workflows, cannot be categorized by platform. However, the workflow for
each readme will document any platform-specific information.
v The IBM On Demand Automation Catalog is available online at
http://www-306.ibm.com/software/tivoli/features/opal/. This site is a resource
for partners who want to develop workflows to be put into the Orchestration
and Provisioning Library.
v Redbooks™ related to Tivoli Intelligent Orchestrator are available at
www.redbooks.ibm.com
viii Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
v The Tivoli Software Library provides a variety of Tivoli publications such as
white papers, datasheets, demonstrations, redbooks, and announcement letters.
The Tivoli Software Library is available on the Web at: http://
publib.boulder.ibm.com/tividd/td/tdprodlist.html
v The Tivoli Software Glossary includes definitions for many of the technical terms
related to Tivoli software. The Tivoli Software Glossary is available at
http://publib.boulder.ibm.com/tividd/glossary/tivoliglossarymst.htm
Accessing publications online
The publications for this product are available online in Portable Document Format
(PDF) or Hypertext Markup Language (HTML) format, or both in the Tivoli
software library: http://publib.boulder.ibm.com/tividd/td/tdprodlist.html
To locate product publications in the library, click the first letter of the product
name or scroll until you find the product name. Then, click the product name.
Product publications include release notes, installation guides, user’s guides,
administrator’s guides, and developer’s references.
Note: To ensure proper printing of PDF publications, select the Fit to page check
box in the Adobe Acrobat Print window (which is available when you click
File → Print).
In addition to accessing the PDF publications, you can also access the online help
which is shipped with Tivoli Intelligent Orchestrator. The online help is available
at the external IBM Tivoli Information Center, located at: http://
publib.boulder.ibm.com/infocenter/tivihelp/index.jsp.
Ordering publications
You can order many Tivoli publications online at the following Web site:
http://www.elink.ibmlink.ibm.com/public/applications/ publications/cgibin/
pbi.cgi
You can also order by telephone by calling one of these numbers:
v In the United States: 800-879-2755
v In Canada: 800-426-4968
In other countries, see the following We b site for a list of telephone numbers:
http://www.ibm.com/software/tivoli/order-lit/
Accessibility
Accessibility features help a user who has a physical disability, such as restricted
mobility or limited vision, to use software products successfully. With this product,
you can use assistive technologies to hear and navigate the interface. Yo u also can
use the keyboard instead of the mouse to operate all features of the graphical user
interface.
Preface ix
Tivoli software training
For Tivoli software training information, refer to the IBM Tivoli Education Web
site: http://www.ibm.com/software/tivoli/education.
Support information
If you have a problem with your IBM software, you want to resolve it quickly. IBM
provides the following ways for you to obtain the support you need:
v Searching knowledge bases: You can search across a large collection of known
problems and workarounds, Technotes, and other information.
v Obtaining fixes: You can locate the latest fixes that are already available for your
product.
v Contacting IBM Software Support: If you still cannot solve your problem, and
you need to work with someone from IBM, you can use a variety of ways to
contact IBM Software Support.
more information about these three ways of resolving problems, see
For
Appendix B, “Support information,” on page 225.
Conventions used in this book
This reference uses several conventions for special terms and actions and for
operating system-dependent commands and paths.
Typeface conventions
The following typeface conventions are used in this reference:
Bold Lowercase commands or mixed case commands that are difficult to
distinguish from surrounding text, graphical user interface (GUI) controls
such as names of fields, icons, or menu choices, keywords, parameters,
options, names of Java™ classes, and objects are in bold.
Italic Variables, titles of publications, and special words or phrases that are
emphasized are in italic. Italics also indicate names for which you must
substitute the appropriate values for your system.
Monospace
Code examples, command lines, screen output, file and directory names
that are difficult to distinguish from surrounding text, system messages,
text that the user must type, and values for arguments or command
options are in monospace.
x Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
Chapter 1. Introduction to troubleshooting
Problem determination, or troubleshooting, is a process of determining why a
product is not functioning in the expected manner. This guide provides
information to help you identify and resolve problems that you encounter when
using Tivoli Intelligent Orchestrator.
Product and system overview
Tivoli Intelligent Orchestrator is an automated resource management solution for
corporate and Internet data centers. Through orchestrated provisioning, it provides
the ability to manage the IT environment in real time, according to defined
business policies, to achieve desired business goals.
Many organizations face challenges in managing their existing IT resources,
evaluating their effectiveness in supporting business needs, and determining how
to meet future needs with existing or new resources. To effectively respond to
change, financial pressures, and competition, businesses need to maintain an
environment that is flexible, responsive, and makes optimal use of available
resources. There are two aspects to creating such an environment:
Creating a flexible business model
Many organizations have difficulty responding to change because business
processes. In some organizations, processes might be too rigid and difficult
to adapt to different situations. Conversely, other organizations struggle
with processes that are unclear, inconsistent, or duplicated by participants
in the processes. To effectively respond to change, businesses must clearly
define their end-to-end business processes and subprocesses. For maximum
flexibility and consistent implementation, subprocesses are designed as
components that can be customized or reused in larger processes.
Creating
a flexible IT environment
To support changing business needs and priorities, the IT infrastructure
that supports the operations and applications in a business must be
optimized for allocation based on demand and re-purposing in different
business contexts.
Tivoli Intelligent Orchestrator helps you to develop an optimized IT environment
by:
Modeling your business processes
You can capture your business processes in workflows. Workflows
automatically and consistently perform the configuration tasks that you
currently perform manually. Tivoli Intelligent Orchestrator provides
workflows that you can customize, or you can create new ones to support
your existing tools and processes. Workflows can automate processes from
configuring and allocating servers, to installing, configuring, and patching
software, and can be either large and complex or can consist of a single
command. The automation of configuration tasks is called automated
provisioning.
Modeling
your data center assets
Tivoli Intelligent Orchestrator manages a virtual representation of your
data center in a data center model. Virtualization exists on several levels to
provide flexible configuration in different contexts:
© Copyright IBM Corp. 2003, 2006 1
v Each asset is represented by a data center object. When you make a
change to an asset with Tivoli Intelligent Orchestrator, the data center
object is updated in the data center model. If a change is made outside
of Tivoli Intelligent Orchestrator, the external change can be identified by
comparing it with the data center object in the data center model.
v For some data center assets, the data center model stores data about the
asset and data about deploying or provisioning the asset separately to
provide a range of implementation options. For example, when a
software package is added to the Tivoli Intelligent Orchestrator software
catalog, you define the software package as a installable file. Yo u can
then create software definitions that describe different configuration
requirements and configuration options for installing the same software
package.
v Yo u can create templates that define standard configurations. For
example, you can create a server template that includes the routing,
software, and storage configuration for a particular application tier.
When a server is added to the application tier, the defined configuration
is automatically applied.
v Yo u can define application topologies that describe requirements for an
application. You can then deploy an application based on the defined
application requirements.
Orchestrating
provisioning
You can configure Tivoli Intelligent Orchestrator to dynamically allocate
resources to applications based on demand or priority. Applications can
share pools of available resources to optimize utilization and eliminate idle
resources or excess allocation of resources.
Product comparison
Tivoli Intelligent Orchestrator includes Tivoli Provisioning Manager, a standalone
product that can be purchased separately, based on your data center needs. Tivoli
Provisioning Manager provides core automated deployment capability, while Tivoli
Intelligent Orchestrator adds policy-based decision-making capabilities.
Tivoli Intelligent Orchestrator uses historical and current demand and performance
data, as well as defined business policies to determine where and when to allocate
resources. When a deployment decision is made, Tivoli Intelligent Orchestrator
runs workflows to automatically make the required configuration and allocation
changes. You can specify the degree of automated provisioning that you require at
the global, application, and tier level.
The following table summarizes the complementary roles of the two products:
Tivoli Intelligent Orchestrator Tivoli Provisioning Manager
Orchestrates automation Coordinates provisioning
Determines why, when, and where: Senses
why an action is necessary, anticipates when
to start provisioning, and prioritizes where
to put resources.
Coordinates what and how: Determines
what data center assets to provision and
provisions them based on your defined
business processes.
2 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
Product architecture
Tivoli Intelligent Orchestrator includes the following main components:
Device-specific
protocols
(For ex., SNMP)
WebSphere
Application Server
SOAP
IIOP/RMI
Policy Engine
JDBC
Command Line
Interface (CLI)
JDBC
Web User
Interface
IIOP/RMI
JDBC
EJB
(
Soap Service
(Web Container)
LDAP
Managed
Data
Center
WebSphere
Application Server
EJB Container)
IIOP/RMI
LDAP
Device-specific
protocols
Device-specific
protocols
Queue
(
WebSphere
Application
Server
JDBC
MQ)
Tivoli Intelligent Orchestrator components
data center model
The data center model is an abstract representation of all physical and
logical assets in a data center that Tivoli Intelligent Orchestrator manages,
and their relationships. The data center model is stored in the Tivoli
Intelligent Orchestrator database. It communicates directly with all other
Tivoli Intelligent Orchestrator components and facilitates communication
between product components.
Database
Device-specific
protocol
Deployment
Engine
JDBC
Automation
Packages
(TC Drivers)
Discovery
policy
engine
The policy engine is responsible for orchestration capability in Tivoli
Intelligent Orchestrator. It encompasses:
data acquisition engine
This component gathers and preprocesses performance metrics
Chapter 1. Introduction to troubleshooting 3
from each managed application environment in the data center. It
captures and filters data from the application, operating system,
and infrastructure layers and communicates the data to other
components.
objective
global resource manager
deployment
The deployment engine is responsible for automated provisioning. It
creates, stores, and runs workflows and communicates their success or
failure in performing an operation. Workflows can be triggered by
commands from an administrator, by external tools that send SOAP
commands to Tivoli Intelligent Orchestrator, or by recommendations from
the policy engine.
automation
An automation package is a collection of workflows, scripts, and other
commands and tools that apply to the operation of a specific type of
software component or a physical device. The deployment engine manages
the deployment of workflows and associated workflow components in an
automation package.
analyzer
This component determines the resource requirements of each
application, and identifies trends and peaks in resource use. Each
managed application environment has an associated objective
analyzer.
This component receives requirements for servers or network
devices from all the objective analyzers, and manages the overall
optimization. It has two primary responsibilities: making optimal
resource allocation decisions, and maintaining a stable application
infrastructure. By considering the different server requirements for
each application environment, it determines where to allocate
resources.
engine
packages
discovery technologies
A discovery technology is an application that performs configuration
change detection. For example Tivoli NetView can discover new hardware
that is added to the data center and is not defined in the data center
model. For hardware that is already defined in the data center model, it
can also identify configuration changes that occur outside of Tivoli
Intelligent Orchestrator.
management interface
Tivoli Intelligent Orchestrator has two management interfaces: a Web
interface, and a command-line interface. The We b interface provides a
graphical representation of the data center, includes wizards to simplify
configuration, and offers features such as reporting and task status
monitoring that are not available from the command-line interface. The
command-line interface provides access to Tivoli Intelligent Orchestrator
features with SOAP. Administrators have the flexibility to perform tasks
such as creating scripts that run specific SOAP commands or setting up
external tools to send SOAP commands in response to an event.
Prerequisites installed with Tivoli Intelligent Orchestrator
Tivoli Intelligent Orchestrator interacts with other components that are installed
with it.
4 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
WebSphere Application Server
WebSphere Application Server hosts management interfaces and
coordinates interaction between other system components.
Database
server
The database server stores the data center model and report and audit
data.
Directory
server
The directory server manages user accounts and user authentication.
Built-in troubleshooting features
Tivoli Intelligent Orchestrator records system activity and events in message logs,
trace logs, and Common Base Event logs. You can view the contents of these log
files in a text editor.
Problem classification
This guide provides information about Tivoli Intelligent Orchestrator tools,
resources, and techniques that can help you identify and resolve the following
types of problems:
v “Problems with installation” on page 55
v “Problems with logging in” on page 62
v “Problems with workflows” on page 64
v “Other common problems” on page 68
For details about problems and resolutions for these issues, see Chapter 7,
“Common problems and known limitations in Tivoli Intelligent Orchestrator,” on
page 55.
Product maintenance
Problems can often be avoided with planning and preparation before you deploy
Tivoli Intelligent Orchestrator. Before you install the software, review the Tivoli
Intelligent Orchestrator Release Notes, the Tivoli Intelligent Orchestrator Installation
Guide, and the Tivoli Intelligent Orchestrator Migration Guide. These documents
contain the following important information:
v Supported operating system levels
v Prerequisite software requirements
v Required software fix packs
v Minimum and recommended memory requirements
v Disk space requirements
v Upgrade considerations
v Known problems, limitations, and recovery procedures
Backing up the system
After you have installed Tivoli Intelligent Orchestrator, ensure that you have a
comprehensive backup and system recovery strategy in place. Because Tivoli
Intelligent Orchestrator does not include tools for backing up or restoring your
system, you should back up your system in accordance with the documentation
that is provided with your operating system or with any specialized backup and
restore software that you use.
Chapter 1. Introduction to troubleshooting 5
You might want to consider the following general recommendations for backing up
your Tivoli Intelligent Orchestrator system:
v Perform a full backup of your existing DB2 Universal Database or Oracle
database. General guidelines for backing up DB2 Universal Database and
Oracle9i Database data are provided in the Tivoli Intelligent Orchestrator Version
5.1 Migration Guide. For more details, refer to the DB2 Universal Database
information center available at
http://publib.boulder.ibm.com/infocenter/db2help/index.jsp
or to the Oracle9i documentation available at
http://www.oracle.com/technology/documentation/oracle9i.html
v Back up your directory server database. Considerations for Microsoft Active
Directory and general guidelines for backing up an IBM Tivoli Directory Server
database are provided in the Tivoli Intelligent Orchestrator Version 5.1 Migration
Guide. For more backup and migration details, refer to the IBM Tivoli Directory
Server Installation and Configuration documentation available at:
http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer5.2.html
v Back up your automation packages (.tcdriver files) using any of the standard
backup and restore tools that you are currently using. For more details on
automation packages, refer to the Developing automation packages section of the
Tivoli Intelligent Orchestrator information center.
For Tivoli Intelligent Orchestrator publications:
1. Go to http://www.ibm.com/software/tivoli/products/intell-orch/
2. Click Technical documentation on the left.
For Tivoli Provisioning Manager publications:
1. Go to http://www.ibm.com/software/tivoli/products/prov-mgr/
2. Click Technical documentation on the left.
Performing periodic checks and maintenance
In addition to the preceding backup guidelines, here are several best practices that
can help you do even more to prevent problems:
v On Windows, periodically back up the user registry following the instructions
provided by the user registry vendor.
v Periodically check that all systems that are running Tivoli Intelligent
Orchestrator have sufficient disk space for runtime and problem determination
data. As your security policy grows, and the number of users, groups, and
protected objects increase, the space requirements for the policy databases,
message logs, trace logs, Common Base Event logs, and any auditing
information can increase as well. For more information, see Chapter 4, “Locating
and configuring log files,” on page 39.
v Regularly check for the availability of fix packs and install them as they become
available. Information on fix packs and other usual information can be found on
the Tivoli Intelligent Orchestrator support Web site, at:
http://www.ibm.com/software/sysmgmt/products/support/
IBMTivoliIntelligentOrchestrator.html
Problem resolution
Use the information in this guide to identify and resolve potential problems. If you
cannot correct the problem, gather the relevant diagnostic information as described
in this guide and then use the information in Chapter 9, “Contacting IBM Tivoli
Software Support,” on page 91 to get further assistance.
6 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
Chapter 2. Problem determination essentials for Tivoli Intelligent Orchestrator
This chapter describes how to approach troubleshooting in Tivoli Intelligent
Orchestrator. Providing methodologies and troubleshooting methods for
identifying problems in Tivoli Intelligent Orchestrator will help you to reduce the
time spent finding resolutions to these problems.
The following sections are described:
v “Problem determination methodologies”
v “Troubleshooting methods for Tivoli Intelligent Orchestrator and prerequisites”
on page 9
Problem determination methodologies
The keys to the problem determination process are understanding the problems
and identifying the causes of problems. Problem determination in Tivoli Intelligent
Orchestrator might be quite a difficult process at times, due to the complexity of
the product, the fact that it integrates with many different products, the number of
software prerequisites the product relies on, and also to the variety of platforms
the product can be installed and configured on.
The following steps are required to identify and resolve a problem in a Tivoli
Intelligent Orchestrator environment in a systematic manner:
v “Defining the problem”
v “Identifying the cause of the problem” on page 8
v “Fixing the problem” on page 9
Defining the problem
The first step is made when you determine that the product is not functioning in
the expected manner. Then, you try to correctly identify the problem, by gathering
information about the context that triggered the problem and about the problem
itself. This phase requires the following steps:
v Identifying the problem symptoms
v Providing a complete description of the error messages generated by the system
During
v How did you first notice the problem?
v Where were you in the process when the problem occurred?
v
this phase, you must be prepared to answer questions like:
– Are you a first-time user of Tivoli Intelligent Orchestrator?
– We re you installing a prerequisite?
– We re you configuring a prerequisite?
– We re you installing Tivoli Intelligent Orchestrator? If yes, was this the first
installation?
– We re you performing post-installation configurations?
– We re you performing a runtime task?
Have you previously performed the same procedure without getting errors?
What was done differently this time?
© Copyright IBM Corp. 2003, 2006 7
v What was the first symptom of the problem? Have you noticed other symptoms
occurring simultaneously?
v Does the problem affect only one system or multiple systems?
v Did you receive an error message indicating what the problem is?
v If the problem can be reproduced, what are the steps that you need to perform
to recreate it?
Identifying the cause of the problem
The information you gathered during the previous phase helped you to correctly
identify the problem and describe the context that triggered it. To be able to
identify and locate the cause of the problem, it is very important that you
understand the Tivoli Intelligent Orchestrator architecture and the interaction
between its components. This will help you to determine the specific component
that is involved with the problem:
v Data center model
v Policy engine
v Deployment engine
v Automation packages
v Discovery technologies
v Management interface
Problems
limited to a single product component have causes that are easier to
identify. Other problems might affect various components and their causes might
be subtle and more difficult to identify. Furthermore, different problems may have
the same symptoms, but different causes. Ideally, you should be able to approach
the problem in such a way so that you can isolate it to a single component.
If you cannot identify the cause of a problem, you might want to seek the
assistance of the Tivoli Support team, who will be able to pinpoint the cause of the
problem and recommend ways to recover from specific situations. For more
information on how to contact the IBM Tivoli Software Support, refer to Chapter 9,
“Contacting IBM Tivoli Software Support,” on page 91.
As Tivoli Intelligent Orchestrator integrates many different products, some of the
problem determination issues that might be considered difficult at times are:
Determining where the problem lies and what it is
The first challenge would be to determine if the problem is a customer
error, triggered by an incorrect use of the product, or an error of the
product itself, that is, a defective piece of software or hardware. Errors
encountered at startup are often missed, resulting in symptoms that are
encountered down the road. It is always worthwhile verifying whether
problems or exceptions are encountered at startup, before debugging a
symptom found down the road. Any problem found in a startup trace
should be addressed first.
Verifying whether the problem can be replicated
You need a non-production environment on which to replicate the problem
and do all tracing and analysis. Having a non-production environment
available would also allow you to test the problem on different platforms.
Checking
whether the problem has occurred before
If the same problem or a similar problem have been dealt with before, it is
very likely that extensive support documentation is provided. Begin by
8 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
searching the available IBM technotes for your product and the available
knowledge bases to determine whether the resolution to your problem is
already documented.
Analyzing
the actions that led up to the problem and generating an action plan
The approach that you should take must make use of all the available
resources and tools. You should start by verifying whether all the hardware
and software prerequisites have been met. While focusing on the problem
itself, you should always take into account the bigger picture of the Tivoli
Intelligent Orchestrator environment.
Fixing the problem
Once the problem is well defined and understood and its cause is correctly
identified, corrective steps are required to solve the problem. As an additional step
to this phase, it is always worthwhile documenting the problem for the future, to
expedite the resolution of further possible problems.
Troubleshooting methods for Tivoli Intelligent Orchestrator and prerequisites
Based on the feedback received from our Support teams, the following methods
have been proposed for troubleshooting Tivoli Intelligent Orchestrator and its
prerequisites. Used in addition to your own product knowledge and experience,
these troubleshooting methods might help you to shorten the time spent looking
for a problem resolution, by better orienting you inside the Tivoli Intelligent
Orchestrator problem determination process.
At all times during the problem determination process, keep the following points
in mind:
v The WebSphere Application Server is central to Tivoli Intelligent Orchestrator
and its components. All components, DB2 Universal Database, IBM Tivoli
Directory Server, SOAP, user interface, deployment engine, policy engine,
command line tools, interact with the WebSphere Application Server. If you do
not know where to start with a problem that you have encountered, start with
WebSphere Application Server. Check the SystemOut.log file for errors. The log
file is located in the following directory:
– On Windows: %WAS_HOME%\logs\<server-name>
– On UNIX: $WAS_HOME/logs/<server-name>
v Isolate the root error using the proposed troubleshooting methods to find a
resolution.
v Take screen shots of specific steps and errors received along the way
v Utilize and exploit the knowledge bases.
methods that are provided for troubleshooting the Tivoli Intelligent
The
Orchestrator are series of tests or questions. Troubleshooting methods are
organized so that you can start with general questions and then continue with
more specific questions that allow you to narrow down the problem space. Based
on the answers to these questions, you can isolate the problem and find a
resolution to it.
Depending on the stage you are encountering the problem, you must identify the
starting troubleshooting method within that stage, and then follow it to find the
resolution to your problem. Many troubleshooting methods will point you to the
knowledge bases.
Chapter 2. Problem determination essentials for Tivoli Intelligent Orchestrator 9
The following stages are described to the Tivoli Intelligent Orchestrator problem
determination process:
1. “Troubleshooting the prerequisite installation”
2. “Tivoli Intelligent Orchestrator installation and subcomponent configuration
verification” on page 17
3. “Tivoli Intelligent Orchestrator runtime problem determination” on page 21
4. “Accessing the knowledge bases” on page 26
5. “Using the available command line tools” on page 26
Troubleshooting the prerequisite installation
The following steps are required during this phase:
1. Ensure that the operating system has all required fixes and levels. For a
complete list of operating system environments that are supported by Tivoli
Intelligent Orchestrator, refer to the Tivoli Intelligent Orchestrator Installation
Guide, Version 5.1.
2. For your specific server topology, ensure that the prerequisite installer installed
the required versions and patches for the prerequisites and their related
components, as described in detail in the Tivoli Intelligent Orchestrator Installation
Guide. Tivoli Intelligent Orchestrator requires the following prerequisites:
v DB2 Universal Database Enterprise Server Edition, Version 8.2, Fix Pack 11 or
2000Solaris
: Oracle 9i
v IBM Tivoli Directory Server, Version 6.0, Fix Pack 1
v WebSphere Application Server, Version 6.0, Fix Pack 2.
After the prerequisite installation, you might want to follow these steps:
3.
v Ensure that the configuration of the IBM Tivoli Directory Server database
was successful. If not, refer to the “IBM Tivoli Directory Server verification”
on page 11 section.
v Perform a subcomponent verification, as described in the following
subsections:
– “DB2 Universal Database verification”
– “IBM Tivoli Directory Server verification” on page 11
– “WebSphere Application Server verification” on page 13
DB2 Universal Database verification
Follow these steps to ensure that the DB2 Universal Database was successfully
started:
1. Use the following command to start the database:
v On UNIX: $ db2start
For example, db2start on AIX has the following output:
$ db2start
07/13/2005 11:54:07 0 0 SQL1063N DB2START processing was successful.
SQL1063N DB2START processing was successful
v On Windows:
– From the services window, start the service DB2 - <instance_name>.
– The default service name in Tivoli Intelligent Orchestrator 5.1 is DB2 -
DB2.
Verify whether the designated port is available by running this command:
2.
netstat -an |grep 50000
3. Verify if all required DB2 Universal Database processes are up and running:
10 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
v On Windows, launch Task Manager and check the Processes window.
v On UNIX, run the following command to list all DB2 Universal
Databaseprocesses: db2_local_ps
required DB2 Universal Database processes include:
The
v db2fmd
v db2dasrrm
v db2fmcd
v db2wdog
v db2sysc
v db2ckpwd
v db2gds
v db2syslog
v db2ipccm
v db2tcpcm
v Db2resync
v db2srvlst
v db2agent (LDAP)
v db2fmp
v db2agent (TIOINTER)
v db2loggr (TC)
v db2loggw (TC)
v db2dlock (TC)
v db2pfchr
v db2pclnr
v db2event
4. Check the db2diag.log file for errors. The file is located in the following
directory:
v On Windows: <DB2_Instance_Home_Directory>
v On UNIX: <instance-owner-home>/sqllib/db2dump
Search the knowledge bases and, if necessary, contact Support. For more
5.
information, refer to Appendix B, “Support information,” on page 225.
IBM Tivoli Directory Server verification
The verification requires these steps:
1. Ensure that the IBM Tivoli Directory Server database was successfully created.
Check the ldapcfg.stat file to see the syntax used at the time of the database
creation. The file is located in the following directory:
v On Windows: C:\IBM\ldap\tmp
v On UNIX: /usr/ldap/tmp
should see an output similar to the following:
You
v Windows:
C:\IBM\ldap\bin\ldapcfg -n -a db2inst1 -w password -d LDAP -l
C: -c -f C:\IBM\ldap\tmp\ldapcfg.dat
v UNIX:
/usr/ldap/bin/ldapcfg -n -a db2inst1 -w th1nk -d ldap -l
/home/db2inst1 -c -f /usr/ldap/tmp/ldapcfg.dat
2. Ensure that the IBM Tivoli Directory Server was successfully started:
a. Verify whether the IBM Tivoli Directory Server database was successfully
started, using the ibmdirctl command line utility:
ibmdirctl -D cn=root -w <password> status
The ibmdirctl command is located in the following directory:
v On Windows: C:\IBM\ldap\bin
v On UNIX: /usr/ldap/bin
To be able to use the ibmdirctl command, the IBM Directory
Note:
Administrator service must be running.
Chapter 2. Problem determination essentials for Tivoli Intelligent Orchestrator 11
Use the Windows Services panel, or issue the ibmdiradm command on
UNIX to verify if the service is running. IBM Directory Administrator
uses the port number 3538 for requests.
You can also start and stop IBM Tivoli Directory Server using the ibmdirctl
command:
v Starting the IBM Tivoli Directory Server:
ibmdirctl -D cn=root -w <password> start
v Stopping the IBM Tivoli Directory Server:
ibmdirctl -D cn=root -w <password> stop
3. You can also use ldapsearch command line utility to verify whether IBM Tivoli
Directory Server is running, configured properly, and responding to queries:
ldapsearch -h <ldapserver> -p 389 -D cn=root -w <password>
-b "" -s base objectclass=*
The ldapsearch command is also located in the following directory:
v On Windows: C:\IBM\ldap\bin
v On UNIX: /usr/ldap/bin
For more information on command line tools that can be used during the IBM
Tivoli Directory Server problem determination process, refer to “IBM Tivoli
Directory Server command line tools” on page 33.
4. Verify whether the DB2 database is successfully started. For more information,
refer to “DB2 Universal Database verification” on page 10.
5. Check the runtime IBM Tivoli Directory Server log files for errors. The logs are
stored in the following directory:
v Windows: <LDAP_Install_Directory>\var\
v UNIX: /var/ldap/
<LDAP_Install_Directory> is the installation directory for the LDAP
where
server.
The log files include:
v ibmslapd.log
v audit.log
v db2diag.log
v ibmds_config.log
v ibmds_config_err.log
If required, edit the ibmslapd.conf file, which is the IBM Tivoli Directory
6.
Server configuration file that controls the server operation. The file is located in
the following directory:
v On Windows: C:\IBM\ldap\etc
v On UNIX: /usr/ldap/etc
IBM Tivoli Directory Server must be restarted for the edits to the
Note:
ibmslapd.conf file to take effect.
The ibmslapd.conf file can be used for:
v Changing the IBM Tivoli Directory Server log locations
v Adding and removing IBM Tivoli Directory Server suffixes
v Controlling server startup parameters and database connection parameters
v Changing administrator user credentials
v Modifying SSL security settings
12 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
7. Search the knowledge bases and, if necessary, contact Support. For more
information, refer to Appendix B, “Support information,” on page 225.
WebSphere Application Server verification
The verification requires the following steps:
1. Perform a WebSphere Application Server installation verification test, by
running the following script:
v On Windows: %WAS_HOME%\bin\ivt.bat
v On UNIX: $WAS_HOME/bin/ivt.sh
the ivt.log file for errors. The log file is located in the following
Check
directory:
v On Windows: %WAS_HOME%\logs
v On UNIX: $WAS_HOME/logs
2. Ensure that the correct WebSphere Application Server version was installed,
and verify the WebSphere Application Server status:
v Use the versionInfo command line utility to display the current level of
WebSphere Application Server. The command is located in the following
directory:
– On Windows: %WAS_HOME%\bin
– On UNIX: $WAS_HOME/bin
Use the serverStatus command line utility to display the WebSphere
v
Application Server status:
serverStatus -all -user <username> -password <password>
The command is located in the following directory:
– On Windows: %WAS_HOME%\bin
– On UNIX: $WAS_HOME/bin
information is logged in the serverStatus.log file, located in the
Tool
%WAS_HOME%\logs directory on Windows, $WAS_HOME/logs on UNIX.
For more information on command line tools that can be used during the
WebSphere Application Server problem determination process, refer to
“WebSphere Application Server command line tools” on page 32.
3. If required, edit the WebSphere Application Server configuration files. The
master configuration repository is located at $WAS_HOME/config. The
configuration files contain the following information:
v Cell configuration information in XML files
v Applications held as EAR files and deployment descriptors
v Templates as XML files that can be used for creating resources
Two administration clients are provided to edit the configuration of all
components:
v The administrative console, available at http://<WAS
hostname.domainname>:9090/admin/
v The wsadmin shell scripting client
The following is a list of useful configuration files:
Security.xml
This file contains server security configuration, and is located in the
$WAS_HOME/config/cells/<cell> directory. When encountering
problems logging in to the WebSphere Application Server
Chapter 2. Problem determination essentials for Tivoli Intelligent Orchestrator 13
administrative console, security can be disabled in this file by changing
the value of the enabled parameter in the second line from true to
false.
By default, <cell> represents the host name.
Server.xml
This file contains server-specific configuration information, and is
located in the $WAS_HOME/config/cells/<cell>/nodes/<node>/servers/
server1 directory.
Java.security
This file contains security properties used by the java.security classes. It
is located in the $WAS_HOME/java/jre/lib/security directory.
Sas.client.props
This file contains security settings that indicate the Java Client how to
access secured applications. It is located in the $WAS_HOME/properties
directory.
Check the runtime WebSphere Application Server log files for errors. The logs
4.
are stored in the following directory:
v On Windows: %WAS_HOME%\logs\<server-name>
v On UNIX: $WAS_HOME/logs/<server-name>
<server-name> is by default server1.
where
The runtime WebSphere Application Server log files include:
SystemOut.log
This log file contains messages generated when the applications
running inside the application server are being started or stopped.
When startingTivoli Intelligent Orchestrator, use the following
command to monitor this log file for any problems:
tail -f SystemOut.log
SystemErr.log
This log file contains Java exceptions and stack traces caused by
applications and their applications.
startServer.log
This log file contains errors that occur during the startup of the
WebSphere Application Server. Look for the following output for a
successful startup:
Server <server-name> open for e-business.
stopServer.log
This log file contains errors that occur during the shutdown of the
WebSphere Application Server. Look for the following output for a
successful shutdown:
Server <server-name> stop completed.
5. Search the knowledge bases and, if necessary, contact Support. For more
information, refer to Appendix B, “Support information,” on page 225.
Problem
determination using SystemOut.log: The following steps are required
when troubleshooting the WebSphere Application Server using SystemOut.log:
1. Look for any error during the WebSphere Application Server startup.
v If you see an error before the point where you see ...ready for e-business,
check the WebSphere Application Server configuration.
14 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
v If no error can be found, look for other symptoms in the log file.
Find the first occurrence of the error in the log file. You will either see an error
2.
message or an error exception with a stack trace.
3. Review the proper error message. It will be either from WebSphere Application
Server or from the application that caught and properly handled the exception.
Review the message and investigate.
4. Check the Java exception with the stack trace:
v Check the stack trace and follow the classes, with the package names, and
try to identify them.
v If all the classes belong to the application server and you can easily identify
them by package names, you probably have a configuration error.
v Tr y to detect the source of the problem based on class names, which are
usually very descriptive.
v If you find any class in the stack trace that belongs to the enterprise
application, you need to check the application.
SystemOut.log
successful startup breakdown:
Administration service
AdminInitiali A ADMN0015I: AdminService initialized
Secure Authentication Services (LDAP)
Configuration A SECJ0215I: Successfully set JAAS login provider
configuration class to
com.ibm.ws.security.auth.login.Configuration.
SecurityDM I SECJ0231I: The Security component’s FFDC Diagnostic Module
com.ibm.ws.security.core.SecurityDM
registered successfully: true.
SecurityCompo I SECJ0309I: Java 2 Security is disabled.
SecurityCompo I SECJ0212I: WCCM JAAS configuration information successfully
pushed to login provider class.
SecurityCompo I SECJ0240I: Security service initialization completed
successfully
SASRas A JSAS0001I: Security configuration initialized.
SASRas A JSAS0002I: Authentication protocol: CSIV2/IBM
SASRas A JSAS0003I: Authentication mechanism: LTPA
SASRas A JSAS0004I: Principal name: todaix05.think.lab.austin.-
ibm.com:389/wasadmin
SASRas A JSAS0005I: SecurityCurrent registered.
SASRas A JSAS0006I: Security connection interceptor initialized.
SASRas A JSAS0007I: Client request interceptor registered.
SASRas A JSAS0008I: Server request interceptor registered.
SASRas A JSAS0009I: IOR interceptor registered.
Map Tivoli Intelligent Orchestrator resource references
ResourceMgrIm I WSVR0049I: Binding DataSource as jdbc/KulaDB
ResourceMgrIm I WSVR0049I: Binding Default Datasource as
ResourceMgrIm I WSVR0049I: Binding QueueConnectionFactory as
ResourceMgrIm I WSVR0049I: Binding DeploymentEnginePrivateRequests
ResourceMgrIm I WSVR0049I: Binding DeploymentEngineHighLevelOutput
ResourceMgrIm I WSVR0049I: Binding TopicConnectionFactory as
jms/factory/TopicConnectionFactory
ResourceMgrIm I WSVR0049I: Binding DeploymentEngineHighLevelOutputForTEC
as jms/factory/DeploymentEngine/HighLevelOutputForTEC
ResourceMgrIm I WSVR0049I: Binding DeploymentEnginePrivateRequests
as queue/thinkcontrol/DeploymentEngine/PrivateRequests
ResourceMgrIm I WSVR0049I: Binding DeploymentEngineHighLevelOutput
as topic/thinkcontrol/DeploymentEngine/HighLevelOutput
ResourceMgrIm I WSVR0049I: Binding ThinkControlInternalMessages as
topic/thinkcontrol/Invocation
DefaultDatasource
jms/factory/QueueConnectionFactory
as jms/factory/DeploymentEngine/
PrivateRequests
as jms/factory/DeploymentEngine/
HighLevelOutput
Chapter 2. Problem determination essentials for Tivoli Intelligent Orchestrator 15
ResourceMgrIm I WSVR0049I: Binding Default_CF as
eis/DefaultDatasource_CMP
ResourceMgrIm I WSVR0049I: Binding AgentRegistry as
jdbc/AgentRegistry
Simple Object Access Protocol ( SOAP)
JMXSoapAdapte A ADMC0013I: SOAP connector available at port 8880
Applications
ApplicationMg A WSVR0200I: Starting application: DefaultApplication
EJBContainerI I WSVR0207I: Preparing to start EJB jar: Increment.jar
EJBContainerI I WSVR0037I: Starting EJB jar: Increment.jar
. . .
ApplicationMg A WSVR0221I: Application started: DefaultApplication
ApplicationMg A WSVR0200I: Starting application: ivtApp
EJBContainerI I WSVR0207I: Preparing to start EJB jar: ivtEJB.jar
EJBContainerI I WSVR0037I: Starting EJB jar: ivtEJB.jar
. . .
ApplicationMg A WSVR0221I: Application started: ivtApp
ApplicationMg A WSVR0200I: Starting application: adminconsole
WebContainer A SRVE0169I: Loading Web Module: adminconsole.
. . .
ApplicationMg A WSVR0221I: Application started: adminconsole
TCEAR Application
ApplicationMg A WSVR0200I: Starting application: TCEAR
EJBContainerI I WSVR0207I: Preparing to start EJB jar: tcEJB.jar
EJBContainerI I WSVR0037I: Starting EJB jar: tcEJB.jar
EJBContainerI I WSVR0207I: Preparing to start EJB jar: jdsEJB.jar
EJBContainerI I WSVR0037I: Starting EJB jar: jdsEJB.jar
WebContainer A SRVE0169I: Loading Web Module: tcWebUI.war.
. . .
WebContainer A SRVE0169I: Loading Web Module: tcSoap.
...
WebContainer A SRVE0169I: Loading Web Module: odiWebServices.
...
WebContainer A SRVE0169I: Loading Web Module: odiWebServicesClient.
...
WebContainer A SRVE0169I: Loading Web Module: jdsWAR.
...
MDBListenerIm I WMSG0042I: MDB Listener RecommendationsMDBPort started
successfully for JMSDestination topic/thinkcontrol/DeploymentEngine/
HighLevelOutput
MDBListenerIm I WMSG0042I: MDB Listener TECEventSenderMDBPort started
successfully for JMSDestination topic/thinkcontrol/DeploymentEngine/
HighLevelOutput
MDBListenerIm I WMSG0042I: MDB Listener DEAdaptorMDBPort started
successfully for JMSDestination topic/thinkcontrol/DeploymentEngine/
HighLevelOutput
ApplicationMg A WSVR0221I: Application started: TCEAR
Agent Manager application
ApplicationMg A WSVR0200I: Starting application: AgentManager
WebContainer A SRVE0169I: Loading Web Module: AgentManager.
...
ApplicationMg A WSVR0221I: Application started: AgentManager
ApplicationMg A WSVR0200I: Starting application: AgentRecoveryService
WebContainer A SRVE0169I: Loading Web Module: AgentRecovery.
...
ApplicationMg A WSVR0221I: Application started: AgentRecoveryService
HTTP Transport Service
HttpTransport A SRVE0171I: Transport http is listening on port 9,080.
HttpTransport A SRVE0171I: Transport https is listening on port 9,443.
HttpTransport A SRVE0171I: Transport http is listening on port 9,090.
16 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
HttpTransport A SRVE0171I: Transport https is listening on port 9,043.
HttpTransport A SRVE0171I: Transport https is listening on port 9,511.
HttpTransport A SRVE0171I: Transport https is listening on port 9,512.
HttpTransport A SRVE0171I: Transport http is listening on port 9,513.
Remote Method Invocation (RMI)
RMIConnectorC A ADMC0026I: RMI Connector available at port 2809
Startup Complete
WsServer A WSVR0001I: Server server1 open for e-business
Tivoli Intelligent Orchestrator installation and subcomponent configuration verification
Tivoli Intelligent Orchestrator configures all of the subcomponents during its
installation. You might also choose to manually configure all or some of the
subcomponents, as required. For more information on Tivoli Intelligent
Orchestrator installation log files, refer to Chapter 5, “Tivoli Intelligent Orchestrator
log files,” on page 45.
Tivoli Intelligent Orchestrator installation
If you are installing Tivoli Intelligent Orchestrator using the Tivoli Intelligent
Orchestrator installer, use the Tivoli Intelligent Orchestrator installation wizard to
enter connection information such as user IDs, passwords, paths, fully qualified
domain information, and ports, to connect to the subcomponents. Tivoli Intelligent
Orchestrator will check the validity of this information. Yo u can review the
configuration and installation information that is displayed at the end of the
installation wizard, when the progress indicator is shown.
The Tivoli Intelligent Orchestrator installer completes the following operations:
v Creates subfolders for the Tivoli common directory.
v Creates the tioadmin user or group if it does not already exist.
v Sets up the license files.
v Copies the Tivoli Intelligent Orchestrator file structure to the Tivoli Intelligent
Orchestrator installation directory.
v Installs the Tivoli Intelligent Orchestrator files.
v Copies the automation packages to <TIO HOME>/drivers.
v Copies the Eclipse plug-ins for online help system.
v Installs the language files.
v Installs GUID.
v If the Install Netview check box is selected, the Netview installer is launched.
v If the Copy Repository CD check box is selected, disk 9 that contains the agent
installers is copied to the specified directory.
v Defines the environment variables.
v Sets up the configuration data. Configuration information entered into the Tivoli
Intelligent Orchestrator wizard is stored in the %TIO_HOME%\config directory on
Windows ($TIO_HOME/config on UNIX). By editing the XML files in this
directory, it ensures that the placeholders are replaced by correct values.
v On Windows, it creates Tivoli Intelligent Orchestrator start and stop desktop
icons.
v Generates the agent manager response file template.
v Sets up the DB2 database.
v Configures the IBM Tivoli Directory Server.
Chapter 2. Problem determination essentials for Tivoli Intelligent Orchestrator 17
v Starts the WebSphere Application Server.
v Configures the WebSphere Application Server, including:
– WebSphere Application Server configuration
– WebSphere Application Server deployment
– WebSphere Application Server setup on UNIX, to run as user configuration
v Enables security for the WebSphere Application Server.
v On Windows, it creates the Windows Service.
v On Windows, it creates keys in Windows registry.
v Reinitializes the database.
v The Tivoli Intelligent Orchestrator uninstaller is automatically created by the
Install Shield MultiPlatform (ISMP).
v If the Install Agent Manager check box is selected, the agent manager installer
is launched.
v
Verifying subcomponent configuration
The following procedures are required to confirm that Tivoli Intelligent
Orchestrator and subcomponents have been successfully configured:
v Verifying Tivoli Intelligent Orchestrator’s connection to DB2 Universal Database.
See “DB2 Universal Databaseconfiguration verification.”
v Verifying the IBM Tivoli Directory Server configuration. See “IBM Tivoli
Directory Server configuration verification” on page 19.
v Verifying Tivoli Intelligent Orchestrator’s connection to IBM Tivoli Directory
Server. See “IBM Tivoli Directory Server connection verification” on page 20.
v Verifying the WebSphere Application Server configuration. See “WebSphere
Application Server configuration verification” on page 21.
Universal Databaseconfiguration verification: Tivoli Intelligent Orchestrator
DB2
uses two mechanisms to connect to the database:
1. Java connection using the information in the dcm.xml file, located at
%TIO_HOME%\config on Windows ($TIO_HOME/config on UNIX):
<config>
<database>
<type>db2</type>
<driver>COM.ibm.db2.jdbc.app.DB2Driver</driver>
<url>jdbc:db2:TC</url>
<name>TC</name>
<username>db2inst1</username>
<password>QJsdAjUEHi8=</password>
</database>
</config>
2. Kula user through the WebSphere Application Server.
In the administrative console, go to Security –> JAAS Configuration –> J2C
Authentication Data to connect to the database.
troubleshooting the DB2 Universal Database connection with Tivoli
When
Intelligent Orchestrator, follow these steps:
1. Try to manually connect to the database, and run the following command to
list all DB2 Universal Database tables:
list tables
18 Tivoli Intelligent Orchestrator Problem Determination and Troubleshooting Guide
Loading...