This document introduces HP Reliable Transaction Router and describes its
concepts for the system manager, system administrator, and applications
programmer.
Confidential computer software. Valid license from HP required for possession,
use or copying. Consistent with FAR 12.211 and 12.212, Commercial
Computer Software, Computer Software Documentation, and Technical Data
for Commercial Items are licensed to the U.S. Government under vendor’s
standard commercial license.
The information contained herein is subject to change without notice. The only
warranties for HP products and services are set forth in the express warranty
statements accompanying such products and services. Nothing herein should
be construed as constituting an additional warranty. HP shall not be liable for
technical or editorial errors or omissions contained herein.
Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.
Intel and Itanium are trademarks or registered trademarks of Intel Corporation
or its subsidiaries in the United States and other countries.
UNIX is a registered trademark of The Open Group. Java is a US trademark
of Sun Microsystems, Inc.
This document was prepared using DECdocument, Version 3.3-1B.
2–1Functional and Object-Oriented Programming Compared . . .2–8
4–1RTR Development Interfaces and their Use ..............4–5
vi
Purpose of this Document
The goal of this document is to assist an experienced system
manager, system administrator, or application programmer to
understand the Reliable Transaction Router (RTR) product.
Document Structure
This document contains the following chapters:
•Chapter 1, Introduction, provides information on RTR
technology, basic RTR concepts, and RTR terminology.
•Chapter 2, Architectural Concepts, introduces the RTR
three-tier model and explains the use of RTR functions and
programming capabilities.
•Chapter 3, Reliability Features, highlights RTR server types
and failover and recovery scenarios.
•Chapter 4, RTR Interfaces, introduces the management and
programming interfaces of RTR.
Preface
•Chapter 5, The RTR Environment, describes the RTR
system management and runtime environments, and
provides explicit pointers to further reading in the RTR
documentation set.
vii
Related Documentation
Table 1 describes RTR documents and groups them by
audience.
Table 1 RTR Documents
DocumentContent
For all users:
HP Reliable Transaction Router
Release Notes
HP Reliable Transaction Router
Getting Started
HP Reliable Transaction Router
Software Product Description
For the system manager:
HP Reliable Transaction Router
Installation Guide
HP Reliable Transaction Router
System Manager’s Manual
For the application
programmer:
HP Reliable Transaction Router
Application Design Guide
1
Describes new features,
corrections, restrictions, and
known problems for RTR.
Provides an overview
of RTR technology and
solutions, and includes the
glossary that defines all
RTR terms.
Describes product features.
Describes how to install
RTR on all supported
platforms.
Describes how to configure,
manage, and monitor RTR.
Describes how to design
application programs for
use with RTR, with both
C++ and C interfaces.
viii
1
Distributed on software kit.
(continued on next page)
Table 1 (Cont.) RTR Documents
DocumentContent
HP Reliable Transaction Router
JRTR Getting Started
2
Provides an overview
of the object-oriented
JRTR Toolkit including
installation, configuration
and Java programming
concepts, with links
to additional online
documentation.
HP Reliable Transaction Router
C++ Foundation Classes
Describes the objectoriented C++ interface that
can be used to implement
RTR object-oriented
applications.
HP Reliable Transaction Router
C Application Programmer’s
Reference Manual
Explains how to design
and code RTR applications
using the C programming
language and the RTR
C API. Contains full
descriptions of the basic
RTR API calls.
2
In downloadable kit.
You can find additional information about RTR, including the
Software Product Descriptions, on the RTR website found
through http://www.hp.com links to middleware products or at
http://www.hp.com/go/rtr .
Reader’s Comments
HP welcomes your comments on this manual. Please send
comments to either of the following addresses:
introduced and defined. All RTR terms
are defined in the glossary at the end of
this document.
User input
Terms and titlesTerms defined only in the glossary are
FERTR frontend
TRRTR transaction router or router
BERTR backend
Refer to . . .References another manual.
See . . .References another part of this manual.
User input and programming examples
are shown in a monospaced font.
Boldface monospaced font indicates
user input.
shown in italics when presented for
the first time. Italics are also used for
titles of manuals and books, and for
emphasis.
Reading Path
x
The reading path to follow when using the Reliable Transaction
Router information set is shown in Figure 1.
Figure1RTRReadingPath
SPD
Getting
Started
Release
Notes
= Glossary
System ManagerApplication Programmer
Application
Installation
Guide
Design
Guide
If Java
JRTR
Getting
Started
(Online Only)
System
Manager’s
Manual
RTR Help
(Online Only)
If C
C Application
Programmer’s
Reference
Manual
If C++
C++
Foundation
Classes
= Tutorial
VM-0818A-AI
xi
This document introduces RTR and describes RTR concepts. It is
intended for the system manager or administrator and for the
application programmer who is developing an application that
works with Reliable Transaction Router (RTR).
Reliable Transaction Router
Reliable Transaction Router (RTR) is failure-tolerant
transactional messaging middleware used to implement large,
distributed applications with client/server technologies. RTR
helps ensure business continuity across multivendor systems and
helps maximize uptime.
1
Introduction
Failure and fault
tolerance
Failure tolerance is supplied by the RTR software that enables
an application to continue even when failures such as node or
site outages occur. Failover is automatic. Fault tolerance is
supplied by systems with hardware that is built with redundant
components to ensure that processing survives failure of an
individual component. Depending on system requirements and
hardware, an RTR configuration can be both failure and fault
tolerant.
Introduction 1–1
Reliable Transaction Router
Interoperability
Networking
Transaction
processing
You use the architecture of RTR to ensure high availability and
transaction completion. RTR supports applications that run
on different hardware and different operating systems. RTR
applications can be designed to work with several database
products including Oracle, Microsoft Access, Microsoft SQL
Server, Sybase, and Informix. For specifics on operating systems,
operating system versions, and supported hardware, refer to the
HP Reliable Transaction Router Software Product Description for
each supported operating system.
RTR can be deployed in a local or wide area network and can use
either TCP/IP or DECnet for its underlying network transport.
A transaction involves the exchange of something for something
else, for example, the exchange of cash for something purchased
such as a book or restaurant meal. Transactions are the fodder
of the commercial world in which we live. Transaction processing
is the use of computers to do the bookkeeping for the physical
transactions in which people engage. RTR is the premier
transaction processing software available for many computer
systems, offering unique benefits to ensure the integrity and
correctness of transactions under its control.
Corporations and institutions benefit when the data they need
is current and kept rapidly up-to-date. This ensures that they
have increased control of their business or institution, and can
react more effectively to changes in the business or institutional
environment.
1–2 Introduction
For example, an inventory management system provides current
inventory for inquiry and can automatically request inventory
updates when needed. Or an production manager can make
production decisions based on current status of manufacturing
elements directly from the shop floor, and a credit card company
can respond with accurate information when a card request for
validation arrives. Or a shipping company can determine and
report the location of a package in transit anywhere in the world,
or a personnel system can enable an employee to update personal
information online. All of these systems perform transaction
processing tasks as designed by their developers.
With RTR, applications can be written to be deployed over a
wide geography to take advantage of distributed resources,
both computers and personnel. Implementing a transaction
processing system using RTR requires analysis, planning, and
considered execution.
RTR Continuous Computing Concepts
RTR provides a continuous computing environment that is
particularly valuable in financial transactions, for example
in banking, stock trading, or passenger reservations systems.
RTR satisfies many requirements of a continuous computing
environment:
•Reliability
•Failure tolerance
•Data and transaction integrity
•Scalability
•Ease of building and maintaining applications
Reliable Transaction Router
•Interoperability with multiple operating systems
RTR additionally provides the following capabilities, essential in
the demanding transaction processing environment:
•Flexibility
•Parallel execution at the transaction level
•Potential for step-by-step growth
•Comprehensive monitoring tools
•Management station for single console system management
•WAN deployability
RTR also ensures that transactions have the ACID properties
that have been established as crucial in a transaction processing
environment. A transaction with the ACID properties is:
•Atomic
•Consistent
•Isolated
Introduction 1–3
RTR Continuous Computing Concepts
•Durable
For more details on transactional ACID properties, see the brief
discussion later in this document in the section Transaction
Integrity and refer to the HP Reliable Transaction RouterApplication Design Guide.
RTR Terminology
In addition to the terms previously defined, the following
terms are either unique to RTR or redefined when used in
the RTR context. If you have learned any of these terms in
other contexts, take the time to assimilate their meaning in the
RTR environment. These and other terms are also defined in
the Glossary of this manual. The terms are described in the
following order:
•Application
1–4 Introduction
•Client, client application
•Server, server application
•Channel
•RTR configuration
•Roles
•Frontend
•Router
•Backend
•Facility
•Transaction
•Transaction controller
•Transactional messaging
•Nontransactional messaging
•Transaction ID
•Tier
•Standby server
•Transactional shadowing
•RTR journal
•Partition
•Key range
•XA
RTR Terminology
RTR Application
Client
An RTR application is user-written software that executes
within the confines of several distributed processes. The RTR
application may perform user interface, business, and server
logic tasks and is written in response to some business need.
An RTR application can be written in one of the supported
languages, C, C++, or Java and includes calls to RTR. RTR
applications are composed of two kinds of actors, client
applications and server applications. An application process
is shown in diagrams as an oval, open for a client application
(see Figure 1–1), filled for a server application (see Figure 1–2).
A client is always a client application, one that initiates
and demarcates a piece of work. In the context of RTR, a client
must run on a node defined to have the frontend role. Clients
typically deal with presentation services, handling forms input,
screens, and so on. A client could connect to a browser running a
browser applet or be a webserver acting as a gateway. In other
contexts, a client can be a physical system, but in RTR and in
this document, physical clients are called frontends or nodes.
You can have more than one instance of a client on a node.
Figure 1–1 Client Symbol
Client
VM-0819A-AI
Introduction 1–5
RTR Terminology
Server
Channel
A server is always a server application, one that reacts to a
client’s units of work and carries them through to completion.
This may involve updating persistent storage such as a database
file, toggling a switch on a device, or performing another
predefined task. In the context of RTR, a server must run on
a node defined to have the backend role. In other contexts,
a server can be a physical system, but in RTR and in this
document, physical servers are called backends or nodes. You
can have more than one instance of a server on a node. Servers
interact with partitions and can have partition states such as
primary, standby, or shadow.
Figure 1–2 Server Symbol
Server
VM-0820A-AI
RTR expects client and server applications to identify themselves
before they request RTR services. During the identification
process, RTR provides a tag or handle that is used for subsequent
interactions. This tag or handle is called an RTR channel.A
channel is used by client and server applications to exchange
units of work with the help of RTR. An application process
can have one or more client or server channels. Channel
management is handled transparently by the C++ and Java
APIs.
RTR configuration
1–6 Introduction
An RTR configuration consists of nodes that run RTR client
and server applications. An RTR configuration can run on
several operating systems including OpenVMS, Tru64 UNIX, and
Windows among others (for the full set of supported operating
systems, refer to the appropriate SPD). Nodes are connected by
network links.
RTR Terminology
Roles
A node that runs client applications is called a frontend
(FE), or is said to have the frontend role. A node that runs
server applications is called a backend (BE). Additionally, the
transaction router (TR) contains no application software but
acts as a traffic cop between frontends and backends, routing
transactions to the appropriate destinations. The router controls
the distributed RTR nodes, and takes care of two-phase commit,
failover and failback.
The router also eliminates any need for frontends and backends
to know about each other in advance. This relieves the
application programmer from the need to be concerned about
network configuration details. The router can reside on a node
running as a frontend or a backend but is often run on a node
where neither backends nor frontends are running. Figure 1–3
shows the symbol for each of the RTR roles.
Figure 1–3 Roles Symbols
FETRBE
Facility
FrontendTransactional
Router
Backend
VM-0821A-AI
The mapping between nodes and roles is done using a facility.
An RTR facility is the user-defined name for a particular
configuration whose definition provides the role-to-node map
for a given application. The facility symbol (see Figure 1–4)
illustrates its use in the RTR environment. Nodes can share
several facilities. The role of a node is defined within the scope
of a particular facility. Normally a facility is defined across all
roles but facility definition depends on application design.
The router is the only role that knows about all three roles.
A router can run on the same physical node as the frontend
or backend, if that is required by configuration constraints,
but such a setup would not take full advantage of failover
characteristics.
Introduction 1–7
RTR Terminology
Figure 1–4 Facility Symbol
VM-0822A-AI
A facility name is mapped to specific physical nodes and their
roles using the CREATE FACILITY command.
Figure 1–5 shows the logical relationship between client
application, server application, frontends (FEs), routers (TRs),
and backends (BEs) in the RTR environment at a specific
location. The database is represented by the cylinder. Two
facilities are shown (indicated by the large double-headed
arrows), the User Accounts Facility and the General Ledger
Facility. The User Accounts Facility uses three nodes, FE, TR,
and BE, while the General Ledger Facility uses only two, TR and
BE in the configuration shown. Its FEs are on nodes not shown
in the figure, at another location.
Figure 1–5 Components in the RTR Environment
User Accounts Facility
FETRBE
Client
application
General Ledger Facility
1–8 Introduction
Server
application
VM-0823A-AI
RTR Terminology
Clients send messages to servers to ask that a piece of work be
done. Such requests may be bundled together into transactions.
An RTR transaction consists of one or more messages that have
been grouped together by a client application, so that the work
done as a result of each message can be undone completely, if
some part of that work cannot be done. If the system fails or is
disconnected before all parts of the transaction are done, then
the transaction remains incomplete.
Transaction
Transaction
Controller
Transactional
messaging
A transaction is a piece of work or group of operations that
must be executed together to perform a consistent transformation
of data. This group of operations can be distributed across many
nodes serving multiple databases. Applications use services that
RTR provides.
Because typically a transaction consists of several operations, a
system or network failure at any step in the process will cause
the transaction to be in doubt. RTR ensures that all transactions
have the ACID properties so that all transactions are in a known
state. (See the description of ‘‘Transactional messaging’’ for
further clarification of transaction integrity.)
With the C++ API, the Transaction Controller manages
transactions (one at a time), channels, messages, and events.
RTR provides transactional messaging in which transactions
are enclosed in messages controlled by RTR.
Transactional messaging ensures that each transaction is
complete, and not partially recorded. For example, a transaction
or business exchange in a bank account might be to move money
from a checking account to a savings account. The complete
transaction is to remove the money from the checking account
and add it to the savings account.
A transaction that transfers funds from one account to another
consists of two individual updates: one to debit the first account,
and one to credit the second account. The transaction is not
complete until both actions are done. If a system performing
this work goes down after the money has been debited from the
checking account but before it has been credited to the savings
account, the transaction is incomplete.
Introduction 1–9
RTR Terminology
With transactional messaging, RTR ensures that a transaction
is ‘‘all or nothing’’—either fully completed or discarded; either
both the checking account debit and the savings account credit
are done, or the checking account debit is backed out and not
recorded in the database. RTR transactions have the ACID
properties.
Nontransactional
messaging
Transaction ID
Tiers
An application will also contain nontransactional tasks such
as writing diagnostic trace messages or sending a broadcast
message about a change in a stock price after a transaction has
been completed.
Every transaction is identified on initiation with a transaction
identifier or transaction ID (TID), with which it can be logged
and tracked. RTR guarantees that TIDs are unique.
To reinforce the use of these terms in the RTR context, this
section briefly reviews other uses of configuration terminology.
A traditional two-tier client/server environment is based on
hardware that separates application presentation and business
logic (the clients) from database server activities. The client
hardware runs presentation and business logic software, and
server hardware runs database or data manager (DM) software,
also called resource managers (RM). This type of configuration is
illustrated in Figure 1–6. (In all diagrams, all lines are actually
bidirectional, even when represented otherwise for clarity. For
a given transaction, the initial action is typically from left to
right.) In Figure 1–6, Application Presentation and Business
Logic are the first tier, and the Database Server is the second
tier.
Further separation into three tiers is achieved by separating
presentation software from business logic on two systems,
and retaining a third physical system for interaction with the
database. This is illustrated in Figure 1–7, where Presentation
and User Interface are the first tier, the Application Server and
Business Logic are the second tier, and the Database Server is
the third tier.
1–10 Introduction
Figure 1–6 Two-Tier Client/Server Environment
Data Manager
software
RTR Terminology
Application Presentation
and Business Logic
(ODBC Model)
Figure 1–7 Three-Tier Client/Server Environment
Presentation/User Interface
Application Server/
Business Logic
RTR extends the three-tier model, which is based on hardware,
to a multilayer, or multicomponent software model.
RTR Software
Components
RTR provides a multicomponent software model where client
applications running on frontends, combined with routers and
server applications running on backends cooperate to provide
reliable service and transactional integrity. Application users
interact with the client (presentation layer) on the node with the
frontend role that forwards messages to the current router. The
router in turn routes the messages to the current, appropriate
backend, where server applications reside, for processing. The
Database
Server
Database
Server
Database
Application
Database
Server
VM-0824A-AI
VM-0825A-AI
Introduction 1–11
RTR Terminology
connection to the current router is maintained until the current
router fails or connections to it are lost.
All RTR software components can reside on a single node but
are typically deployed on different nodes to achieve modularity,
scalability, and redundancy for availability. During initial
application development, it can be convenient to use a single
physical node for all RTR roles and application software.
With different physical systems, if one node goes down or off
line, another router or backend node can take over application
processing. In a slightly different configuration, you could have
an application that uses an external applet component running
on a browser that connects to a client running on the RTR
frontend. Such a configuration is shown in Figure 1–8. In the
figure, the applet is separate from but connects to the client
application written to work directly with RTR.
Figure 1–8 Browser Applet Configuration
1–12 Introduction
Applet
Web Server
Process
RTR Client
Application
RTR FrontendPC Browser
VM-0826A-AI
The RTR client application could be an ASP (Active Server Page)
or a process interfacing to the webserver through a standard
interface such as CGI (Common Gateway Interface) script.
RTR provides automatic software failure tolerance and failure
recovery in multinode environments by sustaining transaction
integrity in spite of hardware, communications, application, or
site failures. This automatic failover and recovery of service can
exploit redundant or underutilized hardware and network links.
RTR Terminology
For example, you could use an underutilized system as a standby
server in certain configurations.
As you modularize your application and distribute its
components on frontends and backends, you can identify
usage bottlenecks, add new nodes, and provide redundancy to
increase availability. Adding backend nodes can help divide
the transactional load and distribute it more evenly. For
example, you could have a single node configuration as shown in
Figure 1–9, RTR with Browser, Single Node, and Database. A
single node configuration can be useful during development, but
would not normally be used when your application is deployed.
Figure 1–9 RTR with Browser, Single Node, and Database
TRFEBE
Database
Browser
VM-0827A-AI
When applications are deployed, often the frontend is separated
from the backend and router, as shown in Figure 1–10.
Figure 1–10 RTR Deployed on Two Nodes
TRFEBE
Client
application
Browser
Server
application
Introduction 1–13
Database
Journal
VM-0828A-AI
RTR Terminology
In this example, the frontend with the client application resides
on one node, and the router with the server application reside
a node that has both the router and backend roles. This is
a typical configuration where routers are placed on backends
rather than on frontends. A further separation of workload
onto three nodes is shown in Figure 1–11. However, in this
configuration, there remain several single points of failure where
one node/role or a network outage can disrupt processing of
transactions.
Figure 1–11 RTR Deployed on Three Nodes
TRFEBE
Database
Browser
VM-0829A-AI
While this three-node configuration separates transaction load
onto three nodes, it does not provide for continuing work if one
of the nodes fails or becomes disconnected from the others. In
many applications, there is a need to ensure that there is a
server always available to access the database.
Standby server
1–14 Introduction
In this case, a standby server will do the job. A standby server
(see Figure 1–12) is a process or application that can take over
when the primary server is not available, due to hardware
failure, application software failure or network outage.
Both the primary and the standby server have the capability
to access the same database, but the primary processes all
transactions unless it is unavailable. On the other hand, the
standby processes transactions only when the primary becomes
unavailable. When not being used to process transactions, the
standby CPU can do other work.
RTR Terminology
The standby server is usually placed on a node other than the
node where the primary server runs, and should be, to avoid
being a single point of failure. Network capability, clustering
or disk-sharing technology, and appropriate software must be
available on both primary and standby backend systems when
running RTR.
Figure 1–12 Standby Server Configuration
Shadow Server
and Transactional
shadowing
Primary Server
Database
TR
BE
Server
application
BE
Server
application
Standby Server
VM-0830A-AI
To increase transaction processing availability, transactions can
be shadowed with a shadow server, as shown in Figure 1–13.
The system where the shadow server runs can be made available
with clustering technology. A shadow server eliminates the
single point of failure that is evident in Figure 1–12. In a
shadow configuration, the second database of Figure 1–13 is
available even when the first is not.
Use of a shadow server is called transactional shadowing and
is accomplished by having a second location, often at a different
site, where transactions are also recorded. Data are recorded in
two separate data stores or databases. The router knows about
both backends and sends all transactions to both backends. RTR
provides the server application with the necessary information to
keep the two databases synchronized.
Introduction 1–15
RTR Terminology
Figure 1–13 Transactional Shadowing Configuration
RTR Journal
Primary Server
TRFE
BE
Server
application
Database
BE
Server
application
Database
Shadow Server
VM-0831A-AI
In the RTR environment, one data store (database or data
file) is elected the primary, and a second data store is made
the shadow. The shadow data store is a copy of the data store
kept on the primary. If either data store becomes unavailable,
all transactions continue to be processed and stored on the
surviving data store. At the same time, RTR makes a record
of (remembers) all transactions stored only on the shadow data
store in the RTR journal by the shadow server.
When creating the configuration used by an application and
defining the nodes where a facility has its frontends, routers,
and backends, the setup must also define which nodes will have
journal files. Each backend in an RTR configuration must have
a journal file to capture transactions when other nodes are
unavailable. When the primary server and data store become
available again, RTR replays the transactions in the journal to
the primary data store through the primary server. This brings
the data store back into synchronization.
1–16 Introduction
With transactional shadowing, there is no requirement that
hardware, the data store, or the operating system at different
sites be the same. You could, for example, have one site running
OpenVMS and another running Windows; the RTR transactional
commit process would be the same at each site. Because the
database resides at both sites, either backend can have an
outage and all transactions will still be processed and recovered.
For full redundancy to assure maximum availability, a
configuration could employ disk shadowing in clusters at
separate sites coupled with transactional shadowing across
sites. Disk shadowing used in specific cluster environments
copies data to another disk to ensure data availability.
Transactional shadowing copies only transactional data.
Additionally, an RTR configuration typically deploys multiple
frontends running client applications with connections to several
routers to ensure continuing operation if a particular router fails.
RTR Server Types
RTR Terminology
Note
Transactional shadowing shadows only transactions
controlled by RTR.
In the RTR environment, in addition to the placement of
frontends, routers, and servers, the application designer must
determine what server capabilities to use. RTR provides four
types of software servers for application use:
•Standby servers
•Transactional shadow servers
•Concurrent servers
•Callout servers
These are described in the next few paragraphs. You specify
server types to your application in RTR API calls.
RTR server types help to provide continuous availability and a
secure transactional environment.
Introduction 1–17
RTR Server Types
Standby server
Standby in a
cluster
The standby server remains idle while the RTR primary
backend server performs its work, accepting transactions and
updating the database. When the primary server fails, the
standby server takes over, recovers any in-progress transactions,
updates the database, and communicates with clients until
the primary server returns. There can be many instances of a
standby server. Activation of the standby server is transparent
to the user.
A typical standby configuration is shown in Figure 1–12,
Standby Server Configuration. Both physical servers running
the RTR backend software are assumed by RTR to connect
to the same database. The primary server is typically in use,
and the standby server can be either idle or used for other
applications, or data partitions, or facilities. When the primary
server becomes unavailable, the standby server takes over and
completes transactions as shown by the dashed line. Primary
server failure could be caused by server process failure or
backend (node) failure.
The intended and most common use of a standby server
is in a recognized cluster environment. In a noncluster or
unrecognized cluster environment, seamless failover of standbys
is not guaranteed. For RTR, clusters supported by OpenVMS
and Tru64 UNIX are recognized clusters, whose processing is
controlled by a lock manager. Windows and Sun clusters can use
disk-sharing, unrecognized cluster technology.
1–18 Introduction
Standby servers are ‘‘spare’’ servers that automatically take over
from the main backend if it fails. This takeover is transparent to
the application.
Figure 1–14 shows a simple standby configuration. The two
backend nodes are members of a cluster environment, and are
both able to access the database.
For any one key range, the main or primary server (Server
application) runs on one node while the standby server (Standby
application) runs on the other node. The standby server process
is running, but RTR does not pass any transactions to it. Should
the primary node fail, RTR starts passing transactions to the
Standby application.
Loading...
+ 70 hidden pages
You need points to download manuals.
1 point = 1 manual.
You can buy points or you can get point for every manual you upload.