HP InfiniBand solution for Oracle Application Server Implementation Manual

HP InfiniBand solution for Oracle Application Server

Overview......................................................................................................................................2

Oracle Application Server overview...............................................................................................2

Advantages of InfiniBand in an Oracle Application Server environment ............................................2

HP InfiniBand solution for Oracle Application Server.........................................................................3

Solution overview.................................................................................................................... 3

Sample HP InfiniBand configuration for Oracle AS 10G and Oracle RDBMS 10G..............................4

Installation process..........................................................................................................................5

Oracle AS on InfiniBand installation process....................................................................................5

Hardware setup and configuration .............................................................................................5

Red Hat Enterprise Linux® installation .........................................................................................5

Installation of InfiniBand software components..............................................................................5

Oracle RDBMS 10G installation.................................................................................................6

Oracle Application Server 10G installation..................................................................................6

Transparent SDP configuration...................................................................................................7

Sample J2EE application installation........................................................................................... 8

For more information....................................................................................................................... 9

Overview

This whitepaper explains how to implement an Oracle Applications Server (AS) J2EE solution for the
database connection using HP ProLiant servers, HP StorageWorks, and HP InfiniBand™ options. The
whitepaper is intended for system administrators, system architects, and systems integrators who are
considering the advantages of HP InfiniBand-based systems in an Oracle AS J2EE environment.

Oracle Application Server overview

Oracle AS is a fully integrated application server – including a J2EE Compatible container, an
enterprise portal, wireless, business intelligence, single sign-on using a LDAP compliant directory, and
much more. It is the integrated nature of Oracle AS that distinguishes it in the marketplace. Oracle
strives not only to provide a container for J2EE applications, but to enable J2EE applications to be
managed more easily, deployed as web services, accessed via a single sign-on, made more secure,
and tied more closely to business level activities.

Typically, the Oracle AS is installed on its own server or set of servers and these servers access an
Oracle database instance on another server across a network. The most typical network connection
between Oracle AS server and the Oracle database server is via TCP/IP over Ethernet.

A typical Oracle Application Server system has the following components:

• One or more application servers

• Oracle database instance

• Public local area network (LAN)

Advantages of InfiniBand in an Oracle Application Server environment

The most common method of communication between application server machines and database
server machines today is TCP/IP over Ethernet. TCP/IP and Ethernet both have limitations. The TCP/IP
processing on the application server and database server machines can result in substantial CPU
utilization on both machines with typical latencies of 50 to 100 microseconds (µs). Additionally,
observed Gigabit Ethernet throughput is limited to approximately 0.8 gigabits per second (Gbps) on
a single network interface card (NIC).

InfiniBand, a standards-based alternative to Ethernet, overcomes the performance limitations of
TCP/IP-based solutions. The key advantages of InfiniBand over TCP/IP and Ethernet is InfiniBand’s
high throughput, low CPU utilization, and ultra-low latency. The current InfiniBand bandwidth of 10
Gbps1provides observed performance over 8 Gbps. CPU utilization of only 1-3 % with latency less
than 5 µs is possible.

InfiniBand supports its own InfiniBand-specific protocol, sockets direct protocol (SDP). This is the
standard wire protocol for the InfiniBand Architecture (IBA) for support of stream sockets
(SOCK_STREAM) networking over IBA.

There are two different versions of SDP protocol that are supported by Oracle. The first is native SDP
support, where the Oracle product implements the SDP directly. The second protocol, called
Transparent SDP, is where the Oracle product is configured to use TCP, and the SDP protocol
provider converts TCP connections to SDP without any configuration changes at the Oracle level.
Native SDP provides support for asynchronous I/O, whereas Transparent SDP only supports
synchronous I/O.

Because it supports asynchronous I/O, native SDP supports one of the most important features of
InfiniBand – remote direct memory access (RDMA). RDMA is a communications technique that allows
data to be transmitted from the memory of one computer to the memory of another computer without

1

InfiniBand 10Gbps bandwidth is represented as 4x or four lanes of traffic each at 2.5 Gbps.

2

passing through either device’s CPU, without needing extensive buffering, and without calling to an
operating system kernel. RDMA can bring about a substantial reduction in CPU utilization on both the
application and database servers.

Oracle’s 10G RDBMS product supports native SDP; however, Oracle products that use an earlier
version of the Oracle DB client can only utilize Transparent SDP. Both Oracle AS 9i and 10G Release
1 use the Oracle 9i DB client. Therefore, they can only use Transparent SDP. While this means that
Oracle AS cannot take advantage of asynchronous I/O or RDMA today, it still benefits from the
additional bandwidth and higher throughput provided by InfiniBand with Transparent SDP. Oracle AS
10G Release 2 will support native SDP.

Applications that connect to the database by using thin JDBC driver are also restricted to Transparent
SDP. Applications that use the OCI driver can use native SDP protocol provided the product support
SDP natively.

HP InfiniBand solution for Oracle Application Server

Solution overview

Many HP customers run J2EE applications in an Oracle Application Server environment using
standards-based HP ProLiant servers and HP StorageWorks Fibre Channel connectivity products.
With the addition of HP InfiniBand options for ProLiant servers, customers can implement a complete
HP-based Oracle Application Server solution using InfiniBand for the database connection.

Figure 1 depicts a sample configuration composed of the following components:

• Application Server

– HP ProLiant DL380 G4 server
– (4) 36GB disk drives

• Database Server

– HP ProLiant DL380 G4 server
– (2) 36GB disk drives
– HP StorageWorks solution

§ HP StorageWorks Modular Smart Array (MSA) or Enterprise Virtual Array (EVA)

§ HP StorageWorks FCA2214DC Fiber Channel HBA

§ HP StorageWorks SAN Switch

• HP InfiniBand cluster interconnect
– HP NC570C dual port PCI-X InfiniBand HCA

– HP 24 port 4x Copper Fabric Switch

3