IBM 6.1.X User Manual

IBM WebSphere Portal software family Your world. Your way.

IBM WebSphere Portal 6.1.X

Performance Tuning Guide

IBM WPLC Performance Team March 2009

Document version 2.1

Contents

PERFORMANCE TUNING OVERVIEW

Environment Considerations

32-bit and 64-bit Considerations Hardware Multithreading (Hyper-Threading)

BASE PORTAL TUNING

Application Server Tuning

JVM Initial and Maximum Heap Size JVM Heap Large Page JVM Heap New Area Size Additional SUN JVM Arguments Session Timeout Web Container Thread Pool Size Security Attribute Propagation VMM Context Pooling ORB Service Tuning For z/OS

WebSphere Portal Services

Navigator Service Registry Service Cache Manager Service

Database Tuning

Datasource Tuning For DB2 DB2 Database Server Tuning Oracle Database Server Tuning

Other Database Considerations Directory Server Tuning Web Server Tuning Operating System Tuning

AIX

Linux

Windows 2003

Solaris

Z/OS Required Fixes

WEB 2.0 THEME TUNING

.................................................................................................................................4

...................................................................................................................5

.........................................................................................................................9

.................................................................................................................11

......................................................................................................................12

........................................................................................................................13

............................................................................................................................15

....................................................................................................................22

......................................................................................................................... 23

..................................................................................................................25

........................................................................................................................................25

.....................................................................................................................................26

.......................................................................................................................... 26

....................................................................................................................................27

......................................................................................................................................29

..............................................................................................................................29

.............................................................................................................................30

...............................................................................................................2

................................................................................................................3

.......................................................................................................3

........................................................................................3

.................................................................................................5

..................................................................................................................7

..............................................................................................................8

.......................................................................................................8

.....................................................................................................9

....................................................................................................... 10

....................................................................................................... 11

...............................................................................................................12

..............................................................................................................14

.........................................................................................................15

........................................................................................................15

.....................................................................................................19

.....................................................................................................21

JVM Initial and Maximum Heap Size Navigator Service Properties Internet Explorer Support of Vary Header Caching Proxy Tuning

......................................................................................................................31

....................................................................................................30

..............................................................................................................30

...............................................................................................31

Web Server Tuning Portlet Caching

MANY PAGES TUNING

DB2 Database Tuning Cache Manager Service Required Fixes

WEB CONTENT MANAGEMENT TUNING

Application Server Tuning WebSphere Portal Service Properties

Cache Manager Service

Navigation Service WCM Object Cache WCM Configuration Service JCR Text Search DB2 Tuning (Authoring Environment)

Multiplatform (LUW)

Z/OS

......................................................................................................................................41

COMPOSITE APPLICATIONS TUNING

Cache Manager Service Properties Composite Applications Best Practices

CLUSTER TUNING

.....................................................................................................................................46

......................................................................................................................... 32

..............................................................................................................................33

................................................................................................................................ 34

......................................................................................................................34

...................................................................................................................35

..............................................................................................................................35

.................................................................................................................36

..............................................................................................................37

.....................................................................................................................38

........................................................................................................................38

...............................................................................................................39

............................................................................................................................39

...................................................................................................................40

.........................................................................................................36

...................................................................................................37

...................................................................................................40

............................................................................................................43

...................................................................................................... 43

.................................................................................................44

Application Server Tuning

Dynacache Custom Properties

z/OS Dynacache Custom Property

Thread Pools

Transport Buffer Size

WMM Context Pooling Web Server Tuning Session Persistence To Database Tuning Vertical Cluster Tuning Required Fixes

OTHER PERFORMANCE TUNING OPTIONS

Improving Portal Startup Performance Managing the Retrieval of User Attributes

Identifying a Full Fetch of User Attributes

Minimum Attribute Set Use of Dynamic Content Features Real-World Network Considerations

Compress Content on the HTTP Server

Enabling Client-Side Caching

..............................................................................................................................51

.................................................................................................................46

...........................................................................................................................47

..................................................................................................................47

................................................................................................................47

......................................................................................................................... 48

.....................................................................................................................50

.................................................................................................................55

...................................................................................................... 46

..................................................................................................46

..............................................................................................49

.....................................................................................................52

..................................................................................................52

..............................................................................................53

.......................................................................................... 54

....................................................................................................... 55

.....................................................................................................56

...........................................................................................56

........................................................................................................57

WEBSPHERE PORTAL CACHES

General Information

Cache Configuration Properties Cache Usage Patterns Cache Instances

Access Control

Portal User Management

Datastore

Model

URL Mappings

Virtual Portals

WSRP

Dynamic Assembly / Process Integration

Policy

Collaboration Services

Miscellaneous Example Scenarios

General Comments

Small Number of Pages and Small Number of Users

Small Number of Pages and Large Number of Users

Portals with Long Session Timeouts

Portals with Many Pages

WEB CONTENT MANAGEMENT CACHES

................................................................................................................................ 68

....................................................................................................................................69

....................................................................................................................................75

.....................................................................................................................................78

....................................................................................................................58

........................................................................................................................58

.....................................................................................................................61

............................................................................................................................62

......................................................................................................................... 62

............................................................................................................. 67

.......................................................................................................................... 74

...........................................................................................................................74

................................................................................................................78

.......................................................................................................................... 79

......................................................................................................................... 82

....................................................................................................................82

............................................................................................................. 84

.....................................................................................................58

.......................................................................................... 77

............................................................................83

................................................................................................ 84

........................................................................................................86

WCM Cache Instances

WCM Item caching

WCM Summary

WCM Basic Caching

Advanced and Resources

Session Cache

.....................................................................................................................................88

Navigator

Absolute path

Missed Items

Library

Library Parent

Draft Summary

User cache

Appendix A. References Appendix B. Credits

................................................................................................................................ 88

....................................................................................................................................88

.................................................................................................................................91

.....................................................................................................................86

........................................................................................................................86

...................................................................................................................87

............................................................................................................87

......................................................................................................................... 87

...........................................................................................................................88

............................................................................................................................88

...........................................................................................................................89

......................................................................................................................... 89

..............................................................................................................................89

............................................................................................................................90

Figures

Figure 1 Portal Access Control Cache Hierarchy Figure 2 Portal Model Cache Hierarchy

.............................................................................................................. 70

.................................................................................................. 63

Tables

Table 1: Additional Sun JVM Settings Table 2: WebSphere Security Attribute Propagation Settings Table 3: VMM Context Pool Setting Table 4: Navigation Service Settings Table 5: Registry Service Settings Table 6: Cache Manager Service Settings Table 7: DB2 Database Domains Table 8: Oracle Database Tuning Table 9: IDS Tuning Table 10: Web Server Tuning Table 11: AIX Network Settings Table 12: Linux Network Settings Table 13: Windows Network Settings Table 14: Solaris Network Settings Table 15: z/OS System Tuning Table 16: Navigation Service Settings for Web 2.0 Theme Table 17: Reverse Proxy Settings Table 18: DB2 Database Settings for Many Pages Table 19: Cache Manager Service Settings for Many Pages Table 20: Cache Manager Service Settings for WCM Table 21: Navigation Service Settings for WCM Table 22: WCM Object Cache Settings Table 23: DB2 z/OS Bufferpool Settings Table 24: DB2 z/OS Default Bufferpool Settings Table 25: Cache Manager Serivce Properties for Application Infrastructure Table 26: Web Server Tuning for Clusters Table 27: WebSphere Session Persistence Tuning Table 28: IDS Tuning in Vertical Cluster

..................................................................................................................................... 22

.................................................................................................................. 8

.................................................................................... 10

.................................................................................................................. 11

................................................................................................................. 12

.................................................................................................................... 13

.......................................................................................................... 14

..................................................................................................................... 15

..................................................................................................................... 20

......................................................................................................................... 23

....................................................................................................................... 25

..................................................................................................................... 26

................................................................................................................ 26

................................................................................................................... 27

........................................................................................................................ 29

....................................................................................... 30

.................................................................................................................... 31

................................................................................................ 34

.................................................................................... 35

............................................................................................. 37

................................................................................................... 38

.............................................................................................................. 38

............................................................................................................. 41

................................................................................................... 42

.......................................................................................................... 48

............................................................................................... 49

............................................................................................................. 51

.................................................................. 43

ABOUT THIS DOCUMENT

This white paper provides a basis for parameter and application tuning for IBM WebSphere Portal for Multiplatform V6.1. Remember that both tuning and capacity are affected by many factors, including the workload scenario and the performance measurement environment. For tuning, the objective of this paper is not to recommend that you use the values we used when measuring our scenarios, but to make you aware of those parameters used in our configuration. When tuning your individual systems, it is important to begin with a baseline, monitor the performance metrics to determine if any parameters should be changed and, when a change is made, monitor the performance metrics to determine the effectiveness of the change.



PERFORMANCE TUNING OVERVIEW

Tuning a WebSphere Portal environment involves tuning and configuring the various systems and components of the environment. This chapter discusses some general concepts and details the specifics of the configuration used in our measurement environments. These specifics entail:

 Configuring the application server and the resources defined for that application

server

 Tuning the database(s) and database server  Tuning the directory server and its database  Tuning the web server and/or proxy server  Tuning the operating system and network  Tuning the WebSphere Portal services

When tuning your individual systems, it is important to begin with a baseline, monitor the performance metrics to determine if any parameters should be changed and, when a change is made, monitor the performance metrics to determine the effectiveness of the change.

In addition to the tuning changes we made in our measurement environments, there are some additional tuning options available which can improve performance in certain circumstances; these will be discussed in a separate section.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Environment Considerations

Before beginning your install of WebSphere Portal you should consider how to use the environment in order to achieve ideal performance. Topics to consider include:

• Choosing between 32-bit and 64-bit JVMs

• Use of hardware multithreading, also known as Simultaneous Multithreading or

Hyper-Threading.

3 2 - B I T A N D 6 4 - B I T C O N S I D E R A T I O N S

The choice of a 32-bit or 64-bit JVM involves some trade-offs. The key advantage of a 64-bit JVM is its vastly larger address space. Heap sizes of 2.5GB or larger can be practical on modern server systems. This can be a significant benefit for applications with high memory demands.

A 64-bit JVM does have disadvantages as well. Machine instructions and memory references in a 64-bit JVM are larger than in a 32-bit JVM. This means that Java objects, which typically contain multiple memory references, are larger in a 64-bit JVM than compared to a 32-bit JVM. Therefore a 64-bit JVM will need a larger heap than a 32-bit JVM for the same population of objects.

The increased size of instructions and memory references imposes a second performance penalty. They increase the demand on the memory subsystem of the system, causing more cache misses and a higher demand for memory bandwidth. As a result, executing a set of operations in a 64-bit JVM can be slower than executing the same operations in a 32-bit JVM.

When considering a deployment of WebSphere Portal 6.1, consider the memory demands your applications will have. If you expect a high demand for memory, the best performance will probably come from a 64-bit JVM. On the other hand, if the memory demand is lower, a 32-bit JVM is likely to give superior performance.

H A R D W A R E M U L T I T H R E A D I N G ( H Y P E R - T H R E A D I N G )

Many modern processor architectures support hardware multithreading. For example, this is known as Hyper-Threading (HT) on Intel processors and Simultaneous Multithreading (SMT) on Power-series processors. Our experience is that using hardware multithreading provides an improvement in capacity in all of the scenarios and platforms reported in this report, so we would recommend its use on platforms where this is an option.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE



BASE PORTAL TUNING

The Base Portal Scenario covers user login, page navigation, and interaction with simple portlets. Users can see a small set of pages, some of which are visible to all authenticated users, with access to others based on their group membership.

We have also benchmarked a number of other scenarios, which focus on different functions or use cases for WebSphere Portal. For example, there are scenarios which make use of Web Content Management (WCM), and a scenario where users have access to thousands of pages. While we have used different tuning to optimize performance for some of those scenarios, the tuning is all based on the tuning done in the Base Portal Scenario.

In all of our measurement environments, we use a separate database server and directory server, in addition to the WebSphere Portal server. We run these servers on separate systems to avoid resource contention on the system running the WebSphere Portal server. This helps improve the maximum capacity achievable.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Application Server Tuning

There are many aspects to configuring and tuning an application server in WebSphere Application Server. We found that those aspects presented here were critical to a correctly functioning and optimally performing WebSphere Portal in our laboratory environment.

For more details on tuning a WebSphere Application Server, see the Tuning Section of the information center located at:

http://www-01.ibm.com/software/webservers/appserv/was/library/

How to get to Admin Console

There are two methods to get to WebSphere Administrative Console.

• Start Server1 and use port 10001

1. In <WAS_root>/profiles/wp_profile/bin

2. ./startServer.sh server1

3. http://yourhost:10001/admin

• Start Portal and use port 10027

1. In <WAS_Root>/profile/wp_profile/bin

2. ./startServer.sh WebSphere_Portal

3. http://yourhost:10027/ibm/console

Customer ports can differ from the ports 10001 or 10027 mentioned on this page. To find out the ports in use for your installation, look for ‘adminhost’ in <wp_profile root>/config/cells/<cell_name>/nodes/<node_name>/serverindex.xml.

The following are settings based on our experience with the Base Portal workloads described above:

J V M I N I T I A L A N D M A X I M U M H E A P S I Z E

Java Virtual Machine heap size: The value of the JVM Heap size is directly related to the amount of physical memory on the system. Never set the JVM heap size larger than the physical memory on the system.

How-To Set: In the WebSphere Administrative Console: Servers  Application Servers  WebSphere Portal  Server Infrastructure: Java and Process ManagementProcess Definition  Java Virtual Machine

- Initial Heap Size

- Maximum Heap Size See JVM Max Heap Size Limits for further discussion.

See instruction on How to get to Admin Console

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

JVM MAXIMUM H E AP SIZE LIMITS

When setting the heap size for an application server, keep the following in mind:  Make sure that the system has enough physical memory for all of the processes to fit

into physical memory, plus enough for the operating system. When more memory is allocated than the physical memory in the system, paging will occur, and this can result in very poor performance.

 We set the minimum and maximum heap sizes to the same values since we’re using

the gencon garbage collection policy available in 1.5 IBM JDK which avoids heap fragmentation, this may not be the best choice if you plan to use a different garbage collection. In our measurement runs, the system is under load for a relatively short time (around 3 hours), and it is running with portlets which do not have large memory requirements. When using portlets which have larger memory requirements, or for continuous operation, it may be possible to reduce heap fragmentation by setting the initial heap size to 320 megabytes.

 After doing any tuning of heap sizes, monitor the system to make sure that paging is

not occurring. As mentioned above, paging can cause poor performance.

 32-bit operating systems have an address space limit of 4GBytes, regardless of the

amount of physical memory in the system. This space limits the maximum size of each individual process in the system. In addition, some operating systems restrict the size of processes to be even less than this limit. Many versions of Windows limit processes to 2GBytes in size; you can find more information at http://support.microsoft.com/default.aspx?scid=kb;en-us;555223.

 The address space limit further restricts the size of the JVM process. If the process

grows larger than the limit imposed by the operating system, it may terminate unexpectedly.

Due to the demands on native memory by WebSphere Portal V6.1 and its underlying components, we chose a maximum heap size of 1408MB in our Windows environments. There is a balance between JVM heap and native memory, all of which must fit within the 2GB restriction in 32-bit Windows. 1408MB was the largest value we could use to successfully measure all of our Windows configurations and workloads. If your application has additional native memory requirements then you may need to choose a smaller maximum heap size. For more information, see the WebSphere Application Server information center.

On Solaris and zLinux, we use 3.5GB heap size in 64-bit environment.

Parameter

Initial and Maximum heap size (Mbytes)

AIX

POWER5

1792 2048 3584 1408 3584 2048

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Linux Solaris

Windows

2003

z/Linux z/OS

J V M H E A P L A R G E P A G E

Large pages can reduce the CPU overhead needed to keep track of heap. With this setting we have seen 10% throughput improvement in our measurements.

This setting does improve performance on Windows, we did not set it for our measurements because Portal doesn’t start reliably when –Xlp is set, sometimes it requires a system reboot to get the jvm to start.

How-to Set: In the WebSphere Administrative Console: Servers -> Application Servers -> WebSphere Portal -> Server Infrastructure: Java and Process Management -> Process Definition -> Java Virtual Machine -> Generic JVM Argument. Add –Xlp.

Large pages are supported by systems running Linux kernels V2.6 or higher. See JVM Large Page Tuning for AIX Operation System.

JVM LARGE P AGE TUNING ON AIX OP ER A TI N G SYSTEM

To use JVM Large Page, AIX operating system must be configured to support large pages.

How-To Set:

1. We use the following steps to allocate 4GB of RAM as large pages (16MB) . We chose this amount based on having 8GB of physical memory in these systems. These values may need to be adjusted on systems with different amounts of physical memory.

vmo -r -o lgpg_regions=256 -o lgpg_size=16777216 bosboot -ad /dev/ipldevice reboot -q vmo -p -o v_pinshm=1 chuser capabilities=CAP_BYPASS_RAC_VMM,CAP_PROPAGATE $USER

2. Add: -Xlp command-line option as described above.

3. In the WebSphere Administrative Console: Servers  Application Servers  WebSphere Portal  Server Infrastructure: Java and Process ManagementProcess Definition-> Environment Entries  New  EXTSHM=OFF (note: When EXTSHM is on it prevents use of large page).

4. Restart Portal Server. To verify if large pages are being used, run the AIX command vmstat -l 1 5 and check the "alp" column which is the active large page used. It should be a non-zero value if large pages are being used.

Parameter AIX

POWER5

Linux

Solaris

Windows

2003

z/Linux z/OS

JVM Heap Large page

-Xlp -Xlp Not Applicable

Not Applicable

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

J V M H E A P N E W A R E A S I Z E

The Generational Garbage Collector introduced in Java 5.0 is efficient to Portal application JVM memory management, and it is set as default by installation with the –Xgcpolicy:gencon commandline option. Use –Xmn to further fine tune the Java heap new area (Nursery). The –Xgcpolicy:gencon option does not apply to Solaris.

How To Set: In the WebSphere Administrative Console: Servers  Application Servers  WebSphere Portal  Server Infrastructure: Java and Process ManagementProcess Definition  Java Virtual Machine -> Generic JVM Arguments:–Xmn256m

Parameter

New Area Size

A D D I T I O N A L S U N J V M A R G U M E N T S

AIX

POWER5

Linux Solaris

-Xmn320m -Xmn256m -Xmn768m -Xmn256m -Xmn1024m -Xmn320m

Windows

2003

z/Linux z/OS

On the Solaris platform, we use the following Java HotSpot parameters to achieve optimum performance.

Table 1: Additional Sun JVM Settings

Parameter Value

-server

-XX:MaxPermSize

-XX:+UseConcMarkSweepGC

-XX:SurvivorRatio

-XX:+UseParNewGC

Offers higher throughput than the "client" mode. 768m Use concurrent mark-sweep collection for the tenured

6 By default concurrent low pause collector uses the

generation. The application is paused for short periods during the collection; we found this collector works best in Portal.

default, single threaded young generation copying collector. Set this parameter to use parallel young generation collector for new area.

Additional Information

-XX:ParallelGCThreads

5 Reduces the number of garbage threads. On the Chip

multithreading processor based system, we set the threads no higher than one quarter of the hardware threads. We also distribute the threads for 6 JVMs. Our system has 128 virtual processors, we set a total of (128/4)=32 GC threads across all the JVMs. So 5 or 6 GC threads per JVM.

-XX:+PrintGCDetails

Print more details at garbage collection. This does not

improve performance, but it provides additional information related to garbage collection activity, which is useful in tuning garbage collection.

-XX:+PrintGCTimeStamps

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Print timestamps at garbage collection. See above.

S E S S I O N T I M E O U T

Session timeout: The default value of Session Timeout is 30 minutes. Reducing this

value to a lower number can help reduce memory consumption requirements, allowing a higher user load to be sustained for longer periods of time. Reducing the value too low can interfere with the user experience.

For Solaris, on a T5240 hardware, we used a much lower think time, 5 seconds, than was used for other platform hardware measurement of 12 seconds. With a lower thinktime, fewer vusers will result in a heavier load on the system. The reason we lowered the thinktime was specifically to decrease the number of vusers required for this measurement. Our pool of LoadRunner vuser licenses was inadequate to generate enough load with the higher think time. With a shorter think time than is used in the other measurements, the duration of each virtual user's interaction with the site is shorter by approximately 2 minutes. To compensate for this, and keep the sessions live on the server for the same period of time, we increased the session timeout by 2 minutes, to 12 minutes.

How To Set: In the WebSphere Administrative Console: Servers  Application Servers  WebSphere Portal  Container Settings: Web Container Settings  Session Management  Session Timeout -> Set Timeout

Parameter

Session timeout

AIX

POWER5

10 minutes 10 minutes 12 minutes 10 minutes 10 minutes 10 minutes

Linux Solaris

Windows

2003

z/Linux z/OS

W E B C O N T A I N E R T H R E A D P O O L S I Z E

Servlet engine thread pool size: Set this value and monitor the results. Increase this value if all the servlet threads are busy most of the time.

How To Set: In the WebSphere Administrative Console: Servers  Application Servers  WebSphere Portal Additional Properties: Thread Pools Web Container  Thread Pool

- Minimum size threads - Maximum size threads

Parameter

Web Container Thread pool size

AIX

POWER5

50 50 50 50 50 50

Linux Solaris

Windows

2003

z/Linux z/OS

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

S E C U R I T Y A T T R I B U T E P R O P A G A T I O N

To reduce the Security Attribute Propagation (SAP) overhead, please use a custom property 'disable Callerlist'. If SAP is not used, you can disable that, to remove the extra overhead to improve the login performance.

If Subject has not been customized, then there is no need to enable Security Attribute Propagation. Security Attribute Propagation can add extra overhead due to some extra processing that is required. However, there are certain configurations where performance might be better with security propagation enabled due to reduction of remote registry calls. See the WebSphere 6.1 InfoCenter (search for 'security attribute propagation') for a discussion of when propagating security attributes is desirable. If you want to enable SAP for functional reasons, you can improve the performance with CallerList tuning mentioned below.

These settings apply to all platforms.

How to Set: In the WebSphere Administrative Console: Security->Secure Administration, Applications, and Infrastructure -> Custom properties ->

Table 2: WebSphere Security Attribute Propagation Settings

Security Attribute Propagation

com.ibm.CSI.disablePropagationCallerList

com.ibm.CSI.rmiOutboundPropagationEnabled

com.ibm.CSI.rmiInboundPropagationEnabled

com.ibm.ws.security.webInboundPropagationEnabled

Name Value

true

false

For com.ibm.CSI.disablePropagationCallerList create a new property, for the other 3 properties, modify their value to “false”.

Note to WAS 7:

In our WAS 7 environment, we add

com.ibm.CSI.disablePropagationCallerList = true

, and use the other 3 default true attributes. For was7, this field is accessed through: Security->Global Security ->CustomProperties->New.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

V M M C O N T E X T P O O L I N G

Tune VMM Context Pooling to improve the performance of concurrent access to an LDAP server.

We changed the following Context Pooling settings line in: <wp_profile_root>/config/cells/<cellname>/wim/config/wimconfig.xml

<config:contextPool enabled="true" initPoolSize="10" maxPoolSize="0" poolTimeOut="0" poolWaitTime="3000" prefPoolSize="30"/>

You can also set them via the administrative console as described in

http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websph ere.base.doc/info/aes/ae/uwim_ldapperfsettings.html

Table 3: VMM Context Pool Setting

Context Pool Setting Default Value Value

initPoolSize 1 10 prefPoolSize

3 30

Number of open connections to maintain to LDAP server.

maxPoolSize

20 0.

A value of 0 allows the pool to grow as large as needed. If access to the LDAP server is shared by many systems, this setting may allow an excessive number of connections to the LDAP server; in such a case, set the maximum pool size to a value appropriate to your environment.

O R B S E R V I C E T U N I N G F O R Z / O S

In the WAS Admin Console, set the ORB Service to be "pass by reference" instead of "pass by value" (default) for both server1 and WebSphere_Portal

How to Set:

- Servers Application Servers server1 Orb Service

- check box for "Pass by Reference"

- Servers Application Servers WebSphere_Portal  Orb Service

- check box for "Pass by Reference"

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

WebSphere Portal Services

WebSphere Portal has a number of configurable “services”; each service has several parameters available to it. This section describes which services we tuned, the tuning values used, and the rationale for those changes.

How to Set:

1. Edit <wp_profile_root>/PortalServer/config/properties/xxxService.properties

2. uncomment the line, then change the size.

3. run <wp_profile_root>/ConfigEngine/ConfigEngine.sh update-properties

The changes should appear on WAS Console -> Resource Environment Providers -> WP_xxxService -> Custom properties

N A V I G A T O R S E R V I C E

The navigator service manages the content model for unauthenticated users, which controls the pages those users are able to see. This content model is periodically reloaded by WebSphere Portal; new pages which are visible to unauthenticated users will not be available until the next reload occurs. Our environment assumes a low rate of change for pages, so we set this reload to only occur once per hour. In a production environment where new pages for unauthenticated users are rarely created, setting this reload time to an hour or more will give better performance. In a test or staging environment where updates to unauthenticated pages need to be seen more often, a lower reload time is more appropriate.

This service also controls the HTTP cache-control headers which will be sent on unauthenticated pages. While our environment did not exploit HTTP page caching, increasing these cache lifetimes in a production environment can reduce load on the portal. For more discussion of the use of HTTP cache-control headers with WebSphere Portal, refer to the “Caching” section of the “Tuning” topic in the WebSphere Portal V6.1 InfoCenter.

Table 4: Navigation Service Settings

Parameter

public.expires (seconds)

public.reload (seconds)

remote.cache.expiration (seconds)

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

NavigatorService.properties

Default

Value

60 3600 Determines cache expiration time for caches

60 3600 Determines cache expiration time for the portal

60 28800 Determines cache expiration for caches outside

Value

Used

outside of WebSphere Portal and for unauthenticated portal pages only. If the setting remote.cache.expiration is also set to a value greater than or equal to 0, the smaller one of the two values is used.

internal cache for unauthenticated pages

of portal server for authenticated as well as for unauthenticated pages

Definition

R E G I S T R Y S E R V I C E

WebSphere Portal maintains information about many resource types in its databases. Some of these resources are replicated into memory for faster access; this is provided by the registry service. This replicated information will be periodically reloaded from the database, thus picking up any changes which may have been made on a peer node in a clustered environment.

The registry service allows configuring a reload time, in seconds, for each type of data which it is managing. In a production environment, we expect this type of information changes very infrequently, so we used very long reload times for the registry service. A full list of the types of information managed by the registry service is in table 4.

Table 5: Registry Service Settings

RegistryService.properties

Parameter

Default

Value

Used

Definition

default.interval 1800 28800 Reload frequency for any object types not

explicitly specified in the file.

bucket.application.interval 600 28800 Reload frequency for application definitions bucket.portlet.interval 600 28800 Reload frequency for portlet definitions bucket.theme.interval 3000 28800 Reload frequency for theme definitions bucket.skin.interval 3500 28800 Reload frequency for skin definitions bucket.client.interval 19000 28800 Reload frequency for client definitions bucket.markup.interval 20000 28800 Reload frequency for markup definitions bucket.transformation

application.interval

600 28800 Reload frequency for transformation

application definitions

bucket.transformation.interval 600 28800 Reload frequency for transformation

definitions

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

C A C H E M A N A G E R S E R V I C E

The cache manager service in WebSphere Portal is used to cache a wide variety of types of information in memory. These caches are somewhat similar to the registries maintained by the registry service, as each type of information gets its own cache. The key differences are:

 The information stored in the cache manager service’s caches tends to be more

dynamic than the information stored in the registry service’s registries.

 The caches used by the cache manager service are limited in size, and entries will

be discarded when the caches become full. The registries used by the registry service are not size-limited; they contain all entries of the specific data type.

 Expiry times are managed individually for each entry in the cache, managed by the

cache manager service. In contrast, when the reload time is reached for a registry, the entire contents of that registry are reloaded.

Each cache has several configurable options. A full discussion of these options, along with a list of the caches in WebSphere Portal V6.1, is given in chapter 2. Table 5 lists the changes which we made to the cache manager service configuration file. Size values are specified in “number of objects” and lifetime values are specified in “seconds”.

Table 6: Cache Manager Service Settings

CacheManagerService.properties

Cache Name

Default Value Value Used

com.ibm.wps.model.factory.ContentModelCache.live.size 1000 2500 com.ibm.wps.ac.ExplicitEntitlements Cache.USER_GROUP.size 1000 2000 com.ibm.wps.model.factory.Navigation

1000 2500

SelectionModelCache.live.size com.ibm.wps.ac.OwnedResourcesCache.enabled true false com.ibm.wps.ac.ProtectedResourceCache.lifetime 5000 14400 com.ibm.wps.datastore.services.Identification.SerializedOidString

2500 5000

Cache.size com.ibm.wps.puma.DN_OID_Cache.size 500 5000 com.ibm.wps.puma.DN_User_Cache.size 500 3000 com.ibm.wps.puma.DN_Group_Cache.size 500 1500 com.ibm.wps.puma.OID_DN_Cache.size 1500 3000 com.ibm.wps.puma.OID_User_Cache.size 1500 3000 com.ibm.wps.puma.OID_Group_Cache.size 1500 5000 com.ibm.wps.ac.groupmanagement.NestedGroupCache.enabled true False com.ibm.wps.ac.RolesCache.enabled true False com.ibm.wps.ac.ChildResourcesCache.lifetime 7200 28800 com.ibm.wps.policy.services.PolicyCacheManager.lifetime 7780 43200

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Database Tuning

D A T A S O U R C E T U N I N G F O R D B 2

Multiple databases are used to hold information in WebSphere Portal V6.1. We used six separate DB2 databases, each representing a separate database domain and having their own datasources. These are:

Table 7: DB2 Database Domains

Database Database name Datasource name

Release Community Customization Feedback Likeminds JCR

All datasources are configured in a similar manner by logging on to the WebSphere Application Server administrative console. For the prepared statement cache size, the path is Resources → JDBC Providers → provider name → Data Sources → datasource name. The provider name and datasource name are based on the names selected for that database during the database transfer step. Look for the parameter Statement cache size.

release reldbDS community commdbDS custom cusdbDS fdbkdb fdbkdbDS Lmdb lmdbDS jcrdb jcrdbDS

For the connection pool settings, the path in the WebSphere Application Server administrative console is Resources → JDBC Providers → Provider name → Data Sources → Datasource name → Connection Pools. The settings are Minimum connections and Maximum connections.

The default settings were used for the prepared statement cache size, and connection pool minimum and maximum sizes.

D B 2 D A T A B A S E S E R V E R T U N I N G

WebSphere Portal V6.1 uses database servers for core functionality. In our measurement environment, we used DB2 database server for the Portal application. The LDAP server, IBM Tivoli Directory Server also included a DB2 database as a repository, but it is largely unseen and was operated as an out of box configuration.

We recommend using a remote database server for the largest capacity. For our measurements we used IBM DB2 Enterprise Edition V9.1 fixpack 5 as our database server. WebSphere Portal V6.1 uses the concept of Database domains to designate either groups of tables belonging to one domain, or even entirely separate databases to store the data specific to each domain.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

We built six separate databases within one database server to house the tables and data needed to support each domain. For the Base Portal and Many Pages measurements, the Release domain is the primary database being exercised.

The databases and related domains supported by Portal V6.1 are:

1. Release (release domain). This is the primary database domain used by the Base Portal and Many Pages Scenarios.

2. Customization (customization domain). This database receives some light traffic in our scenarios.

3. Community (community domain). This database receives some light traffic in our scenarios.

4. JCR (JCR domain). JCR database is used heavily in WCM (Web Content Management) Scenario. This database receives light traffic in all other scenarios measured in our Benchmark report.

5. Likeminds database, used for Likeminds enabled systems. This database is not used in the scenarios measured in our Benchmark report.

6. Feedback database, used by the feedback subsystem. This database is not used in the scenarios measured in this report.

DB2 ON AIX S E T U P

We configure our DB2 database on AIX using the following setup,

• Set the filesystem which will hold the Portal databases to be a Enhanced Journal File System (JFS2) because a large file system is limited to 64GB.

• Turn on concurrent I/O (CIO) for Enhanced Journal File System as this improves performance.

To enable CIO, use the following command to mount the database fileset. Mount –o cio /portaldb

• Increase AIX maximum number of processes per user to 4096.

The default 500 processes per user is too low for database server, we increase

it to 4096 in our AIX environment. To increase it,

chdev –l sys0 –a maxuproc=’4096’

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

While the Portal databases are configured for high capacity performance, various tuning

adjustments may be necessary from time to time. Typically these tuning needs are based on the volume of database traffic and the size of table populations.

Our database tuning settings is documented in the Portal Info Center under ‘Creating

Remote Database’ section.

DB2 ON Z/OS S E T U P

After transferring the database tables, first Identify what tables need to be reorganized. Perform a re-org check to improve performance.

• Run the EJPSDBTC job after database transfer

This job contains the DB2

check and RUNSTATS utility for the JCR, Likemind and Feedback database.

• For details on re-org DB2 database, visit WebSphere Portal Info Center.

Create a Re-org job to re-org all table spaces in WPSDBJCR database.

RECOMMENDED D AT AB ASE MAIN TEN A N C E FOR DB2 LUW

Two of the database attributes, which DB2 relies upon to perform optimally, are the database catalog statistics and the physical organization of the data in the tables. Catalog statistics should be recomputed periodically during the life of the database, particularly after periods of heavy data modifications (inserts, updates, and deletes) such as a population phase. Due to the heavy contention of computing these statistics, we recommend performing this maintenance during off hours, periods of low demand, or when the portal is off-line. The DB2 runstats command is used to count and record the statistical details about tables, indexes and columns. We have used two techniques in our environment to recompute these statistics. The form we recommend is:

db2 runstats on table tableschema.tablename on all columns with distribution on all columns and sampled detailed indexes all allow write access

These options allow the optimizer to determine optimal access plans for complex SQL. A simpler, more convenient technique for recomputing catalog statistics is:

db2 reorgchk update statistics on table all

Not only does this command count and record some of the same catalog statistics, it also produces a report that can be reviewed to identify table organization issues. However, we have found instances where this produces insufficient information for the

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

optimizer to select an efficient access plan for complex SQL, particularly for queries of the JCR database.

We have determined a technique that has the same convenience of the reorgchk command and provides the detailed statistics preferred by the optimizer.

db2 -x -r "runstats.db2" "select rtrim(concat('runstats on table ',concat(rtrim(tabSchema),concat('.',concat(rtrim(tabname),' on all columns with distribution on all columns and sampled detailed indexes all allow write access'))))) from syscat.tables where type='T'" db2 -v -f "runstats.db2"

The first command is used to create a file, runstats.db2, which contains all of the runstats commands for all of the tables. The second command uses the db2 command processor to run these commands.

To determine which tables might benefit from reorganization, we use the command:

db2 reorgchk current statistics on table all > "reorgchk.txt"

For those tables which require reorganization, we use the command:

db2 reorg table tableschema.tablename

to reorganize the table based upon its primary key. You should also ensure that your database servers have adequate numbers of disks.

Multiple disks allow for better throughput by the database engine. Throughput is also improved by separating the database logs onto separate physical devices from the database.

You should ensure that the database parameter MaxAppls is greater than the total number of connections for both the datasource and the session manager for each WebSphere Portal application server instance. If MaxAppls is not large enough, you will see exceptions in your connection pools.

You should use System Managed Storage (SMS) for temporary table spaces to benefit complex SQL which require temporary tables to compute their result sets. This saves time in buffer writes and improves disk utilization.

Database performance is very important for obtaining good performance from WebSphere Portal. The maintenance tasks and practices mentioned here were found to be critical to the performance and correct operation of WebSphere Portal in our lab environment. Additional database maintenance and tuning may be needed in your production environments. For further information on DB2 administration and tuning, refer to the DB2 Information Center.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

O R A C L E D A T A B A S E S E R V E R T U N I N G

WebSphere Portal V6.1 uses database servers for core functionality. In this measurement environment, we used Oracle database server for the Portal application. The LDAP server, IBM Tivoli Directory Server included a DB2 database as a repository.

PLANNING FOR O R AC L E ENTERPRISE E DI TI O N DATABASE

For our Solaris platform measurements we also used Oracle 10g R2 as our database server. WebSphere Portal V6.1 uses the concept of Database domains to designate either groups of tables belonging to one domain, or even entirely separate databases to store the data specific to each domain.

On Oracle, we built a single database and create Oracle users to own the tables and data needed to support each domain. The domains are listed in PortalDatabaseDomain, above. For the Base Portal measurements, the Release domain is the primary database being exercised.

A well designed database can save a lot of trouble later down the road, and improve database performance. We recommend that you refer to the Oracle Administrator’s Guide to help you make informed database design decisions. Here are the key choices we have implemented in our Oracle database.

• To avoid I/O contention and allow for better throughput, you should ensure your database server have adequate number of disks. Our database is on seven stripped disks.

• For better management and performance of storage structures, Oracle-Managed Files are used for database, as well as redo logs, and control files.

• Database block size: 8k

• The following tablespace sizing was required to support roughly a medium sized

Portal, with 100,000 authenticated users, approximately 180 installed portlets and 220 pages, which the load generally consisting of database read operations. We recommend monitoring your tablespace sizing and growth on a regular basis. We used DBCA to create database with the following Tablespace size:

o SYSAUX: 800MB o SYSTEM: 800MB o TEMP: 800MB o UNDOTBS: 1024MB o USERS: 2048MB

• Redo log groups: 500MB each.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

ORACLE O N AI X S ET U P

We configure our Oracle database on AIX using the following setup,

• Set the filesystem which will hold the Portal databases to be a Enhanced Journal File System (JFS2).

• Turn on concurrent I/O (CIO) for database filesystem as this improves performance. Do not enable CIO for Oracle product filesystem, ie, /u01, as Oracle could fail to start.

To enable CIO, use the following command to mount the database fileset.

Mount –o cio /u02

• Increase AIX maximum number of processes per user to 4096.

The default 500 processes per user is too low for database server, we increase

it to 4096 in our AIX environment. To increase it,

chdev –l sys0 –a maxuproc=’4096’

• Enable AIX async I/O, and increase MinServer to 5.

smitty aio  Change/Show Characteristics of Async I/O  MinServers = 5

• We also set in oracle user’s profile as Oracle Installation Guide for AIX recommends,

AIXTHREAD_SCOPE=S

ORACLE ENTERP R I S E EDITION DATABASE P AR AM ETER TUNI NG

Database performance is very important for obtaining good performance from WebSphere Portal. Below is a list of tuning applied on our Oracle database server with the alter system command. Additional database tuning maybe needed in your production environments. For further information on Oracle database tuning, refer to Oracle Performance Tuning Guide at

http://www.oracle.com/technology/documentation/database10g.html

Command used:

Table 8: Oracle Database Tuning

Alter system set <parameter> scope=spfile;

Parameter Value

sessions sga_target pga_aggregate_target processes open_cursors db_files

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

900 1813M 604M 750 1500 1024

RECOMMENDED O R A C LE DATABASE MAINT E N AN CE

Optimizer statistics are a collection of data about the database and the objects in the database, these statistics are used by the query optimizer to choose the best execution plan for each SQL statement. Because the objects in a database can be constantly changing, statistics must be regularly updated so that they accurately describe these database objects, particularly after periods of heavy data modifications (inserts, updates, and deletes) such as a population phase. We have used the following commands in our environment to recompute these statistics:

execute dbms_stats.gather_database_stats(dbms_stats.auto_sample_size, method_opt=>'FOR ALL INDEXED COLUMNS SIZE AUTO',cascade=>TRUE);

O T H E R D A T A B A S E C O N S I D E R A T I O N S

WebSphere Portal maintains some information about users in its database tables, which grow when a user first logs in. We were interested in the steady-state performance of WebSphere Portal, not the performance of a user’s first login to the site. Therefore our performance evaluates after all users logged in at least one time.

One of the most important database tuning factors is bufferpool sizing. It is important to evaluate the database's physical versus logical reads and size the bufferpools to achieve as much as a 95% logical read rate if possible.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Directory Server Tuning

Our measurements used IBM Tivoli Directory Server versions 6.0 as the directory server. These products use a database for storing user information; DB2 Enterprise Server was used for this database in our environment. This database is typically located on the same system as the directory server. If your workload involves creating, updating, or deleting users, then database maintenance described above may be needed on this database.

The following table shows the tuning values used for the directory servers in our Solaris Base Portal Scenario measurements

How-to-Set: These values are in the file /opt/IBM/ldap/V6.0/etc/SchemaV6.0/ibmslapd.conf. You must restart the LDAP server after changing these values.

Table 9: IDS Tuning

Parameter Value

Ibm-slapdACLCacheSize Ibm-slapdEntryCacheSize Ibm-slapdFilterCacheSize Ibm-slapdFilterCacheBypassLimit

250000 250000 250000 7500

The IBM Tivoli Directory Server uses IBM DB2 as the database server. The database instance and alias are named IDSLDAP. We applied the following tuning to this database:

db2 “update db config for idsldap using dbheap 4800” db2 “update db config for idsldap using num_ioservers 10” db2 “update db config for idsldap using num_iocleaners 5” db2 alter bufferpool LDAPBP size 3690 db2 alter bufferpool IBMDEFAULTBP size 88500

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Web Server Tuning

We used IBM HTTP Server 6.1 in our measurement environment. The cluster configuration and the Solaris configuration has a remote web server, find the tuning in Web Server Tuning in Cluster Tuning section. All other configurations have the web server running on the same system as the WebSphere Portal application server. If, during your monitoring, you notice insufficient processor capacity on the system when running the web server and the portal application server on a single system, consider separating the servers onto different systems. We used the following tuning on our web servers:

Table 10: Web Server Tuning

Parameter AIX

POWER5

KeepAliveTimeout

ThreadsPerChild

MaxKeepAliveRequests

5 5 5 5 This value is less than the think

25 25 2000 25 The higher number of threads per

0 0 0 0 Selecting 0 lets an unlimited

Linux

Windows

2003

z/Linux

Additional Information

time defined in our scripts to ensure that testing is conservative. Each user is assumed to open a new TCP connection for each page view. However, in a live environment, it can be helpful to increase the KeepAlive Timeout. A higher timeout value can increase contention for HTTP server processes, if you are running out of HTTP processes, decrease this value.

child on Windows is due to a different process model for IHS on Windows.

number of requests on a single TCP connection.

MaxRequestsPerChild StartServers Access logging

0 0 0 0 2 2 N/A 2 off off off off This was turned off by

commenting out the following configuration line: CustomLog /usr/HTTPServer/logs/access_log common

ThreadLimit ServerLimit

25 25 2000 25 150 120 N/A 180 Set it

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

MaxClient/ThreadsPerChild.

MinSpareThreads MaxSpareThreads MaxClients

25 25 N/A 25 3750 4500 N/A 4500 Set it same as MaxClients. 3750 4500 N/A 4500

We also enabled the server-status module so that we could monitor the number of running and available Web server processes. This enables appropriate tuning of the MaxClients and ThreadsPerChild parameters.

We did additional Web Server tuning in Web 2.0 Scenario. See Web 2.0 section for details.

Note: For z/OS, no Web Server was configured.

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

Operating System Tuning

In any high-load environment, the network must be closely monitored to ensure that its performance is acceptable and consistent. Note that, the following is not to suggest that all network parameters are set to these values, but merely make the reader aware that the network is also an entity in the performance environment and bottleneck resolution process.

A I X

NETWORK T UNI N G

Use smitty->Performance and Resource Scheduling->Tuning Kernel and Network Parameters->Tuning Network Option Parameters->Change/Show Current Parameters to change. These will take effect immediately, improving the network layer performance in high volume environments.

Then remember to ‘Save current parameters for Next Boot’.

Table 11: AIX Network Settings

Parameter Value

tcp_sendspace tcp_recvspace udp_sendspace udp_recvspace somaxconn tcp_nodelayack rfc1323

131072 131072 65536 655360 10000 1 1

WEBS PHE RE POR TAL V6 .1 TUNING GU IDE

+ 67 hidden pages