HP Red Hat Directory White Paper

Red Hat Directory Server 7.1
for HP-UX 11i v2 Integrity
system Performance Tuning

and
Sizing Guidelines

May 23 , 2008

U.S.A.

©2006-2008 Hewlett-Packard Development Company,

L.P.

Always On

Anything

Anytime

Anywhere

Red Hat Directory Server 7.1 Performance Tuning and Sizning Guidelines

History

Version Date Scope / Changes

1.0 8/21/2006

1.1 9/1/2006

1.2 9/7/2006

1.3 5/23/2008

• Initial draft

• First Review Update

• Second Review Update

• New data points Update

i

History...........................................................................................................................................................................i

Overview.......................................................................................................................................................................1

Product Overview....................................................................................................................................................1

Purpose of This Document......................................................................................................................................1

Sizing and Tuning Overview...................................................................................................................................1

Hardware...............................................................................................................................................................1

Operating System and System Parameter Tuning .................................................................................................2

Directory Sever Tuning Overview.........................................................................................................................2

Sizing and Tuning Recommendations........................................................................................................................3

Sizing Guidelines......................................................................................................................................................3

Systems..................................................................................................................................................................3

CPUs......................................................................................................................................................................3

Memory .................................................................................................................................................................4

Tuning Recommendations......................................................................................................................................5

nsslapd-threadnumber............................................................................................................................................5

nsslapd-dbcachesize...............................................................................................................................................6

nsslapd-cachememsize and nsslapd-cachesize ......................................................................................................8

nsslapd-cache-autosize ........................................................................................................................................10

Logging ...............................................................................................................................................................11

Performance Measurements.....................................................................................................................................11

Purpose...................................................................................................................................................................11

Test Result..............................................................................................................................................................11

Data collection 1: (Different Number of CPUs)..................................................................................................11

Data Collection 2: (Different nsslapd-threadnumber).........................................................................................12

Data Collection 3: (Different nsslapd-dbcachesize) ............................................................................................12

Data collection 4: (Different entrycache setting).................................................................................................13

Data Collection 5: (Access Log On vs. Access Log Off)....................................................................................14

Data Collection 6: (SSL Connection Enabled vs. SSL Connection Disabled) ....................................................14

Additional Tuning Reference....................................................................................................................................15

Appendix A: RHDS 7.1 Performance Test Details.................................................................................................15

Test Environment..................................................................................................................................................15

Operating System ................................................................................................................................................15

System Parameter Tuning....................................................................................................................................15

General Directory Server Configuration..............................................................................................................15

Selected Benchmark............................................................................................................................................16

Test Machines......................................................................................................................................................16

Private LAN Configuration .................................................................................................................................16

Performance Monitoring Tools............................................................................................................................16

vsar(Visual System Activity Reporter)................................................................................................................16

Caliper .................................................................................................................................................................17

Performance Data Creation..................................................................................................................................17

Test Data Generation...........................................................................................................................................17

Search Script Generation.....................................................................................................................................17

Appendix B: my_scriptgen.c.....................................................................................................................................17

Appendix C: Test Environment for new data points..............................................................................................18

Operating System ................................................................................................................................................18

Test Machines......................................................................................................................................................18

ii

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Overview

Product Overview

Red Hat Directory Server for HP-UX provides an industry standard centralized directory service to build your
intranet or extranet on. Your directory-enabled applications use the directory service as a common, network­accessible location for storing shared data such as user and group identification, server identification, and access
control information. In addition, you can extend the Red Hat Directory Server to support your entire enterprise with
a global directory service that provides you with centralized management of all your enterprise's resource
information.

Purpose of This Document

This document provides basic sizing and performance tuning guidelines for Red Hat Directory Sever (RHDS)
version 7.1 for HP-UX 11i v2 on Integrity Server. It also provides the performance test results of RHDS 7.1 on HP­UX Integrity (IA64) Servers. The RHDS 7.1 is a 64-bit Directory Server.

The data provided here is intended to help system and network administrators to effectively size and tune different
directory server configurations. Some major topics such as class of machines, amount of memory, and number of
CPUs are covered. This document also covers how to tune some of the key performance related attributes for
RHDS.

Please keep in mind that data presented in this document are measured under controlled environment. Testing for
these guidelines were performed on dedicated HP-UX servers connected by private LAN. No other system activities
were running during the performance testing. For RHDS version 7.1 performance test details, see Appendix A.

Sizing and Tuning Overview

Hardware

The following sizing guidelines should be used when selecting a system to use as a Red Hat Directory Server. They
are discussed in more detail in Section “Sizing Guidelines

• Any HP IA64 architecture with an Intel

be used for Red Hat Directory Server.

• When correctly tuned, the eventual performance bottleneck will be CPU.

• At least 256 MB of memory is required. However, you should plan from 512 MB to 4 GB or more of

RAM for best performance on large directories. As an example, if your directory contains 250K entries
where the average entry size in entrycache is 3860 bytes, and if your directory server only needs to support
exact search requests on the “cn” attribute, you might want to have at least 2.5 GB of memory available for
the directory server to cache all the information needed.

• Approximately 251 MB of disk space for a minimal installation (without loading any user data) is required.

For production systems, you should plan at least 2 GB to support the product binaries, databases, and log
files (log files require 1 GB disk space by default). As an example, for a directory instance with 250K
entries and average entry size on disk is 700 bytes, it requires the minimum of 0.9 GB disk space for
database, default index files and minimum logging (access log is turned off).

® Itanium 2 processor or Intel® Itanium 2 dual core processor can

”.

Page 1

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Operating System and System Parameter Tuning

• PHKL_34032 and PHCO_33675 are installed. These two patches will improve the HP-UX 11i v2 pthread

performance in general.

• Two pthread tuning environment variables PERF_ENABLE and PTHREAD_FORCE_SCOPE_SYSTEM

are set to 1 to force the 1x1 thread model. The default thread models are different on different HP-UX
releases. At the first release of HP-UX 11i v2, it is defaulted to MxN threads; starting from HP-UX 11i v2
September 2004 release and on, it is defaulted to bound threads in an MxN environment. It has been
observed that the 1x1 threads performs better than MxN threads does in general. Our testing shows 1x1
threads gave best performance number for RHDS 7.1.

• Set malloc tuning environment variable _M_CACHE_OPTS=1024:32:0 to turn the thread local cache on.

NOTE: You need to have enough memory to use this tuning variable; otherwise, the server may exit when
it runs out of memory. 10% of the nsslapd-cachememsize may be a good guess for the additional
memory requirement, but test it before using it. By default, the thread local cache is disabled.

• Set maxfiles_lim, the hard file limit per process, to at least 4096.

• Set max_thread_proc, the maximum number of threads per process, to 256.

• Set maxdsize_64bit, the maximum of data-segment s ize for a 64-bit process, to at least the size of

nsslapd-cachememsize * 2.3 + nsslapd-dbcachesize. More information about
nsslapd-cachememsize and nsslapd-dbcachesize will be provided in Section “Tuning

Recommendations

”.

Directory Sever Tuning Overview

The following tuning recommendations apply to Red Hat Directory Server version 7.1 for optimal performance:

• Tune the cache size big enough to fully utilize the cache. There are two caches available: database cache

and entry cache. Detailed information is provided in Section “Tuning Recommendations

• Turn the access log off if you don’t need it. Access log contains information for all the operations. By

default, the access log is turned on.

• Tune the number of operation threads down for directory servers who mainly serve search requests.

Although, fewer threads may gave you better search performance, it may also become the bottleneck when
the directory sever also serves some time-consuming operations such as add or modify. For detailed
information, please see Section “Tuning Recommendations

”.

• Utilize indexes to speed up searches. However RHDS provides an attribute, nsslapd-

idlistscanlimit, to limit the number of IDs that are scanned per index key during a search

operation. When the size of an individual ID list reaches this limit, the server will behave as if no index was
available for that type of search. For more information about nsslapd-idlistscanlimit, please
refer to “Red Hat Directory Server 7.1 Adm i ni st rat or’s Gui de

• Disable unneeded plugins such as referential integrity,

” *

UID uniqueness or schema checking, etc.

Other factors, such as if SSL, replication, or referral is configured, may also affect the performance. Complex or
many ACIs (directory server access control instructions) will also affect performance. SSL performance data is also
covered in Data Collection 6.

”

*

Hewlett-Packard Company, “Red Hat Directory Server Version7.1 Administrator’s Guide”,

http://docs.hp.com/en/7118/ds71admin.pdf.

Page 2

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Sizing and Tuning Recommendations

Sizing Guidelines

Systems

HP Integrity (IA64) Servers

Any HP IA64 architecture with an Intel® Itanium2 Processor or an Intel® Itanium2 dual core Processor and
supports HP-UX 11i v2 (Preferably HP-UX 11i v2 September 2004 or later release) can be utilized as an Red Hat
Directory Server.

For producing this report, a Montecito-based HP Integrity Server is used. A partition of 2 cells with 4 dual-core
processors each is configured within the superdome server SD64B. Within the partition, only one cell is turned on.
The exact search throughput with this configuration (8 CPUs @ 1.6GHz, 64GB memory, and one 18 GB disk) can
reach 6241.56 operations per second without tuning any Directory Server parameters. All the data generated in this
document is based on this specific hardware configuration.

CPUs

The Red Hat Directory Server for HP-UX will utilize multiple processors. When number of processors increases,
the performance gets better (see graph below). Under the performance test environment, changing the test machine
configuration from 1 processor core (2 CPUs) to 4 processor cores (8 CPUs) can increase the performance
throughput about 155%. When configured correctly, the Red Hat Directory Server will generally reach the CPU
limit before it reaches other constrains such as disk or networking I/O. For performance measurement based on
different number of CPUs, please see Table 1

Page 3

14000

12000

10000

operation s / sec

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Performance with different # of CPUs

8000

6 threads
8 threads

6000

4000

2000

0

2468

number of cpus

Figure 1: Performance difference based on different number of CPUs. Measured on Montecito-based test
configuration @1.6GHz /CPUs.

Memory

The Red Hat Directory Server for HP-UX caches entry and indexing information in memory. It requires at least 256
MB of memory for a small deployment, but for large deployments, 512MB to 4GB or more RAM is needed for best
performance. To estimate how much RAM is needed for Directory Server on a HP-UX Integrity system, please use
the following formula:

Total_RHDS_RAM = 1.2 * (base_RAM_need_for_slapd_process + caches)

Where

base_RAM_needed_for_slapd_process = 27 MB + nsslapd-threadnumber * 0.25MB

caches = dbcache + SUM(all entry caches)* 2.3 + import_cache

Explanation:

• 1.2: 20% additional RAM needed for slapd process to handle incoming LDAP operations. 20% is an

estimated number, and it should be sufficient. However, testing is needed to ensure that it is enough before
going into production.

• 27MB: is the size of the slapd process.

• nsslapd-threadnumber *0.25 MB: each thread needs about 0.25 MB of memory.

• dbcache: specified as nsslapd-dbcachesize.

• All entry caches: specified as nsslapd-cachememsize. The 2.3 factor only needs to apply to the

nsslapd-cachememsize of the databases you are doing the work with.

Page 4

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

• Import cache: specified as nsslapd-import-cachesize. If you do not do online import, then

memory allocated for the dbcache can be used for import cache. There is no need to allocate memory for
the import cache in addition to the dbcache.

Tuning Recommendations

This section discusses how to tune some of the very important RHDS attributes.

nsslapd-threadnumber

Here is some information about nsslapd-threadnumber from document: Red Hat Directory Server 7.1

Configuration, Command, and File Reference

• nsslapd-threadnumber

Defines the number of operation threads that the Directory Server will
create during startup. The nsslapd-threadnumber value should be
increased if you have many directory clients performing time-consuming
operations such as add or modify, as this ensures that there are other
threads available for servicing short-lived operations such as simple
searches. This attribute is not available from the server console.

Entry DN: cn=config
Valid Range: 1 to the number of threads supported by your system
Default Value: 30
Syntax: Integer
Example: nsslapd-threadnumber: 60

One common misconception is that more threads are better. When ther e are too many threads configured, thread
contention may affect the performance. As you can see from the graph below, after certain point, as the number of
threads increases, the search throughput decreases in this particular controlled test. In this case, the best
performance occurs when nsslapd-threadnumber is set to 6 or 8 (on a Montecito-based test bed with 8
CPUs). Based on the performance test experiment, the best exact search performance occurs only when nsslapd-
threadnumber is tuned close to the number of CPUs. This performance characteristic is verified with up to 8
CPUs on an Montecito-based test bed.

When tuning this parameter, consider this:

• if the directory only serves non-SSL based search requests, tune nsslapd-threadnmber starting from

where it equals number of CPUs.

• if the directory needs to handle time-consum i ng o perat i o ns whi ch require threads to be blocked for a long

time, such as SSL based searches, tune up nsslapd-threadnumber.

• if there are large numbers of clients concurrently requesting connections, tune up nsslapd-

threadnumber.

• if you experience low CPU utilization under a heavy load, or slow response time, try to tune up nsslapd-

threadnumber and see if performance improves.

†

†

Hewlett-Packard Company, “Red Hat Directory Server 7.1 Configuration, Command, and File Reference”.

http://docs.hp.com/en/7119/ds71cli.pdf

Page 5

14000

12000

10000

operation s / sec

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Performance with different number of worker threads

8000

8 cpus

6000

4000

2000

0

0 10 20 30 40 50 60 70 80 90 100 110 120 130

number of threads

Figure 2: Performance with different nsslapd-threadnumber. Measured on specific Montecito-based test
configuration with 8 CPUs@1.6GHz .

nsslapd-dbcachesize

Here is some information about nsslapd-dbcachesize from: “Red Hat Directory Server 7.1 Configuration,

Command, and File Reference” †

• nsslapd-dbcachesize

This performance tuning related attribute specifies database cache
size. Note that this is neither the index cache nor the entry cache. If
you activate automatic cache resizing, you override this attribute, by
replacing these values with its own guessed values at a later stage of
the server startup.

Entry DN: cn=config,cn=ldbm database,cn=plugins,cn=config
Valid Range: 500KB to 4GB for 32-bit platforms and 500KB to 2^64-1
for 64-bit platforms
Default Value: 10,000,000 bytes
Syntax: Integer
Example: nsslapd-dbcachesize: 10,000,000

The database cache is used by the database to create and manage indexes and perform other database specific work.
nsslapd-dbcache is set globally for the server and is shared by all the database backends on the server.

Tuning dbcachesize can be done by follow these steps:

Step 1: set nsslapd-dbcachesize to some value and then start the directory server.

Page 6

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Step 2: Prime the directory by sending the following ldapsearch command (label it as ldapsearch command 1):

#./ldapsearch –b “<root suffix>” –h <host> -p <port> -D

“cn=directory manager” –w passwd “objectclass=*” > /dev/null

Step 3: check db related monitoring attributes by sending a different ldapsearch request. (label it as ldapsearch
command 2):

# ./ldapsearch -p PORT -b "cn=monitor, cn=ldbm database, cn=plugins,

cn=config" –D "cn=directory manager" -w PASSWD “objectclass=*" |grep dbcache

dbcachehits: 236
dbcachetries: 280
dbcachehitratio: 84
dbcachepagein: 44
dbcachepageout: 0
dbcacheroevict: 0
dbcacherwevict: 0

• Step 4: If dbcachepageout, dbcacheroevict and dbcacherwevict are not equal to zero, you

might want to increase nsslapd-dbcachesize, and repeat from Step 2 until these three attributes
equal to zero or dbcachesize reaches the maximum value or dbcachepagein stops increasing. When
nsslapd-dbcachesize is not big enough, pages need to be discarded
for new pages. This is indicated by attributes dbcachepageout, dbcacheroevict and
dbcacherwevict. For more information about these attributes, please refer to

from the dbcache to make room

and “Red Hat Directory

Server 7.1 Configuration, Command, and File Reference Guide”

To estimate the amount of RAM needed for an optimized database cache, you can use the following formula:

Equation 1: Estimate Size of Database Cache

dbcachesize = SUM(allDB4files)

• Please note: Equation 1 only gives you a very rough estimation for dbcachesize. When estimating database

cache size, only the database (db4) files that your operations need should be included. As an example, if
your directory server only needs to support ex act search requests on the “cn” attribute, you may need just
354MB dbcache instead of 685MB dbcache which is the sum of all database (db4) files for the 250K
databases. For more information about how to manage indexes, please refer to “Red Hat Directory Server

7.1 Administrator’s Guide”

Page 7

14000

13500

13000

12500

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Performance with different size of dbcache

12000

operations / sec

11500

11000

10500

10000

0 50 100 150 200 250 300 350 400 450

dbcache in 1000000 bytes

250k,8cpus

Figure 3: Performance with different nsslapd-dbcachesize. Measured on specific Montecito-based test
configuration with 8 CPUs@1.6GHz.

nsslapd-cachememsize and nsslapd-cachesize

Here is some information about nsslapd-cachememsize and nsslapd-cachesize: “Red Hat Directory

Server 7.1 Configuration, Command, and File Reference”

• nsslapd-cachememsize:

This performance tuning related attribute specifies the cache size in
terms of available memory space. Limiting cachesize in terms of memory
occupied is the simplest method. By activating automatic cache resizing
you override this attribute, replacing these values with its own
guessed values at a later stage of the server startup. If you attempt
to set a value that is not a number or is too big for a 32-bit signed
integer you will receive an LDAP_UNWILLING_TO_PERFORM error message
with additional error information explaining the problem.

Entry DN: cn=Netscaperoot,cn=ldbm database,cn=plugins,cn=config
or cn=UserRoot,cn=ldbm database,cn=plugins,cn=config
Valid Range: 500Kbyte to 4Gbyte for 32-bit platforms and 500Kbyte to
2^64 -1 for 64-bit platforms
Default Value: 10 485 760 (10Mb)
Syntax: Integer
Example: nsslapd-cachememsize:10Mb

• nsslapd-cachesize:

Page 8

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

This performance tuning related attribute specifies the cache size in
terms of the entries it can hold. However, it is worth noting that it
is simpler to limit by memory size only (see nsslapd-cachememsize
attribute). If you attempt to set a value that is not a number or is
too big for a 32-bit signed integer you will receive an

LDAP_UNWILLING_TO_PERFORM error message with additional error
information explaining the problem.

Entry DN: cn=Netscaperoot,cn=ldbm database,cn=plugins,cn=config
or cn=UserRoot,cn=ldbm database,cn=plugins,cn=config
Valid Range: 1 to 2,147,483,647 (or -1 which means limitless) entries
Default Value: -1
Syntax: Integer
Example: nsslapd-cachesize: -1

When the server receives a search request, it will add entries to the entry cache. These two attributes control how
big the entry cache can grow. nsslapd-cachesize specifies how many entries can be cached in the entry
cache. nsslapd-cachememsize specifies the total memory space the entry cache may consume. When the
server reaches the limit specified by either cachesize or cachememsize, whichever comes first, it will remove the
least recently used entries from the entry cache to make room for new entries. For optimum search performance, all
directory entries should be held in the entry cache.

Tuning nsslapd-cachememsize (assume nsslapd-cachesize is set to –1) is very straightforward:

Step 1: set nsslapd-cachememsize to some guessed value.
Step 2: prime the server by executing ldapsearch command 1.
Step 3: execute an ldapsearch command (label it as command 3) to find out some entrycache related attributes:

#./ldapsearch –p PORT –b “cn=monitor,cn=DATABASE_ROOT,cn=ldbm database,

cn=plugins, cn=config” –D “cn=directory manager” –w PASSWORD “objectclass=*”
|grep entrycache

entrycachehits: 7
entrycachetries: 1000015
entrycachehitratio: 0
currententrycachesize: 1073739367
maxentrycachesize: 1073741823
currententrycachecount: 398030
maxentrycachecount: -1

Step 4: If the currententrycachecount is less than total number of entries in your database, and the

maxentrycachesize hasn’t reached the maximum value that is available from your system
yet, increase nsslapd-cacchememsize, and repeat from Step2 until either all the entries are cached or
nsslapd-cachememsize has reached the maximum value from your system. If
nsslapd-cachememsize hasn’t reached the maximum system value when all the entries are cached,

then set nsslapd-cachememsize to be what you got for currententrycachesize from step 3. Allow
some safety margin if possible.

To estimate nsslapd-cachememsize for your database, you can use the following equatio n derived based on
the performance testing. Please note that even though it turns out to be very accurate when tuning cachememsize for
different size of databases in the performance test, you should always test it out before going in to production.

Equation 2: Estimate Size of Entry Cache from Performance Testing:
Entrycache = total_number_of_entries_in_the_database *

average_space_each_entry_needs_in_the_entrycache.

Page 9

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

average_space_each_entry_needs_in_the_entrycache can be found by:

Step 1: set nsslapd-cachememsize to some guessed value that is big enough to give a good sample and set

nsslapd-cachesize to –1.
Step 2: prime the server by executing ldapsearch command 1
Step 3: find out the value of currententrycachesize and currententrycachecount by executing
ldapsearch command 3.
Step 4: average_space_each_entry_needs_in_the_entrycache =
currententrycachesize / currententrycachecount

• Please note: for detailed information about attributes like currententrycachesize,

currententrycachecount, please refer to the

“Red Hat Directory Server 7.1 Administ rat or’s Gui de”

Performance with different % database entries in entrycache

14000

12000

10000

8000

6000

operations / sec

4000

2000

0

0% 20% 40% 60% 80% 100% 120%

% of database entries in entrycache

Figure 4: Performance with different percentage of database entries in entrycache. Measured on specific
Montecito-based test configuration with 8 CPUs@1.6GHz

nsslapd-cache-autosize

• nsslapd-cache-autosize

This performance tuning related attribute which is turned off by

default, specifies the percentage of free memory to use for all the

combined caches. For example, if the value is set to 80, then 80

percent of the remaining free memory would be claimed for the cache. If

you plan to run other servers on the machine, then the value will be

250K

Page 10

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

lower. Setting the value to 0 turns off the cache autosizing and uses

the normal nsslapd-cachememsize and nsslapd-dbcachesize attributes.

Entry DN: cn=config,cn=ldbm database,cn=plugins,cn=config

Valid Range: 0 (turns cache autosizing off) to 100

Default Value: 0

Syntax: Integer

Example: nsslapd-cache-autosize: 80

If you set nsslapd-cache-autosize to a non-zero value between 0 and 100, RHDS will override the
nsslapd-cachememsize and nsslapd-dbcachesize with a value calculated based on the amount of free

memory at the time RHDS starts up. This is often not preferable as free memory might fluctuate from time to time
depending on usage of the system.

Logging

Directory Server provides three types of logs to help you better manage your directory and tune performance. These
logs include:

• Access log: The access log contains detailed information about client connections to the directory. It

provides beneficial troubleshooting information. Access logging is enabled by default.

• Error log: The error log contains detailed messages of errors and events the directory experiences during

normal operations. Error logging is enabled by default.

• Audit log: The audit log contains detailed information about changes made to each database as well as to

server configuration. By default, audit logging is disabled.

In a typical production environment, access log should be turned off, otherwise it will cause excessive disk I/O
despite buffering which will affect performance. For customers who want to have access log turned on, they can put
the access log file on a separate physical disk to improve the performance. In the performance test environment,
turning off the access log can increase the search throughput by 4%. Please see table 6.

Performance Measurements

This section describes performance testing f or Red Hat Di r ect ory Se rver version 7.1 under controlled environment
with databases containing 100, 10K, 100K, 250K, 500K, 1M, 5M
inetorgPerson entries generated by Di rM ar k . Perf o rmance data was measured for exact search on CN only.

Purpose

• Find out how number of CPUs affects the performance.

• Find out how threadnumbers affects the performance.

• Find out how dbcachesize affects the performance.

• Find out how cachememsize and cachesize affect the performance.

• Find out how logging affects the performance.

• Find out the performance differences between a SSL connection and a non-SSL connection.

Test Result

Data collection 1: (Different Number of CPUs)

This set of data is collected to help us understand the performance differences between different numbers of CPUs.

• #of entries: 100k entries

∗

See Appendix C and Appendix D for detail

∗

and 10M∗ entries. The directory entries were

Page 11

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

• RHDS parameter settings: dbcachesize and cachememsize are big enough to cache all the entries, and

nsslapd-threadnumber is set to 6.

Server DirMark

Search

performance
Montecito-based test configuration 2-CPUs @1.6GHz 5135.46
Montecito-based test configuration 4-CPUs @1.6GHz 8337.53 62%
Montecito-based test configuration 6-CPUs @1.6GHz 9999.68 95%
Montecito-based test configuration 8-CPUs @1.6GHz 13106.38 155%

Table 1: RHDS 7.1 performance with different number of CPUs on a Montecito-based test configuration
@1.6GHz/CPU

From Table 1, we can see that as the number of CPUs increased, the performance is getting better.

Performance Increases as #of

CPUs increases (compare to 2

CPU only)

Data Collection 2: (Different nsslapd-threadnumber)

This set of data is collected to help us understand how number of worker threads in ns-slapd process affects the
performance.

• Server: Montecito-based test configuration with 8 CPUs @1.6 GHz each

• #of entries: 100K entries

• RHDS parameter setting: dbcache and entry cache are big enough to hold all the pages, entries.

nsslapd-threadnumber changes as indicated in Table 2.

nsslapd-threadnumber DirMark Search performance

2 5340.03
4 9789.96
6 13209.48

8 12874.27
12 12069.33
16 10613.60
24 9634.56
32 8656.48
64 5183.54
96 5217.02

128 5384.33

Table 2: RHDS7.1 performance with different settings of nsslapd-threadnumber

From Table 2, we can see that as number of thread increases, the search throughput gets worse. Please note when
nsslapd-threadnumber drops from the default of 32 to 6, the exact search throughput increased 53%.

Data Collection 3: (Different nsslapd-dbcachesize)

This set of data is collected to help us understand how dbcachesize affects the performance:

Page 12

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

• Server: Montecito-based 8 CPUs @1.6 GHz

• #of entries: 250K

• RHDS parameter settings: set cachememsize big enough to cache all the entries. dbcachesize is tuned as

shown in Table 3, nsslapd-threadnumber is set to 6

dbcachesize DirMark Exact Search

Performance

500,000 12155.43
1,000,000 12400.40
1,500,000 12412.60
2,000,000 12446.90
4,000,000 12670.00

10,000,000 12935.50

50,000,000 12962.33
100,000,000 13117.60
200,000,000 13013.47
300,000,000 12986.17
400,000,000 13033.00

Table 3: RHDS7.1 performance difference based on different nsslapd-dbcachesize.

Because there is no particular method to determine how big the dbcache is at any given time, data shown in Table 3
is collected based on the full entry cache and different settings of dbcachesize starting from the minimum value of
500,000. When dbcachesize is set to 300,000,000, dbcachehitratio is very close to 100%, thus we can assume
that the dbcache is big enough to hold all the pages. Please note, as dbcachesize increases from 500,000 to
300,000,000 the search throughput only increased 6.8% in our Montecito-based test bed with RHDS 7.1.
Comparing to entrycache size optimization, the size change with dbcache is insignificant. As entrycache size
increases from 0% to 100%, the throughput increased about 66%. (As shown Table 4). Thus it appears that
entrycache has more impact on performance than dbcache. In the performance test, the entries in the LDAP
database are accessed uniformly throughout the test. This characteristic means that increasing the entrycache size
will provide the most performance improvement for the test. However, in practical applications, this may not
always hold true. If a small subset of all the entries tends to be accessed much more frequently than the rest, then
priority to the dbcache instead of the entrycache can be given if memory constraints require a tradeoff.

Data collection 4: (Different entrycache setting)

This set of data is collected to help us understand how entrycache affects the performance.

• Server: Montecito-based test configuration 8 CPUs @1.6GHz

• #of entries: 250K

• RHDS parameter settings: set dbcachesize big enough to cache all the pages. cachesize is tuned as shown in

Table 4, cachememsize is tuned big enough to cache all the entries, nsslapd-threadnumber is set
to 6

% of total database entries
(nsslapd-cachesize)

0% of total entries 7853.53
20% of total entries 7005.90
40% of total entries 7831.47
60% of total entries 9316.37
80% of total entries 11112.33

100% of total entries 13017.20

Table 4: RHDS7.1 performance difference based on different nsslapd-cachesize settings.

DirMark Exact Search Performance

Page 13

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Please note, to control percentage of total entries to be cached, use attribute nsslapd-cachesize. To cache
0% of total entries, set nsslapd-cachesize to 1(minimum value). From Table 4, you can see that as number of
entries can be cached in the entry cache increases, the exact search throughput increases too.

Number of Entries in the Database DirMark Search performance

(cachememsize: 100% or max

cachesize: -1

dbcachesize: 100% or max)

DirMark Search performance

(cachememsize: default

cachesize: 1

dbcachesize: 100% or max)

100 14642.13 8548.93

10,000 13808.97 8164.30
100,000 13135.43 7993.63
250,000 12986.10 7906.10
500,000 13230.17 7830.73

1,000,000 12846.54 7918.02
5,242,885 12289.70 7895.90

10,485,760 11710.23 7544.43

Table 5: Performance difference based on different nsslapd-cachesize/nsslapd-cachememsize for
different size of databases. Measured on a Montecito-based test configuration 8 CPUs @1.6GMHz each.

From Table 5 you can see that if all the entries are cached in the entry cache, and if all the pages are cached in the
dbcache, then the search throughput is close to each other no matter how big the database is. On the other hand, if
none of the entries are cached in the entry cache, the search throughputs for different sizes of databases are close to
each other too. So it turns out that if dbcache is big enough to hold all the pages, then the average exact search
throughput of any size of database with RHDS version 7.1 will be between around 8060 operations/second to 13442
operations/second depending on how big the entrycache is.

Data Collection 5: (Access Log On vs. Access Log Off)

This set of data is collected to help us understand how logging can affect the performance.

• Server: Montecito-based test configuration 8 CPUs @1.6GHz each

• #of entries: 100K

• RHDS parameter settings: dbcachesize and cachememsize are big enough to cache all the entries, and

nsslapd-threadnumber is set to 6.

Access log on or off

ON

OFF

DirMark Search performance

12727.20

13209.48

Table 6: RHDS7.1 performance difference when access log is turned on or off.

From Table 6, you can see that by turning the access log off, the exact search throughput can increase about 4% in a
Montecito-based test bed.

Data Collection 6: (SSL Connection Enabled vs. SSL Connection Disabled)

This set of data is collected to help us understand the performance impact if SSL connection is enabled.

• Server: Montecito-based test configuration 8 CPUs @1.6GHz each

• #of entries: 10K

• RHDS parameter settings: assume dbcachesize and cachememsize are big enough to cache all the entries.

nsslapd-threadnumber is set to 6.

Page 14

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

• Special setting in DirMark: “Bind for every operation” option is selected.

SSL Connection

Enable

Disable

Table 7: RHDS 7.1 performance difference with SSL connection enabled or disabled.

For all the previous data collections, the “bind once as root” option is selected when running DirMark benchmark. It
means once a ldap connection is initialized, all the following search operations will use the same connection. In a
SSL environment, overhead usually occurs during the connection setup time. To investigate how the extra workload
impacts the performance. The “bind for every operation” option is selected when runnin g Di rM ar k benchmark. It
ensures a new connection is created for every search requests.

From Table 7, the connect-and-bind performance of RHDS 7.1 is degraded about 872% when SSL is enabled.

DirMark Search performance

457.3

3988.2

Additional Tuning Reference

There are several ways to mange the directory server’s performance by limiting the amount of resources the server
uses to process client search requests. For example, size limit attribute, time limit attribute, and etc. can be tuned to
limit the resources the server uses. Increasing the checkpoint interval can increase the performance of directory
write operations in certain rare conditions. Please be cautious when tuning transaction log related attributes because
setting these attributes and other configuration attributes inconsistently may cause the directory to be unstable.
Chapter 14, Tuning Directory Server Performance, of the “Red Hat Directory Server 7.1 Administrator’s Guide
provides more detailed information to help optimize directory server performance.

”

Appendix A: RHDS 7.1 Performance Test Details

Test Environment

Operating System

HP-UX 11i v2 June 2006 release was selected to illustrate the performance of Red Hat Directory Server 7.1 on HP
Integrity Servers.

System Parameter Tuning

All the system parameters on the test machines are tuned as recommended in section ”Operating System and System
Parameter Tunig” of Sizing and Turning Overview.

General Directory Server Configuration

In order for all the test data to be comparable against each other, assume the data are collected based on the
following directory server configuration (unless specified separately within each data collection section.)

1. Logging: access log: off

error log: on
audit log: off

2. No referrals

3. No replication configured.

4. db-transaction-logging on (defaul t )

5. db-durable-transaction: on (default)

6. db-checkpoint-interval: 60 (default)

7. nsslapd-lookthroughlimit: 5000 (default)

8. nsslapd-maxdescriptors: 4096 (default)

9. nsslapd-sizelimit: 2000 (default)

10. nsslapd-timelimit: 3600 (default)

11. nsslapd-idlistscanlimit: 4000 (default)

12. legacy replication plug-in: ON (default)

Page 15

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

13. 7-bit check plug-in: on (default)

14. Schema checking: on (default) leave it on because it specifies whether the database schema will be

enforced during entry insertion or modification. When measuring performance data based on exact
searches, this core server configuration attribute won’t affect the performance. So leave it alone as the
default value.

Selected Benchmark

DirectoryMark is selected as a benchmark for measuring performance of Red Hat Directory Server version 7.1. All
the test data in this document is collected base on the DirectoryMark version 1.0. The latest version is 1.3 at the
time of this writing, but it is only available on Windows 2000, 2003 servers, and Sun Solaris.
More information is available at http://www.mindcraft.com/directorymark/

.

Test Machines

All the performance data are collected on the following machines:

Client:
Machine Class Machine Specification
SD32A Within partition, 4-way 1.6GHz ia64 8GB mem OS B.11.23 64-bit, 2 disks 89 GB

Server:
Machine Class Machine Specification
SD64B Within the partition one cell with 4 dual-core Intel Itanium2 9000 processors

counted as 8 1.6GHz CPUs, 64GB memory, OS B.11.23 64-bit, 2 disks 91GB

List 1: Client and Sever machine specifications.

Private LAN Configuration

All the test machines used for performance data collection are directly connected with Ethernet cables running point
to point between cards. All the tests are done assuming there are no heavy network traffics on the private LAN.
The configuration is as following:

Montecito-based server
machine (Itanium 2 dual
core processor)

Ethernet cables

SD32A server (client machine)
(Itanium 2 processor)

Performance Monitoring Tools

vsar(Visual System Activity Reporter)

Vsar is an unsupported performance tool that periodically samples performance information from the kernel and
displays that information on a user’s terminal. For more information about vsar, please refer to:

http://webperf.cup.hp.com/perf_tools/vsarman.txt

.

Page 16

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Caliper

HP Caliper is a general-purpose performance analysis tool for applications on Itanium®-based HP systems. HP
Caliper allows you to understand the performance of your program and to identify ways to improve its run-time
performance. HP Caliper works with any Itanium®-based binary and does not require your applications to have any
special preparation to enable performance measurement.

With Caliper’s aid, we can determine what the system is doing when running benchmark, in this case, DirMark. For
more information for Caliper, please refer to:

http://h21007.www2.hp.com/dspp/tech/tech_TechSoftwareDetailPage_IDX/1,1703,1174,00.html

.

Performance Data Creation

Test Data Generation

The data used for the performance testing is generated by DirMark 1.0. It generates inetorgPerson entries,
where average entry size is 700 bytes, and a sample directory structure. The dbgen script is used to generate the data
and ldif2db is used to populate data to the directory server.

Search Script Generation

DirMark provides script that generates search scri pt s fo r performance testing. However, DirMar k uses rand()
random-number generator whi ch onl y provides 15 bits of precision. For large number databases, a better random­number generator is required. Thus, a different program, my_scriptgen.c, which uses drand48() is created. It will
generate search scripts that only send exact search request on CN. This program is documented in Appendix B.

Appendix B: my_scriptgen.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

void rand_choose(int i, char **cn, char *TestScriptDir, char *TestScriptName, long int rand_array);
void rand_gen(int OperationPerThread, int TotalEntryInDatabase,long int *rand_array);
void main(int argc, char **argv)
{
int OperationPerThread, TotalEntryInDatabase;
char TestScriptDir[200], name_list_file[200], TestScriptName[200], *cn[1000000];
char line[300];
FILE *NameList;
int j=0;
long int rand_array[1000000];

if (argc != 6) {
printf ("\nusage: ./my_scriptgen TestScriptDir TestScriptName NameListFile\n");
printf (" OperationsPerThread TotalEntryInDatabase\n\n");
return;
}

strcpy(TestScriptDir,argv[1]);
strcpy(TestScriptName,argv[2]);
strcpy(name_list_file,argv[3]);
OperationPerThread=atoi(argv[4]);
TotalEntryInDatabase=atoi(argv[5]);

srand48(time(NULL));

/* put all the cns to veriable cn[1000000] from the name_list_file */
NameList = fopen(name_list_file, "r");
while (fgets(line, sizeof(line), NameList) != NULL)
{
line[strlen(line)-1] = '\0';
cn[j]=strdup(line);
j++;

Page 17

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

}
fclose(NameList);

/* DirMark only supports up to 50 clients. Modify the for loop if you need */
/* less than 50 search scripts */
for (int i=0; i<50; i++) {
rand_gen(OperationPerThread, TotalEntryInDatabase,rand_array);
rand_choose(i, cn, TestScriptDir, TestScriptName,rand_array);
}
}
void rand_choose(int index_num,char **cn,char *TestScriptDir,char *TestScriptName, long int rand_array)
{
FILE *search_script;
FILE *rand_file;
int i=0;
char search_script_name[400];
char line[200];
char *counter[]={"0", "1", "2", "3", "4", "5", "6", "7", "8", "9",
"10", "11", "12", "13", "14", "15", "16", "17", "18", "19",
"20", "21", "22", "23", "24", "25", "26", "27", "28", "29",
"30", "31", "32", "33", "34", "35", "36", "37", "38", "39",
"40", "41", "42", "43", "44", "45", "46", "47", "48", "49"};

strcpy(search_script_name,TestScriptDir);
strcat(search_script_name,"/");
strcat(search_script_name, TestScriptName);
strcat(search_script_name, ".client0.script");
strcat(search_script_name, counter[index_num]);

search_script = fopen(search_script_name,"w");

while(rand_array[i] != NULL) {
fprintf(search_script,"ldap_search\n");
fprintf(search_script,"bind_as\n");
fprintf(search_script,"anonymous\n");
fprintf(search_script,"password ignored\n");
fprintf(search_script,"o=Airius.com\n");
fprintf(search_script,"LDAP_SCOPE_SUBTREE\n");
fprintf(search_script,"(cn=%s)\n",cn[rand_array[i]]);
fprintf(search_script,"NULL\n");
fprintf(search_script,"0\n\n");
i++;
}
fclose(search_script);
}

void rand_gen(int OperationPerThread, int TotalEntryInDatabase, long int *rand_array)
{
for (int i = 0; i < OperationPerThread; i++) {
rand_array[i] = drand48() * TotalEntryInDatabase;
}
}

Appendix C: Test Environment for 5M and 10M data points

The test environment for the two new data points at 5M and 10M entries are almost the same as that of previous
ones, except the differences documented at the following:

Operating System

HP-UX 11i v2 September 2006 release was used and PHCO_35997 pthread library cumulative patch was installed.

Test Machines

The performance data are collected on the following machines:

Client:

Page 18

Red Hat Directory Server 7.1 Performance Tuning and Sizing Guidelines

Machine
Class
SD64B Within partition one cell with 4 dual-core Intel Itanium2 9000 processors counted

Server:

Machine
Class
SD64B Within the partition one cell with 4 dual-core Intel Itanium2 9000 processors

Machine Specification

as 8 1.6GHz CPUs, 64 GB memory, OS B.11.23 64-bit, 1 disks 73 GB

Machine Specification

counted as 8 1.6GHz CPUs, 128 GB memory, OS B.11.23 64-bit, 2 disks 143 GB

List 2: Client and Sever machine specifications.

Appendix D: Throughput for 5M database while not enough main memory for entry cache

When all the performance results above are generated in an ideal controlled environment, we also have done some
tests under condition which has not enough main memory to hold entire data entries in entry cache. We used the 5M
entries as our test database, and test setup as described at Appendix C. We set dbcache to 5.5 GB and entry cache to

7.5 GB, throughputs from DirMark client run is 7763 operations per second. This is equivalent to that the entry
cache is holding 40% of 5M entries. We observed that the main memory usage for ns-slapd is 23.2 GB.

Page 19