Novell SUSE Linux Enterprise 10 Quick Start

SUSE Linux Enterprise Real Time 10 SP1 Quick Start

SUSE Linux Enterprise 10 SP1

NOVELL® QUICK START CARD

SUSE Linux Enterprise Real Time is an add-on to SUSE® Linux Enterprise that allows you to run tasks which require
deterministic real-time processing, in a SUSE Linux Enterprise environment. SUSE Linux Enterprise Real Time
meets this requirement by offering several different options for CPU and IO scheduling, CPU shielding and setting
CPU afnities to processes.

system. Dedicated CPUs, together with some predened

Installing SUSE Linux Enterprise Real

memory, work on a number of tasks.

Time

There are two ways to set up SUSE Linux Enterprise Real
Time:

• Install it as an add-on product when installing the SUSE
Linux Enterprise Server 10 SP1.

• Install it on top of an already installed SUSE Linux Enter­prise Server 10 SP1.

SUSE Linux Enterprise Real Time always needs a SUSE Linux
Enterprise Server SP1 base, it cannot be installed in stan­dalone mode. Refer to the SUSE Linux Enterprise Server
Installation and Administration manual, Section “Installing
Add-On Products” at http://www.novell.com/

documentation/sles10/sles_admin/index.html
?page=/documentation/sles10/sles_admin/
data/sec_yast2_sw.html to learn more about in-

stalling add-on products.

The following sections provide a brief introduction to the
tools and possibilities of SUSE Linux Enterprise Real Time.

Using CPU Sets

In some circumstances, it is benecial to be able to run
specic tasks only on dened CPUs. For this reason, the
linux kernel provides a feature called cpuset. Cpusets pro­vide the means to do a so called “soft partitioning” of the

All systems have at least one cpuset that is called /. To re­trieve the cpuset of a specic task with a certain process id
pid, use the command cat /proc/pid/cpuset. To
add, remove, or manage cpusets, a special le system with
le system type cpuset is available. Before you can use
this le system type, mount it to /dev/cpuset with the
following commands:

mkdir /dev/cpuset
mount -t cpuset none /dev/cpuset

Every cpuset has the following entries:

cpus

A list of CPUs available for the current cpuset. Ranges
of CPUs are displayed with a dash between the rst and
the last CPU, else CPUs are represented by a comma
separated list of CPU numbers.

mems

A list of memory nodes available to the current cpuset.

memory_migrate

This ag determines if memory pages should be moved
to the new conguration, in case the memory congura­tion of the cpuset changes.

cpu_exclusive

Denes if this cpuset becomes a scheduling domain, that
shares properties and policies.

1

mem_exclusive

Determines if userspace tasks in this cpuset can only get
their memory from the memory assigned to this cpuset.

tasks

Contains the process ids of all tasks running in this
cpuset.

notify_on_release

If this is set to 1, /sbin/cpuset_release_agent
will be called when the last process leaves this cpuset.
Note, that it is up to the administrator to create a script
or binary that matches the local needs.

memory_pressure

Provides the means to determine how often a cpuset is
running short of memory. Only calculated if memo-
ry_pressure_enabled is enabled in the top cpuset.

memory_spread_page and memory_spread_slab

Determines if le system buffers and I/O buffers are
uniformly spread across the cpuset.

In addition to these entries, the top cpuset also contains
the entry memory_pressure_enabled, which must be
set to 1 if you want to make use of the memory_pressure
entries in the different cpusets.

In order to make use of cpusets, you need detailed hardware
information for several reasons: on big machines, memory
that is local to a CPU will be much faster than memory that
is only available on a different node. If you want to create
cpusets from several nodes, you should try to combine CPUs
that are close together. Otherwise, task switches and
memory access may slow down your system noticeably.

To nd out which node a CPU belongs to, use the /sys
le system. The kernel provides information about available
CPUs to a specic node by creating links in /sys/
devices/system/node/nodeX/.

If several CPUs are to be combined to a cpuset, check the
distance of the CPUs from each other with the command
numactl --hardware. This command is available after
installing the package numactl.

The actual conguration and manipulation of cpusets is
done by modifying the le system below /dev/cpuset.
Tasks are performed in the following way:

Create a Cpuset

To create a cpuset with the name exampleset, just run
mkdir /dev/cpuset/exampleset to create the
respective directory. The newly created set will contain
several entries that reect the current status of the set.

Remove a Cpuset

To remove a cpuset, you only need to remove the cpuset
directory. For example, use rmdir
/dev/cpuset/exampleset to remove the previously
generated cpuset named exampleset. In contrast to

normal le systems, this works even if there are still en­tries in the directory.

Note that you will get an error like rmdir: example-
set: Device or resource busy, if there are still
tasks active in that set. To remove these tasks from the
set, just move them to another set.

Add CPUs to a Cpuset

To add CPUs to a set, you may either specify a comma
separated list of CPU numbers, or give a range of CPUs.
For example, to add CPUs with the numbers 2,3 and 7
to exampleset, you can use one of the following
commands: /bin/echo 2,3,7 >

/dev/cpuset/exampleset/cpus or /bin/echo
2-3,7 > /dev/cpuset/exampleset/cpus.

Add Memory to a Cpuset

You cannot move tasks to a cpuset without giving the
cpuset access to some system memory. To do so, echo
a node number into /dev/cpuset/exampleset/
mems. If possible, use a node that is close to the used
CPUs in this set.

Moving Tasks to Cpusets

A cpuset is just a useless structure, unless it handles some
tasks. To add a task to /dev/cpuset/exampleset/,
simply echo the task number into /dev/cpuset/
exampleset/. The following script moves all user space
processes to /dev/cpuset/exampleset/ and leaves
all kernel threads untouched:

cd /dev/cpuset/exampleset; \
for pid in $(cat ../tasks); do \
test -e /proc/$pid/exe && \
echo $pid > tasks; done

Note, that for a clean solution, you would have to stop
all processes, move them to the new cpuset, and let them
continue afterward. Otherwise, the process may nish
before the for loop nishes, or other processes may start
during moving.

This loop liberates all CPUs not contained in the exam­pleset from all processes. Check the result with the
command cat /dev/cpuset/tasks, which then
should not have any entries.

Of course, you can move all tasks from a special cpuset
to the top level set, if you intend to remove this special
cpuset.

Automatically Remove Unused Cpusets

In case a cpuset is not used any longer by any process,
you might want to clean up such unused cpusets auto­matically. To initialize the removal, you can use the no-
tify_on_release ag. If this is set to 1, the kernel
will run /sbin/cpuset_release_agent when the
last process exits. To remove an unused script, you may,

2

for example, add the following script in /sbin/cpuset

_release_agent:

#!/bin/sh
logger cpuset: releasing $1
rmdir /dev/cpuset/$1

After adding the script to your system, run chmod 755
/sbin/cpuset_release_agent to make the script

executable.

Determine the Cpuset of a Specic Process

All processes with the process id PID have an entry in

/proc/PID/cpuset. If you run the command cat
/proc/PID/cpuset on a PID that runs in the cpuset
exampleset, you will nd the results in /exampleset.

The command taskset can either be used to start a new
process with a given CPU afnity, or to redene the CPU
afnity of a already running process.

Examples

taskset -p pid

Retrieves the current CPU afnity of the process with
PID pid.

taskset -p mask pid

Sets the CPU afnity of the process with PID pid to
mask.

taskset mask command

Runs command with a CPU afnity of mask.

Changing I/O Priorities with ionice

Specifying a CPU Afnity with

taskset

The default behavior of the kernel, is to keep a process
running on the same CPU, if the system load is balanced
over the available CPUs. Otherwise, the kernel tries to im­prove the load balancing by moving processes to an idling
CPU. In some situations, however, it is desirable to set a
CPU afnity for a given process. In this case, the kernel will
not move the process away from the selected CPUs. For
example, if you use shielding, the shielded CPUs will not
run any process that does not have an afnity to the
shielded CPUs. Another possibility is to run all low priority
tasks on a selected CPU to remove load from the other
CPUs.

Note, that if a task is running inside a specic cpuset, the
afnity mask must match at least one of the CPUs available
in this set. The taskset command will not move a process
outside the cpuset it is running in.

To set or retrieve the CPU afnity of a task, a bitmask is
used, that is represented by a hexadecimal number. If you
count the bits of this bitmask, the lowest bit represents the
rst logical CPU as they are found in /proc/cpuinfo. For
example:

0x00000001

is processor #0.

0x00000002

is processor #1.

0x00000003

is processor #0 and processor #1.

0xFFFFFFFE

all but the rst CPU.

Handling I/O is one of the critical issues for all high-perfor­mance systems. If a task has lots of CPU power available,
but must wait for the disk, it will not work as efcient as it
could. The Linux kernel provides three different scheduling
classes to determine the I/O handling for a process. All of
these classes can be ne-tuned with a nice level.

The Best Effort Scheduler

The Best Effort scheduler is the default I/O scheduler,
and is used for all processes that do not specify a differ­ent I/O scheduler class. By default, this scheduler sets its
niceness according to the nice value of the running
process.

There are eight different nice levels available for this
scheduler. The lowest priority is represented by a nice
level of seven, the highest priority is zero.

This scheduler has the scheduling class number 2.

The Real Time Scheduler

The real-time I/O class always gets the highest priority
for disk access. The other schedulers will only be served,
if no real-time request is present. This scheduling class
may easily lock up the system if not implemented with
care.

The real-time scheduler denes nice levels just like the
Best Effort scheduler.

This scheduler has the scheduling class number 1.

The Idle Scheduler

The Idle scheduler does not dene any nice levels. I/O
is only done in this class, if no other scheduler runs an
I/O request. This scheduler has the lowest available pri­ority and can be used for processes that are not time­critical at all.

If a given mask does not contain any valid CPU on the sys­tem, an error is returned. If taskset returns without an error,
the given program has been scheduled to the specied list
of CPUs.

This scheduler has the scheduling class number 3.

To change I/O schedulers and nice values, use the ionice
command. This provides a means to tune the scheduler of

3