Python 7.0pl5 User Manual
Dumpleton
Software
Consulting
Pty Limited
OSE
Version 7.0pl5
Python Manual
19 January 2003
Copyright 2001-2003 Dumpleton Software Consulting Pty Limited
http://www.dscpl.com.au
2
Table o f Contents
Table of Contents...........................3
Manual Overview ..........................7
Python Modules .............................9
Module Descr iptions.......................10
Installation and Setup......................1 0
Additional Information....................11
Logging Facility...........................13
Logging a Message .........................13
Specifying a Log File......................14
Specifying a Log Channel...............16
Logging Python Exceptions............16
Exceptions in a Callback .................17
Program Setup..............................19
Configuration Database...................19
Configuration File...........................2 0
Naming Hierarchies ........................2 1
Environment Variables....................21
Unique Identifiers............................22
Process Identity...............................23
Event Framework.........................25
Scheduling a Job .............................25
Real Time Events............................28
Destroying Agents...........................29
Alarms and Timers..........................29
Recurring Actions ...........................30
Socket Events..................................31
Program Signals..............................32
Program Shutdown ..........................33
Service Agents.............................37
Service Naming...............................38
Service Audience ............................39
Anonymous Service........................39
Service Groups................................40
Service Registry..............................40
Service Announcements..................43
Group Announcements....................44
3
Table of Contents
Service Lookup ...............................44
Service Reports.............................47
Publishing Reports ..........................4 8
Monitoring Reports.........................48
Lifetime of Reports .........................50
Identity of Subscribers .................... 5 2
Existence of Publishers ...................54
Service Requests...........................57
Sending a Request...........................57
Handling a Response....................... 59
Identifying a Response....................60
Detecting a Failure ..........................61
Lack of Response ............................63
Servicing a Request.........................64
Generating a Failure........................6 5
Delaying a Response.......................66
Handling Structured Types..............85
Servlet Framework.......................87
Framework Overview......................87
The HTTP Daemon......................... 88
The File Server................................89
Client Authorisation........................90
User A uthorisation..........................91
HTTP Server Objects ......................91
The Error Servlet............................. 92
The Redirect S ervlet........................93
The Echo Servlet.............................93
The File Servlet............................... 93
Logging of Requests........................94
Servlet Objects .............................95
Processing a Request.......................95
Persistent Connections....................96
Identity of the Sender. .....................68
Invalid Request Method ..................6 9
Local S ervice Requests ...................6 9
Message Exchange .......................71
Exchange Initialisation....................72
Service Availability.........................72
Connection Announcements ...........74
Authorisation of Clients..................75
Distributed Exchange Server...........75
Multiple Exchange Groups..............76
Scalability of the Framework..........7 8
Message Encoding........................79
Supported Data Types..................... 8 0
Mapping of Scalar Types ................8 1
User Defined Types.........................83
Adding New Mappings ...................8 3
Delaying a Response.......................97
Destruction of Servlets....................98
Processing Content.......................... 99
The Form Servlet...........................100
Slow HTTP Clients.......................101
Servlet Plugins............................105
Python Plugin................................105
Module Caching............................107
Writing a Plugin............................108
Plugin Aliasing..............................108
Remote Access ...........................111
The RPC Gateway.........................112
The Client Application..................113
Restricting Client Access ..............113
Duplicate Services.........................114
User Defined Types.......................115
4
Managing User Sessions...............116
The SOAP Gateway......................1 21
The XML-RPC Gateway...............118
Using Multiple Gateways..............123
5
Table of Contents
6
Manual Overview
This manual covers the Python wrappers around the OSE C++ class library. The wrappers make available functionality related to the logging system, the real time events system, the service agent framework for creating distributed applications, the HTTP servlet framework and the RPC over HTTP
interfaces.
Python Modules Lists the available Python modules and their pur-
pose. Includes brief details regarding installation
and setup of the users environment.
Logging Facility Describes the message logging facility,including
how to direct messages to a specific log channel,
how to log messages to a file or to process them
within the actual application.
Program Setup Describes the interface to the configuration data-
base, user environment and other process information.
Event Framework Describes the interface to the event system, how
to schedule jobs and setup callbacks in response
to real time events such as timers, signals and
socket activity.
7
Manual Overview
Service Agents Describes how to create service agents, add them
Service Reports Describes how to subscribe to reports published
Service Requests Describes how to send requests to remote or
Message Exchange Describes how to connect up processes to form a
to groups, subscribe to announcements regarding
specific services or membership of specific service groups.
by specific services. Ie., describes the publish/
subscribe functionality provided by the service
agent framework.
local service agents and how to handle any
response or error which results. Ie., describes the
messaging or request/reply functionality of the
service agent framework.
distributed application, including a decentralised
message exchange and exchange groups.
Message Encoding Describes the Python types which can be used in
messages and how this can be extended to incorporate new scalar data types.
Servlet Framework Describes the HTTP daemon and servlet frame-
work, including the predefined servlets and how
to create customised HTTP server objects.
Servlet Objects Describes how to create new servlet objects
including how to handle forms, client congestion
and delayed responses.
Servlet Plugins Describes how to create servlet plugins and how
to use the Python plugin to dynamically load
servlets at runtime from the file system.
Remote Access Describes the RPCover HTTP interfaces into the
service agent framework, including support for
NET-RPC, XML-RPC and SOAP protocols.
8
Python Modules
OSE includes a number of Python modules. The main module is a wrapper around functionality provided in the OSE C++ class library. Those parts of the OSE C++ class library for which a Python wrapper are provided are the logging system, the real time events system, the service agent framework for
creating distributed applications and the HTTP servlet framework.
Additional modules provide access to the OSE service agent frameworkusing an RPC over HTTP protocol called NET-RPC as well as the XML-RPC and SOAP protocol. Note that the XML-RPC and
SOAP protocols come with restrictions deriving from problems in the respective protocols and the
NET-RPC protocol provides the best integration.
Because interfaces are provided for the OSE service agent framework in both C++ and Python, an application may be spreadacross multiple processes and consist of processes written using either C++ or
Python code. Using shared libraries and dynamic loading, C or C++ code could also be loaded into
Python to perform some functions if desired.
Overall, the Python wrappers provide an interface to the functionality of the OSE C++ class library
which is easierto use than if the C ++ classlibrary were useddirectly. This makes the Python wrappers
ideal for building up the overallstructure of a distributed system, with C++ code being used only when
necessary.
9
Python Modules
Module Descriptions
The Python modules, their names and their purpose are described below.
Module Purpose
netsvc This is the main module and provides wrappers around the
functionality of the OSE C++ class library. It includes all that
is required for building distributed applications using the
service agent framework.
netrpc This module provides a client implementation of the RPC
over HTTP protocol implemented by OSE called NET-RPC.
netsvc.xmlrpc This module includes the XML-RPC gateway for OSE. Any
server code is the same as for when the NET-RPC protocol is
used. The only difference is which gateway you instantiate.
netrpc.xmlrpc This module provides a client implementation of the XML-
RPC protocol. The module is interface compatible with
the"netrpc" module.
netsvc.soap This module includes the SOAP gateway for OSE. Any
server code is the same as for when the NET-RPC protocol is
used. The only difference is which gateway you instantiate.
netrpc.soap This module provides a client implementation of the SOAP
protocol. The module is interface compatible with
the"netrpc" module.
Installation and Setup
The "netsvc" module requires the main OSE C++ class library to be installed, as well as the Python
extension library. The version of "makeit" installed when OSE is installed needs to be run in the
"python" subdirectory of the OSE source code. This final step will install the two Python modules,
a dynamically loadable module which drags in the OSE C++ class libraries and a GUI based debugger
for the service agent framework called "spyon". The exact steps which need to be followed are given
in the "INSTALL" file in the OSE source code.
When the Python modules are installed, they are not installed into your Python installation, but into
the same area that OSE is installed. In order that Python can find the modules, you will need to set your
PYTHONPATH environment variable to include the appropriate library directory in the OSE installation. For OSE 7.0, if installed into its standard location, the directory will be:
/usr/local/ose/7.0/lib/python
10
Additional Information
An OSE installation supports libraries for different architectures. In order that the shared libraries for
your specific platform can be found by the Python module, you should ensurethat the OSE_HOST var-
iable is set to the same value it was set to when OSE was installed. For example:
OSE_HOST=X86_LINUX
If you want to be able to run the "spyon" debugger, your PATH environment variable should include
the OSE bin directory. For OSE 7.0, if installed into its standard location, the directory will be:
/usr/local/ose/7.0/bin
If you want to be able to build up a version of the Python wrappers with a DLL for Win32, you have
two choices. The first requires you to have access to either the Cygnus Win32 toolkit or MKS toolkit,
and the Microsoft C++ compiler. In this case the normal build procedure for OSE is followed. If you
only have access to the Microsoft C++ compiler, a native makefile is provided with the source code in
the "win32" directory. You should follow the instructions contained in that directory.
Note that if you wish to use either the SOAP client or SOAP gateway, you will need to separately obtain and install the "ZSI" package from the "pywebsvcs" project on SourceForge. The project site address is "http://sourceforge.net/projects/pywebsvcs". You must have version 1.2 RC2 or later of the
ZSI package.
Additional Information
As the main Python moduleis a wrapperaround functionality provided in the OSE C++ class libraries,
it may be worthwhile to also consult the manual pages for the corresponding classes in the C++ class
library and the general C++ class library manual. The behaviour of some features is controlled using
environment variables and not all of these may be mentioned in the manual for the Python modules.
11
Python Modules
12
Logging Facility
The logging facility provides you with a mechanism for generating and capturing messages generated
by your application. These can be automatically saved to a log file, or intercepted and dealt with in
some other way. The majority of functionality for this feature is provided by the OTC_Logger class
in the OSE C++ class library.
Some of the features of the logging facility are optional and controlled via environment variables.You
should consult the manual page for the OTC_Logger class and the general OSE C++ class library
manual as a number of these features will not be described here or covered only briefly.
Logging a Message
The logging facility provides you with the ability to log a message string with a specified priority or
level assigned to it. The level is analogous to that used by the UNIX function called "syslog()".
Level Usage
LOG_EMERGENCY A panic condition.
LOG_ALERT A condition that should be corrected immediately,
such as a corrupted system database.
LOG_CRITICAL Critical conditions, such as hard device errors.
LOG_ERROR Errors.
LOG_WARNING Warning messages.
13
Logging Facility
Level Usage
LOG_NOTICE Conditions that are not error conditions, but that
may require special handling.
LOG_INFO Informational messages.
LOG_DEBUG Message that contain information normally of use
only when debugging a program.
To log a message,a handle to an instanceof the Logger class is acquired and the "notify()"member function is called.
import netsvc
logger = netsvc.Logger()
logger.notify(netsvc.LOG_DEBUG,"message")
The format of a message when displayed will be:
DEBUG: message
The string before the ":" corresponds to the level assigned to the message. The remainder of the line
after the ":" is the actual message. If you wish to have the time and process ID appear in the prefix,
call the "enableLongFormat()" member function. Whether the longer form of prefix is enabled
can be queried using the "longFormatEnabled()" member function. It can be disabled using the
"disableLongFormat()" member function.
By default,messages will appear on the standard erroroutput. If you wish to disable the display of messages onto the standard error output, call the "disableStderrOutput()" member function. Conversely, the "enableStderrOutput()" member function can be called to enable display of
messages onto the standard error output if previously disabled. Whether messages are currently being
displayed onto the standard error output can be queried by calling the member function "stder-
rOutputEnabled()".
Specifying a Log File
At any time, messages can be captured into a single file by specifying the name of a log file using the
member function "setLogFile()". If a log file is currently in use, the name of the log file can be
queried using the "logFile()" member function.
logger.setLogFile("/var/tmp/application.log")
14
Specifying a Log File
The string used to specify the name of a log file may incorporate the following special tags.
Tag Purpose
%h Willencode the hostname of the machine into the name of the log file.
%p Will encode the process ID into the name of the log file.
%Y Will encode the current year as 4 digits into the name of the log file.
%y Will encode the current year as 2 digits into the name of the log file.
%m Will encode the current month of the year as a zero padded 2 digit
number into the name of the log file.
%d Willencode the current day of month as a zero padded 2 digit number
into the name of the log file.
When the tags corresponding to dates are used, a new log file will automatically be created when the
value corresponding to a date component changes. The following will for example result in a new log
file being created each day.
logger.setLogFile("/var/tmp/application-%Y-%m-%d.log")
Note that older log files will not be removed automatically, so some other mechanism such as a cron
job will need to be employed to remove them.
The name of a log file can also be set using the OTCLIB_LOGFILE environment variable instead of
calling "setLogFile()". Similarly, output to the standard error output can be disabled using the
OTCLIB_NOLOGSTDERR environment variable and the inclusion of the time and the process ID in
the message prefix enabled using the OTCLIB_LOGLONGFORMAT environment variable. If used,
these environment variables must be set before the application is run or at least before the "netsvc"
module is imported for the first time.
import os
os.putenv("OTCLIB_LOGFILE","/var/tmp/application-%Y-%m-%d.log")
import netsvc
When an application first attempts to open a log file, if i t already exists it will be truncated. If you do
not want the log file truncated, but want messages to be appended to an existing log file, the
OTCLIB_APPENDLOGFILE environment variablemust be set.Again, this needs tobe set prior to the
application being run or at least before the "netsvc" module is imported for the first time.
Note that if any of these environment variables are used, but calls are subsequently made to the corresponding member functions of the Logger class from within the application, the values of the environment variables will effectively be overridden from that point onwards.
15
Logging Facility
Specifying a Log Channel
When logging a message, a log channel may also be specified. If the name of a log channel starts with
a character other than an alphanumeric character, the message will not be displayed on the standard
error output or appear in the log file. If it is displayed or captured in the log file, the name of the log
channel does not appear anywhere in the message. The intent of the log channel is to allow one part of
an application to capture specific messages produced by another part of the application and deal with
them in a special way.
To log a message against a specific log channel, the member function "notifyChannel()"isused.
The name of the log channel is supplied as the first argument.
logger.notifyChannel("VISIBLE",netsvc.LOG_DEBUG,"message")
logger.notifyChannel("#HIDDEN",netsvc.LOG_DEBUG,"message")
Messages logged against a specific log channel, can be captured by calling the member function
"monitorChannel()", supplying the name of the log channel and a callback function.
def callback(channel,level,message):
print (channel,level,message)
logger.monitorChannel("#HIDDEN",callback)
The message supplied to the callback function is the original message and does not contain the prefix
describing the priority or level assigned to the message, nor does it contain any details relating to the
current time or process ID. If you are going to subsequently log the message to a file, you would need
to add these details yourself if you require them.
Only one callback can be associated with a particular log channel. If multiple callbacks are required
for a particular log channel, separate instances of the Logger class should be used. To stop monitoring a specific log channel, the member function "monitorChannel()" is called again but with
"None" supplied in place of the callback function.
If the callback function was a member of a class, it is important to deregister the callback, else a reference to the instance of the class will be maintained and it may not get deleted. You can also deregister
all of the callbacks associated with a particular instance of the Logger class by calling the member
function "destroyReferences()". This would be necessary if the class containing the callbacks
also held a reference to the instance of the Logger class. In this case, a circular reference would exist
and neither object would ever be destroyed.
Logging Python Exceptions
To make the task of logging details of a Python exception easier, the "logException()" function
is provided by the "netsvc" module. This function should only be called from within the context of
a Python "except" clause. The information logged is similar to that displayed by Python when an
exception is not caught and includes details of the exception and a stack trace.
16
Exceptions in a Callback
try:
function()
except SystemExit:
raise
except:
netsvc.logException()
sys.exit()
The details of the exception are logged with level "LOG_ERROR" and a specific log channel is not
specified. If you wanted to log the details of the exception to a specific log channel, or vary the level,
you can use the "exceptionDetails()" function of the "netsvc" module to obtain the same informationthatwould be loggedby the "logException()" function and then call the "notify()"
member function of an instance of the Logger class yourself.
try:
function()
except SystemExit:
raise
except:
details = netsvc.exceptionDetails()
logger.notifyChannel("WARNING",netsvc.LOG_WARNING,details)
pass
If you don’t want the stack trace and only want the description of the exception, use the function "exceptionDescription()" instead. The result of calling either of these functions need not be used
with the logger, but could be displayed using any other available mechanism as well.
Note that the "exceptionDetails()" and "exceptionDescription()" functions are also
available in the "netrpc" module if you are using that in a standalone client application.
Exceptions in a Callback
Whenever a callback is executed, it occurs as a result of a call from C++ code into Python code. Because of the mix of C++ code and Python code, if an exception occurs within the callback function,
Python can’t by itself properly shutdown the application. This is further complicated by the fact that a
callback can be called within the context of a callback from the event dispatcher.
As a consequence, whenany callback into Python code fromC++ occurs, if a Python exceptionoccurs
andthe callback itself doesn’t catch it and deal with it, it will be caught with the details of the exception
being logged. The event dispatcher will then be stopped if it is running and the "SystemExit"exception raised in order to prevent Python from running any further code. The outcome is the same as
when only Python code is being used, except that the details of the exception are displayed using the
logging facility rather than being dumped directly onto the standard error output.
17
Logging Facility
18
Program Setup
As Python is an interpreted language, configuration of an application can be carried out by editing the
actual scripts. In some circumstances however, it is still easier or more practical to rely upon a configuration database or environment variables. When using OSE this is especially the case, as an application can be a mix of C++ and Python code and configuration data may need to be accessible from code
written in both languages.
To support this the Python wrappersprovide an interface tothe configuration database of the OSE C++
class library. The corresponding class in the OSE C++ class library which provides this functionality
is the OTC_Program class. Not all functionality of this class is mirrored in the Python interface a s
Python has its own way of doing most of what is provided by this class. Access is however provided
to aspects of the configuration database and environment variable database. The functionality for generating unique identifiers is also exposed.
Configuration Database
The configuration database is an in memory database. The database may be populated by calls from
within the application, or by loading in a configuration file. The configuration database may also be
saved to a file. In essence, the configuration database is not much more than a dictionary mapping
names to values.
To initially load the configuration database from a file, the "loadConfig()" function is used. A single configuration item may be explicitly merged into the configuration database using the
"mergeConfig()" function. A query can subsequently be made against the configuration database
using the "lookupConfig()" function. If no match is found in the configuration database for the
item in question, the value None is returned.
19
Program Setup
import netsvc
import os
netsvc.loadConfig("database.cfg")
netsvc.mergeConfig("PWD",os.getcwd())
print netsvc.lookupConfig("PWD")
A single configuration item can be removed from the database using the "removeConfig()" function. The configuration database can be completely emptied using the function "removeAllCon-
fig()". The contents of the configuration database can be saved to a file using the
"saveConfig()" function.
netsvc.removeConfig("PWD")
netsvc.saveConfig("database.cfg")
Configuration File
The only real restrictions in regard to naming is that the colon character should not be used anywhere
in a name, a name should not being with an exclamation mark and whitespace should not be used at
the start or end of a name. The colon character cannot be used as it used in a configuration file to separate the name from the value. A leading exclamation mark should not be used as it is used to denote
a comment.
If these characters are used in a name and the configuration database is saved to a file, the results when
that configuration file is read back in will not be the same. The only other special character when used
in a configuration file is a back slash, which when used at the end of the line, indicates the following
line is part of the same value. Note that the leading whitespace and the whitespace either side of the
colon will be ignored when the configuration file is read in.
! comment
single-line-value : value
multi-line-value : value\
value
When reading in a configuration file using "loadConfig()", an exception is raised only if the file
doesn’t exist or the file couldn’t be opened. If there are no errors in the file, the value None is returned.
If there are errors in the file, a string is returned which contains details of the errors and what action
has been taken. By default, the details of the errors are also output via the logging systemon the default
log channel.
If details of any errors should be output on a specific log channel, an optional second argumentcan be
supplied to the "loadConfig()" function giving the name of the log channel. If the value None is
supplied in place of the name of the log channel, the details of the errors will not be output via the logger at all. The value None could be used if you wish to amend the details of the errors before they are
logged.
20
Naming H ierarchies
file = "database.cfg"
errors = netsvc.loadConfig(file,None)
if errors:
errors = "Error reading %s\n%s" % (‘file‘,errors)
netsvc.Logger().notify(netsvc.LOG_DEBUG,errors)
Naming Hierarchies
Ifa naming hierarchy is required,the components of thehierarchy within the name should be separated
by using a period.
compiler.preprocessor.debug-level : 0
compiler.parser.debug-level : 1
compiler.code-generator.debug-level : 0
compiler.assembler.debug-level : 0
In general, the purpose of using a naming hie rarchy is to associate properties with the same name with
different parts of an application, or with different instances of some object. Tocater for default values,
rather than enumerating all possible objects, a wildcard can be specified in place of a single component
in a naming hierarchy. This says to match any component name in this position. Only those items
which need to be different then need to be explicitly specified.
compiler.*.debug-level : 0
compiler.parser.debug-level : 1
When a lookup is made against the database, a check is first made for any entry which matches exactly
the name of interest. If this name is not present, a search is then made of the entries containing a wildcard. If a match is found, the value associated with the wildcard entry will be returned. If there are
multiple wildcard entries which match a lookup against the configuration database, that which has the
longest leading exact match will be used.
Environment Variables
In addition to the configuration database, an interface is also provided to the standard operating system
environment variables.Python does already provide aninterfacefor this, however the Python interface
does have a few quirks which can sometimes make it less than useful.
One problem with the standard Python interface is that when "os.putenv()" is used to set an environment variable, that variable is not then visible using "os.getenv()". This is because
"os.getenv()"uses"os.environ", which is a copy of the environment which is populated at
startup and any changes to environment variables are not reflected in that copy.
As such, although changes to the environment will be seen by subprocesses, they will not be visiblein
the same process. This means that a n environment variable can’t at the same time be used to transfer
information to a different part of the application.
21
Program Setup
To lookup the value of an environmentvariable the function "lookupEnviron()"isused.Ifanew
environment variable needs to be set, or an existing value changed, the function "mergeEnvi-
ron()" is used. Anychanges to the environment variables will be visibleimmediately, but there is no
way to get a list of all environment variables which are set. When a lookup is made but no such environment variable exists, the value None is returned.
netsvc.mergeEnviron("PWD",os.getcwd())
print netsvc.lookupEnviron("PWD")
In addition to these functions, the function "expandEnviron()" is provided. This function accepts
a string and replaces any reference to an environment variable specified using Bourne shell syntax,
with that environment variables actual value. The intent in providing this functionis that it can be used
in conjunction with the configuration database, allowing configuration items to refer to environment
variables.
application.log-files : ${HOME}/logs
Note that the expansion isn’tautomatic when a lookup is made against the configuration database.The
application code will have to explicitly expand the value obtained form the configuration database.
value = netsvc.lookupConfig("application.log-files")
directory = netsvc.expandEnviron(value)
Unique Identifiers
In many applications, it is often useful to be able to create abstract identifiers to uniquely identify objects or resources. These might be used to identify user sessions in a web based application, specific
requests in a distributed messaging system, or even the particular service agent which a request in a
distributed messaging system is targeted at.
Such identifiers may only need to be unique within the context of the lifetime of the application, or
possibly may need to be globally unique. In the case of the latter, to be rigourous this would normally
require an external database to be maintained which tracks what identifiers have been used. In most
cases however, it is not necessary to go to that extent and a simplistic means can be used to generate a
psuedo unique identifier which is sufficient.
To generate such identifiersthe function "uniqueId()" is provided. The function can provide identifiers in either a short or long format. In the short format, the identifier contains components which
identify the host on which the process is running, the process id andan incremental counter. In the long
format, time values are also included which tie the identifier to an instant in time.
id1 = netsvc.uniqueId(netsvc.ID_SHORT_FORMAT)
id2 = netsvc.uniqueId(netsvc.ID_LONG_FORMAT)
If you wish to incorporate your own prefix into the identifier, an optional second argument can be supplied to the "uniqueId()" function.
22
Process Identity
id1 = netsvc.uniqueId(netsvc.ID_SHORT_FORMAT,"$SID?")
The short format identifier is suitable for use within the context of a single process. Duplicates would
only be encounterd if the incremental count of the number of identifiers exceeded what can be stored
within a 32 bit integer value. If this were to occur, the counter would wrap around to zero and conflicts
might thus arise if the existing identifier were still active.
The short format identifier could also be used within the context of a constrained distributed applicationprovided that the nature of the application is such that knowlegeof what the identifier is associated
with is always discarded when the process the identifier is bound to is destroyed. This would be necessary, as the identifier could be reused if the process id was reused at some latter point.
If a better gaurantee of uniqueness over time is required, the long format identifier should be used. In
this case, the identifier also records the time at which the first identifier was generated by the process,
as well as a time delta as to when that particular identifier was generated. Incorporation of time information avoids problems with the incremental counter overflowing and reuse of the same process id at
a latter point in time.
Process Identity
A further feature which is useful in distributed applications is a way of identifying specific processes.
Such an identifier can be generated by combining the name of the host and the process id into a single
string. To facilitate this, the function "processIdentity()" is provided.
identity = netsvc.processIdentity()
23
Program Setup
24
Event Framework
The main support for concurrency in the OSE C++ class libraries comes in the form of a mechanism
for building event driven systems. This is based around a central job queue and a dispatcher, which
takes successive jobs fromthe queue and executes them. To support real time systems, there also exist
a number of event sources which will schedule jobs to trigger an agent to be notified when an event of
interest occurs.The major event sources include timers, signals and the availability of data for reading
on a socket.
The major classes in the OSE C++ class library involved in providing this functionality are the
OTC_Dispatcher, OTC_EVAgent and OTC_Job classes, plus thevarious event classes relatedto
the event sources. In the C++ implementation, communication of events is mainly performed by passing around event objects and having a single event handler method in an agent to deal with them. In
the Python implementation, separate callback functions can be registered by an agent against each
event of interest.
Note that only the major features of the C++ implementation are reflected in the Python interface. Python does not provide a means of creating your own event types or event sources. A Python agent is
also not able to process any events except those from the major event sources.
Scheduling a Job
Scheduling of jobs comes in the form of registering a callback function with the dispatcher for execution. A job may be scheduled as a priorityjob, a standard job, or an idle job. Thetype of job determines
where in the order of existing jobs, a new job will be placed. Any priority jobs are executed before a
standard job is processed. When there are no priority jobs or standard jobs remaining, any pending idle
25
Event Fr amework
jobs will be reclassifiedas standard jobs and subsequently executed. When scheduling a job, if jobs of
the same type already exist, the new job will be placed at the end of the list of jobs of the same type.
To schedulea job the dispatcher member function "schedule()" must be called,supplying the callback function and the type of job. To set the dispatcher running, the member function "run()"is
called. If the only feature of the event system which is used is that of scheduling jobs, the "run()"
function will return when there are no more jobs to execute. A job may prematurely stop the dispatcher
by calling the "stop()" member function. If a callback raises an exception which is not caught and
processed within the callback itself, the details of the exception will be logged, the dispatcher stopped
and Python exited immediately.
def callback(message="hi"):
print message
dispatcher = netsvc.Dispatcher()
dispatcher.schedule(callback,netsvc.IDLE_JOB)
dispatcher.schedule(callback,netsvc.STANDARD_JOB)
dispatcher.schedule(callback,netsvc.PRIORITY_JOB)
dispatcher.run()
The callback supplied when schedulinga job can be a normal function or amember function associated
with an instance of a class. If a callback function is scheduled directly with the dispatcher in this way,
it will be called with no arguments and cannot be cancelled once scheduled.
If it is necessary to pass arguments to a callback function, an instance of the Job class must be used
in place of the actual callback function. The Job class will hold a reference to the real callback function as well as the arguments. When the job is executed it will call the callback function with the supplied arguments.
job = netsvc.Job(callback,("bye",))
dispatcher.schedule(job,netsvc.IDLE_JOB)
In addition to providing a means of supplying arguments to a callback function, the Job class provides
a means of cancelling execution of a callback function. In order to do this, a reference to the instance
of the Job class should be kept. Ifit is subsequently necessary to cancel execution of the callback prior
to it having being called, the "cancel()" member function of the Job class should be called.
job = None
def callback1():
print "hi"
job.cancel()
def callback2():
print "hi"
dispatcher.schedule(callback1,netsvc.PRIORITY_JOB)
job = netsvc.Job(callback2)
dispatcher.schedule(job,netsvc.STANDARD_JOB)
26
Scheduling a Job
All that is occuring here is that when the "cancel()" member function is called, a flag is set. When
the job is executed it will note that the flag is set and will not execute the callback function. If the callbackfunction is amember function of a class, itis importantto ensure that any reference tothe instance
of the Job class is destroyed when nolonger required. If this is not done and the reference is a member
variable of the same class the callback function is a member of, a circular reference will exist and that
instance of the class will not be able to be destroyed.
Any arguments to be passed to the callback function would by default be supplied when the instance
of the Job class is created. If it is necessary to generate an instance of the Job class such that it can
be passed to another part of the program, but the arguments to the callback function are not known at
that time, it is instead possible to supply the arguments at the time the job is scheduled. This is done
byusing the "schedule()" member function of the Job class rather than that of the dispatcher. Any
arguments supplied in this way will override those provided when the instance of the Job class is created.
job = None
def callback1(message):
print message
job.schedule(netsvc.STANDARD_JOB,("override",))
def callback2(message)
print message
job = netsvc.Job(callback1,("default",))
job.schedule(netsvc.STANDARD_JOB)
job = netsvc.Job(callback2)
This would allow for instance a class which accepts callback registrations to return a reference to a
Job class which will later be used to schedule the callback with an as yet undetermined set of arguments. The client who registered the callback could however cancel execution of the callback before
it is called.
Once "cancel()" has been called on an instance of a Job class, whether or not it has already been
scheduled, the callback function will never be executed. To reset the flag which makes the callback
function runnable, the "reset()" member function should be called. To determine if the an instance
of the Job class is still in a runnable state, a truth test can be performed on it.
if job:
# job wasn’t cancelled
job.schedule(netsvc.STANDARD_JOB)
else:
# job was cancelled
pass
If you wish to use the Job class separate to the dispatcher, you can trigger execution of the callback
function by calling the "execute()" member function. If any arguments are supplied to the "exe-
27
Event Fr amework
cute()" member function, these willoverride any which may have been supplied when that instance
of the Job class was created.
Real Time Events
The Python interface provides the ability to register interest in a number of real time events. These are
program shutdown, one off alarms or actions, recurring actions, timers, signals and data activity on
sockets. That an event of interest has occurred is notified by execution of a callback supplied at the
time that interest in an event is registered.
In the C++ implementation, the methods for expressinginterest in a specific type of event were spread
across numerous classes. In the Python interface, all functions for registration of interest in events are
contained within the Agent base class. Any object interested in receiving notification of an event occurring is expected to derive from the Agent class.
The simplest type of notification isn’t really a real time event at all, but a variation on the concept of
scheduling a job with the dispatcher. Instead of calling the "schedule()" member function of the
dispatcher, the "scheduleAction()" member function of the Agent base class is called.
The major difference between using "scheduleAction()" and "schedule()" is that when using "scheduleAction()" you can optionally supply an additional string argument to be used as
an identifier for that job. This identifier can be used to cancel the job before it actually gets executed
by calling "cancelAction()". If thecallback funcion accepts a single argument, the identifierwill
also be passed to the callback function as argument. The identifier can thus be used to distinguish between different jobs calling the same callback function. If an identifier is not explicitly provided, a
unique internal identifier will be created. Whether or not the identifier is set explicitly or created internally, the identifier used is returned as the result of the "scheduleAction()" method.
class Object(netsvc.Agent):
def __init__(self):
self.scheduleAction(self.callback1,netsvc.PRIORITY_JOB)
def callback1(self):
self.scheduleAction(self.callback2,netsvc.IDLE_JOB,"hi")
self.scheduleAction(self.callback2,netsvc.IDLE_JOB,"bye")
def callback2(self,name):
print name
if name == "hi":
self.cancelAction("bye")
dispatcher = netsvc.Dispatcher()
object = Object()
dispatcher.run()
When using the Agent class, you still need to run the dispatcher. You do not need to schedule any
jobs directly with the dispactcher, but any initial agents need to be created prior to the dispatcher being
run. Note that in scheduling a job with a particular identifier, any job already scheduled with that agent
28
Destroying Agents
using the same identifier will first be cancelled. If you want to cancel all jobs scheduled using the
"scheduleAction()" member function you should call the "cancelAllActions()" member
function.
Destroying Agents
Ensuring that any outstanding job is cancelled, or deregistering interest in any event source, is important if you are endeavouring to destroy an agent object. If registrations are not cancelled, a circular reference will exist between data held by the instance of the Agent base class and the derived object.
Suchcircular references defeat the Python reference countingmechanism, meaning that theobject may
never be destroyed.
To combat this particular situation, the member function"destroyReferences()" is included in
the Agent base class.This will cancel all outstandingjobs and cancel any interest inother eventsources as well, destroying any circular references in the process. Provided there are no other references to
the object elsewhere, Python should now be able to destroy it.
If you have circular references within your derived class, you may wish to extend this method in your
own class so as to undo those circular references. Using the same member function name will make it
less confusing to a user of your class as they will only have to call one function. If this is done, you
should ensure however that the last thing the derived version of the method does is call the version of
the method in the immediate base class.
Alarms and Timers
Alarms and timers are a means of having a callback function executed at some point of time in the future. The difference between an alarm and a timer is that an alarm is defined by an absolute value or
point in time, where as a timer is defined by a relative offset in time. For an alarm this means supplying
the clock time in seconds at which the callback should be executed. For a timer this means supplying
the number of seconds from now at which point the callback should be executed.
class Object(netsvc.Agent):
def __init__(self):
offset = 60
now = time.time()
then = now + offset
self.setAlarm(self.callback1,then)
self.startTimer(self.callback2,offset,"timeout-1")
self.startTimer(self.callback2,offset+10,"timeout-2")
def callback1(self):
print "alarm"
def callback2(self,name):
print name
if name == "timeout-1":
self.cancelTimer("timeout-2")
29
Event Fr amework
The member function for setting an alarm is "setAlarm()" and that for starting a timer is "startTimer()".The first argument is the callback function, the second argument is the absolute or relative
time and the third argument is an optional identifier for that alarm or timer. Scheduling an alarm or
timer with an identifier matching that of an alarm or timer which hasn’t yet expired will cause that unexpired alarm or timer to be cancelled.
Bothtypes of events are one off events, with the registrationbeingcancelled oncethe callback has been
executed. The identifier may also be used to cancel an alarm or timer before it expires. To cancel an
alarm use "cancelAlarm()" and to cancel a timer use "cancelTimer()". To cancel all pending
alarms use "cancelAllAlarms()" and to cancel all pending timers use "cancelAllTim-
ers()". If an identifier is not excplicitly provided, an internal identifier will be automatically created
with it being returned as the result of the function being called to schedule the callback.
Recurring Actions
A recurring action is where a job is run at regular intervals. Precisely when the callback function associatedwith a jobis executed isdetermined by a s pecification of theform used by the UNIX cron utility.
The specification consists of five fields each separated by white space. The fields specify:
• minute (0-59),
• hour (0-23),
• day of the month (1-31),
• month of the year (1-12),
• day of the week (0-6 with 0=Sunday).
A field may be an asterisk "*", which always stands for "first-last". Ranges of numbers are al-
lowed. Ranges are two numbers separated with a hyphen. The specified range is inclusive. For example, 8-11 for an "hours" entry specifies execution at hours 8, 9, 10 and 11.
Lists are allowed. A list is a set of numbers (or ranges) separated by commas. For example,
"1,2,5,9" and "0-4,8-12". Step values can be used in conjunction with ranges. Followinga range
with "/number" specifies skips of the number’s value through the range. For example, "0-23/2"
can be used in the hours field to specify the callback function be executed every other hour. Steps are
also permitted after an asterisk, so if you want to say "every two hours", just use "*/2".
Names can also be used for the "month" and "day of week" fields. Use the first three letters of the particular day or month (lower case, or first letter only uppercase).
The day that a callbackfunction is to be executed can be specified by two fields, day of month and day
of week. If both fields arerestricted (ie.,aren’t "*"),the callback functionwill be executedwhen either
field matches the current time. For example, "30 4 1,15 * 5" would cause the callback function
to be executed at 4:30 am on the 1st and 15th of each month, plus every Friday.
To schedulethis type of job, the "scheduleAction()"function is used except that instead of specifying the job type as the second argument, the specification string should be used.
30
Loading...