Advert Management¶

SAGA’s advert module supports the coordination of distributed application components, by exposing and sharing application specific pieces of information between application instances, and by creating and delivering notifications on changes of those information.

The basic usage of the advert module is as follows:

class my_cb (radical.saga.Callback) :

  def cb (self, obj, key, val) :
    print " ----------- callback triggered for %s - %s - %s" % (obj, key, val)
    return True


ad = saga.advert.Directory ("redis://redishost.net/tmp/myapp/%d" % os.getpid (),
                                saga.filesystem.CREATE | saga.filesystem.CREATE_PARENTS)
ad.set_attribute ('start',      time.time())
ad.set_attribute ('user',       os.get_uid())
ad.set_attribute ('iteration',  0)

ad.add_callback  ('iteration', my_cb ())
ad.set_attribute ('iteration', 1)

See also

More examples on how to use the SAGA advert module can be found in the Advert Module section of the chapter_code_examples chapter.

Like all SAGA modules, the advert module relies on middleware adaptors to provide bindings to a specific backend system. Adaptors are implicitly selected via the scheme part of the URL, e.g., redis:// in the example above selects the redis backend.

Note

A list of available adaptors and supported backends can be found in the Developer Documentation part of this documentation.

The rest of this section is structured as follows:

class radical.saga.advert.Entry(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]¶

Bases: Entry, Attributes

add_callback(key, cb)¶

For any attribute change, the API will check if any callbacks are registered for that attribute. If so, those callbacks will be called in order of registration. This registration function will return an id (cookie) identifying the callback – that id can be used to remove the callback.

A callback is any callable python construct, and MUST accept three arguments:

- STRING key: the name of the attribute which changed
- ANY    val: the new value of the attribute
- ANY    obj: the object on which this attribute interface was called

The ‘obj’ can be any python object type, but is guaranteed to expose this attribute interface.

The callback SHOULD return ‘True’ or ‘False’ – on ‘True’, the callback will remain registered, and will thus be called again on the next attribute change. On returning ‘False’, the callback will be unregistered, and will thus not be called again. Returning nothing is interpreted as ‘False’, other return values lead to undefined behavior.

Note that callbacks will not be called on ‘Final’ attributes (they will be called once as that attribute enters finality).

as_dict()¶: return a dict representation of all set attributes

attribute_exists(key, _flow='_down')¶

attribute_exist (key)

This method will check if the given key is known and was set explicitly. The call will also return ‘True’ if the value for that key is ‘None’.

attribute_is_readonly(key)¶: This method will check if the given key is readonly, i.e. cannot be ‘set’. The call will also return ‘True’ if the attribute is final

attribute_is_removable(key, _flow='_down')¶

attribute_is_writeable (key)

This method will check if the given key can be removed.

attribute_is_vector(key)¶: This method will check if the given attribute has a vector value type.

attribute_is_writeable(key)¶: This method will check if the given key is writeable - i.e. not readonly.

clear() → None. Remove all items from D.¶

close(timeout=None, ttype=None)[source]¶: timeout: float ttype: saga.task.type enum ret: None / saga.Task

copy(tgt, flags=0, ttype=None)[source]¶

tgt: saga.Url flags: enum flags ttype: saga.task.type enum ret: None / saga.Task

Copy the entry to another location

Parameters:

target – Url of the copy target.
flags – Flags to use for the operation.

The entry is copied to the given target location. The target URL must be an absolute path, and can be a target entry name or target directory name. If the target entry exists, it is overwritten:

# copy an entry
entry = saga.namespace.Entry("sftp://localhost/tmp/data/data.bin")
entry.copy ("sftp://localhost/tmp/data/data.bak")

classmethod create(url=None, flags=512, session=None, ttype=None)[source]¶: url: saga.Url flags: saga.advert.flags enum session: saga.Session ttype: saga.task.type enum ret: saga.Task

property cwd¶: ttype: saga.task.type enum ret: string / saga.Task

delete_object(ttype=None)[source]¶: ttype: saga.task.type enum ret: None / saga.Task

find_attributes(pattern)¶: Similar to list(), but also grep for a given attribute pattern. That pattern is of the form ‘key=val’, where both ‘key’ and ‘val’ can contain POSIX shell wildcards. For non-string typed attributes, the pattern is applied to a string serialization of the typed value, if that exists.

from_dict(seed)¶: set attributes from dict

fromkeys(value=None, /)¶: Create a new dictionary with keys from iterable and values set to value.

get(key, default=None)¶: Return the value for key if key is in the dictionary, else default.

get_attribute(key)¶: This method returns the value of the specified attribute. If that attribute does not exist, an DoesNotExist is raised. It is not an error to query an existing, but unset attribute though – that will result in ‘None’ to be returned (or the default value, if available).

get_cwd(ttype=None)[source]¶: ttype: saga.task.type enum ret: string / saga.Task

get_name(ttype=None)[source]¶: ttype: saga.task.type enum ret: string / saga.Task

get_session()¶

Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.

The object’s session is also available via the session property.

get_ttl(ttype=None)[source]¶: ttype: saga.task.type enum ret: int / saga.Task

get_url(ttype=None)[source]¶

ttype: saga.task.type enum ret: saga.Url / saga.Task

Return the complete url pointing to the entry.

The call will return the complete url pointing to this entry as a saga.Url object:

# print URL of an entry
entry = saga.namespace.Entry("sftp://localhost/etc/passwd")
print(entry.get_url())

get_vector_attribute(key)¶

See also: saga.Attributes.get_attribute() (key).

As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the get_attribute method.

has_key(key)¶

is_dir(ttype=None)[source]¶

ttype: saga.task.type enum ret: bool / saga.Task

Returns True if path is a directory, False otherwise.

Example:

# inspect an entry
dir  = saga.namespace.Directory("sftp://localhost/tmp/")
if dir.is_dir ('data'):
    # do something

is_entry(ttype=None)[source]¶: ttype: saga.task.type enum ret: bool / saga.Task

is_link(ttype=None)[source]¶: tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task

items() → a set-like object providing a view on D's items¶

iterkeys()¶

itervalues()¶

keys() → a set-like object providing a view on D's keys¶

link(tgt, flags=0, ttype=None)[source]¶: tgt: saga.Url flags: enum flags ttype: saga.task.type enum ret: None / saga.Task

list_attributes()¶: List all attributes which have been explicitly set.

move(tgt, flags=0, ttype=None)[source]¶

Parameters:

target – Url of the move target.
flags – Flags to use for the operation.

ttype: saga.task.type enum ret: None / saga.Task

Move the entry to another location

The entry is copied to the given target location. The target URL must be an absolute path, and can be a target entry name or target directory name. If the target entry exists, it is overwritten:

# copy an entry
entry = rs.namespace.Directory("sftp://localhost/tmp/data/data.bin")
entry.move ("sftp://localhost/tmp/data/data.bak")

property name¶: ttype: saga.task.type enum ret: string / saga.Task

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, default is returned if given, otherwise KeyError is raised

popitem()¶

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

read_link(ttype=None)[source]¶: tgt: saga.Url / None ttype: saga.task.type enum ret: saga.Url / saga.Task

remove(flags=0, ttype=None)[source]¶

Parameters:: flags – Flags to use for the operation.

ttype: saga.task.type enum ret: None / saga.Task

Reove the entry.

The entry is removed, and this object instance is then invalid for further operations.

# remove an entry entry = rs.namespace.Directory(“sftp://localhost/tmp/data/data.bin”) entry.remove ()

remove_attribute(key)¶: Removing an attribute is actually different from unsetting it, or from setting it to ‘None’. On remove, all traces of the attribute are purged, and the key will not be listed on saga.Attributes.list_attributes() () anymore.

remove_callback(key, id)¶

This method allows to unregister a previously registered callback, by providing its id. It is not an error to remove a non-existing cb, but a valid ID MUST be provided – otherwise, a BadParameter is raised.

If no ID is provided (id == None), all callbacks are removed for this attribute.

retrieve_object(ttype=None)[source]¶: ttype: saga.task.type enum ret: any / saga.Task

property session¶

Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.

The object’s session is also available via the session property.

set_attribute(key, val)¶

This method sets the value of the specified attribute. If that attribute does not exist, DoesNotExist is raised – unless the attribute set is marked ‘extensible’ or ‘private’. In that case, the attribute is created and set on the fly (defaulting to mode=writeable, flavor=Scalar, type=ANY, default=None). A value of ‘None’ may reset the attribute to its default value, if such one exists (see documentation).

Note that this method is performing a number of checks and conversions, to match the value type to the attribute properties (type, mode, flavor). Those conversions are not guaranteed to yield the expected result – for example, the conversion from ‘scalar’ to ‘vector’ is, for complex types, ambiguous at best, and somewhat stupid. The consumer of the API SHOULD ensure correct attribute values. The conversions are intended to support the most trivial and simple use cases (int to string etc). Failed conversions will result in an BadParameter exception.

Attempts to set a ‘final’ attribute are silently ignored. Attempts to set a ‘readonly’ attribute will result in an IncorrectState exception being raised.

Note that set_attribute() will trigger callbacks, if a new value (different from the old value) is given.

set_ttl(ttl=-1.0, ttype=None)[source]¶: ttl : int ttype: saga.task.type enum ret: None / saga.Task

set_vector_attribute(key, val)¶

See also: saga.Attributes.set_attribute() (key, val).

As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the set_attribute method.

setdefault(key, default)¶

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

store_object(object, ttype=None)[source]¶: object : <object type> ttype: saga.task.type enum ret: None / saga.Task

update([E, ]**F) → None. Update D from dict/iterable E and F.¶: If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

property url¶

ttype: saga.task.type enum ret: saga.Url / saga.Task

Return the complete url pointing to the entry.

The call will return the complete url pointing to this entry as a saga.Url object:

# print URL of an entry
entry = saga.namespace.Entry("sftp://localhost/etc/passwd")
print(entry.get_url())

values() → an object providing a view on D's values¶

class radical.saga.advert.Directory(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]¶

Bases: Directory, Attributes

add_callback(key, cb)¶

For any attribute change, the API will check if any callbacks are registered for that attribute. If so, those callbacks will be called in order of registration. This registration function will return an id (cookie) identifying the callback – that id can be used to remove the callback.

A callback is any callable python construct, and MUST accept three arguments:

- STRING key: the name of the attribute which changed
- ANY    val: the new value of the attribute
- ANY    obj: the object on which this attribute interface was called

The ‘obj’ can be any python object type, but is guaranteed to expose this attribute interface.

The callback SHOULD return ‘True’ or ‘False’ – on ‘True’, the callback will remain registered, and will thus be called again on the next attribute change. On returning ‘False’, the callback will be unregistered, and will thus not be called again. Returning nothing is interpreted as ‘False’, other return values lead to undefined behavior.

Note that callbacks will not be called on ‘Final’ attributes (they will be called once as that attribute enters finality).

as_dict()¶: return a dict representation of all set attributes

attribute_exists(key, _flow='_down')¶

attribute_exist (key)

This method will check if the given key is known and was set explicitly. The call will also return ‘True’ if the value for that key is ‘None’.

attribute_is_readonly(key)¶: This method will check if the given key is readonly, i.e. cannot be ‘set’. The call will also return ‘True’ if the attribute is final

attribute_is_removable(key, _flow='_down')¶

attribute_is_writeable (key)

This method will check if the given key can be removed.

attribute_is_vector(key)¶: This method will check if the given attribute has a vector value type.

attribute_is_writeable(key)¶: This method will check if the given key is writeable - i.e. not readonly.

change_dir(url, flags=0, ttype=None)[source]¶: url: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

clear() → None. Remove all items from D.¶

close(timeout=None, ttype=None)¶: timeout: float ttype: saga.task.type enum ret: None / saga.Task

copy(url_1, url_2=None, flags=0, ttype=None)[source]¶

Parameters:

src – path of the entry to copy
tgt – absolute URL of target name or directory

url_1: saga.Url url_2: saga.Url / None flags: flags enum / None ttype: saga.task.type enum / None ret: None / saga.Task

Copy an entry from source to target

The source is copied to the given target directory. The path of the source can be relative:

# copy an entry
dir = saga.namespace.Directory("sftp://localhost/tmp/")
dir.copy ("./data.bin", "sftp://localhost/tmp/data/")

classmethod create(url=None, flags=512, session=None, ttype=None)[source]¶: url: saga.Url flags: saga.advert.flags enum session: saga.Session ttype: saga.task.type enum ret: saga.Task

property cwd¶: ttype: saga.task.type enum ret: string / saga.Task

exists(path, ttype=None)[source]¶

Parameters:: path – path of the entry to check

ttype: saga.task.type enum ret: bool / saga.Task

Returns True if path exists, False otherwise.

Example:

# inspect an entry
dir  = saga.namespace.Directory("sftp://localhost/tmp/")
if dir.exists ('data'):
    # do something

find(name_pattern, attr_pattern=None, obj_type=None, flags=2, ttype=None)[source]¶: name_pattern: string attr_pattern: string obj_type: string flags: flags enum ret: list [saga.Url]

find_attributes(pattern)¶: Similar to list(), but also grep for a given attribute pattern. That pattern is of the form ‘key=val’, where both ‘key’ and ‘val’ can contain POSIX shell wildcards. For non-string typed attributes, the pattern is applied to a string serialization of the typed value, if that exists.

from_dict(seed)¶: set attributes from dict

fromkeys(value=None, /)¶: Create a new dictionary with keys from iterable and values set to value.

get(key, default=None)¶: Return the value for key if key is in the dictionary, else default.

get_attribute(key)¶: This method returns the value of the specified attribute. If that attribute does not exist, an DoesNotExist is raised. It is not an error to query an existing, but unset attribute though – that will result in ‘None’ to be returned (or the default value, if available).

get_cwd(ttype=None)¶: ttype: saga.task.type enum ret: string / saga.Task

get_entry(num, ttype=None)[source]¶: num: int ttype: saga.task.type enum ret: saga.Url / saga.Task

get_name(ttype=None)¶: ttype: saga.task.type enum ret: string / saga.Task

get_num_entries(ttype=None)[source]¶: ttype: saga.task.type enum ret: int / saga.Task

get_session()¶

Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.

The object’s session is also available via the session property.

get_ttl(tgt=None, ttype=None)[source]¶: tgt : saga.Url / None ttype: saga.task.type enum ret: int / saga.Task

get_url(ttype=None)¶

ttype: saga.task.type enum ret: saga.Url / saga.Task

Return the complete url pointing to the entry.

The call will return the complete url pointing to this entry as a saga.Url object:

# print URL of an entry
entry = saga.namespace.Entry("sftp://localhost/etc/passwd")
print(entry.get_url())

get_vector_attribute(key)¶

See also: saga.Attributes.get_attribute() (key).

As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the get_attribute method.

has_key(key)¶

is_dir(tgt=None, ttype=None)[source]¶

tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task

Returns True if path is a directory, False otherwise.

Example:

# inspect an entry
dir  = saga.namespace.Directory("sftp://localhost/tmp/")
if dir.is_dir ('data'):
    # do something

is_entry(tgt=None, ttype=None)[source]¶: tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task

is_link(tgt=None, ttype=None)[source]¶: tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task

items() → a set-like object providing a view on D's items¶

iterkeys()¶

itervalues()¶

keys() → a set-like object providing a view on D's keys¶

link(url_1, url_2=None, flags=0, ttype=None)[source]¶: src: saga.Url tgt: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

list(pattern=None, flags=0, ttype=None)[source]¶

Parameters:: pattern – Entry name pattern (like POSIX ‘ls’, e.g. ‘*.txt’)

flags: flags enum ttype: saga.task.type enum ret: list [saga.Url] / saga.Task

List the directory’s content

The call will return a list of entries and subdirectories within the directory:

# list contents of the directory
for f in dir.list() :
    print(f)

list_attributes()¶: List all attributes which have been explicitly set.

make_dir(tgt, flags=0, ttype=None)[source]¶

Parameters:

tgt – name/path of the new directory
flags – directory creation flags

ttype: saga.task.type enum ret: None / saga.Task

Create a new directoy

The call creates a directory at the given location.

Example:

# create a subdir 'data' in /tmp
dir = saga.namespace.Directory("sftp://localhost/tmp/")
dir.make_dir ('data/')

move(url_1, url_2=None, flags=0, ttype=None)[source]¶

Parameters:

src – path of the entry to copy
tgt – absolute URL of target directory

flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

Move an entry from source to target

The source is moved to the given target directory. The path of the source can be relative:

# copy an entry
dir = saga.namespace.Directory("sftp://localhost/tmp/")
dir.move ("./data.bin", "sftp://localhost/tmp/data/")

property name¶: ttype: saga.task.type enum ret: string / saga.Task

open(name, flags=None, ttype=None)[source]¶: name: saga.Url flags: saga.namespace.flags enum ttype: saga.task.type enum ret: saga.namespace.Entry / saga.Task

open_dir(path, flags=None, ttype=None)[source]¶

Parameters:

path – name/path of the directory to open
flags – directory creation flags

ttype: saga.task.type enum ret: saga.namespace.Directory / saga.Task

Open and return a new directoy

The call opens and returns a directory at the given location.

Example:

# create a subdir 'data' in /tmp
dir = saga.namespace.Directory("sftp://localhost/tmp/")
data = dir.open_dir ('data/', saga.namespace.Create)

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, default is returned if given, otherwise KeyError is raised

popitem()¶

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

read_link(tgt=None, ttype=None)[source]¶: tgt: saga.Url / None ttype: saga.task.type enum ret: saga.Url / saga.Task

remove(tgt=None, flags=0, ttype=None)[source]¶: tgt: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

remove_attribute(key)¶: Removing an attribute is actually different from unsetting it, or from setting it to ‘None’. On remove, all traces of the attribute are purged, and the key will not be listed on saga.Attributes.list_attributes() () anymore.

remove_callback(key, id)¶

This method allows to unregister a previously registered callback, by providing its id. It is not an error to remove a non-existing cb, but a valid ID MUST be provided – otherwise, a BadParameter is raised.

If no ID is provided (id == None), all callbacks are removed for this attribute.

property session¶

Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.

The object’s session is also available via the session property.

set_attribute(key, val)¶

This method sets the value of the specified attribute. If that attribute does not exist, DoesNotExist is raised – unless the attribute set is marked ‘extensible’ or ‘private’. In that case, the attribute is created and set on the fly (defaulting to mode=writeable, flavor=Scalar, type=ANY, default=None). A value of ‘None’ may reset the attribute to its default value, if such one exists (see documentation).

Note that this method is performing a number of checks and conversions, to match the value type to the attribute properties (type, mode, flavor). Those conversions are not guaranteed to yield the expected result – for example, the conversion from ‘scalar’ to ‘vector’ is, for complex types, ambiguous at best, and somewhat stupid. The consumer of the API SHOULD ensure correct attribute values. The conversions are intended to support the most trivial and simple use cases (int to string etc). Failed conversions will result in an BadParameter exception.

Attempts to set a ‘final’ attribute are silently ignored. Attempts to set a ‘readonly’ attribute will result in an IncorrectState exception being raised.

Note that set_attribute() will trigger callbacks, if a new value (different from the old value) is given.

set_ttl(tgt=None, ttl=-1.0, ttype=None)[source]¶: tgt : saga.Url / None ttl : int ttype: saga.task.type enum ret: None / saga.Task

set_vector_attribute(key, val)¶

See also: saga.Attributes.set_attribute() (key, val).

As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the set_attribute method.

setdefault(key, default)¶

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None. Update D from dict/iterable E and F.¶: If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

property url¶

ttype: saga.task.type enum ret: saga.Url / saga.Task

Return the complete url pointing to the entry.

The call will return the complete url pointing to this entry as a saga.Url object:

# print URL of an entry
entry = saga.namespace.Entry("sftp://localhost/etc/passwd")
print(entry.get_url())

values() → an object providing a view on D's values¶

Table of Contents

This Page