Advert Management¶
SAGA’s advert module supports the coordination of distributed application components, by exposing and sharing application specific pieces of information between application instances, and by creating and delivering notifications on changes of those information.
The basic usage of the advert module is as follows:
class my_cb (radical.saga.Callback) :
def cb (self, obj, key, val) :
print " ----------- callback triggered for %s - %s - %s" % (obj, key, val)
return True
ad = saga.advert.Directory ("redis://redishost.net/tmp/myapp/%d" % os.getpid (),
saga.filesystem.CREATE | saga.filesystem.CREATE_PARENTS)
ad.set_attribute ('start', time.time())
ad.set_attribute ('user', os.get_uid())
ad.set_attribute ('iteration', 0)
ad.add_callback ('iteration', my_cb ())
ad.set_attribute ('iteration', 1)
See also
More examples on how to use the SAGA advert module can be found in the Advert Module section of the chapter_code_examples chapter.
Like all SAGA modules, the advert module relies on middleware adaptors
to provide bindings to a specific backend system. Adaptors are implicitly
selected via the scheme part of the URL, e.g., redis://
in the example
above selects the redis backend.
Note
A list of available adaptors and supported backends can be found in the Developer Documentation part of this documentation.
The rest of this section is structured as follows:
- class radical.saga.advert.Entry(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]¶
Bases:
Entry
,Attributes
- add_callback(key, cb)¶
For any attribute change, the API will check if any callbacks are registered for that attribute. If so, those callbacks will be called in order of registration. This registration function will return an id (cookie) identifying the callback – that id can be used to remove the callback.
A callback is any callable python construct, and MUST accept three arguments:
- STRING key: the name of the attribute which changed - ANY val: the new value of the attribute - ANY obj: the object on which this attribute interface was called
The ‘obj’ can be any python object type, but is guaranteed to expose this attribute interface.
The callback SHOULD return ‘True’ or ‘False’ – on ‘True’, the callback will remain registered, and will thus be called again on the next attribute change. On returning ‘False’, the callback will be unregistered, and will thus not be called again. Returning nothing is interpreted as ‘False’, other return values lead to undefined behavior.
Note that callbacks will not be called on ‘Final’ attributes (they will be called once as that attribute enters finality).
- as_dict()¶
return a dict representation of all set attributes
- attribute_exists(key, _flow='_down')¶
attribute_exist (key)
This method will check if the given key is known and was set explicitly. The call will also return ‘True’ if the value for that key is ‘None’.
- attribute_is_readonly(key)¶
This method will check if the given key is readonly, i.e. cannot be ‘set’. The call will also return ‘True’ if the attribute is final
- attribute_is_removable(key, _flow='_down')¶
attribute_is_writeable (key)
This method will check if the given key can be removed.
- attribute_is_vector(key)¶
This method will check if the given attribute has a vector value type.
- attribute_is_writeable(key)¶
This method will check if the given key is writeable - i.e. not readonly.
- clear() None. Remove all items from D. ¶
- close(timeout=None, ttype=None)[source]¶
timeout: float ttype: saga.task.type enum ret: None / saga.Task
- copy(tgt, flags=0, ttype=None)[source]¶
tgt: saga.Url flags: enum flags ttype: saga.task.type enum ret: None / saga.Task
Copy the entry to another location
- Parameters:
target – Url of the copy target.
flags – Flags to use for the operation.
The entry is copied to the given target location. The target URL must be an absolute path, and can be a target entry name or target directory name. If the target entry exists, it is overwritten:
# copy an entry entry = saga.namespace.Entry("sftp://localhost/tmp/data/data.bin") entry.copy ("sftp://localhost/tmp/data/data.bak")
- classmethod create(url=None, flags=512, session=None, ttype=None)[source]¶
url: saga.Url flags: saga.advert.flags enum session: saga.Session ttype: saga.task.type enum ret: saga.Task
- property cwd¶
ttype: saga.task.type enum ret: string / saga.Task
- find_attributes(pattern)¶
Similar to list(), but also grep for a given attribute pattern. That pattern is of the form ‘key=val’, where both ‘key’ and ‘val’ can contain POSIX shell wildcards. For non-string typed attributes, the pattern is applied to a string serialization of the typed value, if that exists.
- from_dict(seed)¶
set attributes from dict
- fromkeys(value=None, /)¶
Create a new dictionary with keys from iterable and values set to value.
- get(key, default=None)¶
Return the value for key if key is in the dictionary, else default.
- get_attribute(key)¶
This method returns the value of the specified attribute. If that attribute does not exist, an DoesNotExist is raised. It is not an error to query an existing, but unset attribute though – that will result in ‘None’ to be returned (or the default value, if available).
- get_session()¶
Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.
The object’s session is also available via the session property.
- get_url(ttype=None)[source]¶
ttype: saga.task.type enum ret: saga.Url / saga.Task
Return the complete url pointing to the entry.
The call will return the complete url pointing to this entry as a saga.Url object:
# print URL of an entry entry = saga.namespace.Entry("sftp://localhost/etc/passwd") print(entry.get_url())
- get_vector_attribute(key)¶
See also:
saga.Attributes.get_attribute()
(key).As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the get_attribute method.
- has_key(key)¶
- is_dir(ttype=None)[source]¶
ttype: saga.task.type enum ret: bool / saga.Task
Returns True if path is a directory, False otherwise.
Example:
# inspect an entry dir = saga.namespace.Directory("sftp://localhost/tmp/") if dir.is_dir ('data'): # do something
- items() a set-like object providing a view on D's items ¶
- iterkeys()¶
- itervalues()¶
- keys() a set-like object providing a view on D's keys ¶
- link(tgt, flags=0, ttype=None)[source]¶
tgt: saga.Url flags: enum flags ttype: saga.task.type enum ret: None / saga.Task
- list_attributes()¶
List all attributes which have been explicitly set.
- move(tgt, flags=0, ttype=None)[source]¶
- Parameters:
target – Url of the move target.
flags – Flags to use for the operation.
ttype: saga.task.type enum ret: None / saga.Task
Move the entry to another location
The entry is copied to the given target location. The target URL must be an absolute path, and can be a target entry name or target directory name. If the target entry exists, it is overwritten:
# copy an entry entry = rs.namespace.Directory("sftp://localhost/tmp/data/data.bin") entry.move ("sftp://localhost/tmp/data/data.bak")
- property name¶
ttype: saga.task.type enum ret: string / saga.Task
- pop(k[, d]) v, remove specified key and return the corresponding value. ¶
If key is not found, d is returned if given, otherwise KeyError is raised
- popitem() (k, v), remove and return some (key, value) pair as a ¶
2-tuple; but raise KeyError if D is empty.
- read_link(ttype=None)[source]¶
tgt: saga.Url / None ttype: saga.task.type enum ret: saga.Url / saga.Task
- remove(flags=0, ttype=None)[source]¶
- Parameters:
flags – Flags to use for the operation.
ttype: saga.task.type enum ret: None / saga.Task
Reove the entry.
The entry is removed, and this object instance is then invalid for further operations.
# remove an entry entry = rs.namespace.Directory(“sftp://localhost/tmp/data/data.bin”) entry.remove ()
- remove_attribute(key)¶
Removing an attribute is actually different from unsetting it, or from setting it to ‘None’. On remove, all traces of the attribute are purged, and the key will not be listed on
saga.Attributes.list_attributes()
() anymore.
- remove_callback(key, id)¶
This method allows to unregister a previously registered callback, by providing its id. It is not an error to remove a non-existing cb, but a valid ID MUST be provided – otherwise, a BadParameter is raised.
If no ID is provided (id == None), all callbacks are removed for this attribute.
- property session¶
Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.
The object’s session is also available via the session property.
- set_attribute(key, val)¶
This method sets the value of the specified attribute. If that attribute does not exist, DoesNotExist is raised – unless the attribute set is marked ‘extensible’ or ‘private’. In that case, the attribute is created and set on the fly (defaulting to mode=writeable, flavor=Scalar, type=ANY, default=None). A value of ‘None’ may reset the attribute to its default value, if such one exists (see documentation).
Note that this method is performing a number of checks and conversions, to match the value type to the attribute properties (type, mode, flavor). Those conversions are not guaranteed to yield the expected result – for example, the conversion from ‘scalar’ to ‘vector’ is, for complex types, ambiguous at best, and somewhat stupid. The consumer of the API SHOULD ensure correct attribute values. The conversions are intended to support the most trivial and simple use cases (int to string etc). Failed conversions will result in an BadParameter exception.
Attempts to set a ‘final’ attribute are silently ignored. Attempts to set a ‘readonly’ attribute will result in an IncorrectState exception being raised.
Note that set_attribute() will trigger callbacks, if a new value (different from the old value) is given.
- set_vector_attribute(key, val)¶
See also:
saga.Attributes.set_attribute()
(key, val).As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the set_attribute method.
- setdefault(key, default)¶
Insert key with a value of default if key is not in the dictionary.
Return the value for key if key is in the dictionary, else default.
- store_object(object, ttype=None)[source]¶
object : <object type> ttype: saga.task.type enum ret: None / saga.Task
- update([E, ]**F) None. Update D from dict/iterable E and F. ¶
If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
- property url¶
ttype: saga.task.type enum ret: saga.Url / saga.Task
Return the complete url pointing to the entry.
The call will return the complete url pointing to this entry as a saga.Url object:
# print URL of an entry entry = saga.namespace.Entry("sftp://localhost/etc/passwd") print(entry.get_url())
- values() an object providing a view on D's values ¶
- class radical.saga.advert.Directory(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]¶
Bases:
Directory
,Attributes
- add_callback(key, cb)¶
For any attribute change, the API will check if any callbacks are registered for that attribute. If so, those callbacks will be called in order of registration. This registration function will return an id (cookie) identifying the callback – that id can be used to remove the callback.
A callback is any callable python construct, and MUST accept three arguments:
- STRING key: the name of the attribute which changed - ANY val: the new value of the attribute - ANY obj: the object on which this attribute interface was called
The ‘obj’ can be any python object type, but is guaranteed to expose this attribute interface.
The callback SHOULD return ‘True’ or ‘False’ – on ‘True’, the callback will remain registered, and will thus be called again on the next attribute change. On returning ‘False’, the callback will be unregistered, and will thus not be called again. Returning nothing is interpreted as ‘False’, other return values lead to undefined behavior.
Note that callbacks will not be called on ‘Final’ attributes (they will be called once as that attribute enters finality).
- as_dict()¶
return a dict representation of all set attributes
- attribute_exists(key, _flow='_down')¶
attribute_exist (key)
This method will check if the given key is known and was set explicitly. The call will also return ‘True’ if the value for that key is ‘None’.
- attribute_is_readonly(key)¶
This method will check if the given key is readonly, i.e. cannot be ‘set’. The call will also return ‘True’ if the attribute is final
- attribute_is_removable(key, _flow='_down')¶
attribute_is_writeable (key)
This method will check if the given key can be removed.
- attribute_is_vector(key)¶
This method will check if the given attribute has a vector value type.
- attribute_is_writeable(key)¶
This method will check if the given key is writeable - i.e. not readonly.
- change_dir(url, flags=0, ttype=None)[source]¶
url: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task
- clear() None. Remove all items from D. ¶
- close(timeout=None, ttype=None)¶
timeout: float ttype: saga.task.type enum ret: None / saga.Task
- copy(url_1, url_2=None, flags=0, ttype=None)[source]¶
- Parameters:
src – path of the entry to copy
tgt – absolute URL of target name or directory
url_1: saga.Url url_2: saga.Url / None flags: flags enum / None ttype: saga.task.type enum / None ret: None / saga.Task
Copy an entry from source to target
The source is copied to the given target directory. The path of the source can be relative:
# copy an entry dir = saga.namespace.Directory("sftp://localhost/tmp/") dir.copy ("./data.bin", "sftp://localhost/tmp/data/")
- classmethod create(url=None, flags=512, session=None, ttype=None)[source]¶
url: saga.Url flags: saga.advert.flags enum session: saga.Session ttype: saga.task.type enum ret: saga.Task
- property cwd¶
ttype: saga.task.type enum ret: string / saga.Task
- exists(path, ttype=None)[source]¶
- Parameters:
path – path of the entry to check
ttype: saga.task.type enum ret: bool / saga.Task
Returns True if path exists, False otherwise.
Example:
# inspect an entry dir = saga.namespace.Directory("sftp://localhost/tmp/") if dir.exists ('data'): # do something
- find(name_pattern, attr_pattern=None, obj_type=None, flags=2, ttype=None)[source]¶
name_pattern: string attr_pattern: string obj_type: string flags: flags enum ret: list [saga.Url]
- find_attributes(pattern)¶
Similar to list(), but also grep for a given attribute pattern. That pattern is of the form ‘key=val’, where both ‘key’ and ‘val’ can contain POSIX shell wildcards. For non-string typed attributes, the pattern is applied to a string serialization of the typed value, if that exists.
- from_dict(seed)¶
set attributes from dict
- fromkeys(value=None, /)¶
Create a new dictionary with keys from iterable and values set to value.
- get(key, default=None)¶
Return the value for key if key is in the dictionary, else default.
- get_attribute(key)¶
This method returns the value of the specified attribute. If that attribute does not exist, an DoesNotExist is raised. It is not an error to query an existing, but unset attribute though – that will result in ‘None’ to be returned (or the default value, if available).
- get_cwd(ttype=None)¶
ttype: saga.task.type enum ret: string / saga.Task
- get_name(ttype=None)¶
ttype: saga.task.type enum ret: string / saga.Task
- get_session()¶
Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.
The object’s session is also available via the session property.
- get_ttl(tgt=None, ttype=None)[source]¶
tgt : saga.Url / None ttype: saga.task.type enum ret: int / saga.Task
- get_url(ttype=None)¶
ttype: saga.task.type enum ret: saga.Url / saga.Task
Return the complete url pointing to the entry.
The call will return the complete url pointing to this entry as a saga.Url object:
# print URL of an entry entry = saga.namespace.Entry("sftp://localhost/etc/passwd") print(entry.get_url())
- get_vector_attribute(key)¶
See also:
saga.Attributes.get_attribute()
(key).As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the get_attribute method.
- has_key(key)¶
- is_dir(tgt=None, ttype=None)[source]¶
tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task
Returns True if path is a directory, False otherwise.
Example:
# inspect an entry dir = saga.namespace.Directory("sftp://localhost/tmp/") if dir.is_dir ('data'): # do something
- is_entry(tgt=None, ttype=None)[source]¶
tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task
- is_link(tgt=None, ttype=None)[source]¶
tgt: saga.Url / None ttype: saga.task.type enum ret: bool / saga.Task
- items() a set-like object providing a view on D's items ¶
- iterkeys()¶
- itervalues()¶
- keys() a set-like object providing a view on D's keys ¶
- link(url_1, url_2=None, flags=0, ttype=None)[source]¶
src: saga.Url tgt: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task
- list(pattern=None, flags=0, ttype=None)[source]¶
- Parameters:
pattern – Entry name pattern (like POSIX ‘ls’, e.g. ‘*.txt’)
flags: flags enum ttype: saga.task.type enum ret: list [saga.Url] / saga.Task
List the directory’s content
The call will return a list of entries and subdirectories within the directory:
# list contents of the directory for f in dir.list() : print(f)
- list_attributes()¶
List all attributes which have been explicitly set.
- make_dir(tgt, flags=0, ttype=None)[source]¶
- Parameters:
tgt – name/path of the new directory
flags – directory creation flags
ttype: saga.task.type enum ret: None / saga.Task
Create a new directoy
The call creates a directory at the given location.
Example:
# create a subdir 'data' in /tmp dir = saga.namespace.Directory("sftp://localhost/tmp/") dir.make_dir ('data/')
- move(url_1, url_2=None, flags=0, ttype=None)[source]¶
- Parameters:
src – path of the entry to copy
tgt – absolute URL of target directory
flags: flags enum ttype: saga.task.type enum ret: None / saga.Task
Move an entry from source to target
The source is moved to the given target directory. The path of the source can be relative:
# copy an entry dir = saga.namespace.Directory("sftp://localhost/tmp/") dir.move ("./data.bin", "sftp://localhost/tmp/data/")
- property name¶
ttype: saga.task.type enum ret: string / saga.Task
- open(name, flags=None, ttype=None)[source]¶
name: saga.Url flags: saga.namespace.flags enum ttype: saga.task.type enum ret: saga.namespace.Entry / saga.Task
- open_dir(path, flags=None, ttype=None)[source]¶
- Parameters:
path – name/path of the directory to open
flags – directory creation flags
ttype: saga.task.type enum ret: saga.namespace.Directory / saga.Task
Open and return a new directoy
The call opens and returns a directory at the given location.
Example:
# create a subdir 'data' in /tmp dir = saga.namespace.Directory("sftp://localhost/tmp/") data = dir.open_dir ('data/', saga.namespace.Create)
- pop(k[, d]) v, remove specified key and return the corresponding value. ¶
If key is not found, d is returned if given, otherwise KeyError is raised
- popitem() (k, v), remove and return some (key, value) pair as a ¶
2-tuple; but raise KeyError if D is empty.
- read_link(tgt=None, ttype=None)[source]¶
tgt: saga.Url / None ttype: saga.task.type enum ret: saga.Url / saga.Task
- remove(tgt=None, flags=0, ttype=None)[source]¶
tgt: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task
- remove_attribute(key)¶
Removing an attribute is actually different from unsetting it, or from setting it to ‘None’. On remove, all traces of the attribute are purged, and the key will not be listed on
saga.Attributes.list_attributes()
() anymore.
- remove_callback(key, id)¶
This method allows to unregister a previously registered callback, by providing its id. It is not an error to remove a non-existing cb, but a valid ID MUST be provided – otherwise, a BadParameter is raised.
If no ID is provided (id == None), all callbacks are removed for this attribute.
- property session¶
Returns the session which is managing the object instance. For objects which do not accept a session handle on construction, this call returns None.
The object’s session is also available via the session property.
- set_attribute(key, val)¶
This method sets the value of the specified attribute. If that attribute does not exist, DoesNotExist is raised – unless the attribute set is marked ‘extensible’ or ‘private’. In that case, the attribute is created and set on the fly (defaulting to mode=writeable, flavor=Scalar, type=ANY, default=None). A value of ‘None’ may reset the attribute to its default value, if such one exists (see documentation).
Note that this method is performing a number of checks and conversions, to match the value type to the attribute properties (type, mode, flavor). Those conversions are not guaranteed to yield the expected result – for example, the conversion from ‘scalar’ to ‘vector’ is, for complex types, ambiguous at best, and somewhat stupid. The consumer of the API SHOULD ensure correct attribute values. The conversions are intended to support the most trivial and simple use cases (int to string etc). Failed conversions will result in an BadParameter exception.
Attempts to set a ‘final’ attribute are silently ignored. Attempts to set a ‘readonly’ attribute will result in an IncorrectState exception being raised.
Note that set_attribute() will trigger callbacks, if a new value (different from the old value) is given.
- set_ttl(tgt=None, ttl=-1.0, ttype=None)[source]¶
tgt : saga.Url / None ttl : int ttype: saga.task.type enum ret: None / saga.Task
- set_vector_attribute(key, val)¶
See also:
saga.Attributes.set_attribute()
(key, val).As python can handle scalar and vector types transparently, this method is in fact not very useful. For that reason, it maps internally to the set_attribute method.
- setdefault(key, default)¶
Insert key with a value of default if key is not in the dictionary.
Return the value for key if key is in the dictionary, else default.
- update([E, ]**F) None. Update D from dict/iterable E and F. ¶
If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
- property url¶
ttype: saga.task.type enum ret: saga.Url / saga.Task
Return the complete url pointing to the entry.
The call will return the complete url pointing to this entry as a saga.Url object:
# print URL of an entry entry = saga.namespace.Entry("sftp://localhost/etc/passwd") print(entry.get_url())
- values() an object providing a view on D's values ¶