alibi.utils.distributed module

class alibi.utils.distributed.ActorPool(actors)[source]

Bases: object

__init__(actors)[source]

Taken fom the ray repository: https://github.com/ray-project/ray/pull/5945 Create an Actor pool from a list of existing actors. An actor pool is a utility class similar to multiprocessing.Pool that lets you schedule Ray tasks over a fixed pool of actors. :param actors: List of Ray actor handles to use in this pool. :type actors: list

Examples

>>> a1, a2 = Actor.remote(), Actor.remote()
>>> pool = ActorPool([a1, a2])
>>> print(pool.map(lambda a, v: a.double.remote(v), [1, 2, 3, 4]))
[2, 4, 6, 8]
get_next(timeout=None)[source]

Returns the next pending result in order. This returns the next result produced by submit(), blocking for up to the specified timeout until it is available. :returns: The next result.

Raises

TimeoutError if the timeout is reached.

Examples

>>> pool = ActorPool(...)
>>> pool.submit(lambda a, v: a.double.remote(v), 1)
>>> print(pool.get_next())
2
get_next_unordered(timeout=None)[source]

Returns any of the next pending results. This returns some result produced by submit(), blocking for up to the specified timeout until it is available. Unlike get_next(), the results are not always returned in same order as submitted, which can improve performance. :returns: The next result.

Raises

TimeoutError if the timeout is reached.

Examples

>>> pool = ActorPool(...)
>>> pool.submit(lambda a, v: a.double.remote(v), 1)
>>> pool.submit(lambda a, v: a.double.remote(v), 2)
>>> print(pool.get_next_unordered())
4
>>> print(pool.get_next_unordered())
2
has_next()[source]

Returns whether there are any pending results to return. :returns: True if there are any pending results not yet returned.

Examples

>>> pool = ActorPool(...)
>>> pool.submit(lambda a, v: a.double.remote(v), 1)
>>> print(pool.has_next())
True
>>> print(pool.get_next())
2
>>> print(pool.has_next())
False
map(fn, values, chunksize=1)[source]

Apply the given function in parallel over the actors and values. This returns an ordered iterator that will return results of the map as they finish. Note that you must iterate over the iterator to force the computation to finish. :param fn: Function that takes (actor, value) as argument and

returns an ObjectID computing the result over the value. The actor will be considered busy until the ObjectID completes.

Parameters
  • values (list) – List of values that fn(actor, value) should be applied to.

  • chunksize (int) – splits the list of values to be submitted to the parallel process into sublists of size chunksize or less

Returns

Iterator over results from applying fn to the actors and values.

Examples

>>> pool = ActorPool(...)
>>> print(pool.map(lambda a, v: a.double.remote(v), [1, 2, 3, 4]))
[2, 4, 6, 8]
map_unordered(fn, values, chunksize=1)[source]

Similar to map(), but returning an unordered iterator. This returns an unordered iterator that will return results of the map as they finish. This can be more efficient that map() if some results take longer to compute than others. :param fn: Function that takes (actor, value) as argument and

returns an ObjectID computing the result over the value. The actor will be considered busy until the ObjectID completes.

Parameters
  • values (list) – List of values that fn(actor, value) should be applied to.

  • chunksize (int) – splits the list of values to be submitted to the parallel process into sublists of size chunksize or less

Returns

Iterator over results from applying fn to the actors and values.

Examples

>>> pool = ActorPool(...)
>>> print(pool.map(lambda a, v: a.double.remote(v), [1, 2, 3, 4]))
[6, 2, 4, 8]
submit(fn, value)[source]

Schedule a single task to run in the pool. This has the same argument semantics as map(), but takes on a single value instead of a list of values. The result can be retrieved using get_next() / get_next_unordered(). :param fn: Function that takes (actor, value) as argument and

returns an ObjectID computing the result over the value. The actor will be considered busy until the ObjectID completes.

Parameters

value (object) – Value to compute a result for.

Examples

>>> pool = ActorPool(...)
>>> pool.submit(lambda a, v: a.double.remote(v), 1)
>>> pool.submit(lambda a, v: a.double.remote(v), 2)
>>> print(pool.get_next(), pool.get_next())
2, 4
alibi.utils.distributed.check_ray()[source]

Checks if ray is installed

a bool indicating whether ray is installed or not