ExecutionResources#

class ray.data.ExecutionResources(cpu: float | None = None, gpu: float | None = None, object_store_memory: float | None = None, memory: float | None = None)[source]#

Bases: object

Specifies resources usage or resource limits for execution.

By default this class represents resource usage. Use for_limits or set default_to_inf to True to create an object that represents resource limits.

classmethod from_resource_dict(resource_dict: Dict[str, float])[source]#: Create an ExecutionResources object from a resource dict.

to_resource_dict() → Dict[str, float][source]#: Convert this ExecutionResources object to a resource dict.

classmethod for_limits(cpu: float | None = None, gpu: float | None = None, object_store_memory: float | None = None, memory: float | None = None) → ExecutionResources[source]#

Create an ExecutionResources object that represents resource limits.

Parameters:

cpu – Amount of logical CPU slots.
gpu – Amount of logical GPU slots.
object_store_memory – Amount of object store memory.
memory – Amount of logical memory in bytes.

Returns:

An ExecutionResources with the given limits (defaulting to infinity for any unspecified field).

classmethod zero() → ExecutionResources[source]#

Returns an ExecutionResources object with zero resources.

Returns a cached, shared singleton (functools.cache keyed on cls) – zero() is called all over the scheduler hot path (e.g. .max(zero())) and instances are immutable in practice (every arithmetic op returns a new object and there are no setters), so sharing one instance is safe and avoids the per-call allocation.

classmethod inf() → ExecutionResources[source]#

Returns an ExecutionResources object with infinite resources.

Returns a cached, shared singleton (see zero() for why this is safe).

classmethod combine(resources: Iterable[ExecutionResources], fn: Callable[[float, float], float]) → ExecutionResources | None[source]#

Fold an iterable of ExecutionResources per dimension with fn.

fn(acc, value) combines two per-dimension floats – e.g. operator.add for a sum, or max/min for an element-wise max/min. Accumulates raw floats in a single pass and allocates a single result object, instead of one intermediate per element as reduce(lambda a, b: a.<op>(b), resources) would.

Seeds with the first element (so no per-fn identity is needed) and returns None for an empty iterable, which may be a one-shot generator (so it’s consumed exactly once).

classmethod combine_sum(resources: Iterable[ExecutionResources]) → ExecutionResources[source]#

Sum an iterable of ExecutionResources in a single pass.

Thin wrapper over combine() with addition. Empty folds are common (e.g. completed-ops / downstream-ineligible usage rollups on most iterations), so an empty input reuses the shared zero() singleton instead of allocating.

is_zero() → bool[source]#: Returns True if all resources are zero.

is_non_negative() → bool[source]#: Returns True if all resources are non-negative.

object_store_memory_str() → str[source]#: Returns a human-readable string for the object store memory field.

memory_str() → str[source]#: Returns a human-readable string for the memory field.

copy(cpu: float | None = None, gpu: float | None = None, memory: float | None = None, object_store_memory: float | None = None) → ExecutionResources[source]#: Returns a copy of this ExecutionResources object allowing to override specific resources as necessary

add(other: ExecutionResources) → ExecutionResources[source]#

Adds execution resources.

Parameters:: other – The other ExecutionResources to add to this one.
Returns:: A new ExecutionResource object with summed resources.

subtract(other: ExecutionResources) → ExecutionResources[source]#

Subtracts execution resources.

Parameters:: other – The other ExecutionResources to subtract from this one.
Returns:: A new ExecutionResource object with subtracted resources.

max(other: ExecutionResources) → ExecutionResources[source]#: Returns the maximum for each resource type.

min(other: ExecutionResources) → ExecutionResources[source]#: Returns the minimum for each resource type.

satisfies_limit(limit: ExecutionResources, *, ignore_object_store_memory: bool = False) → bool[source]#

Return if this resource struct meets the specified limits.

Note that None for a field means no limit.

Parameters:

limit – The resource limits to check against.
ignore_object_store_memory – If True, ignore the object store memory limit when checking if this resource struct meets the limits.

Returns:

True if every resource is within the corresponding limit.

scale(f: float) → ExecutionResources[source]#: Return copy with all set values scaled by f.

floordiv(other: ExecutionResources) → ExecutionResources[source]#: Returns the floor division of resources.