AutoscalingPolicy#

pydantic model ray.serve.config.AutoscalingPolicy[source]#

The autoscaling policy that a deployment uses.

Set it through AutoscalingConfig.policy. By default, Serve uses its built-in request-based policy.

field policy_function: str | Callable = 'ray.serve.autoscaling_policy:default_autoscaling_policy'#

Policy function can be a string import path or a function callable. If it’s a string import path, it must be of the form path.to.module:function_name.

field policy_kwargs: Dict[str, Any] [Optional]#

Keyword arguments passed to the policy class constructor when policy_function refers to a class. Ignored when policy_function is a plain function. Values must be JSON-serializable.

classmethod from_serialized_policy_def(policy_config: dict, serialized_policy_def: bytes) → AutoscalingPolicy[source]#

Build a policy from a config dict and a serialized policy.

Parameters:
  • policy_config (dict) – Field values for the policy. This method copies the dict and doesn’t modify it.

  • serialized_policy_def (bytes) – The policy, serialized with cloudpickle.

Returns:

A policy that uses serialized_policy_def instead of importing and serializing the policy again.

Return type:

AutoscalingPolicy

validator policy_kwargs_json_serializable  »  policy_kwargs[source]#
get_policy() → Callable[source]#

Deserialize policy from cloudpickled bytes.

The result is cached to avoid repeated cloudpickle deserialization on every call (e.g. on every autoscaling tick).

Returns:

The policy function or class.

Raises:

ImportError – If the policy depends on a module that isn’t available in the current environment.

Return type:

Callable

get_serialized_policy_def() → bytes | None[source]#

Get the cloudpickled policy.

Returns:

The policy, serialized with cloudpickle.

Return type:

bytes | None

is_default_policy_function() → bool[source]#

Whether this policy is Serve’s built-in request-based policy.

Returns:

True if policy_function is the default policy’s import path.

Return type:

bool

serialize_policy() → None[source]#

Serialize policy with cloudpickle.

Import the policy if it’s passed in as a string import path. Then cloudpickle the policy and set serialized_policy_def if it’s empty.

set_serialized_policy_def(serialized_policy_def: bytes) → None[source]#

Set the cloudpickled policy and clear the cached policy.

Parameters:

serialized_policy_def (bytes) – The policy, serialized with cloudpickle.