class ray.serve.Deployment[source]#

Class (or function) decorated with the @serve.deployment decorator.

This is run on a number of replica actors. Requests to those replicas call this class.

One or more deployments can be composed together into an Application which is then run via serve.run or a config file.


class MyDeployment:
    def __init__(self, name: str):
        self._name = name

    def __call__(self, request):
        return "Hello world!"

    app = MyDeployment.bind()
    # Run via `serve.run` or the `serve run` CLI command.
property name: str#

Unique name of this deployment.

property func_or_class: Callable | str#

Underlying class or function that this deployment wraps.

property num_replicas: int#

Target number of replicas.

property user_config: Any#

Dynamic user-provided config options.

property max_concurrent_queries: int#

[DEPRECATED] Max number of requests a replica can handle at once.

property max_ongoing_requests: int#

Max number of requests a replica can handle at once.

property max_queued_requests: int#

Max number of requests that can be queued in each deployment handle.

property route_prefix: str | None#

HTTP route prefix that this deployment is exposed under.

property ray_actor_options: Dict | None#

Actor options such as resources required for each replica.

bind(*args, **kwargs) Application[source]#

Bind the arguments to the deployment and return an Application.

The returned Application can be deployed using serve.run (or via config file) or bound to another deployment for composition.

options(func_or_class: Callable | None = None, name: DEFAULT | str = DEFAULT.VALUE, version: DEFAULT | str = DEFAULT.VALUE, num_replicas: DEFAULT | int | str | None = DEFAULT.VALUE, route_prefix: DEFAULT | str | None = DEFAULT.VALUE, ray_actor_options: DEFAULT | Dict | None = DEFAULT.VALUE, placement_group_bundles: List[Dict[str, float]] | None = DEFAULT.VALUE, placement_group_strategy: str | None = DEFAULT.VALUE, max_replicas_per_node: int | None = DEFAULT.VALUE, user_config: DEFAULT | Any | None = DEFAULT.VALUE, max_concurrent_queries: DEFAULT | int = DEFAULT.VALUE, max_ongoing_requests: DEFAULT | int = DEFAULT.VALUE, max_queued_requests: DEFAULT | int = DEFAULT.VALUE, autoscaling_config: DEFAULT | Dict | AutoscalingConfig | None = DEFAULT.VALUE, graceful_shutdown_wait_loop_s: DEFAULT | float = DEFAULT.VALUE, graceful_shutdown_timeout_s: DEFAULT | float = DEFAULT.VALUE, health_check_period_s: DEFAULT | float = DEFAULT.VALUE, health_check_timeout_s: DEFAULT | float = DEFAULT.VALUE, logging_config: DEFAULT | Dict | LoggingConfig | None = DEFAULT.VALUE, _init_args: DEFAULT | Tuple[Any] = DEFAULT.VALUE, _init_kwargs: DEFAULT | Dict[Any, Any] = DEFAULT.VALUE, _internal: bool = False) Deployment[source]#

Return a copy of this deployment with updated options.

Only those options passed in will be updated, all others will remain unchanged from the existing deployment.

Refer to the @serve.deployment decorator docs for available arguments.