ray.serve.handle.RayServeHandle#

class ray.serve.handle.RayServeHandle[source]#

A handle used to make requests from one deployment to another.

This is used to compose multiple deployments into a single application. After building the application, this handle is substituted at runtime for deployments passed as arguments via bind().

Example:

import ray
from ray import serve
from ray.serve.handle import RayServeHandle, RayServeSyncHandle

@serve.deployment
class Downstream:
    def __init__(self, message: str):
        self._message = message

    def __call__(self, name: str) -> str:
        return self._message + name

@serve.deployment
class Ingress:
    def __init__(self, handle: RayServeHandle):
        self._handle = handle

    async def __call__(self, name: str) -> str:
        obj_ref: ray.ObjectRef = await self._handle.remote(name)
        return await obj_ref

app = Ingress.bind(Downstream.bind("Hello "))
handle: RayServeSyncHandle = serve.run(app)

# Prints "Hello Mr. Magoo"
print(ray.get(handle.remote("Mr. Magoo")))

Warning

DEPRECATED: This API is deprecated and may be removed in future Ray releases. This API is being replaced by ray.serve.handle.DeploymentHandle. Opt into the new API by using handle.options(use_new_handle_api=True) or setting the environment variable RAY_SERVE_USE_NEW_HANDLE_API=1.

options(*, method_name: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, multiplexed_model_id: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, stream: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, use_new_handle_api: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, _prefer_local_routing: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, _router_cls: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE) → ray.serve.handle.RayServeHandle[source]#

Set options for this handle and return an updated copy of it.

Example:

# The following two lines are equivalent:
obj_ref = await handle.other_method.remote(*args)
obj_ref = await handle.options(method_name="other_method").remote(*args)
obj_ref = await handle.options(
    multiplexed_model_id="model:v1").remote(*args)
remote(*args, **kwargs) → _asyncio.Task[source]#

Issue an asynchronous request to the __call__ method of the deployment.

Returns an asyncio.Task whose underlying result is a Ray ObjectRef that points to the final result of the request.

The final result can be retrieved by awaiting the ObjectRef.

Example:

obj_ref = await handle.remote(*args)
result = await obj_ref