ray.serve.handle.DeploymentResponse

class ray.serve.handle.DeploymentResponse

A future-like object wrapping the result of a unary deployment handle call.

From inside a deployment, a DeploymentResponse can be awaited to retrieve the output of the call without blocking the asyncio event loop.

From outside a deployment, .result() can be used to retrieve the output in a blocking manner.

Example:

from ray import serve
from ray.serve.handle import DeploymentHandle

@serve.deployment
class Downstream:
    def say_hi(self, message: str) -> str:
        return f"Hello {message}!"

@serve.deployment
class Caller:
    def __init__(self, handle: DeploymentHandle):
        self._downstream_handle = handle

async def __call__(self, message: str) -> str:
    # Inside a deployment: `await` the result to enable concurrency.
    response = self._downstream_handle.say_hi.remote(message)
    return await response

app = Caller.bind(Downstream.bind())
handle: DeploymentHandle = serve.run(app)

# Outside a deployment: call `.result()` to get output.
response = handle.remote("world")
assert response.result() == "Hello world!"

A DeploymentResponse can be passed directly to another DeploymentHandle call without fetching the result to enable composing multiple deployments together.

Example:

from ray import serve
from ray.serve.handle import DeploymentHandle

@serve.deployment
class Adder:
    def add(self, val: int) -> int:
        return val + 1

@serve.deployment
class Caller:
    def __init__(self, handle: DeploymentHandle):
        self._adder_handle = handle

async def __call__(self, start: int) -> int:
    return await self._adder_handle.add.remote(
        # Pass the response directly to another handle call without awaiting.
        self._adder_handle.add.remote(start)
    )

app = Caller.bind(Adder.bind())
handle: DeploymentHandle = serve.run(app)
assert handle.remote(0).result() == 2

PublicAPI (beta): This API is in beta and may change before becoming stable.

result(*, timeout_s: float | None = None, _skip_asyncio_check: bool = False) → Any

Fetch the result of the handle call synchronously.

This should not be used from within a deployment as it runs in an asyncio event loop. For model composition, await the response instead.

If timeout_s is provided and the result is not available before the timeout, a TimeoutError is raised.