ray.serve.handle.RayServeHandle#
- class ray.serve.handle.RayServeHandle[source]#
A handle used to make requests from one deployment to another.
This is used to compose multiple deployments into a single application. After building the application, this handle is substituted at runtime for deployments passed as arguments via
bind()
.Example:
import ray from ray import serve from ray.serve.handle import RayServeHandle, RayServeSyncHandle @serve.deployment class Downstream: def __init__(self, message: str): self._message = message def __call__(self, name: str) -> str: return self._message + name @serve.deployment class Ingress: def __init__(self, handle: RayServeHandle): self._handle = handle async def __call__(self, name: str) -> str: obj_ref: ray.ObjectRef = await self._handle.remote(name) return await obj_ref app = Ingress.bind(Downstream.bind("Hello ")) handle: RayServeSyncHandle = serve.run(app) # Prints "Hello Mr. Magoo" print(ray.get(handle.remote("Mr. Magoo")))
Warning
DEPRECATED: This API is deprecated and may be removed in future Ray releases. This API has been replaced by
ray.serve.handle.DeploymentHandle
.- options(*, method_name: str | DEFAULT = DEFAULT.VALUE, multiplexed_model_id: str | DEFAULT = DEFAULT.VALUE, stream: bool | DEFAULT = DEFAULT.VALUE, use_new_handle_api: bool | DEFAULT = DEFAULT.VALUE, _prefer_local_routing: bool | DEFAULT = DEFAULT.VALUE, _router_cls: str | DEFAULT = DEFAULT.VALUE) RayServeHandle [source]#
Set options for this handle and return an updated copy of it.
Example:
# The following two lines are equivalent: obj_ref = await handle.other_method.remote(*args) obj_ref = await handle.options(method_name="other_method").remote(*args) obj_ref = await handle.options( multiplexed_model_id="model:v1").remote(*args)
- remote(*args, **kwargs) Task [source]#
Issue an asynchronous request to the __call__ method of the deployment.
Returns an
asyncio.Task
whose underlying result is a Ray ObjectRef that points to the final result of the request.The final result can be retrieved by awaiting the ObjectRef.
Example:
obj_ref = await handle.remote(*args) result = await obj_ref