ray.serve.handle.RayServeHandle
ray.serve.handle.RayServeHandle#
- class ray.serve.handle.RayServeHandle[source]#
A handle used to make requests from one deployment to another.
This is used to compose multiple deployments into a single application. After building the application, this handle is substituted at runtime for deployments passed as arguments via
bind()
.Example:
import ray from ray import serve from ray.serve.handle import RayServeHandle, RayServeSyncHandle @serve.deployment class Downstream: def __init__(self, message: str): self._message = message def __call__(self, name: str) -> str: return self._message + name @serve.deployment class Ingress: def __init__(self, handle: RayServeHandle): self._handle = handle async def __call__(self, name: str) -> str: obj_ref: ray.ObjectRef = await self._handle.remote(name) return await obj_ref app = Ingress.bind(Downstream.bind("Hello ")) handle: RayServeSyncHandle = serve.run(app) # Prints "Hello Mr. Magoo" print(ray.get(handle.remote("Mr. Magoo")))
Warning
DEPRECATED: This API is deprecated and may be removed in future Ray releases. This API is being replaced by
ray.serve.handle.DeploymentHandle
. Opt into the new API by usinghandle.options(use_new_handle_api=True)
or setting the environment variableRAY_SERVE_USE_NEW_HANDLE_API=1
.- options(*, method_name: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, multiplexed_model_id: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, stream: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, use_new_handle_api: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, _prefer_local_routing: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, _router_cls: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE) ray.serve.handle.RayServeHandle [source]#
Set options for this handle and return an updated copy of it.
Example:
# The following two lines are equivalent: obj_ref = await handle.other_method.remote(*args) obj_ref = await handle.options(method_name="other_method").remote(*args) obj_ref = await handle.options( multiplexed_model_id="model:v1").remote(*args)
- remote(*args, **kwargs) _asyncio.Task [source]#
Issue an asynchronous request to the __call__ method of the deployment.
Returns an
asyncio.Task
whose underlying result is a Ray ObjectRef that points to the final result of the request.The final result can be retrieved by awaiting the ObjectRef.
Example:
obj_ref = await handle.remote(*args) result = await obj_ref