ray.serve.handle.RayServeSyncHandle#

class ray.serve.handle.RayServeSyncHandle[source]#

A handle used to make requests to the ingress deployment of an application.

This is returned by serve.run and can be used to invoke the application from Python rather than over HTTP. For example:

import ray
from ray import serve
from ray.serve.handle import RayServeSyncHandle

@serve.deployment
class Ingress:
    def __call__(self, name: str) -> str:
        return f"Hello {name}"

app = Ingress.bind()
handle: RayServeSyncHandle = serve.run(app)

# Prints "Hello Mr. Magoo"
print(ray.get(handle.remote("Mr. Magoo")))

Warning

DEPRECATED: This API is deprecated and may be removed in future Ray releases. This API is being replaced by ray.serve.handle.DeploymentHandle. Opt into the new API by using handle.options(use_new_handle_api=True) or setting the environment variable RAY_SERVE_USE_NEW_HANDLE_API=1.

options(*, method_name: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, multiplexed_model_id: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, stream: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, use_new_handle_api: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, _prefer_local_routing: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, _router_cls: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE) → ray.serve.handle.RayServeSyncHandle[source]#

Set options for this handle and return an updated copy of it.

Example:

# The following two lines are equivalent:
obj_ref = handle.other_method.remote(*args)
obj_ref = handle.options(method_name="other_method").remote(*args)
obj_ref = handle.options(multiplexed_model_id="model1").remote(*args)
remote(*args, **kwargs) → ray._raylet.ObjectRef[source]#

Issue an asynchronous request to the __call__ method of the deployment.

Returns a Ray ObjectRef whose results can be waited for or retrieved using ray.wait or ray.get, respectively.

obj_ref = handle.remote(*args)
result = ray.get(obj_ref)