ray.serve.start#

ray.serve.start(detached: bool = True, proxy_location: Union[None, str, ray.serve.config.ProxyLocation] = None, http_options: Union[None, dict, ray.serve.config.HTTPOptions] = None, dedicated_cpu: bool = False, grpc_options: Union[None, dict, ray.serve.config.gRPCOptions] = None, **kwargs)[source]#

Start Serve on the cluster.

Used to set cluster-scoped configurations such as HTTP options. In most cases, this does not need to be called manually and Serve will be started when an application is first deployed to the cluster.

These cluster-scoped options cannot be updated dynamically. To update them, start a new cluster or shut down Serve on the cluster and start it again.

These options can also be set in the config file deployed via REST API.

Parameters

  • proxy_location – Where to run proxies that handle ingress traffic to the cluster (defaults to every node in the cluster with at least one replica on it). See ProxyLocation for supported options.

  • http_options – HTTP config options for the proxies. These can be passed as an unstructured dictionary or the structured HTTPOptions class. See HTTPOptions for supported options.

  • grpc_options – [EXPERIMENTAL] gRPC config options for the proxies. These can be passed as an unstructured dictionary or the structured gRPCOptions class See gRPCOptions for supported options.