ray.serve.config.HTTPOptions#

class ray.serve.config.HTTPOptions(*, host: str | None = '127.0.0.1', port: int = 8000, middlewares: List[Any] = [], location: DeploymentMode | None = DeploymentMode.HeadOnly, num_cpus: int = 0, root_url: str = '', root_path: str = '', request_timeout_s: float | None = None, keep_alive_timeout_s: int = 5)[source]#

Bases: BaseModel

HTTP options for the proxies. Supported fields:

  • host: Host that the proxies listens for HTTP on. Defaults to “127.0.0.1”. To expose Serve publicly, you probably want to set this to “0.0.0.0”.

  • port: Port that the proxies listen for HTTP on. Defaults to 8000.

  • root_path: An optional root path to mount the serve application (for example, “/prefix”). All deployment routes are prefixed with this path.

  • request_timeout_s: End-to-end timeout for HTTP requests.

  • keep_alive_timeout_s: Duration to keep idle connections alive when no requests are ongoing.

  • location: [DEPRECATED: use proxy_location field instead] The deployment location of HTTP servers:

    • “HeadOnly”: start one HTTP server on the head node. Serve assumes the head node is the node you executed serve.start on. This is the default.

    • “EveryNode”: start one HTTP server per node.

    • “NoServer”: disable HTTP server.

  • num_cpus: [DEPRECATED] The number of CPU cores to reserve for each internal Serve HTTP proxy actor.

Methods

__init__

Create a new model by parsing and validating input data from keyword arguments.

construct

Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.

copy

Duplicate a model, optionally choose which fields to include, exclude and change.

dict

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

json

Generate a JSON representation of the model, include and exclude arguments as per dict().

update_forward_refs

Try to update ForwardRefs on fields based on this Model, globalns and localns.

Attributes

host

port

middlewares

location

num_cpus

root_url

root_path

request_timeout_s

keep_alive_timeout_s