ray.serve.config.HTTPOptions
ray.serve.config.HTTPOptions#
- pydantic model ray.serve.config.HTTPOptions[source]#
HTTP options for the proxies. Supported fields:
host: Host that the proxies listens for HTTP on. Defaults to “127.0.0.1”. To expose Serve publicly, you probably want to set this to “0.0.0.0”.
port: Port that the proxies listen for HTTP on. Defaults to 8000.
root_path: An optional root path to mount the serve application (for example, “/prefix”). All deployment routes are prefixed with this path.
request_timeout_s: End-to-end timeout for HTTP requests.
keep_alive_timeout_s: Duration to keep idle connections alive when no requests are ongoing.
location: [DEPRECATED: use
proxy_location
field instead] The deployment location of HTTP servers:“HeadOnly”: start one HTTP server on the head node. Serve assumes the head node is the node you executed serve.start on. This is the default.
“EveryNode”: start one HTTP server per node.
“NoServer”: disable HTTP server.
num_cpus: [DEPRECATED] The number of CPU cores to reserve for each internal Serve HTTP proxy actor.
Show JSON schema
{ "title": "HTTPOptions", "description": "HTTP options for the proxies. Supported fields:\n\n- host: Host that the proxies listens for HTTP on. Defaults to\n \"127.0.0.1\". To expose Serve publicly, you probably want to set\n this to \"0.0.0.0\".\n- port: Port that the proxies listen for HTTP on. Defaults to 8000.\n- root_path: An optional root path to mount the serve application\n (for example, \"/prefix\"). All deployment routes are prefixed\n with this path.\n- request_timeout_s: End-to-end timeout for HTTP requests.\n- keep_alive_timeout_s: Duration to keep idle connections alive when no\n requests are ongoing.\n\n- location: [DEPRECATED: use `proxy_location` field instead] The deployment\n location of HTTP servers:\n\n - \"HeadOnly\": start one HTTP server on the head node. Serve\n assumes the head node is the node you executed serve.start\n on. This is the default.\n - \"EveryNode\": start one HTTP server per node.\n - \"NoServer\": disable HTTP server.\n\n- num_cpus: [DEPRECATED] The number of CPU cores to reserve for each\n internal Serve HTTP proxy actor.", "type": "object", "properties": { "host": { "title": "Host", "default": "127.0.0.1", "type": "string" }, "port": { "title": "Port", "default": 8000, "type": "integer" }, "middlewares": { "title": "Middlewares", "default": [], "type": "array", "items": {} }, "location": { "default": "HeadOnly", "allOf": [ { "$ref": "#/definitions/DeploymentMode" } ] }, "num_cpus": { "title": "Num Cpus", "default": 0, "type": "integer" }, "root_url": { "title": "Root Url", "default": "", "type": "string" }, "root_path": { "title": "Root Path", "default": "", "type": "string" }, "request_timeout_s": { "title": "Request Timeout S", "type": "number" }, "keep_alive_timeout_s": { "title": "Keep Alive Timeout S", "default": 5, "type": "integer" } }, "definitions": { "DeploymentMode": { "title": "DeploymentMode", "description": "An enumeration.\n\n.. warning::\n\n **DEPRECATED**: This API is deprecated and may be removed in future Ray releases.", "enum": [ "NoServer", "HeadOnly", "EveryNode" ], "type": "string" } } }
- Config
arbitrary_types_allowed: bool = True
validate_assignment: bool = True
- Fields
- Validators
- field location: Optional[ray.serve.config.DeploymentMode] = DeploymentMode.HeadOnly#
- Validated by
- field middlewares: List[Any] = []#
- Validated by
- validator warn_for_middlewares » middlewares[source]#