ray.serve.config.HTTPOptions#

pydantic model ray.serve.config.HTTPOptions[source]#

HTTP options for the proxies. Supported fields:

host: Host that the proxies listens for HTTP on. Defaults to “127.0.0.1”. To expose Serve publicly, you probably want to set this to “0.0.0.0”.
port: Port that the proxies listen for HTTP on. Defaults to 8000.
root_path: An optional root path to mount the serve application (for example, “/prefix”). All deployment routes are prefixed with this path.
request_timeout_s: End-to-end timeout for HTTP requests.
keep_alive_timeout_s: Duration to keep idle connections alive when no requests are ongoing.
location: [DEPRECATED: use proxy_location field instead] The deployment location of HTTP servers:
- “HeadOnly”: start one HTTP server on the head node. Serve assumes the head node is the node you executed serve.start on. This is the default.
- “EveryNode”: start one HTTP server per node.
- “NoServer”: disable HTTP server.
num_cpus: [DEPRECATED] The number of CPU cores to reserve for each internal Serve HTTP proxy actor.

Show JSON schema

{
   "title": "HTTPOptions",
   "description": "HTTP options for the proxies. Supported fields:\n\n- host: Host that the proxies listens for HTTP on. Defaults to\n  \"127.0.0.1\". To expose Serve publicly, you probably want to set\n  this to \"0.0.0.0\".\n- port: Port that the proxies listen for HTTP on. Defaults to 8000.\n- root_path: An optional root path to mount the serve application\n  (for example, \"/prefix\"). All deployment routes are prefixed\n  with this path.\n- request_timeout_s: End-to-end timeout for HTTP requests.\n- keep_alive_timeout_s: Duration to keep idle connections alive when no\n  requests are ongoing.\n\n- location: [DEPRECATED: use `proxy_location` field instead] The deployment\n  location of HTTP servers:\n\n    - \"HeadOnly\": start one HTTP server on the head node. Serve\n      assumes the head node is the node you executed serve.start\n      on. This is the default.\n    - \"EveryNode\": start one HTTP server per node.\n    - \"NoServer\": disable HTTP server.\n\n- num_cpus: [DEPRECATED] The number of CPU cores to reserve for each\n  internal Serve HTTP proxy actor.",
   "type": "object",
   "properties": {
      "host": {
         "title": "Host",
         "default": "127.0.0.1",
         "type": "string"
      },
      "port": {
         "title": "Port",
         "default": 8000,
         "type": "integer"
      },
      "middlewares": {
         "title": "Middlewares",
         "default": [],
         "type": "array",
         "items": {}
      },
      "location": {
         "default": "HeadOnly",
         "allOf": [
            {
               "$ref": "#/definitions/DeploymentMode"
            }
         ]
      },
      "num_cpus": {
         "title": "Num Cpus",
         "default": 0,
         "type": "integer"
      },
      "root_url": {
         "title": "Root Url",
         "default": "",
         "type": "string"
      },
      "root_path": {
         "title": "Root Path",
         "default": "",
         "type": "string"
      },
      "request_timeout_s": {
         "title": "Request Timeout S",
         "type": "number"
      },
      "keep_alive_timeout_s": {
         "title": "Keep Alive Timeout S",
         "default": 5,
         "type": "integer"
      }
   },
   "definitions": {
      "DeploymentMode": {
         "title": "DeploymentMode",
         "description": "An enumeration.\n\n.. warning::\n\n    **DEPRECATED**: This API is deprecated and may be removed in future Ray releases.",
         "enum": [
            "NoServer",
            "HeadOnly",
            "EveryNode"
         ],
         "type": "string"
      }
   }
}