Serve CLI
Contents
Serve CLI#
serve#
CLI for managing Serve instances on a Ray cluster.
serve [OPTIONS] COMMAND [ARGS]...
build#
Imports the ClassNode or FunctionNode at IMPORT_PATH and generates a structured config for it that can be used by serve deploy
or the REST API.
serve build [OPTIONS] IMPORT_PATH
Options
- -d, --app-dir <app_dir>#
Local directory to look for the IMPORT_PATH (will be inserted into PYTHONPATH). Defaults to ‘.’, meaning that an object in ./main.py can be imported as ‘main.object’. Not relevant if you’re importing from an installed module.
- -k, --kubernetes_format#
Print Serve config in Kubernetes format.
- -o, --output-path <output_path>#
Local path where the output config will be written in YAML format. If not provided, the config will be printed to STDOUT.
Arguments
- IMPORT_PATH#
Required argument
config#
Get the current config of the running Serve app.
serve config [OPTIONS]
Options
- -a, --address <address>#
Address to use to query the Ray dashboard agent (defaults to http://localhost:52365). Can also be specified using the RAY_AGENT_ADDRESS environment variable.
deploy#
Deploys deployment(s) from a YAML config file.
This call is async; a successful response only indicates that the request was sent to the Ray cluster successfully. It does not mean the the deployments have been deployed/updated.
Existing deployments with no code changes will not be redeployed.
Use serve config
to fetch the current config and serve status
to check the status of the deployments after deploying.
serve deploy [OPTIONS] CONFIG_FILE_NAME
Options
- -a, --address <address>#
Address to use to query the Ray dashboard agent (defaults to http://localhost:52365). Can also be specified using the RAY_AGENT_ADDRESS environment variable.
Arguments
- CONFIG_FILE_NAME#
Required argument
run#
Runs the Serve app from the specified import path (e.g. my_script:my_bound_deployment) or YAML config.
If using a YAML config, existing deployments with no code changes will not be redeployed.
Any import path must lead to a FunctionNode or ClassNode object. By default, this will block and periodically log status. If you Ctrl-C the command, it will tear down the app.
serve run [OPTIONS] CONFIG_OR_IMPORT_PATH
Options
- --runtime-env <runtime_env>#
Path to a local YAML file containing a runtime_env definition. This will be passed to ray.init() as the default for deployments.
- --runtime-env-json <runtime_env_json>#
JSON-serialized runtime_env dictionary. This will be passed to ray.init() as the default for deployments.
- --working-dir <working_dir>#
Directory containing files that your job will run in. Can be a local directory or a remote URI to a .zip file (S3, GS, HTTP). This overrides the working_dir in –runtime-env if both are specified. This will be passed to ray.init() as the default for deployments.
- -d, --app-dir <app_dir>#
Local directory to look for the IMPORT_PATH (will be inserted into PYTHONPATH). Defaults to ‘.’, meaning that an object in ./main.py can be imported as ‘main.object’. Not relevant if you’re importing from an installed module.
- -a, --address <address>#
Address to use for ray.init(). Can also be specified using the RAY_ADDRESS environment variable.
- -h, --host <host>#
Host for HTTP server to listen on. Defaults to 127.0.0.1.
- -p, --port <port>#
Port for HTTP servers to listen on. Defaults to 8000.
- --blocking, --non-blocking#
Whether or not this command should be blocking. If blocking, it will loop and log status until Ctrl-C’d, then clean up the app.
- --gradio#
Whether to enable gradio visualization of deployment graph.
Arguments
- CONFIG_OR_IMPORT_PATH#
Required argument
shutdown#
Deletes the Serve app.
serve shutdown [OPTIONS]
Options
- -a, --address <address>#
Address to use to query the Ray dashboard agent (defaults to http://localhost:52365). Can also be specified using the RAY_AGENT_ADDRESS environment variable.
- -y, --yes#
Bypass confirmation prompt.
start#
Start a detached Serve instance on the Ray cluster.
serve start [OPTIONS]
Options
- -a, --address <address>#
Address to use for ray.init(). Can also be specified using the RAY_ADDRESS environment variable.
- --http-host <http_host>#
Host for HTTP servers to listen on. Defaults to 127.0.0.1.
- --http-port <http_port>#
Port for HTTP servers to listen on. Defaults to 8000.
- --http-location <http_location>#
Location of the HTTP servers. Defaults to HeadOnly.
- Options
DeploymentMode.NoServer | DeploymentMode.HeadOnly | DeploymentMode.EveryNode | DeploymentMode.FixedNumber
status#
Prints status information about all deployments in the Serve app.
Deployments may be:
HEALTHY: all replicas are acting normally and passing their health checks.
UNHEALTHY: at least one replica is not acting normally and may not be passing its health check.
UPDATING: the deployment is updating.
serve status [OPTIONS]
Options
- -a, --address <address>#
Address to use to query the Ray dashboard agent (defaults to http://localhost:52365). Can also be specified using the RAY_AGENT_ADDRESS environment variable.