ray.serve.llm.LLMServingArgs#

pydantic model ray.serve.llm.LLMServingArgs[source]#

The configuration for starting an LLM deployment application.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

field llm_configs: List[str | LLMConfig] [Required]#

A list of LLMConfigs, or paths to LLMConfigs, to run.

parse_args() → LLMServingArgs[source]#

Converts this LLMServingArgs object into an DeployArgs object.

classmethod parse_yaml(file, **kwargs) → ModelT#