ray.serve.llm.LLMRouter.get_deployment_options

classmethod LLMRouter.get_deployment_options(llm_configs: List[LLMConfig] | None = None) → Dict[str, Any]

Get the deployment options for the ingress deployment.

Parameters:

llm_configs – The LLM configs to infer the number of ingress replicas from.

Returns:

A dictionary containing the deployment options for the ingress deployment.