ray.serve.llm.deployments.VLLMService.__init__#

async VLLMService.__init__(llm_config: LLMConfig, *, engine_cls: Type[VLLMEngine] | None = None, image_retriever_cls: Type[ImageRetriever] | None = None, model_downloader: LoraModelLoader | None = None)[source]#

Constructor of VLLMDeployment.

Only the llm_config is public api, the other arguments are private and used for testing.

Parameters:

llm_config – LLMConfig for the model.

Keyword Arguments:

  • engine_cls – Dependency injection for the vllm engine class. Defaults to VLLMEngine.

  • image_retriever_cls – Dependency injection for the image retriever class. Defaults to ImageRetriever.

  • model_downloader – Dependency injection for the model downloader object. Defaults to be initialized with LoraModelLoader.