ray.serve.llm.LLMRouter#

class ray.serve.llm.LLMRouter(**kwargs)[source]#

Bases: LLMRouter

Methods

as_deployment

Converts this class to a Ray Serve deployment with ingress.