ray.serve.request_router.RequestRouter.select_available_replicas

RequestRouter.select_available_replicas(candidates: List[RunningReplica] | None = None) → List[RunningReplica]

Select available replicas from the list of candidates.

This method is used to select replicas that are available to take more requests based on the queue length cache. If the queue length is not available in the cache, the replica is considered available. It does not actively probe the replicas for their queue length.

If input candidates is None, all replicas are considered.