ray.serve.request_router.LocalityMixin.rank_replicas_via_locality

LocalityMixin.rank_replicas_via_locality(replicas: List[RunningReplica]) → List[List[RunningReplica]]

Rank the replicas based on the locality preference. Rank 0 is the list of replicas that are on the same node. Rank 1 is the list of replicas that are on the same availability zone. Rank 2 is the list of all other replicas.