ray.serve.request_router.MultiplexMixin.rank_replicas_via_multiplex

MultiplexMixin.rank_replicas_via_multiplex(replicas: List[RunningReplica], multiplexed_model_id: str) → List[List[RunningReplica]]

Rank the replicas based on the multiplexed model ID. Rank 0 is the list of replicas that have the multiplexed model ID. Rank 1 is the list of replicas that have the fewest multiplexed models. Rank 2 is the list of all other replicas.