rank_replicas_via_multiplex#
- MultiplexMixin.rank_replicas_via_multiplex(replicas: List[RunningReplica], multiplexed_model_id: str) List[List[RunningReplica]][source]#
Rank the replicas based on the multiplexed model ID. Rank 0 is the list of replicas that have the multiplexed model ID. Rank 1 is the list of replicas that have the fewest multiplexed models. Rank 2 is the list of all other replicas.
- Parameters:
replicas (List[RunningReplica]) – The replicas to rank.
multiplexed_model_id (str) – The model ID that the request targets.
- Returns:
Three lists of replicas, ordered by rank.
- Return type: