rank_replicas_via_multiplex#

MultiplexMixin.rank_replicas_via_multiplex(replicas: List[RunningReplica], multiplexed_model_id: str) → List[List[RunningReplica]][source]#

Rank the replicas based on the multiplexed model ID. Rank 0 is the list of replicas that have the multiplexed model ID. Rank 1 is the list of replicas that have the fewest multiplexed models. Rank 2 is the list of all other replicas.

Parameters:
  • replicas (List[RunningReplica]) – The replicas to rank.

  • multiplexed_model_id (str) – The model ID that the request targets.

Returns:

Three lists of replicas, ordered by rank.

Return type:

List[List[RunningReplica]]