ray.serve.request_router.RequestRouter.on_new_queue_len_info#

RequestRouter.on_new_queue_len_info(replica_id: ReplicaID, queue_len_info: ReplicaQueueLengthInfo)[source]#

Update queue length cache with new info received from replica.