ray.serve.request_router.RunningReplica.get_queue_len#

async RunningReplica.get_queue_len(*, deadline_s: float) int[source]#

Returns current queue len for the replica. deadline_s is passed to verify backoff for testing.