BackPressureError

exception ray.serve.exceptions.BackPressureError(num_queued_requests: int, max_queued_requests: int, status_code: int = 503, retry_after_s: float | None = None)

Raised when max_queued_requests is exceeded on a DeploymentHandle.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.