BackPressureError#
- exception ray.serve.exceptions.BackPressureError(num_queued_requests: int, max_queued_requests: int, status_code: int = 503, retry_after_s: float | None = None)[source]#
Raised when max_queued_requests is exceeded on a DeploymentHandle.
- Parameters:
num_queued_requests (int) – The number of requests queued when this request was dropped.
max_queued_requests (int) – The deployment’s
max_queued_requests.status_code (int) – HTTP status code to return for the dropped request.
retry_after_s (float | None) – Value for the
Retry-Afterheader, in seconds, or None for no header.
PublicAPI (alpha): This API is in alpha and may change before becoming stable.