BackPressureError#

exception ray.serve.exceptions.BackPressureError(num_queued_requests: int, max_queued_requests: int, status_code: int = 503, retry_after_s: float | None = None)[source]#

Raised when max_queued_requests is exceeded on a DeploymentHandle.

Parameters:
  • num_queued_requests (int) – The number of requests queued when this request was dropped.

  • max_queued_requests (int) – The deployment’s max_queued_requests.

  • status_code (int) – HTTP status code to return for the dropped request.

  • retry_after_s (float | None) – Value for the Retry-After header, in seconds, or None for no header.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.