ray.serve.request_router.LocalityMixin#

class ray.serve.request_router.LocalityMixin(self_node_id: str | None = None, prefer_local_node_routing: bool = False, prefer_local_az_routing: bool = False, self_availability_zone: str | None = None, *args, **kwargs)[source]#

Mixin for locality routing.

This mixin is used to route requests to replicas that are colocated with the handle. It adds necessary attributes and methods to keep track of locality scopes and offer the helpers to apply locality routing and rank replicas based on locality.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

`apply_locality_routing`	Apply locality routing to the pending request.
`rank_replicas_via_locality`	Rank the replicas based on the locality preference.