LocalityMixin#

class ray.serve.request_router.LocalityMixin(self_node_id: str | None = None, prefer_local_node_routing: bool = False, prefer_local_az_routing: bool = False, self_availability_zone: str | None = None, *args, **kwargs)[source]#

Bases: object

Mixin for locality routing.

This mixin is used to route requests to replicas that are colocated with the handle. It adds necessary attributes and methods to keep track of locality scopes and offer the helpers to apply locality routing and rank replicas based on locality.

Parameters:
  • self_node_id (str | None) – ID of the node that the handle runs on.

  • prefer_local_node_routing (bool) – Whether to prefer replicas on the same node as the handle.

  • prefer_local_az_routing (bool) – Whether to prefer replicas in the same availability zone as the handle.

  • self_availability_zone (str | None) – Availability zone that the handle runs in.

  • *args – Positional arguments passed to the next class in the MRO.

  • **kwargs – Keyword arguments passed to the next class in the MRO.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

apply_locality_routing

Apply locality routing to the pending request.

rank_replicas_via_locality

Rank the replicas based on the locality preference.