LocalityMixin#
- class ray.serve.request_router.LocalityMixin(self_node_id: str | None = None, prefer_local_node_routing: bool = False, prefer_local_az_routing: bool = False, self_availability_zone: str | None = None, *args, **kwargs)[source]#
Bases:
objectMixin for locality routing.
This mixin is used to route requests to replicas that are colocated with the handle. It adds necessary attributes and methods to keep track of locality scopes and offer the helpers to apply locality routing and rank replicas based on locality.
- Parameters:
self_node_id (str | None) – ID of the node that the handle runs on.
prefer_local_node_routing (bool) – Whether to prefer replicas on the same node as the handle.
prefer_local_az_routing (bool) – Whether to prefer replicas in the same availability zone as the handle.
self_availability_zone (str | None) – Availability zone that the handle runs in.
*args – Positional arguments passed to the next class in the MRO.
**kwargs – Keyword arguments passed to the next class in the MRO.
PublicAPI (alpha): This API is in alpha and may change before becoming stable.
Methods
Apply locality routing to the pending request.
Rank the replicas based on the locality preference.