Runtime context

These APIs read context from inside a running replica, such as the replica’s identity, the current trace, or the multiplexed model ID of the current request.

Replica and request context

serve.get_replica_context	Returns the deployment and replica tag from within a replica at runtime.
serve.get_trace_context	Get the current OpenTelemetry trace context.
serve.get_deployment_actor	Get a handle to a deployment-scoped actor by name.
serve.get_multiplexed_model_id	Get the multiplexed model ID for the current request.
serve.context.ReplicaContext	Stores runtime context info for replicas.
serve.context.GangContext	Context information for a replica that is part of a gang.

gRPC context

serve.grpc_util.RayServegRPCContext	Context manager to set and get gRPC context.
serve.grpc_util.gRPCInputStream	Async iterator wrapping an incoming gRPC request stream.