ray.serve.get_multiplexed_model_id() str[source]#

Get the multiplexed model ID for the current request.

This is used with a function decorated with @serve.multiplexed to retrieve the model ID for the current request.

import ray
from ray import serve
import requests

# Set the multiplexed model id with the key
# "ray_serve_multiplexed_model_id" in the request
# headers when sending requests to the http proxy.
    headers={"ray_serve_multiplexed_model_id": "model_1"})

# This can also be set when using `DeploymentHandle`.

# In your deployment code, you can retrieve the model id from
# `get_multiplexed_model_id()`.
def my_deployment_function(request):
    assert serve.get_multiplexed_model_id() == "model_1"

PublicAPI (beta): This API is in beta and may change before becoming stable.