ray.serve.get_multiplexed_model_id#

ray.serve.get_multiplexed_model_id() → str[source]#

Get the multiplexed model ID for the current request.

This is used with a function decorated with @serve.multiplexed to retrieve the model ID for the current request.

import ray
from ray import serve
import requests

# Set the multiplexed model id with the key
# "ray_serve_multiplexed_model_id" in the request
# headers when sending requests to the http proxy.
requests.get("http://localhost:8000",
    headers={"ray_serve_multiplexed_model_id": "model_1"})

# This can also be set when using `DeploymentHandle`.
handle.options(multiplexed_model_id="model_1").remote("blablabla")

# In your deployment code, you can retrieve the model id from
# `get_multiplexed_model_id()`.
@serve.deployment
def my_deployment_function(request):
    assert serve.get_multiplexed_model_id() == "model_1"

PublicAPI (beta): This API is in beta and may change before becoming stable.