ray.serve.ingress

ray.serve.ingress(app: starlette.types.ASGIApp | Callable) → Callable

Wrap a deployment class with an ASGI application for HTTP request parsing. There are a few different ways to use this functionality.

Example:

FastAPI app routes are defined inside the deployment class.

from ray import serve
from fastapi import FastAPI

app = FastAPI()

@serve.deployment
@serve.ingress(app)
class MyFastAPIDeployment:
    @app.get("/hi")
    def say_hi(self) -> str:
        return "Hello world!"

app = MyFastAPIDeployment.bind()

You can also use a standalone FastAPI app without registering routes inside the deployment.

from ray import serve
from fastapi import FastAPI

app = FastAPI()

@app.get("/hi")
def say_hi():
    return "Hello world!"

deployment = serve.deployment(serve.ingress(app)())
app = deployment.bind()

You can also pass in a builder function that returns an ASGI app. The builder function is evaluated when the deployment is initialized on replicas. This example shows how to use a sub-deployment inside the routes defined outside the deployment class.

from ray import serve

@serve.deployment
class SubDeployment:
    def __call__(self):
        return "Hello world!"

def build_asgi_app():
    from fastapi import FastAPI

    app = FastAPI()

    def get_sub_deployment_handle():
        return serve.get_deployment_handle(SubDeployment.name, app_name="my_app")

    @app.get("/hi")
    async def say_hi(handle: Depends(get_sub_deployment_handle)):
        return await handle.remote()

    return app

deployment = serve.deployment(serve.ingress(build_asgi_app)())
app = deployment.bind(SubDeployment.bind(), name="my_app", route_prefix="/")

Parameters:

app – the FastAPI app to wrap this class with. Can be any ASGI-compatible callable. You can also pass in a builder function that returns an ASGI app.