Out-of-band communication#

Ray actors typically communicate through actor method calls and share data through the distributed object store. In some use cases, out-of-band communication can be useful instead.

Wrapping library processes#

Many libraries have mature, high-performance internal communication stacks and use Ray as a language-integrated actor scheduler. These libraries handle most communication between actors out-of-band, through their existing communication stacks. For example, Horovod-on-Ray uses collective communication based on NCCL or the Message Passing Interface (MPI), and RayDP uses Spark’s internal RPC and object manager. See Ray Distributed Library Patterns for more details.

Ray Collective#

The Ray collective communication library, ray.util.collective, provides efficient out-of-band collective and point-to-point communication between distributed CPUs or GPUs. See Ray Collective for more details.

HTTP server#

You can start an HTTP server inside an actor and expose HTTP endpoints, so clients outside the Ray cluster can communicate with the actor.

Python

import ray
import asyncio
import requests
from aiohttp import web


@ray.remote
class Counter:
    async def __init__(self):
        self.counter = 0
        asyncio.get_running_loop().create_task(self.run_http_server())

    async def run_http_server(self):
        app = web.Application()
        app.add_routes([web.get("/", self.get)])
        runner = web.AppRunner(app)
        await runner.setup()
        site = web.TCPSite(runner, "127.0.0.1", 25001)
        await site.start()

    async def get(self, request):
        return web.Response(text=str(self.counter))

    async def increment(self):
        self.counter = self.counter + 1


ray.init()
counter = Counter.remote()
[ray.get(counter.increment.remote()) for i in range(5)]
r = requests.get("http://127.0.0.1:25001/")
assert r.text == "5"

You can expose other types of servers the same way, such as gRPC servers.

Limitations#

Ray doesn’t manage out-of-band calls between actors. As a result, features such as distributed reference counting don’t work with out-of-band communication, so don’t pass object refs this way.