Ray Client

What is the Ray Client?

The Ray Client is an API that connects a python script to a Ray cluster. Effectively, it allows you to leverage a remote Ray cluster just like you would with Ray running on your local machine.

By changing ray.init() to ray.init("ray://<host>:<port>"), you can connect to a remote cluster and scale out your Ray code, while maintaining the ability to develop interactively in a python shell.

# You can run this code outside of the Ray cluster!
import ray

# Starting the Ray client. This connects to a remote Ray cluster.
ray.init("ray://<head_node_host>:10001")

# Normal Ray code follows
@ray.remote
def do_work(x):
    return x ** x

do_work.remote(2)
#....

You can also connect using the ClientBuilder API, but this will eventually be deprecated.

# You can run this code outside of the Ray cluster!
import ray

# Starting the Ray client. This connects to a remote Ray cluster.
# `ray.client` will be deprecated in future releases, so we recommend
# using `ray.init("ray://<head_node_host>:10001")` instead.
ray.client("<head_node_host>:10001").connect()

# Normal Ray code follows
@ray.remote
def do_work(x):
    return x ** x

do_work.remote(2)
#....

How do you use the Ray client?

Step 1: set up your Ray cluster

First, you’ll want to create a remote Ray cluster. Follow the directions in Ray Cluster Quick Start to do this.

If using the Ray cluster launcher, the remote cluster will be listening on port 10001 of the head node. If necessary, you can modify this port by setting --ray-client-server-port to the ray start command.

If not using the Ray cluster launcher, you can start the “Ray Client Server” manually on the head node of your remote cluster by running the following:

python -m ray.util.client.server [--host host_ip] [--port port] [--redis-address address] [--redis-password password]

Step 2: Check ports

Ensure that the Ray Client port on the head node is reachable from your local machine. This means opening that port up (on EC2) or proxying from your local machine to the cluster (on K8s).

Step 3: Run Ray code

Now, connect to the Ray Cluster with the following and then use Ray like you normally would:

import ray

# replace with the appropriate host and port
ray.init("ray://<head_node_host>:10001")

# Normal Ray code follows
@ray.remote
def do_work(x):
    return x ** x

do_work.remote(2)

#....

Things to know

Client disconnections

When the client disconnects, any object or actor references held by the server on behalf of the client are dropped, as if directly disconnecting from the cluster.

Versioning requirements

Generally, the client Ray version must match the server Ray version. An error will be raised if an incompatible version is used.

Similarly, the minor Python (e.g., 3.6 vs 3.7) must match between the client and server. An error will be raised if this is not the case.