Anti-pattern: Calling ray.get unnecessarily harms performance#

Avoid calling ray.get() unnecessarily for intermediate steps. Work with object refs directly, and call ray.get() only at the end to get the final result.

When you call ray.get(), the objects must be transferred to the worker or node that makes the call. If you don’t need to manipulate an object, you probably don’t need to call ray.get() on it.

Typically, wait as long as possible before you call ray.get(), or design your program so that it doesn’t need to call ray.get() at all.

Code example#

Anti-pattern:

import ray
import numpy as np

ray.init()


@ray.remote
def generate_rollout():
    return np.ones((10000, 10000))


@ray.remote
def reduce(rollout):
    return np.sum(rollout)


# `ray.get()` downloads the result here.
rollout = ray.get(generate_rollout.remote())
# Now we have to reupload `rollout`
reduced = ray.get(reduce.remote(rollout))

../../_images/unnecessary-ray-get-anti.svg

Better approach:

# Don't need ray.get here.
rollout_obj_ref = generate_rollout.remote()
# Rollout object is passed by reference.
reduced = ray.get(reduce.remote(rollout_obj_ref))

../../_images/unnecessary-ray-get-better.svg

In the anti-pattern example, the call to ray.get() forces the large rollout to be transferred to the driver, and then again to the reduce worker.

In the fixed version, you pass only the object ref to the reduce task. The reduce worker implicitly calls ray.get() to fetch the rollout data directly from the generate_rollout worker, which avoids the extra copy to the driver.

Other anti-patterns related to ray.get() are the following: