Anti-pattern: Closure capturing large objects harms performance

Don’t capture large objects in the closure of a remote function or class. Use the object store instead.

When you define a ray.remote function or class, it’s easy to accidentally capture large objects of more than a few MB implicitly in the definition. This can lead to slow performance or even out-of-memory (OOM) errors, because Ray isn’t designed to handle large serialized functions or classes.

To resolve this problem for large objects, use one of the following two options:

• Use ray.put() to put the large objects in the Ray object store, and then pass object refs as arguments to the remote functions or classes. Better approach #1 in the code example shows this option.

• Create the large objects inside the remote functions or classes by passing a lambda method. Better approach #2 shows this option. It’s also the only option for using unserializable objects.

Code example

Anti-pattern:

import ray
import numpy as np

ray.init()

large_object = np.zeros(10 * 1024 * 1024)


@ray.remote
def f1():
    return len(large_object)  # large_object is serialized along with f1!


ray.get(f1.remote())

Better approach #1:

large_object_ref = ray.put(np.zeros(10 * 1024 * 1024))


@ray.remote
def f2(large_object):
    return len(large_object)


# Large object is passed through object store.
ray.get(f2.remote(large_object_ref))

Better approach #2:

large_object_creator = lambda: np.zeros(10 * 1024 * 1024)  # noqa E731


@ray.remote
def f3():
    large_object = (
        large_object_creator()
    )  # Lambda is small compared with the large object.
    return len(large_object)


ray.get(f3.remote())