ray.data.DataContext.get_current

static DataContext.get_current() → DataContext

Get or create the current DataContext.

When a Dataset is created, the current DataContext will be sealed. Changes to DataContext.get_current() will not impact existing Datasets.

Examples

context = ray.data.DataContext.get_current()

context.target_max_block_size = 100 * 1024 ** 2
ds1 = ray.data.range(1)
context.target_max_block_size = 1 * 1024 ** 2
ds2 = ray.data.range(1)

# ds1's target_max_block_size will be 100MB
ds1.take_all()
# ds2's target_max_block_size will be 1MB
ds2.take_all()

Developer notes: Avoid using DataContext.get_current() in data internal components, use the DataContext object captured in the Dataset and pass it around as arguments.