ray.data.DataContext#

class ray.data.DataContext(target_max_block_size: int, target_shuffle_max_block_size: int, target_min_block_size: int, streaming_read_buffer_size: int, enable_pandas_block: bool, actor_prefetcher_enabled: bool, use_push_based_shuffle: bool, pipeline_push_based_shuffle_reduce_tasks: bool, scheduling_strategy: None | str | PlacementGroupSchedulingStrategy | NodeAffinitySchedulingStrategy | NodeLabelSchedulingStrategy, scheduling_strategy_large_args: None | str | PlacementGroupSchedulingStrategy | NodeAffinitySchedulingStrategy | NodeLabelSchedulingStrategy, large_args_threshold: int, use_polars: bool, eager_free: bool, decoding_size_estimation: bool, min_parallelism: bool, enable_tensor_extension_casting: bool, enable_auto_log_stats: bool, verbose_stats_log: bool, trace_allocations: bool, execution_options: ExecutionOptions, use_ray_tqdm: bool, enable_progress_bars: bool, enable_get_object_locations_for_metrics: bool, use_runtime_metrics_scheduling: bool, write_file_retry_on_errors: List[str], warn_on_driver_memory_usage_bytes: int, actor_task_retry_on_errors: bool | List[BaseException])[source]#

Singleton for shared Dataset resources and configurations.

This object is automatically propagated to workers and can be retrieved from the driver and remote workers via DataContext.get_current().

DeveloperAPI: This API may change across minor Ray releases.

Methods

__init__

Private constructor (use get_current() instead).

get_config

Get the value for a key-value style config.

get_current

Get or create a singleton context.

remove_config

Remove a key-value style config.

set_config

Set the value for a key-value style config.