read_videos#

ray.data.read_videos(paths: str | List[str], *, filesystem: pyarrow.fs.FileSystem | None = None, arrow_open_stream_args: Dict[str, Any] | None = None, partition_filter: PathPartitionFilter | None = None, partitioning: Partitioning | None = None, include_paths: bool = False, include_timestamps: bool = False, fps: int | None = None, resize: Tuple[int, int] | None = None, ignore_missing_paths: bool = False, file_extensions: List[str] | None = ['mp4', 'mkv', 'mov', 'avi', 'wmv', 'flv', 'webm', 'm4v', '3gp', 'mpeg', 'mpg', 'ts', 'ogv', 'rm', 'rmvb', 'vob', 'asf', 'f4v', 'm2ts', 'mts', 'divx', 'xvid', 'mxf'], shuffle: Literal['files'] | None = None, concurrency: int | None = None, override_num_blocks: int | None = None, num_cpus: float | None = None, num_gpus: float | None = None, memory: float | None = None, label_selector: Dict[str, str] | None = None, fallback_strategy: List[Dict[str, Any]] | None = None, max_calls: int | None = None, resources: Dict[str, float] | None = None, accelerator_type: str | None = None, runtime_env: Dict[str, Any] | None = None, ray_remote_args: Dict[str, Any] | None = None)[source]#

Creates a Dataset from video files.

Each row in the resulting dataset represents a video frame. The column names default to “frame”, “frame_index” and “frame_timestamp”.

Examples

>>> import ray
>>> path = "s3://anonymous@ray-example-data/basketball.mp4"
>>> ds = ray.data.read_videos(path)
>>> ds.schema()
Column       Type
------       ----
frame        ArrowTensorTypeV2(shape=(720, 1280, 3), dtype=uint8)
frame_index  int64

Subsample frames to a target frame rate or resize frames to (height, width):

>>> ds = ray.data.read_videos(path, fps=5)
>>> ds = ray.data.read_videos(path, resize=(240, 320))
Parameters:

  • paths (str | List[str]) – A single file or directory, or a list of file or directory paths. A list of paths can contain both files and directories.

  • filesystem (pyarrow.fs.FileSystem | None) – The pyarrow filesystem implementation to read from. These filesystems are specified in the pyarrow docs. Specify this parameter if you need to provide specific configurations to the filesystem. By default, the filesystem is automatically selected based on the scheme of the paths. For example, if the path begins with s3://, the S3FileSystem is used.

  • arrow_open_stream_args (Dict[str, Any] | None) – kwargs passed to pyarrow.fs.FileSystem.open_input_file. when opening input files to read.

  • partition_filter (PathPartitionFilter | None) – A PathPartitionFilter. Use with a custom callback to read only selected partitions of a dataset.

  • partitioning (Partitioning | None) – A Partitioning object that describes how paths are organized. Defaults to None.

  • include_paths (bool) – If True, include the path to each image. File paths are stored in the 'path' column.

  • include_timestamps (bool) – If True, include the frame timestamps from the video as a 'frame_timestamp' column.

  • fps (int | None) – If specified, subsample frames to approximately this target frame rate instead of decoding every frame. Frames are kept at a fixed stride of max(1, round(source_fps / fps)), so the effective rate may differ slightly from fps. frame_index remains the index of the frame in the original video. If fps is greater than or equal to the source frame rate, all frames are kept.

  • resize (Tuple[int, int] | None) – If specified, resize each frame to (height, width) at decode time. This ordering mirrors the size parameter of read_images(). If unspecified, frames retain their original shape.

  • ignore_missing_paths (bool) – If True, ignores any file/directory paths in paths that are not found. Defaults to False.

  • file_extensions (List[str] | None) – A list of file extensions to filter files by.

  • shuffle (Literal['files'] | None) – If "files", randomly shuffle input files order before read. Defaults to None.

  • concurrency (int | None) – The maximum number of Ray tasks to run concurrently. Set this to control number of tasks to run concurrently. This doesn’t change the total number of tasks run or the total number of output blocks. By default, concurrency is dynamically decided based on the available resources.

  • override_num_blocks (int | None) – Override the number of output blocks from all read tasks. By default, the number of output blocks is dynamically decided based on input data size and available resources. You shouldn’t manually set this value in most cases.

  • num_cpus (float | None) – The number of CPUs to reserve for each parallel read worker.

  • num_gpus (float | None) – The number of GPUs to reserve for each parallel read worker. For example, specify num_gpus=1 to request 1 GPU for each parallel read worker.

  • memory (float | None) – The heap memory in bytes to reserve for each parallel read worker.

  • label_selector (Dict[str, str] | None) – Labels required on the node where each read task runs.

  • fallback_strategy (List[Dict[str, Any]] | None) – Alternative label requirements that Ray tries in order if label_selector can’t be satisfied.

  • max_calls (int | None) – The maximum number of read tasks a worker runs before exiting.

  • resources (Dict[str, float] | None) – Custom resources to reserve for each read task, expressed as a mapping from resource name to quantity.

  • accelerator_type (str | None) – The accelerator type required for each read task.

  • runtime_env (Dict[str, Any] | None) – The runtime environment to use for each read task.

  • ray_remote_args (Dict[str, Any] | None) – Additional options passed to ray.remote() for each read task. This argument is deprecated and will be removed in Ray 2.64. Use the named remote parameters instead.

Returns:

A Dataset containing video frames from the video files.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

On this page