ray.data.read_videos#

ray.data.read_videos(paths: str | List[str], *, filesystem: pyarrow.fs.FileSystem | None = None, arrow_open_stream_args: Dict[str, Any] | None = None, partition_filter: PathPartitionFilter | None = None, partitioning: Partitioning | None = None, include_paths: bool = False, include_timestamps: bool = False, ignore_missing_paths: bool = False, file_extensions: List[str] | None = ['mp4', 'mkv', 'mov', 'avi', 'wmv', 'flv', 'webm', 'm4v', '3gp', 'mpeg', 'mpg', 'ts', 'ogv', 'rm', 'rmvb', 'vob', 'asf', 'f4v', 'm2ts', 'mts', 'divx', 'xvid', 'mxf'], shuffle: Literal['files'] | None = None, concurrency: int | None = None, override_num_blocks: int | None = None, num_cpus: float | None = None, num_gpus: float | None = None, memory: float | None = None, ray_remote_args: Dict[str, Any] | None = None)[source]#

Creates a Dataset from video files.

Each row in the resulting dataset represents a video frame. The column names default to “frame”, “frame_index” and “frame_timestamp”.

Examples

>>> import ray
>>> path = "s3://anonymous@ray-example-data/basketball.mp4"
>>> ds = ray.data.read_videos(path)
>>> ds.schema()
Column       Type
------       ----
frame        ArrowTensorTypeV2(shape=(720, 1280, 3), dtype=uint8)
frame_index  int64
Parameters:

  • paths – A single file or directory, or a list of file or directory paths. A list of paths can contain both files and directories.

  • filesystem – The pyarrow filesystem implementation to read from. These filesystems are specified in the pyarrow docs. Specify this parameter if you need to provide specific configurations to the filesystem. By default, the filesystem is automatically selected based on the scheme of the paths. For example, if the path begins with s3://, the S3FileSystem is used.

  • arrow_open_stream_args – kwargs passed to pyarrow.fs.FileSystem.open_input_file. when opening input files to read.

  • partition_filter – A PathPartitionFilter. Use with a custom callback to read only selected partitions of a dataset.

  • partitioning – A Partitioning object that describes how paths are organized. Defaults to None.

  • include_paths – If True, include the path to each image. File paths are stored in the 'path' column.

  • include_timestmaps – If True, include the frame timestamps from the video as a 'frame_timestamp' column.

  • ignore_missing_paths – If True, ignores any file/directory paths in paths that are not found. Defaults to False.

  • file_extensions – A list of file extensions to filter files by.

  • concurrency – The maximum number of Ray tasks to run concurrently. Set this to control number of tasks to run concurrently. This doesn’t change the total number of tasks run or the total number of output blocks. By default, concurrency is dynamically decided based on the available resources.

  • ray_remote_args – kwargs passed to remote() in the read tasks.

  • num_cpus – The number of CPUs to reserve for each parallel read worker.

  • num_gpus – The number of GPUs to reserve for each parallel read worker. For example, specify num_gpus=1 to request 1 GPU for each parallel read worker.

  • memory – The heap memory in bytes to reserve for each parallel read worker.

Returns:

A Dataset containing video frames from the video files.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.