ray.data.datasource.FileBasedDatasource.resolve_compression#

FileBasedDatasource.resolve_compression(path: str, open_args: Dict[str, Any]) → str | None[source]#

Resolves the compression format for a stream.

Parameters:

path – The file path to resolve compression for.
open_args – kwargs passed to pyarrow.fs.FileSystem.open_input_stream when opening input files to read.

Returns:

The compression format (e.g., “gzip”, “snappy”, “bz2”) or None if no compression is detected or specified.