class ray.data.datasource.PathPartitionFilter(path_partition_parser: ray.data.datasource.partitioning.PathPartitionParser, filter_fn: Callable[[Dict[str, str]], bool])[source]#

Bases: object

Partition filter for path-based partition formats.

Used to explicitly keep or reject files based on a custom filter function that takes partition keys and values parsed from the file’s path as input.

PublicAPI (beta): This API is in beta and may change before becoming stable.


__init__(path_partition_parser, filter_fn)

Creates a new path-based partition filter based on a parser.

of(filter_fn[, style, base_dir, ...])

Creates a path-based partition filter using a flattened argument list.



Returns the path partition parser for this filter.