ray.rllib.offline.offline_prelearner.OfflinePreLearner#

class ray.rllib.offline.offline_prelearner.OfflinePreLearner(*, config: AlgorithmConfig, spaces: Tuple[gymnasium.Space, gymnasium.Space] | None = None, module_state: Dict[str, Any] | None = None, **kwargs: Dict[str, Any])[source]#

Maps raw ingested data to episodes and runs the Learner pipeline.

OfflinePreLearner is meant to be used by RLlib to build a Ray Data pipeline using ray.data.Dataset.map_batches when ingesting data for offline training. Ray data is thereby used under the hood to parallelize the data transformation.

It’s basic function is to: (1) Convert the dataset into RLlib’s native episode format (SingleAgentEpisode, since MultiAgentEpisode is not supported yet). (2) Apply the learner connector pipeline to episodes to create batches that are ready to be trained on (can be passed in Learner.update methods).

OfflinePreLearner can be overridden to implement custom logic and passed into AlgorithmConfig.offline(prelearner_class).

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods