OfflinePreLearner#
- class ray.rllib.offline.offline_prelearner.OfflinePreLearner(*, config: AlgorithmConfig, spaces: Tuple[gymnasium.Space, gymnasium.Space] | None = None, module_state: Dict[str, Any] | None = None, **kwargs: Dict[str, Any])[source]#
Bases:
objectMaps raw ingested data to episodes and runs the Learner pipeline.
OfflinePreLearner is meant to be used by RLlib to build a Ray Data pipeline using
ray.data.Dataset.map_batcheswhen ingesting data for offline training. Ray data is thereby used under the hood to parallelize the data transformation.It’s basic function is to: (1) Convert the dataset into RLlib’s native episode format (
SingleAgentEpisode, sinceMultiAgentEpisodeis not supported yet). (2) Apply the learner connector pipeline to episodes to create batches that are ready to be trained on (can be passed inLearner.updatemethods).OfflinePreLearner can be overridden to implement custom logic and passed into
AlgorithmConfig.offline(prelearner_class).- Parameters:
config (AlgorithmConfig) – The AlgorithmConfig to build the RLModule and the Learner connector pipeline from.
spaces (Tuple[gymnasium.Space, gymnasium.Space] | None) – The observation and action space to build the Learner connector pipeline with.
module_state (Dict[str, Any] | None) – The RLModule state to set on the newly built RLModule.
**kwargs (Dict[str, Any]) – Deprecated. A
module_speckeyword argument is still accepted, with a deprecation warning, in place ofconfig.rl_module_spec.
PublicAPI (alpha): This API is in alpha and may change before becoming stable.
Methods