Offline RL API#

Configuring Offline RL#

AlgorithmConfig.offline_data

Sets the config's offline data settings.

AlgorithmConfig.learners

Sets LearnerGroup and Learner worker related configurations.

Configuring Offline Recording EnvRunners#

AlgorithmConfig.env_runners

Sets the rollout worker configuration.

Constructing a Recording EnvRunner#

OfflineSingleAgentEnvRunner

The environment runner to record the single agent case.

Constructing OfflineData#

OfflineData

Loads offline data as a Ray Dataset and samples training batches from it.

OfflineData.__init__

Sampling from Offline Data#

OfflineData.sample

Samples a batch, or returns data iterators, from the offline data.

OfflineData.default_map_batches_kwargs

The default keyword arguments for the Dataset.map_batches() call.

OfflineData.default_iter_batches_kwargs

The default keyword arguments for the Dataset.iter_batches() call.

Constructing an OfflinePreLearner#

OfflinePreLearner

Maps raw ingested data to episodes and runs the Learner pipeline.

Transforming Data with an OfflinePreLearner#

SCHEMA

This is the default schema used if no input_read_schema is set in the config.

OfflinePreLearner.__call__

Maps raw ingested data to episodes and runs the Learner pipeline.

OfflinePreLearner._map_to_episodes

Maps a batch of data to episodes.