ray.rllib.callbacks.callbacks.RLlibCallback.on_episode_end#

RLlibCallback.on_episode_end(*, episode: SingleAgentEpisode | MultiAgentEpisode | EpisodeV2, env_runner: EnvRunner | None = None, metrics_logger: MetricsLogger | None = None, env: gymnasium.Env | None = None, env_index: int, rl_module: RLModule | None = None, worker: EnvRunner | None = None, base_env: BaseEnv | None = None, policies: Dict[str, Policy] | None = None, **kwargs) → None[source]#

Called when an episode is done (after terminated/truncated have been logged).

The exact time of the call of this callback is after env.step([action]) and also after the results of this step (observation, reward, terminated, truncated, infos) have been logged to the given episode object, where either terminated or truncated were True:

  • The env is stepped: final_obs, rewards, ... = env.step([action])

  • The step results are logged episode.add_env_step(final_obs, rewards)

  • Callback on_episode_step is fired.

  • Another env-to-module connector call is made (even though we won’t need any RLModule forward pass anymore). We make this additional call to ensure that in case users use the connector pipeline to process observations (and write them back into the episode), the episode object has all observations - even the terminal one - properly processed.

  • —> This callback on_episode_end() is fired. <—

  • The episode is finalized (i.e. lists of obs/rewards/actions/etc.. are converted into numpy arrays).

Parameters:

  • episode – The terminated/truncated SingleAgent- or MultiAgentEpisode object (after env.step() that returned terminated=True OR truncated=True and after the returned obs, rewards, etc.. have been logged to the episode object). Note that this method is still called before(!) the episode object is finalized, meaning all its timestep data is still present in lists of individual timestep data.

  • env_runner – Reference to the EnvRunner running the env and episode.

  • metrics_logger – The MetricsLogger object inside the env_runner. Can be used to log custom metrics during env/episode stepping.

  • env – The gym.Env or gym.vector.Env object running the started episode.

  • env_index – The index of the sub-environment that has just been terminated or truncated.

  • rl_module – The RLModule used to compute actions for stepping the env. In a single-agent setup, this is a (single-agent) RLModule, in a multi- agent setup, this will be a MultiRLModule.

  • kwargs – Forward compatibility placeholder.