Note
Ray 2.40 uses RLlib’s new API stack by default. The Ray team has mostly completed transitioning algorithms, example scripts, and documentation to the new code base.
If you’re still using the old API stack, see New API stack migration guide for details on how to migrate.
Callback APIs#
Callback APIs enable you to inject code into an experiment, an Algorithm, and the subcomponents of an Algorithm.
You can either subclass RLlibCallback
and implement
one or more of its methods, like on_algorithm_init()
,
or pass respective arguments to the callbacks()
method of an Algorithm’s config, like
config.callbacks(on_algorithm_init=lambda algorithm, **kw: print('algo initialized!'))
.
from ray.rllib.algorithms.dqn import DQNConfig
from ray.rllib.callbacks.callbacks import RLlibCallback
class MyCallback(RLlibCallback):
def on_algorithm_init(self, *, algorithm, metrics_logger, **kwargs):
print(f"Algorithm {algorithm} has been initialized!")
config = (
DQNConfig()
.callbacks(MyCallback)
)
from ray.rllib.algorithms.dqn import DQNConfig
config = (
DQNConfig()
.callbacks(
on_algorithm_init=(
lambda algorithm, **kwargs: print(f"Algorithm {algorithm} has been initialized!")
)
)
)
See Callbacks for more details on how to write and configure callbacks.
Methods to implement for custom behavior#
RLlibCallback#
Abstract base class for RLlib callbacks (similar to Keras callbacks). |
Callbacks invoked in Algorithm#
The main Algorithm process always executes the following callback methods:
Callback run when a new Algorithm instance has finished setup. |
|
Called at the end of |
|
Called at the end of Algorithm.train(). |
|
Callback before evaluation starts. |
|
Runs when the evaluation is done. |
|
Callback run after one or more EnvRunner actors have been recreated. |
|
Callback run when an Algorithm has loaded a new state from a checkpoint. |
Callbacks invoked in EnvRunner#
The EnvRunner actors always execute the following callback methods:
Callback run when a new environment object has been created. |
|
Callback run when a new episode is created (but has not started yet!). |
|
Callback run right after an Episode has been started. |
|
Called on each episode step (after the action(s) has/have been logged). |
|
Called when an episode is done (after terminated/truncated have been logged). |