class ray.rllib.core.rl_module.marl_module.MultiAgentRLModule(*args, **kwargs)[source]#

Bases: ray.rllib.core.rl_module.rl_module.RLModule

Base class for multi-agent RLModules.

This class holds a mapping from module_ids to the underlying RLModules. It provides a convenient way of accessing each individual module, as well as accessing all of them with only one API call. Whether or not a given module is trainable is determined by the caller of this class (not the instance of this class itself).

The extension of this class can include any arbitrary neural networks as part of the multi-agent module. For example, a multi-agent module can include a shared encoder network that is used by all the individual RLModules. It is up to the user to decide how to implement this class.

The default implementation assumes the data communicated as input and output of the APIs in this class are MultiAgentBatch types. The MultiAgentRLModule simply loops through each module_id, and runs the forward pass of the corresponding RLModule object with the associated SampleBatch within the MultiAgentBatch. It also assumes that the underlying RLModules do not share any parameters or communication with one another. The behavior of modules with such advanced communication would be undefined by default. To share parameters or communication between the underlying RLModules, you should implement your own MultiAgentRLModule subclass.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.


add_module(module_id, module, *[, override])

Adds a module at run time to the multi-agent module.


Returns a multi-agent wrapper around this module.

forward_exploration(batch, **kwargs)

Forward-pass during exploration, called from the sampler.

forward_inference(batch, **kwargs)

Forward-pass during evaluation, called from the sampler.

forward_train(batch, **kwargs)

Forward-pass during training called from the learner.


Returns the action distribution class for this RLModule used for exploration.


Returns the action distribution class for this RLModule used for inference.


Returns the initial state of the module.


Returns the state of the multi-agent module.


Returns the action distribution class for this RLModule used for training.


Returns the input specs of the forward_exploration method.


Returns the input specs of the forward_inference method.


Returns the input specs of the forward_train method.


Returns a keys view over the module IDs in this MultiAgentRLModule.

load_state(path[, modules_to_load])

Loads the weights of an MultiAgentRLModule from dir.

remove_module(module_id, *[, ...])

Removes a module at run time from the multi-agent module.


Saves the weights of this MultiAgentRLModule to dir.


Sets the state of the multi-agent module.


Sets up the underlying RLModules.


Returns the underlying module if this module is a wrapper.