ray.rllib.core.rl_module.marl_module.MultiAgentRLModule#

class ray.rllib.core.rl_module.marl_module.MultiAgentRLModule(*args, **kwargs)[source]#

Bases: RLModule

Base class for multi-agent RLModules.

This class holds a mapping from module_ids to the underlying RLModules. It provides a convenient way of accessing each individual module, as well as accessing all of them with only one API call. Whether a given module is trainable is determined by the caller of this class (not the instance of this class itself).

The extension of this class can include any arbitrary neural networks as part of the multi-agent module. For example, a multi-agent module can include a shared encoder network that is used by all the individual RLModules. It is up to the user to decide how to implement this class.

The default implementation assumes the data communicated as input and output of the APIs in this class are MultiAgentBatch types. The MultiAgentRLModule simply loops through each module_id, and runs the forward pass of the corresponding RLModule object with the associated SampleBatch within the MultiAgentBatch. It also assumes that the underlying RLModules do not share any parameters or communication with one another. The behavior of modules with such advanced communication would be undefined by default. To share parameters or communication between the underlying RLModules, you should implement your own MultiAgentRLModule subclass.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

`add_module`	Adds a module at run time to the multi-agent module.
`as_multi_agent`	Returns a multi-agent wrapper around this module.
`foreach_module`	Calls the given function with each (module_id, module).
`forward_exploration`	Forward-pass during exploration, called from the sampler.
`forward_inference`	Forward-pass during evaluation, called from the sampler.
`forward_train`	Forward-pass during training called from the learner.
`from_checkpoint`	Creates a new Checkpointable instance from the given location and returns it.
`get`	Returns the module with the given module ID or default if not found in self.
`get_exploration_action_dist_cls`	Returns the action distribution class for this RLModule used for exploration.
`get_inference_action_dist_cls`	Returns the action distribution class for this RLModule used for inference.
`get_metadata`	Returns JSON writable metadata further describing the implementing class.
`get_train_action_dist_cls`	Returns the action distribution class for this RLModule used for training.
`input_specs_exploration`	Returns the input specs of the forward_exploration method.
`input_specs_inference`	Returns the input specs of the forward_inference method.
`input_specs_train`	Returns the input specs of the forward_train method.
`keys`	Returns a keys view over the module IDs in this MultiAgentRLModule.
`remove_module`	Removes a module at run time from the multi-agent module.
`restore_from_path`	Restores the state of the implementing class from the given path.
`save_to_path`	Saves the state of the implementing class (or `state`) to `path`.
`set_state`	Sets the state of the multi-agent module.
`setup`	Sets up the underlying RLModules.
`unwrapped`	Returns the underlying module if this module is a wrapper.
`update_default_view_requirements`	Updates default view requirements with the view requirements of this module.

Attributes

`CLASS_AND_CTOR_ARGS_FILE_NAME`
`METADATA_FILE_NAME`
`STATE_FILE_NAME`
`framework`