Contents

Learner API

Contents

Learner API#

Learner specifications and configurations#

`FrameworkHyperparameters`	The framework specific hyper-parameters.
`LearnerHyperparameters`	Hyperparameters for a Learner, derived from a subset of AlgorithmConfig values.

TorchLearner configurations#

TorchCompileWhatToCompile

Enumerates schemes of what parts of the TorchLearner can be compiled.

Constructor#

`Learner`	Base class for Learners.
`Learner.build`	Builds the Learner.
`Learner._check_is_built`
`Learner._make_module`	Construct the multi-agent RL module for the learner.

Performing Updates#

`Learner.update`	Do `num_iters` minibatch updates given the original batch.
`Learner._update`	Contains all logic for an in-graph/traceable update step.
`Learner.additional_update`	Apply additional non-gradient based updates to this Algorithm.
`Learner.additional_update_for_module`	Apply additional non-gradient based updates for a single module.
`Learner._convert_batch_type`	Converts the elements of a MultiAgentBatch to Tensors on the correct device.

Computing Losses#

`Learner.compute_loss`	Computes the loss for the module being optimized.
`Learner.compute_loss_for_module`	Computes the loss for a single module.
`Learner._is_module_compatible_with_learner`	Check whether the module is compatible with the learner.
`Learner._get_tensor_variable`	Returns a framework-specific tensor variable with the initial given value.

Configuring Optimizers#

`Learner.configure_optimizers_for_module`	Configures an optimizer for the given module_id.
`Learner.configure_optimizers`	Configures, creates, and registers the optimizers for this Learner.
`Learner.register_optimizer`	Registers an optimizer with a ModuleID, name, param list and lr-scheduler.
`Learner.get_optimizers_for_module`	Returns a list of (optimizer_name, optimizer instance)-tuples for module_id.
`Learner.get_optimizer`	Returns the optimizer object, configured under the given module_id and name.
`Learner.get_parameters`	Returns the list of parameters of a module.
`Learner.get_param_ref`	Returns a hashable reference to a trainable parameter.
`Learner.filter_param_dict_for_optimizer`	Reduces the given ParamDict to contain only parameters for given optimizer.
`Learner._check_registered_optimizer`	Checks that the given optimizer and parameters are valid for the framework.
`Learner._set_optimizer_lr`	Updates the learning rate of the given local optimizer.
`Learner._get_clip_function`	Returns the gradient clipping function to use, given the framework.

Gradient Computation#

`Learner.compute_gradients`	Computes the gradients based on the given losses.
`Learner.postprocess_gradients`	Applies potential postprocessing operations on the gradients.
`Learner.postprocess_gradients_for_module`	Applies postprocessing operations on the gradients of the given module.
`Learner.apply_gradients`	Applies the gradients to the MultiAgentRLModule parameters.

Saving, Loading, Checkpointing, and Restoring States#

`Learner.save_state`	Save the state of the learner to path
`Learner.load_state`	Load the state of the learner from path
`Learner._save_optimizers`	Save the state of the optimizer to path
`Learner._load_optimizers`	Load the state of the optimizer from path
`Learner.get_state`	Get the state of the learner.
`Learner.set_state`	Set the state of the learner.
`Learner.get_optimizer_state`	Returns the state of all optimizers currently registered in this Learner.
`Learner.set_optimizer_state`	Sets the state of all optimizers currently registered in this Learner.
`Learner._get_metadata`

Adding and Removing Modules#

`Learner.add_module`	Add a module to the underlying MultiAgentRLModule and the Learner.
`Learner.remove_module`	Remove a module from the Learner.

Managing Results#

`Learner.compile_results`	Compile results from the update in a numpy-friendly format.
`Learner.register_metric`	Registers a single key/value metric pair for loss- and gradient stats.
`Learner.register_metrics`	Registers several key/value metric pairs for loss- and gradient stats.
`Learner._check_result`	Checks whether the result has the correct format.

LearnerGroup API#

Configuring a LearnerGroup#

The spec for constructing Learner actors.

Coordinator of Learners.