Learner API
Contents
Learner API#
Learner specifications and configurations#
The framework specific hyper-parameters. |
|
Hyperparameters for a Learner, derived from a subset of AlgorithmConfig values. |
TorchLearner configurations#
Enumerates schemes of what parts of the TorchLearner can be compiled. |
Constructor#
Base class for Learners. |
|
Builds the Learner. |
|
Construct the multi-agent RL module for the learner. |
Performing Updates#
Do |
|
Contains all logic for an in-graph/traceable update step. |
|
Apply additional non-gradient based updates to this Algorithm. |
|
Apply additional non-gradient based updates for a single module. |
|
Converts the elements of a MultiAgentBatch to Tensors on the correct device. |
Computing Losses#
Computes the loss for the module being optimized. |
|
Computes the loss for a single module. |
|
Check whether the module is compatible with the learner. |
|
Returns a framework-specific tensor variable with the initial given value. |
Configuring Optimizers#
Configures an optimizer for the given module_id. |
|
Configures, creates, and registers the optimizers for this Learner. |
|
Registers an optimizer with a ModuleID, name, param list and lr-scheduler. |
|
Returns a list of (optimizer_name, optimizer instance)-tuples for module_id. |
|
Returns the optimizer object, configured under the given module_id and name. |
|
Returns the list of parameters of a module. |
|
Returns a hashable reference to a trainable parameter. |
|
Reduces the given ParamDict to contain only parameters for given optimizer. |
|
Checks that the given optimizer and parameters are valid for the framework. |
|
Updates the learning rate of the given local optimizer. |
|
Returns the gradient clipping function to use, given the framework. |
Gradient Computation#
Computes the gradients based on the given losses. |
|
Applies potential postprocessing operations on the gradients. |
|
Applies postprocessing operations on the gradients of the given module. |
|
Applies the gradients to the MultiAgentRLModule parameters. |
Saving, Loading, Checkpointing, and Restoring States#
Save the state of the learner to path |
|
Load the state of the learner from path |
|
Save the state of the optimizer to path |
|
Load the state of the optimizer from path |
|
Get the state of the learner. |
|
Set the state of the learner. |
|
Returns the state of all optimizers currently registered in this Learner. |
|
Sets the state of all optimizers currently registered in this Learner. |
|
Adding and Removing Modules#
Add a module to the underlying MultiAgentRLModule and the Learner. |
|
Remove a module from the Learner. |
Managing Results#
Compile results from the update in a numpy-friendly format. |
|
Registers a single key/value metric pair for loss- and gradient stats. |
|
Registers several key/value metric pairs for loss- and gradient stats. |
|
Checks whether the result has the correct format. |
LearnerGroup API#
Configuring a LearnerGroup#
The spec for constructing Learner actors. |
Coordinator of Learners. |