RLlib Annotations/Decorators

Throughout the RLlib codebase, we will use the following practical class-, method-, and function decorators:

ray.rllib.utils.annotations.override(cls)[source]

Decorator for documenting method overrides.

Parameters

cls (type) – The superclass that provides the overridden method. If this cls does not actually have the method, an error is raised.

Examples

>>> class TorchPolicy(Policy):
... ...
...     # Indicates that `TorchPolicy.loss()` overrides the parent
...     # Policy class' own `loss method. Leads to an error if Policy
...     # does not have a `loss` method.
...     @override(Policy)
...     def loss(self, model, action_dist, train_batch):
...         # ...
ray.rllib.utils.annotations.PublicAPI(obj)[source]

Decorator for documenting public APIs.

Public APIs are classes and methods exposed to end users of RLlib. You can expect these APIs to remain stable across RLlib releases.

Subclasses that inherit from a @PublicAPI base class can be assumed part of the RLlib public API as well (e.g., all trainer classes are in public API because Trainer is @PublicAPI).

In addition, you can assume all trainer configurations are part of their public API as well.

Examples

>>> # Indicates that the `Trainer` class is exposed to end users
... # of RLlib and will remain stable across RLlib releases.
... @PublicAPI
... class Trainer(tune.Trainable):
... ...
ray.rllib.utils.annotations.DeveloperAPI(obj)[source]

Decorator for documenting developer APIs.

Developer APIs are classes and methods explicitly exposed to developers for the purposes of building custom algorithms or advanced training strategies on top of RLlib internals. You can generally expect these APIs to be stable sans minor changes (but less stable than public APIs).

Subclasses that inherit from a @DeveloperAPI base class can be assumed part of the RLlib developer API as well.

Examples

>>> # Indicates that the `TorchPolicy` class is exposed to end users
... # of RLlib and will remain (relatively) stable across RLlib
... # releases.
... @DeveloperAPI
... class TorchPolicy(Policy):
... ...
ray.rllib.utils.annotations.ExperimentalAPI(obj)[source]

Decorator for documenting experimental APIs.

Experimental APIs are classes and methods that are in development and may change at any time in their development process. You should not expect these APIs to be stable until their tag is changed to DeveloperAPI or PublicAPI.

Subclasses that inherit from a @ExperimentalAPI base class can be assumed experimental as well.

Examples

>>> class TorchPolicy(Policy):
...     ...
...     # Indicates that the `TorchPolicy.loss` method is a new and
...     # experimental API and may change frequently in future
...     # releases.
...     @ExperimentalAPI
...     def loss(self, model, action_dist, train_batch):
...         # ...
ray.rllib.utils.annotations.OverrideToImplementCustomLogic(obj)[source]

Users should override this in their sub-classes to implement custom logic.

Used in Trainer and Policy to tag methods that need overriding, e.g. Policy.loss().

Examples

>>> @overrides(TorchPolicy)
... @OverrideToImplementCustomLogic
... def loss(self, ...):
...     # implement custom loss function here ...
...     # ... w/o calling the corresponding `super().loss()` method.
ray.rllib.utils.annotations.OverrideToImplementCustomLogic_CallToSuperRecommended(obj)[source]

Users should override this in their sub-classes to implement custom logic.

Thereby, it is recommended (but not required) to call the super-class’ corresponding method.

Used in Trainer and Policy to tag methods that need overriding, but the super class’ method should still be called, e.g. Trainer.setup().

Examples

>>> @overrides(Trainable)
... @OverrideToImplementCustomLogic_CallToSuperRecommended
... def setup(self, config):
...     # implement custom setup logic here ...
...     super().setup(config)
...     # ... or here (after having called super()'s setup method.