ray.rllib.utils.metrics.metrics_logger.MetricsLogger#

class ray.rllib.utils.metrics.metrics_logger.MetricsLogger[source]#

A generic class collecting and processing metrics in RL training and evaluation.

This class represents the main API used by all of RLlib’s components (internal and user facing) in order to log, collect, and process (reduce) stats during training and evaluation/inference.

It supports: - Logging of simple float/int values (for example a loss) over time or from parallel runs (n Learner workers, each one reporting a loss from their respective data shard). - Logging of images, videos, or other more complex data structures over time. - Reducing these collected values using a user specified reduction method (for example “min” or “mean”) and other settings controlling the reduction and internal data, such as sliding windows or EMA coefficients. - Resetting the logged values after a reduce() call in order to make space for new values to be logged.

import time
from ray.rllib.utils.metrics.metrics_logger import MetricsLogger
from ray.rllib.utils.test_utils import check

logger = MetricsLogger()

# 1) Logging float values (mean over window):
# Log some loss under the "loss" key. By default, all logged values
# under that key are averaged and reported back, once `reduce()` is called.
logger.log_value("loss", 0.001, reduce="mean", window=10)
logger.log_value("loss", 0.002)  # <- no need to repeat arg/options on same key
# Peek at the current (reduced) value of "loss":
check(logger.peek("loss"), 0.0015)  # <- expect average value
# Actually reduce the underlying Stats object(s).
results = logger.reduce()
check(results["loss"], 0.0015)

# 2) Logging float values (minimum over window):
# Log the minimum of loss values under the "min_loss" key.
logger.log_value("min_loss", 0.1, reduce="min", window=2)
logger.log_value("min_loss", 0.01)
logger.log_value("min_loss", 0.1)
logger.log_value("min_loss", 0.02)
# Peek at the current (reduced) value of "min_loss":
check(logger.peek("min_loss"), 0.02)  # <- expect min value (over window=2)
# Actually reduce the underlying Stats object(s).
results = logger.reduce()
check(results["min_loss"], 0.02)

# 3) Log n counts in different (remote?) components and merge them on the
# controller side.
remote_logger_1 = MetricsLogger()
remote_logger_2 = MetricsLogger()
main_logger = MetricsLogger()
remote_logger_1.log_value("count", 2, reduce="sum", clear_on_reduce=True)
remote_logger_2.log_value("count", 3, reduce="sum", clear_on_reduce=True)
# Reduce the two remote loggers ..
remote_results_1 = remote_logger_1.reduce()
remote_results_2 = remote_logger_2.reduce()
# .. then merge the two results into the controller logger.
main_logger.merge_and_log_n_dicts([remote_results_1, remote_results_2])
check(main_logger.peek("count"), 5)

# 4) Time blocks of code using EMA (coeff=0.1). Note that the higher the coeff
# (the closer to 1.0), the more short term the EMA turns out.
logger = MetricsLogger()

# First delta measurement:
with logger.log_time("my_block_to_be_timed", reduce="mean", ema_coeff=0.1):
    time.sleep(1.0)
# EMA should be ~1sec.
assert 1.1 > logger.peek("my_block_to_be_timed") > 0.9
# Second delta measurement (note that we don't have to repeat the args again, as
# the stats under that name have already been created above with the correct
# args).
with logger.log_time("my_block_to_be_timed"):
    time.sleep(2.0)
# EMA should be ~1.1sec.
assert 1.15 > logger.peek("my_block_to_be_timed") > 1.05

# When calling `reduce()`, the internal values list gets cleaned up (reduced)
# and reduction results are returned.
results = logger.reduce()
# EMA should be ~1.1sec.
assert 1.15 > results["my_block_to_be_timed"] > 1.05

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

`__init__`	Initializes a MetricsLogger instance.
`activate_tensor_mode`	Switches to tensor-mode, in which in-graph tensors can be logged.
`deactivate_tensor_mode`	Switches off tensor-mode.
`delete`	Deletes the given `key` from this metrics logger's stats.
`get_state`	Returns the current state of `self` as a dict.
`log_dict`	Logs all leafs (`Stats` or simple values) of a (nested) dict to this logger.
`log_time`	Measures and logs a time delta value under `key` when used with a with-block.
`log_value`	Logs a new value under a (possibly nested) key to the logger.
`merge_and_log_n_dicts`	Merges n dicts, generated by n parallel components, and logs the results.
`peek`	Returns the (reduced) value(s) found under the given key or key sequence.
`peek_results`	Performs `peek()` on any leaf element of an arbitrarily nested Stats struct.
`reduce`	Reduces all logged values based on their settings and returns a result dict.
`reset`	Resets all data stored in this MetricsLogger.
`set_state`	Sets the state of `self` to the given `state`.
`set_value`	Overrides the logged values under `key` with `value`.
`tensors_to_numpy`	Converts all previously logged and returned tensors back to numpy values.

Attributes

tensor_mode