ray.rllib.utils.tf_utils.minimize_and_clip#

ray.rllib.utils.tf_utils.minimize_and_clip(optimizer: torch.optim.Optimizer | tf.keras.optimizers.Optimizer, objective: numpy.array | jnp.ndarray | tf.Tensor | torch.Tensor, var_list: List[tf.Variable], clip_val: float = 10.0) → List[Tuple[numpy.array | jnp.ndarray | tf.Tensor | torch.Tensor, numpy.array | jnp.ndarray | tf.Tensor | torch.Tensor]] | List[numpy.array | jnp.ndarray | tf.Tensor | torch.Tensor][source]#

Computes, then clips gradients using objective, optimizer and var list.

Ensures the norm of the gradients for each variable is clipped to clip_val.

Parameters:

  • optimizer – Either a shim optimizer (tf eager) containing a tf.GradientTape under self.tape or a tf1 local optimizer object.

  • objective – The loss tensor to calculate gradients on.

  • var_list – The list of tf.Variables to compute gradients over.

  • clip_val – The global norm clip value. Will clip around -clip_val and +clip_val.

Returns:

The resulting model gradients (list or tuples of grads + vars) corresponding to the input var_list.