ray.rllib.utils.torch_utils.clip_gradients#

ray.rllib.utils.torch_utils.clip_gradients(gradients_dict: ParamDict, *, grad_clip: float | None = None, grad_clip_by: str = 'value') → numpy.array | jnp.ndarray | tf.Tensor | torch.Tensor[source]#

Performs gradient clipping on a grad-dict based on a clip value and clip mode.

Changes the provided gradient dict in place.

Parameters:

  • gradients_dict – The gradients dict, mapping str to gradient tensors.

  • grad_clip – The value to clip with. The way gradients are clipped is defined by the grad_clip_by arg (see below).

  • grad_clip_by – One of ‘value’, ‘norm’, or ‘global_norm’.

Returns:

If grad_clip_by`="global_norm" and `grad_clip is not None, returns the global norm of all tensors, otherwise returns None.