reduce_multigpu#

ray.util.collective.collective.reduce_multigpu(tensor_list: list, dst_rank: int = 0, dst_tensor: int = 0, group_name: str = 'default', op: ReduceOp = ReduceOp.SUM)[source]#

Reduce the tensor across the group to the destination rank and destination tensor.

Parameters:

  • tensor_list (list) – the list of tensors to be reduced on this process; each tensor located on a GPU.

  • dst_rank (int) – the rank of the destination process.

  • dst_tensor (int) – the index of GPU at the destination.

  • group_name (str) – the collective group name to perform reduce.

  • op (ReduceOp) – The reduce operation.

On this page