reducescatter

ray.util.collective.collective.reducescatter(tensor: Any, tensor_list: list, group_name: str = 'default', op: ReduceOp = ReduceOp.SUM)

Reducescatter a list of tensors across the group.

Reduce the list of the tensors across each process in the group, then scatter the reduced list of tensors – one tensor for each process.

Parameters:

• tensor (Any) – the resulted tensor on this process.

• tensor_list (list) – The list of tensors to be reduced and scattered.

• group_name (str) – the name of the collective group.

• op (ReduceOp) – The reduce operation.