broadcast_multigpu

ray.util.collective.collective.broadcast_multigpu(tensor_list: list, src_rank: int = 0, src_tensor: int = 0, group_name: str = 'default')

Broadcast the tensor from a source GPU to all other GPUs.

Parameters:

• tensor_list (list) – the tensors to broadcast (src) or receive (dst).

• src_rank (int) – the rank of the source process.

• src_tensor (int) – the index of the source GPU on the source process.

• group_name (str) – the collective group name to perform broadcast.