send_multigpu

ray.util.collective.collective.send_multigpu(tensor: Any, dst_rank: int, dst_gpu_index: int, group_name: str = 'default', n_elements: int = 0)

Send a tensor to a remote GPU synchronously.

The function assumes each process owns >1 GPUs, and the sender process and receiver process has equal number of GPUs.

Parameters:

• tensor (Any) – the tensor to send, located on a GPU.

• dst_rank (int) – the rank of the destination process.

• dst_gpu_index (int) – the destination gpu index.

• group_name (str) – the name of the collective group.

• n_elements (int) – if specified, send the next n elements from the starting address of tensor.