MultiGPU In-node and Out of Node P2P Solutions

In-node

• Large BAR support BAR = Base Address Register Making the GPU memory visible BAR 1 Region Suppoted in Radeon Instinct MI25,MI8, MI6

• ROCr Base driver has P2P API support

• ROCr (HSA) AGENT API with Peer to Peer support.

• HCC Language Runtime support of P2P ROCr Agent API.

• HIP Language Runtime support of P2P P2P API’s model after CUDA P2P API’s.

• OpenCL Language Runtime P2P API Peer-to-Peer API with Autocopy support over Intel QPI bus

  • API name - clEnqueueBufferCopyP2PAMD

  • Releasing in OpenCL with ROCm 1.6.2

• HIP based Communication Primitives Helper Library to make it easier to use P2P - In Development

• ROCr level IPC Inter Process Communication API

• IPC is Supported in HIP API

Out of Node

• Remote DMA technology ( RDMA) Peer-to-Peer bridge driver for PeerDirect.

• libibverbs Linux RDMA library YES -since ROCm 1.0.

• PeerDirect Mellanox Peer API for Infiniband.

Standard Frameworks for Out of Node Communication

• OpenUCX UCX is a communication library implementing high-performance messaging for MPI/PGAS frameworks - In Development Source for ROCm.

• OpenMPI Open MPI Project is an open source Message Passing Interface https://www.open-mpi.org In Development.

• MPICH MPICH is a high-performance and widely portable implementation of the Message Passing Interface (MPI) standard (MPI-1, MPI-2 and MPI-3) In Development.

• OpenSHMEM Partitioned Global Address Space & Communication Library - In Development.

• OSU benchmark to test performance.