Peer-to-peer arrangements between graphics processing units (GPUs) are provided herein. A method includes establishing synthetic devices representing GPUs in an address domain associated with a host processor, where the GPUs have a different address domain than the host processor. The method also includes forming a peer arrangement between the GPUs such that data transfers between the GPUs in the different address domain can be initiated by the host processor interfacing with the synthetic devices.