Embodiments are directed to optimizing data transfers between heterogeneous memory arenas. In one scenario, a computer system receives an indication that a data chunk is to be transferred from a first memory arena to a third memory arena, and then determines that for the data chunk to be transferred from the first memory arena to the third arena, the data chunk is to be transferred from the first memory arena to a second memory arena, and from the second memory arena to the third memory arena. The computer system divides the data chunk into smaller data portions and copies a first data portion from the first memory arena to the second memory arena. The computer system then copies the first data portion from the second memory arena to the third memory arena and copies a second data portion from the first memory arena to the second memory arena in parallel.