Each memory cell in an array includes a vertical stack that comprises a bottom electrode, a memory element, and a top electrode. An etch stop dielectric layer is formed over the array of memory cells. A first dielectric matrix layer is formed over the etch stop dielectric layer. The top surface of the first dielectric matrix layer is raised in a memory array region relative to a logic region due to topography. The first dielectric matrix layer is planarized by performing a chemical mechanical planarization process using top portions of the etch stop dielectric layer. A second dielectric matrix layer is formed over the first dielectric matrix layer. Metallic cell contact structures are formed through the second dielectric matrix layer on a respective subset of the top electrodes over vertically protruding portions of the etch stop dielectric layer that laterally surround the array of vertical stacks.