A structure (interconnect or memory structure) is provided that includes a first electrically conductive structure having a concave upper surface embedded in a first interconnect dielectric material layer. A metal-containing cap having a convex bottom surface directly contacts the concave upper surface of the first electrically conductive structure. A metal-containing structure having a planar bottommost surface directly contacts a planar topmost surface of the metal-containing cap. A second electrically conductive structure contacts the planar topmost surface of the metal-containing structure. A second interconnect dielectric material layer is present on the first interconnect dielectric material layer and is located laterally adjacent to an upper portion of the metal-containing cap, the metal-containing structure, and the second electrically conductive structure.