A memory element can include an opening formed within at least one insulating layer formed on an etch stop layer that exposes a first electrode portion and the etch stop layer at a bottom of the opening; a second electrode portion, formed on at least a side surface of the opening and in contact with the first electrode portion, the second electrode portion not filling the opening and being substantially not formed over a top surface of the at least one insulating layer; and at least one memory layer formed on a top surface of the at least one insulating layer and in contact with the second electrode portion, the at least one memory layer being reversibly programmable between at least two impedance states. Methods of forming such memory elements are also disclosed.