Embodiments of semiconductor devices and methods for forming the same are disclosed. In an example, a semiconductor device includes at least one dielectric layer pair including a first dielectric layer and a second dielectric layer different from the first dielectric layer, an interlayer dielectric (ILD) layer in contact with the at least one dielectric layer pair, and one or more capacitors each extending vertically through the ILD layer and in contact with the at least one dielectric layer pair.