A semiconductor device and methods for making the same are disclosed. The device may include: a first transistor structure; a second transistor structure; a capacitor structure comprising a trench in the substrate between the first and second transistor structures, the capacitor structure further comprising a doped layer over the substrate, a dielectric layer over the doped layer, and a conductive fill material over the dielectric layer; a first conductive contact from the first transistor structure to a first bit line; a second conductive contact from the second transistor to a non-volatile memory element; and a third conductive contact from the non-volatile memory element to a second bit line.