A method for manufacturing a semiconductor memory device includes forming a first polysilicon layer on a conductive layer, forming a second polysilicon layer stacked on the first polysilicon layer, and forming a third polysilicon layer stacked on the second polysilicon layer. In the method, a stacked structure of the first, second and third polysilicon layers is patterned into a plurality of stacked structures spaced apart from each other on the conductive layer. Ferroelectric dielectric layers are formed on respective second polysilicon layers of the plurality of stacked structures, and metal layers are formed on the ferroelectric dielectric layers.