A semiconductor memory device including a substrate, bit lines, contacts, a dielectric layer, storage node pads and a capacitor structure. The bit lines are disposed on the substrate and include a plurality of first bit lines and at least one second bit line. The contacts are disposed on the substrate and alternately and separately disposed with the bit lines. The dielectric layer is disposed over the contacts and bit lines. The storage node pads are disposed in the dielectric layer and respectively contact the contacts. The capacitor structure is disposed on the storage node pads and includes a plurality of first capacitors and at least one second capacitor located above at least one second bit line. Therefore, the semiconductor memory device can achieve more optimized device performance.