A semiconductor memory device includes a semiconductor substrate, bit line structures, storage node contacts, isolation structures, a first spacer, a second spacer, and a third spacer. Each bit line structure is elongated in a first direction. The bit line structures are repeatedly arranged in a second direction. Each storage node contact and each isolation structure are disposed between two adjacent bit line structures. The first spacer is partly disposed between each isolation structure and the bit line structure adjacent to the isolation structure and partly disposed between each storage node contact and the bit line structure adjacent to the storage node contact. The second spacer is disposed between each storage node contact and the first spacer. The third spacer is disposed between each storage node contact and the second spacer. A thickness of the third spacer is less than a thickness of the second spacer in the second direction.