A semiconductor memory device includes a semiconductor substrate, a plurality of element isolations, a plurality of first stacked bodies, a second stacked body, and an interlayer insulating film. Distance between each of the first stacked bodies and the second stacked body is longer than distance between adjacent ones of the first stacked bodies. A first void is formed in the interlayer insulating film between the first stacked bodies. A second void is formed in the interlayer insulating film between one of the first stacked bodies and the second stacked body. And, a lower end of the second void is located above a lower end of the first void.