According to an embodiment, a semiconductor memory device comprises a stacked body, a semiconductor layer, a charge accumulation layer, and a first layer. The stacked body includes a plurality of control gate electrodes stacked above a substrate. The semiconductor layer has one end connected to the substrate and faces the plurality of control gate electrodes. The charge accumulation layer is positioned between the control gate electrode and the semiconductor layer. The first layer is surrounded by the stacked body in a plane parallel to the substrate. Moreover, a width in a first direction parallel to the substrate, of the first layer is larger than a width in the first direction of the semiconductor layer.