According to one embodiment, a semiconductor memory device includes: first and second interconnect layers; a plurality of third interconnect layers stacked between the first and second interconnect layers; a first insulating layer passing through the plurality of third interconnect layers, and including one end that is in contact with a first face of the first interconnect layer; a first memory pillar including a first semiconductor layer passing through the plurality of third interconnect layers and a charge storage layer provided between the plurality of third interconnect layers and the first semiconductor layer. A distance between a third face of the first interconnect layer opposite to the first face and the second interconnect layer in the first direction, differs at a position corresponding to the first insulating layer from at positions corresponding to the third interconnect layers.