A semiconductor memory device according to an embodiment includes: a stacked body alternately stacking first insulating layers and gate electrode layers in a first direction; first to third semiconductor layers in the stacked body extending in the first direction; first to third charge accumulation layers; and a second insulating layer in the stacked body extending in the first direction, the second insulating layer contacting the first semiconductor layer or the first charge accumulation layer in a plane perpendicular to the first direction. A first distance between two end surfaces of the gate electrode layer monotonically increases in the first direction in a first cross section parallel to the first direction. A second distance between two end surfaces of the gate electrode layer monotonically increases in the first direction, decreases, and then monotonically increases in a second cross section parallel to the first direction different from the first cross section.