A semiconductor memory device includes a conductive layer; a plurality of electrode layers stacked on the conductive layer; a semiconductor pillar extending through the electrode layers in a stacking direction and electrically connected to the conductive layer; and an insulating layer positioned between the semiconductor pillar and the electrode layers and extending along the semiconductor pillar. The semiconductor pillar has a channel portion extending through the electrode layers and a high impurity concentration portion positioned at a bottom end on a side of the conductive layer. The high impurity concentration portion includes an impurity of a higher concentration than an impurity concentration in the channel portion. The insulating layer has an end portion extending toward a center of the bottom end of the semiconductor pillar, and a boundary of the channel portion and the high impurity concentration portion is positioned above the end portion of the insulating layer.