Provided herein may be a semiconductor memory device and a method of manufacturing the semiconductor memory device. The semiconductor memory device includes a stacked body including interlayer insulating layers and a select line disposed between the interlayer insulating layers, a core insulating layer penetrating the stacked body, a semiconductor pattern extending along a sidewall of the core insulating layer and including an undoped area disposed between the select line and the core insulating layer, doped semiconductor patterns disposed between the semiconductor pattern and the interlayer insulating layers, and a gate insulating layer disposed between the semiconductor pattern and the select line.