There are provided a semiconductor memory device and a manufacturing method thereof. The semiconductor memory device includes: a first etch stop layer; a source layer on the first etch stop layer; a second etch stop layer on the source layer; a stack structure on the second etch stop layer; and a channel structure penetrating the first and second etch stop layers, the source layer, and the stack structure, the channel structure being electrically connected to the source layer. A material of each of the first and second etch stop layers has an etch selectivity with respect to a material of the source layer.