Provided herein are a semiconductor device and a method of manufacturing the semiconductor device. The semiconductor device includes an etch stop pattern having a top surface and a sidewall disposed over a gate stack having interlayer insulating layers alternately stacked with conductive patterns. The semiconductor device also includes a plurality of channel structures passing through the etch stop pattern and the gate stack. The semiconductor device further includes an insulating layer extending to cover the top surface and the sidewall of the etch stop pattern, wherein a depression is included in a sidewall of the insulating layer. The semiconductor device additionally includes a contact plug passing through the insulating layer so that the contact plug is coupled to a channel structure of the plurality of channel structures.