A semiconductor device and a method of fabricating the semiconductor device are provided. The semiconductor device includes a substrate, a gate structure over the substrate, a source/drain regions adjacent to the pair of spacers in the substrate, an etch stop layer next to the gate structure and overlying the substrate, a contact plug extending into the source/drain region and partially overlapping the gate structure, a protective layer over the etch stop layer overlying the substrate and covering the etch stop layer next to the gate structure without the contact plug, and an interlayer dielectric layer over the protective layer. The contact plug has no contact-to-gate short issue to the gate structure.