A method for forming a semiconductor device includes forming a gate structure over a semiconductor substrate. The gate structure includes a gate electrode, at least two hard mask (HM) layers over the gate electrode, and a spacer abutting a side wall of the gate electrode and the at least two hard mask layers. The method further comprises forming a contact etch stop layer (CESL) over the gate structure, exposing at least one of the HM layers after forming the CESL, and removing the exposed at least one of the HM layers.