A method includes forming a gate stack, which includes a gate dielectric and a metal gate electrode over the gate dielectric. An inter-layer dielectric is formed on opposite sides of the gate stack. The gate stack and the inter-layer dielectric are planarized. The method further includes forming an inhibitor film on the gate stack, with at least a portion of the inter-layer dielectric exposed, selectively depositing a dielectric hard mask on the inter-layer dielectric, with the inhibitor film preventing the dielectric hard mask from being formed thereon, and etching to remove a portion of the gate stack, with the dielectric hard mask acting as a portion of a corresponding etching mask.