A method includes forming a gate dielectric layer, forming a metal gate strip over a bottom portion of the gate dielectric layer, and performing a first etching process on the metal gate strip to remove a portion of the metal gate strip. The first etching process is performed anisotropically. After the first etching process, a second etching process is performed on the metal gate strip to remove a residue portion of the metal gate strip. The second etching process includes an isotropic etching process. A dielectric material is filled into a recess left by the etched portion and the etched residue portion of the metal gate strip.