A method of forming a semiconductor device includes forming a dummy metal gate layer including work function metals directly on a base insulator, diffusing the work function metals into the base insulator by annealing, removing the dummy metal gate layer by a wet etching, forming a metal gate on the base insulator, and forming a high-k insulator on the metal gate.