A method of fabricating a replacement gate stack for a semiconductor device includes the following steps after removal of a dummy gate: growing a high-k dielectric layer over the area vacated by the dummy gate; depositing a thin metal layer over the high-k dielectric layer; depositing a sacrificial layer over the thin metal layer; annealing the structure at a high temperature of not less than 800° C.; removing the sacrificial layer; and depositing a metal layer of low resistivity metal for gap fill. Optionally, a second annealing step can be performed after the first anneal. This second anneal is performed as a millisecond anneal using a flash lamp or a laser.