A method that allows integrating complementary metal oxide semiconductor (CMOS) transistors and a non-volatile memory (NVM) transistor on a single substrate is provided. The NVM transistor includes a gate stack containing a high-k tunneling gate dielectric, a floating gate electrode, a high-k control gate dielectric and a control gate electrode. The high-k tunneling gate dielectric is formed form a first high-k dielectric layer employed in formation of a gate dielectric for a p-type field effect transistor (FET), the floating gate electrode is formed from a capping material layer employed in annealing the first high-k dielectric layer, and the high-k control gate dielectric is formed from a second high-k dielectric layer employed in formation of a gate dielectric for an n-type FET.