A method of manufacturing a dual work function semiconductor device is disclosed. In one aspect, the method includes providing a substrate having first and second areas for forming first and second transistor types. The method additionally includes forming a dielectric layer on the substrate, which extends to cover at least parts of the first and second areas. The method additionally includes forming a first metal layer/stack on the dielectric layer in the first area, where the first metal layer/stack comprises a first work function-shifting element. The method additionally includes forming a second metal layer/stack on the first metal layer in the first area and on the dielectric layer in the second area, where the second metal layer/stack comprises a second work function-shifting element. The method additionally includes annealing to diffuse the first work function-shifting element and the second work function-shifting element into the dielectric layer, and subsequently removing the first metal layer/stack and the second metal layer/stack. The method further includes forming a third metal layer/stack in the first and second predetermined areas.