A semiconductor device includes a N-type field effect transistor comprising a N-channel region in a substrate. A high dielectric constant (high-k) layer is disposed on the N-channel region. A diffusion layer including a metal oxide is disposed on the high-k layer. A passivation layer is disposed on the diffusion layer, and a first metal gate is disposed on the passivation layer. The first high-k layer and the N-channel region include metal atoms of a metal element of the metal oxide.