A semiconductor device and a method of forming the same are provided. In one embodiment, the semiconductor device includes a semiconductor substrate, a plurality of channel regions including first, second, and third p-type channel regions as well as first, second, and third n-type channel regions, and a plurality of gate structures. The plurality of gate structures includes an interfacial layer (IL) disposed over the plurality of channel regions, a first high-k (HK) dielectric layer disposed over the first p-type channel region and the first n-type channel region, a second high-k dielectric layer disposed over the first n-type channel region, the second n-type channel region, the first p-type channel region, and the second p-type channel region; and a third high-k dielectric layer disposed over the plurality of channel regions. The first, second and third high-k dielectric layers are different from one another.