A semiconductor device with different gate structure configurations and a method of fabricating the semiconductor device are disclosed. The method includes depositing a high-K dielectric layer surrounding nanostructured channel regions, performing a first doping with a rare-earth metal (REM)-based dopant on first and second portions of the high-K dielectric layer, and performing a second doping with the REM-based dopants on the first portions of the high-K dielectric layer and third portions of the high-K dielectric layer. The first doping dopes the first and second portions of the high-K dielectric layer with a first REM-based dopant concentration. The second doping dopes the first and third portions of the high-K dielectric layer with a second REM-based dopant concentration different from the first REM-based dopant concentration. The method further includes depositing a work function metal layer on the high-K dielectric layer and depositing a metal fill layer on the work function metal layer.