A semiconductor device includes a gate electrode structure of a transistor, the gate electrode structure being positioned above a semiconductor region and having a gate insulation layer that includes a high-k dielectric material, a metal-containing cap material positioned above the gate insulation layer, and a gate electrode material positioned above the metal-containing cap material. A bottom portion of the gate electrode structure has a first length and an upper portion of the gate electrode structure has a second length that is different than the first length, wherein the first length is approximately 50 nm or less. A strain-inducing semiconductor alloy is embedded in the semiconductor region laterally adjacent to the bottom portion of the gate electrode structure, and drain and source regions are at least partially positioned in the strain-inducing semiconductor alloy.