A semiconductor device structure is provided. The semiconductor device structure includes a first stacked nanostructure and a second stacked nanostructure formed over a substrate. The semiconductor device structure includes a first gate structure formed over the first stacked nanostructure, and the first gate structure includes a first portion of a gate dielectric layer and a first portion of a filling layer. The semiconductor device structure includes a second gate structure formed over the second stacked nanostructure, and the second gate structure includes a second portion of the gate dielectric layer and a second portion of the filling layer. The semiconductor device structure includes a first isolation layer between the first gate structure and the second gate structure, and a sidewall of the first portion of the gate dielectric layer extends beyond a sidewall of the filling layer.