A semiconductor device structure is provided. The semiconductor device structure includes an isolation structure formed over a substrate, and a first stacked structure and a second stacked structure extending above the isolation structure. The first stacked structure includes a plurality of first nanostructures stacked in a vertical direction, and the second stacked structure includes a plurality of second nanostructures stacked in the vertical direction. The semiconductor device structure includes a first dummy fin structure formed over the isolation structure, and the first dummy fin structure is between the first stacked structure and the second stacked structure. The semiconductor device structure also includes a first capping layer formed over the first dummy fin structure, and an interface between the first dummy fin structure and the first capping layer is lower than a top surface of a topmost first nanostructure.