A semiconductor device structure is provided. The semiconductor device structure includes a substrate and a dielectric fin structure over the substrate. The semiconductor device structure also includes a semiconductor fin structure adjacent to the dielectric fin structure. The semiconductor device structure also includes a metal gate stack across the dielectric fin structure and the semiconductor fin structure. The semiconductor device structure also includes a source/drain feature over the semiconductor fin structure. The semiconductor device structure also includes a source/drain spacer interposed between the source/drain feature and the dielectric fin structure.