A semiconductor device structure is provided. The semiconductor device structure includes a semiconductor substrate. The semiconductor device structure includes an isolation structure positioned in the semiconductor substrate and adjacent to a first active region of the semiconductor substrate. The semiconductor device structure includes a gate stack disposed over the first active region. The semiconductor device structure includes a first contact structure disposed over the first active region and positioned between the isolation structure and the gate stack. The semiconductor device structure includes a dummy gate stack disposed over the isolation structure and adjacent to the gate stack. The dummy gate stack is not positioned over a portion of the isolation structure next to the first contact structure.