A semiconductor device structure is provided. The semiconductor device structure includes a source/drain contact structure formed over a semiconductor substrate, and a first gate stack formed over the semiconductor substrate and adjacent to the source/drain contact structure. The semiconductor device structure also includes an insulating cap structure formed over and separated from an upper surface of the first gate stack. In addition, the semiconductor device structure includes first gate spacers formed over opposing sidewalls of the first gate stack to separate the first gate stack from the source/drain contact structure. The first gate spacers extend over opposing sidewalls of the insulating cap structure, so as to form an air gap surrounded by the first gate spacers, the first gate stack, and the insulating cap structure.