Embodiments of mechanisms for forming a semiconductor device structure with floating spacers are provided. The semiconductor device structure includes a silicon-on-insulator (SOI) substrate and a gate stack formed on the SOI substrate. The semiconductor device structure also includes gate spacers formed on sidewalls of the gate stack. The gate spacers include a floating spacer. The semiconductor device structure further includes a contact etch stop layer formed on the gate stack and the gate spacers. The contact etch stop layer is formed between the floating spacer and the SOI substrate.