A semiconductor device structure, along with methods of forming such, are described. In one embodiment, a semiconductor device structure is provided. The semiconductor device structure a first source/drain region, a second source/drain region, and a gate stack disposed between the first source/drain region and the second source/drain region. The semiconductor device structure also includes a conductive feature disposed below the first source/drain region. The semiconductor device structure also includes a power rail disposed below and in contact with the conductive feature. semiconductor device structure also includes a dielectric layer enclosing the conductive feature, wherein an air gap is formed between the dielectric layer and the conductive feature.