A semiconductor structure and a method of forming the same are provided. In an embodiment, an exemplary method includes forming a fin-shaped structure extending from a front side of a substrate, recessing a source region of the fin-shaped structure to form a source opening, forming a semiconductor plug under the source opening, exposing the semiconductor plug from a back side of the substrate, selectively removing a first portion of the substrate without removing a second portion of the substrate adjacent to the semiconductor plug, forming a backside dielectric layer over a bottom surface of the workpiece, replacing the semiconductor plug with a backside contact, and selectively removing the second portion of the substrate to form a gap between the backside dielectric layer and the backside contact. By forming the gap, a parasitic capacitance between the backside contact and an adjacent gate structure may be advantageously reduced.