The present disclosure relates to a semiconductor device. The semiconductor device includes a semiconductor on an insulator (SOI) substrate having a bottom substrate, a buried oxide layer on the bottom substrate, and a semiconductor layer on the buried oxide layer. The semiconductor device also includes a first dielectric layer disposed on the semiconductor layer, a first contact structure extending from a top surface of the first dielectric layer through the semiconductor layer and the buried oxide layer and contacting the bottom substrate, and a first trench extending into the semiconductor layer. A width of the first trench is smaller than a width of the first contact structure. The first dielectric layer seals the first trench at or near the top of the first trench to form a vacuum gap.