A method of forming a semiconductor device includes providing a semiconductor substrate. The semiconductor substrate includes a plurality of fins formed thereon, a stress layer formed on the top surface of each of the fins, and a plurality of strip-shaped gate structures formed above the stress layers. The method further includes forming a contact hole etch stop layer. The method further includes forming a first interlayer dielectric layer to fill the gaps between adjacent fins. The first interlayer dielectric layer includes filling voids formed therein. The method further includes back etching the first interlayer dielectric layer to cause the top surface of the first interlayer dielectric layer to be just above the filling voids. The method further includes forming a barrier liner layer over the first interlayer dielectric layer, and forming a second interlayer dielectric layer over the barrier liner layer and the contact hole etch stop layer.