A method includes forming a first dielectric layer over a source/drain region, and forming a source/drain contact plug over and electrically connecting to the source/drain region. A top portion of the source/drain contact plug has a first lateral dimension. An implantation process is performed to implant a dopant into the first dielectric layer. The implantation process results in the source/drain contact plug to have a second lateral dimension smaller than the first lateral dimension. The method further includes forming a second dielectric layer over the etch stop layer, and forming a gate contact plug adjacent to the source/drain contact plug.