A method includes forming a gate stack on a first portion of a semiconductor fin, removing a second portion of the semiconductor fin to form a recess, and forming a source/drain region starting from the recess. The formation of the source/drain region includes performing a first epitaxy process to grow a first semiconductor layer, wherein the first semiconductor layer has straight-and-vertical edges, and performing a second epitaxy process to grow a second semiconductor layer on the first semiconductor layer. The first semiconductor layer and the second semiconductor layer are of a same conductivity type.