A method includes etching a first semiconductor fin and a second semiconductor fin to form first recesses. The first and the second semiconductor fins have a first distance. A third semiconductor fin and a fourth semiconductor fin are etched to form second recesses. The third and the fourth semiconductor fins have a second distance equal to or smaller than the first distance. An epitaxy is performed to simultaneously grow first epitaxy semiconductor regions from the first recesses and second epitaxy semiconductor regions from the second recesses. The first epitaxy semiconductor regions are merged with each other, and the second epitaxy semiconductor regions are separated from each other.