A method includes forming a gate stack over a semiconductor substrate, forming an opening in the semiconductor substrate and adjacent to the gate stack, and performing a first epitaxy to grow a first semiconductor layer in the first opening. An etch-back is performed to reduce a thickness of the first semiconductor layer. A second epitaxy is performed to grow a second semiconductor layer over the first semiconductor layer. The first and the second semiconductor layers have different compositions.