A device includes a semiconductor substrate, isolation regions in the semiconductor substrate, and a Fin Field-Effect Transistor (FinFET). The FinFET includes a channel region over the semiconductor substrate, a gate dielectric on a top surface and sidewalls of the channel region, a gate electrode over the gate dielectric, a source/drain region, and an additional semiconductor region between the source/drain region and the channel region. The channel region and the additional semiconductor region are formed of different semiconductor materials, and are at substantially level with each other.