A method includes recessing isolation regions, wherein a portion of a semiconductor strip between the isolation regions is over top surfaces of the recessed isolation regions, and forms a semiconductor fin. A dummy gate is formed to cover a middle portion of the semiconductor fin. An Inter-Layer Dielectric (ILD) is formed to cover end portions of the semiconductor fin. The dummy gate is then removed to form a first recess, wherein the middle portion is exposed to the first recess. The middle portion of the semiconductor fin is removed to form a second recess. An epitaxy is performed to grow a semiconductor material in the second recess, wherein the semiconductor material is between the end portions. A gate dielectric and a gate electrode are formed in the first recess. The gate dielectric and the gate electrode are over the semiconductor material.