A method includes forming an epitaxy semiconductor layer over a semiconductor substrate, and etching the epitaxy semiconductor layer and the semiconductor substrate to form a semiconductor strip, which includes an upper portion acting as a mandrel, and a lower portion under the mandrel. The upper portion is a remaining portion of the epitaxy semiconductor layer, and the lower portion is a remaining portion of the semiconductor substrate. The method further includes growing a first semiconductor fin starting from a first sidewall of the mandrel, growing a second semiconductor fin starting from a second sidewall of the mandrel. The first sidewall and the second sidewall are opposite sidewalls of the mandrel. A first transistor is formed based on the first semiconductor fin. A second transistor is formed based on the second semiconductor fin.