An integrated circuit structure includes a semiconductor substrate, insulation regions extending into the semiconductor substrate, with the insulation regions including first top surfaces and second top surfaces lower than the first top surfaces, a semiconductor fin over the first top surfaces of the insulation regions, a gate stack on a top surface and sidewalls of the semiconductor fin, and a source/drain region on a side of the gate stack. The source/drain region includes a first portion having opposite sidewalls that are substantially parallel to each other, with the first portion being lower than the first top surfaces and higher than the second top surfaces of the insulation regions, and a second portion over the first portion, with the second portion being wider than the first portion.