A method of forming an integrated circuit device includes forming a gate stack covering a middle portion of a semiconductor fin, forming a gate spacer layer over the gate stack and the semiconductor fin, and patterning the gate spacer layer. The resulting spacers include a gate spacer on a sidewall of the gate stack, and a fin spacer on a sidewall of an end portion of the semiconductor fin. The fin spacer is then etched. When the etching is finished, a height of the fin spacer is smaller than about a half of the height of the semiconductor fin.