A method includes etching a semiconductor substrate to form a semiconductor strip and trenches on opposite sidewalls of the semiconductor strip. A spacer is formed on a sidewall of the semiconductor strip which is used as an etching mask to extend the trenches down into the semiconductor substrate. A dielectric material is filled into the trenches and then planarized to form insulation regions in the trenches. The insulation regions are recessed. After the recessing, top surfaces of the insulation regions are lower than a top surface of the semiconductor strip and a gate structure may be formed thereon.