A semiconductor device having a first region and a second region is provided. The first region has a first protruding structure and a second protruding structure. The second region has a third protruding structure and a fourth protruding structure. First, second, third, and fourth epi-layers are formed on the first, second, third, and fourth protruding structures, respectively. The first and second epi-layers are covered with a first photoresist layer while leaving the third and fourth epi-layers exposed. A dielectric layer is formed over the first photoresist layer and over the third and fourth epi-layers. A portion of the dielectric layer is covered with a second photoresist layer. The portion of the dielectric layer is formed over the third and fourth epi-layers. Portions of the dielectric layer not protected by the first and second photoresist layers are etched. The first and second photoresist layers are removed.