A semiconductor device includes a substrate, fin-shaped structures, a protection layer, epitaxial layers, and a gate electrode. The fin-shaped structures are disposed in a first region and a second region of the substrate. The protection layer conformally covers the surface of the substrate and the sidewalls of fin-shaped structures. The epitaxial layers respectively conformally and directly cover the fin-shaped structures in the first region. The gate electrode covers the fin-shaped structures in the second region, and the protection layer is disposed between the gate electrode and the fin-shaped structures.