A semiconductor device includes a semiconductor substrate; a first impurity diffusion suppression layer and a thicker second impurity diffusion suppression layer formed on the semiconductor substrate in first and second isolated transistor regions; first and second crystal layers formed on the first and second impurity diffusion suppression layers; first and second gate electrodes formed on the first and second crystal layers; first and second p-type channel regions formed in the semiconductor substrate, the first impurity diffusion suppression layer and respective of the first and second crystal layers below the first and second gate electrodes; and first and second source/drain regions formed on both sides of the first and second channel region; wherein the first and second p-type channels have lower impurity concentrations in respective of the first and second crystal layers than in the semiconductor substrate.