A p+ collector layer is provided in a rear surface of a semiconductor substrate which will be an n− drift layer and an n+ field stop layer is provided in a region which is deeper than the p+ collector layer formed on the rear surface side. A front surface element structure is formed on the front surface of the semiconductor substrate and then protons are radiated to the rear surface of the semiconductor substrate at an acceleration voltage corresponding to the depth at which the n+ field stop layer is formed. A first annealing process is performed at an annealing temperature corresponding to the proton irradiation to change the protons into donors, thereby forming a field stop layer. Then, annealing is performed using annealing conditions suitable for the conditions of a plurality of proton irradiation processes to recover each crystal defect formed by each proton irradiation process.