An upper layer (4,5) made of non-doped III-V compound semiconductor is formed on a lower layer (3) made of non-doped III-V compound semiconductor. Impurity source gas is fed through vapor phase diffusion using an organometallic vapor-phase epitaxy device to add an impurity to the upper layer (4,5). The vapor phase diffusion is continued with the feed of the impurity source gas stopped or with a feed amount of the impurity source gas lowered.