A semiconductor device includes a substrate, an insulating layer provided over the substrate, a collection of metal particles exposed on the surface of the insulating layer, and a diamond layer provided on the surface of the insulating layer on which the metal particles are exposed. By controlling the surface density and particle size of the metal particles on the surface of the insulating layer, the surface density of diamond nuclei that are formed on the surface is controlled. Diamond grains are formed by crystal growth using the diamond nuclei as starting material, thereby forming a diamond layer. The control of the surface density of the diamond nuclei results in forming, by the crystal growth, the diamond grains with a grain size exhibiting a relatively high thermal conductivity in the crystal growth initial layer of the diamond layer and improving the thermal conductivity between the diamond layer and the substrate.