A high-k capacitor insulating film stable at a higher temperature is formed. There is provided a method of manufacturing a semiconductor device. The method comprises: forming a first amorphous insulating film comprising a first element on a substrate; adding a second element different from the first element to the first amorphous insulating film so as to form a second amorphous insulating film on the substrate; and annealing the second amorphous insulating film at a predetermined annealing temperature so as to form a third insulating film by changing a phase of the second amorphous insulating film. The concentration of the second element added to the first amorphous insulating film is controlled according to the annealing temperature.