A gate dielectric film, a poly-silicon film, a film of a refractory metal such as tungsten, and a gate cap dielectric film are sequentially laminated on a semiconductor substrate. The gate cap dielectric film and the refractory metal film are selectively removed by etching. Thereafter, a double protection film including a silicon nitride film and a silicon oxide film is formed on side surfaces of the gate cap dielectric film, the refractory metal film, and the poly-silicon film. The poly-silicon film is etched using the double protection film as a mask. Thereafter, the semiconductor substrate is light oxidized to form a silicon oxide film on side surfaces of the poly-silicon film. Accordingly, a junction leakage of a MOSFET having a gate electrode of a poly-metal structure, particularly, a memory cell transistor of a DRAM, can be further reduced.