A gate electrode of a field effect transistor is formed. Next, an offset spacer film with a double-layer structure including a silicon oxide film as a lower-layer film and a silicon nitride film as an upper-layer film is formed on a sidewall surface of the gate electrode. The silicon nitride film serves as a supply source of an element for terminating dangling bonds of silicon in a device formation region. Next, treatment for leaving the offset spacer film intact or treatment for removing the silicon nitride film of the offset spacer film is performed. Thereafter, a sidewall insulating film is formed on the sidewall surface of the gate electrode.