A semiconductor memory device includes a semiconductor substrate having a projection, an upper end portion of the projection being curved, a first element isolation insulating film formed on the substrate surface at the root of the projection, having an upper surface lower than an upper surface of the projection, a second element isolation insulating film formed in the projection, a gate insulating film formed on the projection, and including a charge storage layer, and a gate electrode formed on the gate insulating film. A height of a first portion where the gate electrode is in contact with the gate insulating film above the upper surface of the first element isolation insulating film is smaller than that of a second portion where the gate electrode is in contact with the gate insulating film above an upper end of the second element isolation insulating film.