A charge storage layer interposed between a memory gate electrode and a semiconductor substrate is formed shorter than a gate length of the memory gate electrode or a length of insulating films so as to make the overlapping amount of the charge storage layer and a source region to be less than 40 nm. Therefore, in the write state, since the movement in the transverse direction of the electrons and the holes locally existing in the charge storage layer decreases, the variation of the threshold voltage when holding a high temperature can be reduced. In addition, the effective channel length is made to be 30 nm or less so as to reduce an apparent amount of holes so that coupling of the electrons with the holes in the charge storage layer decreases; therefore, the variation of the threshold voltage when holding at room temperature can be reduced.