A semiconductor memory device includes a first floating gate and a second floating gate of conductivity types with different polarities. Injection of electrons into the first floating gate via a tunnel insulating film is stored through a decrease in holes in a valence band of the second floating gate, and ejection of electrons from the first floating gate via the tunnel insulating film is stored through an increase in holes in the valence band of the second floating gate.