It is an object to provide a nonvolatile semiconductor memory device having excellent writing property and charge-retention property. A semiconductor layer including a channel forming region between a pair of impurity regions which are formed to be apart from each other is provided. In an upper layer portion thereof, a first insulating layer, a floating gate, a second insulating layer, and a control gate are provided. The floating gate has at least a two-layer structure, and a first layer in contact with the first insulating layer preferably has a band gap smaller than that of the semiconductor layer. Furthermore, by setting an energy level at the bottom of the conduction band of the floating gate lower than that of the channel forming region of the semiconductor layer, injectability of carriers and a charge-retention property can be improved.