A nonvolatile semiconductor memory device includes a first insulating layer on a semiconductor layer, a charge storage layer on the first insulating layer, a second insulating layer on the charge storage layer, and a control gate electrode on the second insulating layer. The charge storage layer includes a floating gate layer on the first insulating layer, an interface insulating layer on the floating gate layer, and a charge trap layer on the interface insulating layer, and a lower end of a conduction band of the interface insulating layer is higher than a trap level of the charge trap layer and is lower than a lower end of a conduction band of the charge trap layer.