A nonvolatile semiconductor memory device includes a charge storage layer on a first insulating film, a second insulating film which is provided on the charge storage layer, formed of layers, and a control gate electrode on the second insulating film. The second insulating film includes a bottom layer (A) provided just above the charge storage layer, a top layer (C) provided just below the control gate electrode, and a middle layer (B) provided between the bottom layer (A) and the top layer (C). The middle layer (B) has higher barrier height and lower dielectric constant than both the bottom layer (A) and the top layer (C). The average coordination number of the middle layer (B) is smaller than both the average coordination number of the top layer (C) and the average coordination number of the bottom layer (A).