A semiconductor memory device of an embodiment includes: a semiconductor layer; a gate electrode layer; a first insulating layer provided between the semiconductor layer and the gate electrode layer; a second insulating layer provided between the first insulating layer and the gate electrode layer; and an intermediate layer provided between the first insulating layer and the second insulating layer, the intermediate layer containing a first crystal of a space group Pbca (space group number 61), a space group P42/nmc (space group number 137), or a space group R-3m (space group number 166), and the intermediate layer containing hafnium (Hf), oxygen (O), and nitrogen (N).