According to one embodiment, a semiconductor memory device includes: a first semiconductor layer; first and second insulating layers in contact with the first semiconductor layer; a second semiconductor layer in contact with the first insulating layer; a third semiconductor layer in contact with the second insulating layer; a first conductor; a third insulating layer in contact with the first conductor; a fourth insulating layer provided between the second semiconductor layer and the third insulating layer; a first charge storage layer provided between the second semiconductor layer and the fourth insulating layer; and a fifth insulating layer provided between the second semiconductor layer and the first charge storage layer. The second semiconductor layer, the first conductor, the third to fifth insulating layers, and the first charge storage layer function as a first memory cell.