A semiconductor memory device includes a semiconductor substrate including a plane portion expanding in a first direction and a second direction perpendicular to the first direction, and a pillar portion formed on an upper surface of the plane portion and extending in a stacking direction, a first gate electrode formed on a first gate insulating layer on a lower side surface of the pillar portion, and extending in the first direction, a second gate electrode formed on a second gate insulating layer on an upper side surface of the pillar portion, and extending in the second direction, a variable-resistance element formed on an upper surface of the pillar portion, and an interconnection formed on an upper surface of the variable-resistance element.