A method for forming a semiconductor memory device is disclosed. A substrate is provided. A source diffusion region is formed in the substrate. Two floating gates are on opposite sides of the source diffusion region. A first dielectric cap layer is formed directly on each of the floating gates. An erase gate is formed on the source diffusion region. The erase gate partially overlaps an upper inner corner of each of the floating gates. A second dielectric cap layer is formed on the erase gate and the first dielectric cap layer. A select gate is formed on a sidewall of the first dielectric cap layer in a self-aligned manner. A drain diffusion region is formed in the substrate and adjacent to the select gate.