A stacked body is formed on a silicon substrate by stacking a plurality of insulating films and a plurality of electrode films alternately and through-holes are formed to extend in the stacking direction. Next, gaps are formed between the electrode films using etching the insulating films via the through-holes. Charge storage layers are formed along side faces of the through-holes and inner faces of the gaps, and silicon pillars are filled into the through-holes. Thereby, a nonvolatile semiconductor memory device is manufactured.