A gate TiN layer of adjacent Si pillars among Si pillars contacts at entire channel length in a vertical direction. SiO2 layers are formed, surrounding the Si pillars, and mask material layers on top thereof, and being spaced from each other. Then, a SiN layer is formed surrounding the SiO2 layers. Then, the mask material layers and the SiO2 layers are removed. Then, a P+ layer and N+ layers which upper surfaces are lower than an upper surface position of the SiN layer are formed surrounding each top of the Si pillars by selective epitaxial crystal growth method.