A semiconductor memory device includes a static memory cell having six MOS transistors arranged on a substrate. The six MOS transistors include first and second NMOS access transistors, third and fourth NMOS driver transistors, and first and second PMOS load transistors. Each of the first and second NMOS access transistors has a first diffusion layer, a pillar-shaped semiconductor layer, and a second diffusion layer arranged vertically on the substrate in a hierarchical manner. Each of the third and fourth NMOS driver transistors has a third diffusion layer, a pillar-shaped semiconductor layer, and a fourth diffusion layer arranged vertically on the substrate in a hierarchical manner. The lengths between the upper ends of the third diffusion layers and the lower ends of the fourth diffusion layers are shorter than the lengths between the upper ends of the first diffusion layer and the lower ends of the second diffusion layers.