A method of forming a semiconductor device includes the following processes. A first pillar and a second pillar are formed on a semiconductor substrate. A semiconductor film is formed which includes first and second portions. The first portion is disposed over a side surface of the first pillar. The second portion is disposed over a side surface of the second pillar. The first and second portions are different from each other in at least one of impurity conductivity type and impurity concentration. A part of the semiconductor film is removed by etching back. The first and second portions are etched at first and second etching rates that are different from each other.