A first waveguide member is formed, as viewed from above, in an image pickup region and a peripheral region of a semiconductor substrate. A part of the first waveguide member, which part is disposed in the peripheral region, is removed. A flattening step is then performed to flatten a surface of the first waveguide member on the side opposite to the semiconductor substrate.