A phase modulation structure includes a recording surface including phase angle recording regions in a plurality of calculated element regions corresponding to reconstruction points of an image on a one-to-one basis, each phase angle recording region being formed of a plurality of unit blocks in each of which a phase angle is recorded, the phase angle being calculated based on a phase that is a sum of a plurality of phases of light from the corresponding reconstruction points; and a representative area that is one of divisions of the calculated element region, the representative area being obtained by radially dividing the calculated element region centered on a point on the calculated element region, the point being obtained by extending a normal line from the corresponding reconstruction point to the calculated element region on the recording surface.