A low coherent light from a white light source is emitted to a sample surface. A detour distance in a detour section is adjusted such that an optical path difference between a reference light and a sample light is equal to or shorter than a coherence length of interference light. The interference light is incident on an image sensor only when an inclination angle of a diffraction grating plate and a wavelength of the interference light satisfy a predetermined condition. Thus, an interference fringe image is formed. Based on each of the interference fringe images taken on a wavelength-by-wavelength basis of the interference light and an optical distance between a reference surface and the sample surface along an optical path of a measuring light at the time of taking the interference fringe image, a shape of the sample surface is measured.