A defect detecting device includes an illumination that irradiates a measuring object with illumination light, an imager that images the illumination light reflected by the measuring object, and a detector that detects a defect at a surface of the measuring object based on a captured image obtained by imaging the illumination light by the imager. The captured image includes a plurality of spectral images having different spectral wavelengths, and the detector detects a diffuse reflection region by which the illumination light is diffusely reflected based on the plurality of spectral images, and determines a size of the defect based on the spectral wavelength of the spectral image in which the diffuse reflection region is detected.