An inspection device includes an illumination optical system that irradiates a sample with light having a predetermined wavelength, a detection optical system that includes a photoelectric conversion unit, and a data processing unit that extracts positional information of a foreign substance or a defect on the sample. The light collection optical system includes a polarized light transmission control unit that changes transmission characteristics according to polarization characteristics of the collected reflected light or scattered light. The polarized light transmission control unit includes a birefringence phase difference control unit that causes a predetermined phase difference between a fast phase axis and a slow phase axis of the reflected light or scattered light according to the polarization characteristics of the reflected light or scattered light, and a polarized light transmission unit that selectively transmits light according to a polarization direction of output light of the birefringence phase difference control unit.