When a size of a block on a wafer is equal to or smaller than an optical resolution of imaging optics, room for improvement in a signal-to-noise ratio has not been sufficiently considered in a conventional technique. One feature of the defect determination of the present invention is to include a filter processing for setting a predetermined partial area serving as a predetermined matrix for a first difference image, scanning the first difference image in the partial area, and outputting a second difference image, and a first threshold processing using a first threshold value for the second difference image. As a result, highly sensitive defect detection can be achieved.