An apparatus for reviewing defects including an image processing section (defect classification device section) with a function of estimating a non-defective state (reference image) of a portion in which the defect exists by use of a defect image, and a function of judging criticality or non-flat state of the defect by use of the estimation result. It becomes possible to establish both of a high-throughput image collecting sequence in which any reference image is not acquired and high-precision defect classification, and then to realize both of a high performance classifying function and a high-throughput image collecting function in a defect reviewing apparatus which automatically collects and classifies images of defects existing on a sample of a semiconductor wafer or the like.