A distinguishing size for distinguishing a pseudo defect from a defect caused by a process trouble is stored in a first storage area. Defect data are stored in a second storage area. A processing unit detects a defect on a wafer surface, and stores the defect data in the second storage area. Before a defect detection process is completed for all areas of the wafer surface, a size of a defect detected in a partial area is compared with the distinguishing size stored in the first storage area. If the detected defect has a size equal to or larger than the distinguishing size, an alarm is output through an output unit, whereas if a defect having a size equal to or larger than the distinguishing size is not detected, the defect detection process is executed for the area still not subjected to the defect detection process.