In the inspection of a defect in a fine concave-convex pattern, a spectral waveform of a detection area of an inspection object is detected, area determination as to which area section determined by a pattern type of the inspection object the detection area belongs to is performed, a feature calculation equation and a determination index value which correspond to a determined area section and vary according to defect type is selected, feature calculation on the spectral waveform data in accordance with the selected feature calculation equation is performed, and a calculated feature value and the selected determination index value are compared to perform determination processing according to defect type.