In an ultrasonic inspection performed on an inspection object including a fine and multi-layer structure such as a semiconductor wafer and a MEMS wafer, a defect is detected by: separating a defect present inside from a normal pattern; obtaining an image of the inspection object by imaging the inspection object having a pattern formed thereon to enable a highly sensitive detection; generating a reference image that does not include a defect from the obtained image of the inspection object; generating a multi-value mask for masking a non-defective pixel from the obtained image of the inspection object; calculating a defect accuracy by matching the brightness of the image of the inspection object and the reference image; and comparing the calculated defect accuracy with the generated multi-value mask.