A defect is efficiently and effectively classified by accurately determining the state of overlap between a design layout pattern and the defect. This leads to simple identification of a systematic defect. A defective image obtained through defect inspection or review of a semiconductor device is automatically pattern-matched with design layout data. A defect is superimposed on a design layout pattern for at least one layer of a target layer, a layer immediately above the target layer, and a layer immediately below the target layer. The state of overlap of the defect is determined as within the pattern, over the pattern, or outside the pattern, and the defect is automatically classified.