A defect classification apparatus of the present invention includes a design unit and a diagnosis unit. In the design unit, a model creation unit combines a defect template in a template storage unit with a teaching image to create a defect model, and a classification class setting unit calculates feature amounts of a defect in the defect model and sets a classification class of the defect. The relation between the feature amounts of the defect and the classification class is stored in a storage unit. In the diagnosis unit, a feature amount calculation unit calculates feature amounts of defects from a captured inspection object image of the substrate, and a classification unit classifies the defects of the substrate into classification classes from the relations between the feature amounts of the defects and the classification classes in the storage unit based on the calculated feature amounts of the defects.