Methods and systems for performing active learning for defect classifiers are provided. One system includes one or more computer subsystems configured for performing active learning for training a defect classifier. The active learning includes applying an acquisition function to data points for the specimen. The acquisition function selects one or more of the data points based on uncertainty estimations associated with the data points. The active learning also includes acquiring labels for the selected one or more data points and generating a set of labeled data that includes the selected one or more data points and the acquired labels. The computer subsystem(s) are also configured for training the defect classifier using the set of labeled data. The defect classifier is configured for classifying defects detected on the specimen using the images generated by the imaging subsystem.