A method for inspecting a sample, consisting of receiving a definition of image attributes that are characteristic of defects, and processing an image of the sample so as to identify candidate defects on the sample. The method further includes forming distributions of values of the respective attributes from the candidate defects, and selecting a set of the candidate defects that are characterized by respective candidate attribute values that fall in one or more tails of the distributions. The selected set is presented to a human operator, and respective classifications of the candidate defects in the selected set are received from the operator. A definition of the one or more tails of the distributions is refined responsively to the classifications. The method may be used as a filter to remove false alarms, or nuisances. The method may also be used to categorize the candidate defects into two or more classes.