Computer-implemented methods, carrier media, and systems for creating a defect sample for use in selecting one or more parameters of an inspection recipe are provided. One method includes separating defects into bins based on regions in which the defects are located, defect types, and values of the defects for parameter(s) of a detection algorithm. The method also includes determining a number of the defects to be selected from each bin by distributing a user-specified target number of defects across the bins. In addition, the method includes selecting defects from the bins based on the determined numbers thereby creating a defect sample for use in selecting values of parameter(s) of the detection algorithm for use in the inspection recipe.