Defect printability analysis in a mask or wafer requires the accurate identification of defect images and reference (i.e. defect-free) images, in particular for a die-to-die inspection mode. A method of automatically distinguishing a reference image from a defect image is provided. In this method, multiple images can be accessed and aligned. Then, a common area of the multiple images can be defined. At this point, a complexity of each of the images, as defined by the common area, can be computed. Advantageously, by comparing the complexity of the multiple images, the reference and defect images can be quickly and accurately designated. Specifically, the most complex image is designated the defect image because the defect image must describe the defect. Complexity can be computed using various techniques.