In general, this disclosure describes techniques of reducing the possibility of an image evaluation device incorrectly identifying defects in images of documents. Some defects may be apparent in the images of both sides of a document. For instance, a tear in a document could be apparent in an image of a front side and an image of a rear side of the document. However, the image evaluation device could erroneously identify such a tear in an image of one side of the document. In this case, the tear would not be apparent in the image of the other side of the document. To reduce the possibility that the image evaluation device erroneously identifies such a defect, the image evaluation device may determine whether defects identified in the image of the front side of the document correspond to defects identified in the image of the rear side of the document.