A method is described to increase the efficiency of the removal of defects from document images by reorienting the conceptual framework within which an image is filtered. Rather than arbitrarily applying a filter to an entire landscape of a document image, the disclosure describes a methodology by which a document image is separated into regions of darkness and regions of light, or viewed alternatively, regions of darkness and regions of lack of darkness. Filtering is then adaptively applied to each region to remove defects.