The identification of known normal structures within an image is preferably accomplished using an appearance model. Specifically, an active appearance model, which encapsulates a complete model of the shape and global texture variations of an object from a collection of samples, is utilized to define normal structures within an image by restricting training samples supplied to the active appearance model during a training phase to those that do not contain abnormal structures. Accordingly, the trained appearance model represents only normal variations in the object of interest. When another image with abnormalities is presented to the system, the appearance model cannot synthesize the abnormal structures which show up as errors in a residual image. Accordingly, the errors in the residual image represent potential abnormalities.