An input image (e.g. a digital RGB color image) is subjected to an eye classifier that is targeted at discriminating a complete eye pattern from any non-eye patterns. The red-eye candidate list with associated bounding boxes that are generated by the red-eye classifier are received. The bounding rectangles are subjected to object segmentation. A connected component labeling procedure is then applied to obtain one or more red regions. The largest red region is then chosen for feature extraction. A number of features are then extracted from this region. Then these features are used to determine if the particular candidate red-eye object is a mouth.