Methods and systems for exploiting color for object recognition. A color gradient for each pixel in a gradient image of an object can be calculated. The gradient image can be binarized to produce an image having white walls around characters and other structures. A connected component analysis operation can be performed with respect to black regions in the image to determine bounding boxes for characters and other pictorial elements in the image and thereafter identify character candidates from the image utilizing character metrics. Non-character colors can then be eliminated from the image utilizing an outlier rejection.