Methods and systems for tag recognition in captured images. A candidate region can be localized from regions of interest with respect to a tag and a tag number shown in the regions of interest within a side image of a vehicle. A number of confidence levels can then be calculated with respect to each digit recognized as a result of an optical character recognition operation performed with respect to the tag number. Optimal candidates within the candidate region can be determined for the tag number based on individual character confidence levels among the confidence levels. Optimal candidates from a pool of valid tag numbers can then be validated using prior appearance probabilities and data returned, which is indicative of the most probable tag to be detected to improve image recognition accuracy.