Images may be automatically annotated with place identifiers based on textual metadata associated with the images. The quality of these place identifiers may be scored based on a number of different factors. For example, these annotations may also include confidence and topicality values indicative of the relationships between the place identifier and the text as well as topics of the text of the textual metadata. Images that also have additional location information that includes GPS coordinates or a location specified by a user, may be used to evaluate the quality of a given place identifier, combination of confidence and topicality values, and the overall annotation system. These valuations may, in turn, be used to score the quality of such automatic annotations of other images that do not have such additional location information.