A system and method for ranking images are provided. The method includes receiving a query comprising a semantic part and an abstract part, retrieving a set of images responsive to the semantic part of the query, and computing first scores for the retrieved images in the set of retrieved images. The first score of an image can be based on a relevance of that image to the semantic part of the query (and not to the abstract part of the query). The method further includes identifying a chromatic concept model from a set of chromatic concept models. This identification can be based on the abstract part of the query (and not on the semantic part of the query). The chromatic concept model includes an optionally-weighted set of colors expressed in a perceptually uniform color space. For retrieved images in the set of retrieved images, the method includes computing a chromatic image model based on colors of the image, the chromatic image model comprising a weighted set of colors expressed in the perceptually uniform color space and computing a comparison measure between the chromatic image model and the chromatic concept model. The retrieved images are scored with respective second scores that are based on the computed comparison measures. The retrieved images are ranked based on a combined score for a respective retrieved image which is a function of the first and second scores.