This disclosure describes systems and methods for identifying and ranking relevant and diverse image search results in response to a query. An optimum set of features is identified for every query. The optimum set of features can be selected from a predefined set of features and can be selected based on a variance across features derived from an initial set of objects returned in response to the query. The optimum set of features can then be used to re-rank the initial set of objects or to search for a second set of objects and rank the second set of objects.