A principal-subordinate relationship between two subjects is decided with regard to multiple subjects included in an image. Similarly, with regard also to other images, a principal-subordinate relationship between two subjects is decided with regard to multiple subjects included in each image. In a case where a relationship exists in which a first subject is a subordinate subject and a second subject is principal subject, the system calculates a principal-subordinate relationship value that is higher than in a case where a relationship in which a first subject is a subordinate subject and a second subject is principal subject does not exist. The greater the number of subjects to which the first subject is subordinate from among the multiple subjects, the lower the principal-subordinate relationship value calculated. On the basis of the calculated principal-subordinate relationship values, a subject evaluation value is calculated for every subject.