To provide a classification dictionary learning system and the like that can generate a classification dictionary capable of detecting, with a high classification rate, a particular object of which change in the appearance is large in an image. The classification dictionary learning system includes: setting unit sets the priority degree of the classification between the first class and the second class including the sub-class to a value designated by the user and the like; the classifier learning unit sets the priority degree; the selection unit learns multiple classifiers on the basis of the priority degree for each combination of sub-classes; the weight updating unit updates the weight on the basis of the classification accuracy based on the classifier learned with the selected combination.