A composition determination device includes: a subject detection unit configured to detect a subject in an image based on acquired image data; an actual subject size detection unit configured to detect the actual size which can be viewed as being equivalent to actual measurements, for each subject detected by the subject detection unit; a subject distinguishing unit configured to distinguish relevant subjects from subjects detected by the subject detection unit, based on determination regarding whether or not the actual size detected by the actual subject size detection unit is an appropriate value corresponding to a relevant subject; and a composition determination unit configured to determine a composition with only relevant subjects, distinguished by the subject distinguishing unit, as objects.