There is provided an image processing device that specifies a region including a specific subject on each input image of a plurality of continuous frames. The image processing device includes: subject map generation means that, from feature maps corresponding to features of respective pixels of the input image and representing feature amounts in respective regions of the input image, selects one feature amount of any of the feature maps for each pixel so as to thereby generate a subject map representing similarities of the respective regions of the input image to the subject; and subject region specification means that, on the basis of the subject map, specifies a subject region, which is a region most similar to the subject, in the subject map so as to thereby specify a region which includes the subject on the input image.