An image processing device according to one of the exemplary aspects of the present invention includes: inferring means for selecting, for each of local unknown patches including a target unknown patch, candidate patches from a plurality of input patches based on similarity to the local unknown patch, the local unknown patches being images generated from a part of an unknown image, the plurality of input patches being images generated from a plurality of input images, a subject ID (Identifier) being correlated with the input patches that are generated from an input image to which the subject ID is assigned in the plurality of input image; first score calculation means for calculating a score representing nearness of a candidate patch in the candidate patches to a local unknown patch in the local unknown patches; and patch replacement means for calculating a score summation for the subject ID by summing up scores of the candidate patches being correlated with a same subject ID in the candidate patches of the local unknown patches, and selecting, as a selected patch being used for reconstruction of a reconstruction image, a candidate patch that is correlated with the subject ID for which the score summation is highest from the candidate patches selected for the target unknown patch.