An image processing device according to the present invention includes: a weight calculation unit that determines an area where a feature value of an input image is saved, based on a gradient of a feature value of a pixel of the input image and a direction of the gradient, and calculates a weight for reducing a regularization constraint that is a constraint based on regularization of image processing in the area where the feature value is saved; a regularization term calculation unit that calculates a regularization constraint of a high resolution image restored based on the input image by using the weight; a reconstruction constraint calculation unit that calculates a reconstruction constraint that is a constraint based on reconstruction of the high resolution image; and an image restoring unit that restores the high resolution image from the input image based on the regularization constraint and the reconstruction constraint.