Apparatus includes a change operator for detecting change factor information which causes the deterioration in the image; an iterative processing operator which iterates a processing cycle comprising; a first processing step to apply a first calculation to a first image utilizing the change factor information to generate a second image, calculating difference data between the deteriorated image and the second image; a second processing step to apply a second calculation to the second image utilizing the difference data to generate a third image; and a replacing step to replace the first image with the third image; wherein the processing cycle further includes (a) detecting whether a coincidence exists between the sign of a first difference data and the sign of a second difference data or not, and (b) increasing an absolute value of the difference data utilized in the second processing step when the coincidence is detected in step (a).