An image processing apparatus for reducing fluctuations due to a heat haze in an image including a stationary region and a moving object obtains histogram of gradation distribution for each of a target image to be corrected and a frame for correction, and corrects the target image by using the frame for correction while changing a mixing ratio between the target image and the frame for correction depending on similarity between histograms. To adaptively control a size of image blocks for the histograms, the apparatus calculates two indices having different robustness to deformation of an object from image blocks of each of the target image and the frame for correction, and determines suitability of the image block size based on a relationship of the indices. The difference between the histograms and the sum of absolute differences (SAD) between the pixel values can be used as the two indices.