An image processing apparatus which corrects, based on information of a foreign substance adhered to an optical element arranged in front of an image sensor in an image capturing apparatus, a shadow of the foreign substance in the moving image data generated by performing predictive encoding by referring to image data of a different input frame, the image processing apparatus comprises: a moving image decompression unit which decompresses an image of each frame by obtaining information representing a difference between frames from the moving image data; and a correction unit which corrects the information representing the difference between frames when the information representing the difference between frames that is obtained by the moving image decompression unit in correspondence with a region containing the shadow of the foreign substance represented by the foreign substance information is determined to be larger than a predetermined threshold.