An image processing apparatus which corrects color balance of an image in image data captured by an image capturing apparatus having a color balance correction function, the apparatus comprises a light source estimation unit configured to estimate a capturing light source of the image based on a color value of an object; a first color conversion unit configured to perform first color conversion of converting, based on the capturing light source and the color value, the image into an image in which color balance correction applied depending on the image capturing apparatus that has captured the image data containing the image is weakened; and a second color conversion unit configured to perform second color conversion for the image having undergone the first color conversion, by using an adaptation rate corresponding to a color value defined based on chromatic adaptation of a man under the capturing light source.