An image processing apparatus for applying a color balance correction to input image data, comprises a holding unit which holds information indicating a locus of a change in highlight color when a color temperature for image data is changed on a color space; a highlight color calculation unit which calculates a highlight color from the image data; a distance calculation unit which calculates a distance between the highlight color and the highlight color locus held in the holding unit on the color space; a reliability calculation unit which calculates a reliability for a value of the highlight color calculated by the highlight color calculation unit in accordance with the distance calculated by the distance calculation unit; and a color balance correction unit which applies the color balance correction to the image data using the highlight color and the reliability.