An image processing apparatus has an enlargement processing function of displaying an image on an image display unit at a designated enlargement magnification, and a color conversion processing function of changing a designated conversion source color to a conversion destination color. This image processing apparatus determines at least one of the conversion source color and conversion destination color by acquiring color information from a predetermined area in the displayed image. Upon determining one of the conversion source color and conversion destination color, control according to the enlargement magnification of the enlargement processing function is made so as to assure an amount, which is equal to or larger than a predetermined value, of color information included in the predetermined area.