An image processing apparatus capable of suppressing color fringing in a color image further effectively by image processing. A determination unit determines a region in which signal levels for a color plane in a color image produced by photoelectric conversion of an optical image of a subject exhibit a monotonic increase or a monotonic decrease, as a color fringing region in which color fringing occurs. An estimation unit estimates an intensity of the color fringing in the color fringing region determined by the determination unit. A removal unit deducts an estimate value of the intensity of the color fringing estimated by the estimation unit from the intensity of the color fringing in the color fringing region.