Provided is an image processing apparatus including an extraction unit that extracts representative colors from a target image, a color difference calculation unit that calculates a color difference between the representative color and a pixel of interest of the target image, a change unit that changes a brightness component of the color difference depending on chroma of the representative colors, and a replacement unit that selects the representative color depending on the color difference changed in the change unit to replace a color of the pixel of interest by the representative color.