There is provided an image processing apparatus including circuitry configured to predict an image of a second layer from an image of a first layer using a lookup table which maps combinations of a luminance component, a first color difference component, and a second color difference component of the first layer to predicted pixel values of the second layer corresponding to the combinations. The second layer has a different color gamut from the first layer. The circuitry uses the lookup table in which granularities of the first and second color difference components are lower than a granularity of the luminance component.