Some embodiments provide a computer program for matching colors of a first image to colors of a second image. The computer program identifies a set of transforms for converting the first image to a device-independent color space. The computer program modifies the set of transforms to segment colors of the first image into a set of segmented colors. The computer program applies the modified set of transforms to pixel values of the first image in order to segment the pixels of the first image into the set of segmented colors. The computer program applies the modified set of transforms to pixel values of the second image in order to segment the pixels of the second image into the set of segmented colors. The computer program matches each segmented color in the set of segmented colors in the first image to the segmented color in the second image.