A hue segmentation system and method thereof suitable for image devices are described. First, a plurality of hue segments are generated based on color data of a plurality of colors in a color gamut. Then, the area difference between the hue segments and the color gamut is calculated. If the area difference is greater than a predetermined value, the hue segmentation system searches at least one vertex among the hue segments to divide the segments for generating a plurality of additional hue segments. The above-mentioned steps are performed repeatedly until the area difference is less than the predetermined value. Thus fewer hue segments indicate the color gamut of color processing of the image devices to effectively decrease calculation times and save required memory.