Color processing is executed in consideration of a change in size of a region of interest in a moving image. To this end, a region of interest is set in each frame image of a moving image and an occupied area ratio indicating an amount that the region of interest occupies with respect to the entire frame image is calculated for each of the frame images set with the regions of interest. A temporal increase or decrease of the occupied area ratio is determined based on the occupied area ratios of a plurality of frame images set with the regions of interest, and a color processing method is set for each frame image based on the occupied area ratio and the temporal increase or decrease determination result. Color processing is applied to each frame image by the color processing method set for that frame image.