A method for removing color noise on a slowly varying component contained in color difference component image data of image data which is imported from an image sensor and converted to brightness and the color difference component image data, includes the steps of: sampling pixels of said color difference component image data by thinning out according to a first defined sampling format when not performing a color noise removal process on the slowly varying component; determining if the color noise removal process is necessary to be performed or not; producing the color difference component image data, corresponding to a compressed image data size smaller than an image data size without said color noise removal process, by thinning out according to a second defined sampling format when performing said color noise removal process; and recording the color difference and brightness component image data.