In an endoscope apparatus, a first color separation section separates an image picked up by an image pickup section into a first luminance signal and a first color difference signal, then a first color conversion section and a second color separation section convert the first luminance signal and the first color difference signal to first three primary color signals and second three primary color signals respectively, a signal intensity ratio calculation circuit calculates a signal intensity ratio among the first three primary color signals, matrix coefficients of the second color separation section are changed based on the calculated signal intensity ratio and the second color separation section converts the first luminance signal and the second color difference signal to the second three primary color signals.