From an image pickup element, pixel signals of Bayer array are outputted. A correlation calculation part calculates correlation values with respect to a specified pixel in vertical and horizontal directions. A first interpolation part performs a pixel interpolation process while evaluating the correlation highly. A second interpolation part performs a pixel interpolation process while evaluating the correlation relatively low. A complete signal of RGB outputted from the first interpolation part is converted into a luminance signal in a first color space conversion part, and a complete signal of RGB outputted from the second interpolation part is converted into a color difference signal in a second color space conversion part.