An imaging unit includes a color filter in which filter elements of R, G and B are arranged in a plane thereof. The imaging unit captures an object in a state where infrared light with three frequencies assigned to R, G and B are projected so as to generate first to third frames. A synthesizing unit in a pre-signal processing unit arranges pixel data for the three primary colors including pixel data for R in the first frame, pixel data for G in the second frame and pixel data for B in the third frame in such a manner as to have the same array as the filter elements in the color filter, so as to generate synthesized image signals synthesized in one frame. A demosaicing unit subjects the frame of the synthesized image signals to demosaicing so as to generate frames of the three primary colors.