There is provided a signal processing device including a correction processing unit that acquires a pixel signal output from a sensor on which pixels are disposed in an array in which a spatial frequency of a color pixel which is a pixel acquiring a color component is lower than a spatial frequency of luminance pixels which are pixels acquiring luminance components, and then corrects the pixel signal output from a defective pixel out of the pixels that the sensor includes. During correction of a pixel signal of the color pixel, the correction processing unit performs correction referring to pixel signals of the luminance pixels having a spatial frequency higher than the spatial frequency of the color pixel.