A control device includes: a determination unit determining which mode is designated in a single-eye imaging mode in which imaging is performed by using a single imaging unit obtaining an image signal by imaging an object by an imaging device and a compound-eye imaging mode in which imaging is performed by using plural imaging units; and a gain maximum value control unit changing a gain maximum value which is the maximum value of the gain of the image signal based on the determination result by the determination unit.