A focus detection device includes: an imaging unit having a first pixel and a second pixel each of which receives light transmitted through an optical system and outputs signal used for focus detection; an input unit to which first information regarding a position on an image plane and an exit pupil distance of the optical system is input; a selection unit that selects a first focus detection based on the signal having been output from the first pixel or a second focus detection based on the signal having been output from the second pixel, based on the first information having been input to the input unit; and a focus detection unit that performs the first focus detection or the second focus detection based on a selection by the selection unit.