A focus detection device includes an imaging unit, an image processor, a focus detection circuit, and a control circuit. The imaging unit includes a pixel portion including light-receiving portions. The light-receiving portions are divided into first and second pupil division directions. The image processor performs processing to display an image based on a display pixel signal. The focus detection circuit performs processing for focus detection based on a focus detection pixel signal. The control circuit controls the imaging unit to generate the focus detection pixel signal regarding the first pupil division direction in a first operation, and controls the imaging unit to generate the display pixel signal and the focus detection pixel signal regarding the second pupil division direction in a second operation.