The focus control device includes a focus control section that controls an imaging optical system that is configured so that an in-focus object plane position is changed when an imaging magnification is changed, an image acquisition section that acquires a plurality of images captured through the imaging optical system at a different imaging magnification, and a change-in-magnification detection section that detects a change in magnification that is at least one of a change in the imaging magnification and a change in size of an object within an image among the plurality of images, the focus control section driving the imaging optical system based on an AF evaluation value that indicates the focus state of the imaging optical system and calculated based on the image and the change in magnification to control focus of the imaging optical system.