A focus control device includes an acquisition unit that acquires image data of a plurality of first frames, each of the plurality of first frames being captured at each of positions of a lens in an optical axis direction, the positions being different each other; and a movement unit that, when a focusing position of the lens is not determined based on contrast evaluation values for an area to be focused on in the plurality of first frames, determines a direction for moving the area, based on a magnitude of a contrast evaluation value within the area and a position of an area with the contrast evaluation value larger than or equal to a given threshold in each piece of image data of the plurality of first frames or a plurality of second frames newly acquired by the acquisition unit, and moves the area in the determined direction.