A control apparatus includes an acquiring unit configured to acquire a contrast evaluation value for each of a plurality of areas in an image, and a controlling unit configured to detect an in-focus position of a focus lens for each of the plurality of areas based on the contrast evaluation value. The controlling unit determines whether or not the focus lens approaches the in-focus position of each of the plurality of areas based on the contrast evaluation value. The controlling unit determines a driving direction of the focus lens based on a result of a determination. When detecting the in-focus position of one area of the plurality of areas, the controlling unit determines the driving direction of the focus lens based on the contrast evaluation value of another area of the plurality of areas other than the one area.