An imaging apparatus includes an imaging element for outputting focus detecting signals. A DFE performs phase difference detection-type focus detecting calculation using an output signal of the imaging element. A CPU controls a focus drive circuit depending on the result of calculation and a focus lens is driven so as to perform a focus adjusting operation. The CPU controls the DFE to intermittently execute focus detecting calculation at a predetermined frequency set for a plurality of image frames. In the first mode, AF processing is executed at a rate of once in a few frames and the operation of a circuit unit for performing correlation calculation or the like is stopped in a frame period during which no focus detecting calculation is performed, whereas in the second mode, AF processing is executed for each frame so as to perform a focus adjusting operation following the movement of an object.