When a face is detected from image signals, stability determination time is set for determining if the face is recognized in a stable manner, and a period of time for which the face is consecutively detected starts to be measured. Until the measured period of time exceeds the stability determination time, an AF evaluation value acquired from a normal frame set to include a face frame corresponding to the face is used to exercise TV-AF control. Once the measured time exceeds the stability determination time, TV-AF control is exercised based on AF evaluation values acquired from each of the face frame and the normal frame.