In an image capturing apparatus that carries out TV-AF type autofocus control by setting an AF frame on a predetermined object, for example a person's face, that is detected in an image, in a case in which a in-focus position search direction cannot be determined from an AF evaluation value, changes in the size of the region of the object are detected and focus detection is carried out by determining the in-focus position search direction based on the changes in the size of the region of the object.