An image pickup apparatus that, even when an object is framed-out to be undetectable, is capable of assisting framing with comfort by quick recapturing of the object within the angle of view. A specific object is detected from a live view image picked up by an image pickup device. Movement of the image pickup apparatus is detected, and a zoom operation is performed according to an amount of the detected movement of the image pickup apparatus. When predetermined conditions are satisfied which include a condition that a direction of the movement of the detected object is the same as a direction of the detected movement of the image pickup apparatus, FA zoom control is performed such that the FA zoom operation according to an amount of movement of the image pickup apparatus is restricted.