An image pickup apparatus that is capable of performing a focusing control with sufficient accuracy without restricting a settable range of a focus detection area. An image pickup device has pixels each of which has sub-pixels that receive light beams passing through different pupil areas. A correlation operation area including a focus detection area and a shift area is set in an image indicated by an output from the image pickup device. A defocus amount is calculated in response to a correlation amount found by applying a shift process to image signals from the sub-pixels. If the focus detection area is set at a position where the correlation operation area is set with a default width, the shift area is set with a predetermined width. Otherwise the shift area is set with a narrower width and the correlation operation area is set with a width narrower than the default width.