An image pickup apparatus includes an image pickup element that includes imaging pixels and focus detection pixels and that performs a photoelectric conversion of an object image, a readout unit that reads a pair of image signals from the focus detection pixels with a predetermined period, a storage unit that stores the pair of image signals read by the readout unit and optical information of the pair of image signals, an addition unit that adds a plurality of signals of time-series continuous frames of the pair of image signals, and a conversion coefficient determination unit that determines a conversion coefficient that converts an image shift amount of a pair of image signals of an addition frame obtained by the addition unit into a defocus amount, and the conversion coefficient determination unit determines the conversion coefficient of the addition frame based on the time-series continuous optical information.