An image processing apparatus processes an image obtained by an image capturing apparatus that includes a plurality of first photoelectric converters and a plurality of second photoelectric converters each forming a pair with each other. The second photoelectric converters providing a signal that has a parallax for a signal from the corresponding one of the first photoelectric converters, and a signal reading unit configured to read a first signal as an output signal from the first photoelectric converter and to read a second signal as a combination signal of the first signal and an output signal from a paired one of the second photoelectric converters in image capturing in which the first and second photoelectric converters receive light from an object. The image processing apparatus includes a defocus amount calculating unit configured to calculate a defocus amount using the first signal and the second signal.