An image processing apparatus includes an acquisition unit that acquires a parallax image generated based on a signal from one of a plurality of photoelectric converters that receive light beams passing through partial pupil regions of an imaging optical system different from each other, and a captured image generated by combining a plurality of signals from the plurality of photoelectric converters. A determination unit determines whether the parallax image contains a defect, and an image processing unit corrects, if the determination unit determines that the parallax image contains the defect, one or more pixel values of the parallax image that include the defect, using the captured image to calculate a correction value for each of the one or more pixels in the parallax image, and replaces the one or more pixels in the parallax image with the corresponding correction value.