An image processing apparatus includes a receiving unit configured to receive at least two parallax images that are obtained from a subject image captured via a single optical system, where the at least two parallax images include an image in a first viewpoint direction and an image in a second viewpoint direction, an average calculating unit configured to calculate, for each pixel, an arithmetic average and a geometric average between the image in the first viewpoint direction and the image in the second viewpoint direction, a ratio calculating unit configured to calculate, for each pixel, a ratio of the arithmetic average to the geometric average, and a disparity calculating unit configured to calculate, on a pixel-by-pixel basis, a disparity between the image in the first viewpoint direction and the image in the second viewpoint direction based on the ratio.