An image processing apparatus includes an attention area detection unit, a luminance parallax conversion unit, and a parallax estimation unit. The attention area detection unit is configured to detect an attention area including a desired subject from a standard image. The luminance parallax conversion unit is configured to perform a luminance parallax conversion with respect to the attention area on the basis of a luminance parallax conversion characteristic estimated by using a past frame. The parallax estimation unit is configured to perform parallax estimation on the basis of the standard image and a reference image, a viewpoint position of which is different from that of the standard image, and perform, in the attention area, the parallax estimation by using a luminance parallax conversion result obtained by the luminance parallax conversion unit.