An image processing apparatus includes: a model estimator that models gradient variations of pixel values in an image of a captured object, which is an in-vivo image, according to the pixel values in the image; and an abnormality candidate detector that detects an abnormality candidate area on the object shown in the image according to a difference between a pixel value of each pixel constituting the image and an estimated pixel value of each pixel, the estimated pixel value being determined according to the modeled gradient variations of the pixel values.