An image processing apparatus of the invention includes a first feature value calculation unit adapted to calculate a first feature value for each pixel in an image picked up of living tissue, where the first feature value represents a value of an index which indicates what shape a local region; a second feature value calculation unit adapted to calculate a degree of concentration of a gradient vector as a second feature value based on the gradient vector for each pixel in the image; an evaluation value calculation unit adapted to calculate a geometric evaluation value for each pixel, based on calculation results of the first feature value and the second feature value; and a region extraction unit of separately extracting a candidate region of the linear structure and a candidate region of the massive structure based on a calculation result of the geometric evaluation value.