An image processing apparatus includes: a first distribution calculation unit calculating a distribution of luminance gradient vectors in a first local area that includes a feature point on an image; a second distribution calculation unit calculating distributions of the luminance gradient vectors in second local areas close to the feature point on the image; a selection unit comparing the distribution of the luminance gradient vectors of the first local area with the distributions of the luminance gradient vectors of the second local areas to select the most different distribution of the luminance gradient vectors of the second local area; and a feature descriptor calculation unit calculating a feature descriptor at the feature point on the basis of the distribution of the luminance gradient vectors of the first local area and the distribution of the luminance gradient vectors of the second local area selected by the selection unit.