An image processing apparatus according to this invention includes an information generation unit which generates correspondence information representing correspondence between first images whose viewpoints are different each other, and a selection unit which selects, out of the first images on the basis of the correspondence information, second images whose viewpoints have a predetermined relationship.