An image shift amount calculation apparatus comprises: an obtaining unit that obtains a pair of images having parallax; a generation unit that generates, from the pair of images, an image pair for each of a plurality of hierarchal levels having different resolutions; and a calculation unit calculates an image shift amount for an image pair in a predetermined first hierarchal level among the plurality of hierarchal levels, determines, on the basis of the image shift amount in the first hierarchal level, a second hierarchal level, among the plurality of hierarchal levels, where the calculation of the image shift amount is to end, and calculates an image shift amount of the image pair in the second hierarchal level using the image shift amount in the first hierarchal level.