The image processing apparatus performing a process for displaying a still image acquired by an image capturing apparatus during its panning to follow a moving main object. The image processing apparatus includes a vector acquiring unit configured to acquire, from multiple motion vectors detected in a moving image acquired by the image capturing apparatus during the panning, a background motion vector, and a processing unit configured to determine, using the background motion vector, a background area of the still image where a background object is captured and to perform a magnification display process for magnifying the background area as compared with a case where the display unit displays the entire still image and causing the display unit to display the magnified background area.