This invention can control a code amount while suppressing image quality degradation of a virtual view-point image. An image processing apparatus comprises a reception unit which receives encoded image data from an image capturing device of interest among a plurality of image capturing devices, a determination unit which determines, based on an index value of an image quality represented by encoded image data received in the past from the image capturing device of interest and the index value of an image quality represented by encoded image data transmitted by another image capturing device, a permissible range of a target image quality of a captured image, and a setting unit which sets, in an encoding unit of the image capturing device of interest, an encoding parameter according to an image quality in the determined permissible range.