A system that generates a virtual viewpoint image from multiple viewpoint images captured from different camera viewpoints uses background 3D data representing a three-dimensional shape of a background in an image capturing scene. This background 3D data is generated by using a simulation image representing a view from the camera viewpoint. Then, a difference between the simulation image and an image acquired by performing test image capturing from the same camera viewpoint as the camera viewpoint is detected and the background 3D data is updated based on detection results.