An unnecessary component such as acne is removed completely from a predetermined structure such as a face in a photograph image without manual operation and skills. An acne removal unit fits to a face region as the structure in the image a mathematical model generated according to a statistical method such as AAM using sample images representing the structure without the component to be removed, and an image reconstruction unit reconstructs an image of the face region based on parameters corresponding to the face region obtained by the fitting of the model. An image is then generated by replacing the face region with the reconstructed image. Since the mathematical model has been generated from the sample images of human faces without acne, the model does not include acne. Therefore, the reconstructed face image generated by fitting the model to the face region does not include acne.