An image processing apparatus for acquiring a current video frame and a scene model based on an input video; determining, based on visual elements in the current video frame, visual element models in the scene model to be visual element models necessary to be corrected and visual element models unnecessary to be corrected; and correcting the visual element models necessary to be corrected based on reference visual element models, wherein the reference visual element models are visual element models in the scene model that are determined to be the visual element models unnecessary to be corrected based on each of the current video frame and its previous video frames, similarities among the appearance features of the reference visual element models and the visual element models necessary to be corrected in the previous video frames satisfy a predetermined condition.