An image processing apparatus includes: a division unit configured to divide each of a plurality of frame images into a plurality of regions on a screen; a motion vector detection unit configured to detect a motion vector for each of the divided regions; a separation unit configured to separate a subject vector that indicates motion of a subject from a background vector that indicates motion of a background, based on a panning speed of the apparatus that was detected by a panning detection unit and the motion vector detected by the motion vector detection unit; and an identification unit configured to identify a subject region by weighting the subject vector based on information regarding a distance of the subject that was detected by a distance detection unit for each of the divided regions of the plurality of frame images.