A method and apparatus for image stabilization takes an input image sequence including a plurality of frames, estimates frame-level motion vectors for each frame, and adaptively integrates the motion vectors to produce, for each frame, a motion vector to be used for image stabilization. A copy of the reference image of a frame is displaced by the corresponding adaptively integrated motion vector. In one embodiment, the perimeter of the image sensor is padded with a margin to be used for image compensation. In another embodiment, vertical and horizontal components are treated independently. In still another embodiment, the motion estimation circuitry associated with an MPEG-4 encoder is used to calculate macroblock level vectors, and a histogram is used to compute a corresponding frame-level vector for that frame.