Methods and systems stabilization of a camera image for short term or ‘pole shake’ and longer term ‘pole drift’ are provided. The camera is attached to a fixed structure. The pole drift is over periods of times long enough that the imagery can change significantly between a base reference frame and the current stabilized transient frame. A multitude of mapped frames and associated information is maintained. The slowly varying camera orientation (“pole drift”) is decoupled from the rapid motion jitter (“pole shake”), and separate transformations are calculated for each. Up to three transformations may be combined together for any one input frame. The separate transformations are combined together, so that only one full-blown image transformation computation is performed for each frame. A surveillance system applies a stabilization method.