Local motion estimation is described herein. Each picture of a video is partitioned into blocks for the local motion estimation. An extended-block FFT is calculated for each block, where the extended-block denotes that a certain area around the block is also included for applying FFT. Extending the block for FFT helps to account for the motion of objects that are moving into or out of the block. Phase correlation is applied to attain a set of Motion Vector (MV) candidates for the blocks, and a cost function is evaluated for each MV. If no MV candidate produces a cost function below a pre-defined threshold, a hierarchical variable block matching search is applied and the process is repeated with blocks for finer resolution. Also, predictive MV candidates are used during the block matching search along with temporal constraints tracking to select an MV that yields the minimum cost function.