Aspects of a method and system for hierarchical motion estimation with multi-layer sub-pixel accuracy and motion vector smoothing are presented. Aspects of the system may include hierarchical motion vector computation that enables motion vectors to be computed at each level in the hierarchy based on a distinct pixel resolution level. A smoothing algorithm may be utilized to suppress spurious motion vector generation. The motion vectors computed at one level in the hierarchy may be utilized when computing motion vectors in a subsequent level. A bias value may be computed for each motion vector that provides an evaluation metric that may enable determination of whether the computed motion vector is to be utilized to enable generation of the interpolated image frame.