A system and method for stabilizing video using a census kernel having test points. A subset form a connected circle centered on the kernel's center point. For each candidate pixel, a signature is calculated based on the relative brightness of the candidate pixel and of the pixels corresponding to test points when the kernel overlays the image with the center point over the candidate pixel. A candidate pixel is designated to be a corner pixel by processing the signature to determine that the candidate pixel is significantly brighter or darker than the test point pixels. For each corner pixel, a corresponding pixel in the previous frame is identified by comparing the signature of the corner pixel with signatures of pixels in the previous frame, and a corresponding motion vector is established. A motion model is calculated based on the motion vectors and applied to the digital frame.