The method of analyzing a video sequence for motion estimation comprises computing first matching energies for individual local displacements between frames of the video sequence using a first window around a pixel, and determining a minimum of the first matching energies. Second matching energies for motion configurations each including a plurality of local displacements between frames of the video sequence using the first window around said pixel are also computed. If none of the second matching energies satisfies a comparison criterion with the minimum of the first matching energies, a local displacement providing the minimum of the first matching energies is associated with said pixel. On the other hand, if at least one motion configuration provides a second matching energy satisfying the comparison criterion with the minimum of the first matching energies, a group of local displacements is formed based on local displacements of said at least one motion configuration, and third matching energies are computed for the individual local displacements of that group using a second window smaller than the first window. A local displacement of the group for which the third matching energy is minimal can then be associated with the pixel.