A method for comparing videos includes detecting local features in frames of a video and determining the positions of detected local features in a first frame compared to their respective positions in a succeeding second frame. For at least some of the detected local features, a movement indicator is generated representing the position of a local feature in the first frame relative to the position of that local feature in the second frame, the movement indicator including an orientation, to obtain a plurality of movement indicators. Movement indicators are grouped by orientation ranges. The number of grouped movement indicators in an orientation range are summed to obtain a value for that orientation range. A time series of values for at least one given orientation range is obtained by determining the positions of detected local features in a first frame compared to their respective positions in a succeeding second frame for a plurality of pairs of frames. For at least one orientation range, a first time series of values for a first video is compared with a second time series of values for a second video to determine the similarity of the first and second videos.