The invention relates to video surveillance systems that are used, for example, for surveying public places, stations, streets, industrial estates, buildings or similar. Said video surveillance systems comprise one or more surveillance cameras that are oriented towards surveillance scenes and transfer image data streams, in the form of image sequences, to an evaluation center. The invention also relates to a device (1) for identifying and/or classifying a movement pattern in an image sequence of a surveillance scene comprising a plurality of moving objects, an interface (2) for recording the image sequence, a calculation module (5) for determining an optical flow field (10) in the surveillance scene by evaluating the image sequence and an identification module (6) that is designed in terms of programming and/or circuitry such that the optical field and/or partial areas thereof are compared to one or more patterns in order to identify the movement pattern in the image sequence.