A video surveillance system comprises two or more video cameras each providing a video signal, the cameras each being positioned at a respective known location; a motion detector to detect image motion of an image feature within the field of view of one of the video cameras and for deriving a corresponding physical motion of an object represented by the image feature; and a motion predictor to predict the physical motion of the object with respect to the known locations of the cameras so as to derive an expected time when the object may be observed in the field of view of another one of the video cameras, in which the motion predictor is responsive to routing data defining possible routes from the observed location of the object to the field of view of another of the cameras.