A method and system for training a special object detector to distinguish a foreground object appearing in a sequence of frames for a target domain. The sequence of frames depicts motion of the foreground object in a non-uniform background. The foreground object is detected in a high-confidence subwindow of an initial frame of the sequence, which includes computing a measure of confidence that the high-confidence subwindow includes the foreground object and determining that the measure of confidence exceeds a specified confidence threshold. The foreground object is tracked in respective positive subwindows of subsequent frames appearing after the initial frame. The subsequent frames are within a specified short period of time. The positive subwindows are used to train the special object detector to detect the foreground object in the target domain. The positive subwindows include the subwindow of the initial frame and the respective subwindows of the subsequent frames.