An object detection system including a storage unit, image acquisition module, feature searching module, calculation module, and confirmation module is provided. The image acquisition module obtains a first, second, and third image respectively corresponding to a first, second, and third timepoint of a video sequence from the storage unit, where time difference between the first and second timepoint is less than or equal to that between the second and third timepoint. The feature searching module searches for at least one feature from the first image. When the feature is within a near-distance detected region, the calculation module calculates a motion vector of the feature according to the first and second image; otherwise, it calculates the motion vector of the feature according to the third and one of the first and second image. The confirmation module confirms the existence of a moving object according to the motion vector of the feature.