In an object tracking device, an object detection unit detects an object from video data using a plurality of object detection programs of different detection accuracy. The object tracking unit tracks the object from the detected position of the object by the object detection unit. The control unit controls the object detection programs used by the object detection unit based on the detection result of the object by the object detection unit and the tracking result of the object by the object tracking unit.