A tracking apparatus includes an image data acquisition unit, a tracking process unit, a contrast information acquisition unit, a contrast information similarity evaluation unit, and a control unit. The image data acquisition unit acquires image data. The tracking process unit detects a candidate position of a tracking target in image data. The contrast information acquisition unit acquires contrast information at the candidate position. The contrast information similarity evaluation unit evaluates a similarity between contrast information at a position of the tracking target decided in a past frame and current frame. The control unit decides the position of the tracking target in the current frame based on the evaluation of the similarity.