A tracking device is provided which includes an image information acquiring unit configured to acquire image information in the form of successive frames; and a tracking unit configured to generate a plurality of sub images each smaller than a frame of the acquired image information, to calculate likelihoods with eye images, and to decide locations of eyes using a sub image having a large likelihood value. The tracking unit decides locations of the sub images, based on locations decided by a frame of image information acquired before the one frame.