A tracking apparatus comprises a detection unit that detects a subject in an image obtained by photoelectrically converting light entering through an imaging optical system with an image sensor, a tracking unit that tracks the subject in a plurality of predetermined directions, a determination unit that determines a tracking direction in which the subject is to be tracked among the plurality of directions, and a control unit that controls the tracking unit to perform tracking in the tracking direction and suppress tracking in a direction different from the tracking direction.