In embodiments of multi-pass touch contact tracking, touch input sensor data is recognized as a series of components of a contact on a touch-screen display. The components can be determined to correlate to the contact based on multi-pass nearest-neighbor contact mapping that includes forward nearest-neighbor contact mapping of the components and reverse nearest-neighbor contact mapping of the components. The components can then be associated to represent a tracking of the contact. Subsequent components of the contact can also be determined and associated with the previous components of the contact to further represent the tracking of the contact.