A computer-implemented method for clustering audio fingerprints from different set top boxes into groups based on their similarities is disclosed. The computer server receives audio fingerprints generated at different set top boxes and partitions them into multiple groups such that audio fingerprints within a group correspond to a respective TV program being played at the respective set top boxes. Upon receipt of an audio fingerprint from a set top box, the computer server identifies a group of audio fingerprints that are substantially similar to the audio fingerprint, determines TV program information for the audio fingerprint in accordance with the channel metadata associated with the identified group of audio fingerprints, and returns the TV program information to the requesting set top box.