Systems and methods are provided herein relating to audio matching. Adaptive weighting of popular reference content can be used to more efficiently allocate space in a weighted reference index used to match audio signals. An audio reference index can be maintained that contains a set of audio references wherein each audio reference in the set of audio references is associated with a score. A weighted reference index can be generated based on the audio reference index and the score associated with each audio reference wherein respective audio references are up-weighted or up-scored based at least in part of user popularity. The benefits in using adaptive weighting of popular reference content can improve the accuracy of an audio matching system.