A system and method of measuring audience reaction to media content includes obtaining images or other information of each individual of the audience as the individual views the content and analyzing the images or other information with software to generate emotional response source data for each individual. The emotional response source data identifies the emotion or emotions or change of emotions of the individual as interpreted by the software. The emotional response source data of the individuals is aggregated to identify a set of primary emotions of the audience, and thereafter, the emotional response source data is re-evaluated in view of the primary emotions to refine and more accurately classify the emotional responses of each individual of the audience.