Methods, systems, and apparatus, including computer-readable media storing executable instructions, for enhanced video conference management. In some implementations, a participant score for each participant in a set of multiple participants in the communication session. Each of the participant scores is based on at least one of facial image analysis or facial video analysis performed using image data or video data captured for the corresponding participant during the communication session. Each of the participant scores is indicative of an emotional or cognitive state of the corresponding participant. The participant scores are used to generate an aggregate representation of the emotional or cognitive states of the set of multiple participants. During the communication session, output data is comprising the aggregate representation of the emotional or cognitive states of the set of multiple participants is provided for display.