Provided are a sound output control apparatus, a sound output control system, a sound output control method, and a program that can appropriately thin out the output of pieces of sound data. A sound data reception section receives a plurality of pieces of sound data transmitted from transmission apparatuses that are different from each other. A selection section selects a portion of the plurality of pieces of sound data on the basis of at least one of a result of a voice activity detection process performed on each of the pieces of sound data or moving averages of volumes of sounds represented by the pieces of sound data. A sound data transmission section outputs the selected portion of the pieces of sound data.