An audio signal processing system includes: a receiving section which receives a plurality of audio signals, a process period specifying section which specifies a silent period in other audio signals except a particular audio signal that are received by the receiving section, respectively, a sound volume level adjusting section which decreases a sound volume level of the particular audio signal, in a period specified by the process period specifying section as a silent period in any of the other audio signals, and an outputting section which outputs the particular audio signal whose sound volume level is adjusted by the sound volume level adjusting section.