Audio control system and audio control method that can appropriately control sound volume of an audio part while preventing inaccurate determination regarding condition of a passenger are provided. An audio control system includes an audio part installed in a compartment of a vehicle, a photographing part that photographs the compartment from a front side reference position in the compartment and obtains a photographed image also representing distance to an object within a photographing range in the compartment, a determination part that determines, based on the photographed image obtained by the photographing part, at least one of whether or not a passenger other than a driver in the vehicle is asleep and whether or not the passenger is making a call on a mobile phone, and a control part that turns down sound volume of the audio part if the determination part determined the passenger is asleep or making a call.