An audio surround processing system receives an audio source signal having at least two audio channels and generates a number of additional surround sound signals in which an amount of artificially generated ambient energy is controlled in real-time at least in part by an estimate of ambient energy that is contained in the audio source signal. The system may divide the audio source signal into two sets of components; a first set of components and a second set of components. The first set of components may be in a range of frequency that is less than a range of frequency of the second set of components. An ambience estimate control coefficient may be generated using the transformed first set of components. An overall gain may be determined using the ambience estimate control coefficient. The overall gain may be used in generation of the additional surround sound signals.