A system for generating and outputting one or more audio output signals has a decomposition module, a signal processor, and an output interface. The decomposition module can receive two or more audio input signals, to generate a direct component signal, having direct signal components of the audio input signals, and to generate a diffuse component signal, having diffuse signal components of the audio input signals. The signal processor can receive the direct component signal, the diffuse component signal and direction information, to generate one or more processed diffuse signals depending on the diffuse component signal, for each of the one or more audio output signals, to determine, depending on the direction of arrival, a direct gain, to apply the direct gain on the direct component signal to obtain a processed direct signal, and to combine the processed direct signal and a processed diffuse signal to generate the audio output signal.