Multi-channel audio signals arranged to achieve original two-dimensional sound image localization are input, and the audio signal of each channel, included in the input multi-channel audio signals, is distributed to individual output channels. Each of the distributed signals is multiplied by a corresponding coefficient determined independently for each of the output channels, in accordance with a deviation from the original two-dimensional sound image localization. Then, the audio signals distributed to the individual output channels and multiplied by the corresponding coefficients are summed up, separately for each of the output channels. Thus, the summed-up audio signals of the individual output channels are output as multi-channel audio signals having the sound image localization varied in accordance with the deviation. If the deviation from the original two-dimensional sound image localization is varied over time, a panning effect can be achieved.