An encoding device (1) for converting a first number (M) of input audio channels into a second, smaller number (N) of output audio channels comprises at least one conversion unit (12) for converting a first signal (Lf; Rf; Co) and a second signal (Lr; Rr; Le) into a third signal (L; R; C) and a fourth signal (Ls; Rs; Cs). The third, dominant signal contains most of the signal energy of the first and second signals, while the fourth, residual signal contains the remainder of said signal energy. The encoding device is arranged for using the third signal (L; R; C) to produce an output signal and for outputting the fourth signal (Ls; Rs; Cs). A decoding device (2) for converting a first number (N) of input audio channels into a second, larger number (M) or output audio channels comprises at least one conversion unit (24) for converting a first signal (L; R; C) and a second signal (Ld; Rd; Ld) into a third signal (Lf, Rf; Co) and a fourth signal (Lr; Rr; Le). The first, dominant signal contains most of the signal energy of the third and fourth signal, while the second, residual signal contains the remainder of said signal energy. The encoding device is arranged for receiving at least one-second signal (Ld; Rd; Cd).