A decoder for generating an audio output signal having one or more audio output channels from a downmix signal having three or more downmix channels, wherein the downmix signal encodes three or more audio object signals is provided. The decoder includes an input channel router and at least two channel processing units. Each channel processing unit of the at least two channel processing units is configured to generate one or more of at least two processed channels depending on side information and depending on one or more of the three or more downmix channels received by the channel processing unit from the input channel router.