An audio encoder (109) has a hierarchical encoding structure and generates a data stream comprising one or more audio channels as well as parametric audio encoding data. The encoder (109) comprises an encoding structure processor (305) which inserts decoder tree structure data into the data stream. The decoder tree structure data comprises at least one data value indicative of a channel split characteristic for an audio channel at a hierarchical layer of the hierarchical decoder structure and may specifically specify the decoder tree structures to be applied by a decoder. A decoder (115) comprises a receiver (401) which receives the data stream and a decoder structure processor (405) for generating the hierarchical decoder structure in response to the decoder tree structure data. A decode processor (403) then generates output audio channels from the data stream using the hierarchical decoder structure.