The present document relates to a method of layered encoding of a frame of a compressed higher-order Ambisonics, HOA, representation of a sound or sound field. The compressed HOA representation comprises a plurality of transport signals. The method comprises assigning the plurality of transport signals to a plurality of hierarchical layers, the plurality of layers including a base layer and one or more hierarchical enhancement layers, generating, for each layer, a respective HOA extension payload including side information for parametrically enhancing a reconstructed HOA representation obtainable from the transport signals assigned to the respective layer and any layers lower than the respective layer, assigning the generated HOA extension payloads to their respective layers, and signaling the generated HOA extension payloads in an output bitstream. The present document further relates to a method of decoding a frame of a compressed HOA representation of a sound or sound field, an encoder and a decoder for layered coding of a compressed HOA representation, and a data structure representing a frame of a compressed HOA representation of a sound or sound field.