A method for compressing a HOA signal being an input HOA representation with input time frames (C(k)) of HOA coefficient sequences comprises spatial HOA encoding of the input time frames and subsequent perceptual encoding and source encoding. Each input time frame is decomposed (802) into a frame of predominant sound signals (XPS(k−1)) and a frame of an ambient HOA component ({tilde over (C)}AMB (k−1)). The ambient HOA component ({tilde over (C)}AMB (k−1)) comprises, in a layered mode, first HOA coefficient sequences of the input HOA representation (cn(k−1)) in lower positions and second HOA coefficient sequences (cAMB,n(k−1)) in remaining higher positions. The second HOA coefficient sequences are part of an HOA representation of a residual between the input HOA representation and the HOA representation of the predominant sound signals.