Disclosed is a non-transitory computer readable storage medium which receives, by an audio decoder (operating in a specific playback environment different from a reference channel configuration), an audio signal for the reference channel configuration. The audio signal includes audio sample data and encoder-generated loudness metadata which includes a plurality of portions of loudness metadata for a plurality of playback environments. The plurality of portions of loudness metadata includes one or more respective portions of loudness metadata for each playback environment in the plurality of playback environments. The medium also selects one or more portions of specific loudness metadata (based on the specific playback environment), from among the plurality of portions of loudness metadata for the plurality of playback environments. The one or more portions of specific loudness metadata relating to the specific playback environment determine loudness adjustment gains from the one or more portions of specific loudness metadata for the specific playback environment, apply the loudness adjustment gains as a part of overall gains applied to the audio sample data to generate output audio data.