Control data templates are generated independent of a plurality of audio elements based on user input. The user input relates to parameter values and control inputs for operations. In response to receiving audio elements after the control data templates are generated, audio objects are generated to store audio sample data representing the audio elements. Control data is generated based on the parameter values and the control inputs for the operations in the control data templates. The control data specifies the operations to be performed while rendering the audio objects. The control data is then stored separately from the audio sample data in the audio objects. The audio objects can be communicated to downstream recipient devices for rendering and/or remixing.