Method for improving audio time stretching starts with processor receiving input media file and rate schedule. Rate schedule includes identification of portions of input media file and playback output rates associated with each of the portions of input media file. Writer thread being executed by processor generates timeline that is linked list including segments that respectively represent portions of input media file and associated playback output rates as desired playback rate curves. Generating timeline include writing each segment in linked list to end of linked list. Processor then performs time scaling of media file based on rate schedule to generate output media file. Performing time scaling of media file includes identifying by reader thread being executed by processor a segment in timeline corresponding to desired output time, and translating by reader thread desired output time to corresponding input time based on identified segment. Other embodiments are described.