Systems and methods are disclosed herein for selective audio segment compression for accelerated playback of media assets. A playback speed of the video segment of a media asset is calculated based on the duration of the video segment and a received playback time period. A priority weight for each of the various audio segments is then determined. The audio segments with the lowest priority weight are removed from the group of various audio segments. The system then determines whether the duration of the remaining audio segments exceeds the received playback time period. If so, the system modifies the remaining audio segments by removing another audio segment with the lowest priority weight from the remaining audio segments. The system then rechecks whether the received playback time period is exceeded. If not, the system generates for playback the video segment based on the video playback speed and the remaining audio segments.