A system and method is provided for improving media playback and seek time for media files such as encoded audio data. As the media is decoded during playback, frame position information is stored in a frame table. This frame position information may be used for pause/resume functionality, and for seek operations. If a seek operation is to a position not stored in the frame table, the media data is parsed to produce frame position information that is added to the frame table until the seek location is reached. Frame table data may be stored in several different forms, including periodic intervals that may be selected based on the length of the media data, and also contemporaneous data useful for resuming from a pause command.