A system and method for controlling a media player for replacement content, such as dynamic ad insertion. The system tracks video watermarks from a content stream, where the input content timeline is being modified by a user exercising the transport controls of a digital video recorder (DVR). A Detector Engine receives decoded video frames and extracts a time-offset field, a VP1 payload, and a Cyclic Redundance Check (CRC) field in each video frame. A Content Timeline Tracker monitors and analyzes the output of the Detector Engine to produce a piecewise linear approximation of the content timeline, wherein playback rate changes by a user in an upstream device can be tracked. This enables the playback of auxiliary content which is synchronized to a watermark timeline recovered from the received content in cases where the recovered timeline has a non-linear mapping to real time. When the estimated speed is changing due to user-controlled trick play of recorded content, estimated speed deviates from the user intended speed profile because of imperfect playback of the media player. The system includes additional filtering of estimated speed to produce a Boolean updated speed which is asserted sparsely at estimated control segment endpoints in an attempt to delineate constant slope (constant speed) control segments.