A server system hosts one or more virtual machines. The server system receives a first media stream that includes first content corresponding to a plurality of frames of video data and generates a first digest segment that corresponds to the first media stream, wherein the first digest segment includes a representation of the plurality of frames but does not include the video data. The server system stores the first digest segment in a cache at the server system; and receives a playback position of the first media stream at a first client device. The server system uses the playback position from the first client device and the first digest segment stored in the cache to perform processing to recreate a representation of the playback of the first media stream on the first client device.