Systems and methods are presented for adaptively streaming a time-ordered sequence of video frames from a video server, across a wide area network, to a remote client video device, to accommodate advanced video display modes. Based on indications received from the remote client, the video server obtains and sends to the remote client encodings of video frames adapted to facilitate smooth, timely transitions between display modes at the client, and optimal playback quality across all modes, subject to various operational constraints. Advanced playback modes include fast forward and reverse, slow forward and reverse, frame-by-frame stepping, and other modes that specify displaying video frames on the client at a rate different from the normal display rate, or that specify displaying video frames in a different order than the normal time-ordering of the sequence of video frames. The systems and methods may be incorporated into collaborative online video editing environments.