The described technology is directed towards having multiple paths through streamed media content, such as a video. The content may be represented as a state machine of states, in which each state corresponds to one or more periods of one or more segments, and transitions to one or more other states. When a state is able to transition to different states, one or more criteria may be used to select one of the transition paths to a next state. Segments corresponding to unknown paths (where the transition decision is not yet available) may be selected and streamed for buffering via a multiple path buffering mechanism.