An adaptive stream player that has control over whether a retrieved stream is cached in a local stream cache. For at least some of the stream portions requested by the player, before going out over the network, a cache control component first determines whether or not an acceptable version of the stream portion is present in a stream cache. If there is an acceptable version in the stream cache, that version is provided rather than having to request the stream portion of the network. For stream portions received over the network, the cache control component decides whether or not to cache that stream portion. Thus, the cache control component allows the adaptive stream player to work in offline scenarios and also allows the adaptive stream player to have rewind, pause, and other controls that use cached content.