A streaming device may request and download multi-layer video segments based on a number of factors including the artistic interest associated with the video segments and/or the status of a buffer area managed by the streaming device. The multi-layer segments may be coded using scalable coding techniques or a combination of scalable coding and simulcast coding techniques by which each of the video segments may be coded into one or more representations of different qualities and/or bitrates. When requesting the multi-layer segments, the streaming device may ensure that the fullness of the buffer area falls between a buffer underflow threshold and a backfilling threshold under various network conditions. The streaming device may estimate the available network bandwidth in order to facilitate the scheduling decisions. The streaming device may consider the artistic interest associated with the video segments during scheduling and may give priority to those segments with higher artistic interest.