The present disclosure includes methods and systems for streaming high-performance virtual reality video using adaptive rate allocation. In particular, an adaptive rate allocation system partitions a panorama video into segments or tiles and assigns priorities to each tile or segment based on input (e.g., a viewport of field-of-view) from a user client device. Further, the adaptive rate allocation system streams each tile or segment to the user client device according to the adaptive rate allocation, which maximizes bandwidth efficiency and video quality. In this manner, the adaptive rate allocation system delivers higher quality content to regions in the panorama video where a user is currently looking/most likely to look.