A system and method for effectuating optimized 360° immersive video viewing experience. A media input stream is coded into a first set of bitstreams having different qualities, wherein each GOP starts with an intra-coded (I) frame followed by a set of frames including at least one predictive-coded (P) frame. The media input stream is also coded into a second set of bitstreams having different qualities, wherein each GOP starts with an I-frame followed by a plurality of X-frames, each X-frame having a slice header of a P-frame and comprising blocks of intra-coded data only (e.g., I-blocks). During playout of the 360° immersive video content, when a viewport changes during mid-GOP and/or if bandwidth allocation changes, higher quality data in an X-frame is muxed into the coded video sequence for providing enhanced viewing experience with minimum latency.