The present invention is a system and method for restructuring a group of pictures in such a manner as to provide for random access into the group of pictures without substantially degrading the quality of service for the video content. When encoding a current picture, if the current picture is an I picture, the current picture is encoded utilizing intra coding. If the current picture is not an I picture, and is not referenced by future pictures, the current picture is compressed using an arbitrary number of pictures in the past as a reference. If the current picture is not an I picture, and the current picture is not referenced by future pictures, the picture is compressed and the number of reference pictures in the past are limited so that picture prior to the last I picture are not used as a reference. This process continues until the conclusion of the group of pictures.