A media stream receiver is provided for scalable physical layer flow of packetized media streams. The media stream receiver replicates the processing block in time, rather than in hardware, through the use of a single shared memory and pointer alignment calculations, which combines multiple buffering stages as the single, shared memory buffer to offer redundancy and alignment, while acting as a receiver buffer to account for packet delay variations. By doing so the media stream receiver can perform a vertical interval switch between received media streams.