The present invention relates to an apparatus and method for client-side compositing of video streams. The method includes receiving, by the video display device, a multiplexed data stream from a remote video server. The multiplexed data stream includes substreams, where the substreams includes a descriptor substream, at least one media substream, and a compositing-instruction substream. The method further includes demultiplexing, by the video display device, the multiplexed data stream into the substreams according to the descriptor substream and displaying, by the video display device, the at least one media substream on a display screen according to the compositing-instruction substream, where the compositing-instruction substream includes instructions on a composition of the at least one media substream.