System and methodologies are provided herein for reconstructing a video signal from multiple video streams. Various aspects described herein can utilize a least square estimate (LSE) algorithm to jointly decode multiple video bitstreams that are generated from a common original video sequence at different bit rates. As described herein, the LSE algorithm can reconstruct an original video sequence by determining and computing a weighted sum of collocated video information reconstructed from different video bitstreams. The weights applied can be adaptively determined to minimize the mean square error (MSE) of the reconstructed video sequence as compared to the original.