A method and apparatus provide the ability to resample video frame into various resolutions, and to predict, quantize, and entropy code the video signal for spatially scalable compression and networking applications. The solution involves a unified re-sampling and estimation-theoretic prediction, quantization, and entropy coding framework, which by design is tailored to allow base layer coding information to be fully accessible and usable by enhancement layers; and for the enhancement layer to account for all available information from both layers for superior compression performance. Specialization may include specific techniques for coding and networking scenarios, where the potential of the unified resampling and estimation-theoretic framework is realized to considerably improve the overall system performance over existing schemes.