There are provided scalable video encoders, scalable video decoders, and corresponding methods for scalable video encoding and decoding. A scalable video encoder includes an encoder for encoding a block in a picture by performing a motion compensated temporal filtering (MCTF) prediction step for the block using a first weighting factor applied to a first reference picture, while using a different default update weighting factor applied to a second reference picture for a corresponding MCTF update step for the block.