A system and method for video coding include an encoder and decoder. The encoder/decoder respectively include a base layer encoding/decoding apparatus, at least one enhancement layer encoding/decoding apparatus, and an encoder/decoder drift control apparatus. The encoder drift control apparatus is configured to determine the amount of local error drift for the encoder according to local information of the base layer encoding apparatus and the enhancement layer encoding apparatus and control the value of an encoder leaky factor according to the amount of error drift. The decoder drift control apparatus is configured to determine the amount of local error drift for the decoder according to local information of the base layer decoding apparatus and the enhancement layer decoding apparatus and control a decoder leaky factor according to the amount of error drift.