A video encoding method including determining a phase shift between pixels of an enhancement layer image and a reference layer image, according to a scaling factor between the enhancement layer image and the reference layer image and a color format of the enhancement layer; selecting at least one filter coefficient set corresponding to the phase shift, from filter coefficient data including filter coefficient sets; generating an up-sampled reference layer image by extending a resolution of the reference layer image according to the scaling factor by performing interpolation filtering on the reference layer image by using the filter coefficient set; obtaining a prediction error between the up-sampled reference layer image and the enhancement layer image; generating an enhancement layer bitstream including the prediction error and a scalable codec; and generating a base layer bitstream by encoding the base layer images.