An image formed in units of macroblocks of 16×16 pixels attached with a band area having a width of “a” pixels serving as a margin area is extracted as a motion compensation image from a reference frame and is regarded as an input image of a filtering process. “a” is a value that is determined in accordance with the number of taps of an FIR filter. A filtering process is performed using such a motion compensation image as an input image, and a prediction image of 16×16 pixels is output as an output image of the filtering process. The prediction image is added to an output image of an inverse orthogonal transformation circuit in an adder circuit, and an image as a result of the addition is used as a macroblock constituting a decoded frame.