A deblocking filter performs filtering of decoded image data obtained by decoding image data encoded for each block, so as to remove block distortion. If at least one of block sizes on adjacent sides of two adjacent blocks is extended larger than a predetermined block size, a filter setting unit sets the tap length to an extended length to increase the strength of distortion removal, or sets a filtering object pixel range to an extended range. When a macroblock having an extended size is used, the degree of smoothing is increased, and pixels including those distant from the block boundary are subjected to filtering. Consequently, even when various block sizes are employed or when blocks of extended sizes are used, images of high image quality can be achieved.