A frame image of decoded image data in which an image stream generated by performing an orthogonal transformation and a lossy compression coding has been decoded is inputted so as to be stored in a frame memory for each block, and based on the stored frame image and a newly input frame image, a motion vector for each block is detected, and a motion vector variance map generator calculates a variance value map by calculating a variance value of the motion vectors. Based on the variance value map and a threshold, an outline domain included in the frame image is extracted, and a band limiting filter performs band limitation for each block in regard to the outline domain so as to accomplish removal of noise from the decoded image.