A reference area is extracted from image data, and from the pixels included in the reference area, extracted are a plurality of small sections in which pixels are arranged so as to be symmetrical with respect to the center pixel of the reference area in the horizontal and vertical directions, and point-symmetrical with respect to the center pixel, while overlap of pixels is allowed between each of the small sections. Determination is made as to whether or not image data included in the reference area matches a predetermined pattern. A characteristic of the image data in the reference area is determined based on a number of characteristics of the plurality of small sections. Based on the results of the determination, it is judged whether or not to perform smoothing on the center pixel of the reference area. Then, the smoothing is performed on the judged center pixel.