When performing arithmetic processing on unprocessed image data with use of a target pixel and reference pixels in its proximity, buffers of a number corresponding to the number of lines required for the arithmetic processing, which are a first buffer and a second buffer, are used as intermediate storage units. Each buffer has a capacity that is smaller than a size of a line of the unprocessed image data and equal to or larger than a size required for the arithmetic processing in a main scanning direction. In each arithmetic processing, a pixel from each line in the unprocessed image data is input one by one to a storage region at the right end of a corresponding buffer, and a pixel is read and output from each position of each buffer that is determined according to a positional relationship between a target pixel and its reference pixel. Each time arithmetic processing is performed in each line, data is shifted one pixel from the right end to the left end in the first buffer and the second buffer.