In an image processing apparatus that processes image data in units of blocks, each block having a predetermined number of pixels, a number of pixels in the block whose pixel values are different is calculated. A predetermined pattern for the block is specified in accordance with the calculated number of pixels whose pixel values are different and the locations of the different pixels, and the image data is compressed by outputting the specified pattern and the pixel values of only the number of pixels whose pixel values are different. It is then determined whether the image data is composed of a single color based on the output pattern and the pixel values.