An image processing device includes a histogram generation unit configured to generate a density histogram on the basis of image data of an original document, a white-reference value detection unit configured to acquire as a white reference value a density value at a density distribution peak in a predetermined white side range of the density histogram, a background-removal level determination unit configured to determine a background-removal level value having the maximum density to remove a background component by using a reference table on the basis of the white reference value, and a background removal unit configured to remove the background component from the image data by using the white reference value and the background-removal level value.