A histogram preparer reads image data scanned from an original sheet by the line, preparing a histogram of pixel numbers of respective background densities of a read line, a background density estimator estimates a background density of the read line as a density at a local maximal value in view of a highlight end of the histogram, a background removal reference value setter sets up a background removal reference value as a reference margin based on a pixel number normalized on the estimated background density, a background removal threshold determiner increments the estimated background density by the background removal reference value to set a provisional background removal threshold of the read line, and determines a background removal threshold of the read line as a weighting combination of the provisional background removal threshold and a background removal threshold of a previous line, and a background remover removes a background of the read line based on the determined background removal threshold.