An image processing apparatus includes a down-converter and an image converter converting first image data into higher quality second image data. The image converter includes a prediction tap extracting unit extracting, as prediction taps, a plurality of pixels used for predicting a subject pixel of the second image data from the first image data, a class tap extracting unit extracting, as class taps, a plurality of pixels used for classifying the subject pixel from the first image data, a classification unit classifying the subject pixel based on the class taps, a coefficient output unit outputting a coefficient corresponding to a class of the subject pixel from among coefficients, determined beforehand by learning, corresponding to the plurality of classes, and a calculator determining the subject pixel by performing prediction calculations by using the coefficient corresponding to the class of the subject pixel and the prediction taps.