An image processing apparatus calculates estimated pixel values of respective pixels of a provisional high-resolution image by interpolation on basis of pixel values in a reference frame. Interior of the reference frame is segmentalized into an edge region, a texture region or a flat region and others on a basis of pixel values of respective pixels. The respective pixels in the reference frame are set as target pixels one by one in sequence. Corresponding positions on the provisional high-resolution image are calculated, of the respective target pixels in decimal accuracy on a basis of information on the segmented regions including the target values. The estimated pixel values are modified so that differences from the pixel values of the target pixels to provisionally estimated pixel values obtained from the estimated pixel values of the provisional high-resolution image for pixels around the corresponding positions of the target pixels, becomes smaller. Modified pixel values and then obtained.