An image processing apparatus and a control method thereof are provided. The image processing apparatus includes: an image input interface configured to receive an image content including a first reference frame and a second reference frame; a memory; and a processor configured to generate an interpolation frame based on the first and second reference frames, and the processor is configured to group a plurality of pixels included in at least one of the first reference frame and the second reference frame into a plurality of pixel groups, and individually store each of the plurality of pixel groups in the memory, and load, into an internal memory, pixel values corresponding to a predetermined range with reference to a target pixel region of the interpolation frame with respect to a first pixel group from among the plurality of pixel groups, and load, into the internal memory, pixel values of a range which is smaller than the predetermined range with respect to a second pixel group of the plurality of pixel groups, and to determine pixel values of the target pixel region.