An image processing apparatus for encoding or decoding an image, wherein, even when it is desired to process a motion picture having, e.g., a high resolution, the capacity of a memory necessary for storing a predictive frame is reduced. In an image processing device such as an image encoding or decoding device for encoding or decoding an image, the processing device encodes or decodes the image in an intra-child-frame correlation system or in an inter-child-frame correlation system. In the system, a frame of the image to be processed is treated as a parent frame, the parent frame is divided into a plurality of child frames, and an intra-child-frame correlation or an inter-child-frame correlation is used for each child frame.