An image data processing apparatus includes a memory for storing coefficient data obtained by performing a wavelet transform on image data, a calculation unit for calculating a memory address of the memory storing the coefficient data, based on an encoding parameter of the image data and an amount of data of the coefficient data, a memory area reservation unit for reserving a memory area where the coefficient data is to be written, in accordance with the memory address calculated by the calculation unit, and a write unit for rearranging the coefficient data in the order of inverse wavelet transform and writing the order-rearranged coefficient data onto the memory area reserved by the memory area reservation unit.