An image data processing apparatus is provided and includes a variable length decoding section to decode and extract quantized DCT coefficients from an encoded signal input thereto and extract encoded information from side information added to the encoded signal. Classification adaptation processing sections use the input signal supplied from a decoding section and a creation signal supplied from a signal storage section to determine the reliability of each of motion compensating vectors of the encoded signal supplied from a coded information storage section, selects that one of the motion compensating vectors which has the highest reliability as a selected vector, and produces a creation signal based on the selection vector.