A motion compensation apparatus performs motion compensation on a current block having a non-rectangular shape, using a reference image stored in a frame memory and includes: a reference block memory for storing part of the reference image; a frame-memory-transfer control unit which identifies, in the reference image, a pixel block having a rectangular shape and including a reference block having a non-rectangular shape and used for motion compensation, and transfers pixel data of the identified pixel block from the frame memory to the reference block memory and a motion compensation processing unit which generates a prediction block of the current block, using the reference block included in the pixel block stored in the reference block memory.