An image encoding device encodes a block-based target image. The image encoding device comprises, in a transform skip mode in which orthogonal transform processing of the target image is skipped, a motion compensation predictor configured to generate a prediction image corresponding to the target image by performing motion compensation prediction using a plurality of reference images, an evaluator configured to evaluate a degree of similarity between the plurality of reference images on a pixel-by-pixel basis, a subtractor configured to calculate prediction residuals each indicating a difference between the target image and the prediction image on a pixel-by-pixel basis, a rearranger configured to rearrange the prediction residuals based on a result of evaluation by the evaluator and an encoder configured to encode the prediction residuals rearranged by the rearranger.