The amount of encoding of hierarchically encoded data is reduced with an improvement in the accuracy of a predicted image by including information for accurate derivation of a position of a reference layer pixel corresponding to a pixel in a higher layer, which is a target layer, in the hierarchically encoded data. A hierarchical moving image decoder includes a parameter set decoding unit decoding inter-layer phase correspondence information and a predicted image generator generating a predicted image on the basis of the inter-layer phase correspondence information.