A moving image decoder (1) includes an intermediate estimated prediction mode deriving section (124) for transforming a prediction mode of each neighbor partition into an intermediate prediction mode included in an intermediate prediction set which is a sum of prediction sets (PS); and an estimated prediction mode deriving section (125) for deriving an estimated prediction mode by estimating a prediction mode of a target partition based on the intermediate prediction mode of each neighbor partition which is obtained by the transform.