A prediction direction difference calculating unit derives a prediction direction difference in intra prediction modes of a plurality of reference blocks used for an intra prediction processing of a block to be coded. A most probable mode determining unit and a coding tree selecting unit determine a most probable mode as an intra prediction mode candidate of the block to be coded based on the intra prediction modes of the reference blocks used for deriving the prediction direction difference and the prediction direction difference, and creates a coding tree assigned with a bit stream having a shorter code length than other prediction modes for the determined most probable mode. A variable length coding unit codes information for specifying an intra prediction mode of the block to be coded according to the coding tree.