Encoding and decoding are uniformly carried out for a plurality of chroma formats. Based on a control signal for providing a chroma format type of an input moving image signal, in the case of a chroma format of 4:2:0 or 4:2:2, a first intra prediction mode deciding unit and a first intra prediction image generation unit are applied to a luminance component of the input moving image signal, and a second intra prediction mode deciding unit and a second intra prediction image generation unit are applied to a chrominance component. In the case of a chroma format of 4:4:4, the first intra prediction mode deciding unit and the first intra prediction image generation unit are applied to all color components to carry out encoding, and a variable length encoding unit multiplexes the control signal as encoding data to be applied to a moving image sequence unit on a bit stream.