An image data of pictures constituting moving image data is encoded to generate an encoded video stream. In this case, the image data of the pictures constituting the moving image data is classified into a plurality of levels and encoded to generate a video stream having the image data of the pictures at the respective levels. Hierarchical composition is equalized between a low-level side and a high-level side, and corresponding pictures on the low-level side and the high-level side are combined into one set and are sequentially encoded. This allows a reception side to decode the encoded image data of the pictures on the low-level side and the high-level side with a smaller buffer size and a reduced decoding delay.