A 3D encoding apparatus encodes an input image and generates and outputs a three dimensional stream including a basic stream and an extended stream including a plurality of image groups. A 3D encoding apparatus includes a setter that sets a specific delay value for each image group based on the a first delay value as a delay value of the basic stream and a second delay value as a delay value of the three dimensional stream. The setter sets the specific delay value to a value not more than smaller one of the first delay value and the second delay value. The first virtual buffer and the second virtual buffer perform the buffer simulation for n+1th image group based on information about the specific delay value of the nth image group set by the setter in the basic stream and three dimensional stream.