A decoding method of decoding encoded data per unit from among units included in a picture is provided. The decoding method decodes a first flag which indicates whether a removal time of the encoded data from a buffer is set per unit. The buffer is for storing the encoded data. The image decoding method further decodes a second flag indicating whether an interval between removal times of the units is constant when the removal time is set per unit. The image decoding method removes the encoded data from the buffer per unit and at an interval according to the second flag, and decodes the removed encoded data.