A compressed motion compensated video sequence is decoded using reference pictures (R) and motion vectors for deriving intermediate pictures (I,B) from reference pictures. The maximum vertical extent of the motion vector corresponds to a number of lines in the image data. A picture derived from the reference picture and motion vectors is decoded once the vertical extent of the reference picture received exceeds the maximum vertical extent of a motion vector from a starting position. Further set(s) of motion vectors for deriving further picture(s) can be received and for each picture to be derived, the image data is decoded using a respective further set of motion vectors after an area of a respective reference picture has been decoded to a maximum vertical extent of a motion vector from a starting position.