A system for encoding and decoding a sequence of frames of video data. The system includes encoding processing circuitry configured to encode a sequence of source video frames using other source frames as reference frames. The encoding processing circuitry is also configured, when encoding a new source frame that has a different resolution to the resolution of a reference frame, to generate a scaled copy of the reference frame for the new source frame. The encoding processing circuitry is also configured to encode the new source frame using the scaled copy of the reference frame to provide output encoded video image data. The system also includes a video decoder configured to decode the encoded scaled copy of the reference frame and decode the encoded source frame using the decoded scaled copy of the reference frame to provide output video image data for the source frame.