The invention relates to a method for processing a video data set, which comprises a sequence of original images, into a modified video data set which, in addition to the sequence of original images, comprises virtual images derived therefrom in one or more data processing units, wherein an SfM analysis is carried out for each of the original images and a respectively assigned original position of a recording device which was used to record the original image, virtual positions are formed for a recording device for recording virtual images, for each virtual position of the recording device with the associated virtual initial image, a respective assignment is created to at least one of the original positions of the recording device with the associated original image, for each assignment, a homography is determined between the virtual initial image and the associated original image, and for each virtual initial image, a virtual final image is formed from the associated original image in that the original image is formed in the associated final image using the respectively determined homography.