A system for using decoder information in video super resolution processing. A compressed video buffering module is used for receiving a compressed video stream and a decoder module is used for decoding the compressed video stream into an uncompressed stream and extracting motion vector information from the uncompressed stream. A video super resolution deep neural network processor module is used for processing the uncompressed stream in conjunction with the motion vector information to produce a video super resolution stream. An output buffer module is used for buffering the video super resolution stream for subsequent output.