System and methods are provided for evaluating quality of a video sequence. For example, a video sequence including one or more image frames is received, the video sequence being associated with a video processing system. A frame quality factor for each image frame in the video sequence is generated based on a comparison between the image frame and a reference image frame. A fluctuation value and a weight value of the image frames are generated, the fluctuation value of the image frames indicating a variation of the frame quality factors of the image frames, the weight value of the image frames being determined based on an average of the frame quality factors of the image frames. A video quality factor is output based on the frame quality factors of the image frames, the fluctuation value of the image frames, and the weight value of the image frames.