A measure of visual quality of processed images relative to unprocessed images is generated in real-time. The measure of visual quality closely correlates with a human's actual perception of the processed image relative to the original image. The measure of visual quality is computed based on a measure of discrepancy (e.g., mean square errors) between the processed and unprocessed images and the variance of each image in the pixel domain or the transform domain may be determined. If the processed image is unavailable, a prediction of the processed image may be used in place of the processed image. The prediction of a processed image may involve predicting the variance values for processed image blocks. The visual quality measure may be used in a feedback loop to improve processing or encoding.