System and method embodiments are provided for achieving improved View Synthesis Distortion (VSD) calculation and more accurate distortion estimation of encoded video frames. An embodiment method includes obtaining a depth map value for a video frame and determining a weighting factor for depth distortion in accordance with the depth map value. The weighting factor maps a pixel range of the depth map value to an output function having higher values for closer image objects and lower values for farther image objects. The VSD for the video frame is then calculated as a function of absolute horizontal texture gradients weighted by a depth distortion value and the weighting factor determined in accordance with the depth map value.