A system is provided and includes a memory and a control module. The memory stores first and second pixel data corresponding to first and second image frames. The control module performs stereo vision matching including: accessing the first and second pixel data; determining initial disparity values that indicate differences between positions of the first and second pixel data; determining matching merit values for the first pixel data, where each of the matching merit values is indicative of a reliability level of a corresponding one of the initial disparity values; determining weights based on (i) the initial disparity values, (ii) differences between intensity values of the first and second pixel data, and (iii) the matching merit values; and refining the initial disparity values, based on the weights, to provide refined disparity values. The control module estimates a depth of a feature or an object based on the refined disparity values.