A stereo image matching system includes an image processing unit for converting input images inputted from a first and a second image acquisition unit into digital signals to output first and second pixel data; and an image matching unit for computing at least two of an upward, a downward, a forward and a backward message of each pixel by using data values of the first and the second pixel data that are located on a same epipolar line to calculate a disparity value of each pixel by using the computed messages corresponding to adjacent pixel(s). The stereo image matching system employs a parallel pipeline VLSI configuration with a time complexity of O(N). Thus, a plurality of image lines are used for matching, so that correct distance image information is obtained regardless of the conditions of the surrounding environment.