A system for automating the quality evaluation of a translation. The system may include a computer having a processor and memory device operably connected to one another. A source text in a first language may be stored within the memory device. A target text comprising a translation of the source text into a second language may also be stored within the memory device. Additionally, a plurality of executables may be stored on the memory device and be configured to, when executed by the processor, independently identify a test sample comprising one or more blocks, each comprising a matched set having a source portion selected from the source text and a corresponding target portion selected from the target text.