A translation system receives a test pair that includes a source test phrase in a first language and a target test phrase in a second language. The test pair can be evaluated by comparing its components with phrases in primary pairs. The test source phrase can be compared to a primary source phrase that is the phrase most commonly translated by the machine translation system into the test target phrase. The test target phrase can be compared to a primary target phrase that is the phrase into which the target source phrase is most often translated. If one and/or both comparisons are sufficiently dissimilar, the machine translation system can be modified by deleting the test pair, by flagging it for human review, or in other ways.