BLEU score
BLEU (bilingual evaluation understudy) is an automatic metric for machine translation quality. It scores a translation from 0 to 100 by measuring how much its wording overlaps with one or more human reference translations of the same text. Higher overlap means a higher score.
How it works
BLEU compares the machine's output against a human reference and counts matching runs of words. It rewards output that reuses the reference's exact phrasing and penalises output that strays, then folds those matches into a single number.
Because it only looks at word overlap, BLEU is fast, cheap and perfectly repeatable: the same input always produces the same score. That is also its limit. A translation can say the right thing in different words and score poorly, or match many words while mangling the meaning and score respectably.
How SourceTarget uses it
SourceTarget calculates BLEU as one input among several, never as the final verdict. It is useful for comparing engines on the same content and for tracking whether a change helped or hurt.
On its own it cannot say whether a single sentence is fit to publish, so it feeds the composite quality score alongside metrics that judge meaning, not just words, and the result routes work to the right level of human review.
BLEU score compared with COMET score
| BLEU score | COMET score | |
|---|---|---|
| What it measures | Word overlap with a reference | Predicted human judgement of meaning |
| Strength | Cheap, transparent, repeatable | Correlates far better with human ratings |
| Weakness | Blind to correct paraphrases | A model prediction, weaker in low-resource languages |
| Best used for | Comparing engines quickly | Judging adequacy and fluency |