Detecting Translation Hallucinations with Attention Misalignment

has advanced considerably because the early days of Google Translate in 2007. Nonetheless, NMT programs nonetheless hallucinate like every other mannequin — particularly in terms of low-resource domains or when translating between uncommon language pairs.

When Google Translate offers a consequence, you see solely the output textual content, not the likelihood distributions or uncertainty metrics for every phrase or sentence. Even for those who don’t want this data, information of the place the mannequin is assured and the place it isn’t may be actually worthwhile for inner functions. As an example, easy components may be fed to a quick and low cost mannequin, whereas extra assets may be allotted for tough ones.

However how can we assess and, most significantly, “calibrate” this uncertainty? The very first thing that involves thoughts is to judge the distribution of output possibilities for every token, for instance, by calculating its entropy. That is computationally easy, common throughout mannequin architectures, and, as may be seen under, really correlates with instances the place the NMT mannequin is unsure.

Nonetheless, the constraints of this strategy are apparent:

1. First, the mannequin could also be selecting between a number of synonyms and, from the token choice perspective, be unsure. 
2. Second, and extra importantly, that is only a black-box methodology that explains nothing in regards to the nature of the uncertainty. Maybe the mannequin actually hasn’t seen something comparable throughout coaching. Or maybe it merely hallucinated a non-existent phrase or a grammatical building.

Current approaches handle this downside moderately properly, however all have their nuances: 

• Semantic Entropy [1] clusters mannequin outputs by semantic that means, however requires producing 5–10 outputs for a single enter, which is computationally costly (and admittedly, once I tried to breed this on my labelled dataset, the noticed semantic similarity of phrases in these clusters was questionable). 
• Metrics like xCOMET [2] obtain SOTA-level QE on the token stage, however require fine-tuning 3.5 billion parameters of an XLM-R mannequin on costly quality-annotated information and, other than that, perform as a black field. 
• Mannequin introspection [3] by way of saliency evaluation seems to be fascinating but additionally has interpretation points.

The strategy proposed under could make uncertainty computation environment friendly. Since most NMT setups have already got two fashions — a ahead mannequin (language1 → language2) and a backward mannequin (language2 → language1) — we are able to leverage them to compute interpretable uncertainty indicators.

After producing a translation with the ahead mannequin, we are able to “place” the inverted translation-source pair into the backward mannequin utilizing instructor forcing (as if it generated it itself), then extract the transposed cross-attention map and evaluate it with the corresponding map from the ahead mannequin. The outcomes under present that this strategy permits acquiring interpretable indicators on the token stage typically.

Moreover, there is no such thing as a have to retrain a heavy NMT mannequin. It’s adequate to coach a light-weight classifier on options from the matrix comparability whereas conserving the principle mannequin’s weights frozen.

What does proper and mistaken appear like

Let’s begin with a easy French → English translation instance the place every little thing is obvious simply from the visualization.

“Elle aime manger des pommes → She likes to eat apples”

Now let’s evaluate this with a damaged NMT translation:

“La femme dont je t’ai parlé travaille à cette université” (right translation: “The girl I advised you about works at this college”)

What the mannequin produced:

“The girl whose spouse I advised you about that college”

The place did this additional “spouse” come from?

Bidirectional Cross-Verify

Computation of bidirectional consideration. Please notice that the backward mannequin makes use of instructor forcing. It receives the pre-generated English translation and checks whether or not it matches again to the unique French supply, whereas no new French sentence is generated. That is an alignment verification, not a round-trip translation.

def get_bidirectional_attention(dual_model, src_tensor, tgt_tensor):
    """Extract ahead/backward cross-attention and reciprocal map."""
    dual_model.eval()
    with torch.no_grad():
        fwd_attn, bwd_attn = dual_model.get_cross_attention(src_tensor, tgt_tensor)

    # Align to full goal/supply lengths for element-wise comparability
    B, T = tgt_tensor.form
    S = src_tensor.form[1]
    fwd_aligned = torch.zeros(B, T, S, gadget=src_tensor.gadget)
    bwd_aligned = torch.zeros(B, T, S, gadget=src_tensor.gadget)
    if T > 1:
        fwd_aligned[:, 1:T, :] = fwd_attn
    if S > 1:
        bwd_aligned[:, :, 1:S] = bwd_attn.transpose(1, 2)

    reciprocal = fwd_aligned * bwd_aligned
    return fwd_aligned, bwd_aligned, reciprocal

All reproducible code is accessible through the project’s GitHub repository.

At first of my work on the subject, I attempted utilizing direct round-trip translation. Nonetheless, because of the poor efficiency of single-GPU-trained fashions and likewise due to the interpretation ambiguity, it was tough to match the supply and round-trip consequence on the token stage, as sentences might fully lose their that means. Furthermore, evaluating the eye matrices of the again and ahead fashions for 3 totally different sentences — the supply, the interpretation, and the reproduced supply from the round-trip — would have been pricey.

When Patterns Are Much less Apparent: Chinese language → English

For language pairs with comparable construction (like French↔English), the “1-to-1 token sample” is intuitive. However what about typologically distant languages?

Chinese language → English includes: 

• Versatile phrase order. Chinese language is SVO like English, however permits topicalization and pro-drop.
• No areas between phrases. Tokenizers should phase earlier than subword splitting.
• Logographic writing system. Characters map to morphemes, not phonemes.

The eye maps develop into more durable to interpret simply by trying on the image, nonetheless the discovered options nonetheless handle to seize alignment high quality.

Let’s take a look at this instance of a semantic inversion error:

这家公司的产品质量越来越差客户都很不满意 (right translation: This firm’s product high quality is getting worse, clients are very dissatisfied)

The mannequin output:

The standard of merchandise of the corporate is more and more glad with the client.

The phrase “不满意” means “dissatisfied”, however what the mannequin had produced is precisely reverse. Not even mentioning that the entire translation result’s nonsense.

Regardless of the sample being considerably much less visually noticeable, a trainable QE classifier continues to be in a position to seize it. That is exactly why we extract 75 consideration alignment–based mostly options of varied sorts, as defined in additional element under.

Experimental Setup

The NMT core is deliberately stored undertrained for the setup. A near-perfect translator produces few errors to detect. Nonetheless, to construct a high quality estimation system, we want translations that typically (and even typically) fail, and embody omissions, hallucinations, grammatical errors, and mistranslations.

Whereas a completely rubbish mannequin would make no sense, just because there could be no reference for the classifier on what is true, the mannequin used on this setup (~0.25–0.4 BLEU over a validation a part of the dataset) ensures a gradual provide of various error sorts, thereby creating a good coaching sign for the QE.

The structure makes use of easy scaled dot-product consideration as an alternative of extra superior choices (linear consideration, GQA, and so forth.). This retains the eye weights interpretable: every weight represents the likelihood mass assigned to a supply place, with out approximations or kernel tips that will even be good to think about — however but out of the scope of this experiment. Discovering methods to enhance the strategy for extra optimized consideration construction is an efficient ahead level to go.

Knowledge and Annotation

Token-level binary high quality labels had been annotated through “LLM-as-a-judge” strategy utilizing Gemini 2.5 Flash. The annotation immediate had clear and strict guidelines:

• BAD: mistranslations, mistaken tense/kind, hallucinated content material, incorrect syntax, UNK tokens.
• OK: right that means, legitimate synonyms, pure paraphrasing.

Every translation was tokenized, and the judging mannequin created labels for each token. It additionally supplied a reference translation and a minimal post-edit. In whole, this gave approx. 150,000 labeled tokens with 15–20% “BAD” fee.

Coaching Pipeline

Step 1: Prepare bidirectional NMTs. Ahead (src→tgt) and backward (tgt→src) fashions had been skilled collectively on parallel information. Each share the identical structure however separate parameters.

class DualTransformerNMT(nn.Module):
    """Bidirectional translator used for QE function extraction."""
    def __init__(self, src_vocab_size, tgt_vocab_size, d_model,
                 n_heads, n_layers, d_ff, max_length):
        tremendous().__init__()
        self.zh2en = TransformerNMT(src_vocab_size, tgt_vocab_size,
                                    d_model, n_heads, n_layers, d_ff, max_length)
        self.en2zh = TransformerNMT(tgt_vocab_size, src_vocab_size,
                                    d_model, n_heads, n_layers, d_ff, max_length)

class QEClassifier(nn.Module):
    """Token-level BAD likelihood head."""
    def __init__(self, input_dim=75, hidden_dim=128, dropout=0.2):
        tremendous().__init__()
        self.input_projection = nn.Linear(input_dim, hidden_dim)
        self.hidden1 = nn.Linear(hidden_dim, hidden_dim)
        self.hidden2 = nn.Linear(hidden_dim, hidden_dim // 2)
        self.output = nn.Linear(hidden_dim // 2, 1)

Step 2: Generate translations. The ahead mannequin translated the QE annotation set. These translations (with their pure errors) turned the coaching information for high quality estimation.

Step 3: Extract consideration options. For every translated sentence, 75-dimensional function vectors had been extracted per token place utilizing the strategy described under.

def extract_all_features(dual_model, src_tensor, tgt_tensor, attention_extractor):
    """Extract per-token QE options utilized in coaching/inference."""
    # Bidirectional cross-attention
    fwd_attn, bwd_attn = dual_model.get_cross_attention(src_tensor, tgt_tensor)

    # 75 consideration options (25 base x context window [-1,0,+1])
    attn_features = attention_extractor.extract(
        fwd_attn, bwd_attn, src_tensor, tgt_tensor
    )[0]  # [T, 75]

    # Elective entropy function (top-k normalized output entropy)
    entropy = compute_output_entropy(dual_model.zh2en, src_tensor, tgt_tensor)[0]

    # Ultimate mixed vector utilized in ablation: [T, 76]
    options = torch.cat([attn_features, entropy.unsqueeze(-1)], dim=-1)
    return options

Step 4: Prepare QE classifier. A small MLP classifier (128 → 64 → 1) was skilled on the extracted options with frozen translator weights.

# Freeze translator weights, prepare QE head solely
dual_model.freeze_translation_models()

n_bad = max(int(y_train.sum()), 1)
n_ok = max(int(len(y_train) - y_train.sum()), 1)
pos_weight = torch.tensor([n_ok / n_bad], gadget=gadget, dtype=torch.float32)

classifier = QEClassifier(input_dim=input_dim, hidden_dim=128, dropout=0.2).to(gadget)
optimizer = torch.optim.Adam(classifier.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss(pos_weight=pos_weight)

for batch_x, batch_y in train_loader:
    optimizer.zero_grad()
    logits = classifier(batch_x.unsqueeze(1)).squeeze(1)
    loss = criterion(logits, batch_y)
    loss.backward()
    optimizer.step()

Alignment function sorts

1. Focus (12 options) — The place is the mannequin trying?

def extract_focus_features(fwd_attn, bwd_attn, tgt_pos, src_content_mask):
    """
    Extract top-k alignment scores and their backward counterparts.

    Args:
        fwd_attn: [S] ahead consideration from goal place to all sources
        bwd_attn: [S, T] backward consideration matrix
        tgt_pos: present goal place index
        src_content_mask: [S] boolean masks for content material (non-special) tokens

    Returns:
        options: [12] focus function vector
    """
    # Masks to solely take into account content material supply tokens
    fwd_scores = fwd_attn * src_content_mask

    # Get top-3 attended supply positions
    top_k = 3
    top_fwd_scores, top_src_indices = torch.topk(fwd_scores, top_k)

    options = torch.zeros(12)
    options[0:3] = top_fwd_scores  # Ahead top-1, top-2, top-3

    # For every high supply, test backward alignment energy
    for i, src_idx in enumerate(top_src_indices):
        bwd_from_src = bwd_attn[src_idx, :]  # [T] - how supply seems to be again
        top_bwd_scores, _ = torch.topk(bwd_from_src, 3)
        options[3 + i*3 : 3 + (i+1)*3] = top_bwd_scores

    return options

Hallucinated tokens typically (but not all the time) have subtle consideration. The mannequin fails to floor the reality between the supply and the goal and tries to “look” in all places. A assured translation typically focuses sharply on 1–2 supply positions or not less than has a definite sample.

2. Reciprocity (2 options) — Does the alignment cycle again?

def extract_reciprocity_features(fwd_attn, bwd_attn, tgt_pos, src_content_mask):
    """
    Verify if consideration alignment types a closed cycle.

    Returns:
        hard_reciprocal: 1.0 if actual match, 0.0 in any other case
        soft_reciprocal: dot product overlap (steady measure)
    """
    # Ahead: discover greatest supply place for this goal
    fwd_scores = fwd_attn * src_content_mask
    best_src = fwd_scores.argmax()

    # Backward: does that supply level again to us?
    bwd_from_best_src = bwd_attn[best_src, :]  # [T]
    best_tgt_from_src = bwd_from_best_src.argmax()

    # Laborious reciprocity: actual place match
    hard_reciprocal = 1.0 if (best_tgt_from_src == tgt_pos) else 0.0

    # Mushy reciprocity: consideration distribution overlap
    # Excessive worth = ahead and backward "agree" on alignment
    fwd_normalized = fwd_scores / (fwd_scores.sum() + 1e-9)
    bwd_normalized = bwd_attn[:, tgt_pos] / (bwd_attn[:, tgt_pos].sum() + 1e-9)
    soft_reciprocal = (fwd_normalized * bwd_normalized).sum()

    return hard_reciprocal, soft_reciprocal

For instance, if “spouse” (from the instance above) attends to place 3 in French, however place 3 doesn’t attend again to “spouse,” the alignment is spurious.

3. Sink (11 options)

When unsure, transformers typically dump consideration onto “secure” particular tokens (SOS, EOS, PAD):

def extract_sink_features(fwd_attn, bwd_attn, src_tensor, tgt_tensor,
                          SOS=1, EOS=2, PAD=0):
    """
    Extract consideration sink options - consideration mass on particular tokens.
    """
    # Determine particular token positions in supply
    src_is_sos = (src_tensor == SOS).float()
    src_is_eos = (src_tensor == EOS).float()
    src_is_pad = (src_tensor == PAD).float()

    # Measure consideration mass going to every particular token kind
    sink_sos = (fwd_attn * src_is_sos).sum()  # Consideration to SOS
    sink_eos = (fwd_attn * src_is_eos).sum()  # Consideration to EOS
    sink_pad = (fwd_attn * src_is_pad).sum()  # Consideration to PAD
    sink_total = sink_sos + sink_eos + sink_pad

    # Backward sink: test if best-aligned supply additionally exhibits uncertainty
    best_src = fwd_attn.argmax()
    bwd_from_best = bwd_attn[best_src, :]
    tgt_is_special = ((tgt_tensor == SOS) | (tgt_tensor == EOS) |
                      (tgt_tensor == PAD)).float()
    bwd_sink = (bwd_from_best * tgt_is_special).sum()

    # Asymmetry: disagreement in uncertainty ranges
    sink_asymmetry = abs(sink_total - bwd_sink)

    # Prolonged options: entropy-based measures
    content_mask = 1.0 - src_is_sos - src_is_eos - src_is_pad
    fwd_content = fwd_attn * content_mask
    fwd_content_norm = fwd_content / (fwd_content.sum() + 1e-9)

    max_content = fwd_content.max()  # Peak consideration to content material
    focus = max_content / (fwd_content.sum() + 1e-9)  # How peaked?

    return {
        'sink_total': sink_total,
        'sink_sos': sink_sos,
        'sink_eos': sink_eos,
        'sink_pad': sink_pad,
        'bwd_sink': bwd_sink,
        'sink_asymmetry': sink_asymmetry,
        'max_content': max_content,
        'focus': focus,
        # ... plus entropy options
    }

Why Context Issues

Translation errors create a cascade, the place a dropped phrase impacts neighbors. By together with options from positions t-1 and t+1, we enable the classifier to detect these ripple patterns. That is clearly not the overwhelming sign (particularly when the actual semantic “neighbor” may very well be positioned distantly within the sentence), however it’s already sturdy sufficient to deliver the worth. Combining it with “topological” token-linking strategies might make these options much more significant.

Token-Stage Comparability: What Every Sign Sees

Now let’s take a look at a extra detailed breakdown – if attention-alignment may very well be matched along with output distribution entropy scores. Are they carrying the identical data or might doubtlessly increase one another?

“Entropy” column under is the normalized top-k output entropy values (ok=20) of the ahead mannequin’s softmax distribution, leading to 0–1 scale. A worth close to 0 means the mannequin is assured in a single token; a worth close to 1 means likelihood is unfold evenly throughout different candidates.

French: “spouse” hallucination

Supply: La femme dont je t’ai parlé… → MT: “the girl whose spouse i advised you about…”

Chinese language: “glad” semantic inversion

Supply: 这家公司的产品质量越来越差客户都很不满意 → MT: “the standard of merchandise of the corporate is more and more glad with the client”

Repetition Errors: How Consideration Catches Poor Mannequin Artifacts

Supply: Il est évident que cette approche ne fonctionne pas correctement (It’s apparent that this strategy doesn’t work correctly)

MT: “it’s apparent that this strategy doesn’t work correctly function correctly”

What “Assured Translation” Appears to be like Like

Supply: Le rapport mentionne uniquement trois incidents, pas quatre 

MT: “the report mentions solely three incidents, not 4”

Scaling up

Now let’s discover how the strategy works at scale. For this, I did a fast ablation test, which had the purpose to look at the affect of attention-based options. Might it’s that they carry nothing worthy of extra calculations, in contrast with a easy output entropy? 

Methodology: the dataset was break up on the sentence stage into 70% coaching, 15% validation, and 15% take a look at units. The very best epoch was chosen utilizing validation ROC-AUC for threshold independence, and the classification threshold was tuned on validation F1(BAD). The ultimate metrics had been reported on the held-out take a look at set solely.

Function Contributions

Prolonged Metrics for Mixed options (entropy+consideration)

When mixed, options work higher than every sign alone throughout each language pairs, probably as a result of they seize complementary error sorts.

Ultimate ideas

Might the identical strategy be used for duties past translation? Entropy captures “The mannequin doesn’t know what to generate,” whereas consideration captures “The mannequin isn’t grounded within the enter.” For RAG programs, this implies combining perplexity-based detection along with consideration evaluation over retrieved paperwork. For summarization — constructing grounded hyperlinks between supply textual content tokens and people within the abstract. 

Limitations

Computation value. Working the backward mannequin primarily will increase inference time.

It’s a glassbox mannequin solely. You want entry to consideration weights and, due to this fact, it received’t work on API-based fashions. Nonetheless, when you have entry — you received’t have to change the core mannequin’s weights. You’ll be able to plug in any pretrained encoder-decoder, freeze it, and prepare solely the QE head.

Uncertainty doesn’t imply errors. It simply signifies that the mannequin is not sure. The mannequin typically flags right paraphrases as errors as a result of consideration patterns differ from these which the mannequin met throughout coaching — much like how a human translator may be not sure if they’d by no means encountered something prefer it earlier than.

Attempt It Your self

All code, fashions, and the annotated dataset are open supply:

# Clone the repository
git clone https://github.com/algapchenko/nmt-quality-estimation
cd nmt-quality-estimation
pip set up -r necessities.txt

# Run interactive demo (downloads fashions mechanically)
python inference/demo.py --lang zh-en --interactive
# Or use programmatically:
from inference.demo import QEDemo

# Initialize (downloads fashions from HuggingFace mechanically)
demo = QEDemo(lang_pair='zh-en')

# Translate with high quality estimation
consequence = demo.translate_with_qe("她喜欢吃苹果")

print(consequence['translation'])  # "she likes consuming apples"
print(consequence['tokens'])       # ['she', 'likes', 'eating', 'apples']
print(consequence['probs'])        # (P(BAD) per token)
print(consequence['tags'])

# Highlighted output
print(demo.format_output(consequence))
# Translation: she likes consuming apples
# QE: she likes consuming apples
# BAD tokens: 0/4

Sources

References

[1] S. Farquhar, J. Kossen, L. Kuhn, Y. Gal, Detecting Hallucinations in Giant Language Fashions Utilizing Semantic Entropy (2024), https://www.nature.com/articles/s41586-024-07421-0 

[2] COMET, GitHub repository, https://github.com/Unbabel/COMET 

[3] W. Xu, S. Agrawal, E. Briakou, M. J. Martindale, M. Carpuat, Understanding and Detecting Hallucinations in Neural Machine Translation through Mannequin Introspection (2023), https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00563/