PaperlayerView exact revision on arXiv

Source-linked AI summary

TermJudge: A Document-Level Metric Judging, Not Counting, Terminology in Machine Translation Evaluation

Nicolas Dahan, Fran{\cc}ois Yvon, Rachel Bawden

arXiv:2609.35017v1cs.AI

TL;DR

Terminology metrics can conflate errors with valid variation by penalising divergence from a fixed reference. TERMJUDGE judges divergences in document context and aggregates occurrence-level verdicts. It ranks first in the reported meta-evaluation, and glossary-guided prompting improves term translation on two NLP test corpora.

  • Problem

    Existing terminology metrics penalise divergences from fixed references, conflating translation errors with valid terminological variation.

    Source: §1 · p. 1

  • Method

    TERMJUDGE assigns each term occurrence an interpretable verdict, settling glossary-conforming forms deterministically and assessing divergences through error and document-consistency judgments.

    Source: §3 · p. 3

  • Results

    TERMJUDGE ranks first in SPA under all three aggregations and in segment-level acc∗ eq on BioMQM-Terms, while glossary-guided prompting improves term translation on two NLP test corpora.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The paper’s evaluations distinguish judging whether divergences are errors from simply counting departures from expected forms.

    Source: §1 · p. 1, §6 · p. 10

    The metric requires a domain glossary and upstream detection, labelling, and alignment steps whose errors can propagate to verdicts.

    Source: Limitations · p. 11

Abstract

from arXiv · show

Existing automatic metrics for evaluating terminological use in machine translation (MT) penalise any divergence from a fixed reference, conflating translation errors with the valid terminological variation that human translators routinely produce. We introduce TermJudge, a document-level terminology metric that assigns an interpretable verdict to every term occurrence: glossary-conforming occurrences are settled deterministically, while divergences are assessed under a two-step LLM-as-judge procedure using the full document context: the first detects and labels terminology errors; the second sorts valid document-level variations from inconsistencies. Validated against expert error annotations and document-level human MQM scores, TermJudge ranks first in both system- and segment-level meta-evaluation, ahead of glossary-conformity and quality-estimation baselines. When applied to eight systems translating academic documents, under two prompting conditions, we observe that glossary injection improves terminology translation in all paired comparisons, by removing genuine errors rather than valid variation. TermJudge is released as open-source code.

1 Introduction

Existing terminology metrics can treat any departure from an expected form as an error, even when it is a valid translation choice. TERMJUDGE judges divergences in context and reports interpretable verdicts, with supporting resources and empirical evaluations.

1 Introduction

  • TERMJUDGE distinguishes translation errors from valid terminological variation instead of treating every divergence from an expected form as a failure.It assigns interpretable verdicts to occurrences and provides an aggregate score.Source: §1 · pp. 1–2
  • The paper introduces BioMQM-Terms, a resource with a bilingual glossary of 729 English–French biomedical concepts and 13,200 aligned term–translation pairs for ten MT systems.The resource is derived from bio-MQM and released with the code.Source: §1 · p. 2
  • TERMJUDGE ranks first under every aggregation in document-level meta-evaluation on BioMQM-Terms, ahead of ten alternative metrics.The evaluations also include agreement with fine-grained human error annotations on a geoscience corpus and glossary-guided prompting on two NLP test corpora.Source: §1 · p. 2

2 Related Work

Related work evaluates terminology through matching expected forms or measuring consistency, while general MT metrics have limitations for specialised, document-level terminology. Research on specialised discourse documents valid variation that these approaches do not distinguish from errors.

2 Related Work

  • Fine-tuned general MT metrics can lose correlation with human judgments on unseen specialised domains and reduce translation quality to a single scalar.Their document-level application remains questionable (Vernikos et al., 2022; Deutsch et al., 2023b; Dahan et al., 2026a).Source: §2 · p. 2
  • LLM-as-a-judge methods offer flexible qualitative judgments, but documented biases and instability make validation against human judgments important.Existing approaches annotate general error categories without tracking consistency of a concept across a document (Zheng et al., 2023; Mathur et al., 2020; Deutsch et al., 2023a).Source: §2 · p. 2
  • Terminology metrics match output against reference terms or glossary entries, but unlisted reductions, acronyms, and lexical variants can count as failures regardless of contextual adequacy.Term hit rate and glossary-based matching approaches are described by Farajian et al. (2018), Haque et al. (2023), ibn Alam et al. (2021), and Alam et al. (2021).Source: §2 · p. 2
  • Consistency metrics reward reuse of one translation across a document, an assumption that is blind to motivated alternations and warranted chiefly for prescriptive terminologies.These approaches can be cheap and reference-free, but do not distinguish undue changes from motivated variation.Source: §2 · pp. 2–3
  • Specialised discourse research describes terminological variation for discourse and stylistic reasons, but existing metrics do not judge whether a divergence is such a variation or an error.Variation can include graphical, morphosyntactic, reduction, expansion, and lexical forms (Cornejo Cárcamo et al., 2025).Source: §2 · p. 3

3 The TERMJUDGE Metric

TERMJUDGE evaluates term occurrences against references and judges divergences to produce document-level verdicts and scores. Its pipeline uses aligned document data, separates error and consistency judgments, and specifies a configuration for running the metric.

3 The TERMJUDGE Metric

  • TERMJUDGE assigns a verdict to each term occurrence and aggregates verdicts into a document-level score; glossary matches are settled without an LLM call, while divergences proceed through error and consistency judgments.The two judgments distinguish translation correctness from whether an error-free variation is justified by its document context.Source: §3 · p. 3, Figure 1 · p. 3

3.1 Data Requirements and Preprocessing

  • The metric requires a domain glossary and translated documents with aligned source and target segments, but no human reference translation.Preprocessing identifies glossary concepts and their occurrences, labels source-form variation, and aligns occurrences to target spans; lemma matching alone would miss about 15% of occurrences detected by CONCORDANCER on PARANLP and IWSLT23.Source: §3.1 · pp. 3–4

3.2 Evaluation Pipeline

  • For each source form, TERMJUDGE selects a reference, detects divergences, judges translation errors with Q1, then sends error-free divergences to Q2 for a consistency judgment.A glossary translation is used when available; otherwise the system’s most frequent translation is validated, with an LLM-generated reference used if it is judged erroneous.Source: §3.2 · pp. 4–5
  • Q1 judgments are propagated across occurrences of a distinct divergent translation, while Q2 is judged per occurrence; the document score is the mean penalty across scored occurrences, with lower scores better.Propagation and the matching shortcut reduce LLM calls by 2.55 on IWSLT23 while leaving system rankings nearly unchanged, with 25 of 28 pairwise orderings preserved.Source: §3.2 · p. 5

3.3 Metric Configuration

  • The experiments use gpt-4.1-mini as the judge, chosen for low cost and because it belongs to a different model family from the evaluated systems.The prompts, context, references, and decoding settings are specified in the paper’s configuration.Source: §3.3 · p. 5

4 Experimental Setup

The experiments validate TERMJUDGE on occurrence-level terminology errors, document-level human quality judgments, and contemporary MT outputs. The setup spans geoscience, biomedical, and NLP corpora, with distinct annotation and comparison procedures.

4.1 Datasets

  • The evaluation uses STEP for occurrence-level validation, bio-MQM for document-level human-score validation, and PARANLP and IWSLT23 to rank contemporary MT systems.The corpora are paired with terminology resources; all are English–French.Source: §4.1 · p. 5

4.3 Document-Level Meta-Evaluation with BioMQM-Terms

  • BioMQM-Terms adds a 729-concept glossary and 13,200 aligned term–translation pairs to bio-MQM’s 50 documents, enabling terminology-focused comparison with document-level human scores.Human scores aggregate MQM error severity over aligned term spans; TERMJUDGE scores can be micro- or macro-averaged over occurrences, source forms, or concepts.Source: §4.1 · p. 6, §4.3 · p. 7

4.2 Occurrence-Level Validation on STEP

  • STEP’s 3,373 scored occurrences are mapped to 15 shared classes, with multi-label human annotations reduced using a fixed severity-based precedence.UNJUSTIFIED maps to inconsistency, while CONFORMING and JUSTIFIED map to no-error.Source: §4.2 · pp. 6–7

4.4 System Ranking on the Test Corpora

  • Eight MT outputs per corpus cover four open-weight LLMs under baseline and glossary-injected prompting; Table 3 scopes STEP agreement to 3,373 exhaustively judged occurrences.The Table 3 caption identifies 1,019 errors, including 371 with a family-A label, and 2,354 non-errors; the supplied passages do not provide agreement values.Source: §4.4 · pp. 7–8, Table 3 · p. 8

5 Results and Analyses

TERMJUDGE ranks first in document-level meta-evaluation, and glossary prompting improves terminology outcomes across the tested systems and corpora. Occurrence-level analysis shows strong error detection but weaker fine-grained classification, while the ablation attributes the metric’s advantage to its typed judges.

5.1 Occurrence-Level Agreement on STEP

  • On STEP, TERMJUDGE reaches binary balanced accuracy 0.728, but its 15-class macro-F1 is 0.196 as content-altering errors are often detected with the wrong label.Invented calques are a weak point: 40 of 65 are accepted as correct.Source: §5.1 · p. 8
  • Hapax binary accuracy is 0.664 versus 0.787 for non-hapax occurrences, while human inconsistency labels and TERMJUDGE’s UNJUSTIFIED verdicts capture different cases.The STEP human guidelines record variation without judging whether it is justified; TERMJUDGE penalises variation only when its consistency judge finds it unjustified.Source: §5.1 · p. 8

5.2 Document-Level Meta-Evaluation

  • TERMJUDGE ranks first in SPA (0.844) and segment-level acc∗ eq (0.606), significantly leading every metric on segment-level acc∗ eq.On SPA, its advantage is significant over eight of ten metrics, with ties for GEMBAdiv and CometKiwidiv.Source: §5.2 · p. 8, Table 4 · p. 9
  • Excluding hapax chains removes the SPA difference between TERMJUDGE and other metrics, though it remains significantly ahead of general QE on segment-level acc∗ eq.The reported SPA differences in this reduced variant are not statistically significant (p ≥0.17).Source: §5.2 · p. 9

The comparison metrics as an ablation of TERMJUDGE

  • In the ablation, document-level divergence scoring with generic QE reaches SPA 0.769–0.803; replacing it with typed Q1/Q2 judges raises SPA to 0.844 and adds interpretable labels.Restricting general QE to terminology spans changes little or can hurt (SPA 0.775 permissive, 0.689 strict), while glossary-conformity scoring yields 0.723–0.730.Source: The comparison metrics as an ablation… · p. 9

5.3 System Ranking on PARANLP and IWSLT23

  • For all four models on both corpora, base+terms scores better than baseline; wrong-equivalent errors fall from 11.7% to 8.4% on PARANLP and 18.6% to 9.3% on IWSLT23.Glossary prompting also reduces justified and unjustified variation moderately; the suppressed variation is mostly not valid.Source: §5.3 · p. 9, Table 6 · p. 10

The comparison metrics as an ablation of TERMJUDGE

  • About four divergences out of five follow established document usage, while glossary prompting reduces synonym inconsistency but slightly raises neutralisation.The example judgements distinguish document usage, first mention, source-distinction erasure, purposeless synonym switching, and wrong terms; the pattern holds on IWSLT23, where small counts warrant caution.Source: The comparison metrics as an ablation… · p. 10

6 Conclusion

TERMJUDGE distinguishes terminology errors from valid variation by judging document-context divergences rather than simply counting them. It performs strongly in the reported validations and finds that glossary injection improves terminology translation on two test corpora.

6 Conclusion

  • TERMJUDGE settles expected-translation matches deterministically and judges divergences in document context, producing interpretable verdicts aggregated into a document-level score.Hapax occurrences go through error detection and, if error-free, are labelled conforming without the consistency stage.Source: §6 · p. 10
  • On STEP, error detection reaches 0.728 binary balanced accuracy; on BioMQM-Terms, TERMJUDGE ranks first across three SPA aggregations and segment-level acc∗ eq.It is significantly ahead of every comparison metric at the segment level.Source: §6 · p. 10
  • On two NLP test corpora, glossary injection improves terminology translation for every model by removing genuine errors rather than legitimate variation.The pipeline can extend to other domains given a glossary and aligned occurrences, and to other language pairs by adapting its prompts.Source: §6 · p. 10

Limitations

TERMJUDGE's validity depends on its judge model, preprocessing, and evaluation scope. Fine-grained labels, some occurrence types, and cross-corpus comparisons have specific limitations.

Limitations

  • Verdicts depend on one proprietary judge model, and stability across judge models and prompt paraphrases was not measured; pseudo-reference circularity is mitigated but not eliminated.No open-weight judge was tested.Source: Limitations · p. 11
  • Out-of-glossary hapaxes remain the hardest case, with binary agreement about 12 points below non-hapax occurrences; Step 2 may conflate distinct words when ignoring diacritics and hyphens.The passage gives élevé and élève as an example.Source: Limitations · p. 11
  • Validation supports binary error detection and aggregated document scores, but fine-grained per-code agreement is substantially lower at 15-class macro-F1 0.196.The judge identifies wrong translations more reliably than their mechanism, so fine-grained codes are diagnostic and indicative.Source: Limitations · p. 11
  • Occurrence-level validation uses consensus labels from two annotators and one MT system, while document-level validation covers EN–FR and ten bio-MQM systems.Annotator bias cannot be separated from metric error, and judge accuracy may differ across language pairs and domains.Source: Limitations · p. 11
  • TERMJUDGE requires a domain glossary and upstream detection, variation-labelling, and alignment whose errors propagate; scores are relative to the supplied terminology, and full-corpus evaluation incurs non-trivial LLM cost.Term detection and LLM alignment were checked on PARANLP only; excluding the least reliable detections and alignments left rankings unchanged.Source: Limitations · p. 11

A Pipeline Overview and a Worked Example

The pipeline preprocesses documents into aligned term occurrences before TERMJUDGE selects references and judges divergences. Examples illustrate variation labelling, consistency judgments, and manual checks of detection and alignment.

A Pipeline Overview and a Worked Example

  • The pipeline runs upstream preprocessing to produce aligned term occurrences; TERMJUDGE proper begins at reference selection, and STEP supplies gold occurrences and alignments.The three other corpora use source-target segments aligned by construction.Source: A Pipeline Overview and a Worked Example · p. 14
  • Every occurrence is aligned by gpt-4.1-mini, which copies the term translation from its target segment; spans and glossary terms are lemmatised for Step 2 matching.Applying one aligner to every occurrence avoids span differences caused by assigning occurrences to different alignment components.Source: A Pipeline Overview and a Worked Example · p. 15
  • Attested source forms are labelled relative to their concept's head term using categories such as graphical, morphosyntactic, reduction, expansion, lexical, or combined variation.An earlier version of the labelling received 84% category agreement; combined variants were under-detected but represented under 2% of occurrences.Source: A Pipeline Overview and a Worked Example · p. 15
  • Manual PARANLP checks found term-detection precision of 91.5%, recall of 95.5%, and correct alignment for 78 of 80 sampled occurrences.The alignment result was equal across the two prompting conditions.Source: A Pipeline Overview and a Worked Example · p. 15
  • Table 8 illustrates source-side variation categories relative to a concept's head term, while Table 9 shows consistency judgments distinguishing legitimate from unjustified rendering of a source variation.In Table 9, rendering slab like downgoing slab is justified at first mention and while following document usage, but unjustified once the variant has a distinct translation; a paraphrase for subduction initiation is labelled an A2 error.Source: Table 8 · p. 15, Table 9 · p. 15

B Manual Validation of the Preprocessing

The preprocessing validation samples term detection and alignment on PARANLP, using a blinded manual assessment of preprocessing rather than translation quality. It identifies errors in both stages and tests whether excluding suspect occurrences changes system comparisons.

B Manual Validation of the Preprocessing

  • One author assessed preprocessing items without knowing the system or prompting condition; the questions concerned preprocessing, not translation quality.Items were sampled from 11,122 scored occurrences in 34 documents, shared across eight outputs because source-side detection is system-independent.Source: B Manual Validation of the Preprocessing · p. 15
  • The manual checks separately assessed whether detected forms referred to the linked concept, which glossary-concept occurrences were missed, and whether aligned spans matched the system's term translation.The detection-precision sample over-represented non-glossary variants, so results were weighted by detection-type share in the corpus.Source: B Manual Validation of the Preprocessing · pp. 15–16
  • Detection errors concentrated in non-glossary variants, while three missed occurrences were metric, pattern, and DBN5; other missed domain terms were absent from the glossary.Some false detections linked ordinary words such as “Table” or “split” to technical concepts.Source: B Manual Validation of the Preprocessing · p. 16
  • Among spans empty or absent from the target, three of four alignment errors assigned translations to words the system had not translated.The remaining error was a defective copy of the translation, analogiede for analogie.Source: B Manual Validation of the Preprocessing · p. 16
  • Removing non-glossary variants or occurrences with empty or absent spans left the four-model rankings within each prompting condition unchanged, with base+terms still scoring better for all four models.The excluded non-glossary variants represented 21.3% of occurrences.Source: B Manual Validation of the Preprocessing · p. 17

C Dataset Construction Details

The BioMQM-Terms glossary and STEP corpus were assembled through complementary manual passes, then linked to concept and source-form counts.

C Dataset Construction Details

D A Typology of Terminological Errors

The typology distinguishes term-level translation errors from document-level consistency patterns, and TERMJUDGE adapts it for judging. A controlled STEP comparison supports a compact label set, while the appendix reports aggregation and penalty-scale findings.

D A Typology of Terminological Errors

  • The 23 fine-grained term-error codes span seven families, while family B captures document-level consistency, including individually correct translation variation.Families B1–B4 describe consistency patterns among individually correct translations; B5–B7 combine inconsistency with incorrectness.Source: D A Typology of Terminological Errors · p. 17

D.1 Why the Compact Typology

  • The compact typology keeps family A fine-grained, groups families C–H, and delegates family B to Q2 to avoid counting document inconsistency twice.The grouped families capture common error families whose individual codes often have low support.Source: D A Typology of Terminological Errors · p. 17, §D.1 · p. 21, Table 13 · p. 21
  • 2.8 points more accuracy, 5.3 points more error precision, 78 fewer false positives, and 17 fewer false negatives favored the compact typology over the full version on STEP.The controlled comparison changed only the typology exposed to Q1; the binary gold was the same.Source: Table 14 · p. 21, §D.1 · p. 21

E STEP Confusion Matrix

F Macro Aggregation Levels

G Penalty Scale and Sensitivity

  • The score maps gravities 1/2/3 to weights 1/3/4; grouped-family errors receive the most lenient sub-code penalty as a uniform benefit-of-the-doubt rule.The rule bounds penalties by what the grouped verdict establishes.Source: G Penalty Scale and Sensitivity · p. 22
  • Raising family F’s weight reduced mean SPA from 0.833 to 0.774, while the adopted minimum-weight region kept all 64 configurations above comparison metrics.Re-scoring saved verdicts across the weight grid required no additional LLM calls.Source: G Penalty Scale and Sensitivity · p. 22

H Meta-Evaluation without Hapax

Without single-occurrence chains, the apparent ranking changes are not statistically significant, while TERMJUDGE retains advantages on two reported comparisons.

H Meta-Evaluation without Hapax

  • All system-pair SPA differences involving TERMJUDGE were within statistical noise (p ≥ 0.17); its apparent reordering ahead of glossary-conformity baselines was not significant.The reduced analysis excludes single-occurrence chains and covers 48 documents and 275 segments.Source: H Meta-Evaluation without Hapax · p. 22
  • TERMJUDGE remained significantly better than five sentence-level rows on segment-level acc∗ eq and had the best document-level acc∗ eq among terminology-aware families.Source: H Meta-Evaluation without Hapax · p. 22

I Exhaustive Judging Path on IWSLT23

On eight IWSLT23 outputs, exhaustive judging reproduced the default ranking at higher LLM-call cost. The appendix also explains how consistency labels distinguish coherent variation from inconsistency.

I Exhaustive Judging Path on IWSLT23

J Consistency Labels: Grounding and Distribution

  • The consistency labels treat first mention, explicitation, register, facet, and document usage as motivations for coherent variation, while unjustified switches and neutralisation mark inconsistency.Synonym merge also counts as coherent when source forms differ only by syntactic reordering.Source: J Consistency Labels: Grounding and… · pp. 23–24

K Judgement Counts and Cost

This section describes the prompts and procedures for generating French terminology references, extracting translated terms, and labelling head-term variants, including document-level consistency decisions and combined change-type codes. It also quantifies the cost of LLM judgements and notes that reference forms follow established usage rather than being mechanically derived from variation categories.

K Judgement Counts and Cost

  • About 12,000 judgements ($6) covered BioMQM-Terms, 6,500 ($7) the eight IWSLT23 outputs, and 46,000 ($50) the eight PARANLP outputs at gpt-4.1-mini list price.Deterministic shortcuts, Q1 propagation, and caching reduce calls; STEP’s exhaustive validation used about 3,700 judgements for 3,377 occurrences.Source: K Judgement Counts and Cost · pp. 24–25

L LLM Prompts

  • Prompts inject the corpus domain and occurrence-specific context; judges answer in JSON with justification before label and decode at temperature 0.The preceding-segment context contains five aligned segment pairs.Source: L LLM Prompts · p. 26

L.2 Q2: Consistency Judgement

  • Q2 judges document consistency only after correctness is established, identifying neutralisation or unjustified synonym switches while allowing contextually justified divergences.Its ordered decision procedure distinguishes coherent choices from synonym inconsistency and gratuitous divergence.Source: §L.2 · p. 29
  • Earlier outputs from the same system establish the document’s usage for consistency judgments, but do not establish that the French is correct.Consistency is measured against the system’s own established renderings.Source: §L.2 · p. 30

L.3 Reference Selection

  • Reference selection prefers a glossary candidate unless it mismatches the document’s sense; otherwise it favors idiomatic, appropriate French, using frequency and order to break ties.A concise established term can beat a longer, more frequent paraphrase.Source: §L.3 · p. 31

L.4 Reference Generation

  • Reference generation asks for an established French term for the source form; its source-variation category guides the choice but does not constrain French variation.Established usage decides the French form.Source: §L.4 · p. 34
  • Reference generation derives a canonical French form from established usage, with rules for acronyms, scope, synonyms, unchanged forms, and cases where a form should not be invented.The output includes a justification, generatability flag, reference, and confidence.Source: §L.4 · pp. 34–35

L.5 Term Alignment

  • Preprocessing extracts each detected term’s translation from its target sentence and copies it verbatim, preserving its casing and grammatical form.The alignment prompt asks for only the translated term.Source: §L.5 · p. 36

L.6 Variation Labelling

  • Variant classification uses context examples and a typology covering graphical, reduction, expansion, lexical, and morphosyntactic variation.The typology includes subtypes such as spelling changes, constituent deletion or addition, lexical substitution, and changes in structure or inflection.Source: §L.6 · p. 38
  • The instructions treat every supplied pair as a true variation and classify the types of changes, using a combined code when two or more types occur.For combined cases, codes are joined with +, such as VG1+VE1.Source: §L.6 · p. 39
  • Acronyms count as VG1 alone only when every letter maps to a word in the head term; otherwise, the added meaning requires a combined code.For example, TDT for “topic detection” is VG1+VE1 because “tracking” is absent from the head term.Source: §L.6 · pp. 39, 41
  • When a variant preserves part of a multiword head term, the rules distinguish partial abbreviation VG5 from full acronym VG1 and use VL2/VL3 rather than VL1 for partial replacement.The examples classify “ASR system” as VG5 and replacement of “measure” in “similarity measure” as VL2.Source: §L.6 · pp. 39, 41

L.1 Q1: Error Judgement

Q1 judges whether each aligned French rendering is an attested, domain-appropriate translation of its English source term, not whether it matches one expected form. It assigns one specific error label when the rendering is defective.

L.1 Q1: Error Judgement

  • The judgment is absolute: the target must convey the source concept and be an attested, idiomatic form, but need not match a single expected form.The expected form depends on the source form; supporting context and prior renderings inform the decision but are not verdicts.Source: §L.1 · pp. 26–27
  • Wrong-equivalent labels distinguish neighbouring concepts, descriptive paraphrases, invented calques, untranslated terms, Do-Not-Translate violations, wrong components, and general words replacing terms.The typology assigns one most-specific label to the defect.Source: §L.1 · p. 26
  • Structural labels separate misparsed heads or constituent order, misread relations between constituents, and English structures left untransposed into French.A misread semantic relation differs from a correctly understood relation expressed with an untransposed compact structure.Source: §L.1 · pp. 26–27
  • Other labels cover faulty grammar or phraseology, added, omitted, or hallucinated content, and inappropriate target-language expression or register.For an omitted aligned target, use content_altered unless the French segment conveys the concept elsewhere.Source: §L.1 · p. 27
  • A correct established term, terminological synonym, standard abbreviation, accepted variant, inflection, or domain loanword is no_error; document consistency is judged separately.Every alignment is assessed, even when it matches a prior system rendering.Source: §L.1 · pp. 26–27
  • The decision procedure first establishes an attested French term, then checks concept validity before assigning the single most specific defect label.Preposition or explicitation errors are classified by whether the relation was misread, left untransposed, or only grammatically marked incorrectly.Source: §L.1 · p. 27

Related papers

Loading 2609.35017v1…