Source-linked AI summary
Towards English - Mizo Legal Machine Translation: Corpus Creation, Analysis, and Benchmarks
Telem Joyson Singh, Sanabam Boyshana, Sanasam Ranbir Singh, Priyankoo Sarmah
TL;DR
Mizo speakers have limited access to Indian court judgments because proceedings are conducted in English and the language has little translation support. The paper builds and analyzes a professionally translated English–Mizo legal corpus, then evaluates a fine-tuned model. The model improves over its base system on a legal test set, but still borrows English words more often than human translators.
Problem
Mizo is outside the Eighth Schedule and has little institutional translation support, while Indian court proceedings are conducted in English.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The corpus documents selective English borrowing and inconsistent Mizo diacritics, leaving diacritic normalization an open preprocessing choice.
Source: §1 · p. 1, §4.3 · p. 4The corpus is released unnormalized because it remains unclear whether Mizo models should retain or strip diacritics.
Source: §4.3 · p. 4
Abstract
from arXiv · showhide
Indian court proceedings are held in English, so speakers of languages with limited translation technology have limited access to them. We present the first English Mizo parallel dataset of Indian court judgments. It contains 13,168 sentence pairs from the Gauhati High Court and the Supreme Court of India. Professional native Mizo translators translated complete documents; sentence pairs were manually aligned, and two independent reviewers checked the results. Our study finds that English words are borrowed in a very selective way: official and legal terms are almost always kept in English, but words for everyday things like actions, situations, and results are usually expressed in native Mizo. Mizo diacritics are also used inconsistently across the corpus. When we fine tune the NLLB 200 1.3B model on our training data, it scores 27.47 BLEU on a balanced legal test set of 1,000 sentence pairs, an improvement of 7.16 BLEU over the base model. However, it still uses English words about three times more often than a human translator does.
1. Introduction
The paper addresses the lack of legal translation resources for Mizo, a language outside India’s Eighth Schedule, by creating a professionally translated and validated corpus of Indian court judgments. It analyzes selective English borrowing and Mizo orthographic variation, then benchmarks a fine-tuned translation model.
1. Introduction
- Mizo has little institutional translation support because it is outside the Eighth Schedule, while existing parallel data is poorly matched to legal writing.The paper notes that legal test data had also been unavailable to measure the domain gap.Source: §1 · p. 1
- The first English–Mizo legal corpus contains 13,168 sentence pairs from Gauhati High Court and Supreme Court judgments, translated by professionals and independently validated.Sentence alignment was manually corrected, and two annotators achieved Cohen’s κ = 0.90.Source: §1 · pp. 1–2
- English borrowing is selective: institutional and statutory names are retained almost always, while terms for people, processes, and outcomes are usually rendered in native Mizo.The paper links retention to referential rigidity, where replacing an English form could create ambiguity.Source: §1 · p. 1
- Mizo diacritics vary substantially in the corpus, leaving normalization an open preprocessing question for tokenization and system comparability.Circumflex tone marking appears in 2.70% of Mizo segments.Source: §1 · p. 1
- 27.47 BLEU on the 1,000-pair held-out legal test set, a +7.16 BLEU improvement over the original NLLB-200 1.3B model.The fine-tuned model still borrows English words about three times more often than a human translator.Source: §1 · p. 1
2. Background
Mizo’s language and writing-system properties, limited NLP coverage, and the specific demands of legal translation motivate extending legal MT resources to this non-scheduled language.
2.1. The Mizo language
- Mizo is agglutinative and tonal, and its Latin-based writing includes optional circumflex tone marks and the letter t. with a dot below.Religious texts dominate available digital Mizo, while technical and administrative writing is scarce; common-law concepts were not lexicalized as a coherent native system.Source: §2.1 · p. 2
2.2. Language technology for Mizo
- Most Indian-language NLP resources target the twenty-two scheduled languages, leaving Mizo largely outside major datasets and models.Existing Mizo NLP work and multilingual coverage mainly use general-domain or religious and crawled web text, rather than legal-domain resources.Source: §2.2 · p. 2
2.3. Legal-domain translation and corpora
- Legal judgments contain nested sentences and fixed citation patterns, where near-synonyms such as accused and convict are not interchangeable and errors can harm litigants.Indian legal NLP resources and datasets have largely stopped at scheduled languages; this corpus extends that work to Mizo.Source: §2.3 · p. 2
3. Corpus Construction
The corpus uses official English judgments from two Indian courts and combines document-level professional translation with manual sentence alignment and independent validation.
3. Corpus Construction
- The corpus draws on publicly available Gauhati High Court and Supreme Court judgments spanning civil, criminal, and other cases.The judgments were not additionally anonymized beyond their original publication.Source: §3 · p. 2
- Professional native Mizo translators with degrees in law, English, or Mizo translated complete documents, allowing context to support ambiguity resolution and terminology consistency.A second translator peer-reviewed each translated document.Source: §3 · p. 2
- Sentence pairs were generated from embedding-based candidate matches, manually corrected for complex alignments, and independently checked by two native Mizo speakers.Disagreements went to a third translator for majority resolution; inter-annotator agreement was Cohen’s κ = 0.90.Source: §3 · p. 2
4. Corpus Analysis
The corpus analysis characterizes lexical diversity, borrowing, and spelling consistency in English–Mizo judgments. It finds selective English retention for fixed legal terms alongside inconsistent Mizo diacritics and unresolved normalization choices.
4.1. Descriptive and lexical statistics
- Mizo vocabulary has 16% more types than English despite similar token counts, and its growth shows no clear plateau at 13,168 sentence pairs.Mizo also has more hapax types (35.45% versus 24.00%), while its higher Heaps exponent is 0.576 versus 0.506 for English.Source: §4.1 · pp. 2–3, Figure 2 · p. 3
4.2. Lexical borrowing and terminology retention
- 55.30% of Mizo running tokens are strings found in the English vocabulary, but retention varies sharply by legal term.Institutional labels and statutory instruments are retained at rates approaching 100%, while accused (8.5%), conviction (5.0%), judgment (23.6%), and evidence (37.1%) are mostly rendered in native Mizo.Source: §4.2 · p. 3, Table 4 · p. 4
- English forms are preserved for legally fixed references, while compositional conditions, processes, and conclusions are more often lexicalized in Mizo.The authors illustrate this distinction with native forms including thubuai, dan, hriattirna, and thuneihna.Source: §4.2 · p. 3
4.3. Mizo orthographic consistency
- Mizo-specific diacritics are inconsistently represented: circumflexes occur in 0.22% of tokens, and the t/t. contrast is largely not maintained.There are 159 types attested both with and without circumflexes; t. occurs in 282 tokens across 52 types.Source: §4.3 · p. 4, Table 6 · p. 4, Table 7 · p. 4
- Whether to retain or strip diacritics remains unresolved, so the corpus is released unnormalised for downstream users to choose a convention.The choice affects vocabulary size, segmentation cost, and comparability of published results.Source: §4.3 · p. 4
5. Experimental Setup
The experiments use a training split from the new corpus and a deliberately stratified legal benchmark, evaluating NLLB-200 both before and after fine-tuning. The setup specifies the model, implementation, and complementary evaluation metrics.
5.1. Training data
- The 11,522-pair training split is formed after removing benchmark tests and sentences with more than 90% overlap with a test sentence.It contains 245,838 English and 248,039 Mizo tokens.Source: §5.1 · p. 4, Table 8 · p. 5
5.2. Test data
- The 1,000-pair test set is stratified across legal terms, abbreviations, named entities, statutes, and general legal writing.The strata contain 230, 220, 200, 150, and 200 pairs, respectively; token-level training coverage is 96.6% for English and 96.5% for Mizo.Source: §5.2 · pp. 4–5, Figure 3 · p. 5, Figure 4 · p. 5
5.3. Model
- The study evaluates NLLB-200 1.3B off the shelf and after fine-tuning, using the fine-tuned system as NLLB-FT.NLLB-200 is selected as one of the few large-scale open-source translation models with explicit Mizo support.Source: §5.3 · p. 5
5.4. Fine-tuning and implementation
- Fine-tuning uses the 1.3-billion-parameter model for five epochs on a single NVIDIA A100 GPU, with batch size 4 and learning rate 5×10−5.The implementation uses Hugging Face Transformers, AdamW, a linear warm-up schedule, and English-to-Mizo translation.Source: §5.4 · p. 5
5.5. Metrics
- Evaluation reports BLEU, chrF++, and TER, combining n-gram precision, character-level similarity, and normalized edit effort.Lower TER indicates fewer edits needed to convert a translation to the reference.Source: §5.5 · p. 5
6. Results
Fine-tuning improves all reported overall metrics over the off-the-shelf model, while performance varies by stratum and is weakest where the model must choose Mizo words instead of copying English.
6. Results
- +7.16 BLEU, +7.83 chrF++, and −12.13 TER after legal-domain fine-tuning, compared with the off-the-shelf model.BLEU rises from 20.31 to 27.47, chrF++ from 42.56 to 50.39, and TER falls from 77.81 to 65.68.Source: §6 · p. 5
6.1. Analysis by stratum
- The analysis compares system output with the human translator’s reference choices to distinguish mistakes from acceptable differences in borrowing.It examines three critical strata using automatic extraction of terms, entities, and acronyms followed by manual inspection.Source: §6.1 · p. 5
- BLEU varies by more than 14 points across strata, reaching 34.05 for abbreviations and 32.18 for statute names, but only 19.71 for legal terms and 21.14 for general prose.Scores are highest where source text can be copied and lowest where the model must select an appropriate Mizo word.Source: §6.1 · pp. 5–6
7. Error Analysis
The main error is over-retention of English legal terms where the reference uses Mizo; named entities and acronyms are copied reliably overall, though some entity forms are systematically altered.
7. Error Analysis
- The error analysis combines automatic extraction of terms, entities, and acronyms with manual inspection of flagged segments.It compares outputs with human translator choices rather than an abstract notion of correctness.Source: §7 · p. 6
7.1. Legal terminology
- In legal-term segments, the model matches the reference in 65.2%, but over-retains English in 21.7% versus 8.3% in the opposite direction.It borrows roughly three times more often than the translator; five substitutions also change legal meaning.Source: §7.1 · p. 6
7.2. Named entities
- The model reproduces 814 of 842 reference-copied entity mentions exactly (96.7%), but 15 other forms show systematic alterations.These include singularizing plural institutional names, rewriting statute titles, and orthographic drift in low-frequency names.Source: §7.2 · p. 6
7.3. Abbreviations and acronyms
- The model copies 203 of 208 reference-retained acronym mentions exactly (97.6%).The few differences include dropped or loosely translated acronyms and a minor letter-case change.Source: §7.3 · p. 6
8. Conclusion
The paper introduces a validated English–Mizo legal corpus and reports selective borrowing alongside inconsistent diacritic use. Fine-tuning improves BLEU, but over-borrowing remains, motivating terminology-aware modeling and evaluation.
8. Conclusion
- The first English–Mizo parallel corpus in the legal domain contains 13,168 sentence pairs from Indian court judgments, translated at document level and independently validated after manual alignment.The conclusion characterizes institutional and law names as retained nearly unchanged, while descriptions of events, processes, and outcomes use native Mizo; diacritic normalization remains unsettled.Source: §8 · p. 6
- +7.16 BLEU on a carefully sampled legal test set accompanies a remaining pattern in which the fine-tuned model copies English words about three times more often than a human translator.The conclusion links remaining problems to areas where English borrowing is most likely.Source: §8 · p. 6
- The paper suggests terminology-aware objectives or retention-lexicon constrained decoding, and terminology-sensitive metrics because BLEU and chrF++ credit copying.Source: §8 · p. 6
10. Generative AI Use Disclosure
Generative AI tools were used only to edit and polish the manuscript, not to produce research content, the corpus, or experiments.
10. Generative AI Use Disclosure
- Generative AI tools were used only for manuscript language editing and polishing, not for research content, the corpus, or experiments.Source: §10 · p. 7