Cross-Lingual Legal QA for Vietnamese Labour Law: Retrieval, Translation, and Verifier-Guided Correction

1 Introduction

• Cross-lingual legal QA faces both weak lexical overlap between English queries and Vietnamese statutes and answers that may misstate citations, exceptions, or shall/may distinctions.These risks motivate dense multilingual retrieval and checking answer claims against retrieved statutory evidence.Source: §1 · p. 1
• The paper frames Vietnamese cross-lingual legal QA as a retrieval, verification, and translation-faithfulness problem, evaluated with automatic diagnostics on 231 questions and human judgements on 27.The two evaluation views diverge, so the paper analyzes them separately.Source: §1 · pp. 1–2
• The contributions are a 231-pair bilingual suite with 75 hard-case annotations, a two-stage verifier-guided correction pipeline, and six retrieved-evidence diagnostics.The suite is paired with a 953-article corpus; diagnostics cover citations, modality, exceptions, procedures, conclusions, and support.Source: §1 · p. 2
• Dense retrieval performs best for direct English queries, translation placement does not measurably change automatic diagnostic scores, and procedural completeness is lowest across B2–B4.The introduction also notes that the retrieval gap makes retrieval a potentially more consequential bottleneck than translation placement in this setting.Source: §1 · p. 2

2.1 Legal QA with Retrieval-Augmented Generation

• RAG systems address legal grounding and retrieval across domains, but the cited work does not verify after generation that answers faithfully reflect retrieved statutes.Beauchemin et al. (2024) study Quebec automobile insurance law; Redelaar et al. (2024) address precondition-heavy Dutch statutes; Gokhan and Briscoe (2025) improve retrieved-context coherence.Source: §2.1 · p. 2

2.2 Multilingual and Cross-Lingual Legal NLP

• Multilingual legal NLP includes cross-lingual resources and QA systems, yet the cited systems do not incorporate dedicated verification against retrieved statutory text.Examples include Nguyen et al. (2025), Kabir et al. (2025), and Akarajaradwong et al. (2025a, 2025b).Source: §2.2 · p. 2

2.3 Vietnamese Legal NLP

• Vietnamese legal resources cover reasoning benchmarks, LLM error analysis, and legal entailment, while prior Vietnamese–English work addresses retrieval rather than end-to-end answer correction.VLegal-Bench (Dong et al., 2025), Le (2026), and ViLegalNLI (Duong et al., 2026) inform the paper’s reasoning and verifier design.Source: §2.3 · p. 2

2.4 Legal Faithfulness and Evaluation

• Prior legal-faithfulness work identifies irrelevant or fabricated citations and omissions of core elements or exceptions; this paper extends those concerns with modality and procedural dimensions.The cited error categories motivate exception and completeness dimensions, while modality and procedure are described as task-specific extensions.Source: §2.4 · p. 3

3.1 Dataset Construction

• The suite contains 231 bilingual Labour Code QA pairs drawn from VLegal-Bench and ALQAC, with gold article citations; 88 records require at least three gold articles.Questions include Vietnamese originals and English translations; a community-derived collection was excluded because citation coverage was incomplete.Source: §3.1 · p. 3

3.2 Hard-Case Annotation

• 75 questions are tagged for five hard-case phenomena involving exceptions, cross-article definitions, numeric or temporal thresholds, and synthesis of non-adjacent articles.The counts are 41 exception-conditional cases, 12 cross-article definitions, 11 numeric thresholds, 6 temporal thresholds, and 5 multi-article synthesis cases.Source: §3.2 · p. 3

3.3 Retrieval Corpus

• The retrieval corpus contains 953 articles from the Labour Code and implementing decrees, with each article indexed as a single retrieval chunk.The articles use BGE-M3 embeddings in a Qdrant vector store, preserving complete statutory units for article-level citation evaluation.Source: §3.3 · p. 3

4.1 Pipeline Overview

• The pipeline is modular, with configurable retrieval mode and evidence language across retrieve, generate, verify, and correct stages.This configuration supports cross-lingual comparisons while keeping verifier and evaluator components fixed.Source: §4.1 · p. 3

4.2 Retrieval

• For B2, GPT-4o-mini translates English questions to Vietnamese before retrieval; B3 and B4 submit English queries directly to BGE-M3 dense retrieval.Main pipeline runs retrieve top 5 articles, while the standalone retrieval ablation uses top 10; number-level citation recall is treated as an upper bound.Source: §4.2 · pp. 3–4

4.3 Generation

• Qwen3-8B generates answers from the question and retrieved statutes using a prompt that requires explicit article identifiers for factual claims.In B4, retrieved Vietnamese passages are translated into English before generation.Source: §4.3 · p. 4

4.4 Two-Stage Verification

• The verifier decomposes answers into claims, checks citation reachability and obligation or exception signals, then classifies eligible claims as supported, contradicted, or not addressed.Claims that miss exception signals pass through uncorrected because rewriting without direct supporting evidence risks another distortion; only contradicted claims proceed to correction.Source: §4.4 · p. 4

4.5 Correction

• A single-pass Qwen3-8B rewrite targets claims with unreachable citations or entailment contradictions, using the original claim, retrieved passage, and failure type.The rewrite is instructed to preserve the answer’s overall structure while making the claim faithful to the passage.Source: §4.5 · p. 4

4.6 Evaluation Metrics

• Evaluation combines retrieval metrics with six evidence-faithfulness dimensions, but automatic scores are not expert validation and conclusions about legal correctness remain provisional.Four dimensions use rule-based matchers, while conclusion consistency and support preservation use GPT-4o-mini; B1 lacks three English-calibrated subscores and is not directly comparable to B2–B4.Source: §4.6 · p. 4

5.1 Baseline Configurations

• B3/B4/C3/C4 form a 2×2 translation comparison under dense retrieval, while B1 and B2 are end-to-end references outside it.C3/C4 are sensitivity controls because their query translations were generated independently.Source: Table 1 · p. 5
• B1–B4 share a 953-article corpus, Qwen3-8B generator, two-stage verifier, and evaluator; query language, retrieval mode, and evidence language vary.B1 is monolingual Vietnamese; B2 translates the English question before retrieval, B3 retrieves directly from English, and B4 translates retrieved Vietnamese passages before generation.Source: §5.1 · pp. 4–5

5.2 Retrieval Mode Ablation

• The standalone retrieval ablation compares dense BGE-M3, hybrid RRF, and BGE-M3 sparse-only retrieval on 231 English questions against the Vietnamese corpus at k = 10.It isolates retrieval mode independently of the end-to-end pipeline and targets the retrieval step shared by B3 and B4.Source: §5.2 · p. 5

5.3 Query-Language vs. Retriever Disentanglement

• A 2×2 query-language × retriever-type matrix adds Vietnamese-query dense retrieval and EN→VI-query dense retrieval controls at k = 10.These controls separate query-language effects from retriever-type effects.Source: §5.3 · p. 5

6.1 Retrieval Performance

• B1 had the strongest retrieval, B2 tracked it closely, and direct English-query systems B3 and B4 trailed with identical retrieval metrics.Gold citations were ambiguous across instruments; the permissive matching rule over-credits, so absolute retrieval values are upper bounds, though system rankings remain unchanged.Source: §6.1 · p. 5, Table 2 · p. 6

6.2 Retrieval Mode Ablation

• Under dense retrieval, EN→VI queries achieved R@5 = 0.394 versus 0.358 for direct English queries; switching from hybrid to dense yielded no consistent gain.BGE-M3 sparse performed poorly for English-to-Vietnamese retrieval, and hybrid rank fusion did not improve over dense retrieval.Source: §6.2 · pp. 5–6

6.3 Retrieved-Evidence Diagnostics

• B4 led overall among cross-lingual baselines, but its +0.012 overall contrast with B3 was not distinguishable from zero, and other controlled contrasts were also null.B4 led on exception preservation, procedural completeness, and conclusion consistency; B3 led on citation and support preservation. B1’s overall score covers only three dimensions and is not comparable.Source: §6.3 · p. 6, Table 3 · p. 7

6.4 Verifier Behaviour

• Fewer than 27% of outputs passed both verifier stages with every claim supported; B4 had the lowest citation-failure, contradiction, and correction rates among cross-lingual baselines.B3 had a higher citation-failure rate than B4 despite identical retrieved passages, locating that difference after retrieval without identifying a specific generation mechanism.Source: §6.4 · p. 6

6.5 Hard-Case Phenomena

• The 75 tagged hard cases were harder to retrieve than the other 156 questions, with multi-article chaining the hardest phenomenon.Diagnostic scores measured groundedness in retrieved passages, not legal correctness, and tagged cases did not receive lower diagnostic scores.Source: §6.5 · p. 6, Table 4 · p. 7

6.6 Correction Effectiveness (E2)

• On the corrected subset, overall diagnostic gains came almost entirely from citation preservation; exception and procedural dimensions moved only marginally.The corrector abstained from rewriting FAIL_CONDITION and not_addressed claims, while support preservation was flat or slightly negative on English-output systems.Source: §6.6 · p. 7

6.7 Verifier-Corrector Ablation (System Level)

• Correction significantly improved citation preservation on B2–B4 and overall diagnostics on B3 and B4, but B2 showed no reliable overall gain.The system-level comparison scored uncorrected answers against corrected outputs while holding retrieval and generation fixed.Source: §6.7 · p. 7

6.8 Human Evaluation

• Human elicitation formats produced different orderings: B3 received the best criterion ratings, while B4 was preferred in holistic ranking.On the 27-question sample, inter-annotator agreement was modest and imprecisely estimated; human factual grounding was inversely associated with automatic support preservation (ρ = −0.345, p < 0.001).Source: §6.8 · pp. 7–8, Table 6 · p. 8

• R@5 is 0.358 for dense BGE-M3 versus 0.032 for learned-sparse retrieval, while hybrid RRF offers no consistent gain.Source: §7 · p. 8
• Translation placement has no statistically detectable effect on automatic diagnostics, and human judgements do not fully align with those diagnostics.Source: §7 · p. 8
• Verifier-guided correction improves citation preservation by 0.022–0.034 but yields no reliable gains in the remaining dimensions.Source: §7 · p. 8
• Procedural completeness is the lowest-scoring diagnostic, with a mean of 0.32, and changes little under correction.Source: §7 · p. 8
• Future work should target procedural grounding, validate the six diagnostics with legal experts, and expand benchmarks across legal domains and language pairs.Source: §7 · p. 8

• The 231-question benchmark covers only Vietnamese labour law, so findings should not be assumed to generalize to other domains, jurisdictions, legal systems, or language pairs.Source: Limitations · p. 8
• Gold article numbers span multiple instruments and versions, and the authors have not reviewed whether 2025 instruments changed cited rules, leaving temporal-label risk unbounded.A system could be penalized for a legally current answer or credited for an obsolete one.Source: Limitations · pp. 8–9
• Four diagnostic dimensions are rule-based and two use GPT-4o-mini, which also performs pipeline tasks; the evaluator is therefore not independent of the scored components.Source: Limitations · p. 9
• Support preservation and conclusion consistency assess answers against retrieved passages, so faithful answers based on the wrong provision can still be legally wrong.Source: Limitations · p. 9
• The retrieval conclusion covers BGE-M3 on one language pair and corpus, not untested sparse methods such as BM25 on translated documents, SPLADE, or bilingual lexical expansion.Source: Limitations · p. 9
• C3/C4 translations produced identical top-five passages for only 179 of 231 questions; their matched-subset contrast was null, so B3–B4 is the clean comparison.Source: Limitations · p. 9
• Translation contrasts did not reach significance, and intervals roughly ±0.015 wide cannot detect effects smaller than that; this is not evidence of equivalence.Source: Limitations · p. 9
• All systems use Qwen3-8B at k=5 with one sample per question, leaving sampling variance, sensitivity to k, and transfer to larger or instruction-tuned generators unmeasured.Source: Limitations · p. 9

Ethical Considerations

• The system is a research prototype, not a production legal-advice tool, and its outputs should not guide legal decisions without qualified counsel.Source: Ethical Considerations · p. 9
• The corpus includes 2019 and 2026 Labour Code versions, and unreviewed 2025 amendments could make retrieved provisions outdated or misleading without version control and legal review.Source: Ethical Considerations · p. 9
• GPT-4o-mini translations may alter the meaning of legal terms without direct English equivalents, so answers should be checked against the Vietnamese text.Source: Ethical Considerations · p. 9
• The 231-question set was curated by the authors, and human annotators were graduate students with Vietnamese legal training who consented and received standard research-assistant rates.Source: Ethical Considerations · p. 9
• Among corrected outputs, automatic diagnostic scores worsened for 14.3%–46.4% across systems, with B2 highest; legal-expert review is advised before deployment.Source: Ethical Considerations · pp. 9–10

A Reproducibility

• Qwen3-8B generates answers; BGE-M3 supplies dense and learned-sparse retrieval vectors, while GPT-4o-mini handles translation, claim splitting, NLI verification, and judging.GPT-4o-mini is also the fallback generator when the local endpoint is unavailable.Source: A Reproducibility · p. 11
• Retrieval uses Qdrant with BGE-M3 dense cosine similarity and learned-sparse vectors; hybrid mode fuses both with native RRF.Dense and sparse prefetch each use 3k, k = 5; retrieval metrics are computed at k ∈{1, 3, 5, 10}.Source: A Reproducibility · p. 11
• Evaluation reports standard retrieval metrics and six evidence diagnostics, using rule-based matchers for four dimensions and GPT-4o-mini judgments for two.Multi-MRR averages reciprocal ranks over unique citation numbers per question.Source: A Reproducibility · p. 11
• Human evaluation uses two blinded annotators to rate 27 stratified questions on completeness, factual grounding, and legal correctness, and to rank four systems.Ordinal agreement is assessed with Krippendorff’s α, with 95% confidence intervals from 10,000 question-level resamples.Source: A Reproducibility · p. 11
• Sampling and bootstrap analyses use seed 42, scores are stored to three decimals, and all systems cover all 231 questions.Separate task-specific prompts handle translation, generation, claim splitting, verification, correction, and evaluation.Source: A Reproducibility · p. 11

B Additional Results

• The representative-organization case is the correction stage’s only degenerate output: citation rewriting creates a repetition loop and drops the overall diagnostic score to 0.089.The raw answer is described as clean before correction.Source: B Additional Results · p. 11
• Table 8 presents the B4 hard-case breakdown by phenomenon; multi-article chaining is identified as the hardest retrieval phenomenon.Source: Table 8 · p. 12
• Correction gains concentrate in citation preservation; support preservation is flat or slightly negative on English-output systems, and B1 lacks modality, exception, and procedural sub-scores.Those B1 sub-scores were not computed because the matchers were calibrated for English-language outputs.Source: Table 10 · p. 12
• For q-259, the answer omits the consequence of misinformation; the verifier flags FAIL_CONDITION, but the corrector abstains by design, leaving procedural completeness at 0.1.Source: B Additional Results · p. 12
• Under the strict Labour-Code-only rule, R@5 was .396→.378 (B1), .394→.381 (B2), .358→.343 (B3), .358→.343 (B4), .394→.385 (C3), and .395→.385 (C4).On B3, 15 of 148 top-5 hits (10.1%) are credited solely through a non-Code instrument, affecting 10 of 231 questions.Source: B Additional Results · p. 12
• Per-item tables distinguish full-set changes, which include unchanged and uncorrected outputs, from corrected-subset changes that isolate verifier-triggered items.Unchanged items have |∆| ≤0.001.Source: Table 11 · p. 12

C Dataset Construction Details

• VLegal-Bench contributes 156 eligible records and ALQAC contributes 80; removing five exact duplicates yields the final 231, while VietLegal is excluded for incomplete article-level citation coverage.Records lacking a question, answer target, or mappable citation were removed before the source counts.Source: C Dataset Construction Details · p. 13
• The 953-article corpus spans 20 instrument versions, but bare-number citation normalization creates cross-instrument ambiguity; 136 records have expansions touching post-2024 instruments, and English references lack independent legal-expert validation.Hard-case labels were assigned by one rater, so inter-annotator agreement is unavailable.Source: C Dataset Construction Details · pp. 13–14