PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-lingual Biography Enrichment via Claim Extraction and Alignment

Yifei Song, Ziyang Chen, Emil Sayilov, Claire Gardent

arXiv:2608.23390v1cs.CLcs.AI

TL;DR

The paper addresses the limited coverage of long-tail figures in English Wikipedia by enriching existing English biographies with evidence from non-English biographies. It introduces claim extraction, alignment, and controlled rewriting, finding that selected claims usually offer the strongest supported-addition versus hallucination trade-off while Azerbaijani remains difficult.

  • Problem

    English Wikipedia may underspecify long-tail figures despite richer information in non-English editions, motivating evidence-based cross-lingual biography enrichment.

  • Method

    The framework extracts English claims from paired biographies, aligns them to select non-English enrichment evidence, and uses the selected claims for controlled rewriting.

  • Results

    Claim-based evidence offers the strongest supported-addition versus hallucination trade-off in most language-generator settings, without an evident writing-quality penalty.

  • Takeaways & Limitations

    Selected claims reduce long-context burden and improve interpretability for cross-lingual biography enrichment, while reliable performance still depends on robust extraction and alignment.

  • Takeaways & Limitations

    The evaluation covers limited languages and entity groups, and its human quality assessment is a small sanity check covering 15 biography pairs, one generator, and three annotators.

Abstract

from arXiv · show

English Wikipedia is often treated as the default encyclopedic source, yet non-English Wikipedia editions can contain richer locally grounded information for long-tail figures. We study cross-lingual biography enrichment: enriching an existing English biography with facts supported by a non-English biography about the same person. Focusing on women from non-English-speaking contexts, we introduce \textsc{CLAW-4L}, a benchmark consisting of 300 Wikipedia biography pairs linking an English biography with its French, Chinese or Azerbaijani counterpart, along with claim annotations and a fine-grained claim-pair relation corpus. We propose a claim-based enrichment framework that extracts English claims from both biographies, aligns them to identify enrichment evidence from the non-English biography, and rewrites the English biography using the selected claims. Our results show that non-English Wikipedia biographies provide valuable evidence for improving English biography coverage, while lower-resource settings remain challenging.

1 Introduction

The paper studies enriching underspecified English biographies with evidence from richer non-English biographies, using claim extraction and alignment to select additions before rewriting.

  • Non-English Wikipedia editions can provide richer locally relevant information than English Wikipedia for long-tail figures.
  • Translation and Claims representations produce more supported additions than Raw, while Claims substantially reduces hallucination.
  • Cross-lingual biography enrichment adds information from a non-English biography that is absent from an existing English biography.
  • The framework extracts claims into a shared English space, aligns them, and uses selected enrichment claims as structured rewriting evidence.
  • The paper introduces CLAW-4L and benchmarks for biography enrichment, claim extraction, and claim-pair relation classification.The resources cover French, Chinese, and Azerbaijani biographies paired with English biographies.

2 Related Work

Prior biography-generation work focuses on structured data or retrieved web evidence, while this paper uses human-written non-English Wikipedia biographies and fine-grained claim alignment for enrichment.

  • Earlier systems generate short biographies from infoboxes, Wikidata, or other structured data rather than enriching existing full biographies.
  • Retrieval-grounded systems generate full-length Wikipedia content from similar pages and web evidence, including biographies of women with limited evidence.
  • This work complements retrieval-based approaches by using human-written non-English Wikipedia biographies as cross-lingual encyclopedic evidence.
  • Existing claim-extraction methods often use task-specific schemas or language-specific tools, whereas recent LLM methods decompose text into atomic or verifiable claims.
  • The proposed relation space distinguishes exact, enrichment, contradictory, and irrelevant claim relations beyond binary or three-way classification.

3 Task and Method

The task rewrites an existing English biography while preserving its content and incorporating additional information supported by a paired non-English biography.

  • Given paired English and non-English biographies of the same person, the system generates an enriched English biography.
  • The generated biography preserves the original English content while adding information supported by the non-English biography.
  • The rewriting input combines the original English biography with Cadd, the English claims extracted from the non-English biography but not the English biography.
  • A claim is defined as a predicate with its arguments and modifiers, optionally including a corresponding hedge.

4 Datasets

The paper releases three CLAW-4L resources covering biography pairs, claim extraction, and claim-pair relations, with non-English biographies substantially richer than English counterparts.

  • CLAW-4L: CLAW-4L contains 300 English–non-English biography pairs about women from French, Chinese, or Azerbaijani Wikipedia.
  • CLAW-4L-CX: CLAW-4L-CX contains 600 manually annotated sentences with English claims, including 300 English sentences and 100 per non-English language.
  • CLAW-4L-RC: CLAW-4L-RC contains 600 English claim pairs labeled by semantic relation for alignment, contradiction, or irrelevance.
  • CLAW-4L: The biography pairs are constructed from Wikidata-linked entities and cleaned articles, with a language-calibrated token ratio used to favor richer non-English biographies.
  • CLAW-4L: 3.4× more tokens, 3.4× more sentences, and 2.9× more claims occur in non-English biographies on average.
  • Annotation: The benchmark includes balanced claim-complexity sampling and human verification of automatically drafted claims.

5 Processing Claims

The paper classifies and extracts claims across English and non-English biographies, then aligns them to select information that can enrich the English biography. This pipeline identifies most non-English claims as enrichment evidence after filtering covered claims.

  • Classifying Claim Pairs: The claim-pair classifier uses a coarse-to-fine schema that distinguishes alignment, contradiction, irrelevance, and fine-grained enrichment relations.Aligned pairs are further classified as exact alignment, one-sided enrichment, or mutual enrichment.
  • Classifying Claim Pairs: Qwen3.5-9B is the strongest open-source text-only classifier, achieving 95.5 ARC, 89.5 Align-FG, and 93.2 Overall.Its performance is close to GPT-5.1, which achieves 95.9 ARC, 90.8 Align-FG, and 93.3 Overall.
  • Extracting Claims: X-Claimify performs best across languages, especially under Exact-Aligned-F1, so the paper uses it to extract English reference claim sets.These claims support enrichment-claim selection and downstream generation evaluation.
  • Aligning Claims: The alignment pipeline retrieves semantically similar candidates above cosine 0.7 and within the top five, then applies LLM-based relation verification.The threshold retains at least one candidate for 96% of target-side claims while serving only as a permissive coarse filter.
  • Selecting Enrichment Claims: 86.2% of non-English-side claims are selected as enrichment evidence: 26.2% add detail to partially aligned English claims and 48.6% lack a relevant English counterpart.Only 13.9% are already covered by English.

6 Enriching English Biographies

The study compares raw, translation-based, and claim-based rewriting of English biographies using common generators and claim-level evaluation. Claim-based evidence generally provides the strongest supported-addition versus hallucination trade-off, with lower-resource exceptions and no evident writing-quality penalty.

  • Methods: The evaluation compares raw cross-lingual, machine-translated, and claim-based evidence while keeping the rewriting task and generators aligned across settings.The three generators are Qwen3.6-27B, Gemma-4-31B-it, and Mistral-3.2-24B-it.
  • Evaluation: Generated biographies are evaluated by extracting claims and comparing them with the original English and target-side reference claim pool.Metrics cover claim growth, factual support, hallucination, and the overall Bal. trade-off score.
  • Results: Claim-based evidence achieves the highest average #Valid at 20.47 and the lowest hallucination at 23.54, compared with translation’s 19.05 #Valid and 29.59 hallucination.Translation raises #Valid over raw evidence, from 15.89 to 19.05, but hallucination remains nearly unchanged: 29.49 versus 29.59.
  • Results: Claim-based evidence usually improves the supported-addition versus hallucination trade-off by reducing long-context burden, cross-lingual generation difficulty, and implicit evidence alignment.The only reported Bal. setting favoring translation over claims is Azerbaijani with Mistral.
  • Results: French and Chinese substantially outperform Azerbaijani, while Mistral achieves the highest Bal. in every language-evidence setting, largely through lower hallucination rates.The Azerbaijani exception is consistent with more difficult lower-resource claim extraction.
  • Human Evaluation: Claims obtains the highest Readability/Fluency score of 3.933 and Wikipedia-Style Writing score of 3.822, while Raw leads Coherence/Integration at 3.911.The human evaluation is described as a descriptive sanity check rather than a significance-tested comparison.
  • Qualitative Analysis: Claim-based outputs generally add more targeted, fine-grained events and relations, whereas raw outputs retain a more narrative style and translation is more sensitive to source and translation noise.Remaining errors include incorrect temporal, geographic, and educational details, unsupported over-specific enumerations, and repetitive generation in one difficult translation case.

7 Conclusion

The paper introduces a benchmark and claim-based framework for enriching existing English biographies with evidence from corresponding non-English Wikipedia biographies. Experiments find improved factual trade-offs in most settings without an evident writing-quality penalty, while lower-resource cases remain challenging.

  • Contributions: CLAW-4L pairs English biographies with French, Chinese, and Azerbaijani biographies of the same women, with resources for claim extraction and alignment.The framework maps both biographies into a shared English claim space before controlled rewriting.
  • Findings: Claim-based evidence offers the strongest supported-addition versus hallucination trade-off in most language-generator settings.Selected claims reduce long-context burden and improve interpretability relative to raw biographies and translations.
  • Findings: Human evaluation finds no evident writing-quality penalty from using selected claims as rewriting evidence, while Azerbaijani results show that lower-resource settings remain challenging.The conclusion frames these findings as complementary evidence about factual trade-offs and language-resource limitations.

Limitations

The benchmark and evaluation are limited in scope, source reliability, pipeline robustness, and deployment-oriented validation. The study also relies on model-assisted construction and leaves broader multilingual enrichment for future work.

  • Benchmark scope: The benchmark covers women’s biographies in only French, Chinese, and Azerbaijani, not the full diversity of Wikipedia languages, scripts, regions, or biography types.The authors identify broader language and entity coverage as future work.
  • Source reliability: Wikipedia statements are not assumed universally factual, so conflicts may reflect errors, temporal mismatch, or missing context rather than reliable disagreement.The framework surfaces conflicts for downstream reconciliation instead of resolving factual truth against external sources.
  • Pipeline errors: Automatic claim extraction and relation judgment can propagate errors into generation, especially for lower-resource languages such as Azerbaijani.The authors report that upstream processing can miss useful non-English evidence.
  • Evaluation and reproducibility: Human evaluation is a small-scale sanity check covering 15 biography pairs, one generator, and three expert annotators, so it is not comprehensive editorial or deployment validation.It also does not compare enriched outputs with original English biographies, and annotators saw method labels.
  • Evaluation and reproducibility: Exact reproduction may be affected by changes to the closed-source GPT-5.1 models used for reference claim extraction and prompt-assisted construction.The authors release prompts, scripts, and open-weight alternatives.
  • Beyond English: The current formulation generates only English biographies, leaving bidirectional and many-to-many enrichment across lower-resource editions for future work.The proposed direction would align claims from multiple language editions.

B.9 CLAW-4L-CX Statistics

CLAW-4L-CX is a 600-sentence human annotation benchmark designed to preserve claim-count diversity while selecting sentences with richer, more verifiable factual content. Its construction supports evaluating claim extraction against human references.

  • 300 English and 300 non-English-language sentences form CLAW-4L-CX, with exactly one selected sentence from each biography.
  • On the English side, selected sentences contain an average of 2.63 claims and include 25 instances per country across the 1-, 2-, 3-, and 4+-claim buckets.
  • English selection improved coverage rank by +0.4107, fact-density rank by +0.2095, rewrite-rank by +0.1965, and overall score by +0.2722.
  • Non-English selection improved coverage rank by +0.3285, fact-density rank by +0.2221, rewrite-rank by +0.4030, and overall score by +0.3179.
  • The benchmark balances broad claim-count coverage with higher-quality claim-bearing sentences, making it suitable for evaluating extraction against human annotation.Higher coverage, fact density, and rewrite-rank indicate better factual capture, richer content, and easier verification or refinement.
  • CLAW-4L-RC contains 600 manually validated claim pairs labeled as aligned, contradicted, or not relevant, with four balanced aligned sub-labels.The aligned sub-labels distinguish exact equivalence, one-sided enrichment, and mutual enrichment relations.

C.4 Results

CLAW-4L-RC evaluation shows GPT-5.1 is strongest overall and Qwen3.5-9B is the closest open-source model, while fine-grained enrichment distinctions remain difficult. Structured infoboxes do not consistently help and can add noise for stronger models.

  • GPT-5.1 achieves the strongest performance across ARC, Align-FG, and Overall, while Qwen3.5-9B is the closest open-source model.
  • 95.5 vs. 95.9 in ARC is the gap between Qwen3.5-9B and GPT-5.1, with both substantially outperforming other open-source model families.
  • Align-FG scores are consistently lower than ARC, and mutual partial alignment (A↔B) is typically the hardest relation.The pattern indicates greater difficulty distinguishing one-sided and bidirectional non-conflicting enrichment than making top-level relation judgments.
  • GPT-5.1-parsed infoboxes do not consistently improve performance and often slightly degrade stronger models such as Qwen3-14B, Qwen3.5-9B, and GPT-5.1.Smaller models occasionally benefit, suggesting that the value of structured context depends on model capacity and input fidelity.
  • Within Qwen3 and Qwen3.5, performance improves monotonically with model size, whereas the Ministral-3 3B model outperforms its larger variants.

D Cross-lingual Claim Extraction Methods

The cross-lingual extraction methods normalize biographies into English claims through combinations of decomposition, decontextualization, verification, and disambiguation. X-Claimify is the most reliable framework overall, while operation order affects redundancy and exact matching.

  • The methods differ in whether they explicitly decompose, decontextualize, select verifiable content, resolve ambiguity, and use context or staged execution.
  • Five adapted frameworks—X-FactScore, X-FactCheck-GPT, X-DnDScore, X-VeriScore, and X-Claimify—map English and non-English biographies into a shared English claim space.
  • Claim extraction quality is evaluated against three human reference sets using classifier-based claim-pair matching rather than exact string matching.
  • X-Claimify achieves the strongest Exact-Aligned-F1 across English, French, and Chinese and the best or near-best Aligned-F1.Its performance is useful downstream because alignment requires recovering source content while preserving comparable event-level granularity.
  • X-DnDScore is weaker because decomposing before decontextualizing can duplicate contextual information across claims, producing overlapping or redundant verbalizations.Methods that resolve context before final decomposition produce more factually independent claims and improve exact matching.
  • GPT-5.1 remains the reference extraction backbone because it is strongest and most consistent across languages, despite Gemma-4-31B-it outperforming it on French Aligned-F1.On French, Gemma-4-31B-it reaches 82.62 versus GPT-5.1's 78.05 on Aligned-F1 but trails on Exact-Aligned-F1, 70.18 versus 73.56.

E.2 Claim-based Evaluation of Translated Biographies

The paper evaluates translation models by extracting English claims from source and translated biographies, retrieving candidate alignments, and scoring coarse and exact claim preservation. Exact alignment is substantially harder, open-source LLMs outperform dedicated multilingual translation models, and language-specific model choices are made accordingly.

  • Evaluation protocol: Claim-based evaluation extracts English reference claims from non-English biographies and compares them with claims extracted from candidate English translations.Candidate claim pairs are retrieved with MPNet using a 0.7 cosine threshold and top-5 nearest neighbors, then scored with Aligned-F1 and Exact-Aligned-F1.
  • Results: Exact-Aligned scores are consistently much lower than coarse Aligned scores across language pairs and models.Exact matching is sensitive to paraphrasing, granularity shifts, and small factual perturbations, whereas coarse matching tolerates slight differences in granularity.
  • Results: The three open-source LLMs consistently outperform the two dedicated multilingual translation models under both Aligned-F1 and Exact-Aligned-F1.The authors attribute this pattern partly to model scale and partly to better handling of long-context, discourse-dependent biography translation.
  • Language-specific selection: French-to-English is the easiest setting overall, while Chinese-to-English is the most challenging.Azerbaijani uses Gemma-4-31B-it; French uses Mistral-3.2-24B-it; and Chinese uses Qwen3.6-27B, prioritizing semantic claim preservation.
  • Language-specific selection: The selected language-specific translation models are used in the main Machine Translation and Generation Pipeline experiments.These choices connect the claim-based translation evaluation to the subsequent enrichment experiments.

F Target-Side Enrichment Claim Selection

The enrichment pipeline retrieves candidate English claims for each non-English claim, classifies their relations, and selects non-English-specific, complementary, conflicting, and unmatched claims as enrichment evidence. English biographies cover only a small fraction of non-English claims, leaving substantial room for enrichment.

  • Claim alignment: Each non-English claim is matched against candidate English claims using MPNet retrieval with a 0.7 similarity threshold and top-5 truncation.Top-5 is used as a middle operating point between broader candidate coverage and verifier-call cost.
  • Claim selection: Exact matches and English-more-specific alignments are treated as already covered, while non-English-more-specific, complementary, conflicting, and unmatched claims remain enrichment candidates.Conflicts are retained because differences between human-written biographies may require contextual reconciliation rather than being treated as arbitrary noise.
  • Claim coverage: 13.9% of 89.3 average non-English claims are already covered by exact matches or English-more-specific alignments.Target-additive alignments account for 26.2%, while unmatched claims form the largest category at 48.6%.
  • Claim coverage: 86.2% of 89.3 average non-English claims are selected as enrichment candidates.This supports treating non-English biographies as complementary evidence rather than parallel paraphrases of English biographies.
  • Evaluation: The main analysis averages supported additions and hallucination rates across all three non-English languages and three generators for each evidence format.Qualitative analysis inspects contrasting high- and low-support cases across Raw, Translation, and Claims outputs.

G.4 Human Evaluation of Writing Quality

Human evaluation rates generated biographies for readability, coherence, and Wikipedia-style writing across languages, methods, and length strata. Writing quality generally declines with length, while Claims scores highest in the longest bucket, and three-rater mean scores show substantial reliability.

  • Evaluation design: Three Ph.D. researchers independently rate 45 outputs on 1–5 Readability/Fluency, Coherence/Integration, and Wikipedia-Style Writing scales.The sample contains three languages, five length-stratified biography pairs per language, and Raw, Translation, and Claims outputs for each pair.
  • Results: Writing quality generally declines as sampled biographies become longer for all methods.Claims has the highest scores on all three dimensions in the longest bucket, but each method–bucket cell contains only three outputs.
  • Results: Claims has the highest scores on readability, coherence, and Wikipedia-style writing in the longest sampling bucket.This pattern is descriptive and includes a 6,384-word repetitive Azerbaijani Translation output in the Translation B5 cell.
  • Reliability: Krippendorff’s α ranges from 0.507 to 0.537 for the subjective discourse-level judgments.The statistic uses ordinal distance for the Likert ratings.
  • Reliability: ICC(2,3) for the mean of three ratings ranges from 0.801 to 0.817.ICC(2,1) measures single-rating reliability, while ICC(2,3) measures reliability of the three-rater mean.
Loading 2608.23390v1…