PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Lingual Clinical Annotation Projection as Constrained Text Generation: A Six-Language Study

Álvaro Rey-Blanes, Francisco J. Moreno-Barea, Francisco J. Veredas

arXiv:2609.11450v1cs.CLcs.AI

TL;DR

The paper addresses how to transfer clinical annotations across languages while preserving exact target-text spans. It uses constrained direct LLM tagging with deterministic validation and character-offset reconstruction, alongside candidate-based and hybrid baselines, and reports stronger direct-projection performance across the evaluated settings.

  • Problem

    Cross-lingual clinical projection must preserve semantic labels and recover exact target spans despite translation variation and uneven multilingual annotation resources.

  • Method

    The study inserts entity tags directly into immutable target documents, validates text and tag integrity, reconstructs character offsets, and compares this workflow with supervised candidate and hybrid ML–LLM projection.

  • Results

    Direct LLM projection consistently outperformed candidate-based, hybrid, and previous state-of-the-art approaches across six target languages and three clinical entity types.

  • Takeaways & Limitations

    The formulation provides a practical mechanism for extending annotated clinical resources where manually annotated corpora and language-specific NLP resources are limited.

  • Takeaways & Limitations

    Generalisability remains unknown beyond Spanish as the source, six European target languages, three entity types, and the MultiClinCorpus benchmark; semantic correctness and downstream corpus utility were not additionally validated.

Abstract

from arXiv · show

Background: To determine whether cross-lingual clinical annotation projection can be formulated as a text-preserving, document-level generative task that produces verifiable character-level annotations for multilingual clinical corpus construction, and to characterize its robustness and computational trade-offs relative to candidate-based projection pipelines. Methods: We developed a constrained LLM projection workflow that inserts entity tags directly into immutable target-language text, followed by deterministic validation and character-offset reconstruction. We evaluated it alongside supervised candidate-span projection and hybrid ML-LLM refinement for transferring Spanish Disease, Symptom, and Procedure annotations into six languages. Evaluation used MultiClinAI gold standard with strict span matching and character-overlap F1 Results: Direct LLM projection achieved the strongest and most consistent performance. GLM 5.2 obtained a mean Strict F1 of 0.9201 across 18 language-entity combinations, while locally deployable Gemma4:31B achieved 0.9133. The best LLM configuration improved Strict F1 over the previous state of the art in all 18 settings, by 0.0564-0.1512, yielding 55,416 grounded mentions with reconstructed offsets. Conclusions: Direct LLM-based projection enables high-quality multilingual clinical annotation transfer and provides a practical approach for extending clinical NLP resources to languages with fewer annotated datasets and language-specific tools. Combined with local inference and deterministic validation, it can substantially reduce expert time and cost for multilingual clinical corpus construction.

1 Background and Significance

Clinical annotation projection addresses uneven multilingual resources by transferring expert labels from annotated source documents to equivalent target texts. The central challenge is preserving semantic equivalence while recovering exact target-language spans and character offsets despite translation variation.

  • 1 Background and Significance: Uneven distribution of annotated clinical resources makes cross-lingual projection an alternative to independently annotating every language.English has more datasets and models, while clinical annotation elsewhere is costly and time-consuming.
  • 1 Background and Significance: Translation can expand, contract, reorder, or reformulate mentions, so projection must recover exact target boundaries rather than copy source offsets.Clinical terminology, abbreviations, numerical expressions, and differing annotation granularity further complicate span recovery.
  • 1 Background and Significance: MultiClinCorpus transfers Spanish Disease, Symptom, and Procedure annotations into English, Dutch, Italian, Romanian, Swedish, and Czech.MultiClinAI provides a common benchmark for multilingual clinical information extraction and corpus construction.
  • 1 Background and Significance: Earlier projection methods used alignments, dictionaries, translation, similarity, contextual representations, or candidate-span ranking, but token correspondences still require valid entity-span reconstruction.MultiClinAI therefore distinguishes exact entity-type and offset matching from character-overlap agreement.
  • 1 Background and Significance: The study asks whether generative semantic matching can preserve immutable target text while producing verifiable corpus-ready annotations without explicit candidate enumeration.It evaluates constrained document-level tagging with deterministic validation and character-offset reconstruction against supervised and hybrid approaches.

2 Objective

The study tests whether cross-lingual clinical annotation projection can be reformulated as constrained, text-preserving document-level generation. It also examines robustness across languages and entity types alongside accuracy and computational trade-offs against candidate-based approaches.

  • 2 Objective: The objective is to determine whether constrained document-level generation can produce accurate, verifiable character-level annotations for multilingual clinical corpus construction.The evaluation covers robustness across languages and entity types.

3 Materials and Methods

The study compares supervised candidate-span projection, hybrid ML–LLM refinement, and direct LLM tagging on Spanish-to-six-language clinical annotation transfer. All strategies use official MultiClinAI evaluation, while the direct workflow preserves target text and reconstructs character offsets deterministically.

  • 3 Materials and Methods: Evaluation uses Strict F1 and character-overlap F1, with strict matching requiring exact entity type and start and end offsets.Gold-standard test annotations were hidden and predictions were scored through the official MultiClinAI evaluation server.
  • 3 Materials and Methods: The benchmark pairs Spanish clinical documents with English, Czech, Italian, Dutch, Romanian, and Swedish translations covering Disease, Procedure, and Symptom annotations.The complete test partition contains 3,260 parallel documents, while reported results use the official gold-standard subset.
  • 3 Materials and Methods: Three strategies identify target spans at different levels: supervised candidate classification, selective hybrid refinement, and direct document-level LLM projection.The approaches differ in candidate generation and in the role assigned to the LLM.
  • 3 Materials and Methods: The supervised pipeline generates target-document windows near source-entity lengths and ranks them using a binary classifier over surface, positional, structural, and semantic features.Candidate windows include local hard negatives and sampled soft negatives, with classifiers trained separately by language pair and entity type.
  • 3 Materials and Methods: The hybrid pipeline sends the best machine-learning projection to an LLM, retaining exact matches and correcting uncertain or incorrect cases before grounding them to target-document offsets.Exact string matching accepts unique occurrences and resolves multiple matches using the occurrence closest to the corresponding Spanish start offset.
  • 3 Materials and Methods: The direct workflow prompts an LLM to insert normalized entity tags into immutable target text, then validates tag structure, source–target counts, and text preservation before reconstructing offsets.The prompt prohibits translation, paraphrasing, commentary, and any modification beyond XML tag insertion; outputs are parsed sequentially to obtain offsets.

4 Results

Direct document-level projection achieved the strongest and most consistent exact-span performance across languages and entity types, outperforming candidate-based and hybrid approaches. It also improved over the previous best method in every evaluated setting, while exposing clear computational trade-offs.

  • 4.1 Candidate-based Projection and the Effect of Generative Refinement: 0.2462 mean absolute Strict F1 gain from LLM refinement improved every completed ML comparison, while character-overlap F1 rose from 0.6878 to 0.9087.Refinement also reduced both false-positive and false-negative projections, but remained dependent on the upstream candidate projection.
  • 4.2 Robustness of Document-level Projection across Languages and Entity Types: 0.9201 mean Strict F1 made GLM 5.2 the strongest direct projection model, followed by Gemma4:31B at 0.9133 and Qwen3.6:35B at 0.8300.GLM 5.2 led in 13 of 18 language–entity combinations, while Gemma4:31B led in five.
  • 4.2 Robustness of Document-level Projection across Languages and Entity Types: 0.9719 character-overlap F1 for GLM 5.2 exceeded its Strict F1, indicating that many residual errors involved boundary mismatches rather than unrelated-span localisation.Gemma4:31B and Qwen3.6:35B reached macro-averages of 0.9710 and 0.9222, respectively.
  • 4.3 Exact-boundary Recovery and Improvement over Previous Projection Methods: 0.0866 absolute Strict F1 improvement raised the best document-level configuration from 0.8348 to 0.9214 over the strongest previous MultiClinCorpus method.The improvement occurred in all 18 language–entity combinations and ranged from +0.0602 to +0.1512.
  • 4.4 Accuracy-Efficiency Trade-offs: 6.54 s per document for GLM 5.2 contrasted with 27.01 s for Gemma4:31B and approximately 0.1 s for trained ClinicalAligner3.ClinicalAligner3 additionally required approximately 2–4 h of task-specific training, whereas the LLM approaches were applied without task-specific training.

5 Discussion

The study finds that constrained document-level LLM generation can outperform candidate-based projection while preserving verifiable target-text grounding, but it entails higher inference costs and remains bounded by evaluation and deployment limitations.

  • Direct LLM projection substantially outperformed candidate-based, hybrid, and previous state-of-the-art approaches by combining document-level reasoning with deterministic text preservation and offset reconstruction.The approach avoids explicit alignment and candidate generation while producing reproducible character-level annotations.
  • A strict F1 improvement of +0.0564 to +0.1512 over the previous state of the art demonstrated stronger semantic localisation and more accurate complete-span reconstruction.The largest gains were observed for Procedure entities.
  • LLM performance varied by language: English, Italian, and Romanian were strongest, Dutch was most challenging, and Czech and Swedish remained competitive despite weaker resource representation.These results suggest resource availability alone does not determine projection performance.
  • Locally deployable models can keep projection within institutional infrastructure, which matters when governance, privacy, or data-sharing restrictions preclude external inference services.Gemma4:31B showed a consistent advantage over the previous state of the art without specialised alignment architecture.
  • Candidate-based methods remain computationally efficient after training, whereas direct LLM projection exchanges higher per-document inference cost for higher exact-span accuracy and no task-specific training.The trade-off is relevant when choosing between large-scale repeated processing and simpler, more accurate local workflows.
  • 5.1 Limitations: The benchmark used one Spanish source, six European target languages, and three entity types, so generalisation to other languages, corpora, sources, and annotation structures remains unknown.The authors call for external validation on additional clinical corpora and language pairs.
  • 5.1 Limitations: Structural validation cannot establish semantic correctness, and the study did not include bilingual or clinical-expert error analysis or downstream evaluation of projected corpora.Efficiency estimates and model comparisons may also vary with implementation, infrastructure, hardware, and future model versions.

6 Conclusion

The study concludes that text-preserving document-level generation provides accurate, verifiable multilingual annotation projection across six languages and three entity types. Direct LLM projection outperformed competing approaches and produced a practical route for extending clinical resources.

  • Direct LLM projection consistently outperformed supervised, hybrid, and previous state-of-the-art approaches across six target languages and three clinical entity types.It avoided explicit alignment, candidate generation, and task-specific training.
  • The best LLM configurations produced 55,416 projected clinical mentions while preserving exact textual grounding and reproducible character offsets.The result transfers expert annotations from one source language into six target languages.
  • Despite higher inference costs than specialised alignment systems, local deployment, deterministic validation, and offset reconstruction make the workflow practical and auditable for multilingual clinical corpus construction.The approach may reduce expert annotation required when extending existing resources across languages.

Funding

The work was supported by MICINN project PID2024-155334OB-I00, with open-access funding from Universidad de Málaga / CBUA.

  • The study was supported by MICINN project PID2024-155334OB-I00, and Universidad de Málaga / CBUA funded the open-access charge.

A Prompts

The appendix presents prompts for bilingual match classification, span correction, and constrained cross-lingual projection. These prompts require structured outputs, preserve translated text, and restrict projection to specified clinical XML tags.

  • Match classification: The match-classification prompt distinguishes exact, mid, and no matches using meaning rather than literal wording, with uncertainty resolved toward mid match.Non-exact decisions require a brief justification and the output must be JSON only.
  • Span correction: The span-correction prompt uses document context and the source-language reference to suggest accurate target-language entity strings for problematic entities.It requests a JSON array containing entity identifiers and suggestions.
  • Cross-lingual projection: The English projection prompt supplies tagged Spanish and untagged translated documents as inputs and returns only the tagged translated document.The prompt restricts edits to insertion of XML tags from the active label set.
Loading 2609.11450v1…