PaperlayerView exact revision on arXiv

Source-linked AI summary

SAGE: Semantic Anchor-Guided Evolution for Grounded Medical QA Data Synthesis

Chuan Li, Chengyu Wang, Cen Chen, Ye Lyu, Mingyuan Fan, Ming Gao

arXiv:2610.08093v1cs.CLcs.AI

TL;DR

Scarce expert-annotated medical QA data and privacy constraints limit clinical model development. SAGE grounds small-model synthesis in public taxonomies and iteratively combines atomic with associative generation; the resulting models outperform self-derived and document-based methods across benchmarks.

  • Problem

    High-quality expert-annotated medical training data are scarce, and privacy and resource constraints complicate assembling large corpora or using external APIs.

    Source: §1 · p. 1

  • Method

    SAGE uses public taxonomies as semantic anchors for small-model medical QA synthesis, iteratively combining atomic and associative generation from minimal seeds.

    Source: §1 · p. 2

  • Results

    Across medical QA benchmarks, SAGE-trained models outperform self-derived and document-based methods, achieving 68.12% average accuracy on five datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SAGE offers a practical pathway for developing medical AI assistants from compact knowledge bases without relying on massive corpora.

    Source: §6 · p. 8

    SAGE's effectiveness partly depends on the scope and organization of taxonomies such as MeSH, and generated scenarios may not capture nuanced open-ended clinical reasoning.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Developing reliable models for clinical tasks, such as Medical Question Answering (QA), is severely constrained by the limited availability of high-quality, expert-annotated training data. This challenge is exacerbated by stringent privacy requirements and the impracticality of utilizing large open-source corpora or proprietary cloud APIs within resource-limited clinical settings. To address these obstacles, we introduce SAGE (\textit{Semantic Anchor-Guided Evolution}), a novel data synthesis framework that enables small, locally deployed models to generate high-quality medical training data. SAGE leverages lightweight, publicly available taxonomies such as MeSH as semantic anchors, imposing a structured prior to effectively guide and ground the data generation process. At its core, SAGE iteratively interleaves atomic (individual concept-based) and associative (relation-based) synthesis, bootstrapping training data from minimal seeds. This approach eliminates the need for large collections of medical documents or reliance on external APIs, providing a practical solution for on-premises data creation. Extensive experiments across multiple medical question-answering benchmarks demonstrate that models fine-tuned with SAGE-synthesized data consistently outperform those trained using self-derived or conventional document-based paradigms, highlighting tangible improvements in data efficiency and resource utilization for medical LLM development. Code is available at https://github.com/DIaacKr/SAGE.

1 Introduction

SAGE uses public medical taxonomies as semantic anchors for small-model synthesis of grounded QA data, combining atomic and associative generation. The paper reports that SAGE-trained models outperform self-derived and document-based methods across medical QA benchmarks.

1 Introduction

  • High-quality expert-annotated medical datasets are scarce, while assembling large corpora is costly and constrained by privacy and ethical requirements.These constraints hamper adapting general-purpose LLMs into reliable medical experts.Source: §1 · p. 1
  • Self-derived synthesis risks ungrounded outputs and hallucinations, whereas document-based synthesis requires extensive corpora and can favor memorization over clinical reasoning.The paper contrasts both paradigms with SAGE's use of semantic anchors.Source: §1 · p. 2
  • SAGE uses lightweight public taxonomies such as MeSH as semantic anchors to guide small, locally deployed models in generating grounded medical QA data.The framework is designed without reliance on large corpora or external APIs.Source: §1 · p. 2
  • SAGE achieves state-of-the-art performance across medical QA benchmarks, with 68.12% average accuracy on five datasets, outperforming self-derived and document-based methods.The introduction also reports that SAGE exceeds results from several high-quality open-source datasets and models trained on massive data.Source: §1 · p. 2
  • Its parallel iterative strategy combines atomic generation for factual coverage with associative generation for more complex reasoning, balancing exploitation and exploration.The process bootstraps from minimal seeds and expands the synthesized dataset through ongoing calibration.Source: §1 · p. 2

2 Related Work

Related work describes self-derived and document-based approaches to synthetic training data, noting trade-offs in factual control and resource requirements.

2 Related Work

  • Self-bootstrapping methods expand training data from small seed sets, but their filtering often favors semantic richness or complexity rather than factuality.The cited discussion identifies this as a critical concern for medical training because hallucinations can affect fine-tuning.Source: §2 · p. 2
  • Document-based synthesis grounds generated data in source documents, but depends heavily on access to large, high-quality collections that may be unavailable in low-resource settings.The passage describes Easy Datasets and Wrap as examples of document-based approaches.Source: §2 · p. 3

3 Methodology

SAGE alternates language-model generation in text space with projection onto a fixed MeSH taxonomy, whose active anchor population evolves across epochs. It combines concept-focused and relation-oriented generation with consistency and diversity checks to select and expand QA instances.

3.1 Problem Formulation and Neural-Symbolic Evolution

  • SAGE treats a locally deployed model as a neural variation operator, while a fixed taxonomy anchors and consolidates its text-space exploration into an evolving population.Candidate questions are mapped back to the taxonomy and may contribute updated anchors to the next epoch.Source: §3.1 · p. 3, §3 · p. 3

3.2 MeSH-Structured Semantic Anchors

  • Seed questions are mapped to MeSH concepts that form anchor configurations, providing a structured semantic scaffold without prescribing a complete reasoning path.The mapping retains concepts under category and hierarchy-depth constraints; nonempty seed mappings initialize the anchor population.Source: §3.2 · p. 3

3.3 Dual-Track Neural Generation

  • Singleton anchors use atomic generation grounded in taxonomy definitions and sampled clinical qualifiers, while multi-concept anchors use associative generation to explore relations.Atomic generation can produce parallel candidates with different qualifiers when available.Source: §3.3 · pp. 3–4

3.4 Symbolic Selection and Anchor Evolution

  • Answer-consistency and diversity checks select candidates; projection identifies newly surfaced MeSH concepts, which become singleton anchors for later atomic generation and update the active population.Retained projected configurations and singleton mutation configurations are combined for the next epoch.Source: §3.4 · pp. 4–5

4 Experiments

Across the evaluated medical QA benchmarks, SAGE outperforms self-derived and document-based baselines and matches or exceeds large-scale datasets and models. The experiments compare common synthesis and fine-tuning setups, ablations, and performance across data scales and out-of-distribution tasks.

4.1 Experimental Setup

  • The study uses Qwen3-4B with thinking disabled for synthesis and fine-tunes fresh Qwen3-4B-Base models for five epochs under a shared protocol.The stated protocol uses a learning rate of 1 × 10−5 and global batch size 16.Source: §4.1 · p. 5

4.2 Baselines and Benchmarks

  • SAGE is compared with self-derived, document-based, and open-source dataset paradigms across ten medical QA benchmarks, using accuracy for every task.The benchmarks cover seed-aligned domains, cross-domain generalization, robustness, and advanced reasoning.Source: §4.2 · p. 5

4.3 Main Results

  • SAGE consistently outperforms baseline methods and matches or exceeds large-scale datasets and models fine-tuned on massive CoT data.Table 1 reports performance on mainstream medical QA benchmarks, with units in percentage.Source: Table 1 · p. 6, §4.3 · p. 6

4.4 Ablation Study

  • Ablations report marked degradation when either question type is removed, while Figure 3 covers diverse benchmarks including out-of-distribution tasks.The study also conducted ablations of four components and extended validations of mapping, corpus selection, and factual filtering.Source: §4.4 · pp. 6–7, Figure 3 · p. 6
  • SAGE maintains strong performance across data scales, rivals much larger models, and consistently outperforms baselines on out-of-distribution QA benchmarks.The scaling analysis compares SAGE with CoT-Self-Instruct from 1k to 10k samples; the text also reports fewer tokens than baselines and efficient offline semantic mapping.Source: §4.4 · p. 7, Figure 4 · p. 7

5 Discussion

SAGE’s dual-track synthesis is presented as combining factual grounding with complex medical reasoning. The discussion reports a balance of factuality, reasoning, and utility, alongside semantic depth and combinatorial diversity as data scale increases.

5.1 Qualitative Case: Mitigating Hallucination and Enhancing Reasoning.

  • Document-based methods may produce factual content but focus on simple recall, while self-derived methods can emphasize reasoning yet lack factual verification and may hallucinate.The discussion illustrates SAGE’s alternative with atomic generation from Chronic Pain followed by associative generation using related concepts for clinical scenarios.Source: §5.1 · p. 7
  • Starting from Chronic Pain, atomic questions introduce related concepts that then scaffold associative clinical scenarios integrating accurate knowledge with advanced reasoning.The described two-stage process is presented as mitigating self-derived hallucination risks and overcoming document-based shallow reasoning limitations.Source: §5.1 · p. 7

5.2 Quantitative Assessment: Factuality, Reasoning, and Utility.

  • SAGE achieves a superior balance of factuality and reasoning, while Wrap has high factuality but limited reasoning depth and ScaleQuest shows moderate reasoning with hallucinations.SAGE also consistently minimized confabulations against baselines on TruthfulQA and MedHALT.Source: §5.2 · pp. 7–8

5.3 Combinatorial Validity and Diversity.

  • As data scale increases, SAGE maintains high semantic depth and combinatorial diversity, whereas concept-generation baselines produce broad, shallow, recurring topics and rapidly lose diversity.The analysis maps generated concepts to MeSH and calculates average semantic depth and combination diversity in batches of 1,000 samples.Source: §5.3 · p. 8, Figure 7 · p. 8

6 Conclusion

The conclusion presents SAGE as a taxonomy-grounded framework for small local models to produce structured medical QA data under privacy constraints. It identifies a practical route to medical AI development from a compact knowledge base.

6 Conclusion

  • The conclusion frames SAGE as addressing medical LLM data scarcity under strict privacy constraints without relying on massive corpora.Its stated pathway uses a compact knowledge base.Source: §6 · p. 8
  • SAGE grounds atomic and associative generation in public taxonomies so small local models can autonomously produce pedagogically structured data.The conclusion describes this as a practical pathway for developing medical AI assistants from a compact knowledge base.Source: §6 · p. 8
  • Dynamic diversity is measured by discovery rate R = Nnew/B, where Nnew is the number of unique unseen concept combinations and B is batch size.Figure 8 compares dynamic diversity as generated samples increase.Source: §6 · p. 9, Figure 8 · p. 9

Limitations

SAGE’s limitations concern the scope of its semantic resources and generated scenarios, the extent of sample review, and the evaluation tasks covered.

Limitations

  • SAGE’s effectiveness partly depends on the scope and organization of taxonomies such as MeSH; domains without mature resources may need task-specific schemas.Generated scenarios may also miss nuanced, open-ended reasoning found in authentic clinical narratives.Source: Limitations · p. 9
  • The study did not comprehensively double-check every generated sample with clinicians, leaving some subtle clinical inaccuracies potentially unexamined.The authors also identify dynamic anchor expansion, multimodal knowledge, and adaptation beyond medicine as future directions.Source: Limitations · p. 9
  • Evaluation primarily covers QA benchmarks; broader validation on interactive tasks such as patient dialogue simulation or diagnostic decision support remains future work.Source: Limitations · p. 9

Ethics Statement

The study uses publicly available, deidentified resources and synthetic training data, with MeSH concepts and a 1,000-instance medical QA seed set supporting synthesis.

Ethics Statement

  • The study uses publicly available, deidentified resources and synthetic training data, without human subjects, personal health data, or protected clinical records.The authors warn that synthetic-data models may reproduce or amplify biases and that outputs are not clinical advice; deployment requires clinical validation, regulatory compliance, and human supervision.Source: Ethics Statement · p. 9

A.1 Knowledge Base Description

  • MeSH organizes biomedical descriptors hierarchically, encodes semantic relations, and supplies synonyms used for lightweight string-matching; selected anchors are medically relevant mid-to-deep terms.The described categories include anatomy, diseases, chemicals and drugs, diagnostic and therapeutic techniques, psychiatry and psychology, and phenomena and processes.Source: §A.1 · p. 12

A.2 Seed Dataset Overview

  • The synthesis pipeline starts from m1k, which provides concept distributions and few-shot examples as scaffolds for small-model generation.The seed contains 1,000 diverse instances selected from 37,000 difficult questions drawn from an initial pool of 196,000 public samples, with verified Chain-of-Thought solutions.Source: §A.2 · pp. 12–13

A.3 Benchmark Datasets

The section describes a diverse medical benchmark suite and reports SAGE’s performance across weaker models, non-QA tasks, factuality tests, and document-based comparisons. Its cross-scale result places a 4B model near larger-model performance.

A.3 Benchmark Datasets

  • The benchmark suite includes medical entrance-exam questions, USMLE-derived questions, biomedical QA, expert-written biomedical questions, and expert-crafted multi-step reasoning questions.Source: §A.3 · p. 13

B Implementation Details and Hyperparameters

  • Candidate generation uses temperature 0.7, top-p 0.95, and a 4,096-token limit; associative generation draws two examples and self-consistency uses three auxiliary answers.Two candidates are generated for each eligible anchor configuration.Source: B Implementation Details and… · p. 13

C Training on Weaker Models

D Generalization on non-QA Benchmark

E Performance Ceiling and Cross-Scale Comparison

F Quantitative Assessment of Hallucinations Beyond LLM-as-a-Judge

G In-depth Comparison with Document-Based Methods

  • With identical seed cases or a matched 3.51M-token PubMed corpus, document-based baselines improve only marginally and remain significantly below SAGE.The authors attribute the difference to SAGE’s conversion of dictionary-style facts into clinical reasoning instructions, rather than simply greater text exposure.Source: G In-depth Comparison with… · pp. 14–15

H Robustness and Evaluation of the Mapping Function

The mapping evaluation contrasts extraction precision and downstream performance with computational and privacy costs. The following section discusses factual grounding and filtering choices.

H Robustness and Evaluation of the Mapping Function

  • 36.52% precision for naive string matching contrasts with SAGE’s 12.21% recall and 55.68% precision, reflecting a deliberate trade-off against broad, shallow terms.The authors argue that precise, deep anchors provide greater clinical density for synthesis than numerous noisy concepts.Source: H Robustness and Evaluation of the… · pp. 15–16, Table 6 · p. 16
  • 68.12% vs. 68.14% downstream performance: SAGE mapping nearly matches the online MoD tool while running offline; MoD requires external requests and is about 40× slower.Source: H Robustness and Evaluation of the… · p. 16

I Ablation on Factual Grounding and Filtering Strategies

J Qualitative Analysis of Baseline Methods

On extended benchmarks, SAGE consistently achieves leading performance; the qualitative cases contrast Wrap’s superficial response with a factual grounding error in CoT-Self-Instruct.

J Qualitative Analysis of Baseline Methods

K Evaluation on Extended Benchmarks

L Token Efficiency Analysis

SAGE combines low response-token lengths with stable benchmark performance across runs, while outperforming fine-tuning on original training splits. Its prompt templates support atomic and associative question generation.

L Token Efficiency Analysis

  • 230.3 average tokens for SAGE responses compares with 2655.2 for Qwen3-4B thinking mode and 2947.3 for M23K.Wrap averages 57.5 tokens; the authors describe SAGE as balancing downstream effectiveness and inference-time token efficiency.Source: L Token Efficiency Analysis · p. 18

M Stability Across Independent Runs

N Comparison with Fine-Tuning on Original Training Sets

  • Fine-tuning on original MedMCQA, MedQA, and PubMedQA training splits performs substantially below SAGE, possibly because their short labels offer limited reasoning-oriented supervision.The original splits contain approximately 10k, 100k, and 200k samples, respectively; SAGE uses explanation-rich instruction data grounded in semantic anchors.Source: N Comparison with Fine-Tuning on… · pp. 18–19, Table 12 · p. 19

O Prompt Templates

Related papers

Loading 2610.08093v1…