PaperlayerView exact revision on arXiv

Source-linked AI summary

ASCRIBE: Atomic and Significance-Based Reasoning for Thai Clinical SOAP Note Generation

Tarm Kalavantavanich, Teerawut Ponarchar, Pattaramanee Arsomngern, Jenta Wonglertsakul, Watcharakorn Chuthong, Chiraphat Boonnag, Knot Pipatsrisawat, Titipat Achakulvisut

arXiv:2610.01234v1cs.CL

TL;DR

Clinical SOAP note generation is burdensome, while Thai lacks public datasets and existing reasoning methods can omit important information or produce unsupported content. ASCRIBE ranks extracted atomic facts by clinical significance before generating notes, alongside releases of Thai clinical data. Its prompt and GRPO reward formulations improve reported results, including against Gemini 3.1 Pro on completeness.

  • Problem

    Thai SOAP note research lacks public datasets, while existing reasoning methods can omit clinically important information and generate unsupported content.

    Source: §1 · p. 1

  • Method

    ASCRIBE extracts atomic clinical facts, ranks them by significance, and generates SOAP notes from the reasoning trace; the authors also synthesize S-ICOPD from real paired transcripts and notes.

    Source: §1 · p. 1

  • Results

    As a prompt, ASCRIBE improves frontier LLMs by up to 10.3 points over standard prompting; synthetic-data-trained Gemma-4-E4B surpasses Gemini 3.1 Pro on completeness by over 10 points as a GRPO reward.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The released synthetic encounters and ThaiClinicBench provide a practical foundation for developing medical scribes for Thai and other low-resource languages.

    Source: §8 · p. 9

    The study covers a small number of outpatient encounters, a single SLM, and only Thai; its synthetic transcripts may not capture the diversity and complexity of real clinical conversations.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Automatic SOAP note generation can ease the documentation burden on physicians, but existing reasoning methods often omit clinically important information and generate unsupported content. Progress in Thai is further hindered by the lack of publicly available datasets. We propose ASCRIBE, a physician-inspired reasoning framework that ascribes a clinical-significance level to each extracted atomic fact in the conversation before summarization, making a general-purpose LLM a more reliable scribe. We also release ThaiClinicBench, the first de-identified Thai clinical summarization benchmark of real encounters, together with a synthetic training corpus derived from real clinical notes. As a prompt, ASCRIBE outperforms chain-of-thought prompting on GPT-5.4 and Gemini 3.1 Pro across the physician-aligned LLM-judge metrics and improves on standard prompting by up to 10.3 points on the completeness LLM-judge metric. As a GRPO reward, it enables a Gemma-4-E4B model trained solely on synthetic data to match Gemini 3.1 Pro in factual precision and surpass it in completeness. Code and data can be found at https://github.com/loolootech/ascribe.

1 Introduction

Clinical note generation is time-consuming, and Thai clinical summarization faces both data and reasoning gaps. The paper introduces ASCRIBE and releases Thai clinical datasets, reporting gains from its prompting and reward formulations.

  • Clinical documentation burdens physicians, while Thai conversational nuances and the lack of public Thai SOAP datasets leave data and reasoning gaps.Source: §1 · p. 1
  • ASCRIBE extracts atomic facts, ranks them by clinical significance, and generates SOAP notes from that reasoning trace.Source: §1 · p. 1
  • ASCRIBE prompting improves frontier LLMs by up to 10.3 points over standard prompting; as GRPO rewards, it lets synthetic-data-trained Gemma-4-E4B surpass Gemini 3.1 Pro on completeness by over 10 points.Source: §1 · p. 2
  • Real-data-seeded synthetic transcripts match real-transcript SFT performance and surpass it on completeness under GRPO.Source: §1 · p. 2
  • The authors release S-ICOPD, a synthetic training corpus, and ThaiClinicBench, a de-identified benchmark of real Thai clinical encounters.Source: §1 · pp. 1–2

2 Related Work

Prior work spans clinical note generation, medical reasoning, and note evaluation. The cited literature also motivates structured reasoning over clinical facts and rewards that assess reasoning traces as well as final notes.

  • Clinical note research progressed from section-aware SOAP pipelines to English datasets and methods including in-domain pretraining, extractive-abstractive summarization, transcript synthesis, and prompting [Krishna et al., 2021; Abacha et al., 2023b,a; Yim et al., 2023; Grambow et al., 2022; Neupane et al., 2024; Ge et al., 2025; Kernberg et al., 2024].Source: §2 · p. 2
  • Although CoT has supported medical reasoning, including licensing-exam performance, evidence reports degraded clinical-text performance from hallucination and weak grounding [Wei et al., 2022; Wu et al., 2025].Source: §2 · p. 2
  • Clinical-note evaluation work addresses omissions, clinical-impact grading, and claim-level precision and recall because overlap metrics such as ROUGE correlate inconsistently with physician judgment [Abacha et al., 2023c; Schumacher et al., 2023; Asgari et al., 2025; Xie et al., 2024].Source: §2 · p. 2
  • Prior RLVR work adapted claim-level precision and recall as a GRPO reward [Shao et al., 2024; Jhaveri et al., 2025].Source: §2 · p. 2
  • Unlike approaches rewarding only the final note, the paper also rewards reasoning traces through clinical fact extraction and significance attribution.Source: §2 · p. 2

3 Proposed Method

The method structures Thai SOAP generation around atomic facts ranked by clinical significance, then uses that reasoning in prompting and fine-tuning. It also synthesizes encounter transcripts from clinical notes as training data.

3.1 ASCRIBE: Reasoning over Atomic Facts and Clinical Significance

  • ASCRIBE extracts atomic facts from the transcript, classifies their clinical significance into a reasoning trace, and uses that trace to condition SOAP-note generation.The task is framed as sequence-to-sequence generation of a four-section SOAP note.Source: §3 · p. 2, Figure 1 · p. 3, §3.1 · p. 3
  • Atomic facts are designed as standalone, single-idea, nonduplicative units that ground the model’s reasoning in explicit, verifiable information.Compound statements are split unless a clause is needed to preserve the main fact’s meaning.Source: §3.1 · p. 3
  • Six significance levels rank facts by potential impact on patient safety and care, spanning current-visit assessment, management, required documentation, and longitudinal relevance.Physicians co-designed the levels around SOAP support and longitudinal care.Source: §3.1 · pp. 3–4

3.2.1 Supervised Fine-Tuning (SFT)

  • LoRA-based supervised fine-tuning compares note-only targets with targets that prepend a reference reasoning trace to the SOAP note.The two settings train on clinical transcripts.Source: §3.2.1 · p. 4

3.2.2 Reinforcement Learning with Verifiable Rewards (RLVR)

  • GRPO uses five rewards to optimize reasoning and note quality, including atomic-fact matching, significance labels, SOAP format, completeness, and factual grounding.The factual-grounding and completeness rewards use SOAPPrecision and ClaimEntailment, respectively.Source: §3.2.2 · pp. 4–5

3.3 Synthesizing Encounter Transcripts

  • Synthetic transcript generation plans profile-consistent intent paths from a weighted graph, then has doctor and patient agents realize each intent as dialogue.An LLM judge ranks candidate paths; generated dialogue sections are validated and may be regenerated using feedback.Source: §3.3 · p. 5

4 Datasets

The study uses a private Thai encounter corpus, releases a real-encounter benchmark, and constructs a synthetic training corpus from clinical notes. Reference facts, labels, and SOAP targets are generated for training and evaluation.

4 Datasets

  • The study uses ICOPD, ThaiClinicBench, and S-ICOPD, with real conversation transcripts manually produced by human annotators.Source: §4 · p. 5

4.1 Internal Clinical OPD Dataset (ICOPD)

  • ICOPD contains 400 consented Thai outpatient encounters with transcripts, physician-written notes, and LLM-constructed reference SOAP notes; its 300/50/50 split is for training, validation, and test.Conversations average 328 words (1.34k characters).Source: §4.1 · pp. 5–6

4.2 ThaiClinicBench

  • ThaiClinicBench is a manually de-identified, consented set of 44 clinic encounters paired with physician-written SOAP notes, designated as an out-of-distribution test set.The authors describe it as the first Thai medical-scribe benchmark released for research.Source: §4.2 · p. 6

4.3 Synthetic ICOPD Dataset (S-ICOPD)

  • S-ICOPD synthesizes transcripts from physician-written ICOPD notes used as patient profiles, yielding 295 training and 50 validation transcripts; every text in the corpus is model-generated.Five training cases were excluded because the pipeline did not produce usable transcripts.Source: §4.3 · p. 6

4.4 Reference Training Target Generation

  • Gemini 2.5 Pro generates reference atomic facts and significance labels for rewards, evaluation, and SFT, while SOAP targets are drafted by Gemini 3.1 Pro and refined with GPT-5.4.Source: §4.4 · p. 6

5 Evaluation Metrics

The evaluation combines lexical and semantic similarity with two LLM-judge measures of clinical quality. ClaimEntailment assesses completeness, while SOAPPrecision assesses factual grounding.

  • The evaluation includes ROUGE-L and BLEURT for lexical and semantic similarity, alongside adapted LLM judges intended to assess clinical quality and validated for physician alignment.The judges assess generated notes directly rather than extracting claims first.Source: §5 · p. 6
  • ClaimEntailment measures completeness by checking whether reference atomic facts are entailed by the generated note, while CE-Impact restricts that check to the three highest significance levels.The restricted levels are Diagnosis-impacting, Treatment-impacting, and Required.Source: §5 · p. 6
  • SOAPPrecision measures factual grounding by checking whether each generated statement is explicitly supported by the encounter transcript, allowing paraphrases and terminology normalization.Unsupported, inferred, or hallucinated content is penalized.Source: §5 · p. 6

6 Experimental Settings

The experiments compare zero-shot prompting strategies for frontier models and GRPO reward formulations for a small language model. The baselines cover standard prompting, chain-of-thought, and structured reasoning, as well as three alternative GRPO rewards.

  • GPT-5.4 and Gemini 3.1 Pro are evaluated with standard SOAP prompting, chain-of-thought, and the proposed structured-reasoning prompt.Source: §6 · p. 6
  • Gemma-4-E4B-it is the base model for fine-tuning comparisons against ROUGE-L, BGE-M3, and ClinicEval GRPO baselines.Source: §6 · p. 6
  • The GRPO baselines use lexical overlap, embedding similarity, or completeness and factual-grounding rewards.These correspond to ROUGE-L, BGE-M3, and ClinicEval, respectively.Source: §6 · p. 6

7 Experimental Results

ASCRIBE improves clinical note quality through structured prompting and clinically grounded GRPO rewards, with results across real and synthetic training, clinical metrics, and ASR transcripts. The experiments also show that conventional lexical metrics do not consistently track clinical quality.

Effectiveness of RLVR

  • With real-data GRPO, ASCRIBE rewards yield Gemma-4-E4B's best clinical performance: ICOPD SP/CE/CE-Impact scores are 87.1/62.1/71.2, and it surpasses Gemini 3.1 Pro across judge metrics and GPT-5.4 on CE.Table 2 compares ASCRIBE prompting and fine-tuning with baselines on ICOPD and ThaiClinicBench.Source: Table 2 · p. 7, §7 · pp. 7–8

Effectiveness of Structured Reasoning

  • On ThaiClinicBench, GPT-5.4's ASCRIBE prompt improves SP/CE/CE-Impact over standard prompting by +5.8/+6.9/+10.3, outperforming CoT prompting in the zero-shot setting.ASCRIBE also outperforms standard prompting for Gemini 3.1 Pro and largely shows the same trend for Gemma-4-E4B.Source: §7 · p. 7

SFT vs. GRPO

  • GRPO with clinically grounded rewards substantially improves SP, CE, and CE-Impact, while SFT generally achieves higher ROUGE-L against curated reference notes.BLEURT differences between SFT and GRPO are less consistent.Source: §7 · p. 7

Reward Design

  • ROUGE-L optimization maintains high SP but severely reduces CE; ClinicEval improves CE at comparable SP, and ASCRIBE rewards improve both SP and CE.The experiments also report that optimizing BGE-M3 improves CE but yields the lowest SP among GRPO baselines.Source: §7 · p. 8

Ablation of Structured Reasoning Prompt and Rewards

  • Replacing the standard prompt with ASCRIBE improves SP by +1.3 on ICOPD and +1.4 on ThaiClinicBench; its reward adds further SP/CE/CE-Impact gains on both datasets.With the ASCRIBE prompt, replacing ClinicEval with the ASCRIBE reward adds +2.3/+0.3/+0.2 on ICOPD and +1.1/+0.5/+0.3 on ThaiClinicBench.Source: §7 · p. 8

Effectiveness of Synthetic Training Data

  • Synthetic-only data performs comparably to real-data SFT and improves GRPO CE and CE-Impact with comparable SP, while naively mixing real and synthetic data can reduce performance.On ThaiClinicBench, synthetic-only standard-prompt SFT has SP/CE/CE-Impact of 83.7/49.1/56.1 versus 83.9/50.6/56.5 for real-data SFT; the authors hypothesize that corpus distribution differences may explain degradation when mixing.Source: §7 · p. 8

Clinical vs. Conventional Evaluation

  • Conventional metrics do not consistently reflect clinical quality: lexical-similarity scores can be high without corresponding CE gains, while ClinicEval GRPO improves CE and CE-Impact at ROUGE-L cost.The findings suggest conventional and clinical metrics capture complementary aspects of note quality.Source: §7 · p. 9

Robustness to ASR Transcripts

  • ASR transcripts lower CE and SP for all models; ASCRIBE variants improve absolute ASR performance in most settings, although GRPO models generally have larger drops from human transcripts.The figure compares CE and SP on human-annotated versus ASR transcripts; ASCRIBE rewards improve GRPO ASR SP over ClinicEval (72.7 vs. 65.1–69.9).Source: Figure 3 · p. 8, §7 · p. 9

8 Conclusion

The conclusion presents ASCRIBE as a structured reasoning framework for Thai SOAP note generation, using atomic clinical facts and their significance to improve prompting and GRPO rewards. It also highlights synthetic-data training and the release of Thai clinical resources.

Conclusion

  • ASCRIBE models atomic clinical facts and their clinical significance, improving SOAP-note prompting and providing an effective reward formulation for GRPO fine-tuning.The conclusion reports competitive performance using synthetic clinical encounters without additional real clinical conversations for model tuning.Source: §8 · p. 9
  • The authors release synthetic clinical encounters and ThaiClinicBench as resources for developing clinical scribes in Thai and other low-resource languages.They describe the benchmark as a practical foundation for clinically reliable medical scribes.Source: §8 · p. 9

Limitations

The section outlines limitations in the study’s scope, evidence, evaluation, and robustness, alongside a human alignment study comparing the LLM judge’s grounding labels with a Thai board-certified physician’s.

Limitations

  • The study covers few outpatient encounters, one SLM, and Thai only; its LLM judges were validated on limited encounters against one board-certified physician, precluding inter-rater agreement.The authors also note that real physician user studies of generated results are lacking.Source: Limitations · p. 9
  • Synthetic transcripts may not capture real conversation diversity and complexity, use physician-written notes as seed profiles, and have no established optimal mixing strategy with real data.The synthetic encounters were re-summarized to prevent leakage; naive real–synthetic mixing yielded inconsistent improvements.Source: Limitations · p. 9
  • ClaimEntailment and SOAPPrecision also serve as GRPO rewards, so evaluation partly reuses models’ training signals.Source: Limitations · p. 9
  • ASR errors substantially degrade performance, particularly for GRPO-trained models, motivating more robust handling of noisy transcripts.Source: Limitations · p. 9

D.2 SOAPPrecision Judge

  • Cohen’s κ was 0.696 between the LLM judge and a Thai board-certified physician independently labeling each generated SOAP statement for factual grounding.The physician used the same evaluation protocol as the LLM judge, assessing generated SOAP notes against the reference doctor–patient conversation.Source: §D.2 · p. 25

E Additional Training Details

This section specifies the fine-tuning and GRPO configurations used for Gemma-4-E4B-it and explains how the significance reward handles facts matched to multiple references.

E Additional Training Details

  • Gemma-4-E4B-it was supervised-fine-tuned with LoRA for six epochs, selecting the checkpoint with the lowest validation loss.Training used sequence length 8,192, learning rate 9 × 10−5, and batch size 16.Source: E Additional Training Details · p. 27
  • GRPO training used 300 optimization steps, eight sampled generations per prompt, and a maximum completion length of 4,096 tokens.Reward weights were λformat = 3, λAF = 2, λsig = 1, λcomp = 4, and λfact = 3.Source: E Additional Training Details · p. 27
  • For the significance reward, a generated fact is correct if its predicted label matches any reference fact matched to it.The matched reference facts are represented as C(i).Source: E Additional Training Details · p. 27

F Additional Details on ASR Robustness Experiments

The ASR robustness experiment compares physician-aligned LLM-judge metrics on manually transcribed and ASR-generated ThaiClinicBench transcripts.

F Additional Details on ASR Robustness Experiments

G Qualitative Results

The qualitative examples show which clinical details different prompting and training settings preserve. The accompanying data-governance statement describes consent, de-identification, and controlled research access.

G Qualitative Results

  • In the first example, Gemini 3.1 Pro with the structured prompt entailed 8/11 facts, compared with 6/11 for free-form CoT and 7/11 for real-data GRPO.The structured prompt retained chemical exposure and normal limb strength, while the GRPO note missed the two-day symptom duration and full-day chemical exposure.Source: G Qualitative Results · p. 27, Table 8 · p. 28, Table 9 · p. 29
  • In the second example, real-data GRPO entailed 22/28 facts, synthetic-only GRPO 21/28, and ClinicEval 17/28.The real-data model retained pain during phonation and a follow-up instruction about tonsillar swelling or exudate; the synthetic-only model missed contextual details.Source: G Qualitative Results · p. 27, Table 11 · p. 30, Table 12 · p. 31

H Ethics & Data Governance Statement

Loading 2610.01234v1…