PaperlayerView exact revision on arXiv

Source-linked AI summary

ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation

Jiarui Jin, Haoyu Wang, Xingliang Wu, Xiaocheng Fang, Xiang Lan, Zihan Wang, Deyun Zhang, Bo Liu, Yingying Zhang, Xian Wu, Hongyan Li, Shenda Hong

arXiv:2602.04279v3cs.CL

TL;DR

Existing medical MLLMs remain unreliable for ECG interpretation, producing hallucinated analyses and struggling with missing signal or image modalities. ECG-R1 combines protocol-guided corpus generation, a modality-decoupled architecture with Interleaved Modality Dropout, and evidence-rewarded reinforcement learning. The paper reports improved diagnostic accuracy and interpretation quality over prior state-of-the-art methods, while emphasizing qualified clinical oversight.

  • Problem

    Existing medical MLLMs have inadequate ECG diagnostic reliability because interpretation data can contain clinical errors and models struggle with signal-image modality differences.

  • Method

    ECG-R1 combines protocol-guided instruction generation, a modality-decoupled architecture with Interleaved Modality Dropout, and reinforcement learning with ECG Diagnostic Evidence Rewards.

  • Results

    ECG-R1 consistently outperforms prior state-of-the-art methods in diagnostic accuracy and interpretation quality while remaining stable under varying data completeness.

  • Takeaways & Limitations

    The study supports using ECG-R1 as a more reliable research model for evidence-grounded ECG interpretation, while current MLLM outputs require independent clinical verification.

  • Takeaways & Limitations

    ECG-R1 is intended for research use under qualified clinical oversight, and real-world deployment requires prospective multi-site validation and clinical governance.

Abstract

from arXiv · show

Electrocardiography (ECG) serves as an indispensable diagnostic tool in clinical practice, yet existing multimodal large language models (MLLMs) remain unreliable for ECG interpretation, often producing plausible but clinically incorrect analyses. To address this, we propose ECG-R1, the first reasoning ECG MLLM designed for reliable ECG interpretation via three innovations. First, we construct the interpretation corpus using \textit{Protocol-Guided Instruction Data Generation}, grounding interpretation in measurable ECG features and monograph-defined quantitative thresholds and diagnostic logic. Second, we present a modality-decoupled architecture with \textit{Interleaved Modality Dropout} to improve robustness and cross-modal consistency when either the ECG signal or ECG image is missing. Third, we present \textit{Reinforcement Learning with ECG Diagnostic Evidence Rewards} to strengthen evidence-grounded ECG interpretation. Additionally, we systematically evaluate the ECG interpretation capabilities of proprietary, open-source, and medical MLLMs, and provide the first quantitative evidence that severe hallucinations are widespread, suggesting that the public should not directly trust these outputs without independent verification. Code is available at \href{https://github.com/PKUDigitalHealth/ECG-R1}{here}.

1. Introduction

Existing medical MLLMs remain unreliable for ECG interpretation because they produce hallucinated analyses and struggle with signal-image modality differences. ECG-R1 addresses these problems with protocol-guided data generation, modality-robust training, and evidence-rewarded reasoning.

  • Incorrect ECG interpretations can lead to inappropriate treatment and severe health consequences, motivating reliable automated interpretation.
  • Existing medical MLLMs show inadequate diagnostic reliability, while their outputs may appear clinically complete despite hallucinated statements and low diagnostic accuracy.
  • Most current medical MLLMs analyze ECG images but not signals, and modality-missing conditions cause performance degradation and cross-modal inconsistency.
  • ECG-R1 generates structured interpretations from physiological features and monograph-defined thresholds and diagnostic logic.
  • ECG-R1 applies reinforcement learning with ECG Diagnostic Evidence Rewards to optimize structured intermediate reasoning alongside final-answer correctness.
  • ECG-R1 uses Interleaved Modality Dropout to simulate missing modalities and token-order changes, with theoretical guarantees for robustness and cross-modal consistency.

2. Method

ECG-R1 combines protocol-grounded corpus construction, decoupled multimodal encoding, Interleaved Modality Dropout, and evidence-based reinforcement learning for reliable ECG interpretation.

  • Framework overview: ECG-R1’s framework comprises instruction corpus curation, architecture design, and a training strategy for multimodal ECG interpretation.The model takes instruction text, an ECG image, and an ECG time-series signal, then generates an interpretation.
  • Protocol-guided data generation: Protocol-Guided Instruction Data Generation extracts structured physiological evidence and uses standardized monograph-derived ECG protocols to generate grounded supervision.The extractor measures 14 feature sequences per lead across 12 leads, including intervals, amplitudes, morphology, ST descriptors, and QT/QTc values.
  • Protocol-guided data generation: The curated responses follow a fixed schema containing a six-step protocol reasoning trace, summary narrative, and final diagnosis.The six steps comprise five protocol phases plus a final medical reasoning step.
  • Decoupled modalities encoding: ECG-R1 decouples image and time-series processing by using separate encoders, projectors, and modality-specific token positions.Time-series tokens are injected at an explicit <ecg> tag, while image tokens are injected at <image>, avoiding reliance on paired modalities and a shared projector.
  • Interleaved Modality Dropout: Interleaved Modality Dropout samples modality removal and token-block order swaps during training, optimizing mixture risk across test-relevant environments.Theoretical results bound worst-environment risk by mixture risk under a coverage assumption and establish robustness and consistency guarantees in the ECG setting.
  • Evidence-based reinforcement learning: ECG Diagnostic Evidence Rewards extend reinforcement learning beyond format and final-answer correctness by rewarding structured intermediate reasoning grounded in ECG evidence.The RL stage uses protocol-structured targets with a six-step interpretation trace, summary, and final diagnosis.

3. Experiments

ECG-R1 is evaluated on grounded interpretation, modality-missing robustness and consistency, and cardiologist-rated reliability. It outperforms prior ECG-specialized models while reducing hallucination-related errors and maintaining stronger cross-modal agreement.

  • Grounded ECG Interpretation: Existing non-ECG-specialized MLLMs produce structurally complete analyses but remain clinically unreliable, with GPT-5.1 reaching only 31.48 Diagnosis Accuracy and medical MLLMs generally below 30.00.Their weaker Analysis Relevance, Lead Evidence Validity, and ECG Feature Grounding reveal systematic hallucinations beneath superficially comprehensive outputs.
  • Grounded ECG Interpretation: ECG-R1 improves grounded interpretation over GEM, reaching 80.29 Diagnosis Accuracy, 6.51 Analysis Completeness, 4.74 Analysis Relevance, and 5.81 Lead Evidence Validity.It also achieves a +17.49 average absolute gain over GEM on ECG Feature Grounding, Evidence-Based Reasoning, and Clinical Diagnostic Fidelity.
  • Robust and Consistent ECG Interpretation: Under modality missing conditions, GEM loses up to 28.0% relative Diagnosis Accuracy and 44.9% relative Analysis Relevance when only the time-series modality remains.ECG-R1 shows consistently smaller relative drops and still surpasses GEM with both modalities available.
  • Robust and Consistent ECG Interpretation: ECG-R1 achieves stronger cross-modal consistency than GEM, whose BLEU-4 and ROUGE-L scores are 0.33 and 0.43, while ECG-R1 reaches an SBERT-Score of 0.97.The comparison uses interpretations generated from time-series-only and image-only inputs on the same cases.
  • Cardiologist Evaluation: Four licensed cardiologists rate ECG-R1 more favorably than GEM across reliability criteria, with stronger relevance and completeness and fewer severe factual errors.Cardiologist evaluation covered 100 randomly sampled test cases and predefined clinical criteria.

4. Conclusion

ECG-R1 targets reliable ECG interpretation through protocol-guided data generation, modality dropout, and evidence-rewarded reinforcement learning. Experiments show consistent gains over prior methods and stability when modalities are missing.

  • Conclusion: ECG-R1 combines Protocol-Guided Instruction Data Generation, Interleaved Modality Dropout, and ECG Diagnostic Evidence Rewards to improve ECG interpretation reliability.The system grounds interpretations in structured clinical logic, supports incomplete inputs, and strengthens evidence-based reasoning.
  • Conclusion: ECG-R1 consistently outperforms prior state-of-the-art methods in diagnostic accuracy and interpretation quality while remaining stable under modality missing conditions.The paper presents this as progress toward reliable ECG interpretation.

Impact Statement

The paper addresses the clinical and research implications of unreliable ECG interpretation by MLLMs, while emphasizing that deployment remains bounded by safety, validation, and governance requirements.

  • Potential Impact: The work aims to reduce hallucinated interpretations and improve report consistency under real-world data imperfections if developed and validated responsibly.It also supports research on safer AI-assisted ECG workflows.
  • Impact and Limitations: Incorrect or incomplete ECG interpretations may affect downstream clinical assessment and patient management in this high-stakes medical application.This motivates treating model outputs as auxiliary references rather than autonomous clinical decisions.
  • Impact and Limitations: The proposed methods and models are intended for research purposes and require qualified clinical oversight, verification, and final clinician decision-making.The paper also identifies prospective multi-site validation, governance, training, monitoring, and regulatory review as deployment requirements.

A.2. Language-based ECG Analysis

Language-based ECG interpretation remains an early research area with only a small set of recent MLLM approaches. Existing systems span time-series analysis, image understanding, multimodal fusion, and related ECG tasks.

  • Language-based ECG Analysis: Recent ECG MLLMs include ECG-Chat for time-series report generation, PULSE for ECG-image diagnosis, and anyECG-Chat for reporting, waveform localization, and open-ended QA.GEM performs cross-modal fusion, while UniECG supports evidence-based interpretation and signal generation.
  • Language-based ECG Analysis: Prior approaches relying on pretrained general-purpose LLM knowledge for data generation may suffer hallucinations, while ECG omni-perception models still show limitations.These concerns motivate more grounded and reliable ECG interpretation methods.

B. Experiment Details

The experiments evaluate ECG interpretations using diagnosis, grounding, reasoning, consistency, and cardiologist-oriented reliability and usefulness metrics, with specified training settings for ECG-R1.

  • Automated evaluation metrics: DiagnosisAccuracy (0–2) measures whether generated diagnoses are correct, specific, and supported by ECG findings.Sub-items with scores above zero are counted as correct, and the reported value is their percentage among all sub-items.
  • Automated evaluation metrics: GroundedECGUnderstanding, EvidenceBasedReasoning, and ClinicalDiagnosticFidelity each use 0–100 scales to assess ECG evidence, logical support, and structured clinical interpretation.Their rubrics distinguish comprehensive, clear, and absent evidence or reasoning.
  • Consistency metrics: BLEU-4, ROUGE-L, and SBERT-Score measure cross-modal agreement through local n-gram overlap, sequence-level overlap, and embedding-based semantic similarity.Higher values indicate stronger agreement under the respective lexical, structural, or semantic criterion.
  • Cardiologist evaluation: Cardiologist evaluations rate analytical relevance, accuracy, completeness, reasoning quality, findings novelty, clinical value, and overall satisfaction on 1–5 scales.The rubrics assess factual correctness, ECG feature coverage, clinician-like reasoning, decision support, and perceived output quality.
  • Training settings: Training uses one SFT epoch with learning rate 2e-5, per-device batch size 4, gradient accumulation of 2, frozen ECG and image encoders, and IMD probabilities ps=0.1 and pd=0.5.The RL stage trains only the LLM component while keeping the encoders frozen.

C.1. Dataset Comparison

The dataset comparison contrasts ECG-Grounding with ECG Protocol-Guided Grounding CoT using qualitative and quantitative evaluations under controlled training conditions.

  • Qualitative comparison: The protocol-guided corpus avoids fundamental medical errors and incomplete diagnostic coverage found in ECG-Grounding qualitative examples.The comparison highlights omissions involving intervals, conduction, axis assessment, and ischemia–infarction differentiation.
  • Quantitative comparison: ECG Protocol-Guided Grounding CoT consistently outperforms ECG-Grounding across all seven metrics after SFT on identical samples with matched training-set sizes, architecture, and hyperparameters.No RL training is applied in this comparison.
  • Quantitative comparison: 74.32 to 79.33 (+5.01) is the diagnosis accuracy improvement from ECG-Grounding to ECG Protocol-Guided Grounding CoT.The largest reported gain is clinical diagnostic fidelity, increasing from 62.98 to 83.51 (+20.53).

C.3. ECG-Bench Results

On ECG-Bench abnormality detection, generic proprietary MLLMs perform near randomly, while ECG-R1 remains competitive with ECG-specialized methods despite not being explicitly optimized for this task.

  • Overall results: Across all five ECG-Bench datasets, proprietary MLLMs exhibit near-random performance, indicating that generic multimodal capabilities are insufficient for accurate abnormality detection.The evaluation uses macro AUC, macro F1, and hamming loss for multi-label datasets, and accuracy for other datasets.
  • Overall results: ECG-R1 achieves new state of the art on CSN and G12EC while showing only marginal gaps to the best methods on the remaining datasets.The comparison includes ECG-specialized models such as ST-MEM, METS, MERL, PULSE, and GEM.

C.4. Ablation Study on Interleaved Modality Dropout

The ablation study shows that Interleaved Modality Dropout improves ECG-R1 robustness when images are missing and increases agreement between image-only and signal-only interpretations.

  • Robustness under modality missing: 36.77 to 77.91 is the modality-missing performance change when IMD is enabled, a robustness recovery gain of 41.14.Without IMD, performance collapses when the image modality is absent; with IMD, only time-series signals are provided.
  • Robustness under modality missing: A 1.7 peak-performance trade-off accompanies IMD, yielding a 24.2x efficiency ratio between robustness recovery gain and peak-performance reduction.Without IMD, omni-modality inputs achieve slightly higher peak accuracy.
  • Cross-modality consistency: With IMD, cross-modality consistency reaches BLEU-4 0.69, ROUGE-L 0.73, and SBERT-Score 0.97 across image-only and time-series-only interpretations.These scores are averaged over the entire test set and indicate modality-invariant semantic agreement.
  • Overall effect: IMD improves both robustness and cross-modality consistency, with empirical gains consistent with the paper’s theoretical analysis.The authors characterize IMD as a training strategy for reliability under modality-missing conditions.

C.5. Ablation Study on ECG Diagnostic Evidence Rewards

EDER improves both diagnostic accuracy and the quality of ECG interpretations during reinforcement learning, while stabilizing generation behavior and preserving evidence statements.

  • EDER consistently improves terminal diagnosis accuracy and multiple evidence-grounded interpretation metrics.The reported gains include analysis completeness and relevance, lead evidence validity, ECG feature grounding, evidence-based reasoning, and clinical diagnostic fidelity.
  • Without EDER, mean rollout length progressively contracts during RL training.This pattern is associated with omitting key diagnostic evidence statements when optimization is dominated by terminal diagnostic correctness.
  • With EDER, rollout length remains more stable, encouraging preservation of salient evidence statements.Process-level rewards impose sustained constraints on evidence-driven interpretation and mitigate length collapse.
  • EDER generally lowers output entropy and reduces its fluctuations during RL training.The analysis attributes this behavior to clearer constraints on evidence-driven interpretation.

C.6. Numerical Analysis of the DiagnosisAccuracy Metric Evaluated by GPT-4o and DeepSeek v3.1 Terminus

The section argues that GPT-4o and DeepSeek v3.1 Terminus apply materially different DiagnosisAccuracy evaluation logic, with GPT-4o’s leniency potentially inflating scores and masking clinically important errors.

  • GPT-4o’s semantic-relevance-based scoring can tolerate fatal omissions and logically inconsistent over-diagnoses.Accurate secondary descriptions such as heart rate or rhythm may compensate for missing clinically important findings under this evaluation tendency.
  • GPT-4o’s binary treatment of any score above zero can amplify partial credits into a 100% sample-level accuracy rate.Aggregating multiple diagnostic points into one score creates a minimal denominator and masks bottlenecks in complex cases.
  • Representative cases show both evaluators identifying some correct rhythm or conduction findings while differing in their treatment of ischemia, infarction, and specificity.GPT-4o sometimes misses key ischemic findings or gives unclear diagnoses, while DeepSeek also identifies unsupported findings but evaluates them more granularly.
  • DeepSeek v3.1 Terminus decomposes diagnoses exhaustively, exposing unsupported or incorrect findings as zero-credit items.Examples include unsupported anterior ischemia, left main disease inferred from aVR, inferior infarct, and vague lateral ischemia diagnoses.
  • The authors conclude that DeepSeek v3.1 Terminus offers greater clinical reference value and statistical power despite lower absolute DiagnosisAccuracy values.The stated rationale is that lenient assessment can condone medical risks, whereas stricter decomposition better exposes clinically relevant errors.
  • ECG-R1’s remaining limitation is that rare clinically important conditions need targeted data curation and reward optimization.QT/QTc interval estimation also remains challenging because numerical supervision is uneven in the training corpus.

D. Complete Proofs for Section 2.4

The proof formalizes interleaved modality dropout as optimization over modality-missing and token-order environments, then relates mixture risk to worst-environment robustness and cross-view consistency.

  • IMD samples transformations over test environments by dropping modalities and optionally swapping modality-token block order.The transformation distribution includes modality-drop trials and token-order swap trials with probabilities p_d and p_s.
  • The induced mixture risk averages model risk across all sampled test environments.Training therefore evaluates multiple modality configurations rather than a single fixed input pattern.
  • The framework defines excess risk relative to Bayes-optimal environment-specific predictors and introduces view gaps for single-modality inputs.Intrinsic view gaps may be non-zero because the modalities can contain unequal information.
  • Pinsker’s inequality and Jensen’s inequality connect KL-based excess risk to total-variation consistency gaps.The proof applies these inequalities to compare predictions across modality views and token-block orders.
  • Under the coverage assumption q(τ) ≥ α for every test environment, the theorem bounds worst-environment risk using the mixture-risk formulation.The proof selects the highest-risk environment and applies the lower probability coverage condition.
  • If the model family cannot realize the Bayes-optimal conditional distributions, the stated excess-risk interpretation acquires an additional approximation term.The paper identifies this as a standard statistical-learning qualification.

E.1. Inference Results

The inference examples show ECG-R1 producing structured diagnoses from rhythm, conduction, voltage, ischemia, and interval evidence, while distinguishing conduction- or hypertrophy-related changes from acute pathology.

  • The reported answer for another case is sinus rhythm with left anterior fascicular block and left ventricular hypertrophy by voltage only.The accompanying analysis states that voltage criteria support LVH while precordial progression does not indicate additional abnormalities.
  • ECG-R1 reports normal QTc and no acute ischemic changes in cases where conduction abnormalities explain repolarization findings.One example gives QTc approximately 440 ms and describes no significant ST-segment threshold violations or pathologic Q waves.
  • ECG-R1 identifies sinus rhythm with PACs, left axis deviation, bifascicular block, and an abnormal ECG in one representative case.The diagnosis combines right bundle branch block with left anterior fascicular block and attributes abnormality to conduction delays.
  • Some generated analyses include conflicting interpretations, such as combining a conduction-based diagnosis with possible lateral ischemic changes.The passages present these findings as part of the inference outputs rather than as a single consistent diagnostic conclusion.
  • A separate case reports sinus tachycardia at 120 bpm with PACs, left axis deviation, inferior infarction evidence, and possible anterior myocardial infarction.The anterior finding is described as less definitive because significant Q waves are absent in the anterior leads.
  • Another analysis identifies atrial fibrillation from an irregularly irregular 118 bpm rhythm without organized P waves.The same case also reports RBBB, left anterior fascicular block, LVH by voltage criteria, and no acute ischemia or infarction.
Loading 2602.04279v3…