PaperlayerView exact revision on arXiv

Source-linked AI summary

ECG-Scroll: A Long-Horizon, Streaming Benchmark and Agent Environment for Interpretation of Ambulatory Electrocardiograms

Haitao Li, Chenglin Li, Zhengyao Ding, Ziyu Li, Yiheng Mao, Zhengxing Huang

arXiv:2609.33117v1cs.LGcs.AIcs.CLeess.SP

TL;DR

Long ambulatory ECGs contain brief events in streams that cannot be interpreted all at once, yet existing ECG MLLMs focus on short, fully visible recordings. ECG-Scroll provides a streaming benchmark and agent environment, and its evaluations show that models can recognize prompted events but struggle to decide autonomously when to scan and commit findings.

  • Problem

    Long ambulatory ECGs contain brief, clinically decisive events in streams that cannot fit one diagnostic-resolution context, while existing ECG resources focus on short, fully visible recordings.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    ECG-Scroll streams annotated ambulatory ECGs chunk by chunk in a causal benchmark and fixed agent environment with objective verifiers, timestamped memory, and signal-grounded tools.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Models recognize and commit events when prompted but fail to choose when to scan and write on their own, a planning collapse that persists across scales and reasoning effort.

    Source: §5 · pp. 8–9, §6 · p. 9

  • Takeaways & Limitations

    ECG-Scroll makes autonomous stop-and-commit control over streaming signals measurable as a long-horizon agent capability.

    Source: §6 · p. 9

Abstract

from arXiv · show

Multimodal large language models (MLLMs) can now interpret a standard ten-second, twelve-lead electrocardiogram (ECG) with clinically grounded, reward-verified reasoning. Real cardiac monitoring is different. Ambulatory (Holter) and telemetry recordings span hours to days and are read as they stream in, and their clinically decisive findings are paroxysmal, brief episodes buried in an otherwise unremarkable trace. Such a recording cannot be held in one context at diagnostic resolution, and its future has not yet happened, so a reader must work online, deciding what to measure now, committing evidence to memory as it passes, and reporting events as they occur. We recast long-duration ECG interpretation as a long-horizon, online (streaming, causal) sequential decision process and introduce ECG-Scroll. As a benchmark, long ambulatory recordings are streamed to an agent chunk by chunk, and it must localize, quantify, and promptly flag paroxysmal events without access to future signal; because the underlying signal is retained, every answer is checkable against objective ground truth, giving rule-based rather than judge-based rewards, and the streaming formulation adds a metric batch evaluation cannot express, the detection latency between an event's onset and the moment the agent records it. As an agent environment, it is a fixed, gym-style interaction layer that exercises three competencies single-glance ECG models never touch: Memory, Tool use through signal-grounded measurement rather than reading pixels, and Planning of what to measure now and when to commit. We release 390 whole-recording instances spanning 2,536 hours of two-lead ambulatory ECG and evaluate a signal-threshold rule agent alongside off-the-shelf LLM agents online, characterizing how they use memory, tools, and planning and where the benchmark's head-room lies.

1 INTRODUCTION

Long-duration ambulatory ECGs contain brief, decisive events in streams too long for one-shot interpretation, requiring online decisions about what to measure, remember, and report. ECG-Scroll contributes a verifiable streaming benchmark and agent environment to evaluate those demands.

1 INTRODUCTION

  • Brief paroxysmal findings in hours-to-days ambulatory recordings are buried in otherwise unremarkable traces and cannot be interpreted in one fixed context or single glance.Telemetry is read as it streams, and the clinical value of a finding degrades with reporting delay.Source: §1 · pp. 1–2
  • Streaming makes memory, tool use, and planning necessary: agents must retain passing evidence, measure waveforms, and choose when and how finely to scan.The signal's future is unavailable, and elapsed signal may be expensive or impossible to revisit.Source: §1 · p. 2
  • ECG-Scroll evaluates a benchmark and fixed agent environment designed to make memory, signal-grounded tool use, and planning necessary and measurable.The benchmark provides rule-based rewards and detection latency; the environment provides causal actions and signal and diagnostic tools.Source: §1 · p. 2
  • 390 whole-recording instances spanning 2,536 hours across five databases form a diverse streaming benchmark with rule-based verification and a detection-latency metric.Tasks span AF episode detection and burden, ischemia, ventricular ectopy, rare-event search, and change detection.Source: §1 · p. 2
  • The agent environment uses causal actions and clinician-ordered signal and diagnostic tools, while the empirical study evaluates a signal-threshold rule agent and off-the-shelf LLM agents online.The study also tests LLM agents with and without diagnostic tools.Source: §1 · p. 2

2 RELATED WORK

Prior ECG MLLMs ground diagnoses in waveform evidence but generally read short, fully visible recordings in one pass. ECG-Scroll brings memory, tool use, and planning into a causal streaming setting where the recording itself makes those capabilities unavoidable.

2.1 ECG MULTIMODAL LLMS AND REASONING

  • Recent ECG MLLMs build on waveform–clinical-text alignment, yet GEM, ECG-R1, PULSE, and anyECG-chat use differing methods; all read short, fully-visible resting ECGs in a single forward pass.GEM uses a dualencoder over ECG time series, image, and text; ECG-R1 (Jin et al., 2026) trains on a protocol-guided corpus with diagnostic-evidence reinforcement learning. PULSE (Liu et al., 2024b) instruction-tunes rendered ECG images, while anyECG-chat handles flexible-length, multi-record input.Source: §2.1 · pp. 2–3
  • ECG-Scroll targets the gap left by ECG MLLMs that interpret short, fully visible resting ECGs in a single pass rather than streaming recordings with callable measurements and online planning.Earlier systems include GEM, ECG-R1 (Jin et al., 2026), PULSE (Liu et al., 2024b), and anyECG-chat.Source: §2.1 · pp. 2–3

2.2 AGENTIC LLMS: MEMORY, TOOLS, AND PLANNING

  • ECG-Scroll adapts the agent-literature triad of planning, memory, and tool use to streaming ECG, where the past must be remembered rather than reread and event timing is objectively verifiable.The ECG agent observes elapsed signal through measurements and commits cursor-timestamped findings to memory.Source: §2.2 · p. 3, Figure 1 · p. 3

3 THE ECG-SCROLL BENCHMARK

ECG-Scroll repurposes annotated ambulatory recordings as a streaming benchmark with an online causal formulation, whole-recording instances, and objective task verifiers. Its evaluation captures event reporting latency as well as whether events are detected.

3 THE ECG-SCROLL BENCHMARK

  • Existing ECG QA resources use short, fully visible resting ECGs, while annotated long-duration ambulatory recordings had not been used to evaluate MLLMs or agents.ECG-Scroll bridges the resources by using episode- and beat-level annotations as time-stamped ground truth for online reading.Source: §3 · p. 3

3.1 PROBLEM FORMULATION

  • Each problem instance pairs a recording over [0, T] with a hidden timeline of clinical events, including event type, lead, and onset/offset interval.The recording is revealed chunk by chunk rather than handed over in full.Source: §3.1 · pp. 3–4
  • The agent acts causally on elapsed signal while the future remains unobservable, using a monotonic cursor and structured memory; the event timeline is hidden.The agent's observation includes elapsed signal, the last tool result, and memory.Source: §3.1 · p. 4
  • SKIM advances the cursor by a chosen duration, MEASURE runs a chosen tool over elapsed signal, and WRITE commits a timestamped finding for the final answer.Skim granularity is the core planning lever; rewind is costed, and a strict variant forbids it, making memory the only access to the past.Source: §3.1 · p. 4
  • Detection latency measures the delay from an event's onset to its first matching timestamped memory entry at temporal IoU ≥ τ; unrecorded events count as missed.This rule-based metric captures a streaming outcome that batch evaluation cannot produce.Source: §3.1 · p. 4

3.2 DATA AND STREAMING

  • The suite uses real two-lead ambulatory recordings from five PhysioNet databases, streamed in fixed-length chunks of 30 seconds by default.Tools access the retained signal for elapsed spans through WFDB rather than requiring image rendering.Source: §3.2 · pp. 4–5
  • 390 whole-recording instances across 247 recordings and 2,536 hours span five databases, with AF, ischemia, ventricular ectopy, rare-event search, and change-detection tasks.The AF core comprises 214 episode-detection and AF-burden instances; AF episodes are sparse and brief, with median duration near 16 seconds, and recording-level burden is strongly U-shaped.Source: §3 · p. 3, §3.2 · p. 5

3.3 TASK SUITE AND VERIFIERS

  • The task suite uses objective rule-based verifiers: episode and ischemia detection use temporal F1, burden uses relative error, and change detection and rare-event search use localization tolerance.No model-as-judge is used.Source: §3.3 · p. 5

4 THE ECG-SCROLL AGENT

ECG-Scroll’s fixed, gym-style environment defines how agents interact with streaming recordings through causal actions, clinically ordered measurement tools, diagnostic tools, and a typed memory ledger.

4 THE ECG-SCROLL AGENT

  • The fixed, gym-style interaction layer separates how agents interact with recordings from what the benchmark evaluates.It is designed for policies ranging from the studied rule and LLM agents to future trained models.Source: §4 · p. 5

4.1 ACTIONS

  • SKIM advances the cursor and returns a lossy profile, MEASURE runs a tool on elapsed signal, and WRITE commits a timestamped finding to memory.Choosing the SKIM duration—from coarse coverage to fine localization—is the core planning lever; rewinds can be costed or forbidden.Source: §4.1 · pp. 5–6

4.2 SIGNAL TOOLS: A CLINICIAN’S READING ORDER

  • Signal tools follow a clinician’s reading order and return numeric measurements, not diagnostic labels, so agents must reason from measurements to conclusions.The tools cover rate and rhythm, P waves, QRS complexes, ST segments, T waves, and intervals; they can return compact summaries or per-beat arrays.Source: §4.2 · p. 6

4.3 DIAGNOSTIC TOOLS

  • Diagnostic tools return ready-made classifications, and comparing agents with versus without them tests whether labels improve online localization and latency.Without diagnostic tools, agents must reason from measurements alone.Source: §4.3 · p. 7

4.4 MEMORY

  • The typed findings ledger records each WRITE with its type, lead, interval, value, confidence, and cursor timestamp, making memory inspectable and detection latency measurable.Because elapsed signal is costly or impossible to revisit, an event not written down as it passes is effectively lost.Source: §4.4 · p. 7

5 EXPERIMENTS

The experiments compare self-directed scanning with chunk-by-chunk judging, using shared tools and verifiers. Models can recognize events when coverage is handled for them, but struggle to plan and commit findings across long recordings.

5 EXPERIMENTS

  • The study compares Planning agents, which choose where to scan and when to write, with Judge agents, which classify elapsed chunks while the environment handles coverage.A Rule Agent provides a signal-grounded measurement-plus-threshold reference; it is not an upper bound, and language models exceed it on a few cells.Source: §5 · pp. 7–8
  • Planning agents remain near the floor on long-horizon tasks, while diagnostic labels do not rescue episode detection, ischemia, change detection, or AF burden.They perform better on PVC burden and rare-event search when a few well-placed commits suffice; scores are around 0.5–0.6 for PVC burden and up to 0.46 for rare-event search under +dx.Source: §5 · p. 8
  • With planning removed and diagnostic labels available, Judge agents recover AF burden and episode detection and reach the signal-grounded reference on ischemia and rare-event search.Without the diagnostic label, Judge agents collapse and flood nearly every chunk as an event.Source: §5 · pp. 8–9, Table 4 · p. 8
  • Across tasks, planning agents often skim or measure but leave half to two-thirds of recordings without a committed finding.DeepSeek instead overcommits on long atrial-fibrillation recordings, but its writes mostly fail the overlap criterion.Source: §5 · p. 9, Figure 3 · p. 9

6 CONCLUSION

ECG-Scroll frames long-recording ECG interpretation as an online, causal decision process and provides a benchmark and agent environment. The experiments identify autonomous stop-and-commit control as the skill models most lack.

6 CONCLUSION

  • ECG-Scroll combines a benchmark with objective verifiers and a fixed agent environment for studying memory, tool use, and planning in long-recording ECG interpretation.Models recognize and commit events when the environment handles coverage, but struggle to decide when to look and write across scales and reasoning effort.Source: §6 · p. 9

A ADDITIONAL DATASET STATISTICS

Figure 4 summarizes instance coverage across five task types and five databases, alongside how much of each stream is occupied by target events.

A ADDITIONAL DATASET STATISTICS

  • Target events occupy only a small fraction of each stream, with rare-event targets approaching instantaneous duration.The figure description contrasts AF and ischemic ST episodes with near-instantaneous rare-event targets.Source: A ADDITIONAL DATASET STATISTICS · p. 12, Figure 4 · p. 12
  • Figure 4(a) counts instances across five task types and five databases.The supplied description gives the dimensions of the coverage display, not the individual counts.Source: Figure 4 · p. 12, A ADDITIONAL DATASET STATISTICS · p. 12
  • Figure 4(b) plots the fraction of each stream occupied by target events on a log axis.The target-event fractions range from AF and transient ischemic ST episodes to near-instantaneous rare events.Source: Figure 4 · p. 12

B TASK VERIFIERS

Task verifiers use objective rules to score interval detection, burden estimates, change points, ischemia episodes, and rare-event localization.

B TASK VERIFIERS

  • Predicted intervals match ground truth when temporal IoU is at least τ=0.5 by default.The verifier scores matched intervals using task-specific rules.Source: B TASK VERIFIERS · p. 12
  • Episode detection and localization outputs intervals and uses temporal F1 over IoU-matched AF episodes.The task is evaluated on AFDB and LTAFDB.Source: B TASK VERIFIERS · p. 13
  • Burden quantification outputs a scalar, with AF burden evaluated on AFDB and LTAFDB and PVC burden on MITDB and LTDB.Examples include time-in-AF divided by recording duration and the ventricular ectopic beat fraction.Source: B TASK VERIFIERS · p. 13
  • Change detection scores a predicted change point as correct when it falls within δ=10 s of the ground-truth time.The example is sinus-to-AF onset, evaluated on AFDB.Source: B TASK VERIFIERS · p. 13
  • Ischemia detection uses temporal F1 for ST-episode intervals with stricter IoU τ=0.75, and a richer variant scores lead and ST magnitude.The task is evaluated on EDB; the richer variant measures |∆ST| within ϵ mV.Source: B TASK VERIFIERS · p. 13
  • Rare-event search localizes a near-instantaneous beat by midpoint tolerance, scoring a hit within δ=15 s.The passage notes that IoU is unreachable at streaming chunk resolution for an approximately 1 s target.Source: B TASK VERIFIERS · p. 13

C AGENT PROMPTS AND ACTION INTERFACE

The agent prompts distinguish fixed-chunk judging from agent-controlled scanning, while both configurations commit findings to a shared memory ledger for scoring.

C AGENT PROMPTS AND ACTION INTERFACE

  • The Planning agent chooses each cursor advance, tool call, measurement window, and finding to write while scanning elapsed signal only.It can skim, measure, write, or finish; under +dx, its measure action can also select diagnostic tools.Source: C AGENT PROMPTS AND ACTION INTERFACE · p. 13
  • Planning prompts prescribe large skims over quiet stretches and small skims or measurements around peaks before confirming and writing an event span.The prompt advises extending boundaries before and after an event to identify its start and end.Source: C AGENT PROMPTS AND ACTION INTERFACE · p. 13
  • The Judge agent receives fixed 30-second chunks in time-ordered batches and flags chunks it believes contain an event.It does not control the cursor and makes one flag-chunks call per batch.Source: C AGENT PROMPTS AND ACTION INTERFACE · pp. 13–14
  • The Judge prompt directs AF episode flagging from ready-made rhythm-class and AF-fraction labels, while the signal-only configuration provides measurements for model reasoning.Both configurations commit flagged or written spans to the same timestamped memory ledger and are scored identically.Source: C AGENT PROMPTS AND ACTION INTERFACE · p. 14

D SUCCESSFUL PLANNING TRAJECTORIES

On PVC burden, the Planning agent demonstrates a coarse-to-fine scan-and-write loop, though signal-only performance still underestimates the true burden.

D SUCCESSFUL PLANNING TRAJECTORIES

  • 0.96 score with predicted burden 0.0078 versus true 0.0070 on MIT-BIH record 118 with diagnostic tools; signal-only scoring was 0.83 with predicted burden 0.0055 versus true 0.0089 on record 202.The two examples show successful burden estimation with different tool configurations and records.Source: D SUCCESSFUL PLANNING TRAJECTORIES · p. 14
  • The diagnostic-tool trajectory measures elevated peaks with beat classification, then writes tight intervals around returned PVC times.On MIT-BIH record 118, it skips writing when the tool reports no ventricular beats.Source: D SUCCESSFUL PLANNING TRAJECTORIES · p. 14
  • The signal-only trajectory uses elevated ectopic counts or wide-beat fraction as cues and writes around returned ectopic times.The example includes a measurement with ectopic_count=1 and wide_frac=0.009 at 476.6 seconds.Source: D SUCCESSFUL PLANNING TRAJECTORIES · p. 14
  • Both runs use adaptive granularity and task-appropriate tools; the reported gap to the Rule Agent is recall among beats, not failure of the scan loop.The described pattern is coarse skimming over quiet stretches, fine measurement at peaks, and a tight commit per confirmed beat.Source: D SUCCESSFUL PLANNING TRAJECTORIES · pp. 14–15

Related papers

Loading 2609.33117v1…