ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

• Existing resources leave gaps in multi-visit aggregation, clinical event provenance, and characterization of clinical long-context failures.Clinical benchmarks are dominated by single-encounter or single-document prompts (Kweon et al., 2024; Fleming et al., 2024; Ma et al., 2024; Ben Abacha et al., 2025); open-domain memory benchmarks lack clinical semantics and event-level provenance (Maharana et al., 2024; Wu et al., 2025; Hu et al., 2025). Long-context pathologies such as lost-in-the-middle attention (Liu et al., 2024) and miscalibrated abstention (Kadavath et al., 2022) have not been characterized by specific clinical cognitive operation.Source: §1 · p. 1
• ClinTraceBench comprises 385 verified MIMIC-IV-derived dialogues with event-ID provenance, nine clinical reasoning tasks, and L0–L4 deterministic plus L5 human-audit validation at 98.92% agreement.The tasks span fact recall, temporal trends, encounter-level linkage, retrospective propagation, contradiction, cross-patient comparison, treatment recall, observed treatment response, and abstention.Source: §1 · pp. 1–2
• Full context wins on pooled accuracy but is not uniformly best; dense retrieval is competitive at lower cost, while compressed and agentic memories show task-specific failures.Reported failures include aggregation tax, loss of finding–diagnosis relations, and abstention drift.Source: §1 · p. 2
• The evaluation compares eight representation strategies across four backbones on 6,271 questions, producing 32 cells and 200,672 predictions.The backbones are DeepSeek-V3, GPT-4o-mini, Haiku 4.5, and Sonnet 4.6.Source: §1 · p. 2
• A sentence-level T3 injection probe tests preservation with the attribution sentence inserted before construction for equal input and after construction for update or staleness.This design isolates signal preservation from backbone capacity and is presented as generalizable beyond clinical settings.Source: §1 · p. 2

EHR-derived clinical evaluation

• Prior EHR benchmarks use real chart data but are dominated by single-document or single-encounter prompts, leaving multi-visit aggregation, encounter attribution, and abstention under unstated facts untested.Examples include EHRNOTEQA, MEDALIGN (Fleming et al., 2024), CLIBENCH (Ma et al., 2024), MEDEC (Ben Abacha et al., 2025), DR.BENCH (Gao et al., 2023), and MEDHELM (Bedi et al., 2025).Source: §2 · p. 2

Temporal and longitudinal clinical reasoning

• Temporal benchmarks and studies cover clinical temporal extraction, multi-encounter instruction tuning, and longitudinal summarization or prediction, but lack answer-to-turn provenance and controlled signal-preservation probes.The cited work includes i2b2 temporal challenges (Sun et al., 2013), TIMER (Cui et al., 2025), and Kruse et al. (2025).Source: §2 · p. 2

Long-context and retrieval evaluation in NLP

• Long-context benchmarks and needle probes study retrieval and attention drift, while calibration and abstention research motivates ClinTraceBench’s clinical T9 task.LONGBENCH (Bai et al., 2024), Kamradt (2023), and Liu et al. (2024) address long-context behavior; RAG and dense-retrieval embeddings motivate an evaluated strategy family, and Kadavath et al. (2022) establish calibration and abstention as measurable capabilities.Source: §2 · p. 2

Memory benchmarks and agentic memory

• Conversational-memory benchmarks and agentic-memory methods motivate testing whether memory-style representations preserve enough longitudinal clinical signal to compete with full-context EHR feeding.The lineage includes LOCOMO (Maharana et al., 2024), LONGMEMEVAL (Wu et al., 2025), MEMORYAGENT-BENCH (Hu et al., 2025), Mem0 (Chhikara et al., 2025), AMEM (Xu et al., 2025), and MemGPT (Packer et al., 2023).Source: §2 · p. 2

Position of our work

• ClinTraceBench’s contribution is integrating longitudinal EHR trajectories, event-level provenance, nine reasoning tasks, layered validation, and controlled injection for paired history-representation comparisons.Validation includes L0–L4 deterministic checks and L5 human audit; the injection probe isolates representation-level signal preservation from backbone capacity.Source: §2 · pp. 2–3

Cohort

• 100 patients per disease were sampled across diabetes, hypertension, CKD, and CAD, with complexity-stratified selection and deterministic inclusion criteria.Each patient required at least two admissions more than seven days apart, five labs, two prescriptions, and two index-disease ICD codes; dialogues preserve structured fields within narrative turns.Source: §3 · p. 3

Task layers and suite

• The nine tasks separate evidence preservation and access (T1, T3, T7), integrative reasoning (T2, T4, T6, T8), and diagnostic or epistemic probes (T5, T9).They cover lab retrieval, trends, attribution, retrospective evidence propagation, contradiction detection, cross-patient comparison, treatment facts, post-treatment lab change, and abstention; T1/T3/T7 results are framed as fidelity rather than reasoning.Source: §3 · pp. 3–4

Benchmark construction

• Figure 2 reports 400 patients yielding 385 verified dialogues, 6,271 evaluation questions, and 200,672 predictions through the source-verifiable construction pipeline.The pipeline uses deterministic event IDs, validation gates, and L0–L4 checks plus an L5 human audit, which yielded 98.92% agreement.Source: §3 · p. 3, Figure 2 · p. 4

Task suite

• In the primary T3 preservation probe, the attribution sentence enters the dialogue before Mem0, A-Mem, or llm-summary is constructed.The post-construction setting instead tests staleness after sentence injection; the primary setting scores positive-class recall on injected positives.Source: §3 · p. 4

Task limitations

• T5, T8, and T9 have interpretation limits: T5’s trivial majority baseline is 0.919, T8 measures post-treatment lab change rather than causal response, and T9’s always-abstain baseline scores 100%.T5 and T9 are therefore used for relative ordering, not as absolute benchmarks.Source: §3 · p. 4

Evaluation set

• 6,271 stratified questions were fixed across all 32 cells, with pooled Wilson 95% CI ±0.6 pp and per-task minimum-detectable effects of 5.1–27.2 pp.T8 was weakly powered at 17.1 pp and T4 under-powered at 27.2 pp; all 12 headline comparisons remained significant after Holm–Bonferroni correction.Source: §3 · p. 4, Figure 2 · p. 4

Headline metrics

• Pooled accuracy covers all 6,271 questions, while macro accuracy averages nine task accuracies; pooled accuracy is dominated by T1 and T9 because task sizes are unbalanced.The paper directs representation comparisons that should not be task-mix-weighted to macro accuracy and the per-task surface.Source: §3 · pp. 4–5

History representation strategies

• Eight strategies cover no-context, last-visit, full-context, BGE-M3 retrieval, structured-timeline and llm-summary compression, plus Mem0 and A-Mem agentic memory.Dense retrieval uses top-K=5 complete-visit chunks; llm-summary uses a single 500-token DeepSeek summary. Mem0 stores extracted facts, while A-Mem uses atomic notes.Source: §4 · p. 5

Backbones

• The four answering backbones are DeepSeek-V3, GPT-4o-mini, Claude Haiku 4.5, and Claude Sonnet 4.6.Their context windows are 64k, 128k, 200k, and 200k, respectively.Source: §4 · p. 5

Memory prep model

• DeepSeek-V3 prepares Mem0 and A-Mem artifacts before questions are asked, holding construction-side extraction fixed across all four answering backbones.Thus, non-DeepSeek memory cells use DeepSeek-extracted artifacts with GPT-4o-mini, Haiku, or Sonnet answering.Source: §4 · p. 5

5.1 Main accuracy

• 67.2 / 63.0 / 70.6 / 69.8% pooled accuracy for full-context across DeepSeek-V3 / GPT-4o-mini / Haiku / Sonnet leads every backbone; dense-retrieval ranks second and structured-timeline third.Macro accuracy preserves this ordering; full-context, dense-retrieval, and structured-timeline also stay above compressed strategies on T5 under macro-F1, and structured-timeline leads on T8. The main ordering survives exclusions of T5, T8, T9, and the T3 probe, as well as rebuilding agentic memories with each answering backbone.Source: §5.1 · p. 5, Table 1 · p. 6

5.2 SP1: The aggregation bottleneck

• On T2, T6b / T6c, and T8, compressed strategies lose multi-value aggregation signal; on T7, full-context reaches 0.942–0.965 while Mem0 reaches 0.535–0.733 and llm-summary 0.349–0.512.Agentic memories do not store per-visit value pairs, 500-token summaries cannot encode multi-visit trajectories, and dense retrieval can still fail when aggregation crosses non-contiguous chunks.Source: §5.2 · p. 6

5.3 SP2: Backbone-specific extraction profiles

• +62.7 pp for Haiku, +49.6 pp for Sonnet, +43.6 pp for DeepSeek-V3, and +29.8 pp for GPT-4o-mini are the respective blind-to-full-context gaps.Haiku rises from 0.079 blind to 0.706 full-context; GPT-4o-mini has the highest blind score (0.332) and lowest full-context score (0.630), so context use varies by backbone rather than a single capacity dimension.Source: §5.3 · pp. 6–7

5.4 SP3: Non-monotonic abstention under longer context

• 0.994 vs. 0.907 for Haiku, 0.926 vs. 0.747 for Sonnet, 0.883 vs. 0.701 for DeepSeek, and 0.895 vs. 0.698 for GPT-4o-mini show last-visit-only abstention exceeds full-context.Full-context improves abstention over blind on every backbone, but the last-visit-only advantage is significant in pooled paired data (0.924 vs. 0.763, ∆=0.161, McNemar p = 1.7 × 10−38). T9 is a relative ordering because its trivial-baseline caveat limits absolute interpretation; fabricated values account for ≈54% of genuine full-context over-answers after parser-missed verbose abstentions are discounted.Source: §5.4 · p. 7

5.5 SP4: Controlled preservation probe: compressed representations drop the finding–diagnosis relation even under equal input

• At best 1/19 (5.3%) injected-positive T3 instances were recovered by Mem0 or A-Mem, and llm-summary recovered 0/19, even when attribution text was present before construction.Inspection indicates the lab fact is usually retained but its finding–diagnosis link omitted, a relation-level rather than access-level loss. This equal-input probe covers Mem0, A-Mem, and llm-summary only; structured-timeline was not run in this setting. In the secondary post-construction probe, all four upstream-built strategies answered “No” on all 600 questions in all 16 cells, scoring 0.500 on balanced gold; this measures update staleness, not discarding.Source: §5.5 · p. 8, Table 3 · p. 8

5.6 Retrieval, cost, and Pareto efficiency

• $25.76 and 0.706 for full-context × Haiku dominate $106.21 and 0.698 for full-context × Sonnet; structured-timeline remains off the 11-cell Pareto frontier despite 0.58–0.60 accuracy.BGE-M3 retrieval hit@5 is 0.926 across 4,771 retrieval-eligible questions; dense retrieval is within 1–4 pp of full-context accuracy in the main comparison.Source: §5.1 · p. 5, §5.6 · p. 8, Figure 6 · p. 9

5.7 Illustrative failure modes

• Four case studies distinguish T3 constant-No collapse, T9 over-answering, Haiku-specific T2 trend failure, and T6 cross-patient confusion as different failure modes.The cases illustrate why aggregate accuracy can obscure the distinct errors associated with different representation strategies.Source: §5.7 · p. 8

What compact representations get right

• Dense retrieval lands within 1–4 pp of full-context pooled and within 0–9 pp on every task except small-n T4; structured timelines lead numeric retrieval at T1 0.886–0.923.BGE-M3 surfaces chunk-localizable answers reliably; Mem0 and AMem hold their own on T4 and T9 when compression aligns with gold density.Source: §6 · p. 9

What compact representations get wrong

• Preprocessing can erase per-visit pairs needed for trends and deltas, while relation-preservation strategies recover only 0–5.3% of injected positives despite retaining the underlying lab fact.After construction, those strategies and structured-timeline reach the constant-No floor on T3, indicating that derived artifacts can also go stale.Source: §6 · p. 9
• T5’s trivial always-contradicts baseline scores 91.9%, no cell beats it, and full-context improves T9 over blind but degrades versus last-visit-only on every backbone.Trivial always-abstain scores 100% on T9 by construction.Source: §6 · p. 9

Cost flips “biggest backbone wins”

• Under full-context, Haiku—not Sonnet—is on the Pareto frontier, costing $25.76 versus $106.21 and scoring 0.706 versus 0.698.The discussion argues memory-equipped clinical agents should be benchmarked on the cost–quality frontier.Source: §6 · p. 9

Memory size ≠ accuracy

• Mem0’s total character budget is uncorrelated with patient-level T1 accuracy (Pearson r = −0.129), suggesting architecture matters more than budget.The discussion contrasts atomic facts, visit-level notes, and prose, and notes that more facts can introduce retrieval noise.Source: §6 · p. 9

Future work

• Future work includes multi-EHR replication, longer horizons, dynamic memory updates, real longitudinal notes and conversations, and extending the equal-input probe.The proposed probe extension covers structured-timeline and the full cohort; a future release will rebalance T5 and enlarge T8 to remove trivial-class ceilings.Source: §6 · p. 9

7 Conclusion

• Full context remains the accuracy upper bound because compressed representations lose relational signal during preprocessing, and greater backbone capacity does not recover it.Even when the finding–diagnosis attribution sentence is present before representation construction, compressed strategies typically retain the fact but drop its relation.Source: §7 · p. 9
• Abstention discipline does not improve monotonically with context length, complicating the case for feeding longer charts.Source: §7 · p. 9
• The full-context cost–quality Pareto frontier inverts the “biggest backbone wins” heuristic, with smaller backbones dominating.Source: §7 · p. 9
• ClinTraceBench provides a source-verifiable benchmark with 9 tasks, 385 verified dialogues, 8 history representation strategies, 4 frontier backbones, 6,271 questions, and 200,672 predictions.Source: §7 · p. 9

Synthetic dialogues

• The 98.92% audit and deterministic verifier establish source fidelity and gold correctness, not conversational realism or transfer to real clinical conversations.Key clinical fields are covered at ≥98%, hallucination checks pass at 99.8–100%, and independent gold recomputation mismatches on <0.5% of items; real-note validation remains future work.Source: Limitations · pp. 9–10

Scope

• The benchmark uses English MIMIC-IV hospital admissions and a single 385-dialogue time horizon; generalization, multi-year follow-up, and dynamic memory updates remain untested.T5 uses controlled rather than natural-EHR contradiction, T8 measures post-treatment lab change rather than causal efficacy, and T9 covers only unstated lab and medication facts; full-context is a coverage upper bound, not a deployment target.Source: Limitations · p. 10

Residual artifacts

• Approximately 1,400 empty API responses (0.7% of 200,672) remain in headline numbers, while residual truncation and length–severity confounding also limit interpretation.Excluding empty responses would inflate Sonnet / Haiku full-context by at most +0.2 pp; every SP1–SP4 contrast holds with ∆<0.5 pp under either policy. About 13% of Sonnet T8 rows hit the original cap, and long-context degradation is −13 to −14 pp across quartiles, with length and disease severity entangled.Source: Limitations · p. 10

Agentic-memory prep confound

• Backbone-matched preparation did not improve accuracy in any of six agentic-memory cells, so the fixed DeepSeek-V3 prep model does not explain weak results.On the stratified 20-patient subset, accuracy changes ranged from −0.045 to −0.244 and at most 1/19 T3 attribution positives were recovered; the check corroborates Table 1 ordering only at that subset scale.Source: Limitations · p. 10

Task-design limits and planned revisions

• T5, T6, T8, and T9 are not discriminable from majority-class predictors in accuracy terms, while T4’s 53 questions leave it under-powered.T1, T3, T4, and T7 carry the cleanest representation signal; macro-F1 recovers usable signal on T5 and T8. Planned T5 rebalancing and T8 expansion are not part of this benchmark version.Source: Limitations · p. 10

Ethics statement

• Authors accessed MIMIC-IV only after CITI training and signing its PhysioNet Data Use Agreement; no re-identification was attempted.Synthetic dialogues derive from structured fields such as dates, lab values, ICD codes, and drugs, and contain no protected health information.Source: Ethics statement · p. 10
• The 385 dialogues are LLM-generated from de-identified structured records, not real clinical communications, and have not been clinically validated.The benchmark measures retrieval and reasoning fidelity over EHR-derived dialogues, not safety, diagnostic accuracy, treatment appropriateness, or patient outcomes.Source: Ethics statement · p. 10
• ClinTraceBench must not serve as a proxy for clinical readiness, clinical decision support, deployment-oriented training, or any patient-facing application.Its evaluation scope does not establish clinical safety or effectiveness.Source: Ethics statement · p. 10
• The question set, event-ID provenance, gold answers, predictions, scoring harness, and injection generator are released, but dialogue text is not redistributed.Regenerating the dialogues requires PhysioNet credentialing and MIMIC-IV access because they inherit its redistribution restrictions.Source: Ethics statement · p. 10

A Validation gates

• 366/370 audited questions agreed with automated gold answers (98.92%), while T7 and T9 had 2% and 6% FAIL rates, respectively.The stratified L5 audit found both task-level rates below the 10% fix threshold.Source: A Validation gates · p. 13
• L0–L4 apply deterministic schema, anchor, gold-recompute, leakage, and distribution checks; L5 human spot-checks a stratified subsample.L0–L3 have hard-failure thresholds, while L4 distribution coverage is a soft warning.Source: A Validation gates · p. 13

B Per-task subtype diagnostics; B.1 Subtype overview

• Across T3, T7, and T9, categorical subtypes separate; T5 follows its controlled-versus-natural design, and T8’s small-magnitude bins are harder across backbones.Figure 7 shows the full-context subtype overview for T1, T3, T5, T6, T7, and T8 across four backbones.Source: B Per-task subtype diagnostics · p. 13, §B.1 · p. 13, Figure 7 · p. 13

B.2 T3 four-class attribution detail

• In T3, injected attribution classes collapse to approximately zero under compressed memories but are recovered by dense and full-context strategies.Not-injected classes are near ceiling for most strategies, while dense and full-context can occasionally add noise; this qualitative pattern holds across backbones.Source: §B.2 · p. 13, Figure 8 · p. 14

B.3 T5 controlled vs. natural contradiction

• T5 controlled-subtype accuracy rises from 0.32 with no-ctx to 0.97 with full-context, while natural lab-non-glucose accuracy falls from 0.80 to 0.07.The natural-subtype decline is attributed to confounding lab evidence triggering spurious contradictions under dense and full-context.Source: §B.3 · p. 14, Figure 9 · p. 14

B.4 T4 retrospective evidence retrieval

• T4 uses strict event-UID set-F1 on only 53 questions, so its authors treat the task as a limited rather than headline finding.The pool is biased toward a handful of condition families; Figure 10 reports its composition and performance across backbones and memory baselines.Source: §B.4 · p. 14, Figure 10 · p. 14

B.5 T8 observed treatment-response bins

• T8 small negative and positive bins remain near zero across baselines, while small flat-bin accuracy is rescued only by structured timeline and full-context.Large and medium bins rise from 0.29–0.65 at no-ctx to 0.69–0.94 at full-context; these bins proxy observed lab-change difficulty, not causal treatment response.Source: §B.5 · p. 14

B.6 T9 silent-evidence kind: lab vs. other

• T9’s primary-versus-secondary axis is degenerate, so the appendix stratifies silent-evidence questions by lab versus other anchors.The pool contains 1,147 lab-anchored and 353 other-anchored questions; Figure 12 reports accuracy, abstention, and over-answer rates by anchor kind.Source: §B.6 · p. 14, Figure 12 · p. 16

C Cohort, retrieval, and compression footprints; C.1 Patient cohort; C.2 BGE-M3 dense-retrieval hit rate; C.3 Compression footprints

• The benchmark comprises 6,271 questions from 385 verified dialogues; dense retrieval reaches hit@1 = 0.595 and hit@10 = 0.979, while compression accuracy is not explained by representation size.Retrieval hit rates are reported for 1,177 eligible questions, excluding T9; Mem0 size and T1 accuracy have Pearson r = −0.129 across 210 patients.Source: C Cohort, retrieval, and compression… · p. 15, §C.1 · p. 15, §C.2 · p. 15

D Resource consumption; E Pipeline-quality diagnostics; E.1 Empty responses (D1)

• Representation-driven accuracy gains often do not require the most expensive cell, and the pipeline checks document empty responses and parser failures as potential confounds.The 32 cells cost $527.87 total. After retries, about 0.7% of prediction rows remain empty, concentrated in GPT-4o-mini and Haiku under retrieval-style contexts; DeepSeek has none, and these rows remain in headline accuracy numbers.Source: D Resource consumption · p. 15, E Pipeline-quality diagnostics · p. 15, §E.1 · p. 15

E.2 Parse failures (D2)

• About 6.5% of 200,672 responses are unparseable, dominated by Claude refusal-as-prose under no-context blind; Haiku no-context is the largest contributing cell.These parse failures are concentrated in a small number of cells and do not change memory-strategy ordering under richer contexts.Source: §E.2 · p. 16, Figure 18 · p. 18

E.3 Output length and truncation

• T4 is the only task where the 120-token ceiling binds meaningfully, causing 86–88% truncation on Haiku and Sonnet under the worst memory baseline; T1 and T8 bind next most.T3, T5, T6, and T7 are essentially un-truncated across backbones.Source: §E.3 · pp. 17–18, Figure 20 · p. 18

F Robustness and stratification

• Robustness checks examine per-patient skew, per-disease drift, per-tertile calibration, and bootstrap-CI width for headline estimates averaged over patients, diseases, and within-task difficulty tertiles.These four checks address sensitivity dimensions the paper identifies as important.Source: F Robustness and stratification · p. 18

F.1 Per-patient accuracy distribution (D5)

• Full-context accuracy distributions are uniformly narrower than compressed-representation distributions across backbones, indicating less between-patient skew, not just a higher mean.Figure 21 shows per-patient accuracy violins, with medians and patient-level scatter.Source: §F.1 · p. 18, Figure 21 · p. 18

F.2 Per-disease drift (D6)

• Hypertension has the weakest pooled full-context accuracy across tasks (mean 0.615), reflecting its overrepresentation of longitudinal and comparison questions rather than a disease-specific knowledge gap.The benchmark covers CKD, Coronary, Diabetes, and Hypertension.Source: §F.2 · p. 18, Figure 22 · p. 19

F.3 Per-tertile calibration of the stratifier (D7)

• Under full context, difficulty tertiles are monotonically calibrated for T1 and T9, but T8’s “high” tertile is easiest, an anti-alignment flagged in the main paper.The analysis tests whether generation-time tertiles predict difficulty.Source: §F.3 · pp. 18–19, Figure 23 · p. 19

F.4 Bootstrap CI widths per cell (D8)

• Headline rankings are robust to within-cell sampling noise: bootstrap CI width is driven by sample size, with T4 and T8 widest and T1, T7, and T9 tightest.CI widths were estimated for 288 cells using 100 resamples per cell.Source: §F.4 · p. 19

G Supplementary analyses: balanced metrics, sensitivity, and ablations

• Supplementary re-scoring analyses reused existing model runs, while the Table 10–12 ablations used a stratified 20-patient subset of the 385-dialogue cohort.The appendix collects analyses referenced from Sections 3–6.Source: G Supplementary analyses: balanced… · p. 19

G.1 Class-balanced metrics for the skewed tasks

• Against the 0.478 majority baseline, T5 macro-F1 exceeds it for full-context, dense-retrieval, and structured-timeline on every backbone; T8 macro-F1 is led by structured-timeline, then full-context and dense-retrieval.T6's best accuracy is 0.609 against a 0.423 majority baseline; T9 remains a relative diagnostic probe.Source: Table 5 · p. 19, Table 6 · p. 19, §G.1 · p. 19

G.3 Parser-strictness sensitivity

• T8 has limited power for moderate differences and T4 is under-powered; lenient T9 parsing accepts synonymous abstentions, while T7 matching accepts option labels or full option text.Source: §G.3 · p. 19, Table 7 · p. 20

G.4 T9 over-answer composition

• 0.930 vs. 0.838: last-visit-only remains above full-context on T9 abstention accuracy, while parser leniency narrows the gap from 0.161 to 0.092.Among 307 strict-parser non-abstentions, 93% of the “other” bucket were missed verbose abstentions; discounting those, fabricated values were roughly 54% of genuine over-answers.Source: Table 8 · p. 20, §G.4 · p. 20

G.5 Backbone-matched memory preparation

• Matching each answering backbone to its own memory-preparation model changed only the preparation model and improved accuracy in none of the six cells.Source: §G.5 · p. 20, Table 10 · p. 21

G.6 Dense-retrieval and summary-length ablations

• Larger retrieval K gives no consistent accuracy gain, doubling summary budget lowers overall accuracy for three of four backbones, and excluding exploratory tasks or T3 leaves strategy ordering unchanged.Source: §G.6 · p. 21, Table 11 · p. 21, Table 12 · p. 21

G.8 Substrate pilot: dialogue versus a bare event table

• The bare structured-event table retained events but omitted explicit narrative relations targeted by T3, T4, and T5, so those tasks could not be evaluated equivalently from it.The pilot does not validate real-note realism; comparison against a visit-organized table is left to future work.Source: §G.8 · p. 21

H Case studies

• Four illustrative cases show representational loss (T3), uncritical answering (T9), a Haiku reasoning-capacity ceiling (T2), and cross-document aggregation breakdown (T6).Source: Figure 25 · p. 22, H Case studies · p. 22

I Full per-task strategy-by-backbone matrix

• Table 14 covers per-task accuracy across all 32 strategy-backbone cells on 6,271 questions, totaling 200,672 evaluations.The task sample sizes range from 53 for T4 to 1,500 each for T1 and T9.Source: Table 14 · p. 22
• On T2, full-context Claude Sonnet scored 0.474, while on T3, full-context DeepSeek scored 0.938.For T2, dense-retrieval Claude Sonnet scored 0.432; for T3, full-context DeepSeek was the highest-scoring listed pair.Source: I Full per-task strategy-by-backbone… · p. 23
• On T5, full-context GPT-4o-mini scored 0.897; on T6, structured-timeline DeepSeek scored 0.609.The T6 structured-timeline DeepSeek score exceeded the other listed T6 strategy-backbone scores.Source: I Full per-task strategy-by-backbone… · pp. 24–25
• On T7, full-context Claude Haiku scored 0.965, while on T8, structured-timeline Claude Haiku scored 0.537.These were the highest-scoring listed pairs for T7 and T8, respectively.Source: I Full per-task strategy-by-backbone… · pp. 25–26
• On T9, last-visit-only Claude Haiku scored 0.994, the highest-scoring listed T9 strategy-backbone pair.Source: I Full per-task strategy-by-backbone… · p. 26