PaperlayerView exact revision on arXiv

Source-linked AI summary

CohortFlow: A Steerable Clinical Cohort Curation Agent with Localized Semantic Correction

Neil Chen, Adrian Serapio, Hanxue Gu, Evan Lee, Kang Wang, Yang Yang

arXiv:2610.36100v1cs.AI

TL;DR

Clinical cohort criteria can change during execution, while conventional pipelines may repeat work for unchanged requirements. CohortFlow applies localized specification updates and reconciles them with a dependency-aware execution graph. It reports similar requirement-level accuracy to full regeneration alongside lower correction latency, and demonstrates selective reuse in a proof-of-concept backend evaluation.

  • Problem

    Conventional cohort-curation pipelines may repeat interpretation and computation after revisions even when most of the cohort definition remains unchanged, and execution order alone does not identify semantic dependencies.

    Source: §1 · p. 1

  • Method

    CohortFlow represents corrections as localized Task Delta updates to a structured cohort specification and reconciles a persistent execution graph using semantic dependencies.

    Source: §1 · p. 2

  • Results

    77.0% requirement micro-F1 for localized steering versus 76.4% for full regeneration; mean correction latency falls 64.8% after completion and 44.3% during execution.

    Source: §5.1 · p. 7, §5.2 · p. 8

  • Takeaways & Limitations

    The evaluation reports comparable requirement-level accuracy with lower correction latency and model invocations, and representative cohort revisions preserve unaffected computations while recomputing affected graph portions.

    Source: §6 · p. 9, §7 · p. 9

    Stage 2 is a 13-state proof of concept, not a comprehensive evaluation of reuse efficiency or clinical cohort validity; Stage 1 uses one 8B model under 4-bit quantization and constructed correction trajectories.

    Source: §6 · p. 9

Abstract

from arXiv · show

Language model agents for clinical cohort curation should accommodate evolving eligibility criteria without repeatedly processing unchanged requirements. We introduce CohortFlow, a steerable agent that represents natural language corrections as localized updates to a structured cohort specification and reconciles them with a persistent execution graph. Explicitly recorded semantic dependencies enable affected computations to be selectively invalidated while preserving valid prior results. On 100 scenarios derived from clinical-trial eligibility criteria, comprising 126 correction turns, localized steering achieves 77.0% requirement micro-F1 versus 76.4% for full regeneration using the same 8B model, a small numerical advantage that is not statistically significant ($p=0.493$). It reduces mean correction latency by 64.8% after completion and 44.3% during execution, with 38.1% fewer model calls. In a separate proof-of-concept evaluation over MIMIC, all 13 cohort states execute successfully, while correction trajectories demonstrate selective preservation of unaffected computations and recomputation of changed or dependent nodes. Together, these results show that localized semantic steering can substantially reduce correction latency and model calls while maintaining strong requirement-level accuracy.

1 Introduction

Clinical cohort criteria often change as clinicians inspect results, but conventional systems may reprocess the entire request. CohortFlow treats corrections as localized changes and uses semantic dependencies to preserve valid computation.

1 Introduction

  • Cohort curation is iterative, yet conventional pipelines may repeat interpretation and computation when a revised request leaves most of the cohort definition unchanged.Corrections can arrive during execution; restarting discards valid intermediate work, while editing only the final result can leave execution inconsistent with revised intent.Source: §1 · p. 1
  • CohortFlow applies natural-language corrections without restarting, maintaining an explicit cohort specification and execution state so revisions follow semantic dependencies rather than execution order.This can preserve computation that remains valid while corrections are incorporated during execution.Source: §1 · p. 2
  • The framework introduces localized Task Delta updates, semantic validity-aware invalidation, and stateful steering for heterogeneous EHR workflows, including resumable execution and sequential or parallel correction scheduling.Its evaluation addresses semantic correctness, computation reuse, and adaptation efficiency.Source: §1 · p. 2

2 Related Work

Prior work supports user intervention in agent workflows, dependency-aware reuse, and LLM-assisted cohort curation. CohortFlow instead treats the evolving cohort definition itself as persistent computational state.

2.1 Interactive and Steerable LLM Agents

  • Interactive agents expose persistent intermediate state for user intervention; CohortFlow represents the evolving cohort definition itself as persistent computational state through localized Task Deltas.Related systems include CowPilot, InterDeepResearch, and Plover; the cited examples are CowPilot and InterDeepResearch (Pan et al., 2026) and Plover.Source: §2.1 · p. 2

2.2 Dependency-Aware Execution and Reuse

2.3 LLMs for EHR Cohort Curation and Phenotyping

  • LLM cohort systems support structured queries, interactive refinement, and reasoning over patient data, while CohortFlow focuses on consistency between a revised cohort definition and ongoing computation.Examples include Criteria2Query, Criteria2Query 2.0, Criteria2Query 3.0, and EHRAgent.Source: §2.3 · p. 3

3 Methods

CohortFlow represents cohort requirements as a structured specification and persistent execution graph. Corrections update that specification locally, while semantic dependencies determine which computations can be reused or must be recomputed.

3.1 System Overview

  • A natural-language request becomes a database-independent TaskSpec with typed requirements, Boolean logic, and temporal relations, then compiles to an execution DAG whose artifacts persist across interactions.Corrections parse and ground only new or modified requirements; the graph identifies prior computations that remain valid.Source: §3.1 · p. 3

3.2 Clinical Concept Resolution and Structured Execution

  • Clinical predicates are grounded to MIMIC-IV identifiers, with normalized exact matching tried first and an IDF-weighted overlap score used to rank retrieved dictionary candidates otherwise.Grounded requirements compile to deterministic SQL operations supporting quantitative constraints, event counts, admission attributes, and temporal relations.Source: §3.2 · pp. 3–4

3.3 Clinical Note Retrieval and Adjudication

  • Free-text criteria use hybrid note retrieval, cross-encoder reranking, and sigmoid-converted relevance scores; a fixed threshold retains evidence for patient-level adjudication.When Boolean structure permits, structured predicates first restrict retrieval to patients for whom the note criterion can still affect cohort membership; per-patient limits keep evidence for existing patients stable under unchanged settings.Source: §3.3 · p. 4

3.4 Mid-Execution Steering

  • Corrections become semantic deltas to the current TaskSpec, and only new or modified requirements are compiled before deterministic merging preserves unchanged representations and stable identifiers.Corrections are processed in semantic FIFO order, and safe interruption boundaries preserve completed work for compatible resumption.Source: §3.4 · pp. 4–5

3.5 Sequential and Parallel Steering

  • Sequential steering waits for the active semantic action, while parallel steering interprets the first queued correction concurrently but commits it only after that action completes.Parallel mode overlaps interpretation latency without parallelizing the execution DAG or changing the committed TaskSpec sequence.Source: §3.5 · p. 5

3.6 Dependency-Aware Invalidation and Reuse

  • Graph reconciliation reuses completed nodes when identity and semantic fingerprint remain unchanged and artifacts and parent computations remain reusable; semantic changes invalidate affected computations, not all later ones.Note processing separately reuses corpus artifacts and patient decisions for surviving patients, while changes to note semantics or retrieval policy invalidate corresponding results.Source: §3.6 · p. 5

4 Experiments

The experiments assess semantic steering against gold structured specifications and test downstream execution and reuse separately. The benchmark contains curated correction trajectories, while backend testing uses frozen specifications and corrections.

4.1 Experimental Setup

  • 100 manually reviewed scenarios derived from distinct clinical-trial studies provide gold TaskSpecs for initial requests and corrections, including 126 turns and eight intentionally ambiguous corrections.Evaluation uses Qwen3-8B with identical inference settings for Stage 1.Source: §4.1 · pp. 5–6

4.2 Stage 1: Semantic Steering

  • Stage 1 compares localized steering with full regeneration using the same model, evaluates corrections after completion and during predicate compilation, and scores requirement-level and related semantic accuracy.Of 126 correction turns, 111 are evaluated mid-run and 15 after completion; efficiency measures include latency, LLM calls, and token counts.Source: §4.2 · p. 6

4.3 Stage 2: Backend Execution

  • Stage 2 executes manually frozen TaskSpecs over MIMIC and applies three frozen corrections across 13 cohort states to assess successful materialization and graph reuse or invalidation.This stage evaluates backend executability and correction-aware reconciliation, not clinical correctness of the resulting cohorts.Source: §4.3 · pp. 6–7

5 Results

Localized steering achieves requirement-level accuracy similar to full regeneration while reducing correction latency and model calls. All 13 tested cohort states execute, and three correction trajectories illustrate selective reuse, though they do not provide a comprehensive reuse evaluation.

5.1 Semantic Accuracy

  • 77.0% requirement micro-F1 for steering versus 76.4% for restart was not a statistically significant difference (p = 0.493).The paired bootstrap interval for the +0.6 percentage-point difference was [−1.23, 2.45].Source: §5.1 · p. 7
  • 90.5% temporal-relation accuracy for steering versus 89.5% for restart across 95 jointly scoreable states provided insufficient evidence of a significant difference.The paired randomization test gave p = 1.000.Source: §5.1 · p. 8
  • 85.7% Boolean logical-equivalence accuracy for steering versus 94.3% for full regeneration was not a statistically significant difference across 35 jointly scoreable states.The paired randomization test gave p = 0.500; the small sample warrants cautious interpretation.Source: §5.1 · p. 8
  • 1/8 intentionally ambiguous corrections elicited the expected clarification from steering, versus 4/8 for restart, indicating a limitation in clarification handling.Across all 126 planned correction states, valid-output coverage was 94.4% for steering and 99.2% for restart.Source: §5.1 · p. 8

5.2 Steering Efficiency

  • 64.8% lower post-completion latency and 44.3% lower mid-run latency accompanied 38.1% fewer LLM calls, but total token use was 16.5% higher.Mean latency fell from 48.6 s to 17.1 s after completion and from 69.7 s to 38.8 s during execution; larger prompt context accounts for the total-token trade-off.Source: §5.2 · p. 8

5.3 Backend Execution and Reuse

  • All 13 Stage-2 cohort states completed backend execution and materialized a patient cohort across the tested operation types.The tested operations included condition, patient, procedure, laboratory, medication, admission, Boolean, and temporal operations.Source: §5.3 · p. 8
  • Three correction trajectories showed selective reuse: unchanged computations were preserved, while changed or dependent computations were recomputed.The potassium-threshold change invalidated its predicate and downstream computations; widening a temporal constraint preserved predicate evaluations but recomputed the relation and downstream cohort computations. The three trajectories are proof of concept, not a comprehensive quantitative evaluation of reuse efficiency.Source: §5.3 · pp. 8–9

6 Discussion and Limitations

Localized steering reduces revision costs while preserving requirement-level accuracy, but individual predicate preservation does not ensure correct global composition. The evaluation is limited in model, trajectory, workload, and clinical-validity scope.

6 Discussion and Limitations

  • Comparable requirement-level accuracy accompanies substantially lower correction latency and model invocations, but Boolean-logic accuracy and clarification performance remain weaker.Preserving individual requirements may not preserve their global composition when corrections change predicate interactions.Source: §6 · p. 9
  • Stage 1 uses one 8B model with 4-bit quantization and constructed correction trajectories, and excludes clinical-note retrieval and reasoning.The authors leave evaluation across model families and scales, prospective clinician corrections, and full note-processing workloads for future work.Source: §6 · p. 9
  • Stage 2’s 13 cohort states demonstrate backend executability and representative reuse and invalidation, not comprehensive reuse efficiency or clinical cohort validity.Source: §6 · p. 9
  • Successful execution does not establish clinical correctness; the system is intended to support, not replace, expert cohort definition and review.Source: §6 · p. 9

7 Conclusion

The paper presents a steerable clinical cohort-curation agent that applies localized specification updates and semantic validity-aware execution. Its evaluation reports comparable requirement-level accuracy with lower correction costs, alongside representative selective reuse in backend experiments.

7 Conclusion

  • Localized steering maintains comparable requirement-level semantic accuracy while substantially reducing correction latency and model invocations.Source: §7 · p. 9
  • The framework reconciles revisions with an explicit execution graph, preserving valid prior computation instead of restarting the entire workflow after each intervention.Source: §7 · p. 9
  • Backend experiments demonstrate selective reuse across representative cohort revisions.Source: §7 · p. 9

B Example of Mid-Run Cohort Revision

An end-to-end example traces a natural-language correction through a cohort-specification change while retaining unaffected computation.

B Example of Mid-Run Cohort Revision

B.1 Initial Request and Task Specification

The initial example request asks for patients with type 2 diabetes or hypertension. Its TaskSpec makes the two included predicates and their disjunctive relation explicit.

B.1 Initial Request and Task Specification

  • The example’s initial request is to find patients with type 2 diabetes or hypertension.Source: §B.1 · p. 12
  • The TaskSpec represents the two conditions as stable predicates p1 and p2 combined by disjunction group g1.Source: §B.1 · p. 12
  • After clinical concept resolution, Figure 2 evaluates the two condition branches independently and combines them according to the specification’s Boolean structure.Source: §B.1 · p. 12

B.2 Localized Correction

The under-40 correction is handled as a localized addition: it is added at the root while the existing disjunction is retained.

B.2 Localized Correction

  • The correction interpreter identifies “Patients must be under 40 years old” as a localized semantic change rather than regenerating the complete specification.It emits an ADD action for the requirement at the ROOT with operator AND.Source: §B.2 · pp. 12–13
  • The corrected graph retains the type 2 diabetes–hypertension disjunction and adds the age predicate through conjunction.Node inspection shows reuse of the unchanged hypertension result and new execution for the age predicate.Source: §B.2 · p. 13, Figure 2 · p. 13
  • The added requirement is compiled into predicate p3 and attached by conjunction, leaving the existing disjunction g1 intact.Source: §B.2 · p. 13

B.3 Revised Task Specification

The revised task specification keeps the original condition disjunction and conjoins the new age predicate; execution reuses unchanged work and runs the new predicate.

B.3 Revised Task Specification

  • The specification maps p1 and p2 to type 2 diabetes and hypertension, and p3 to the included patient-age predicate age < 40.Source: §B.3 · p. 14
  • The revised logical expression is (p1 ∨ p2) ∧ p3, with the original predicates and disjunction structurally unchanged.Source: §B.3 · p. 14
  • The corrected graph preserves the original disjunction, combines it with the age predicate through AND, reuses the unchanged hypertension result, and executes the age predicate as new work.Source: §B.3 · p. 14, Figure 3 · p. 15
Loading 2610.36100v1…