PaperlayerView exact revision on arXiv

Source-linked AI summary

Clinical Intent Extraction: A FHIR-Aligned Representation and the CIRCA Benchmark

Alexander Apartsin, Yehudit Aperstein

arXiv:2609.29479v1cs.CL

TL;DR

Prospective clinical actions matter for safe follow-up, yet existing standards do not recover them from free-text clinical communication. The paper defines a FHIR-informed representation and harmonizes five corpora into CIRCA; the audited agreement stratum matched human decisions 88.4% of the time.

  • Problem

    Prospective actions can determine safe follow-up after discharge, yet existing interoperability standards do not recover them from free-text clinical communication.

    Source: §1 · pp. 1–2

  • Method

    The authors define CIE and a FHIR-informed CIR with request-intent and modality axes, then harmonize five corpora into CIRCA using three-model consensus and human review.

    Source: §1 · p. 2, §4 · p. 7

  • Results

    88.4% of audited agreement-stratum cases matched human decisions, compared with a 60.4% match rate in the adjudicated disagreement stratum.

    Source: §7.1 · p. 13

  • Takeaways & Limitations

    The authors argue that separate request-intent and modality axes matter because a single-axis schema would discard conditional or prohibited actions.

    Source: §1 · p. 2

    CIR output is not a clinical order, and the resource is intended for research and system development rather than autonomous clinical use.

    Source: §8.1 · p. 18

Abstract

from arXiv · show

Prospective clinical actions, the follow-ups, orders, referrals, and instructions that deter-mine what happens to a patient next, are annotated today in thin fragments across incom-patible corpora: each records a text span and one coarse category. We introduce Clinical Intent Extraction (CIE), the task of recovering these actions as complete structured rec-ords, and the Clinical Intent Representation (CIR), which decomposes each action into its verb, type, coded target, timing, and condition, and adds two axes prior datasets do not jointly represent: request-intent, the authority behind the action (proposal, plan, order, or option, aligned to HL7 FHIR), and modality, a seven-valued scale of clinical strength. Re-expressing five heterogeneous corpora (CLIP, MedDec, ap_parsing, PaniniQA, SIMORD) in the CIR yields CIRCA: 10,011 harmonized intents spanning two note distributions, with a human-validated subset, source-to-CIR crosswalks, and a deterministic FHIR R4 mapper. CIRCA is built by three-model consensus that auto-accepts high-agreement in-tents and routes the rest to human review; the audited agreement stratum matches human decisions 88.4% of the time. Benchmarking five existing models without task-specific training exposes the gap CIRCA targets: given the span, they label type well (85 to 91%) but get all four closed fields right only 18 to 35% of the time. All artifacts are released, with MIMIC-derived layers shared as stand-off annotations under PhysioNet credentialed access.

1 Introduction

Clinical notes contain prospective actions needed for safe follow-up, but clinical intent extraction must recover them from free text because interoperability standards carry intent only after it is structured. The paper introduces a FHIR-informed representation and harmonizes five existing corpora into CIRCA.

1 Introduction

  • A large share of test results lack documented follow-up [1], and roughly 40% of hospital discharges leave pending results, many actionable and unnoticed [2].Extracting prospective actions in structured form supports follow-up tracking, care-gap detection, and interoperable care plans.Source: §1 · p. 1
  • CLIP [6], MedDec [7], ap_parsing [8], PaniniQA [9], and SIMORD / MEDIQA-OE [10] annotate different pieces of prospective actions, but their schemas, note types, and granularities differ.None jointly represents the authority or strength of an intended action, limiting uniform downstream reasoning.Source: §1 · p. 2
  • CIE and the FHIR-informed CIR represent actions with a closed taxonomy, verbatim and normalized targets with standard codes, separated temporal fields, request-intent, and modality.The two non-redundant axes represent an action’s authority and clinical strength; CIR records serialize deterministically to FHIR R4 request resources where representable.Source: §1 · p. 2
  • CIRCA re-expresses five heterogeneous corpora as 10,011 intents across 1,185 notes and 942 patients, spanning two note distributions.It includes 2,145 human-validated intents, 255 reviewed negatives, per-layer licensing and provenance, and source-to-CIR crosswalks.Source: §1 · p. 2
  • The baseline evaluates existing large language models without task-specific training on span-level labeling and whole-note extraction tracks.The first predicts CIR fields given spans; the second finds spans from notes without anchors, and both are evaluated against human gold.Source: §1 · p. 2

2 Related work

Related work treats clinical actions as spans, temporal events, assertions, or standardized entities, but does not jointly represent prospective intent with request authority and directive strength across harmonized corpora. CIRCA instead structures prospective actions and uses model agreement to propose annotations for human validation.

Clinical action and order extraction; Harmonization and standards

  • Prior corpora annotate spans with coarse categories, while CIR makes action, type, target, timing, condition, request-intent, and modality explicit across harmonized resources.Among surveyed resources, CIR alone captures all attribute groups and harmonizes across corpora.Source: §2 · pp. 2–4, Table 1 · p. 4

Prospective intent versus temporal annotation

  • Temporal annotation alone does not identify intended action: future-return statements, conditional returns, and already scheduled imaging differ in prospective force.CIE makes prospective intent the unit and represents time as one attribute, distinguishing proposed, planned, ordered, optional, conditional, or prohibited actions.Source: §2 · p. 3

Assertion, certainty, and negation

  • Assertion status characterizes present findings, whereas CIR modality grades prospective directive strength; its polarity layer maps negated directives to prohibited.The paper relates this layer to assertion, negation, context, speculation, and hedging detection, while distinguishing the objects and questions being annotated.Source: §2 · p. 3

LLMs for annotation across domains

  • CIRCA uses agreement among three independent LLMs as a candidate-generation signal, not ground truth, and sends every disagreement to human validation.The passage cautions that unvalidated generative labels vary in quality and may hallucinate, with added privacy and expertise concerns in healthcare.Source: §2 · p. 3

Request authority and speech acts

  • CIR request-intent distinguishes proposal, plan, order, and option; the authors know of no prior clinical-NLP corpus annotating equivalent authority over narrative text.The axis is linked to speech-act semantics and the FHIR request pattern, and maps to FHIR request resources.Source: §2 · p. 4

Harmonization and standards

  • Existing harmonization maps terminology or entities to common models and FHIR, but the reviewed resources do not annotate clinical notes at scale with both authority and strength axes.CIRCA harmonizes prospective-action annotations and releases a labeled dataset with source-to-CIR crosswalks and FHIR mappings.Source: §2 · pp. 3–4

Clinical corpora, transfer, and de-identification; Synthetic and surrogate clinical data

  • CIRCA spans MIMIC-III discharge text and ACI-Bench dialogues to assess transfer, while surrogate-filling MIMIC de-identification placeholders preserves readable text and usable date intervals.It releases labels and cross-corpus mappings rather than model-generated notes, using surrogate-filled MIMIC text under a Business Associate Agreement and zero-retention pipeline.Source: §2 · pp. 4–5

3 The CIE task and the CIR schema

Clinical Intent Extraction identifies prospective actions in clinical notes and represents each as a structured record. The CIR distinguishes the authority behind an action from its clinical strength and aligns the authority axis with FHIR R4.

3 The CIE task and the CIR schema

  • CIE extracts actions that should happen after the note’s writing time, representing each with five required fields—action, type, target, request-intent, and modality—plus provenance.Other fields are populated only when stated; past actions and diagnoses are out of scope, and polarity is an internal normalization attribute rather than a released schema field.Source: §3 · p. 5, Table 2 · p. 5

3.1 Axis value definitions

  • Request-intent grades how binding an action is, while modality grades its clinical strength; the seven modality values are compared in four coarse classes.The coarse classes are active (ordered, planned, recommended), weak (considered, optional), conditional, and prohibited; the fine modality value remains in the record.Source: §3 · pp. 5–6, Table 3 · p. 6

3.2 Alignment with FHIR

  • CIR records map to FHIR R4 request resources, and every mappable dataset intent—every type except “other”—produces a schema-valid resource, not a claim of full profile conformance.The mapper is deterministic and validated against the R4 schema; condition is carried as a trigger, not a clinical reason.Source: §3.2 · p. 6, Table 4 · p. 6

3.3 Authority and strength as separate axes

  • 986 intents have conditional, prohibited, optional, or considered strengths with no request-intent equivalent, showing authority and strength are not interchangeable.Separately, 9% (903) of 10,011 intents have a strength not predicted by the authority-to-strength mapping; the axes are strongly associated (Cramér’s V ≈ 0.67) but neither recovers the other.Source: Table 5 · p. 7, §3.3 · p. 7

4 Dataset construction

CIRCA construction turns partial source annotations into full CIR records through model re-extraction, agreement-based routing, and targeted human review. Source annotations serve as evidence rather than ground truth, and the agreement stratum is released with audited accuracy rather than treated as gold.

Dataset construction

  • Source spans and labels anchor model searches, but crosswalked types are only candidates and no source span is assumed correct or in scope.For anchored intents, models infer all CIR fields beyond the coarse source label; unanchored additions are located by the models themselves.Source: §4 · pp. 7–8
  • Three independent models re-extract and label intents from notes, extending sparse annotations, filtering retrospective spans, and normalizing fields before comparison.The models are Claude Sonnet 4.5, Llama 3.3 70B Instruct, and Nova Pro; they add missed prospective actions and drop retrospective content such as “s/p” and “given” spans.Source: §4 · p. 7
  • Intent clusters with grounded agreement on five critical fields are auto-accepted; disagreements and intents found by only some models go to human review.Clustering uses weighted span overlap, normalized target, and type (0.5 / 0.3 / 0.2), accepted above 0.45; Figure 1 depicts the routing pipeline.Source: §4 · p. 8, Figure 1 · p. 8
  • A trained annotator confirms or corrects each routed span using its highlighted text, about 160 characters of local context, and the pre-filled label—not the whole note.This makes each human decision a short span-level check rather than a full-note rereading.Source: §4 · p. 8
  • The span-level benchmark matches the anchor-hinted annotation condition, whereas whole-note evaluation is harder because it removes localization and source hints.The recall gap between tracks reflects the combined cost of removing both localization and the source hint.Source: §4 · p. 8
  • The agreement stratum is audited and released with measured accuracy, while construction-involved models are distinguished from independent gpt-oss baselines.Claude Sonnet 4.5, Llama 3.3 70B, and Nova Pro also appear among the evaluated baselines and are treated as construction-involved references.Source: §4 · pp. 8–9

5 Dataset composition

CIRCA harmonizes prospective clinical intents from five heterogeneous corpora into a structured dataset, with sparse fields, imbalanced value distributions, and a human-reviewed core. Its examples show how CIR records map to FHIR resources, but do not validate executable clinical behavior.

5 Dataset composition

  • CIRCA harmonizes 10,011 intents over 1,185 notes from five heterogeneous corpora; the corpus draws cover 2,270 source notes with negligible overlap.The sources include four MIMIC-III layers with different annotation tasks and public simulated ACI-Bench dialogue, providing a second note distribution.Source: §5 · p. 9, Table 6 · p. 9

5.1 Field presence

  • All intents populate five mandatory fields; time and condition are filled in 34% and 16%, while a from-scratch pass estimates true presence at 29% and 21%.Fill rates are a proxy for genuine presence, not a direct measurement.Source: §5.1 · p. 9, Figure 2 · p. 10

5.2 Value distributions

  • Medication comprises 51% of intents and continue alone 30% of verbs; 98% are plan or order, while 90% are recommended, ordered, or planned.The common-case skew makes macro-averaged, per-class evaluation more informative than raw accuracy; timing, condition, and duration are usually unpopulated.Source: Figure 3 · p. 10, §5.2 · p. 10

5.3 Distributions across the two note sources

  • Medication comprises 60% of MedDec and PaniniQA intents, 57% of CLIP, 46% of ap_parsing, but 32% of SIMORD, whose dialogue-derived orders span more target types.This different intent mix makes testing discharge-text-trained models on dialogue-derived notes meaningful.Source: §5.3 · p. 11

5.4 Human-validated core subset

  • Three-model consensus auto-accepts 2,613 intents (26%) and routes 7,398 (74%) to human adjudication; the reviewed core confirms 2,145 intents and includes 255 non-intent negatives.The stratified core reviews 2,400 candidate spans, including 450 from the agreement stratum, 1,651 from disagreement, and 299 targeting rare modality values.Source: Table 6 · p. 9, §5.4 · p. 11

5.5 Worked examples

  • Worked examples map source sentences to CIR records and FHIR resources, illustrating downstream uses without validating executable behavior or clinical safety.The examples include a paired prohibition that produces two intents from one sentence.Source: §5.5 · p. 11, Table 7 · p. 12

6 The CIE benchmark

CIRCA defines a reproducible CIE benchmark with two evaluation tracks, patient-level splits, a matching rule, and reference evaluation code. It evaluates field accuracy and exact-record performance while distinguishing span-anchored extraction from the harder whole-note task.

Benchmark design

  • CIRCA provides a fixed intent-extraction task, two evaluation tracks, patient-level splits, a matching rule, and reference evaluation code with dataset loaders.The input is a note and the output is a set of CIR records.Source: §6 · p. 12

Tracks and splits

  • The span-level track supplies gold spans, while whole-note models must find spans themselves; patient-level splits separate 470 validated notes into 20-note dev and 450-note test sets.The validated notes contain 2,145 gold intents; the span-level track draws 600 spans from the validated core.Source: §6 · p. 12

Matching rule

  • Intent matching uses weighted span-token overlap, target agreement, and type agreement, with a 0.45 threshold, one-to-one matching, and a code cannot-link for differing resolved codes.Type accuracy is measured on matched intents rather than fixed at 100%.Source: §6 · p. 12

Metrics

  • Span-level evaluation reports per-field accuracy and the all-four-exact rate; whole-note evaluation reports recall and per-field accuracy on matched intents.Whole-note precision is only a lower bound because gold intents are sampled per note; §7.6 provides true precision and F1 on an exhaustive 24-note gold.Source: §6 · p. 12

Cross-corpus transfer

  • Held-out-corpus transfer splits are de-duplicated by patient to prevent leakage, despite 13 shared notes and 21 patients appearing across source corpora.SIMORD/ACI-Bench shares no notes or patients with the MIMIC corpora.Source: §6 · p. 12

7 Results

Across benchmark tracks, models recover the four closed CIR fields far less reliably as a complete record than they recover type, with action and context-sensitive attributes driving errors. Human validation, cross-corpus transfer, and exhaustive-note evaluation further characterize where the benchmark is reliable and where models struggle.

7.1 Human validation

  • 88.4% of audited agreement-stratum pre-labels matched human decisions, versus 60.4% in the disagreement stratum, while human review still overturned some agreed labels.Among validated spans, 2,145 intents were confirmed and 255 rejected; rejection rose from 2.9% in the agreement stratum to 14.4% in the disagreement stratum. A 300-intent audit found field accuracy from 45% for time to 94% for reason; timing models agreed about 90% but over-generated values, so human review corrected the gold, usually to blank.Source: §7.1 · p. 13

7.2 Baseline model evaluation

  • 18–35% all-four-exact accuracy across five models trails type accuracy of 85–91%; Sonnet leads overall, while Nova Pro is marginally higher on request-intent (76% vs 75%).The 600-span comparison is at full coverage; the independent gpt-oss baselines achieve 19–20% all-four accuracy. In whole-note extraction, condition is stated correctly on about 66% of matched intents and is omitted on most remaining cases; timing, frequency, and goal are also often omitted, with prompt instructions reducing timing and frequency misses but not condition under-extraction.Source: Table 8 · p. 14, §7.2 · p. 14

7.3 Error analysis

  • 29–55% all-four-exact accuracy across corpora contrasts with 86–100% type accuracy, while free-text ap_parsing is weakest on request-intent (62%) and modality (57%).Structured discharge corpora provide more explicit authority and strength cues. Modality errors are mostly adjacent-strength confusions, such as ordered labeled planned (39) or recommended (28), rather than jumps to prohibition.Source: §7.3 · p. 15, Table 10 · p. 15
  • 72–79% action accuracy and 38–47% all-four accuracy follow collapsing start, adjust, and continue into one medication-management class, compared with original ranges of 45–63% and 18–35%.The exact verb remains in the release and the collapse is only a sensitivity analysis: a local span cannot resolve the start-versus-continue distinction without the admission medication list. Continue was labeled start 49 times and start labeled continue 21 times.Source: §7.3 · pp. 15–16, Table 11 · p. 16

7.4 Results on target and sparse attributes

  • 40–62% of spans yield the correct target category, while time and condition values are usually right when present but their presence is often misjudged.On true-present cases, value accuracy is 75–84% for time and 88–96% for condition; presence F1 is 49–60% for time and 71–77% for condition. Standard target-code benchmarking is not yet possible because the validated gold lacks resolved LOINC, SNOMED, and RxNorm codes.Source: §7.4 · p. 16, Table 12 · p. 16

7.5 Cross-corpus transfer baseline

  • 79% to 71% for type, 75% to 68% for request-intent, and 70% to 61% for modality mark the in-distribution-to-held-out accuracy drops of a TF-IDF/logistic-regression transfer baseline.Held-out ap_parsing and SIMORD had the lowest reported type accuracies, 43% and 54%; held-out macro-F1 was 42 for type, 44 for request-intent, and 52 for modality, reflecting heavy class imbalance.Source: §7.5 · pp. 16–17, Table 13 · p. 17

7.6 Whole-note validation

  • 96.4% of 330 exhaustive-gold intents from 24 notes were present in the released candidate pool, while model precision was 78–96% at 61–80% recall.True timing and condition prevalence was 29% and 21%, respectively, close to the annotation fill-rates; the 24-note sample makes these results indicative.Source: §7.6 · p. 17, Table 14 · p. 17

8 Access, provenance, and licensing

CIRCA supports research on prospective clinical actions, including follow-up tracking, FHIR serialization, and cross-corpus robustness. Its MIMIC-derived data are shared as stand-off annotations under credentialed access, and the resource is intended for research and system development, not autonomous clinical use.

8.1 Intended uses

  • CIRCA enables follow-up tracking, FHIR request-resource serialization, and cross-corpus robustness evaluation across discharge summaries and dialogue-derived notes.Validated timing and modality can help track deadlines while suppressing reminders for prohibited or not-yet-triggered conditional actions.Source: §8.1 · pp. 17–18
  • CIR output is a structured extraction, not a clinical order; schema validity does not establish clinical correctness, so the resource is not for autonomous clinical use.Unchecked errors involving prohibited or conditional actions could cause harm.Source: §8.1 · p. 18

8.2 Data provenance and compliance

  • Four source corpora use credentialed MIMIC-III notes, while ACI-Bench uses public simulated dialogue; model processing ran through AWS Bedrock under a HIPAA BAA and zero-retention configuration.The BAA is an additional safeguard and does not replace each corpus's data-use agreement.Source: §8.2 · p. 18
  • Across 100 notes and 4,632 placeholders, KART left none unclassified and generated synthetic surrogates with within-note consistency and shared-year date shifting.The shared year offset preserves month and day, maintaining gaps between dated events for timing extraction.Source: §8.2 · p. 18

8.3 Availability and licensing

  • The release shares MIMIC-derived CIR artifacts as stand-off annotations, not source text; each layer inherits its source rights, and only the SIMORD/ACI-Bench layer includes redistributable text.The deposit includes crosswalks, KART mappings, a deterministic FHIR mapper, and a reference evaluator; the mapper, evaluator, and crosswalks use an open code license.Source: §8.2 · p. 18, §8.3 · p. 19, Table 15 · p. 19

9 Limitations

CIRCA’s limitations concern how construction agreement, field-level gold, and span coverage are measured, as well as the scope of benchmarking and validation. Audits qualify agreement estimates, while broader data, model, and standards validation remain future work.

9 Limitations

  • Audited accuracy for the five routing-checked critical fields is 88.4%, but sparse-field agreement can overstate correctness: time agreement is about 90% versus 45% human-validated accuracy.All three models shared an over-generation bias, so the audit—not agreement—is the accuracy estimate to cite for sparse fields.Source: §9 · p. 19
  • Agreement measurements cover 8 to 40 documents per corpus and have wide intervals except in the scaled temperature-0 run; extending that run is a next step.The medication arbiter applies only to notes with structured admission and discharge medication lists, absent from the assessment-and-plan corpus; source annotations also act as evidence and recall targets, motivating a no-source-hint ablation.Source: §9 · p. 19
  • The main benchmark compares prompted models with human gold; Sonnet, Llama, and Nova are construction-involved references, while gpt-oss models are independent baselines.A TF-IDF plus logistic-regression cross-corpus baseline establishes a transfer gap; a strong trainable-encoder baseline on the released split remains future work.Source: §9 · p. 19
  • All 2,145 intents have human-checked critical fields, but sparse fields are audited on 300 intents and target codes remain unresolved in the gold.Records are core-field gold; sparse-field and code quality are quantified by audits.Source: §9 · pp. 19–20
  • Per-corpus counts and routing appear in Table 6, but a full dataset card will add candidate-versus-clustered intent counts; over-segmented source gold bounds recovery of some labels on three corpora.Split sizes are reported in §6, and the source-gold limitation is characterized in Appendix A.2.Source: §9 · p. 20
  • Span coverage is limited to source anchors and LLM extraction: an exhaustive pass estimates 3.6% missed intents on 24 notes, with a larger study needed to tighten the estimate.Estimated completeness is 96.4%.Source: §9 · p. 20
  • The mapper emits schema-valid FHIR R4 resources for every mappable intent, but profile-conformance and terminology-binding validation remain future work.Reliability uses a second annotator’s relabeling of 211 intents and shows moderate-to-substantial kappa; full double annotation and expert-clinician validation are future stages.Source: §9 · p. 20

10 Conclusion

The paper defines Clinical Intent Extraction and introduces CIRCA, a harmonized dataset and FHIR-informed representation for prospective clinical actions. Existing models label type well but struggle to recover complete structured intents.

10 Conclusion

  • Existing models achieve 85 to 91% on type labeling but only 18 to 35% on all-four-field labeling given the span, with action and modality hardest.The benchmark covers five existing models evaluated against human gold.Source: §10 · p. 20
  • CIRCA contains 10,011 intents across 1,185 notes and 942 patients from five heterogeneous corpora and two note distributions.It includes a human-reviewed core of 2,145 validated intents plus 255 reviewed negatives.Source: §10 · p. 20
  • The FHIR-informed Clinical Intent Representation adds request-intent and modality as separate axes, while CIRCA provides source-to-CIR crosswalks and a schema-valid FHIR R4 mapper.Artifacts include per-layer licensing, an exhaustive from-scratch whole-note gold set, and a second-annotator reliability study.Source: §10 · p. 20
  • A three-model consensus routes disagreements to human adjudication, and its audited agreement stratum matches human judgment 88.4% of the time.Validation by expert clinicians remains future work.Source: §10 · p. 20

Data availability

CIRCA is archived on Zenodo with its annotations and supporting resources. MIMIC-III-derived layers are distributed as stand-off annotations for credentialed users to rehydrate, while only the public ACI-Bench layer includes text.

Data availability

  • CIRCA’s Zenodo deposit includes annotations, crosswalks, surrogate mappings, modified code, a FHIR R4 mapper, evaluator, and rebuild script; MIMIC-III-derived layers contain no source note text.Credentialed users rehydrate those layers using their own MIMIC-III access, and their use remains governed by the PhysioNet/MIMIC-III data-use agreement; only the public ACI-Bench layer is distributed with text.Source: Data availability · p. 20

Appendix A. Construction and validation details … A.5 Polarity-preserving action metric and component attribution

The appendix evaluates harmonized annotations through inter-model agreement, source-gold recovery, confidence intervals, deterministic normalization, and action metrics. It distinguishes pipeline consistency from human-validated correctness and explains how source-gold scope and action-verb differences affect interpretation.

A.1 Inter-model agreement

  • Inter-model agreement on target, type, and both axes held across MIMIC discharge summaries and ACI-Bench dialogue-derived notes; agreement measures pipeline consistency, not correctness.Table 16 reports per-corpus agreement on aligned clusters and gives action as exact and polarity-preserving rates; human validation is the measure of correctness.Source: §A.1 · p. 23, Table 16 · p. 23

A.2 Recovery of source gold

  • Unmatched source labels included out-of-scope diagnoses and retrospective labels in MedDec, appointment logistics and section headers in CLIP, and past care in PaniniQA.For MedDec, 57% (174 of 306) unmatched labels were Category 3 diagnoses and 25% carried retrospective markers.Source: §A.2 · pp. 23–24
  • Recall on ap_parsing and SIMORD was 77–83%, with 87–94% type accuracy; lower recall elsewhere reflects source-gold granularity, not necessarily missed prospective actions.Source corpora annotate only a slice of note intents, and their labels are provided to annotators as evidence, so source-gold recall measures consistency with that signal rather than independent correctness.Source: §A.2 · p. 23, Table 17 · p. 24

A.3 Decoding and confidence intervals

  • At temperature 0, ap_parsing agreement was 69% exact action and 88% polarity-preserving action, with target 98%, type 87%, and modality 94%.The scaled run covered 20 documents and 169 clusters; exact agreement is primary, while polarity-preserving agreement is secondary.Source: §A.3 · p. 24

A.4 Deterministic normalization layers

  • Deterministic normalization standardizes temporal phrases, maps unconditional negative directives to prohibited modality, preserves conditional hold triggers, and canonicalizes actions and targets.Medication reconciliation overrides an action only when every model identifies the same drug and the medication lists give one answer; codeable targets resolve to standard codes, with conflicting drug codes kept in separate clusters.Source: §A.4 · pp. 24–25

A.5 Polarity-preserving action metric and component attribution

  • The polarity-preserving action metric treats nearby verbs such as start and adjust as less severe disagreement than start versus stop, using a prespecified clinical-distance threshold.Component attribution counts an improvement only when the paired confidence interval for its leave-one-out effect excludes zero; a code-aware cannot-link constraint separates clusters with conflicting standard codes.Source: §A.5 · p. 25
  • 85% action accuracy on 1,221 validated medication intents was unchanged before and after the medication arbiter, whose contribution is cross-model consistency.The arbiter uses admission medication lists to resolve start-versus-continue decisions that span-level annotators did not see.Source: §A.5 · p. 25
Loading 2609.29479v1…