Source-linked AI summary
A Neurosymbolic Approach for Constructing Planning Domain Models from Clinical Narratives
Ranveer Singh, Saurabh Mathur, Michael Skinner, Prasad Tadepalli, Kristian Kersting, Sriraam Natarajan
TL;DR
Clinical narratives are difficult to convert into probabilistic planning models because they are linguistically variable, sparse, and omit implicit but logically necessary events. NSPIN combines LLM-based extraction and sequence imputation with symbolic PPDDL induction and precondition refinement. Across 2,660 appendectomy notes from 9 surgeons, the approach generalizes to unseen data, while sequence imputation and neural refinement improve predictive and model-generalization behavior.
Problem
Clinical narratives lack structured execution traces and omit implicit intermediary actions, preventing existing action-model induction methods from directly producing logically sound planning domains.
Method
NSPIN uses a pretrained LLM to extract and impute structured predicate sequences, then applies symbolic induction and LLM-guided precondition refinement to construct probabilistic PPDDL models.
Results
NSPIN produces domain descriptions that generalize to unseen data, with imputation improving downstream prediction and refinement improving generalizability by pruning spurious temporal correlations.
Takeaways & Limitations
Combining LLM-based extraction and imputation with symbolic induction is a promising approach for constructing useful PPDDL-like procedural models from clinical narratives.
Takeaways & Limitations
Evaluation beyond laparoscopic appendectomy and refinement of probabilistic effects remain important directions for future work.
Abstract
from arXiv · showhide
Surgical procedures such as laparoscopic appendectomy are complex, high-stakes processes, yet formalizing their workflows for decision support remains a significant challenge. Inducing probabilistic planning domain models in this setting is particularly difficult due to the lack of structured event data and the prevalence of implicit actions in clinical narratives, which neither empirical symbolic methods nor Large Language Models (LLMs) can adequately address on their own. We introduce NSPIN, a neurosymbolic framework for inducing probabilistic planning domain models from unstructured clinical narratives. Our method extracts and imputes structured event sequences from raw text using a pretrained LLM, then induces a PPDDL model and refines its preconditions with LLM-proposed revisions, guided by empirical validation. We evaluate the approach on 2,660 laparoscopic appendectomy notes written by 9 surgeons. NSPIN yields models that generalize to unseen notes, and expert clinical review indicates its induced knowledge is largely consistent with surgical practice.
Introduction
The paper frames surgical workflow formalization as a need for interpretable, generalizable planning models and introduces NSPIN to derive them from clinical narratives. It targets laparoscopic appendectomy, using LLM extraction and imputation with symbolic planning-model induction.
- Motivation: Clinical notes capture patient-specific findings and responses, but formal models could generalize this experience across cases for safety, training, and clinical AI.Such models may support simulator improvement, operative-note review, and equitable access to high-quality care.
- Motivation: Formal planning languages represent procedures with interpretable predicates and parameterized action schemas that clinicians can validate.PDDL describes actions through conditions and state changes, providing a declarative representation of workflow structure.
- Challenge: Clinical narratives vary linguistically and omit obvious-but-essential information, making conversion to logically consistent symbolic representations difficult.The omitted information can be necessary to maintain a valid formal state.
- Approach: NSPIN combines LLM-based extraction and imputation with symbolic induction to transform unstructured surgical notes into formal probabilistic PDDL domains.The framework uses LLM grounding to account for explicit and implicit actions and observations.
- Evaluation: The evaluation uses 2,660 real-world surgical notes written by 9 surgeons to test whether the induced models generalize to unseen surgical workflows.The study focuses on laparoscopic appendectomy, a frequent procedure with a relatively straightforward workflow and a dense note corpus.
Background
The background distinguishes deterministic and probabilistic planning representations and explains why existing induction methods do not directly fit raw clinical narratives. NSPIN addresses this mismatch by combining LLM capabilities for extraction and imputation with symbolic reasoning.
- Planning Representations: Classical PDDL represents planning domains with predicates, initial states, goals, and parameterized action schemas whose preconditions and effects define valid transitions.A grounded action is applicable when its preconditions are satisfied by the current state.
- Planning Representations: PPDDL extends classical PDDL by representing each action’s effects as a probability distribution over multiple possible outcomes.This probabilistic representation captures stochastic action effects rather than assuming one guaranteed result.
- Induction Challenges: Traditional model acquisition is manual or relies on structured state-action-state trajectories, assumptions that are brittle or unavailable for clinical narratives.Clinical notes require parsing and often omit implicit intermediary steps, producing incomplete and potentially unsound traces.
- Neurosymbolic Approach: A sampled surgical sequence can use learned preconditions to constrain candidate actions while probabilistic observations provide context for transition-model sampling.In the illustrated example, adhesions observed after port placement contribute context for adhesiolysis.
- Neurosymbolic Approach: NSPIN combines neural representation learning with symbolic reasoning to bridge raw narrative extraction and formal planning-domain construction.This combination is intended for settings requiring probabilistic models and lacking clean domain descriptions or executable feedback.
- Neural Support: LLMs can map varied clinical language to structured forms and impute latent events or affordances omitted by human writers.Their pretrained domain knowledge can also help filter implausible interactions and state changes.
The NSPIN Framework
NSPIN converts unstructured clinical narratives into probabilistic planning models by combining LLM-based extraction and imputation with symbolic induction and validation-guided precondition refinement.
- Motivation: NSPIN addresses heterogeneous, sparse clinical narratives in which structured action data are unavailable and implicit steps are omitted.These omissions and clinician-specific writing styles complicate logically consistent planning-model construction.
- Schema Creation: The framework first uses an LLM to generate a formal domain schema containing typed object categories, predicates, and parameterized action types.Initial action schemas define vocabulary and argument structure but leave preconditions and effects empty.
- Sequence Generation: For each note, LLM-generated grounded predicate sequences are aggregated into a consensus sequence using majority filtering, argument voting, and temporal rank aggregation.Kemeny aggregation orders retained instances by minimizing contradicted precedence weight.
- Sequence Generation: NSPIN imputes omitted actions or observations that are logically necessary or contextually implied, retaining additions only when supported across candidate sequences.The enriched sequence strictly extends the extracted sequence with consensus-supported instances and represents the workflow used for model construction.
- Planning Model Construction: The system induces initial PPDDL action schemas from empirical co-occurrences and outcome frequencies, then uses LLM-proposed precondition revisions constrained by training and validation performance.A revision is accepted only if it preserves precondition satisfaction and does not degrade validation performance.
Empirical Evaluation
Across held-out surgical notes, NSPIN’s imputation and refinement improved predictive domain-model performance, while expert review found its induced knowledge clinically valid and useful for synthesis.
- Evaluation setup: 2,660 notes from 9 surgeons were evaluated using LLM-based baselines and an NSPIN variant without final refinement.The evaluation used average results across 9 folds on held-out data.
- Q1: Sequence imputation: Imputation improved action-set prediction across Top-1 accuracy, Top-3 accuracy, and average negative log-likelihood.The improvement indicates that imputed sequences recovered clinically obvious information omitted from notes.
- Q2: Precondition refinement: LLM-based precondition refinement improved next-action Top-1 and Top-3 accuracy, reduced Avg NLL, and significantly reduced TADR.Refinement removed descriptive findings from a pathology precondition while retaining the requirement that the specimen be retrieved first.
- Q3: Comparison with LLM baselines: NSPIN outperformed LLM-only and LLM-Example baselines across observation-prediction metrics, with higher Top-1 and Top-3 accuracy and lower average negative log-likelihood.The evaluation describes the symbolic model as providing a stronger explanation of post-action observations than directly generated LLM domains.
- Q3: Comparison with LLM baselines: After refinement, NSPIN surpassed both neural baselines in next-action accuracy, likelihood, and TADR, whereas the unrefined model had higher TADR.The results support symbolic validation of procedural priors before using them in the induced planning model.
- Q4: Qualitative evaluation: Expert review found imputed steps and effects clinically valid, while synthesized notes preserved broad workflow structure but sometimes missed infrequently observed constraints.Examples included recovering fascia closure and appendix-base securing, but generating an unwarranted suprapubic incision before camera-port placement.
Conclusion
The work addresses linguistic variability and observational sparsity in clinical notes by combining LLM-based extraction and imputation with symbolic induction. NSPIN produces domain descriptions that generalize to unseen data and supports future clinical decision-support systems capable of reasoning about complex procedures.
- NSPIN targets linguistic variability and observational sparsity when translating unstructured clinical narratives into formal clinical-planning representations.
- LLM-based imputation improves predictive performance by recovering omitted procedural steps that are logically essential for plan consistency.
- A single neural-refinement step improves domain-description generalizability by pruning spurious temporal correlations that data-driven induction may treat as hard preconditions.
- Evaluation beyond appendectomy and refinement of probabilistic effects remain important directions for future work.
- NSPIN represents progress toward clinical decision-support systems capable of reasoning about complex medical procedures.