PaperlayerView exact revision on arXiv

Source-linked AI summary

NSIDDx: A Design Framework for Neuro-Symbolic, Practitioner-First Differential Diagnosis in Low-Resource Settings

Aarav Singh

arXiv:2609.00256v1cs.CL

TL;DR

The paper addresses the gap between semantic benchmark accuracy and verifiable clinical reliability in open-ended differential diagnosis, especially in low-resource settings. It presents NSIDDx, a neuro-symbolic, practitioner-first framework with ternary symptom encoding, contradiction-sensitive scoring, and clinician override, and characterises failure modes that make oversight routine.

  • Problem

    Differential diagnosis places substantial cognitive demands on practitioners, while low-resource settings often lack specialist support and decision-support tools; accuracy alone does not reveal which outputs to distrust.

  • Method

    NSIDDx combines ternary symptom encoding, contradiction-sensitive and coverage-penalising scoring, negation-retaining phenotype graphs, symbolic transparency, and practitioner override in an offline neuro-symbolic pipeline.

  • Results

    The evaluation characterises vocabulary mismatch, confidence-without-grounding, and negation inversion as systematic failure modes of open-ended LLM+RAG diagnosis, while a case study demonstrates the override mechanism.

  • Takeaways & Limitations

    For low-resource clinical NLP, the paper argues that diagnostic AI should prioritise tractable disagreement and clinician collaboration alongside accuracy, with offline deployability treated as a design goal.

  • Takeaways & Limitations

    Evaluation uses 750 diagnostically interesting and complete CUPCase cases, which may overestimate performance relative to fragmented routine clinical notes; real-world EHR validation remains necessary.

Abstract

from arXiv · show

LLM-based diagnostic systems achieve high semantic accuracy on benchmarks, but open-ended evaluation on clinically uncommon presentations reveals a systematic gap between headline accuracy and verifiable clinical reliability. We evaluate an LLM+rare-disease-RAG pipeline across two cohorts and show that the paradigm produces confident outputs that are frequently unverifiable and systematically resistant to clinician interrogation. We present NSIDDx (Neuro-Symbolic Integrated Differential Diagnosis System), a design framework arguing that DDx systems in low-resource settings must treat the clinician as an active reasoning agent. We instantiate this through a neuro-symbolic pipeline with ternary symptom encoding, contradiction detection, audit strings, and practitioner override - running offline on consumer hardware. We distill five design principles for clinician-in-the-loop clinical NLP and invite the prospective studies needed to validate the claim at scale.

1 Introduction

The paper argues that diagnostic AI in low-resource settings must make reasoning transparent, auditable, and modifiable so practitioners can interrogate uncertainty and disagreement. NSIDDx instantiates this practitioner-first philosophy while framing clinician involvement as necessary but not yet clinically validated.

  • Motivation: Differential diagnosis places a heavy cognitive burden on practitioners who must weigh present and absent symptoms, consider rare diseases, and reason without specialist support.The paper emphasizes that low-resource clinicians need uncertainty surfaced rather than concealed.
  • Problem: 75.2% semantic accuracy at DDx@5 falls to 6.6% exact match, while 83.5% of rare scanner outputs lack tokenically verifiable relationships to the ground-truth diagnosis.These results illustrate the gap between benchmark-style semantic performance and clinically verifiable outputs.
  • Problem: LLM diagnostic outputs provide answers without reasoning processes that clinicians can interrogate, modify, or reject, including in knowledge-graph-grounded pipelines.The graph may inform generation while remaining invisible to the clinician.
  • System response: NSIDDx uses an LLM for narrative synthesis and explanations while independently validating reasoning through symbolic layers grounded in HPO/MONDO and PrimeKG.Practitioners can inspect a propositional-logic audit string and the graph, then add symptoms, remove candidates, and rerun scoring.
  • Design principles: The framework distills five clinician-in-the-loop principles: surface contradiction, preserve negation structurally, make override cheap, provide multi-level explanation, and support offline deployment.These principles are presented as actionable guidelines for clinical NLP.
  • Scope: The paper does not claim diagnostic superiority and acknowledges that its argument for clinician involvement remains theoretical, demonstrated by one case study, and unvalidated clinically.Prospective validation is explicitly invited.

2 Related Work

Related systems demonstrate pieces of NSIDDx’s functionality, but the paper positions NSIDDx as integrating negation modelling, practitioner override, offline operation, unstructured text input, and knowledge-graph grounding. Its distinction is architectural integration rather than individual feature novelty.

  • LLM-based diagnosis: Existing LLM diagnostic systems provide labels without inspectable logic chains or mechanisms for clinician override.This limitation is described for both general LLM-based systems and Health-LLM’s two-pass RAG pipeline.
  • Knowledge-graph diagnosis: MedKGI optimizes iterative history-taking over positive findings, whereas NSIDDx explicitly represents and exposes absent symptoms after history collection.MedKGI neither elicits nor encodes negative findings in the described comparison.
  • Knowledge-graph grounding: Binary ICD-9 vectors in a PrimeKG-grounded prognosis system cannot represent explicit symptom denials, and its reasoning chain is not inspectable.The paper contrasts this opacity with NSIDDx’s audit trail.
  • Neuro-symbolic explainability: Neuro-symbolic diabetes systems expose auditable reasoning with negation support but use structured tabular data and statically defined rules rather than unstructured clinical narratives.The comparison identifies a modality and flexibility boundary rather than a general deficiency.
  • Feature comparison: NSIDDx’s claimed novelty is the integration of five dimensions: negation modelling, override interface, offline capability, unstructured text input, and knowledge-graph grounding.The paper explicitly states that the claim is not individual feature novelty.

3 System Architecture

NSIDDx combines conversational extraction, ontology resolution, LLM-generated differentials, symbolic audit trails, dual-source scoring, negation-aware graphs, rare-disease retrieval, and practitioner-controlled reruns. Its ternary representation makes explicit denials computationally actionable, while hybrid scoring is used to avoid small-profile bias in full-database scans.

  • Pipeline overview: NSIDDx is organized as a modular pipeline with five core stages for processing clinical histories into differential diagnoses.The architecture combines language-model components with symbolic knowledge sources and practitioner control.
  • History and extraction: Clinical history is condensed into structured positive and negative findings, which are resolved into present and explicitly denied HPO identifier sets.The extraction pipeline uses Presence of: and Denies: prefixes before literal or semantic HPO matching.
  • Differential generation: The LLM differential stage receives the clinical summary and confirmed symptom lists, producing fixed Evidences → Reasoning → Diagnosis entries with ABSENT markers.Disease names are resolved by exact then substring matching, with symptom-profile diseases preferred over umbrella terms.
  • Auditability: Propositional-logic strings encode case-specific symptom conjunctions that imply each diagnosis, using negated propositions for absent symptoms as practitioner-readable audit trails.The strings may be generated symbolically from HPO matches or by the LLM.
  • Ternary-hybrid scoring: The matrix score represents patient and disease profiles with ternary vectors: +1 present, −1 explicitly denied, and 0 unrecorded; negative scores indicate contradictions.Contradictions arise when a patient denies a required symptom or presents one the disease expects absent.
  • Ternary-hybrid scoring: The hybrid score rewards coverage of patient symptoms while penalizing large disease profiles with only incidental overlap through a Jaccard term.Both scoring formulas run independently against HPO/MONDO and PrimeKG.
  • Negation graph: The negation-retaining phenotype graph keeps present, absent, history, and candidate nodes visible, with PrimeKG edges linking diseases, symptoms, cooccurrences, and modifiers.Explains-edge specificity weights are derived from disease and symptom counts; modifier handling is identified as a limitation.
  • Rare-disease retrieval: The optional ZebraMap rare-disease scanner retrieves candidates for presentations with eight or more symptoms and sends them through the same dual-source symbolic scoring pipeline offline.The scanner uses MedEmbed and FAISS, then maps UMLS candidates to MONDO identifiers.

4 Evaluation

The evaluation compares an LLM+rare-disease-RAG diagnostic pipeline across random and ontology-resolvable CUPCase cohorts using increasingly permissive matching metrics. Semantic performance is similar, while exact and token agreement diverge because of vocabulary alignment and ontology limitations; the rare scanner remains a clinician-reviewed hypothesis generator.

  • Evaluation setup: The study evaluates 750 CUPCase cases across a 500-case random CUP cohort and a 250-case exact-match WELL cohort.WELL represents an upper bound for automated symbolic performance because diagnoses have ontology entries, curated phenotype profiles, and resolvable names.
  • Evaluation setup: Four matching metrics—exact, substring, token, and semantic—make the gap between verifiable and apparent accuracy explicit.Semantic matching uses cosine similarity of sentence embeddings with a 0.7 threshold.
  • Primary DDx accuracy: 75.2% versus 73.6% DDx@5 semantic accuracy is statistically equivalent across cohorts (χ2, p = 0.700).Phenotype extraction rates are also statistically indistinguishable (MWU, p = 0.976), reducing support for extraction quality as the explanation for cohort differences.
  • Primary DDx accuracy: 6.6% versus 14.0% exact accuracy and 34.4% versus 46.8% token accuracy diverge significantly, reflecting vocabulary alignment with ontology entries.The reported differences are consistent with vocabulary mismatch rather than differential LLM clinical reasoning.
  • Scanner analysis: 5.0–8.0% semantic accuracy from MONDO and KG symbolic scorers indicates near-zero performance associated with HPO phenotype annotation sparsity rather than scoring-formula failure.The rare scanner recovers cases missed by the LLM DDx, but candidates are ranked by embedding similarity without phenotypic pathway validation.
  • Scanner analysis: The rare scanner functions as a hypothesis generator requiring clinician review, not as a standalone decision threshold.Its discriminative gap is statistically significant but clinically insufficient as a decision threshold.

5 Qualitative Failure Mode Analysis

Failure analysis identifies four intervention categories across both cohorts, including polarity errors that convert denied symptoms into positive evidence. The design implications emphasize structural negation, auditability, and clinician oversight for failures that remain domain-level or vocabulary-level problems.

  • Failure taxonomy: Four intervention categories organize automated failure modes across both evaluation cohorts.Categories 1 and 2 are statistically equivalent across cohorts, while Category 3 diverges significantly.
  • Negation inversion: 26 cases mapped explicitly denied symptoms to affirmative HPO terms, allowing negated findings to enter scoring as positive evidence.Examples include denied pain mapping to Pain insensitivity and denied hemoptysis mapping to Hemoptysis.
  • Negation inversion: In a sarcoidosis case, six of seven absent symptoms were LLM-inferred rather than explicitly denied, producing a matrix score of −0.024 for the correct diagnosis.Without explicit polarity encoding, this failure class is structurally undetectable.
  • Design implications: Domain-level blind spots and extraction failures require clinician oversight regardless of ontology coverage.Vocabulary-boundary failures call for synonym normalization and an audit trail so clinicians can verify tokenically unverifiable hits.
  • Design implications: The PL string and negation graph provide explanation modalities independent of score magnitude when symbolic path confirmation is unavailable.Surfacing active contradictions and negation inversions is itself the design goal rather than a requirement for changing the scoring system.

6 Case Study

The case study shows how extraction errors created a symbolic contradiction despite a correct LLM nomination, then demonstrates practitioner override correcting the symptom state and restoring Sarcoidosis to the top rank. The authors present this as a mechanism demonstration rather than a generalisable recovery-rate estimate.

  • Automated extraction failure: The automated pipeline extracted five present and seven absent symptoms, including six negatives not explicitly denied in the case narrative.Splenomegaly was also missed because its surface form was absent from the HPO synonym dictionary.
  • Automated extraction failure: Sarcoidosis received MONDOm = −0.024 and MONDOh = 0.512, placing it last in symbolic ranking despite the LLM nominating it first.The negative matrix score and PL audit trail surfaced the contradiction rather than suppressing it.
  • Practitioner override: The practitioner removed six spurious absent symptoms, retaining only ¬palpitations as a true absent finding.The intervention corrected the automatically generated negative list using the raw case narrative.
  • Practitioner override: Adding clinically inferred splenomegaly after removing the spurious negatives raised Sarcoidosis to rank 1 in the scored differential.The scores changed from MONDOm = −0.024 to 0.098 and MONDOh = 0.331 after both interventions.
  • Convergent confirmation: The corrected state contained seven present HPO IDs and one absent finding, with independent database sweeps confirming Sarcoidosis at top ranks.The corrected present vector included dyspnea, lymphadenopathy, pleural effusion, subcutaneous nodules, granuloma, and splenomegaly.
  • Interpretation: The case demonstrates that contradiction surfacing and practitioner correction can function as designed, but does not establish a generalisable recovery rate.The authors state that the three interventions represent capabilities relevant to 208 CUPCase cases and require prospective user studies for generalisability.

7 Position: Five Design Principles for Clinician-in-the-Loop Clinical NLP

The authors propose five clinician-in-the-loop design principles centered on making diagnostic disagreement visible, editable, and usable in low-resource settings. NSIDDx operationalizes these principles through explicit negation, auditable override, multiple explanation formats, and offline deployment.

  • Five principles: The five principles are surface contradiction, preserve negation structurally, make practitioner override auditable, provide multiple explanation levels, and design for offline deployment.The principles are presented as well-motivated hypotheses requiring prospective studies of usability and generalisability.
  • Principle 1: Surface Contradiction: Surface contradiction by presenting symbolic and LLM outputs side-by-side, using disagreement to flag uncertainty, extraction failure, or missing clinician knowledge.In the sarcoidosis case, the negative matrix score identified an extraction error rather than hiding it.
  • Principle 2: Preserve Negation Structurally: Preserve denied symptoms as first-class scoring, graph, and reasoning elements by encoding them as −1 and rendering them as grey graph nodes.The sarcoidosis contradiction from spurious absent symptoms motivates this structural treatment of negation.
  • Scope boundary: The ternary encoding cannot distinguish structural absence from clinically meaningful absence at a particular presentation stage, so clinical judgment remains necessary.The override interface is intended to admit distinctions that the encoding cannot represent by design.
  • Principle 3: Enable Auditable Practitioner Override: Enable auditable practitioner override so clinicians can add, remove, or modify symptoms and propagate changes through scoring, graphs, and reasoning chains in real time.The /add_symptom and /remove_symptom actions trigger immediate rescoring and regeneration of outputs.
  • Principle 4: Provide Multiple Levels of Explanation: Provide PL audit strings, phenotype graphs, and raw numerical scores because no single explanation format works for all clinicians.The framework treats explanation format as a layered choice rather than forcing one representation.
  • Principle 5: Design for Offline Deployment on Consumer Hardware: Design for offline deployment on commodity hardware because low-resource settings cannot rely on cloud APIs.This requirement shapes choices including quantized local models and deterministic scoring alongside probabilistic LLM output.

8 Conclusion

The paper presents NSIDDx as a neuro-symbolic framework that prioritizes tractable clinician disagreement over raw accuracy in low-resource diagnostic AI. Its open-ended evaluation identifies recurring LLM+RAG failure modes, while the case study demonstrates override and the paper calls for prospective validation.

  • Conclusion: NSIDDx combines explicit negation, ternary-hybrid scoring, full-stack override, and offline deployment in a clinician-in-the-loop diagnostic framework.The framework is intended for low-resource settings where clinician oversight must remain active.
  • Conclusion: Open-ended evaluation characterizes vocabulary mismatch, confidence-without-grounding, and negation inversion as failure modes making clinician oversight a routine requirement.The conclusion frames these as systematic issues of the LLM+RAG paradigm under clinically uncommon presentations.
  • Conclusion: The paper identifies offline deployability on consumer hardware as a prerequisite for equitable access in the intended settings.A case study demonstrates the override mechanism, while prospective studies are invited to validate the broader claim at scale.

Limitations

NSIDDx has important scope, data, representation, and validation limitations. The authors emphasize unresolved uncertainty in symptom matching and scoring, incomplete phenotype coverage, and the need for prospective clinician and real-world evaluation.

  • Knowledge and language coverage: PrimeKG incompletely covers common diseases, although the HPO/MONDO matrix remains operational and LLM-generated differentials are not suppressed.The system is therefore most valuable for complex, multisystem presentations where general practitioners benefit from structured support.
  • Knowledge and language coverage: The pipeline currently supports only English, reasons over a single encounter, and cannot guarantee complete absent-symptom reporting.These boundaries limit applicability to multilingual, chronic, or evolving presentations and to histories where pertinent negatives were not elicited.
  • Evaluation and validation: The evaluation covers 750 selected CUPCase reports, which may overestimate performance relative to fragmented or incomplete routine clinical notes.Real-world EHR validation remains necessary.
  • Evaluation and validation: The semantic evaluation metric cannot distinguish genuine synonymy from embedding artefacts without expert review.Across CUPCase, 83.5% of rare scanner outputs and 41.6% of DDx@5 semantic hits lacked a tokenically verifiable relationship to the ground-truth diagnosis.
  • Evaluation and validation: The human-in-the-loop demonstration used a researcher rather than a clinician in a live diagnostic setting.It shows that the override mechanism functions as designed but provides no evidence about clinician behaviour; prospective user studies are required.
  • Scoring and representation: The scoring formulas prioritize interpretability over diagnostic accuracy, use untuned equal weights, and may represent phenotype modifiers too coarsely.The modifier-binding gap arises because extracted modifiers enter the scoring vector as independent +1 entries despite graph-level relationships.

Ethical and Societal Implications

NSIDDx frames diagnostic AI as locally deployable, assistive information for practitioners rather than an autonomous decision-maker. Its ethical rationale emphasizes access, inspectable disagreement, data sovereignty, and safeguards against over-reliance, while acknowledging that clinical deployment still requires validation and governance.

  • Ethical and societal rationale: Positive impact is defined as enabling practitioners without specialist networks or reliable infrastructure to inspect and correct differential-diagnosis suggestions.The intended value is access to structured reasoning and potentially overlooked candidates, not benchmark accuracy alone.
  • Assistive use and oversight: NSIDDx presents rankings, audit strings, and graphs as information to reason with, while making disagreement and rerunning the pipeline easy.The system is explicitly assistive and does not make diagnoses.
  • Assistive use and oversight: Automation bias remains a risk because clinicians may anchor on system outputs despite divergent judgment.The interface invites disagreement, but the authors state that over-reliance cannot be fully designed away.
  • Risk-bounded rare-disease support: The rare-disease scanner reverses the usual hoofbeats heuristic but is optional, off by default, and restricted to presentations with at least eight symptoms.Low-scoring candidates, including negative matrix scores, are visually deprioritized.
  • Governance and equity: The prototype requires institutional review, privacy compliance audits, and phased clinical validation before deployment.Local processing, no cloud dependency, and auditable reasoning are intended to facilitate these governance requirements.
  • Governance and equity: English-only operation creates a significant equity gap for many low-resource settings, while local execution supports patient data sovereignty.Multilingual extension is identified as a priority, and no patient information is sent to external servers or APIs.

B Evaluation Data

The evaluation uses a 750-case subset of CUPCase organized into random and exact-match cohorts, with the full corpus omitted because of consumer-hardware inference limits. The scoring engine reports four source- and formula-specific scores, and Table 7 explains how discrepancies should be interpreted.

  • Evaluation data: 750 CUPCase cases form the evaluation set: 500 randomly sampled cases and 250 exact-match cases.The random cohort represents clinically uncommon presentations broadly, while the exact-match cohort represents the upper bound of ontology coverage.
  • Evaluation data: The full 3,562-case CUPCase corpus remains unevaluated because inference time on consumer hardware is constrained.The reported sample reflects one to three minutes per case on the target hardware.
  • Evaluation data: The dataset consists of real-world published BMC patient case reports drawn from the CUPCase benchmark.The evaluation spans two cohorts with different sampling purposes.
  • Scoring outputs: Each candidate receives four scores: HPO_m and KG_m for the matrix formula, plus HPO_h and KG_h for the hybrid formula.Both formulas are applied to both knowledge sources.
  • Scoring outputs: Table 7 interprets four-score combinations and flags HPO/KG discrepancies as possible knowledge-graph noise for practitioner review.The table is intended as a clinical reading guide for score combinations.
Loading 2609.00256v1…