PaperlayerView exact revision on arXiv

Source-linked AI summary

Teaching LLMs to Recommend and Defer in Underrepresented Epilepsy Care

Shreyas Rajesh, Kartik Sharma, Tonmoy Monsoor, Mehmet Yigit Turali, Richard Idro, Juliana Kayaga, Robert Sebunya, Tracy Tushabe Namata, Jessica Nichole Pasqua, Vwani Roychowdhury, Rajarshi Mazumder

arXiv:2606.31036v1cs.LG

TL;DR

In resource-constrained Ugandan epilepsy care, LLMs need to adapt prescribing recommendations to local practice and defer uncertain cases. MANANA learns deployment-specific guidance from local prescription errors, while Bayesian prompt averaging enables selective prediction, reaching 99% top-1 agreement on the most confident quarter of Cohort B cases.

  • Problem

    LLM decision support for Ugandan pediatric epilepsy must adapt to local prescribing practice because direct prompting can apply inappropriate prescribing priors.

  • Method

    MANANA learns auditable, deployment-specific prompt memories from patient notes, physician prescriptions, and observed errors using single-agent or multi-agent variants.

  • Results

    99% top-1 agreement was achieved on the most confident 25% of Cohort B cases, and 95% on the most confident 50%.

  • Takeaways & Limitations

    Bayesian prompt averaging supports selective prediction by trading coverage for physician-regimen agreement among non-deferred cases.

  • Takeaways & Limitations

    BPA provides retrospective confidence scores rather than a deployment-ready deferral rule, so thresholds require prospective selection and monitoring.

Abstract

from arXiv · show

Specialist epilepsy expertise is scarce in resource-constrained settings, making LLM-based decision support attractive for frontline clinicians managing longitudinal treatment. Such systems must adapt to local prescribing practice and know when to defer. We study this problem in Ugandan pediatric epilepsy care, predicting anti-seizure medication regimens from longitudinal unstructured clinic notes. Standard prompting achieves non-trivial agreement with physician prescriptions, but neurologist review shows that many errors reflect distribution-miscalibrated prescribing defaults rather than failures to parse the local record. We introduce MANANA, a non-parametric prompt-learning framework that learns local prescribing guidance from a small patient-level training set. MANANA converts observed prescription errors into auditable prompt memories, instantiated in single-agent and multi-agent variants, and improves over classical ML models, direct LLM prompting, and prompt-optimization baselines across two independently collected Ugandan cohorts. We further propose Bayesian prompt averaging, which converts the learned prompt trajectory into prescription likelihoods and an uncertainty-based deferral signal. On the independently collected held-out cohort, this improves visit-level top-3 prescription accuracy by 4-8 percentage points over prompt-optimization baselines and enables selective prediction: the system can auto-handle the most confident half of cases at 95% precision, or the most confident quarter at 99% precision, while deferring lower-confidence cases for specialist review.

1 Introduction

The paper studies longitudinal anti-seizure medication recommendation in Ugandan epilepsy care, where specialist access is scarce and prescribing practice differs from common Western priors. It introduces MANANA for error-grounded, deployment-specific prompt learning and Bayesian prompt averaging for uncertainty-aware decision support.

  • Motivation: Epilepsy affects around 0.8% of the global population, with nearly 80% of affected people living in LMICs where specialist care remains scarce.Diagnosis and treatment often fall to generalist providers in these settings.
  • Study setting: The study predicts anti-seizure medication regimens across serial Ugandan clinical visits from longitudinal clinic notes and evaluates agreement with treating physicians’ prescriptions.Predictions use patient notes rather than structured diagnostic codes or high-resource EHR medication histories.
  • Motivation: Direct prompting achieves non-trivial physician-prescription agreement, but neurologist audits identify systematic errors driven by Western prescribing priors rather than missing medical knowledge.The central challenge is applying the wrong prior to the target health system.
  • MANANA: MANANA learns deployment-specific prescribing guidance from patient notes and physician prescriptions through agents that propose regimens, analyze errors, and consolidate recurring lessons into an interpretable prompt memory.The framework is non-parametric and does not require clinician-written rules or expert review of model reasoning traces.
  • Bayesian prompt averaging: Bayesian prompt averaging treats learned prompt states across the adaptation trajectory as an ensemble of estimators for uncertainty-aware medication decision support.Each learning round contributes a distinct prompt state reflecting a different stage of evidence accumulation and plausible local prescribing rules.

2 MANANA

MANANA adapts an LLM through an explicit memory state learned from local patient cases, using a Predictor–Inspector–Architect loop instead of weight updates. Its Single and Multi variants produce memory trajectories that Bayesian prompt averaging converts into confidence scores for prediction and deferral.

  • Memory-based adaptation: MANANA learns an explicit memory state from 50 local patient cases pairing pre-prescription histories with clinician-prescribed regimens, without updating model weights.Learning proceeds through memory-state updates conditioned on future predictions.
  • Three-agent learning loop: The Predictor proposes three regimens over a 10-drug action space, the Inspector compares them with clinician prescriptions, and the Architect updates memory from structured evidence.All three components are LLM calls using the same base model with role-specific prompts.
  • Auditable updates: MANANA separates error diagnosis from memory updates, retaining candidate learnings in an auditable evidence process rather than rewriting one mutable global prompt.This distinguishes its unit of update from TextGrad’s textual-gradient descent over a mutable prompt variable.
  • Architect variants: MANANA-Single appends learned rules, while MANANA-Multi manages specialist agents that extract clinical signals and can spawn, edit, prune, or remain unchanged.Single-rule additions require support from at least N = 2 learnings originating from distinct clinic visits.
  • Uncertainty estimation: Bayesian prompt averaging treats learned memory states along the trajectory as candidate models and uses posterior predictive mass as a confidence score for specialist deferral.Top-1 prediction selects the regimen with maximum posterior predictive mass; top-3 prediction selects the three highest-mass regimens.

3 Experimental Setup

The study evaluates ASM-regimen recommendation from longitudinal clinician-authored notes in two independently collected Ugandan pediatric epilepsy cohorts. Methods are compared using visit-level top-3 agreement with physician-selected active medication sets, with Cohort B reserved for transfer evaluation.

  • Clinical cohorts and preprocessing: 332 patients and 1,040 visits comprise Cohort A, while 367 patients and 1,509 visits comprise Cohort B.Both cohorts come from Ugandan referral centers staffed by specialist pediatric neurologists and contain longitudinal outpatient epilepsy notes.
  • Prediction task and metrics: Each method receives the current pre-prescription note, prior notes, and prior regimens, then returns three candidate ASM regimens.Evaluation uses normalized active post-visit drug sets and compares them with the treating physician’s selected regimen.
  • Baselines and comparators: MANANA is compared with standard prompting, classical clinical-feature predictors, EpiPick, TextGrad, ExpeL, and DSPy optimized with GEPA.EpiPick is included as a rule-based comparator for monotherapy.
  • Implementation details: All learning methods train and validate on Cohort A, while Cohort B is held out for transfer evaluation.The Cohort A learning pool contains 70 patients, stratified into 50 training and 20 validation patients across five seeds.

4 Results and Discussion

MANANA improves longitudinal anti-seizure medication regimen prediction across Ugandan cohorts and supports calibrated, uncertainty-aware deferral. Bayesian prompt averaging strengthens top-1 confidence separation, enabling high-precision selective prediction for specialist-constrained care.

  • Physician-regimen agreement: MANANA improves physician-regimen agreement across most visits and regimen strata, outperforming classical baselines and remaining competitive with prompt-optimization methods.Evaluation uses Cohort A training patients and tests held-out Cohort A patients plus independently collected Cohort B.
  • Uncertainty and deferral: BPA achieves the same Top-3 accuracy as majority vote while improving Top-1 accuracy and confidence separation between correct and incorrect predictions.The method converts the learned MANANA-Multi trajectory into an uncertainty signal for selective prediction.
  • Uncertainty and deferral: 99% top-1 agreement is achieved on the most confident 25% of Cohort B cases with MANANA-Multi.Deferral operating points trade retained-case physician-regimen agreement against coverage according to specialist capacity.
  • Calibration: BPA confidence tracks empirical top-1 accuracy across cohorts and learned-prompt variants, with high-confidence bins substantially more accurate than low-confidence bins.The reliability analysis translates this separation into a selective-prediction operating curve over non-deferred cases.
  • Ablations and clinician review: Ablations test MANANA loop components, model transfer, LLM backbones, BPA weighting, MIMIC-IV reproducibility, and comparison with CONSILIUM.These experiments assess whether the results depend on a single component or model choice.

5 Related Work

Prior clinical LLM and prompt-optimization systems do not jointly learn site-specific prescribing rules from small narrative-note datasets and decide when to defer. MANANA instead updates prompt memories from prescription failures and derives deferral uncertainty from its learned prompt trajectory.

  • LLMs for clinical decision support: Clinical LLMs have been evaluated for medical QA, simulated EHR workflows, medication prediction, and epilepsy-variable extraction, while medication models typically use coded or structured EHR features.These systems do not learn site-specific prescribing rules from a small clinic dataset or recover complete serial ASM regimens from narrative outpatient notes.
  • LLMs for clinical decision support: Existing clinical systems do not combine small-data site-specific prescribing guidance, complete serial ASM-regimen recovery from narrative notes, and deferral to specialist review.The paper frames these capabilities as gaps in prior clinical decision-support systems.
  • Prompt optimization and self-improving agents: Prompt-search, textual-gradient, and reflection-style agents adapt LLM behavior through feedback-built text artifacts; MANANA is compared directly against DSPy, TextGrad, and ExpeL.For clinical prescription learning, MANANA treats the failure mode as the unit of update rather than allowing salient individual cases to become brittle rules.
  • Uncertainty and selective prediction: Prior calibration, Bayesian ensembling, conformal prediction, and learning-to-defer work studies when models should abstain or hand off to humans, whereas BPA derives uncertainty from the learned prompt trajectory.BPA treats prompt states across rounds as local estimators whose posterior concentration provides a practical deferral signal.

6 Limitations and Future Work … D.1 Is the model just copying the previous prescription?

The paper identifies deployment, clinical-validity, uncertainty, distribution-shift, baseline, and evaluation limitations while showing that MANANA-Multi predicts regimen changes beyond simply copying prior prescriptions. Future work centers on prospective deferral calibration and clinical analysis of learned prescribing guidance.

  • 6 Limitations and Future Work: Deployment-ready deferral remains unresolved because BPA confidence thresholds were selected retrospectively on held-out data rather than before clinical use.The learned confidence scores and memories could nevertheless identify low-confidence visits for specialist review and expose local prescribing rules.
  • 6 Limitations and Future Work: Physician-regimen agreement does not establish clinical optimality, because local prescriptions also reflect formulary limits, availability, affordability, and follow-up constraints.Future work should clinically analyze MANANA’s learned memories and specialist agents.
  • A Extended Related Work: The appendix situates the study within clinical LLM decision support, prompt optimization, self-improving agents, uncertainty quantification, and selective prediction, while distinguishing benchmark evaluation from longitudinal clinic prescribing.Prior uncertainty methods motivate treating prompt states as an empirical distribution and using posterior concentration for deferral without claiming prospective conformal validity.
  • B Clinical Cohorts and Distribution Shift: The two Ugandan cohorts are related but shifted clinical environments: Cohort B has heavier polytherapy and a different medication-use fingerprint, whereas Cohort A is more carbamazepine- and valproate-centered.This motivates reporting cohort-specific results rather than treating the combined dataset as homogeneous.
  • B Clinical Cohorts and Distribution Shift: Preprocessing separates clinical context from prescriptions, normalizes drug labels across documentation variants, and excludes current-visit treatment decisions while retaining prior prescriptions as longitudinal history.A manual audit of 200 Cohort A patients found the input-output splits consistent with the intended prediction task.
  • C Baseline Definitions: The evaluation compares non-agentic clinical prediction, direct LLM prompting, and frozen-model prompt optimization before specialist-audit and expert-derived systems, using pediatric adaptations where necessary.EpiPick is used as the closest epilepsy-specific rule-based comparator despite being designed for patients whose seizures begin at age 10 years or older.
  • D.1 Is the model just copying the previous prescription?: 85.1% on Cohort A and 76.6% on Cohort B are the previous-regimen-copy baseline’s aggregate accuracies, establishing a continuation floor for later-visit EM@3.The test also evaluates change visits, where the prescribed active drug set differs from the previous visit and copy accuracy is 0%.
  • D Interpreting the Main-Results Evaluation: 81.7% (Cohort A) and 83.5% (Cohort B) overall, with 39.0% and 55.6% on change visits, show MANANA-Multi learns a non-trivial revision component beyond continuation.On Cohort B, it is the only method to beat the copy baseline outright; TextGrad and MANANA-Single reach 41.7% and 47.3% on change visits.

D.2 Clinical validity of EM@3 under outcome-stratified evaluation · E Single-Agent Audit and Expert System CONSILIUM

Outcome-stratified evaluation treats reduced seizure burden as corroboration of the prescribed regimen and finds MANANA-Multi strongest on this validated subset. The appendix also uses neurologist review of single-agent failures to motivate CONSILIUM as an expert-designed comparator.

  • D.2 Clinical validity of EM@3 under outcome-stratified evaluation: Physician-agreement metrics use the prescribed regimen as ground truth, but clinical validity varies with seizure outcomes across visits.Reduced seizure burden validates the current regimen’s next prescription, whereas increased seizures make the revised regimen an unvalidated re-attempt.
  • D.2 Clinical validity of EM@3 under outcome-stratified evaluation: Agreement rankings are not claimed for visits where seizures worsened, because matching a revised regimen may reflect agreement with a clinical guess rather than correctness.The prior regimen has failed in these visits, so the revised prescription remains unvalidated.
  • D.2 Clinical validity of EM@3 under outcome-stratified evaluation: 342 visits, or 92% of annotated Cohort A V2/V3 visits, form the outcome-validated subset after excluding ambiguous seizure-frequency outcomes.Improved visits are annotated “Reduced” or “Seizure-free”; “Unchanged” and “Other” are excluded.
  • D.2 Clinical validity of EM@3 under outcome-stratified evaluation: Drug-resistant epilepsy requires longer follow-up, multi-trial outcome data, and likely a reformulated label for proper evaluation.The paper defines drug resistance as failure after adequate trials of two tolerated, appropriately chosen and used antiepileptic drug schedules.
  • D.2 Clinical validity of EM@3 under outcome-stratified evaluation: 86.0% EM@3, MANANA-Multi outperforms the frozen single-agent baseline at 80.7% and prompt-learning baselines TextGrad at 79.7% and MANANA-Single at 82.3%.This is the cleanest physician-agreement claim available because the prescribed regimen is corroborated by reduced seizure burden.
  • E Single-Agent Audit and Expert System CONSILIUM: Neurologist review found recurring clinical patterns in single-agent errors, motivating CONSILIUM as an expert-designed comparator.The appendix positions CONSILIUM as the bridge from direct prompting to an expert-designed reference system.

E.1 Single-Agent Failure Audit · E.2 Expert-Designed CONSILIUM Comparator

Neurologist review found that single-agent errors repeatedly involved local, longitudinal, pediatric, and safety-sensitive reasoning gaps, motivating CONSILIUM’s specialist-lens decomposition and synthesis. CONSILIUM separates independent specialist reports, epileptologist ranking, and pharmacologist safety review, while ablations test whether the council structure is reducible.

  • E.1 Single-Agent Failure Audit: Single-agent misses clustered around active-medication timelines, seizure classification, pediatric context, drug access, polytherapy escalation, interaction risk, and infectious triggers.Neurologists often judged predictions clinically plausible while identifying recurring reasoning gaps such as timeline drift, missed classification, pediatric dosing, formulary constraints, and unsafe combinations.
  • E.1 Single-Agent Failure Audit: The audit sampled 20 Cohort A patients across 60 visits, intentionally enriching cases with correct and incorrect monotherapy and polytherapy predictions.The sample included 10 incorrectly predicted polytherapy patients, 5 correctly predicted polytherapy patients, 3 incorrectly predicted monotherapy patients, and 2 correctly predicted monotherapy patients.
  • E.1 Single-Agent Failure Audit: 104/120 (86.7%) neurologist-visit judgments accepted the LLM, while 16/120 (13.3%) did not.The binary endpoint coded agreement with both the LLM and physician, or with the LLM only, as LLM accepted.
  • E.1 Single-Agent Failure Audit: 90.0% exact binary agreement, Cohen’s κ 0.57, Gwet’s AC1 0.87, and PABAK 0.80 characterized inter-rater agreement across 60 shared visits.Because LLM acceptance was common, the paper treats feedback and comments descriptively to identify failure modes rather than claim clinical superiority.
  • E.1 Single-Agent Failure Audit: CONSILIUM assigns recurring reasoning gaps to specialist agents, requests independent reports, and requires final synthesis rather than relying on one monolithic prompt.This design tests whether separating clinical lenses provides more reliable reasoning than adding generic epilepsy knowledge.
  • E.2 Expert-Designed CONSILIUM Comparator: CONSILIUM routes each visit to specialist agents, has an epileptologist combine their reports into three ranked regimens, and uses pharmacologist review for dosing, safety, and interactions.If the pharmacologist raises concerns, the critique returns to the epileptologist before final regimens are produced.
  • E.2 Expert-Designed CONSILIUM Comparator: All-agents-combined controls for prompt content by placing CONSILIUM’s specialist roles in one direct prompt, testing whether council structure adds value beyond listing the same lenses.Appendix L ablations report that the full council is strongest overall, while different specialists matter in different cohort–visit slices.

F Cross-Model Transfer Learning … G.3 Model Capacity and Specialist Decomposition

MANANA memories transfer from stronger to smaller models, improving performance across most cohort–visit pairs and acting as portable clinical guidance. Across additional LLM backbones, specialist decomposition is capacity-dependent, with single-buffer MANANA more consistent on the smallest models.

  • F Cross-Model Transfer Learning: 120B-derived MANANA memory improves native 20B learning on five of six cohort–visit pairs, including all three visits in Cohort A.Cohort B shows a small drop at V1 but gains at V2 and V3.
  • F Cross-Model Transfer Learning: 120B→20B transfer also benefits TextGrad, but its transferred memories are less consistently aligned with clinical task structure than MANANA.
  • F Cross-Model Transfer Learning: Transferred MANANA memory acts as portable clinical guidance, distilling recurring prescribing corrections for smaller models that would otherwise learn weaker memories.This supports offline adaptation with larger models and deployment with smaller models.
  • G Additional Open Source Model Experiments: MANANA is replicated across additional Bedrock-available LLM backbones spanning different scales and architectures, using identical pipelines, patient splits, and 15 training rounds.Test inference uses the round selected by held-out evaluation accuracy.
  • G.1 Models Evaluated: The evaluated backbones include Gemma-3-12B, Ministral-3-14B, GPT-OSS-20B, Gemma-3-27B, Qwen3-32B, and GPT-OSS-120B.
  • G.2 Test-Set Performance Across Models: Held-out test performance is reported for each LLM backbone and MANANA variant, with EM@3 (%) evaluated on Cohort A and Cohort B test sets.The best-per-backbone round is selected by held-out evaluation accuracy.
  • G.3 Model Capacity and Specialist Decomposition: Specialist decomposition helps only when model capacity supports reliable prompt construction and coordination; smaller Gemma-3-12B and Ministral-3-14B models often struggled with Multi.Single performs more consistently than Multi on the smallest evaluated models.

H MANANA Component Ablations … J.1 Beta-Binomial BPA Derivation

The merged sections define MANANA component ablations, report a qualitative neurologist audit of MANANA-Multi versus CONSILIUM, and derive the Beta-Binomial Bayesian prompt-averaging score. The ablations isolate memory, inspection, architectural consolidation, quorum, and in-context-learning contributions, while the derivation models top-3 hits and candidate positions under a shared empirical distribution.

  • H MANANA Component Ablations: Component ablations remove one MANANA-loop element at a time while preserving the remaining system, using identical 50-train/20-eval splits and 15 rounds across Cohorts A and B.Evaluations cover no-buffer and no-inspector variants for both single-buffer and multi-agent MANANA, plus a single-buffer no-architect condition.
  • H MANANA Component Ablations: No-buffer removes cross-round candidate memory while retaining the learned prompt state, isolating candidate-buffer effects from persistent learned state.The Architect still receives the current prompt state and current Inspector reports.
  • H MANANA Component Ablations: No-inspector replaces structured diagnostic reports with raw patient notes, three predictions, and ground truth, testing whether raw signals suffice for Architect updates.The removed diagnostic decomposition includes match status, error type, root cause, and missed signal.
  • H MANANA Component Ablations: No-architect appends Inspector-produced learnings directly to the shared rule list, removing quorum checks, deduplication, rewriting, and cross-case synthesis.This condition is defined only for the single-buffer variant.
  • I Clinician Review of MANANA: The neurologist audit reviewed 120 system-visit outputs across 20 longitudinal patients, finding near-ceiling ratings and emphasizing option coherence qualitatively.MANANA-Multi often proposed related variants of one clinical plan, whereas CONSILIUM sometimes proposed individually plausible but directionally different options.
  • I Clinician Review of MANANA: The clinician observations motivate a planned deeper audit of when model recommendations are clinically coherent.The current review is presented primarily as a qualitative audit because neurologist ratings were near ceiling across answered fields.
  • J MANANA Bayesian Prompt Averaging Ablations: For each retained memory state, Bayesian prompt averaging scores three outcomes: the physician regimen appears first, appears in positions 2–3, or is absent from the three candidates.Counts c_k,1, c_k,>1, and u_k define the state-specific top-3 hit count h_k = c_k,1 + c_k,>1.
  • J.1 Beta-Binomial BPA Derivation: The Beta-Binomial derivation factors likelihood into a state-specific top-3 hit probability and conditional candidate-position distribution, using π = (0.85, 0.11, 0.04) with θ_k ∼ Beta(1, 1).The resulting integrated score supplies Beta-Binomial BPA weights; a Dirichlet-Multinomial variant instead estimates position distributions separately by memory state.

J.2 Weighting Ablations · K MIMIC-IV

Weighting ablations found similar predictions across three BPA variants, indicating that the deferral signal is not tied to one fragile weighting rule. MIMIC-IV provides a reproducibility and robustness setting, but differs substantially from the primary Ugandan longitudinal outpatient epilepsy-care task.

  • J.2 Weighting Ablations: Three BPA weighting variants produce similar predictions, suggesting the deferral signal is not an artifact of a fragile weighting rule.The compared variants are Beta-Binomial BPA, linear weighting, and Dirichlet-Multinomial BPA.
  • K MIMIC-IV: MIMIC-IV differs from the primary task because it uses retrospective hospital discharge summaries from a high-resource US system rather than longitudinal Ugandan outpatient notes.The Ugandan task infers locally appropriate anti-seizure medication regimens from sparse documentation before or during serial visits.
  • K MIMIC-IV: MIMIC-IV is used for reproducibility and robustness because it is the closest public, credentialed-access source with epilepsy-related clinical notes and medication records.The cohort is adult and hospital-based, with median age 49 years, IQR 34–60, and 55.3% female admissions.
  • K MIMIC-IV: 1,977 admissions from 1,257 patients remain after filtering epilepsy admissions, restricting the action space, extracting ground truth, and removing audited leaky admissions.The final cohort contains 31.6% monotherapy and 68.4% polytherapy, with a maximum ground-truth regimen size of 4.
  • K MIMIC-IV: Ground truth comes from the prescription table and targets oral or enteral, scheduled, active-at-discharge ASMs newly started during admission.Fosphenytoin is excluded, and the MIMIC action space contains 15 ASMs meeting the 4% frequency threshold.
  • K MIMIC-IV: Discharge-side sections are removed so the cleaned input retains pre-admission and admission evidence without hospital discharge decisions.An LLM performs the cleaning, followed by regex removal of residual discharge-tail headers and exclusion of manually audited leaky admissions.
  • K MIMIC-IV: Held-out MIMIC-IV performance uses test split seed 42, with identical 50-patient training and 20-patient validation splits across learning systems.MANANA-Single, MANANA-Multi, TextGrad, and ExpeL use their best learned checkpoints, while Base Prompt evaluates the unadapted single-agent template.
  • K MIMIC-IV: Table 15 evaluates top-1 and top-3 exact-match rates over the 15-ASM action space, including monotherapy and polytherapy stratification for top-3 exact match.EpiPick is evaluated on all monotherapy admissions, whereas Poly@3 does not apply to monotherapy-only evaluation.

L CONSILIUM Council Ablations

The CONSILIUM council is evaluated through leave-one-out and only-one ablations. Results are reported as EM@3 percentages across visits in each cohort.

  • CONSILIUM ablations test leave-one-out and only-one council configurations.
  • The ablation table reports EM@3 percentages across visits for each cohort.

M Learned Artifacts Across Optimization Methods

Learned artifacts differ in grounding, specificity, and whether they preserve clinical memory. TextGrad and ExpeL encode broad or generic guidance, DSPy-GEPA sharpens task instructions without accumulated memory, while MANANA variants retain auditable local clinical signals.

  • Overview: Representative artifacts illustrate qualitative differences among optimization methods, while complete artifacts are supplied in the released code documentation.The section focuses on differences that matter for interpreting the results.
  • TextGrad: TextGrad learns detailed prose containing dosing, laboratory, formulation, pill-burden, stocking, and cost assumptions unsupported by prescribed-drug-set labels.Its global instruction artifact resembles a guideline document but is weakly grounded in the available supervision.
  • ExpeL: ExpeL learns plausible generic heuristics—continuation, locally common first-line agents, and weight-based dosing—but not recurring drug- or seizure-specific local error patterns.Its ranked experience memory remains at the level of generic best practice.
  • DSPy-GEPA: DSPy-GEPA optimizes interface behavior for directive parsing, formatting, dose adequacy, and avoiding unnecessary additions, but lacks persistent cross-patient clinical memory.Its artifact is an instruction for the prediction program rather than an accumulating clinical memory.
  • MANANA-Single: MANANA-Single learns compact, evidence-gated correction rules naming specific drugs and situations, including continuation, levetiracetam addition, and avoiding escalation after fever-related breakthrough.These rules survive the Architect’s quorum check across cases.
  • MANANA-Multi: MANANA-Multi converts recurring errors into specialist signal extractors that surface clinical observations for the Predictor instead of directly writing prescriptions.This makes its learned memory auditable as clinical signal extraction rather than a hidden decision policy.
Loading 2606.31036v1…