PaperlayerView exact revision on arXiv

Source-linked AI summary

CRC-Router: Risk-Constrained Routing for Medical Agentic AI Systems

Xueyang Li, Mingze Jiang, Gelei Xu, Jun Xia, Ching-Hao Chiu, Mengzhao Jia, Danny Z. Chen, Yiyu Shi

arXiv:2609.30714v1cs.AI

TL;DR

Medical agentic systems need reliable routing because autonomous errors may propagate into downstream clinical decisions. CRC-Router combines uncertainty signals and predictive scores, estimates per-finding wrong-accept risk, and uses Conformal Risk Control to calibrate thresholds. It achieves the best empirical risk–coverage trade-off among compared methods in standalone and MedRAX-integrated settings.

  • Problem

    Medical agentic AI lacks a routing mechanism that explicitly calibrates accept-versus-escalate decisions to a user-specified risk constraint.

    Source: §I · p. 1

  • Method

    CRC-Router maps multiple uncertainty signals and predictive scores to per-finding wrong-accept risk, then uses Conformal Risk Control to calibrate routing thresholds.

    Source: §I · p. 1

  • Results

    CRC-Router achieved the best empirical risk–coverage trade-off across standalone and MedRAX settings, with 3.9% macro wrong-accept risk at 91.3% coverage standalone.

    Source: 1) CRC-Router Standalone Results · p. 6, 2) MedRAX Integration Results · p. 6, §VI · p. 7

  • Takeaways & Limitations

    The results support CRC-Router as a modular, model-agnostic routing module for risk-constrained medical AI deployment.

    Source: §VI · p. 7

Abstract

from arXiv · show

Agentic AI systems are increasingly being explored in medical imaging to improve throughput and reduce clinician workload; however, safe deployment remains challenging because autonomous errors may propagate into downstream clinical decisions. A central requirement is therefore not only strong predictive performance, but also a reliable routing mechanism that determines when the system should proceed autonomously and when a case should be escalated for further review. To address this gap, we propose CRC-Router, a risk-constrained, uncertainty-aware routing module that is applicable to both conventional medical prediction models and agentic medical AI systems. CRC-Router combines multiple complementary uncertainty signals with the predictive score to construct a per-finding routing feature vector, maps this vector to an estimated wrong-accept risk using a lightweight per-finding risk model, and then applies Conformal Risk Control (CRC) to calibrate acceptance thresholds under a user-specified risk target. Instantiated on chest X-ray multi-finding triage using the NIH ChestX-ray14 dataset, CRC-Router achieves the strongest empirical risk--coverage trade-off among the evaluated baselines, both as a standalone routing layer and as a plug-in module integrated with the state-of-the-art MedRAX agent. These results demonstrate both the effectiveness of CRC-Router in selective medical automation and its modular, model-agnostic compatibility with existing predictive and agentic medical pipelines. Code is publicly available at https://github.com/XLIAaron/CRC-Router

I. INTRODUCTION

Medical agentic AI needs routing that controls whether predictions proceed autonomously or are escalated, because errors can propagate into downstream clinical decisions. CRC-Router applies explicit risk control to this decision and is positioned as a modular layer for predictive and agentic systems.

I. INTRODUCTION

  • Medical agentic errors can propagate beyond isolated predictions into downstream clinical decisions, so safe deployment requires accounting for risk as well as predictive performance.The passage notes that outputs may affect diagnosis and treatment, with potential consequences including delayed care, financial burden, or patient harm.Source: §I · p. 1
  • Existing uncertainty-aware agentic methods use uncertainty as a coordination heuristic, not as a formally calibrated rule enforcing a user-specified risk constraint.The cited examples include tool orchestration, trajectory propagation, and adaptive intervention [4], [5].Source: §I · p. 1
  • CRC-Router maps multi-signal uncertainty and predictive scores to per-finding risk estimates, then calibrates routing thresholds to a user-specified safety target.The module is designed to work with conventional predictors and agentic systems, including as a plug-in for MedRAX.Source: §I · pp. 1–2

A. Medical Agentic AI and Selective Automation

  • Selective prediction often treats rejection as terminal, whereas medical deployment may route a case to a stronger model, downstream agent, or human expert.The related-work discussion also notes that selective-prediction methods commonly rely on confidence scores or learned rejection functions rather than explicitly calibrated wrong-accept risk.Source: A. Medical Agentic AI and Selective… · p. 2

B. Uncertainty Estimation for Risk-Aware Routing

Single predictive signals provide incomplete evidence for routing: confidence and entropy may miss model disagreement, intrinsic ambiguity, or distribution shift. Conformal Risk Control can calibrate decisions to a user-specified expected-risk target, but uncertainty must still be translated into an actionable accept-or-escalate rule.

B. Uncertainty Estimation for Risk-Aware Routing

  • Predictive confidence and entropy summarize the predictive distribution, but alone may miss miscalibration, out-of-distribution inputs, or the source of uncertainty.Confidence can be near-certain on unrecognizable or far-out-of-distribution inputs, while entropy cannot distinguish model disagreement, intrinsic ambiguity, or distribution shift.Source: B. Uncertainty Estimation for… · p. 2
  • Epistemic uncertainty concerns uncertainty in the model, aleatoric uncertainty reflects intrinsic observation ambiguity, and distributional scoring can capture atypicality relative to training features.These signals motivate combining predictive entropy, epistemic and aleatoric uncertainty, and distributional scoring rather than relying on one confidence score.Source: B. Uncertainty Estimation for… · p. 2

C. Conformal Calibration and Risk Control

  • Conformal prediction can provide coverage guarantees, while Conformal Risk Control calibrates decisions against a user-specified expected-risk target; neither alone makes set-valued outputs an accept-or-escalate action.The paper applies CRC to wrong-accept risk for per-finding routing decisions.Source: C. Conformal Calibration and Risk… · p. 2

III. METHOD

CRC-Router frames medical image triage as per-finding accept-or-escalate routing: maximize autonomous coverage while keeping wrong-accept risk below a user-specified target. Its two stages construct uncertainty and risk estimates, then calibrate deployment thresholds with Conformal Risk Control.

III. METHOD

  • The router accepts a finding for autonomous action or escalates it for additional review, with wrong-accept risk measuring autonomous incorrect decisions.The review may involve a stronger model, downstream agent, or human expert.Source: A. Problem Formulation · p. 2
  • Coverage is the fraction of cases handled autonomously, and the objective is to maximize it subject to per-finding risk Rd ≤ α.Here α is the user-specified target risk level, and the case study uses chest X-ray multi-finding triage.Source: A. Problem Formulation · pp. 2–3
  • CRC-Router has two stages: multi-signal uncertainty construction with per-finding risk modeling, followed by CRC-based threshold calibration for deployment routing.The figure summarizes the pipeline from uncertainty and risk modeling to threshold calibration and accept-versus-escalate decisions.Source: A. Problem Formulation · p. 3, Figure 1 · p. 3

B. Multi-Signal Uncertainty Vector

CRC-Router constructs a per-finding uncertainty representation from committee predictions and an image-level distributional score. Predictive entropy summarizes aggregate output uncertainty, but does not distinguish its source and is insufficient alone for risk-sensitive routing under distribution shift.

B. Multi-Signal Uncertainty Vector

  • CRC-Router builds a per-finding multi-signal uncertainty representation from committee predictions and an image-level distributional score for downstream risk modeling.The committee produces probability predictions for each finding; their mean probability is used as an aggregated prediction.Source: B. Multi-Signal Uncertainty Vector · p. 3

1) Predictive Entropy:

  • The uncertainty vector has four components, and predictive entropy summarizes uncertainty in the aggregated committee prediction.Entropy does not distinguish whether uncertainty arises from model disagreement or intrinsic ambiguity in the input.Source: B. Multi-Signal Uncertainty Vector · p. 3, 1) Predictive Entropy · p. 3
  • Predictive entropy is a compact posterior-level signal and baseline for selective prediction, but is insufficient alone for risk-sensitive routing under distribution shift.The passage identifies this as a limitation of entropy by itself.Source: 1) Predictive Entropy · p. 3

2) Epistemic Uncertainty:

CRC-Router separates uncertainty from committee disagreement from uncertainty within individual predictions. It estimates epistemic uncertainty by committee-prediction variance and aleatoric uncertainty by average committee-member entropy.

2) Epistemic Uncertainty:

  • Epistemic uncertainty is estimated as the variance of committee predictions; larger variance indicates greater disagreement and elevated model uncertainty.The passage relates disagreement to limited training support, model instability, or sensitivity to representation differences.Source: 2) Epistemic Uncertainty · p. 3

3) Aleatoric Uncertainty:

  • Aleatoric uncertainty is estimated as average entropy across committee members and reflects ambiguity within individual predictions, including noisy or visually ambiguous findings.Cases with similar predictive entropy can differ in whether uncertainty comes from model disagreement or consistently uncertain predictions.Source: 3) Aleatoric Uncertainty · p. 3

4) Mahalanobis-Style Distributional Score:

The method adds a handcrafted-feature Mahalanobis score to posterior-based uncertainty signals, then combines these signals with predictive scores for per-finding risk modeling.

4) Mahalanobis-Style Distributional Score:

  • A diagonal Mahalanobis model fitted on training data produces one image-level score from handcrafted radiomics-style intensity, texture, and histogram features.This model-agnostic signal is independent of committee outputs and adds low computational overhead.Source: 4) Mahalanobis-Style Distributional… · pp. 3–4
  • The Mahalanobis score supplies an image-level signal for outlier-like or shifted inputs, complementing posterior-based uncertainty measures.The handcrafted features are used instead of deep embeddings.Source: 4) Mahalanobis-Style Distributional… · pp. 3–4
  • The method augments each finding’s multi-signal uncertainty vector with its predictive score, creating a five-dimensional routing feature vector.A separate risk model is fitted for each finding and supervised using the binary error indicator on an out-of-sample calibration split.Source: 4) Mahalanobis-Style Distributional… · p. 4

C. Conformal Risk Control for Threshold Calibration

CRC converts per-finding predicted wrong-accept risk into a calibrated accept-versus-escalate threshold. The target risk is user-specified, and the stated control relies on exchangeability between calibration and deployment samples.

C. Conformal Risk Control for Threshold Calibration

  • The uncertainty-to-risk model maps routing features to a per-finding score estimating the probability that a prediction is incorrect.CRC calibrates an acceptance threshold on that score to control expected wrong-accept risk at the user-specified level.Source: C. Conformal Risk Control for… · p. 4
  • For each finding, the threshold-induced loss is an indicator of a wrong-accept event, and its expectation defines the threshold-dependent wrong-accept risk.The same threshold determines coverage, linking calibration to the risk–coverage objective.Source: C. Conformal Risk Control for… · p. 4
  • CRC calibrates a per-finding threshold on a separate calibration split to meet the user’s expected-risk target under calibration–deployment exchangeability.The calibrated threshold is then used to implement the deployment-time routing rule.Source: C. Conformal Risk Control for… · p. 4
  • The resulting module combines multi-signal uncertainty, learned risk estimation, and CRC calibration while aiming to preserve autonomous coverage.It can attach to predictive or agentic medical systems without modifying their underlying predictors.Source: C. Conformal Risk Control for… · p. 4

IV. EXPERIMENTS

The experiments assess standalone CRC-Router on NIH ChestX-ray14 and its integration with MedRAX on NIH ChestX-ray14 and CheXpert, using risk–coverage evaluation.

IV. EXPERIMENTS

  • Standalone evaluation uses NIH ChestX-ray14, a patient-level 0.5/0.25/0.25 train/calibration/test split, and per-disease CRC thresholds calibrated at α = 5%.The calibration data are divided equally between risk-model training and CRC calibration; the committee contains five multi-label classifiers.Source: §IV · p. 4
  • The standalone comparison includes MaxProb, MCD-MI, and Conformal Prediction, with baseline thresholds selected to approximately match CRC-Router’s macro coverage.Risk is evaluated at the matched operating point.Source: §IV · pp. 4–5
  • MedRAX integration is evaluated on NIH ChestX-ray14 and CheXpert, including unguided MedRAX and CRC-Router with test-time augmentation; the MoE condition is NIH-only.CheXpert uses stratified subsamples for risk-model training, calibration, and testing.Source: §IV · p. 5
  • Evaluation reports empirical wrong-accept risk per disease, per-disease coverage, and the number of diseases meeting the target risk; Table I covers selective methods at approximately 91% macro coverage.Table II reports per-disease wrong-accept risk for MedRAX integration at α = 5% on NIH ChestX-ray14 and CheXpert.Source: §IV · p. 5, Table I · p. 5, Table II · p. 5
  • Figure 2 presents risk–coverage curves and prevalence enrichment among escalated cases on NIH ChestX-ray14.The experiment sets α = 5%.Source: Figure 2 · p. 6, §IV · p. 6

1) CRC-Router Standalone Results:

CRC-Router achieves low macro wrong-accept risk at high coverage and enriches the escalated queue with cases from higher-prevalence disease groups.

1) CRC-Router Standalone Results:

  • 3.9% macro wrong-accept risk at 91.3% macro coverage is the lowest risk among selective routing methods at matched coverage.Accept-All has 9.2% macro wrong-accept risk; CRC-Router reduces risk by 58% while sacrificing 8.7 percentage points of coverage.Source: 1) CRC-Router Standalone Results · p. 6
  • Across 10 diseases with non-trivial escalation rates, escalated-case prevalence reaches 1.5–7.0× natural population prevalence.The four near-100%-acceptance diseases are omitted from Fig. 2(b) because they produce negligible escalation queues.Source: 1) CRC-Router Standalone Results · p. 6
  • Diseases omitted from Fig. 2(b) produce negligible escalation queues.The passage identifies the omission reason but does not specify the diseases in this request's evidence set.Source: 1) CRC-Router Standalone Results · p. 6

2) MedRAX Integration Results:

CRC-Router achieves the strongest empirical risk–coverage trade-off in standalone and MedRAX-integrated settings. The integration results and feature ablation support its modular routing approach and show contributions from both uncertainty and prediction signals.

2) MedRAX Integration Results:

  • On NIH ChestX-ray14, CRC-Router (MoE) reaches 91.3% coverage at 3.8% risk; CRC-Router (TTA) raises MedRAX coverage from 62.9% to 81.5% and lowers risk from 5.7% to 4.8%.On CheXpert, CRC-Router (TTA) lowers macro risk from 20.9% to 4.6% while coverage rises from 65.6% to 68.4%.Source: 2) MedRAX Integration Results · p. 6

3) Ablation Study:

  • Removing epistemic uncertainty lowers macro coverage by 3.1%; removing predictive score p̄ lowers it by 6.2%, the largest overall reduction.The full model attains 91.3% macro coverage at the target risk level; removing any individual component decreases coverage.Source: 3) Ablation Study · p. 7

VI. CONCLUSIONS

  • Across standalone and MedRAX-integrated settings, CRC-Router achieves the best empirical risk–coverage trade-off among compared methods.The conclusion describes the module as applicable to conventional prediction models and agentic medical AI systems.Source: §VI · p. 7

Related papers

Loading 2609.30714v1…