PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Uncertainty Quantification

Jiaxin Zhang, Prafulla Kumar Choubey, Kung-Hsiang Huang, Caiming Xiong, Chien-Sheng Wu

arXiv:2601.15703v1cs.AIcs.CL

TL;DR

Long-horizon agents remain vulnerable to compounding epistemic errors, while existing uncertainty estimation and reflection methods provide incomplete control. AUQ combines uncertainty-aware memory with targeted reflection, achieving superior performance and trajectory-level calibration across closed-loop and deep-research tasks.

  • Problem

    Existing UQ methods diagnose risk without resolving it, while self-reflection can trigger blindly or incessantly, leaving reliable long-horizon control unresolved.

  • Method

    AUQ combines System 1 uncertainty-aware memory with System 2 reflection, activating targeted correction when self-evaluated confidence falls below a reliability threshold.

  • Results

    AUQ delivers superior performance, calibration, and self-awareness across extensive experiments; with one-step memory, it retains a +5.0% performance advantage.

  • Takeaways & Limitations

    The results suggest principled agentic UQ can support more reliable and adaptive autonomous agents by combining fast uncertainty propagation with reflective calibration.

  • Takeaways & Limitations

    AUQ assumes the underlying LLM can express uncertainty reliably, but this capability diminishes in models with fewer than 7 billion parameters.

Abstract

from arXiv · show

Although AI agents have demonstrated impressive capabilities in long-horizon reasoning, their reliability is severely hampered by the ``Spiral of Hallucination,'' where early epistemic errors propagate irreversibly. Existing methods face a dilemma: uncertainty quantification (UQ) methods typically act as passive sensors, only diagnosing risks without addressing them, while self-reflection mechanisms suffer from continuous or aimless corrections. To bridge this gap, we propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals. Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary. This enables the agent to balance efficient execution and deep deliberation dynamically. Extensive experiments on closed-loop benchmarks and open-ended deep research tasks demonstrate that our training-free approach achieves superior performance and trajectory-level calibration. We believe this principled framework AUQ represents a significant step towards reliable agents.

1 Introduction

The paper frames long-horizon agent reliability as a control problem: early epistemic errors can spiral into irreversible failures, while existing UQ and reflection methods do not resolve this gap. It proposes training-free Dual-Process Agentic UQ, using uncertainty-aware memory and reflection to constrain execution and target deliberation.

  • Motivation: Early grounding errors can propagate through an agent’s context, biasing later planning toward irreversible failure states.Agents therefore need to detect deviation before errors propagate.
  • Framework: The dual-process architecture combines System 1 Uncertainty-Aware Memory with System 2 Uncertainty-Aware Reflection to constrain fast execution and guide targeted deliberation.Memory implicitly propagates confidence and explanations, while reflection uses them to support inference-time correction.
  • Motivation: Existing UQ methods quantify compounded epistemic risk but lack mechanisms to resolve it, whereas unguided reflection can be inefficient or sycophantic.This creates a gap between passive diagnosis and active correction, especially without ground-truth labels.
  • Framework: AUQ transforms uncertainty into bidirectional control signals through Forward Propagation for constraint and Inverse Calibration for problem-solving.Uncertainty-Aware Memory retains verbalized confidence and explanations to create a soft cognitive constraint.
  • Contributions and Evaluation: AUQ formally separates Forward Uncertainty Propagation from Inverse Uncertainty Calibration and introduces trajectory-level evaluation for long-horizon reliability.The training-free framework is evaluated across ALFWorld, WebShop, and DeepResearch, with reported superior performance and trajectory-level calibration.

2 Preliminaries

The paper models long-horizon agents as history-dependent policies in a partially observable environment, where reliability fails when implicit beliefs diverge from latent states. It distinguishes environmental stochasticity from reducible cognitive deficiency and formulates uncertainty handling as forward propagation and inverse calibration.

  • Agent Modeling: Long-horizon agents are modeled as POMDPs E = (S, A, Ω, T, R) with history-dependent policies π(a_t|h_t).The observed history is h_t = (o_0, a_0, . . . , o_t), while the true state remains latent.
  • Agent Modeling: Reliability failure occurs when the implicit belief b_t(s_t) = P(s_t|h_t) diverges from the true state while the policy continues acting on that flawed belief.
  • Agentic Uncertainty: Aleatoric uncertainty represents environmental stochasticity and irreducible noise, whereas epistemic uncertainty represents reasoning limitations such as hallucinations, logic gaps, and memory failures.Epistemic uncertainty is reducible through better reasoning strategies or external knowledge.
  • Agentic Uncertainty: The Spiral of Hallucination occurs when epistemic errors committed to history h_t become contextual constraints that influence future steps.
  • Propagation and Calibration: The forward problem propagates confidence through time, estimating trajectory validity while accounting for accumulated historical risk.
  • Propagation and Calibration: The inverse problem seeks a corrected action a∗ when P(V_t|h_t) < δ, maximizing the likelihood of a reliable outcome through posterior optimization.The paper frames this low-confidence correction as Test-Time Calibration and an inverse search for a reliable plan.

3 Methodology

The methodology introduces a Dual-Process UQ framework that uses fast uncertainty propagation through memory and slow uncertainty resolution through reflection. A confidence-triggered switching policy balances reliability with inference efficiency, while trajectory-level calibration evaluates confidence over complete agent trajectories.

  • Dual-Process Framework: The Dual-Process UQ framework separates uncertainty handling into a fast memory-augmented System 1 and a slow reflection-based System 2.System 1 performs forward uncertainty propagation, whereas System 2 performs inverse uncertainty calibration.
  • System 1: Uncertainty-Aware Memory: Uncertainty-Aware Memory preserves verbalized confidence and semantic explanations so prior reliability signals remain available as the context window shifts.Semantic uncertainty propagation conditions subsequent actions on articulated doubts, encouraging information gathering over high-commitment exploitation.
  • System 2: Uncertainty-Aware Reflection: Uncertainty-Aware Reflection treats explanations as diagnostic constraints, generating corrected actions through inverse optimization and consistency-weighted Best-of-N reflection.The method selects candidates using a score that rewards both confidence and semantic consistency.
  • Adaptive Switching Policy: The dual-process policy activates System 2 when self-evaluated confidence falls below threshold τ, while higher-confidence actions use System 1 directly.The threshold is empirically determined and typically lies in τ ∈[0.8, 1). Corrected results are written back with updated confidence.
  • Trajectory-Level Calibration: Trajectory-Level Calibration aggregates stepwise confidence using end-state, weakest-link, or average-confidence beliefs because local calibration does not capture whole-trajectory failure.The framework defines trajectory confidence aggregation to evaluate reliability across complete long-horizon tasks.

4 Experiments

Experiments evaluate AUQ across embodied, ecommerce, and open-ended research benchmarks using performance, calibration, and discrimination metrics. AUQ improves task outcomes and reliability through uncertainty-aware reflection, memory, dynamic retrieval, and cost-sensitive deliberation.

  • Evaluation Setup: AUQ is evaluated on ALFWorld, WebShop, and DeepResearch Bench against System 1, self-correction, and unguided reflection baselines.Metrics cover Success Rate (SR), DeepResearch RACE scores, Trajectory-ECE, Trajectory Brier Score (T-BS), and discriminative AUROC.
  • Calibration and Discrimination: UAM improves calibration by retaining verbalized uncertainty, while AUQ’s internal confidence signals distinguish successful from failed trajectories and target System 2 computation.Compared with ReAct, the framework corrects 14.3% of failures while regressing on only 3.6% of successes.
  • Closed-Loop Benchmarks: 42.5% SR (+13.6%) on WebShop and the reported ALFWorld result demonstrate task-performance gains over baselines.The passage reports the WebShop figure and a preceding ALFWorld result as 42.5% SR (+13.6%) and SR (+10.7%), respectively.
  • DeepResearch Bench: AUQ achieves an Overall Score of 52.09 on DeepResearch Bench, exceeding the strongest closed baseline at 49.71 and the best open-source competitor at 50.62.Its advantages are attributed particularly to Insight (54.21) and Comprehensiveness (51.60), with inverse UQ triggering deeper reflection on epistemic gaps.
  • Memory and Generalization: With h = 1, AUQ maintains significantly higher performance (+5.0), and adaptive memory expansion yields +17.9% for GPT-5.1 in the constrained h = 5 setting.AUQ also outperforms ReAct by +11.0% with limited memory and +7.7% with full memory across diverse models.
  • Cost Efficiency: τ ≈0.9 is the reported efficiency sweet spot, whereas τ = 0.95 produces diminishing accuracy returns and exponentially higher inference costs.The Pareto analysis describes over-verification of trivial steps at the higher threshold.

5 Conclusion

The paper proposes a dual-process agentic UQ framework that bridges calibration and autonomous reasoning. It combines fast memory-aware uncertainty propagation with slow reflective calibration to mitigate hallucination spirals and improve performance, calibration, and self-awareness.

  • The proposed dual-process agentic UQ framework bridges calibration and autonomous reasoning.
  • System 1 performs fast, memory-aware uncertainty propagation, while System 2 performs slow, reflective calibration.
  • Extensive experiments show that the approach mitigates the hallucination spiral and achieves superior performance, calibration, and self-awareness.

Limitations

The framework assumes that the underlying LLM can express uncertainty, with verbalized confidence correlating strongly with correctness in strong models.

  • Limitations: The framework depends on the underlying LLM possessing a latent ability to express uncertainty.The authors identify this premise as a limitation that defines the scope of the current work and motivates future research.
  • Limitations: Verbalized confidence correlates strongly with correctness in strong LLM models such as GPT-5.

Ethical Considerations

The framework aims to enhance autonomous-agent reliability but may encourage users to over-rely on verbalized confidence, especially in high-stakes domains where calibrated agents can still hallucinate.

  • Ethical Considerations: AUQ improves agent calibration but may foster automation bias and over-reliance on verbalized confidence.Users may perceive high confidence as a guarantee of factual correctness.
  • Ethical Considerations: Even calibrated agents can hallucinate in high-stakes domains such as medical or legal research.The passage identifies these domains as examples where residual hallucination risk remains consequential.
  • Ethical Considerations: The work frames AUQ as a framework for enhancing the reliability of autonomous agents.

A Appendix · Appendix Contents · A.1 Related Work

The related work frames reliable agentic AI around uncertainty quantification, error propagation, failure attribution, calibration, and self-correction. It motivates approaches that move beyond diagnosis or externally triggered correction toward uncertainty-aware, adaptable agent behavior.

  • A.1 Related Work: Classical calibration works well for discriminative tasks, but uncertainty quantification for open-ended LLM generation remains non-trivial and has bifurcated into logit-based and linguistic-based approaches.Semantic Entropy is identified as an example of a logit-based method.
  • A.1 Related Work: ReAct- and Reflexion-style reasoning-acting systems enable long-horizon tasks but remain brittle in dynamic environments because errors propagate across sequential interactions.This problem is described as the “Curse of Recursion” or “Error Propagation.”
  • A.1 Related Work: Automated failure-attribution methods analyze agent trajectories after episodes, using critique models to trace causes such as planning brittleness and grounding errors.Related work also studies coordination collapse in multi-agent systems, where individual delusions can spread to system-wide failures.
  • A.1 Related Work: Failure-attribution and coordination-collapse studies provide taxonomies of why and where agents fail, but remain predominantly diagnostic or depend on separate debugging layers.These approaches generally do not directly control the agent during execution.
  • A.1 Related Work: Agentic uncertainty research addresses challenges distinct from static text generation, especially the sequential nature of confidence and uncertainty propagation through trajectories.UProp and SAUP are cited as early frameworks explicitly modeling uncertainty propagation.
  • A.1 Related Work: Self-Refine, Self-Reflection, and Reflexion show that iterative refinement can improve outputs, but typically require explicit failure signals or ground-truth oracles to trigger correction.This limitation is especially relevant to open-ended reasoning tasks lacking environmental feedback.
  • A.1 Related Work: Critique-and-refine methods add verification through external tools or self-generated critiques, yet can produce sycophantic confirmation when the model lacks knowledge to solve the problem.STeCa improves trajectory-level calibration through reward modeling and supervised fine-tuning, but requires expensive expert-trajectory training and adapts poorly to new base models.

A.2 Formal Mathematics in Problem Formulation

The formalization models trajectory validity recursively as a forward probabilistic process and reliable planning as an inverse Bayesian calibration problem. Forward uncertainty accumulates across actions, while Best-of-N Reflection approximates posterior optimization over latent reasoning paths.

  • A.2.1 Forward Problem: Recursive Validity Estimation: Trajectory validity P(V_t|h_t) is defined recursively as the probability that history remains valid and the current action is correct.V_t ∈ {0, 1} indicates whether the trajectory through step t is free of critical epistemic errors.
  • A.2.1 Forward Problem: Recursive Validity Estimation: The recursive product makes P(V_t = 1) non-increasing with t, so one near-zero correctness probability can invalidate every later trajectory state.The formulation identifies this propagation as the Spiral of Hallucination.
  • A.2.1 Forward Problem: Recursive Validity Estimation: Forward UQ approximates joint trajectory validity by multiplying verbalized confidence scores, or conservatively taking their minimum.The passage gives P(V_t|h_t) ≈ ∏_{i=0}^t ĉ_i and the alternative min(ĉ_0:t).
  • A.2.2 Inverse Problem: Latent Variable Calibration: When forward confidence is low, reliable-plan generation is treated as an inverse search using inference-time computation without updating model parameters.Reflection is given as an example of this test-time calibration process.
  • A.2.2 Inverse Problem: Latent Variable Calibration: The inverse objective maximizes posterior action quality by introducing latent reasoning path z and an optimality variable O, where O = 1 implies success.z may represent a Chain-of-Thought or Reflection explanation.
  • A.2.2 Inverse Problem: Latent Variable Calibration: Bayesian expansion weights each latent path by generation prior and likelihood P(O = 1|z, h_t), representing reasoning consistency.The posterior is obtained by marginalizing over z and applying Bayes’ rule.
  • A.2.2 Inverse Problem: Latent Variable Calibration: Best-of-N Reflection approximates the intractable latent-path integral through Monte Carlo sampling, reweighting candidates by consistency or confidence, then selecting the highest-weighted action.The derivation characterizes this procedure as a particle-based approximation of optimal inverse calibration.

A.3 Detailed Experimental Setup · A.3.1 Datasets and Evaluation Protocols

The evaluation spans three benchmarks covering deterministic planning, noisy web navigation, and open-ended research synthesis. Their protocols assess success rate in the first two settings and dynamically weighted RACE quality in the third.

  • A.3.1 Datasets and Evaluation Protocols: Three benchmarks span rigid logic-heavy planning to open-ended creative synthesis, testing whether Agentic UQ generalizes across modalities.The selected tasks cover deterministic planning, stochastic interaction, and open-ended reasoning.
  • A.3.1 Datasets and Evaluation Protocols: ALFWorld is a text-based simulated household task requiring low-level action sequences to achieve high-level goals.It aligns TextWorld with ALFRED and evaluates embodied decision making in household environments.
  • A.3.1 Datasets and Evaluation Protocols: ALFWorld uses deterministic planning with long-horizon dependencies, where one omitted step can cause failure and test cognitive continuity.The Seen Evaluation Set contains 140 unique environments, and performance is measured by Success Rate (SR).
  • A.3.1 Datasets and Evaluation Protocols: WebShop simulates noisy ecommerce navigation across 1.18 million products, requiring agents to search, browse, and select options under user constraints.Its stochastic interactions test filtering noise and verifying information before commitment.
  • A.3.1 Datasets and Evaluation Protocols: WebShop samples 140 episodes from the standard Development Set to match ALFWorld’s sample size, using Success Rate (SR) as its evaluation metric.The sampled episodes preserve a consistent evaluation scale across the two closed-loop benchmarks.
  • A.3.1 Datasets and Evaluation Protocols: DeepResearch Bench contains 100 PhD-level tasks across 22 fields, split evenly between Chinese and English, requiring autonomous web research and comprehensive reports.It evaluates open-ended synthesis rather than a single correct action sequence.
  • A.3.1 Datasets and Evaluation Protocols: DeepResearch Bench uses dynamically weighted RACE scoring for comprehensiveness, insight, instruction following, and readability, while excluding FACT to isolate reasoning.The criteria adapt to the reference report’s complexity rather than relying on static grading.

A.3.2 Baselines and Variants … Summary of Trajectory Refinement

AUQ combines uncertainty-aware memory with threshold-triggered reflection, using verbalized confidence and explanations to propagate uncertainty and selectively refine actions or research plans. The qualitative trajectories show targeted correction, improved efficiency, and remaining risks from over-correction and capability limits.

  • Baselines Implementation: AUQ combines UAM uncertainty propagation with UAR correction, switching to reflection when confidence falls below threshold τ.The full variant preserves corrected actions and new confidence metadata in memory, combining local correction with long-term uncertainty propagation.
  • Calculation of Consistency-Weighted Confidence: Consistency-weighted confidence separates semantic action consistency from epistemic confidence, distinguishing conflictual uncertainty from uniformly low-confidence task difficulty.Structured environments use normalized action-string matching, while open-ended research uses model-based equivalence checks for semantically equivalent plans.
  • A.3.3 Implementation Details: Reflection samples N = 3 parallel responses and refines low-confidence paths for at most D = 3 turns, with full-history expansion reserved for stubborn uncertainty.In limited memory with h = 5, adaptive expansion reruns reflection only when the best candidate still fails threshold τ.
  • A.3.4 Evaluation Metrics and Protocols: Trajectory-level evaluation aggregates confidence using end-state, average, or minimum-process beliefs, then measures calibration with T-ECE, T-BS, and AUROC.Open-ended reports are judged by Gemini-2.5-Pro using RACE across comprehensiveness, insight/depth, instruction following, and readability.
  • Comparative Outcome Analysis (AUQ vs. ReAct): 14.3% Correction outweighed 3.6% Regression, while AUQ used 13.7 steps versus ReAct’s 16.2 steps on shared-success tasks.AUQ corrected early hallucination spirals but occasionally over-reflected correct actions; shared failures remained bounded by the base model’s capabilities.
  • The Economics of Reliability: Success-Weighted Cost: AUQ trades reflection overhead for fewer futile environmental interactions, reducing total solution steps and achieving approximately +20% Success Rate despite ≈1.4× per-trajectory token cost in ALFWorld.Its inference-time correction occurs within one episode, contrasting with Reflexion’s multiple full trials and their multiplied token cost.
  • Key Improvements Analysis: In qualitative cases, AUQ prevented a greedy bowl-manipulation loop by identifying the missing lamp and completed the ALFWorld task in 8 steps versus the baseline’s 30+ steps.For deep research, three parallel reflection branches cumulatively refined the plan, avoiding local optima and producing more authoritative, granular queries.
  • Summary of Trajectory Refinement: In the deep-research case, AUQ raised confidence from 0.88 to 0.96 while expanding the plan from 4 to 9 subtopics and separating spending from population analysis.The refined plan targeted IPSS and UN, switched demographic research to academic_search, retained general_search for consumer trends, and added market-sizing methodology queries.
Loading 2601.15703v1…