PaperlayerView exact revision on arXiv

Source-linked AI summary

CalVerT: Augmenting Agents with Calibrated Verifier Telemetry Improves Action and Learning in Knowledge-Intensive Tasks

Ashwin Vinod, Ying Ding, Elias Stengel-Eskin

arXiv:2606.21777v1cs.CLcs.AI

TL;DR

Knowledge-intensive QA agents lack reliable signals about answer confidence and evidence support, causing unsupported commitments and redundant retrieval. CalVerT adds calibrated confidence and grounding telemetry to agent states, improving the accuracy-cost tradeoff across training-free and trained settings, including +5.9% F1 on HotpotQA distractor with Qwen3-8B.

  • Problem

    Knowledge-intensive QA agents lack measurements of answer confidence and evidence support, contributing to unsupported commitments and redundant retrieval.

  • Method

    CalVerT augments each agent state with calibrated self-confidence and per-claim grounding signals before action selection.

  • Results

    CalVerT improves the accuracy-cost tradeoff across prompt-only, framework-substitution, and GRPO settings, including +5.9% F1 for Qwen3-8B on HotpotQA distractor.

  • Takeaways & Limitations

    Calibrated telemetry provides a framework-agnostic mechanism for better agent control by reducing redundant actions and encouraging retrieval when appropriate.

  • Takeaways & Limitations

    CalVerT is evaluated on single-answer multi-hop QA and does not yet support answer-recall tasks requiring complete answer sets.

Abstract

from arXiv · show

LLM agents in knowledge intensive question answering take retrieval and reasoning actions with incomplete knowledge about whether their current answer is uncertain, unsupported, or already complete. This produces two failure modes: committing to confident but unsupported answers, which hurts accuracy, and over-retrieving when the evidence in hand already suffices, resulting in wasted compute. To give agents a more complete picture of the state space they are operating in, we introduce calibrated verifier telemetry (CalVerT), which augments the agent's state with additional telemetry: a calibrated self-confidence score and a grounding verifier score. We show that CalVerT can improve agents in both training-free and training-based settings. On four QA benchmarks, we find that CalVerT raises F1 by triggering retrieval in cases where agents over-rely on parametric knowledge, while cutting redundant retrieval in cases where agents have sufficient context to answer. We show that CalVerT can augment existing QA frameworks without training. Moreover, CalVerT also improves trained systems: by simply augmenting an agent's state with telemetry, we observe improvements after reinforcement learning, as compared to an agent with identical training but no CalVerT telemetry.

1 Introduction

CalVerT augments QA agents with calibrated confidence and grounding telemetry to address parametric over-trust and over-retrieval. The telemetry improves action selection in training-free settings and raises reinforcement-learning performance, including +5.9% and +3.3% F1 gains on HotpotQA distractor for two Qwen models.

  • Motivation: Knowledge-intensive QA agents face parametric over-trust and over-retrieval, respectively risking correctness and increasing latency and cost.Over-retrieval occurs when agents continue retrieving after sufficient evidence is available.
  • CalVerT: CalVerT supplies two turn-level signals: calibrated confidence in the current answer and whether that answer is entailed by retrieved evidence.Both signals are exposed as telemetry to improve the agent’s view of its operating state.
  • CalVerT: Confidence and grounding are complementary: confidence cannot distinguish confidently wrong from unverified correct answers, while grounding cannot distinguish missing evidence from conflicting evidence.Using both signals addresses limitations that either verifier would have alone.
  • CalVerT: CalVerT adds calibrated verbal self-confidence and per-claim grounding scores to the agent state before each commit, retrieve, refine, or decompose action, without fixed thresholds.The agent incorporates telemetry naturally into decision-making rather than following predefined confidence or grounding cutoffs.
  • Results: +5.9% F1 was observed for Qwen3-8B and +3.3% F1 for Qwen3-30B-A3B on the HotpotQA distractor subset under identical training conditions.The gains compare agents whose states include CALVERT telemetry against otherwise identical agents without it.

2 Related Work

Prior approaches typically expose only one signal at a time for retrieval and action selection, whereas CalVerT conditions decisions on both calibrated confidence and grounding telemetry. This extends multi-hop agent loops with external verifier signals while supporting retrieval gating and commitment decisions.

  • Single-signal action selection: Existing approaches expose either confidence or grounding per turn, limiting the agent’s ability to choose effectively between actions.Training-free methods use fixed schedules, self-reflection, entropy thresholds, complexity routing, iterative decomposition, or single-signal gates.
  • Cost-aware retrieval gating: Prior retrieval-gating methods use a single confidence score, fixed before a trajectory or read per turn, whereas CalVerT uses calibrated verbal confidence for retrieval and commitment.The cited prior methods include Jiang et al. (2023), Jeong et al. (2024), Wang et al. (2026), Zubkova et al. (2025), Yao et al. (2025), and Zhao et al. (2023).
  • Multi-hop agent loops: Multi-hop agent loops interleave retrieval and reasoning with internal stopping criteria but do not incorporate external verifier signals.CalVerT adds calibrated confidence and grounding scores to the agent state before each action, conditioning decisions on two orthogonal uncertainty signals.

3 Calibrated Verifier Telemetry

CalVerT augments a ReAct-style agent with four verifier-telemetry scalars that inform action selection over evolving evidence and subgoals. The loop supports commit, retrieve, refine, and decompose actions, then composes a final answer once subgoals are resolved or the turn budget is exhausted.

  • Agent Loop: The agent decomposes each question into a subgoal DAG and chooses among commit, retrieve, refine, and decompose using the running evidence pool.Commit stores the current answer and supporting facts for later composition.
  • Verifier Telemetry Scores: CalVerT supplies four scalars from two channels: self-confidence (NVC, SC) and grounding (gmean, gmin).NVC is normalized verbal confidence, SC is generator self-consistency, gmean is mean claim entailment, and gmin identifies the worst-grounded claim.
  • Agent Loop: Self-confidence is computed when a subgoal is first visited and cached, whereas grounding is recomputed each turn over the candidate beam and evidence pool.The grounding minimum highlights the weakest link among candidate claims.
  • Agent Loop: The telemetry values are presented in the prompt without hardcoded thresholds, so the agent reasons over calibrated inputs rather than fixed gates.The loop ends when the subgoal DAG is resolved or the turn budget is exhausted, followed by a composer pass producing the final answer.

4 Experiments and Results

Across four open-domain QA benchmarks, CalVerT improves retrieval decisions by raising F1 while reducing redundant actions in many settings. The gains transfer across adaptive-retrieval frameworks and reinforcement-learning training, though prompt-only telemetry depends on model scale.

  • Experimental setup: Evaluation spans four QA benchmarks with 300 sampled development questions per benchmark, using BM25 retrieval and WiTQA cross-encoder reranking.The benchmarks are HotpotQA-distractor, 2WikiMultihopQA, MuSiQue, and WiTQA.
  • Prompt-level results: On three multi-hop benchmarks, telemetry shrinks the action budget while F1 holds or improves, except Mistral-24B on HotpotQA, with ∆F1=−2.7 and ∆turns/ex=−2.61.The prompt, planner, retriever, and composer remain fixed between telemetry and no-telemetry conditions.
  • Prompt-level results: On WiTQA, telemetry increases retrieval and improves F1, including Qwen3-32B ∆F1=+4.7, adapting to over-retrieval or parametric over-trust.WiTQA uses a binary closed-book versus retrieve-then-answer decision.
  • Framework portability: Telemetry boosts F1 on four of five adaptive-retrieval frameworks; SUGAR’s F1 is essentially unchanged while retrieval cost falls by 34 retrieval calls.CALVERT replaces each framework’s native retrieval gate with DINCO confidence scores while preserving the baseline framework otherwise.
  • Reinforcement-learning results: Telemetry-aware GRPO improves untrained-baseline EM by +9.0% on Qwen3-8B and +6.5% on Qwen3-30B-A3B, while exceeding telemetry-free GRPO by +7.0% EM / +5.9% F1 and +4.0% EM / +3.3% F1, respectively.The comparison uses a 1,600-question training pool, a 200-question held-out set, and results at step 100.
  • Analysis and limitations: Telemetry gains grow with difficulty: HotpotQA gains increase monotonically with hop count, while WiTQA gains concentrate on tail-popularity subjects.Confidence and grounding jointly align with actions: commit correlates with high values, whereas retrieve correlates with low or absent grounding; prompt-only telemetry can underperform at Qwen3-8B scale but improves with training.

5 Conclusion

CALVERT exposes turn-level confidence and grounding signals to QA agents, which use them to decide when to retrieve, refine, decompose, or commit. Across prompt-only, framework-substitution, and GRPO training settings, CALVERT improves the accuracy-cost tradeoff by reducing redundant actions and encouraging retrieval when appropriate.

  • CALVERT exposes turn-level confidence and grounding signals to QA agents.These signals provide telemetry for agent control.
  • Agents use CALVERT telemetry to decide when to retrieve, refine, decompose, or commit.
  • Across prompt-only, framework-substitution, and GRPO training settings, CALVERT improves the accuracy-cost tradeoff by reducing redundant actions and encouraging retrieval when appropriate.The gains provide a framework-agnostic mechanism for better agent control.

Limitations · A Calibration scores of the telemetry signals

CalVerT’s commit-centric action vocabulary is suited to single-answer multi-hop QA but can fail on answer-recall tasks requiring every relevant item. Its telemetry signals are calibrated on task-specific benchmark populations, with reliability diagrams indicating strong calibration for DINCO and appropriately confident MiniCheck scores.

  • Limitations: CalVerT commits once it finds a sufficient answer, matching the single-answer multi-hop QA setting used across its four benchmarks.This behavior is intentional and tied to the agent’s action vocabulary.
  • Limitations: The commit-centric vocabulary is unsuitable for answer-recall tasks where the gold answer must contain every relevant item.The passage attributes this failure mode to the action vocabulary rather than the telemetry signal.
  • Limitations: A signal-agnostic alternative is to accumulate a complete answer set before terminating.The supplied passage presents this as a way to avoid the commit-centric limitation.
  • A Calibration scores of the telemetry signals: The framework calibrates confidence and evidence-grounding signals by evaluating their scoring methods on task-specific benchmarks.This calibration is required for the telemetry signals to be usable in the framework.
  • A Calibration scores of the telemetry signals: DINCO confidence is evaluated with Qwen3-32B on 300 randomly sampled closed-book TriviaQA validation questions using seed 42.TriviaQA is selected because its distribution closely matches the closed-book knowledge regime described in the passage.
  • A Calibration scores of the telemetry signals: Reliability diagrams show MiniCheck’s well-populated high-confidence region at or above the diagonal, indicating confidence in correctly grounded answers.The figure reports MiniCheck results on AggreFact-RAG.
  • A Calibration scores of the telemetry signals: DINCO bars closely track the diagonal across the well-populated region, yielding strong calibration on TriviaQA.The figure presents empirical accuracy by confidence bin and per-bin sample counts.
  • A Calibration scores of the telemetry signals: Table 5 reports calibration metrics for both telemetry signals on populations matching CalVerT’s operating regime.The supplied passage identifies the table’s scope but does not provide its numerical values.

B CIs Across Benchmarks · C FLOP analysis of prompt only setups

The prompt-only evaluation tests CalVerT across matched question pairs with bootstrap confidence intervals, while a separate TFLOP analysis measures the computational overhead of adding telemetry across models and benchmarks.

  • B CIs Across Benchmarks: CalVerT’s prompt-only evaluation uses 300 samples with a single seed, comparing +tel and -tel settings on identical question pairs.The 95% confidence intervals are reported in Table 6.
  • B CIs Across Benchmarks: 10,000 paired index resamples from the 300-sample dev split produce confidence intervals for each prompt-only experiment.The resampling procedure uses paired +tel and -tel answers.
  • B CIs Across Benchmarks: The bootstrap experiment reproduces the seed run’s F1 improvements and shifts in agent action turns.This supports consistency between the bootstrap and single-seed evaluations.
  • C FLOP analysis of prompt only setups: The computational analysis measures the TFLOPs overhead introduced by CalVerT’s telemetry addition.The analysis compares telemetry and no-telemetry settings.
  • C FLOP analysis of prompt only setups: Telemetry includes verbalized confidence metrics from DINCO and grounding confidence scores from MiniCheck.These components define the confidence telemetry used in the FLOP comparison.
  • C FLOP analysis of prompt only setups: Qwen3-32B and Mistral-24B are evaluated in tel and notel settings on two benchmarks using a torch flop counter.Results report average TFLOPs over 300 samples in Table 7.

D Additional Analysis

Telemetry acts as an action router that reallocates an agent’s action budget toward the prevailing failure mode rather than uniformly reducing search. On multi-hop benchmarks it shrinks the budget, while on WiTQA it expands retrieval for tail entities.

  • Telemetry as Action Router: Telemetry reallocates the agent’s action budget against the prevailing failure mode instead of uniformly reducing search.This routing behavior is described as telemetry’s role when exposed to harder questions.
  • Telemetry as Action Router: On multi-hop benchmarks, telemetry shrinks the action budget, whereas on WiTQA it expands retrieval for tail entities.The multi-hop result is reported in Table 1, and the WiTQA behavior is shown in Fig. 3.

E Implementation Details · E.1 GRPO Training Details

The experiments use open-weight Qwen and Mistral generators with a fixed Bespoke-MiniCheck-7B grounding verifier, while GRPO trains a telemetry-informed role agent on verified hard HotpotQA questions. The reward combines token F1 and EM with costs for retrieval, refinement, and decomposition actions.

  • E Implementation Details: Prompt-level experiments use Mistral-Small-24B-Instruct and Qwen3-32B, while training-time experiments use Qwen3-8B and Qwen3-30B-A3B.Qwen3-32B is run in non-thinking mode, and all models use the same generator family for training-time experiments.
  • E Implementation Details: The grounding verifier is fixed to Bespoke-MiniCheck-7B across all experimental cells.Generators are served with vLLM on NVIDIA GH200 nodes.
  • E.1 GRPO Training Details: The role-beam runner combines planner, per-turn role agent, DINCO sampler, and composer roles with a hybrid BM25 + bge-base retriever.Retrieval uses RRF k=60 and top-3 results; the planner, DINCO, MiniCheck, and composer are frozen, while only the per-turn role agent is trained as a LoRA adapter.
  • E.1 GRPO Training Details: Only the per-turn role agent is trained with a LoRA adapter using rank 16, lr 1×10−4, and AdamW.The adapter is hosted on the Tinker training service, while the frozen components run on a local vLLM backend.
  • E.1 GRPO Training Details: Each rollout receives token-based F1 and EM rewards for the composer’s predicted answer against the gold answer.The reward definitions are F1(τ) = F1(apred, agold) and EM(τ) = EM(apred, agold).
  • E.1 GRPO Training Details: Action costs are 0.01 for turns, 0.02 for retrieval, 0.02 for refinement, and 0.04 for decomposition, while commits incur only the base step cost.The available actions are commit, retrieve, refine, and decompose.
  • E.1 GRPO Training Details: The rollout reward is Rout(τ) = F1(τ) + 0.1 EM(τ) −c⊤n(τ), and GRPO samples G=4 rollouts per prompt to compute group-relative advantages.Advantages compare each rollout reward with the mean reward within its sampled group.
  • E.1 GRPO Training Details: GRPO training uses 1,600 questions from HotpotQA distractor dev[0:1600], selected because the released dev split contains verified hard questions.The policy learns when to retrieve, refine, decompose, or commit from verifier telemetry.

E.2 Evaluation on Closed Source Models … F.3 No-telemetry ablation system prompt

CalVerT improves closed-source-model performance and is most effective when multi-hop questions require complex retrieval, while its prompts expose telemetry-guided action choices absent from the no-telemetry ablation. The paper also tests portability across adaptive retrieval frameworks and isolates the roles of confidence and grounding signals.

  • E.2 Evaluation on Closed Source Models: CalVerT raises F1 for gpt-4o and gpt-4.1 on both 2Wiki and HotpotQA in 100 sampled test examples.The largest gains occur on 2Wiki: +13.2 F1 for gpt-4o and +3.1 for gpt-4.1; HotpotQA gains are +2.0 F1 for both.
  • E.2 Evaluation on Closed Source Models: Action turns barely change across closed-source-model settings, with the only reported regression being a −1.0 EM drop for gpt-4.1 on HotpotQA.Overall F1 still improves for that case.
  • E.3 Performance with Individual Telemetry: Neither DINCO nor MiniCheck alone matches their combination on 2Wiki; DINCO alone increases turns and can trigger repeated decomposition and refinement until the action budget is exhausted.This indicates that grounding telemetry is important for convergence in the multi-hop setting.
  • E.3 Performance with Individual Telemetry: On single-hop WiTQA, all telemetry configurations achieve nearly the same accuracy and only marginally improve over the no-telemetry baseline.The paper attributes this to WiTQA’s single gold evidence pool, unlike 2Wiki’s multi-hop, multi-document setting.
  • E.3 Performance with Individual Telemetry: CalVerT’s action routing is primarily effective for complex multi-hop reasoning and retrieval from multiple sources.The individual-telemetry results contrast this setting with the easier single-hop WiTQA benchmark.
  • E.4 Adaptive Retrieval Frameworks: CalVerT is tested for portability on five adaptive retrieval frameworks, including threshold-based, internal-state, and semantic-entropy retrieval policies.The reported calibration settings include TARG’s τ = 0.045, SeaKR’s δ = −6, and SUGAR’s τlo = .81 and τhi = 1.0.
  • F Prompts: The telemetry prompt gives the agent calibrated DINCO confidence and MiniCheck grounding signals to guide commit, retrieve, refine, and decompose actions.It exposes NVC, self-consistency, aggregate grounding, minimum grounding, and per-claim support scores.
  • F.3 No-telemetry ablation system prompt: The no-telemetry ablation retains the same four actions and budget-aware guidance but removes confidence and grounding readings from the system prompt.Commit is based on believing the answer is correct, while retrieval, refinement, and decomposition remain available as described actions.

G Full Example: agent actions flip upon exposed telemetry

In a Qwen3-32B MuSiQue example, exposing calibrated telemetry flips the agent from committing to a plausible but wrong answer to retrieving evidence and producing the correct answer.

  • No telemetry: Without telemetry, the agent commits to “Elisabeth Shue,” causing the composer to emit the wrong final answer.It commits to the first plausible candidate after observing that answer.
  • Telemetry exposed: With telemetry, low grounding leads the agent to retrieve and refine rather than commit prematurely.The agent observes Molly Ringwald with dinco:0.22, then Lea Thompson with g_min:0.35, and continues searching.
  • Telemetry exposed: At t4, g_min:0.93 accompanies “Claudia Wells,” prompting commitment and the correct composed answer.The gold answer is Claudia Wells for the MuSiQue question.
Loading 2606.21777v1…