Source-linked AI summary
Self-Reports Are Not Verification: Environment-Grounded Auditing of LLM Operators in Evolutionary Search
Enrong Pan, Ryan Zhou, Ting Hu
TL;DR
LLM self-reports are convenient but need external verification because their accuracy, causal influence, and optimization properties are uncertain. The paper audits them in an evolutionary Contexto search where every valid intermediate proposal receives an exact rank, finding inflated confidence and no detectable rationale or selection effects on report accuracy. The results support treating self-reports as claims grounded against environmental outcomes, with validation still needed beyond this exact-feedback setting.
Problem
Intermediate agent proposals often remain ungraded, limiting evidence about whether confidence and rationales are reliable self-reports.
Method
The paper audits LLM-guided evolutionary search in Contexto, where every valid proposal receives an exact rank without human annotation.
Results
Confidence overstates top-100 success by factors of 4.8 to 9.3, while inherited rationales and selection produce no detectable improvement or transmission in report accuracy.
Takeaways & Limitations
Self-reports should be treated as claims requiring external verification rather than as evidence of their own reliability.
Takeaways & Limitations
The findings are most directly supported in one exact-rank semantic search environment with three similarly sized model families; larger models, other architectures, and partial or noisy feedback require separate evaluation.
Abstract
from arXiv · showhide
Language model agents increasingly propose actions, observe external feedback, and explain their own behavior. Their confidence and rationales are convenient monitoring signals, but convenience is not verification. We introduce an environment-grounded audit in which every intermediate proposal receives an exact outcome. A language model operates an evolutionary Contexto search whose feedback function assigns every valid guess an exact rank without human annotation. Across 200 runs spanning five configurations and three model families, four reporting configurations produce 12,249 self-reports. We test three assumptions: stated confidence is calibrated, inherited rationales affect later proposals, and fitness-based selection improves report quality. All three fail. Operators overstate top-100 success by factors of 4.8 to 9.3, while calibration and discrimination dissociate across model families. Controlled interventions on 754 inherited rationales bound any measured benefit of the genuine rationale to roughly 250 ranks. Neither fitness-based nor random selection produces a detectable selection differential or parent-to-offspring transmission in report accuracy, despite sharply different search behavior. Agent self-reports should therefore be treated as claims to verify against the environment, not as evidence of their own reliability.
1 Introduction
The paper tests whether LLM self-reports can verify intermediate proposals when an environment grades every valid action exactly. Across an evolutionary Contexto audit, confidence, inherited rationales, and selection all fail to provide reliable report verification.
- Motivation: Every valid proposal receives an exact environment outcome, enabling confidence and rationale reports to be evaluated without human annotation.The audit uses Contexto ranks as external evidence for intermediate proposals.
- Research questions: The audit asks whether confidence is accurate, inherited rationales affect later behavior, and selection improves report quality.These questions are tested within an LLM-guided evolutionary search loop.
- Study design: 12,249 self-reports were emitted across 200 runs spanning five configurations and three model families.Separate controls replayed 754 variation events under four inherited-rationale conditions and included random selection.
- Findings: Confidence overstates top-100 success by factors of 4.8 to 9.3, while the strongest ranking ability coincides with the largest calibration error.The results distinguish discrimination among proposals from numerical calibration of stated probabilities.
- Findings: Inherited-rationale replacements do not detectably change proposal identity or returned rank, and selection does not improve report accuracy despite changing search behavior.The findings separate report accuracy, causal influence, and optimization rather than treating narration as verification.
2 Related Work
The paper connects external-feedback agents, LLM-guided evolutionary search, confidence calibration, and rationale faithfulness. It brings together questions that prior work commonly studies separately: numerical correctness, causal influence, and selection for accurate self-reports.
- Agents and external verification: Agent research studies environment interaction, verbal feedback, process supervision, and output judging, but these approaches differ in how they externally verify intermediate behavior.The paper positions dense environment feedback as a way to grade every intermediate proposal.
- LLM-guided evolutionary search: LLMs serve as semantic variation operators in evolutionary search, while prior work mainly evaluates whether they improve the objective rather than report accuracy.This paper asks whether survivor selection also favors accurate self-reports.
- Confidence calibration: Calibration measures agreement between stated probabilities and observed frequencies, whereas discrimination measures whether higher confidence accompanies better outcomes.The paper applies both concepts to intermediate search proposals instead of final answers.
- Rationale faithfulness: Rationale faithfulness asks whether a free-text explanation reflects the process producing a prediction, not merely whether the explanation sounds plausible.The paper extends perturbation work by intervening on rationales inherited by descendants.
3 A Setting Where Every Self-Report Is Gradeable
The testbed combines Contexto’s exact rank feedback with an LLM-guided evolutionary algorithm, making every valid intermediate proposal gradeable. Offspring are generated by mutation or crossover, then archive-based selection retains the best-ranked candidates as parents.
- A Setting Where Every Self-Report Is Gradeable: Contexto returns an exact similarity rank for every valid guess, with rank 1 denoting the target and lower ranks being better.The largest rank observed in the runs is 79,552.
- Evolutionary search and selection: The LLM creates offspring through mutation or crossover, proposing a category and three candidate words for evaluation.Mutation operators request progressively larger semantic departures, while crossover combines the two best current parents.
- Evolutionary search and selection: The best rank among an offspring’s newly graded words serves as its fitness, and the search optimizes toward rank 1.Report accuracy is evaluated separately on the first proposed word that receives a grade.
- Evolutionary search and selection: Archive-based truncation selection retains the best five individuals as the next parent population.All individuals in the cumulative archive are ranked by their best returned rank before retention.
- Experimental configuration: The study uses Qwen-3 14B, Gemma-4 12B, and Ministral-3 14B across 50 runs per evolutionary configuration and ten targets.The design also includes Direct-Qwen runs and EA-Qwen runs with reporting disabled.
4 Is the Operator’s Stated Confidence Right?
The audit finds that stated confidence is systematically overconfident, while calibration and discrimination vary independently across model families. Evolutionary search also changes calibration, but the paper does not claim a mechanism for that difference.
- Operators overstate top-100 success by factors of 4.8 to 9.3 across model families.EA-Qwen reports 47.7% versus 10.0% observed success; EA-Gemma reports 75.3% versus 12.3%; EA-Ministral reports 67.9% versus 7.3%.
- Positive calibration error appears in every occupied confidence bin across all configurations.Figure 3 compares stated top-100 probabilities with observed top-100 frequencies; hollow markers indicate bins with fewer than 30 reports.
- EA-Gemma has the strongest ranking but worst calibration, whereas EA-Qwen has the best calibration but weaker ranking.EA-Gemma has median ρ = −0.450 and median ECE 0.652; EA-Qwen has ECE 0.363 and ρ = −0.230.
- Across target games, EA-Gemma ranks better in nine of ten games, while EA-Qwen has lower ECE in all ten.This cross-game contrast reinforces that ranking ability and probability accuracy are distinct properties.
- Median AUROC is 0.655 for EA-Qwen, 0.717 for EA-Gemma, 0.625 for EA-Ministral, and 0.510 for Direct-Qwen.Values above 0.5 indicate useful separation, while Direct-Qwen is near chance.
- EA-Qwen’s pooled ECE is 0.377 versus 0.421 for Direct-Qwen, with 59.6% of the difference persisting after state reweighting.The paper reports the difference without claiming a mechanism; access to more graded history does not explain the remainder.
5 Do Inherited Rationales Affect Later Proposals?
Inherited rationales do not measurably affect descendant proposals or returned ranks in the controlled search. Across rationale replacements, the estimated effects do not favor the genuine rationale and bound any benefit to roughly 250 ranks.
- 754 inherited-rationale variation events were replayed under controlled replacement conditions.The rationale can affect descendants during mutation, but not the proposal it accompanies because the proposal is emitted first; crossover does not use the rationale.
- The genuine rationale does not improve returned rank against unrelated, filler, or no-rationale replacements.Hodges–Lehmann estimates are +73, 0, and +50 respectively, with 95% intervals [−5, +477], [−242, +138], and [0, +279].
- The intervention intervals rule out a genuine-rationale benefit larger than about 250 ranks.The unrelated-rationale comparison limits the benefit to five ranks.
- Rationale replacements leave proposal identity detectably unchanged.First-word agreement is 15.5% for unrelated rationales, 16.3% for filler, and 15.1% for no rationale; Jaccard overlap ranges from 0.138 to 0.142.
- The conclusion applies only to rationales inherited by descendants in this search.It does not test whether rationales help human readers or whether a rationale generated before an action can guide that same action.
6 Does Selection Respond to Self-Report Quality?
The study finds that neither fitness-based nor random selection favors accurate self-reports or transmits report accuracy, despite producing different search behavior. Fitness selection changes task search rather than report quality.
- Selection requirements: Selection requires both a differential favoring accurate reporters and parent-to-offspring transmission of report accuracy.The analysis treats these as the two conditions required for a response to selection.
- Selection differential: +0.039 probability error and −0.146 bucket distance show no detectable selected-minus-discarded accuracy preference across 1,214 events.Permutation tests gave p = 0.713 and p = 0.253, respectively.
- Heritability: −0.023 probability-error and +0.057 bucket-distance correlations show no detectable parent-to-offspring transmission across 1,018 pairs.The reported p-values were 0.524 and 0.911, respectively.
- Selection comparison: Neither fitness nor random selection shows a detectable report-accuracy differential or parent-to-offspring association.The figure compares probability error and bucket distance, alongside solved runs and archive re-selections.
- Search behavior: Fitness and random selection produce sharply different search outcomes: one target has five versus one solved run, with median guesses of 446 versus 1,828.Random selection also reselects 542 previously unselected archive members, whereas fitness selection produces none in the corresponding control runs.
7 Discussion and Limitations
The discussion separates self-report accuracy, causal influence, and optimization, arguing that each requires distinct tests and that dense environmental feedback makes reports checkable. It also limits the claims to validated settings and warns against overgeneralization.
- Discussion: Self-reports can be inaccurate, causally inert, or ignored by optimization, so these failures require different remedies.The paper distinguishes calibration, information flow, and external verification rather than treating narration as a single property.
- Environment grounding: Exact feedback on every proposal creates a verification surface where confidence can be evaluated as a prediction and rationales as causal inputs.Suggested dense feedback sources include partial test suites, simulator state, constraint checks, and process-level rewards.
- Broader impacts: Applications should validate the audit in their own environments and retain independent checks on consequential actions.The paper warns that overgeneralization could either discard useful self-reports or overtrust dense synthetic feedback.
- Scope and limitations: The results are most directly supported in one semantic search environment with exact rank feedback and three similarly sized model families.Larger models, other architectures, and partial or noisy feedback require separate evaluation.
8 Conclusion
The paper concludes that fluent self-reports cannot substitute for verification: confidence, rationale influence, and selection for report accuracy each fail distinct tests. Reliable evaluation should ground reports in external outcomes and separate task optimization from report-quality optimization.
- Conclusion: Confidence is systematically inflated, inherited rationales do not measurably change later proposals or ranks, and selection does not select or transmit report accuracy.These are presented as distinct failures across accuracy, causal influence, and optimization.
- Conclusion: Self-reports should be treated as inexpensive claims whose reliability is established by environmental outcomes.The conclusion frames external grading as the basis for evaluating predictions and causal inputs.
- Additional discussion: The source of EA-Qwen’s calibration advantage remains unresolved.The reported effect persists after matching search difficulty, but the paper does not identify its cause.
- Additional discussion: Selecting on self-report quality would make the report an optimization target and forfeit the causal inertness that makes the audit trustworthy.The paper argues that grading self-reports costs the search nothing and gains the operator nothing only while reports remain outside fitness.
A.1 Future work
The paper identifies boundaries for interpreting its audit and proposes extensions involving adaptive search, denser grading environments, and selection signals protected from contamination.
- Adaptive search: The fixed-parameter design does not test whether self-report quality co-varies with an adapting search state.Releasing the constraint would enable that question.
- Domains with dense grading: Program synthesis with partial-credit tests and interactive theorem proving are proposed as denser-grading settings for extending the audit.Both environments grade intermediate progress rather than only final answers.
- Selection that does not corrupt the signal: Selection on an unobserved self-report grade or on held-out proposals could avoid corrupting the signal, but neither scheme is tested here.
- Scope caveats: The Direct-Qwen comparison is exploratory because it covers five target words.
- Scope caveats: AUROC is undefined for runs containing only one outcome class, so AUROC comparisons use an outcome-dependent subset.
B Estimation details
The estimation procedure standardizes Direct-Qwen’s calibration against EA-Qwen’s search-state distribution and defines how correlation summaries and uncertainty intervals are computed.
- Calibration standardization: Direct-Qwen’s within-stratum ECE is weighted by EA-Qwen’s search-state distribution for difficulty standardization.
- Correlation estimation: 47 of 50 EA-Qwen runs, 49 of 50 EA-Gemma runs, and 47 of 50 EA-Ministral runs contribute to Figure 4’s correlation analysis.Runs lacking enough distinct outcomes are excluded.
- Uncertainty estimation: Figure 4 centroids are per-run medians with 95% percentile bootstrap intervals over 10,000 resamples of runs, using both coordinates from each resample.
C Reproducibility details
The reproducibility materials document reporting conventions, run accounting, hardware, licenses, filtering rules, and the mutation prompts used to generate search hypotheses.
- Reporting conventions: Figure 7 reports individual runs as light marks and per-run medians with 95% resampling intervals; AUROC counts show defined runs over launched runs.AUROC is undefined when a run lacks either positive or negative top-100 outcomes.
- Proposal handling: Previously submitted and known invalid words are excluded run-wide, while the environment returns ⊥ for words outside its accepted vocabulary.If all proposals are invalid or excluded, sentinel fitness 109 is assigned.
- Compute: Experiments used NVIDIA A100 and A30 GPUs, parallelized across up to ten GPUs, with an approximate upper estimate of 1,680 aggregate GPU-hours.Exact per-run times, occupancy, and A100 memory variant were not retained.
- Compute: GPU choice reflected speed and availability rather than a method requirement, so compatible hardware can reproduce the runs more slowly.
- Existing assets and terms: The work uses Qwen-3 14B, Gemma-4 12B, and Ministral-3 14B under Apache License 2.0.3, plus MIT-licensed Ollama and Phosphor Icons.Contexto is accessed as a hosted service, and its implementation, vocabulary, and game content are not redistributed.
- Prompt specifications: The implementation records four mutation prompts, crossover, Direct-Qwen, and self-report instructions verbatim; prompt wording is part of the reproducible method.Mutation operators include small refinement, medium reinterpretation, medium-large adjacency, and large directional changes.