Source-linked AI summary
Beyond Accuracy: How Procedural Traces Shift the Decision Criterion of LLM Overseers
Zihan Chen, Di Zhu, Lei Zheng, Weiling Li
TL;DR
As LLMs audit other LLMs, it remains unclear whether elaborated procedural traces change overseers’ detection or rejection decisions, even though those traces feed later control decisions. A controlled audit finds detection stays near ceiling while detailed traces shift susceptible overseers toward rejecting correct work.
Problem
As LLMs audit other LLMs, it remains unclear whether elaborated procedural traces change detection or rejection decisions, even though traces feed later control decisions.
Source: §1 · pp. 1–2Method
Five LLM overseers judged 19 compliance tasks in a controlled audit varying trace detail and evidence-option labels while holding recommendations and evidence fixed.
Source: §1 · p. 2Results
Detection stayed near ceiling, while one susceptible overseer’s false rejection of correct answers rose from 58% to 96% as traces grew more detailed.
Source: §1 · p. 2, §7 · p. 10Takeaways & Limitations
AI auditors should be evaluated by their decision criterion and false-alarm behavior alongside accuracy, because procedural traces can shape downstream oversight decisions.
Source: §1 · p. 2Evidence is limited to 19 compliance items in one domain with single-criterion violations and visible disconfirming reviews, leaving concealed errors untested.
Source: §6.3 · p. 10
Abstract
from arXiv · showhide
Organizations increasingly use oversight loops where one large language model (LLM) audits another's outputs alongside procedural traces of claimed steps. A common concern about such LLM-as-a-judge pipelines is that detailed traces make overseers gullible. Using signal detection theory, we audit five LLM overseers on 19 compliance tasks (4,551 analyzed judgments), varying only trace detail and evidence labeling. With disconfirming evidence always visible, error detection remains near ceiling. Instead, elaborate traces shift the decision criterion toward rejection, increasing false alarms in susceptible overseers. Without option labels, human-validated reason coding shows about 60% of false alarms cite an inability to tie evidence to its option. Labels eliminate this stated reason, yet residual rejection of correct work persists in those overseers and rises with trace detail. Procedural traces thus act as governance artifacts that shape oversight decisions. AI auditors should be evaluated by their decision criterion and false-alarm behavior, alongside accuracy.
1 Introduction
The paper asks whether procedural traces make LLM overseers gullible or instead more suspicious, and tests this while holding audited answers and evidence fixed. It finds that detection remains near ceiling while elaborate traces increase false rejection among susceptible overseers.
1 Introduction
- A procedural trace is a natural-language account of claimed task steps, not necessarily a faithful execution log or hidden chain of thought.It can narrate due diligence whether or not the underlying work was solid.Source: §1 · pp. 1–2
- Longer traces could either induce overtrust and missed errors or raise suspicion and increase rejection of correct work, making the direction an empirical question.Source: §1 · p. 2
- Accuracy-only evaluations conflate error-detection sensitivity with rejection bias, leaving prior LLM-judge work unable to isolate trace effects when answer quality and evidence are fixed.The study asks whether elaboration shifts rejection, whether evidence-to-option attribution explains the shift, and how effects vary across models.Source: §1 · p. 2
- The audit varies trace detail and evidence labeling across five overseers and 19 compliance tasks while keeping recommendations and underlying evidence fixed.Of 4,560 judgments, 4,551 parsed cleanly and were analyzed.Source: §1 · p. 2
- Hit rates range from 99.3% with no trace to 100% with the most detailed trace, while susceptible overseers increasingly reject correct work.In the most susceptible overseer, false rejection rises from 58% to 96%; pooled odds of false rejection rise about 44% at each trace step.Source: §1 · p. 2
- The paper frames traces as governance artifacts that can recalibrate a downstream control, and recommends evaluating decision criteria and false alarms alongside accuracy.Evidence labeling is described as a cheap partial remedy, not a complete one.Source: §1 · p. 2
2 Related work
Prior work studies judge biases, explanation effects, and trace faithfulness, but does not isolate how procedural traces shift an AI overseer's decision threshold when answer quality is held fixed. This paper distinguishes binary oversight and criterion shifts from preference or quality judgments summarized by accuracy alone.
2 Related work
- LLM-judge research catalogs position, verbosity, self-preference, and sycophancy biases, while often using preference or quality judgments and a single accuracy metric.This paper instead studies binary oversight and models sensitivity and decision criterion separately.Source: §2 · p. 3
- Human-AI research finds that explanations can improve appropriate reliance or inflate confidence without improving decisions, and that decision aids can shift response bias without changing sensitivity.Whether AI overseers show the same susceptibility, and in which direction, remains open.Source: §2 · p. 3
- Trace-faithfulness research asks whether a model's trace reports its own reasoning truthfully; this study asks how a trace alters another model's decision threshold.Source: §2 · p. 3
- The governance-artifact framing treats traces as audit trails for later control decisions that can redistribute perceived verification responsibility between worker and overseer.The cited control literature distinguishes configured controls from how they are enacted.Source: §2 · p. 3
3 Theory and hypotheses
The theory separates an overseer's ability to distinguish flawed from sound work from its threshold for rejecting work. It predicts that performed rigor may shift that threshold through attribution gaps and residual over-skepticism, with effects varying by overseer.
3 Theory and hypotheses
- Sensitivity (d′) measures discrimination between flawed and sound recommendations, whereas criterion (c) measures rejection bias, with more negative c indicating stronger predisposition to reject.Accuracy can change because discrimination changes or because the overseer becomes more suspicious.Source: §3 · p. 3
- The gullibility hypothesis predicts that more elaborate traces lower sensitivity and hit rates, while the criterion-shift hypothesis predicts more rejection and false alarms without necessarily reducing sensitivity.The hypotheses are not mutually exclusive.Source: §3 · p. 3
- The attribution hypothesis predicts that labeling each evidence excerpt with its option reduces false rejection by closing a gap in evidence-to-option attribution.Source: §3 · p. 3
- Residual-rigor and heterogeneity hypotheses predict that rejection shifts persist with attribution supplied but vary across overseers, with some robust in both attribution conditions.The proposed residual mechanism treats a criterion not affirmatively confirmed as potentially wrong.Source: §3 · pp. 3–4
- The proposed model links added trace detail to a higher verification burden and false alarms through attribution gaps and an absence-of-evidence rule, moderated by attribution, calibration, and evidence visibility.Source: §3 · p. 4
4 Method
The method uses a controlled audit in which correctness is fixed by construction and only the displayed trace and evidence labels vary. Five LLM overseers judge compliance recommendations, and the analysis separates detection from false rejection while coding stated reasons.
4 Method
- A controlled audit tests competing trace effects by varying displayed trace and evidence labels while holding recommendation correctness fixed by construction.Source: §4 · p. 4
4.1 Task and stimuli
- The 19 hand-validated compliance tasks use Yelp reviews and known best and error options, with each error violating one structured criterion and its disconfirming evidence visible.This setup makes every planted error detectable from the displayed evidence.Source: §4.1 · p. 4
4.2 Manipulations
- Four trace levels progress from none to a false claim of verifying the violated criterion, while attribution changes only whether excerpts identify the option they describe.The recommendation, evidence content, and ordering remain fixed across conditions.Source: §4.2 · p. 4, Figure 1 · p. 5
4.3 Overseers, design, and procedure
- The design crosses five overseers, 19 items, two correctness levels, four traces, two attribution conditions, and three runs for 4,560 judgments.Of these, 4,551 parsed cleanly; the nine unparsed returns were dropped, and inference clusters on item.Source: §4.3 · pp. 4–5
4.4 Measures and identification
- The analysis computes hit and false-alarm rates plus d′ and c, fits item-clustered regressions, and codes stated rejection reasons with LLM and human checks.A right-reason hit is a rejected error whose reason names the criterion actually violated; two human annotators assess a stratified subsample of 103 reasons.Source: §4.4 · p. 5
5 Results
Across five overseers, trace detail leaves error detection near ceiling but shifts susceptible models toward rejecting correct work; attribution reduces, but does not remove, that shift. Stated-reason coding and robustness checks characterize the remaining rejection and its variation across models.
5.1 No gullibility: detection is intact (RQ1, H1a)
- Pooled hit rates ranged from 99.3% to 100% across trace levels, and 98.1% of hits named the violated criterion.Errors were single-criterion violations with disconfirming reviews visible, so the result does not establish detection under concealed errors.Source: §5.1 · p. 6
5.2 The criterion slides toward rejection (RQ2, H1b)
- Each one-level increase in trace dose multiplied the pooled odds of falsely rejecting correct work by 1.44 (95% CI [1.28, 1.63]).Pooled accuracy fell from 0.80 to 0.70 while d′ stayed near 2.4 to 2.7.Source: §5.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
5.3 Attribution removes a large component (RQ3, H2)
- Attribution reduced false alarms at the flawed-verification trace by 32 points for qwen3-next-80b, 28 for gemma-3-27b, and 21 for minimax-m2.5.Across models, attribution multiplied the odds of false rejection by 0.34 (95% CI [0.17, 0.67]).Source: §5.3 · p. 7
5.4 A residual rigor effect survives attribution (RQ3, H3)
- Even with attribution, false alarms rose with trace detail for three overseers, while the two opus overseers remained flat.For gemma-3-27b, minimax-m2.5, and qwen3-next-80b, attributed-arm false alarms rose from 5% to 30%, 26% to 54%, and 25% to 65%, respectively.Source: §5.4 · p. 7, Table 2 · p. 7
5.5 What the residual is, and heterogeneity (RQ4, H4)
- The attribution-gap complaint accounted for about 60% of false alarms without labels and fell to zero under attribution for both judges.Two LLM judges classified 934 reasons with Cohen’s κ = 0.89; human annotations validated the attribution-gap labels.Source: §5.5 · p. 7, Table 3 · p. 8
- Remaining rejections included absence-of-evidence over-skepticism, which increased with trace detail for minimax-m2.5 and qwen3-next-80b but was flat for the opus pair.The five systems differed in scale, vendor, weight availability, and post-training, so the study leaves the source of their robustness differences open.Source: §5.5 · p. 8
5.6 Confidently wrong (RQ4)
- For qwen3-next-80b, mean confidence was 94.4 on false rejections and 94.9 on correct accepts, a 0.5-point gap on a 0-to-100 scale.By contrast, gemma-3-27b’s confidence dropped 15.7 points when it over-rejected.Source: §5.6 · p. 8
5.7 Robustness
- The trace-dose and attribution effects remained stable across item-exclusion, fixed-effects, random-intercept, and item-cluster-bootstrap checks.The trace effect was not merely verbosity: attribution shifted false alarms at fixed trace length, and the longest trace was not always most punitive.Source: §5.7 · p. 8
6 Discussion
The discussion finds that visible evidence lets overseers detect errors while elaborate traces shift susceptible overseers toward rejecting correct work. It distinguishes a removable evidence-attribution gap from residual over-skepticism and argues that oversight should be designed and evaluated for calibration.
6 Discussion
- Detection holds while the decision criterion slides; attribution removes one component, but residual over-skepticism remains even when confidence does not fall.The discussion distinguishes these criterion shifts from a loss of sensitivity.Source: §6 · p. 8, §6.1 · p. 9
6.1 Theoretical implications
- Under full attribution, detailed traces still lead susceptible overseers to demand affirmative proof of every criterion, making residual over-skepticism the more general finding.The authors connect this result to faithfulness research and describe audit trails as decision-shaping governance artifacts.Source: §6.1 · p. 9
6.2 Managerial and design implications
- False alarms can return valid outputs for rework, consuming scarce review capacity and recreating the bottleneck oversight automation was intended to relieve.The authors recommend labeling evidence by option, treating trace detail as a design variable, separating trace verification from output judgment, and auditing the criterion.Source: §6.2 · p. 9
- At equal costs, detailed traces raise expected cost from 202 to 300 units per 1,000 evenly split audits, despite reducing misses from 1.4% to 0%.Break-even requires a missed error to cost about fifteen times a false alarm; the authors recommend tracking false alarms on known-good work and criterion c, not accuracy alone.Source: §6.2 · p. 9
6.3 Limitations and future work
- Because disconfirming evidence was always visible, the no-gullibility result leaves open whether traces can mask genuinely concealed errors.The study covers 19 compliance items in one domain; the authors identify buried-error settings as a natural test.Source: §6.3 · p. 10
7 Conclusion
The paper tests whether procedural traces make LLM overseers gullible and finds that, with disconfirming evidence visible, they instead increase rejection of correct work among susceptible overseers. It attributes over-rejection partly to evidence attribution and partly to residual over-skepticism, which accuracy-only evaluation can miss.
7 Conclusion
- Across 4,551 compliance-task judgments with disconfirming evidence visible, elaborate traces make susceptible overseers reject correct work rather than miss errors.The study reports that detection remains near ceiling.Source: §7 · p. 10
- One evidence-attribution label removes the attribution-gap component of over-rejection, but rigor-driven over-skepticism persists.The residual rejection of correct work rises with trace detail.Source: §7 · p. 10
- Accuracy-only evaluation cannot reveal over-rejection that reflects a shifted decision criterion rather than reduced sensitivity.The conclusion calls for evaluating decision criteria and false-alarm behavior alongside accuracy.Source: §7 · p. 10