PaperlayerView exact revision on arXiv

Source-linked AI summary

Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines

Yirui Zhang, Yash Raj Shrestha

arXiv:2610.15897v2cs.CLcs.AI

TL;DR

Review is commonly evaluated by its gain over the generator, a measure that mixes verification with capability transfer. Across 96 review directions, review rarely beats the stronger model alone; an equal-call selector is more accurate and cheaper.

  • Problem

    Gain over the generator does not distinguish accuracy added beyond either model from answers supplied by a stronger reviewer, complicating assessment of whether a second call is worthwhile.

    Source: §1 · p. 1

  • Method

    The paper separates review’s gain over the generator from its gain over the stronger model and evaluates review, answer selection, and solo models across 48 pairs in four domains.

    Source: §1 · p. 1

  • Results

    Only 9 of 96 review directions beat the stronger model alone, while equal-call selection is 4.2 and 2.7 points more accurate than the two review directions and cheaper in every pair.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Studies should report the stronger model alone alongside gain over the generator, and treat choosing between independent answers as the equal-call baseline to beat.

    Source: §6 · pp. 8–9

    The paper checks only one alternative prompt on four directions and leaves other prompts, multiple rounds, training, and the reviewer’s internal causes for future work.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result. Reviewing another model's answer is usually judged by its gain over the generator. We show that this confuses verification, which adds accuracy beyond either model alone, with capability transfer, where a stronger reviewer supplies answers it could have given on its own. Measured against the stronger model alone, across 48 pairs of 12 open-weight models (3B to 72B) in math, knowledge, code and reasoning, review beats it in only 9 of 96 review directions, and does worse the larger the capability gap. The reason is anchoring: the reviewer mostly follows the generator, discarding its own correct answers. Review costs 2.6 to 2.8 times as much as the stronger model alone, while a selector using the same two calls is 2.7 to 4.2 points more accurate than review and cheaper. Studies of verification should report the stronger model alone, not only the gain over the generator.

1 Introduction

Review is often judged by its gain over the generator, but that measure mixes verification with capability transfer from a stronger reviewer. Across four domains, review rarely beats the stronger model alone, while equal-call selection is more accurate and cheaper.

1 Introduction

  • A gain over the generator can reflect verification beyond either model alone or capability transfer from a stronger reviewer, so the paper also measures gain over the stronger model.The stronger-model comparison is the final answer’s accuracy minus the more accurate model’s solo accuracy.Source: §1 · p. 1
  • A selector can add accuracy only on questions where the second model is right and the stronger model is wrong; this share is the pair’s headroom.The authors expect review to collect this headroom because the reviewer sees both the question and another model’s attempt.Source: §1 · p. 1
  • Only 9 of 96 review directions beat the stronger model alone; review does worse as the capability gap grows.Seven wins came from a weaker model reviewing a close stronger one.Source: Figure 1 · p. 2, §1 · p. 2
  • The evaluation covers 48 pairs of 12 open-weight models, 3B–72B parameters, across math, knowledge, code, and reasoning, with review run in both directions.The four task sets contain 2,500 math, 2,500 knowledge, 832 code, and 901 reasoning questions.Source: §1 · pp. 1–2
  • With the same two calls, selection is more accurate than review and costs less in every pair; it chooses between the models’ independent answers.Source: §1 · p. 2, Figure 1 · p. 2

2 Related Work

Prior work often measures verification by gain over the generator or studies peer influence without testing review against both the stronger model and an equal-call choice. This paper connects review to anchoring and compares it with independent-answer selection.

2 Related Work

  • Most related verifier studies choose among the generator’s sampled answers and measure gain over the generator; when the generator is stronger, that gain is also a gain over the stronger model.The closest cited coding comparison raises Codex from 71.6% to 89.7%, still below Claude’s 91.4% alone (Xiang et al., 2026).Source: §2 · p. 3, Table 1 · p. 3
  • Prior studies had not tested review against both the stronger model alone and an equal-call choice, or traced the source of review’s gains; this work evaluates 96 directions across four domains.The authors characterize review’s gains as mostly capability transfer.Source: §2 · p. 3
  • Research on model conformity covers candidate order, self-generation, peer majorities, and peer deference; this study tests full solutions across real capability gaps and finds anchoring even without a user or majority.Self-review preserves over 90% of a model’s correct answers, pointing to the other model’s answer rather than a second pass as the relevant influence.Source: §2 · pp. 3–4
  • Unlike routers and cascades, review and selection act after answers exist; selection chooses between two independent models’ answers, and under the two-call budget it is more accurate on average and cheaper in every pair.The authors present this equal-call choice as a yardstick not used by prior work comparing review.Source: §2 · p. 4

3 Framework and Experimental Setup

The framework distinguishes review’s gain over the generator from its gain over the stronger model, then evaluates review across paired model directions and four task domains.

3.1 Review Protocol and Comparative Gains

  • The framework tracks gain over the generator (∆G) and stronger model (∆S), since a stronger reviewer’s ∆G can come entirely from the capability gap.The gap is the accuracy difference between the stronger and weaker models answering alone.Source: §3.1 · p. 4
  • Review gain is decomposed into reviewer-only correct answers adopted, generator-only correct answers lost, and changes where both models agree; only positive ∆S shows verification.The protocol has the reviewer keep the generator’s answer when judged correct, or correct it otherwise, and return a complete solution.Source: §3.1 · pp. 4–5

3.2 Tasks and Models

  • The study evaluates 12 open-weight models from 3B to 72B on MATH, MMLU-Pro, BigCodeBench, and three reasoning benchmarks, using 12 high- and low-headroom pairs per domain.All domains are program-graded; an unparseable reviewer answer leaves the generator’s answer as final.Source: §3.2 · p. 5, Table 2 · p. 5

3.3 Statistical Inference and Uncertainty

  • Question-paired bootstrap intervals quantify sampling uncertainty, while regressions relate ∆S and ∆G to capability gap, optionally controlling for headroom; slopes are read as linear associations.Source: §3.3 · p. 5

4 Results

Review rarely beats the stronger model alone, and its shortfall grows with the capability gap. It misses potential gains because the final answer tends to follow the generator, discarding answers the reviewer could have supplied.

4.1 RQ1: What does review do?

  • 22 of 48 weaker-reviewer directions fall significantly below G alone, with an average change of −1.6 points.The damage is more common at gaps of 14 points or more: 10 of 13 directions, versus 12 of 35 closer directions.Source: §4.1 · p. 6
  • Only 9 of 96 review directions beat the stronger model alone; review’s average shortfall is 3.1 points and grows with the gap.Qwen3-8B reviewing Llama-3.1-8B in math finishes 6.4 points below Qwen3-8B alone.Source: §4.1 · pp. 5–6, Figure 3 · p. 6

4.2 RQ2: Why does review miss the gains a second model could offer?

  • The two solo answers offer 7.5 points of oracle headroom on average, but post-review answer choice recovers only 0.6 or 2.8 points over the stronger model.The figures are for stronger-reviewing-weaker and weaker-reviewing-stronger directions, respectively.Source: Figure 4 · p. 6, §4.2 · p. 7
  • When only the reviewer is right alone, the final answer is right just 40% of the time for stronger reviewers and 17% for weaker reviewers.Across questions, the final answer differs from G’s less often than the two solo answers differ in correctness, leaving reviewer-only headroom largely unused.Source: §4.2 · p. 7
  • A stronger reviewer’s 7.6-point gain over G comes mainly from capability transfer: +8.4 points on reviewer-only questions, against −1.8 on generator-only questions and +0.9 elsewhere.Where the models agree, review itself changes accuracy by only +0.9 points with a stronger reviewer and −0.5 with a weaker one.Source: §4.2 · p. 7
  • Among close pairs where a weaker reviewer beats the stronger model, G’s correct answers survive 92% of the time, versus 72% when review hurts.Far-apart pairs put more at stake: only the stronger model is right alone on 30% of questions, compared with 14% for closer pairs.Source: §4.2 · p. 7

5 Using the second model

The section compares the cost and accuracy of review, selection, and an external judge against the stronger model alone. Review underperforms that baseline, while selection improves accuracy and beats review at the same two calls.

5 Using the second model

  • The selector lets both models answer independently, then uses a classifier fitted separately for each pair to choose the answer estimated more likely to be correct.Its cost proxy is parameters × (input + output tokens); review costs both calls, with the generator’s full answer included in the reviewer’s input.Source: §5 · pp. 7–8

5.1 RQ3: Is a second model worth its cost?

  • Review costs 2.8 times as much for −3.0 points when the stronger model reviews, and 2.6 times for −1.6 points when the weaker model reviews.Review is significantly above the stronger model in 1 and 7 of 48 pairs, respectively, and below it in 23 and 21.Source: §5.1 · p. 8
  • The selector costs 1.9 times as much as the stronger model alone and gains 1.2 points, recovering about 15% of the 7.6 points of headroom.It is significantly above the stronger model in 19 pairs and below it in none; JEV gains 4.1 points for an API fee of $0.04–$0.21 per 1,000 questions.Source: §5.1 · p. 8

5.2 RQ4: Select or review?

  • At the same two calls, selection is 4.2 points more accurate than stronger-reviewing-weaker and 2.7 points more accurate than weaker-reviewing-stronger, at lower cost.Selection costs 0.69 and 0.82 times as much, respectively, and is more accurate in every domain.Source: §5.2 · p. 8

6 Conclusion

The conclusion argues that judging review by its gain over the generator conflates verification with capability transfer. Across the studied review directions, reviewers often follow the generator; the authors recommend comparing review with the stronger model alone and selection between answers.

6 Conclusion

  • A gain over the generator can conflate verification with capability transfer, so the authors recommend reporting the stronger model alone and the decomposition by which model is right alone.They also recommend reporting that decomposition alongside ∆G, since ∆G alone credits a stronger reviewer’s own ability to review.Source: §6 · pp. 8–9
  • Across 96 review directions in four domains, review rarely beats the stronger model alone because the reviewer tends to follow the generator’s answer.The authors conclude that a second model is better used for its own answer as a candidate to choose from than as a reviewer.Source: §6 · p. 8
  • Before adding review, practitioners should check whether one call to the better model already wins; if paying for a second call, selection is the baseline to beat.The authors say reviewers often give up a correct answer of their own after reading the generator’s answer, even when asked to commit first.Source: §6 · p. 9

Limitations

The authors identify limits in model and task scope and note that their measurements do not explain what inside the reviewer causes it to follow the generator.

Limitations

  • The study covers open-weight models from 3B to 72B parameters and one thinking-on reviewer in two math directions; closed frontier models and more thinking-on models remain for future work.The reported model scope therefore does not establish whether the findings extend to those untested models and settings.Source: Limitations · p. 9
  • All tasks are in English and program-graded; comparison on open-ended generation, where no program can grade answers, is left for future research.Source: Limitations · p. 9
  • The authors measured how often final answers follow the generator and checked one alternative prompt on four directions, but did not test what inside the reviewer causes this behavior.Other prompts, multiple rounds, and training are left for future work; verification and capability transfer are defined by outcome, not the reviewer’s internal process.Source: Limitations · p. 9

A Model pairs

The appendix describes how model pairs were selected and gives the prompts used for solving and reviewing tasks. Pair selection varied model tiers and headroom, with additional rules for some pairs.

A Model pairs

  • Within each domain, models were sorted by development accuracy into strong, middle, and weak tiers; pairs sampled the six tier combinations at high and low weaker-model headroom.For some pairs involving Qwen2.5-72B, the next pair in headroom order was chosen to save compute; the original pairs were later run in both directions as well.Source: A Model pairs · p. 11
  • For code and reasoning, all questions were used to sort models into accuracy tiers; reasoning reused the knowledge prompts unchanged.Source: A Model pairs · p. 11, B Prompts · p. 12

B Prompts

  • Solve prompts asked each model to answer alone, while review prompts supplied the generator’s response and asked the reviewer to check and correct it if needed.The prompts used domain-specific formats, including boxed final answers for math, option-letter answers for knowledge and reasoning, and complete Python code blocks for code tasks.Source: B Prompts · pp. 12–13

C Selector and JEV

The selector estimates which of two answers is correct, while JEV reads the question and both answers to return correctness probabilities. The section also describes selector training and a third-call reference judge.

C Selector and JEV

  • JEV reads the question and both full answers, returns correctness probabilities, and averages two calls with reversed answer order.If answers agree, a call fails, or probabilities tie, the stronger model’s answer is returned.Source: C Selector and JEV · p. 13
  • The selector uses answer-level features, pairwise agreement, and question features; reasoning reuses the knowledge selector with its subject and source indicators set to zero.Math and knowledge use held-out test questions; code and reasoning use five-fold cross-fitting.Source: C Selector and JEV · pp. 13–14
  • A three-call design would let the reviewer choose between independent answers, but it is excluded to compare methods at the same call count.A commit-first variant was tested on four directions; it retained 55% of reviewer-only correct answers, versus 61% with the main prompt.Source: C Selector and JEV · p. 14
  • The selector estimates each answer’s probability of correctness and returns the higher-scoring answer, breaking ties in favor of the stronger model.Each pair has its own selector; it averages an L2 logistic regression and a gradient-boosted tree.Source: C Selector and JEV · p. 14

D Cost with output tokens weighted four times

Weighting output tokens four times more heavily lowers review’s cost relative to the stronger solo model, while the selector’s accuracy advantage over review remains and it costs less on average.

D Cost with output tokens weighted four times

  • 2.2 and 2.0 times: review’s cost relative to the stronger model alone under output-weighted cost, versus 2.8 and 2.6 before.The selector’s cost is 1.9 times the stronger model’s; it costs 0.89 times as much as stronger-model review and 0.98 times as much as weaker-model review.Source: D Cost with output tokens weighted… · p. 14
  • The selector costs less than review in 34 and 28 of 48 pairs, respectively, and retains its accuracy advantage while costing less on average.Accuracy is unchanged by the cost measure; the selector can cost slightly more than weaker-model review in math, knowledge, and reasoning.Source: D Cost with output tokens weighted… · p. 14

E Per-domain results and robustness

Review is below the stronger model on average in every domain and direction except one knowledge comparison; the selector beats both review directions in every domain.

Bootstrap

Per domain

  • Review falls below the stronger model in every domain and both directions except weaker-model review in knowledge, which is level at +0.1 [−0.3, +0.5].Code is the weakest domain for review: stronger-review lifts the generator by only 2.7 points, and no weaker reviewer improves on it.Source: E Per-domain results and robustness · p. 14
  • The selector is above the stronger model in every domain except code and above both review directions in every domain.Source: E Per-domain results and robustness · p. 14

Robustness checks

Resampling model pairs

  • Resampling model pairs widens intervals, but every average keeps its sign; the oracle after stronger-model review is the exception that reaches below zero.The authors treat this as a robustness check, since models recur across pairs and pairs are not fully independent.Source: E Per-domain results and robustness · p. 15

F Further checks

Further checks compare selection and review under equal data access, examine when reviewers retain their own correct answers, and test alternative prompts, reasoning settings, and output limits.

All eight cells

  • When the stronger model reviews, review-only outcome cells contain 2.9% and 1.9% of questions, while it takes up 8.4% of its own correct answers and loses 9.8%.Source: F Further checks · p. 15

Does the reviewer follow G?

  • The final answer matches G’s on 57.7% of questions with a stronger reviewer and 69.1% with a weaker one; most lost reviewer-only correct answers end on G’s wrong answer.Those losses become G’s wrong answer 53.3% and 65.0% of the time, respectively; in code, the reviewer’s program never equals G’s, but the final program does on 21% of questions.Source: F Further checks · p. 15

When does the reviewer keep its own answer?

  • Across directions, keeping reviewer-only correct answers correlates with reviewer strength relative to G; within directions, longer reviewer solutions correlate with keeping fewer.On held-out math and knowledge questions, each 10-point larger gap predicts a 1.9- and 1.5-point lower ΔS for stronger- and weaker-reviewer directions; the authors describe these as correlations, not causal effects.Source: F Further checks · pp. 15, 17

Self-review as a control

  • Self-review retained 99.8% of reviewer-only right answers in math and 91% in knowledge, versus 65% and 44% after reviewing G.Accuracy under self-review stayed within five questions of solo accuracy, supporting the authors’ interpretation that losses largely come from being shown G’s answer.Source: F Further checks · pp. 15–16

A prompt without the instruction to retain

  • Removing the instruction to retain did not eliminate following G: the commit-first prompt kept 55% of reviewer-only correct answers, versus 61% with the main prompt.The authors report that following G is not an effect of the retain instruction.Source: F Further checks · p. 16

Thinking on

  • With thinking enabled, review still fell 2.8 points [1.0, 4.6] and 3.0 [0.8, 5.2] below Qwen3-8B alone when reviewing weaker and stronger models.It appeared to stop following the weaker generator but still followed the stronger one; the authors leave the reason and generality for future work.Source: F Further checks · p. 16

Output length

  • Only 1.7% and 1.8% of review outputs hit the 4,096-token limit, versus 1.8% for the stronger solo model, making truncation an unlikely explanation for review losses.Source: F Further checks · p. 16

A selector without labels

  • 1.2 points [0.9, 1.4]: the trained selector exceeds the stronger model alone, and is 4.2 [3.8, 4.6] and 2.7 [2.4, 3.1] points above stronger- and weaker-model review.A rule selecting the answer with higher mean token log-probability is 3.7 points [3.4, 4.1] below the stronger model alone.Source: F Further checks · p. 17

G Additional figures

The additional figures and tables organize review outcomes by model-gap and reviewer direction, track how final answers relate to the stronger model, and document domain-level breakdowns and robustness checks. They also describe selector features and cost comparisons.

G Additional figures

  • Figure 6 groups review directions by model-gap range and outcome level, separating stronger-reviewer from weaker-reviewer cases.The caption notes that only three outcome levels occur when the weaker model reviews the stronger one.Source: Figure 6 · p. 18
  • Figure 7 plots how often the reviewer differs from G in correctness against how often the final answer does, with points below the dashed line indicating movement toward G.Each point is one review direction, and crosses show means weighted equally across domains.Source: Figure 7 · p. 18
  • Table 5 lists selector features and counts by domain, while Table 6 compares review and selector costs under two token-cost schemes and counts pairs where the selector is cheaper.The logistic model uses one answer and question features; the tree also uses both answers and pair features.Source: Table 5 · p. 20, Table 6 · p. 20
  • Tables 7–8 report domain-level mean ∆G and ∆S or accuracy differences, while Table 10 distinguishes question-only from model-pair-and-question resampling.The 48-pair inventory gives question counts by domain and flags an earlier math run for one pair.Source: Table 4 · p. 19, Table 7 · p. 21, Table 8 · p. 21
  • Table 9 assesses robustness by leaving out each model in turn and by adding completed pairs.Its leave-one-model-out column reports the lowest and highest mean over the 12 models.Source: Table 9 · p. 21
  • Table 11 decomposes ∆G into correctness-pattern components, while Table 12 reports odds ratios and kept shares and labels the relationships as correlations, not causal effects.Table 11 also notes that percentages may not sum to 100 because of rounding.Source: Table 11 · p. 22, Table 12 · p. 22
Loading 2610.15897v2…