PaperlayerView exact revision on arXiv

Source-linked AI summary

Are You Sure You're Sure? Two Confounds in a Sycophancy Benchmark

Atharv Gupta, Akshat Jindal, Lavanya Nigam, Aryan Sood

arXiv:2609.32867v1cs.CL

TL;DR

Sycophancy benchmarks script a user’s pushback, but template features that vary alongside the tested feature can confound what their scores measure. This paper audits SycEval by adding controlled comparisons for naming and instruction placement; both affect measured caving, and the results motivate three checks for benchmark authors.

  • Problem

    SycEval attributes differences in caving between preemptive and in-context objections to timing, but its templates vary other features alongside timing, limiting what the comparison isolates.

    Source: §1 · p. 2

  • Method

    The paper audits SycEval by building missing template arms that vary naming or instruction placement while holding the other feature fixed.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    Naming raises follow rate by 14.1–49.5pp, and the naming-controlled timing comparison reverses on three of five model conditions; instruction placement also shifts caving in opposite directions across models on shared items (p = 0.0059).

    Source: §1 · p. 2, §5.2 · p. 6

  • Takeaways & Limitations

    The authors distill the audits into three checks covering naming, instruction position, and how sycophancy is reported.

    Source: §1 · p. 2

    The multiple-choice instruction-position contrast for Qwen2.5-3B depends on item selection, and the compared instructions are worded slightly differently rather than being one string moved unchanged.

    Source: §5.2 · p. 6

Abstract

from arXiv · show

Sycophancy is a language model's tendency to cave when a user pushes back, abandoning a correct answer for the user's. Several benchmarks now measure it by scripting an objection and recording how often the model caves. Because that objection is a prompt template, whatever else the template varies is measured along with the property it claims to isolate. We audit SycEval, which reports that objections raised before a model answers (preemptive) cause more caving than those raised after (in-context), and attributes the gap to timing. Two features of its templates vary alongside the property each is meant to test. First, SycEval's objections escalate through four strength levels, and at the two weakest only the preemptive template names a target answer, so timing and naming vary together. We build the missing comparison and test it on multiple-choice questions and SycEval's own free-form pipeline. Naming a target answer raises the follow rate, the share of samples matching the user's assertion, by 14.1--49.5 percentage points (pp), and once both templates name one, the timing comparison reverses on three of the five model conditions we test: models cave \emph{less} under preemptive objections than in-context ones, opposite to SycEval. Second, the output-format instruction benchmarks append for automatic grading also varies with objection placement. Moving it from the pushback into the question flips its effect in opposite directions across models on the same items ($p=0.0059$). The same effect appears in SycEval's free-form pipeline: relocating its instruction lowers caving by 5.1pp on Llama-3.1-8B, while an equivalence test confirms no effect on Qwen3-4B. Both confounds live in the template rather than the models under test, so both are correctable: we close with three checks benchmark authors can apply before publishing.

1 Introduction

SycEval compares objection timing, but its templates also vary in other features, so the reported gap may not isolate timing. The paper audits naming and instruction placement, finding that both affect measured caving and offering benchmark authors a three-item checklist.

1 Introduction

  • SycEval measures whether an objection changes a model’s answer, distinguishing flips toward the correct answer from harmful flips away from it.Its headline comparison attributes a gap between in-context and preemptive objections to when the objection arrives.Source: §1 · pp. 1–2
  • Because benchmark templates vary alongside the feature they are meant to test, a measured gap may reflect both features rather than the intended one alone.Source: §1 · p. 2
  • Naming raises follow rate by 14.1–49.5pp, and matching naming reverses the timing comparison in three of five model conditions.Both free-form runs use SycEval’s datasets, rebuttal templates, and judge protocol.Source: §1 · p. 2
  • Moving the output-format instruction from pushback to question reverses its effect across two models on shared items (p = 0.0059).The same effect appears in SycEval’s free-form pipeline.Source: §1 · p. 2
  • The audit concludes with three checks benchmark authors can run on their templates before publishing, covering naming, instruction position, and reporting.Source: §1 · p. 2

2 Related work

Prior work studies sycophancy across different tasks and interaction settings, while benchmark research shows that measurement conditions can change results. The paper positions its audit as a test of template choices that benchmarks commonly treat as fixed.

Sycophancy as a phenomenon

  • Sycophancy has been linked to training preferences for agreeable answers, and activation patching has traced it to a late-layer override of an already-correct representation.A bare “are you sure?” flipped answers on 46% of classifications averaged across ten models and seven tasks.Source: §2 · p. 3

Sycophancy benchmarks

  • SycEval varies objection timing and strength, while SYCON-Bench varies pushback rounds, ELEPHANT social context, PENDULUM modality, and BrokenMath theorem-proving behavior.These benchmarks fix their template choices, including turn count, pushback content, and whether an answer is named.Source: §2 · p. 3

Measurement conditions confound benchmark results

  • Measurement framing and format can shift benchmark scores: question versus assertion framing changes measured sycophancy, while multiple-choice versus open-ended phrasing shifts measured safety by 5 to 20pp.Other cited work reports formatting shifts in accuracy and option-order effects; automated sycophancy judges also raise a human-validation concern.Source: §2 · p. 3

3 Problem formulation

SycEval’s preemptive and in-context objections differ in both timing and whether they name a target answer, leaving timing confounded. The paper builds the missing comparison and frames a general audit that varies the suspected confound and intended variable separately.

3.1 The confound in SycEval’s templates

  • SycEval varies objection timing and strength across four nested levels, from a bare objection to an added fake citation.Source: §3.1 · p. 3
  • The preemptive version removes the model’s prior turn but also names a target answer, so the original design has no arm separating timing from naming.Source: §3.1 · p. 3
  • The added INCONTEXT_NAMED arm holds timing fixed to compare naming, then holds naming fixed to compare PREEMPTIVE against INCONTEXT_NAMED.The latter comparison isolates timing, which SycEval’s original design does not provide.Source: §3.1 · p. 4
  • At Simple and Ethos strength, only the preemptive template names a target answer; at Justification and Citation, both timing conditions name one.Because SycEval aggregates all four strengths, the asymmetry inflates its headline gap rather than explaining it outright.Source: §3.1 · p. 4

3.2 Formulation

  • A template confound is a feature that varies with the intended variable across conditions, making the reported difference measure both jointly.Source: §3.2 · p. 4
  • The audit builds the missing arm, then checks whether the effect persists, shrinks, reverses, or disappears when the confound is held fixed.It also estimates the confound’s own effect by holding the intended variable fixed; the two audits examine naming and instruction position.Source: §3.2 · p. 4

4 Experimental methodology

The experiments measure user-following across multiple-choice and free-form tasks, with paired per-item contrasts and different answer-grading procedures. The setup uses repeated sampled completions and tests supported null results with equivalence tests or Bayes factors.

4.1 Metric

  • Follow rate is the share of sampled completions matching the user’s assertion; on baseline-correct items it equals cave rate, while baseline-incorrect follows are progressive.Source: §4.1 · p. 4
  • Multiple-choice answers use exact string matching, while a second language model grades free-form answers under SycEval’s protocol.A blind annotation check found the judge tracked annotator majority at least as closely as annotators tracked one another.Source: §4.1 · p. 4
  • At full precision, Qwen3-4B’s 0.0% cave rate in two multiple-choice position arms leaves little room for a suppressive effect, but its free-form rate is 35.6%.The paper reports its smallest naming effect and a confirmed null position effect in free-form as well.Source: §4.1 · p. 4
  • The study samples k = 20 completions per item per arm at temperature 0.7, reporting per-item proportions rather than a single greedy completion.Greedy decoding can hide minority caving behavior.Source: §4.1 · pp. 4–5
  • Table 1 reports naming and timing contrasts in percentage points across five model conditions, with free-form entries pooled over four rebuttal strengths.Naming compares INCONTEXT_NAMED − INCONTEXT; timing compares PREEMPTIVE − INCONTEXT_NAMED.Source: Table 1 · p. 5

4.2 Setup

  • The experiments cover three models across multiple-choice and free-form formats, with the naming and instruction-position audits using overlapping but distinct model conditions.Free-form runs use SycEval’s AMPS and MedQuAD datasets; a separate Qwen2.5-7B-Instruct writes rebuttals and judges responses.Source: §4.2 · p. 5
  • Contrasts are paired per item and tested with Wilcoxon signed-rank tests; reported nulls use a two one-sided equivalence test or Bayes factor.The study draws multiple-choice items from four datasets and free-form items from SycEval’s two datasets.Source: §4.2 · p. 5

5 Experiments

Naming substantially changes measured follow rates, and controlling it reverses the timing comparison in several conditions. The placement of output-format instructions also changes caving, with effects that vary by model; tests address possible quantization and truncation explanations.

5.1 The naming confound

  • 45.7pp on Qwen2.5-3B and 48.9pp on Llama-3.1-8B: naming a target answer raises multiple-choice follow rates, while controlled timing comparisons reverse to −14.3pp and −33.1pp, respectively.Qwen3-4B at full precision is an exception: its timing comparison remains +20.2pp, with the smallest naming effect measured.Source: §5.1 · p. 5
  • 49.5pp at Simple and 47.5pp at Ethos, where only the preemptive template names an answer; with naming held fixed, free-form timing shifts follow rate by −14.4pp.At Justification and Citation, both timing templates already name an answer and the naming gap falls to a few points without significance; the free-form reversal also appears in SycEval’s own pipeline.Source: §5.1 · p. 6

5.2 The instruction-position confound

  • On 36 shared items, moving the instruction into the question raises caving by 19.0pp on Qwen2.5-3B and lowers it by 7.8pp on Qwen3-4B; the cross-model difference is significant (p = 0.0059).The Qwen2.5-3B contrast is +14.3pp across its 42 correctly answered items, so the shared-item result is not presented as a confirmed effect on that model alone; the two instructions also differ slightly in wording.Source: §5.2 · p. 6
  • On Llama-3.1-8B, moving the instruction into the question changes median replies from one character to 284, while the caving shift is −7.0pp and not significant (p = 0.174).Qwen3-4B replies remain one character either way; Llama’s most constrained condition has 80.1% caving, versus 59.5% with no instruction.Source: §5.2 · p. 7
  • On Llama-3.1-8B, moving the unchanged instruction into the question lowers caving by 5.1pp; on Qwen3-4B, the +0.7pp effect is indistinguishable from zero by equivalence testing.Llama’s regressive caving falls from 54.7% to 42.8%, while its progressive rate barely moves; the two models’ instruction-position effects differ by −6.1pp.Source: §5.2 · p. 7

5.3 Is the position effect an artifact?

  • Full-precision Qwen3-4B does not support quantization as the cause of the position effect, and uneven truncation does not track caving changes (p = 0.87).The full-precision estimate grows from −7.8pp at 4-bit to −11.1pp; adjusting for truncation changes the cross-model estimate from −6.1pp to −6.3pp.Source: §5.3 · pp. 7–8

6 A checklist for sycophancy benchmarks

The checklist asks benchmark authors to test whether template details confound the comparisons they intend to make. It covers naming, instruction placement, and reporting the direction of sycophantic changes.

6 A checklist for sycophancy benchmarks

  • Check whether timing conditions differ in naming a candidate answer; if they do, hold naming fixed before attributing a gap to timing.The timing gap then reflects more than timing alone.Source: §6 · p. 8
  • Check where each instruction appears, because placement can move the measurement by an amount that depends on the model.Identical wording across turns does not guarantee identical measured effects.Source: §6 · p. 8
  • Report regressive and progressive rates separately: pooled follow rates can shift with the item pool’s room for progressive flips, even before model differences enter.Source: §6 · p. 8

7 Limitations

The audit covers a small sample of six model conditions across two families and only one benchmark, limiting how broadly its findings can be generalized.

7 Limitations

  • Six conditions across two model families remain a small sample, frontier models were not evaluated, and whether these confounds recur beyond the single audited benchmark remains open.The free-form cross-model comparison is less tightly controlled because rebuttals differ by subject model; the authors cannot rule out rebuttal content as part of the measured difference.Source: §7 · p. 8

8 Conclusion

The conclusion argues that template details can shape which model appears more sycophantic, even when the models’ behavior is unchanged. The appendix catalogs the prompt arms used to vary naming and instruction placement.

8 Conclusion

  • The audit concludes that naming and instruction placement confound SycEval’s measurements, while the models tested cave at every rebuttal strength.The authors focus scrutiny on the measurement rather than disputing the sycophancy construct.Source: §8 · p. 8

A Prompt templates, verbatim

  • Table 2 indexes the prompt arms by manipulated feature: naming varies while conversation structure is held fixed, and instruction position varies while wording is held fixed.Source: Table 2 · p. 11

A.1.1 Naming-confound arms

  • The naming-confound arms compare SycEval’s in-context and preemptive prompts with an in-context condition that adds a named target answer; additional variants remove “CONCISE” and the trailing exclamation marks.The preemptive template is single-turn, unlike the shared two-turn scaffold of the other arms.Source: §A.1.1 · p. 11

A.1.2 Instruction-position arms

  • The multiple-choice position arms vary the instruction and its turn while sharing the same two-turn scaffold and fabricated argument per item.The filler sentence is length-matched to the concise instruction, controlling for rebuttal length alone.Source: §A.1.2 · p. 12

A.2 Free-form

  • The free-form pipeline uses SycEval’s generation protocol, with a separate rebuttal writer that never sees the question and a second model that judges responses.Source: §A.2 · p. 13

A.2.1 Naming-confound arms

  • In the free-form naming arms, the in-context prompt names no answer by construction, while the added named condition names one; at Justification and Citation, both timing arms already name an answer.Source: §A.2.1 · p. 13

A.2.2 Instruction-position arms

  • The free-form instruction-position arms move the same instruction between turns, and also combine both placements with the naming-on condition.Source: §A.2.2 · pp. 13–14

B Full per-arm results

Instruction position produces a significant cross-model interaction on shared items, while Qwen3-4B is the exception to the naming-controlled timing reversal. The full results also show that instruction-position effects may differ in mechanism across models.

B Full per-arm results

  • For Qwen3-4B in full precision, two runs with fresh random draws over 82 items (42 baseline-correct, 40 baseline-incorrect), at k = 20, agreed within one pp on every contrast.Source: Table 3 · p. 14
  • Table 6 covers Llama-3.1-8B across all six naming-confound arms, reporting regressive and progressive rates and follow rate for 120 items (60 baseline-correct, 60 baseline-incorrect), with k = 20.Source: Table 6 · p. 15
  • The largest strength-ladder step is +16.8pp between Ethos and Justification, where both timing templates begin naming an answer.At Justification and Citation, where both templates name an answer, the naming effect is null.Source: B Full per-arm results · p. 16, Figure 3 · p. 16

C Full instruction-position results

  • Table 9 assigns zero follow rate to items with wholly ungradable draws, keeping all instruction-position contrasts on the same item set.Its unconstrained arms are reported for completeness but excluded from claims because Qwen3-4B outputs are often truncated.Source: Table 9 · p. 17
  • On 36 shared items, Qwen2.5-3B’s position contrast is +19.0pp (p = 0.046), while the cross-model interaction is −26.8pp (p = 0.0059).The position row is the paper’s claim; the other rows in Table 11 are diagnostic.Source: C Full instruction-position results · p. 17, Table 11 · p. 17

D The Llama-3.1-8B mechanism, in full

  • TURN1_LOCK raises caving on Llama-3.1-8B but lowers it on Qwen3-4B (−5.0pp, p = 0.017), a second cross-model sign flip.Llama’s position contrast is non-significant, and the authors leave open whether its partial reply suppression reflects a different mechanism from Qwen3-4B’s lockout.Source: D The Llama-3.1-8B mechanism, in full · p. 18, Table 12 · p. 18

E Judge validation

The authors validate SycEval’s free-form judge against human labels and explain how its one-way error affects comparisons. They also document item-paired tests and the treatment of ambiguous truncated replies.

E Judge validation

  • The judge matches annotator majority at κ = 0.70, at least as closely as annotators agree (κ = 0.63), and matches unanimous labels at κ = 0.93.Source: E Judge validation · p. 18
  • Judge disagreements mostly label annotator-correct replies incorrect, but this one-way error is roughly equal across arms and largely cancels in arm-difference results.Source: E Judge validation · pp. 18–19
  • Truncated replies are excluded because annotators also disagree on them (κ = 0.14), leaving replies without clear labels unscored for either party.Source: E Judge validation · p. 19

F Full statistics

  • Contrasts pair responses item by item and use Wilcoxon signed-rank tests, rather than subtracting pooled rates.Pairing also cancels item difficulty when testing differences between arms.Source: F Full statistics · pp. 19–20
  • A within-model contrast is not treated as a confirmed effect by itself when its significance depends on which item pool is tested.The cross-model claim rests on a direct interaction test using shared items.Source: F Full statistics · p. 20

F.1 Equivalence tests and Bayes factors

Equivalence tests distinguish confirmed nulls from uncertain contrasts, while the position-interaction verdict depends on how tests are grouped for correction. The authors adopt a primary-hypothesis family, under which the interaction survives correction.

F.1 Equivalence tests and Bayes factors

  • Qwen2.5-3B’s +19.0pp multiple-choice position contrast is not equivalent to zero within ±4.5pp, while Qwen3-4B’s multiple-choice result remains undecided.For free-form Qwen3-4B, the contrast is equivalent to zero within ±4.5pp (TOST p = 0.0415; BF01 = 11.89).Source: §F.1 · p. 20

F.2 Holm correction, three ways

  • The correction family changes the position-interaction verdict; the adopted family includes one primary test per model per confound, not replications and diagnostic controls.Source: §F.2 · p. 20

Primary hypotheses only (the standard we adopt)

Every contrast in the paper, as one family (conservative)

G Compute and reproducibility

The reported runs specify GPU, weight precision, sampling, and elicitation settings, alongside the separate model used for free-form rebuttals and judging.

G Compute and reproducibility

  • Runs used NVIDIA A10G GPUs (24GB); quantized Qwen2.5-3B and Qwen3-4B used 4-bit weights, while Qwen3-4B and Llama-3.1-8B full-precision runs used bf16.Source: G Compute and reproducibility · pp. 21–22
  • Free-form rebuttals and response judgments used a separate Qwen2.5-7B-Instruct model.Source: G Compute and reproducibility · p. 22
  • Each reported cell used k = 20 sampled completions at temperature 0.7, followed by greedy forced-answer elicitation capped at 6 tokens.Parse failures were negligible: at most 3 of 9,840 draws in the reported naming arms and none in the position arms.Source: G Compute and reproducibility · p. 22
  • Baseline follow/cave rates ranged from 4.7% to 59.5%; Qwen3-4B’s 4.7% multiple-choice rate limited room for a suppressive effect there.Its free-form unaided rate was 35.6%, with room to move in either direction.Source: G Compute and reproducibility · p. 22
Loading 2609.32867v1…