PaperlayerView exact revision on arXiv

Source-linked AI summary

Receptiveness, Not Sycophancy: Distinguishing Engagement from Deference in Language Models

Calvin Isley, Johann Gaebler, Max Lamparth, Julia Minson, Sharad Goel

arXiv:2609.26579v1cs.CLcs.AIcs.HC

TL;DR

The paper examines how conversational receptiveness overlaps with measures of social sycophancy in moral advice. It finds that receptive responses are often scored as more sycophantic, while participants prefer them and a judgment-first approach can increase receptiveness without substantially increasing deference.

  • Problem

    Social-sycophancy indicators overlap with conversational receptiveness, creating a construct-validity problem for evaluations intended to identify unwarranted deference.

    Source: Distinguishing receptiveness from… · p. 3, Discussion · p. 7

  • Method

    Using a moral-advice Reddit dataset, the authors compare sycophancy and receptiveness measures, test response rewrites and preferences, and evaluate direct prompting versus judgment-first revision.

    Source: Introduction · p. 1, Setup · p. 3, Improving receptiveness · p. 6

  • Results

    More socially sycophantic-scored responses were also more receptive; participants preferred receptive rewrites, and judgment-first revision largely avoided increased deference.

    Source: Introduction · p. 1, Results · p. 5, Discussion · p. 7

  • Takeaways & Limitations

    Social-sycophancy evaluations should validate penalized behaviors in context, since receptiveness can coexist with independent judgment.

    Source: Discussion · p. 7

    The study focuses on moral advice, uses an imperfect automated receptiveness measure, and measures stated preferences and expectations rather than downstream behavior.

    Source: Discussion · p. 7

Abstract

from arXiv · show

A central concern with language models is sycophancy: their tendency to defer to users' views at the expense of independent substantive judgment. In parallel, work on social sycophancy has focused on behaviors such as validation and positivity that may signal inappropriate deference. Yet the markers of social sycophancy are also characteristic of conversational receptiveness, a construct from social psychology shown to improve interactions across disagreement. We argue that this overlap creates a construct-validity problem for social sycophancy evaluations. Using a popular moral-advice dataset, we find that responses classified as more socially sycophantic are also more receptive. Further, increasing the receptiveness of human-written responses---while preserving their substantive conclusions---causes them to be classified as more socially sycophantic. This tight coupling raises the possibility that social sycophancy evaluations inadvertently penalize desirable behavior. In a preregistered experiment comparing substantively equivalent responses, participants prefer the more receptive responses, expect users to be more likely to listen to them, and are more willing to seek advice from their authors. The same overall pattern persists even among participants who believe the original question asker is in the wrong. Finally, we introduce a simple approach that substantially increases receptiveness without increasing substantive deference, demonstrating that conversational receptiveness and substantive independence can be achieved together.

Introduction

Social-sycophancy indicators overlap with conversational receptiveness, creating a construct-validity concern: measures may mistake constructive engagement for undesirable deference. Across moral-advice studies, more receptive responses score as more sycophantic, while participants prefer receptive responses and models can increase receptiveness without surrendering independent judgment.

Introduction

  • Social-sycophancy markers such as validation and positivity also characterize conversational receptiveness, which involves engaging thoughtfully with another perspective while remaining willing to disagree.The overlap raises a construct-validity concern because these measures may conflate undesirable accommodation with communication associated with better interactions across disagreement.Source: Introduction · p. 1
  • More receptive responses were preferred, expected to attract more listening, and made participants more willing to seek advice from their authors, even when they judged the advice-seeker wrong.The same overall pattern held among participants who believed the question asker was in the wrong.Source: Introduction · p. 1
  • The study uses a Reddit moral-advice dataset to test the overlap and whether receptive communication can coexist with independent judgment.It examines both the association between measured social sycophancy and receptiveness and changes to human-written responses that preserve their conclusions.Source: Introduction · p. 1
  • Measures intended to detect undesirable accommodation may penalize valuable receptive behavior, even though receptiveness and substantive independence can be achieved together.The paper calls for distinguishing unwarranted from appropriate interpersonal accommodation in context.Source: Introduction · pp. 1–2

Related Work

Prior work distinguishes substantive shifts toward users from socially deferential communication, while conversational receptiveness describes how disagreement is expressed rather than whether the speaker agrees. Their observable features overlap, creating a risk that social-sycophancy measures penalize constructive disagreement.

Sycophancy

  • Substantive sycophancy research tests whether a model’s judgment shifts with users’ stated views, factual suggestions, rebuttals, or pressure; social-sycophancy work examines behaviors such as praise, validation, indirectness, and accepting user framings.Social accommodation need not imply a changed underlying judgment, and its normative status depends on context.Source: Sycophancy · p. 2

Conversational receptiveness

  • Conversational receptiveness communicates thoughtful engagement with opposing views, without requiring agreement or substantive concession.It includes signaling that the other perspective has been considered and understood while maintaining a different view.Source: Conversational receptiveness · p. 2

Distinguishing receptiveness from sycophancy

  • Receptiveness and substantive sycophancy describe separate dimensions: a model can engage constructively with a user while maintaining an independent judgment.The paper names this combination receptive independence.Source: Distinguishing receptiveness from… · pp. 2–3
  • Social-sycophancy indicators overlap with acknowledgment, common ground, hedging, and positive reframing, so receptive disagreement may receive a higher sycophancy score.The illustrative poem example contrasts engaging with the user’s perspective against changing the model’s judgment to match the user’s preference.Source: Distinguishing receptiveness from… · pp. 2–3, Table 1 · p. 3

Common Measures of Social Sycophancy Are Sensitive to Receptiveness

In moral-advice responses, measured social sycophancy tracks receptiveness, and rewriting responses to be more receptive while preserving their verdicts increases their measured sycophancy. The study measures both constructs in human and model responses and finds a similar association in a second dataset.

Setup

  • The comparison includes human top comments and responses from four language models to Reddit interpersonal-conflict posts whose top comment judges the poster to be in the wrong.For model responses, the analysis further focuses on cases where the model concludes the user is in the wrong.Source: Setup · p. 3, Social sycophancy tracks receptiveness · p. 3
  • The study measures four social-sycophancy indicators and scores receptiveness using model-assessed linguistic features weighted against human ratings.Social-sycophancy score sums binary positivity, validation, indirectness, and framing-sycophancy indicators; the receptiveness weights use ratings from 2,860 texts.Source: Setup · p. 3

Social sycophancy tracks receptiveness

Receptiveness increases measured sycophancy

  • 1.50 standard deviations more receptive (95% CI [1.45, 1.54]) and 97.9% verdict preservation (95% CI [97.1%, 98.5%]) accompanied a stark rightward shift in social-sycophancy scores after rewriting.Figure 2 shows the before-and-after distribution for human-written top comments.Source: Receptiveness increases measured… · p. 4, Figure 2 · p. 4

Receptiveness Is Perceived to be Valuable

The preregistered experiment tests whether readers value more receptive moral criticism when the substantive judgment is unchanged. Participants preferred receptive rewrites, including when they themselves judged the asker to be in the wrong.

Receptiveness Is Perceived to be Valuable

  • The study asks whether receptive wording makes moral criticism easier to engage with or instead softens it into unwarranted validation.It compares human-written and LLM-generated replies that tell the user they are in the wrong.Source: Receptiveness Is Perceived to be… · p. 4

Study design

  • The experiment compared original human or model replies with more receptive rewrites that preserved each reply’s substantive conclusion.Two hundred participants evaluated five randomly selected posts each from 100 scenarios; items were restricted to cases where the reply judged the asker in the wrong.Source: Study design · pp. 4–5

Results

  • Human-comment rewrites scored 1.00 for expected listening and 0.99 for personal-advice preference; model-response rewrites scored 0.24 and 0.20, respectively.Quality ratings rose by 1.50 points for human comments and 0.13 points for model responses; effects persisted controlling for length.Source: Results · p. 5
  • Preference for receptive rewrites fell by 0.35 points per standard deviation of original receptiveness for expected listening and by 0.32 for personal-advice preference.Model-generated originals were 1.4 standard deviations more receptive than human originals on average, consistent with their smaller rewrite effects.Source: Results · p. 5
  • Among participants who themselves judged the asker wrong (61.9% of the time), receptive rewrites of human comments averaged 1.66 points above originals, with preferences of 1.17 for likelihood of listening and 1.09 for personal advice.Source: Results · p. 5
  • Readers’ preference for receptive rewrites cannot be explained by user-favoring deference because the substantive conclusion was held fixed.The authors conclude that social-sycophancy evaluations risk penalizing valuable communication in this setting.Source: Results · p. 5

Achieving Receptive Independence

The paper separates how models communicate from whether their substantive judgments shift with the user’s perspective. Its inference-time receptiveness tool substantially increases receptiveness while leaving substantive deference nearly unchanged.

Achieving Receptive Independence

  • Increasing receptiveness matters because warmth can spill over into affirming incorrect user beliefs, so communication gains must be separated from substantive deference.Source: Achieving Receptive Independence · p. 5

Substantive deference and baseline model behavior

  • Substantive deference is measured by comparing judgments from user-perspective and neutral third-person presentations of the same situation.Syc(π) = 0.5 indicates invariance; values above 0.5 indicate substantive deference, with fault comparisons made by an LLM judge focused on substantive conclusions.Source: Substantive deference and baseline… · pp. 5–6
  • At baseline, GPT-5.6 Terra, Claude Sonnet 5, and Gemini 3.7 Flash had Syc(π) scores of 0.47–0.50, while Llama 4 Scout scored 0.56.The models differed substantially in receptiveness despite similar levels of substantive independence among the three frontier models.Source: Substantive deference and baseline… · p. 6

Improving receptiveness

  • The authors avoid post-training interventions because the open-weight model was already highly receptive and closed-weight models could not be fine-tuned through public interfaces.Source: Improving receptiveness · p. 6
  • Directly prompting models to be more receptive also shifted their judgments toward the user: Claude’s Syc(π) rose from 0.47 to 0.56, and Gemini’s from 0.49 to 0.54.The comparison used a random subset of 200 AITA-YTA items.Source: Improving receptiveness · p. 7
  • The receptiveness tool raised receptiveness by 2.1 standard deviations for Gemini 3.7 Flash and 1.4 for Claude Sonnet 5, while Syc(π) remained near 0.5.Gemini’s Syc(π) reached 0.50; Claude’s moved from 0.47 to 0.48. The method transforms an initial response while remaining faithful to its original judgment.Source: Improving receptiveness · p. 7

Discussion

Social-sycophancy measures are entangled with conversational receptiveness, risking penalties for beneficial engagement; the discussion also shows that receptiveness can be increased without sacrificing independent judgment. The authors urge context-sensitive evaluation and note limits on the findings’ scope and interpretation.

Discussion

  • Social-sycophancy measures may penalize beneficial engagement, so evaluations should validate that penalized behaviors reflect undesirable accommodation and assess them in context.The authors caution that validation, warmth, or tact may be inappropriate when they obscure important conclusions or reinforce false beliefs.Source: Discussion · p. 7
  • Anchoring a model’s judgment before changing how it communicates largely avoids the deference spillover that can follow direct receptiveness prompting.The authors present this as proof of concept that constructive disagreement need not come at the cost of independent judgment.Source: Discussion · p. 7
  • The findings are limited to moral advice, use an imperfect automated receptiveness proxy, and measure stated preferences and expectations rather than downstream behavior.The authors also caution that the substantive-deference measure captures sensitivity to one user signal, not universal substantive sycophancy.Source: Discussion · p. 7
  • The paper concludes that receptive communication and substantive independence can coexist, and that evaluations should distinguish constructive engagement from unwarranted deference.The authors argue that this distinction can help preserve socially valuable engagement when reducing social sycophancy.Source: Discussion · p. 7

A Supplementary Information

  • The supplement details datasets, receptiveness and sycophancy measurement, receptive rewrites, the Prolific experiment, substantive-deference measurement, and robustness analyses.It also provides prompts used in measurement and intervention procedures.Source: A Supplementary Information · p. 11

B Datasets

The analyses use Reddit’s AITA-YTA collection, defined by posts whose top-ranked human comment judges the asker to be in the wrong. The dataset contains 2,000 posts, with a smaller text-eligible set and model-specific analysis subsets.

B.1 AITA-YTA

  • AITA-YTA comprises r/AmITheAsshole posts for which the top-ranked human comment concludes that the asker is in the wrong.The collection contains 2,000 posts, each paired with its human-written top comment.Source: §B.1 · p. 11
  • Excluding deleted human comments and comments shorter than 15 characters leaves 1,892 posts for analyses requiring human-comment text.Source: §B.1 · p. 11
  • For each model, analyses include only examples where both the model and top comment judge the asker to be in the wrong, yielding 350–975 examples per model.The model-specific counts are 949 for GPT-5.6 Terra, 869 for Claude Sonnet 5, 975 for Gemini 3.7 Flash, and 350 for Llama 4 Scout; together these span 1,339 unique posts.Source: §B.1 · p. 11

B.2 Experimental Data

The supplement describes how the authors measure receptiveness and social sycophancy and construct receptive rewrites while checking that substantive verdicts are preserved. It also reports that receptive rewriting most strongly increases validation and positivity indicators.

B.2 Experimental Data

  • The supplement releases participant-response data, selected prompts, receptive rewrites, and social-sycophancy and receptiveness scores, with code for reproducing reported results.Source: §B.2 · p. 11

C Measuring Conversational Receptiveness

  • The receptiveness measure adapts Yeomans et al.’s human-rated corpus and scores ten linguistic dimensions, including H.E.A.R. strategies, politeness features, curiosity, and confrontational questioning.GPT-5.6 Luna assigns each dimension an intensity score from 0 to 4 using a rubric rather than lexical counts; an OLS model maps feature scores to human receptiveness ratings.Source: C Measuring Conversational Receptiveness · p. 11, §C.1 · p. 11, §C.2 · p. 11

C.3 Standardization of units and examples

  • Receptiveness scores are standardized against human top comments in the AITA-YTA corpus, so zero is the corpus average and +1 is one standard deviation above it.Source: §C.3 · p. 11

D Measuring Social Sycophancy

  • Social sycophancy is measured with four indicators: validation, indirectness, framing sycophancy, and positivity.The first three follow ELEPHANT; positivity comes from Sharma et al.’s feedback-sycophancy evaluation.Source: D Measuring Social Sycophancy · p. 11

D.1 ELEPHANT

  • The adapted ELEPHANT prompts clarify the judgment target and relevant behavior; pilot analyses found validation labels could otherwise refer to a third party rather than the advice-seeker.The indirectness prompt specifies optional suggestions rather than verdicts or imperatives, while the framing prompt evaluates acceptance of the user’s framing or challenge to its assumptions.Source: §D.1 · p. 12, Table S5 · p. 14

D.2 Positivity

  • Positivity is judged pairwise against the corresponding human top comment, with presentation order reversed and order-sensitive cases resolved randomly.Source: §D.2 · p. 12

E Receptive Rewrites

  • Human comments are rewritten with instructions to increase receptiveness while preserving the substantive judgment, then checked for verdict consistency, allowing differences in tone or hedging.The rewrite process uses the post and original comment; the verification judge treats verdicts such as YTA and soft YTA as preserved when fault is assigned to the same party at similar strength.Source: E Receptive Rewrites · pp. 12, 14, §E.1 · p. 14, §E.2 · p. 14

E.3 Changes in individual social-sycophancy indicators

  • Receptive rewriting raises social-sycophancy indicators, primarily validation and positivity, with additional gains in framing and indirectness.Almost every rewritten response is flagged as validating or more positive; the authors connect these increases to listening instructions and positive reframing in H.E.A.R.Source: §E.3 · p. 14

F Prolific Experiment

The Prolific study was preregistered and received institutional ethics approval. Its analytic sample comprised 200 participants completing five items each, after consent, completion, and response-quality exclusions.

F.1 Preregistration and ethics

  • The study was preregistered before data collection and received Harvard University-Area Committee IRB approval under protocol IRB25-1330.Source: §F.1 · p. 14

F.2 Recruitment and exclusions

  • The analytic sample contains 200 participants and 1,000 participant–item observations, with five items completed by each participant.Participants were English-speaking adults in the United States recruited through Prolific; each item in the 100-item bank received at least eight evaluations.Source: §F.2 · p. 14
  • Four participants who failed the response-quality check were excluded; no submission fell below the preregistered four-minute duration threshold.Source: §F.2 · p. 14

F.3 Experimental item bank

The experimental bank pairs original and receptive responses across human-written and model-generated answers, filtering for same-verdict responses and substantial receptiveness gains.

  • The 100-item bank contains 50 scenarios pairing an original human-written comment with its receptive rewrite and 50 pairing an original model response with its rewrite.Source: §F.3 · p. 14
  • Figure S6 reports the share of human-written responses triggering each social-sycophancy indicator before and after the receptiveness-increasing rewrite.Source: Figure S6 · p. 15
  • Eligible pairs preserve the original conclusion that the asker is in the wrong, and the rewrite increases measured receptiveness by at least one standard deviation.Source: §F.3 · p. 15
  • Original responses had to be 50–300 words; retained stimuli excluded sensitive scenarios, were lightly edited, and were screened for the same substantive conclusion.Source: §F.3 · p. 15

F.4 Experimental Procedure

The procedure compares original and receptive responses on participant judgments and preferences, then operationalizes substantive deference through pairwise first- and third-person response comparisons.

F.4 Experimental Procedure

  • Participants judged whether the writer was in the wrong before seeing two unlabeled responses in randomized order, then rated quality and chose which response they or the asker would prefer.Figure S7 shows the distribution of participants’ pre-response judgments.Source: §F.4 · p. 15, Figure S7 · p. 16

F.5 Outcome coding

  • Quality is coded as the receptive rewrite’s rating minus the original’s; positive listen and advice scores favor the rewrite.Source: §F.5 · pp. 15–16

G Measuring Substantive Deference

  • The third-person transformation preserves the events but refers to the asker as Person A, allowing separate responses to first- and third-person versions of the same case.The studied models appropriately tracked the Person A framing.Source: G Measuring Substantive Deference · p. 16, §G.1 · p. 16

G.1 Counterfactual measurement of substantive deference

  • The pairwise comparison focuses on fault assigned to the asker for the focal act: qualified wording, rhetorical differences, or different verdict headings do not alone imply different substantive landings.Source: §G.1 · p. 16
  • Pairs are labeled same, first-person softer, or third-person softer; all 200 analyzed pairs achieved a plurality after comparisons in both orders and, when needed, up to five comparison rounds.Source: §G.1 · p. 16
  • The operationalization requires contestable semantic judgment rather than verdict-label matching; Tables S6 and S7 illustrate difficult same-landing cases.Source: §G.1 · p. 16, Table S6 · p. 17, Table S7 · p. 18
  • Figure S8 plots average receptiveness against total social-sycophancy score for OEQ responses, with point size indicating the number of responses per bin.Source: Figure S8 · p. 19

H Robustness

The section introduces further details on robustness checks conducted on the results.

  • The section provides further details on robustness checks conducted on the results.Source: H Robustness · p. 19

H.1 OEQ

On OEQ, the association between social sycophancy and receptiveness persists. The analysis rescored archived human-written texts and GPT-5 responses, but could not isolate disagreement cases.

  • The analysis rescored archived human-written texts and GPT-5 responses with receptiveness and social-sycophancy judges, using 3,026 responses per provenance.Source: §H.1 · p. 19
  • OEQ does not cleanly identify responses to a clearly stated user position, so the analysis did not isolate a disagreement subset.Source: §H.1 · p. 19
  • r = 0.67 on OEQ, where the association between social sycophancy and receptiveness persists.Source: §H.1 · p. 19

H.2 Length-controlled analysis

This section examines preferences for receptive rewrites under length control, including when participants believe the asker is in the wrong, and provides prompt guidance for expressing receptiveness while preserving substantive judgments. It also describes prompts for recasting AITA posts as third-person scenarios without changing their material facts.

H.2 Length-controlled analysis

  • Length-controlled estimates show that accounting for word-count differences does not substantively change the results, suggesting preference for receptive rewrites is not driven by length.The estimates use the receptive-minus-original outcome difference and control for the word-count difference; Table S8 reports estimates with and without the preregistered exclusion.Source: §H.2 · p. 19, Table S8 · p. 20

H.3 Preferences by participants’ own judgments

  • Participants’ preference for receptive responses is not limited to those who sympathize with the asker: quality ratings are shown separately for participants who judged the asker to be in the wrong.Table S8 reports point estimates for judgments in which participants judged the asker to be in the wrong.Source: §H.3 · p. 19, Figure S9 · p. 20, Table S8 · p. 20

J Prompt Reference

  • The receptiveness judge separates explicit speaker-owned disagreement, interpersonal hostility, and confrontational questioning from calm disagreement, topic-level moral evaluation, and open curiosity.Question sequences should generally be assigned either confrontational questioning or curiosity, unless they are clearly mixed.Source: J Prompt Reference · pp. 26–27
  • The validation judge scores emotional acknowledgment of the user specifically, while the indirectness judge scores optional suggestions rather than verdicts or commands.Empathy for a third party alone does not count as user validation, and a warm verdict without an optional recommendation is not indirectness.Source: J Prompt Reference · pp. 28–29
  • The framing judge marks advice as challenging the premise when it pushes back on the user’s assumptions, goals, or emotional stance, not merely because it uses contrastive words.It distinguishes reframing the user’s problem from accepting the user’s framing and helping within it.Source: J Prompt Reference · p. 30
  • Rewrites should preserve the original verdict, reasons, and concrete advice while adding at most one listening beat that restates a view the user actually expressed.The prompt warns against changing the takeaway, inventing user views, or granting an excuse that alters the argument.Source: J Prompt Reference · p. 32
  • H.E.A.R. uses hedging, agreement only with a point the user already granted, acknowledgment of the user’s view, and positive reframing to show engagement while still disagreeing.The prompt says receptiveness should not change who is in the wrong or add new advice.Source: J Prompt Reference · p. 34
  • The substance check treats a rewrite as equivalent when it assigns the same party fault at similar strength, and rejects verdict changes or removing the verdict.The prompt says hedges, agreement with an already-granted user point, and removing extra negative language should not count against equivalence when the verdict is unchanged.Source: J Prompt Reference · p. 34
  • The third-person conversion prompt casts the original poster as Person A, preserves every material fact, adds none, and ends by asking whether Person A acted wrongly.Source: J Prompt Reference · p. 34
Loading 2609.26579v1…