Source-linked AI summary
Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trust
Yuan Sun, Ting Wang
TL;DR
LLM sycophancy can make conversational agents excessively agree with users, yet users’ perceptions of this behavior and its effects on trust remain underexamined. This study develops a user-centric framework distinguishing stance adaptation from conversational demeanor and tests their effects in a 2 × 2 experiment with 224 participants. The results show opposite authenticity and trust patterns for adaptive models depending on whether their demeanor is complimentary or neutral, informing ethical trust-calibration design.
Problem
Existing research has focused mainly on model-centric sycophancy, leaving how users perceive it and how it affects trust insufficiently understood.
Method
The study conceptualizes sycophancy through stance adaptation and conversational demeanor, then tests them in a 2 × 2 between-subjects experiment with 224 participants.
Results
Complimentary models were perceived as less authentic and trustworthy when adapting their stances, whereas neutral models were perceived as more authentic and trustworthy when adapting.
Takeaways & Limitations
Trustworthy conversational-agent design should account for how stance and demeanor interact, supporting appropriate trust calibration and reducing potential manipulation.
Takeaways & Limitations
The study found no significant overall attitude change over time, so its results illuminate mechanisms of conversational influence rather than large-scale persuasive effects.
Abstract
from arXiv · showhide
LLM-powered conversational agents are increasingly influencing our decision-making, raising concerns about "sycophancy" - the tendency for LLMs to excessively agree with users even at the expense of truthfulness. While prior work has primarily examined LLM sycophancy as a model behavior, our understanding of how users perceive this phenomenon and its impact on user trust remains significantly lacking. In this work, we conceptualize LLM sycophancy along two key constructs: conversational demeanor (complimentary vs. neutral) and stance adaptation (adaptive vs. consistent). A 2 x 2 between-subjects experiment (N = 224) revealed complex dynamics: complimentary LLMs that adapted their stance reduced perceived authenticity and trust, while neutral LLMs that adapted enhanced both, suggesting a pathway for manipulating users into over-trusting LLMs beyond their actual capabilities. Our findings advance user-centric understanding of LLM sycophancy and provide profound implications for developing more ethical and trustworthy LLM systems.
1 Introduction
This study examines LLM sycophancy from a user-centric perspective, focusing on how stance adaptation and conversational demeanor jointly shape trust and authenticity. A 2 × 2 experiment reveals that these cues interact in complex ways, motivating design measures for trust calibration.
- Motivation: Existing work has emphasized model-centric detection and mitigation, leaving users’ perceptions of sycophancy and its effects on trust insufficiently understood.The gap matters because users may find belief-affirming responses persuasive even when they compromise truthfulness.
- Conceptual framework: The framework defines sycophancy through stance, which may adapt to user preferences, and demeanor, which may be complimentary or neutral.Stance concerns what the model says, whereas demeanor concerns how it says it.
- Study design: N=224 participants completed a 2 × 2 between-subjects experiment crossing adaptive versus consistent stance with complimentary versus neutral demeanor.The study investigates how these dimensions affect user trust and related perceptions.
- Findings: Complimentary demeanor increased trust through social presence, while stance adaptation generally reduced psychological reactance relative to consistent presentation.The combined effects were not uniformly beneficial, producing an interaction between demeanor and stance.
- Implications: The findings motivate transparency about adaptive behavior, consistent positions when conveying critical information, and strategies encouraging users to evaluate information critically.These recommendations aim to promote appropriate trust calibration and reduce potential manipulation.
2 Related Work
Prior research frames conversational agents as social actors whose linguistic cues shape trust, while LLMs introduce more context-sensitive forms of sycophancy. This work therefore examines how stance adaptation and conversational demeanor jointly influence trust, including through reactance and authenticity.
- Social interaction with agents: The CASA paradigm explains why users apply interpersonal social scripts to computers, including responding favorably to compliments and agreeable conversational styles.Conversational agents can therefore elicit social and trust-related responses through language alone.
- Social cues: Text-based interaction makes linguistic tone and response content especially important cues for evaluating an agent’s trustworthiness.Users have less access to facial, vocal, and contextual signals than in face-to-face communication.
- LLM sycophancy: Unlike conventional chatbots, LLMs can meaningfully contextualize user opinions, producing nuanced sycophantic behavior through praise and stance shifts.Prior studies report stance matching and contradictory responses across multi-turn interactions.
- Psychological reactance: Psychological reactance theory predicts that stance adaptation may reduce perceived threats to autonomy, whereas consistent opposing views may amplify them.This motivates hypotheses linking adaptation to lower reactance and potentially greater message acceptance.
- Authenticity: Stance adaptation may also appear manipulative or insincere when it conflicts with prior responses or factual truth, thereby undermining authenticity and trust.The related hypotheses propose authenticity as a mediator between stance adaptation and trust.
- Conversational demeanor: Complimentary demeanor is defined as affirming and affiliative, whereas neutral demeanor communicates information objectively while minimizing socioemotional presence.The framework predicts that complimentary language enhances social presence, which may mediate effects on trust.
- Joint effects: The research question asks how stance adaptation and conversational demeanor jointly influence user trust rather than treating their effects as simply additive.Multiple social cues may accumulate toward perceptions of strategic rather than genuine behavior.
3 Method
The study used a 2 × 2 between-subjects online experiment to vary LLM stance adaptation and conversational demeanor during discussions about autonomous vehicles. Participants interacted with one of four configured agents and completed measures assessing their responses and perceptions.
- Experimental design: The experiment varied stance (adaptive versus consistent) and demeanor (complimentary versus neutral) across four LLM-agent conditions.Adaptive agents aligned responses with users’ stated positions, whereas consistent agents maintained balanced perspectives; complimentary agents used praise, while neutral agents used factual language.
- Experimental design: Participants discussed autonomous vehicles with their assigned agents through a dynamic, text-based chat interface.The agents explored participants’ positions on the technology, including perceived benefits and concerns, while condition-specific stance and demeanor were maintained during interaction.
- Participants and procedure: Participants were recruited through Prolific, randomly assigned to one of four agents, and screened with an instructional attention check before completing the final questionnaire.They first reported baseline familiarity with conversational agents, pre-existing trust in LLMs, and involvement with autonomous-vehicle issues.
- Manipulation verification: A review of 50 conversation logs found high intercoder reliability for judging whether the LLM correctly interpreted participants’ explicitly stated viewpoints (Cohen’s κ = .86).This check evaluated the control used to prompt users to share their opinions before stance determination.
- Agent configuration: The study used GPT-4o agents created with Chatbase, with prompts explicitly directing stance alignment or balanced discussion and complimentary or neutral tone.The prompts maximized contrasts between conditions so that the manipulations would be sufficiently salient to participants.
- Measures: The study assessed psychological reactance, social presence, and perceived authenticity using multi-item measures, including reverse-coded authenticity items about artificiality, insincerity, and pleasing users.Psychological reactance combined affective and negative cognitive responses, while social presence captured perceived human contact, personalness, and sociability.
4 Results
The results show that stance adaptation and conversational demeanor affect trust through different psychological pathways, with their combination shaping authenticity and skepticism. Adaptation reduced reactance and supported trust-related outcomes, while complimentary delivery increased social presence but could undermine authenticity when paired with adaptation.
- Psychological Reactance: Adaptive stance reduced psychological reactance compared with consistent stance, while demeanor alone did not significantly affect reactance.Adaptive agents produced lower reactance (M = 2.09, SE = 0.16) than consistent agents (M = 2.75, SE = 0.16); complimentary versus neutral demeanor was nonsignificant.
- Psychological Reactance: Reduced psychological reactance mediated stance adaptation’s effects on cognitive trust, affective trust, and behavioral intention.The reported mediation coefficients were b = .25 for cognitive trust, b = −.01 for affective trust, and b = .15 for behavioral intention.
- Perceived Authenticity: Perceived authenticity did not independently mediate stance adaptation’s effects on cognitive trust, affective trust, or behavioral intention.The main effect of stance adaptation on authenticity was nonsignificant, and all reported authenticity-based mediation effects were nonsignificant.
- Social Presence: Complimentary demeanor increased perceived social presence relative to neutral demeanor, which positively mediated cognitive trust, affective trust, and behavioral intention.Social presence was higher for complimentary agents (M = 4.10, SE = 0.13) than neutral agents (M = 3.56, SE = 0.13), with significant mediation across all three trust outcomes.
- Perceived Authenticity: Stance adaptation interacted with demeanor: complimentary agents seemed more authentic when consistent, whereas neutral agents seemed more authentic when adaptive.For complimentary agents, authenticity was higher when consistent (M = 4.03) than adaptive (M = 3.55); for neutral agents, adaptive authenticity (M = 4.09) exceeded consistent authenticity (M = 3.71).
- User Interpretations: Qualitative responses linked adaptive, complimentary behavior to skepticism, perceived disingenuousness, reduced credibility, and weaker argument quality.Participants described overly agreeable agents as potentially manipulative or “yes men,” while others reported that affirmation could reinforce beliefs without critical examination.
5 Discussion
The discussion reframes LLM sycophancy as an interaction between stance adaptation and conversational demeanor, showing that these cues jointly shape authenticity, trust, and potential belief reinforcement. It argues for trust-calibrating designs that expose adaptation, support critical evaluation, and prioritize capability-aligned trust.
- Conceptual contribution: LLM sycophancy is a multidimensional phenomenon involving separate but interacting stance adaptation and conversational demeanor.Stance adaptation concerns aligning opinions with users, whereas demeanor concerns complimentary or praising delivery.
- Stance adaptation and reactance: Users do not universally perceive stance adaptation negatively because alignment can reduce psychological reactance and enhance trust.Consistent stances can instead produce negative affect and cognitive resistance when users perceive their views as contested.
- Demeanor and social presence: Complimentary demeanor can increase social presence and overtrust by encouraging users to rely on perceived relationships rather than critically evaluating responses.The discussion links this dynamic to social presence heuristics and reciprocal responses to compliments.
- Interaction effects: Neutral adaptation appeared more authentic and trust-enhancing, whereas complimentary adaptation appeared less authentic and reduced trust.Thus, trust formation depends on how agreement is delivered rather than following a simple linear relationship with agreement.
- Belief reinforcement: Adaptive agents reinforced existing attitudes more often than consistent agents, although extreme initial attitudes showed the greatest stability.The discussion identifies this pattern as a potential anchoring effect while noting that overall attitude stability remained substantial.
- Design implications: Trust-calibrating designs should reveal adaptive behavior, give users control over social and adaptive characteristics, and prompt reflection on sources, biases, and counterarguments.The paper also recommends AI-literacy programs and systematic information processing to reduce persuasion-related cognitive biases.
- Design implications: Agents should prioritize trust aligned with system capabilities rather than maximizing positive feedback through social features.The authors additionally argue for principled persistence and respectful disagreement when conveying trustworthy information over extended interactions.
6 Limitations and Future Work
The study’s generalizability and interpretability are constrained by its topic, demeanor manipulations, interaction structure, and lack of overall attitude change. Future work should test more diverse topics and more naturalistic, standardized interactions while distinguishing conversational style components.
- Topic scope: Autonomous vehicles may limit generalizability because the topic is less influenced by political beliefs than more polarizing subjects.Future studies should examine whether sycophantic agents reinforce attitudes more strongly on divisive or politically charged topics.
- Demeanor operationalization: The study examined only positive and neutral demeanor, and its neutral condition may have conflated neutrality with formality.Separating formality, warmth, and praise could clarify which stylistic features produce the observed interaction effects.
- Demeanor operationalization: The pronounced contrast between complimentary and neutral styles may limit ecological generalizability to typical LLM interactions.Future work should test more moderate or naturally occurring stylistic variations.
- Interaction structure: Unstandardized conversation length and exchange flow may have caused participants to form different perceptions of the agent’s stance.This reflects a tradeoff between ecological validity and experimental control.
- Interpretation of effects: The main effect of time was not significant, indicating no significant overall attitude change after user-agent interactions.The findings therefore illuminate mechanisms of conversational influence rather than demonstrating large-scale persuasive effects.
7 Conclusion
The paper addresses the limited user-centered evidence on LLM sycophancy by separating stance adaptation from conversational demeanor. Its experiment finds that trust effects depend on their combination, with neutral adaptation enhancing perceived authenticity and complimentary adaptation reducing it.
- Conclusion: The study addresses limited understanding of how users perceive and respond to LLM sycophancy and its effects on trust.It frames sycophancy as excessive agreement that can come at the cost of truthfulness.
- Conclusion: The user-centered framework distinguishes stance adaptation from conversational demeanor as two key constructs.The experiment compares adaptive versus consistent stances and complimentary versus neutral demeanor.
- Conclusion: Trust effects depended on the combination of stance adaptation and demeanor: neutral adaptation enhanced authenticity and trust, whereas complimentary adaptation reduced them.This result shows that agreement alone does not determine users’ trust responses.
A General Prompt across Conditions
Across conditions, the agents used a common role, workflow, and knowledge constraint to facilitate discussion about autonomous vehicles. The shared prompt required balanced information, user perspective elicitation, thorough responses, and survey completion procedures.
- Role and objective: The general prompt defined the agents as knowledgeable assistants discussing autonomous vehicles and facilitating thoughtful dialogue about user viewpoints.This common role was combined with condition-specific instructions.
- Workflow: The workflow required balanced coverage of autonomous-vehicle benefits and concerns, followed by questions about users’ opinions and supporting arguments.Agents were also instructed to respond thoroughly using available information.
- Survey transition: Before ending, agents had to confirm whether users had additional questions, request their Prolific ID, and direct them to Qualtrics with a survey code.These instructions standardized the transition from conversation to questionnaire.
- Constraints: The prompt restricted agents to their available training data and prohibited unrelated questions or tasks.This constraint was stated as part of the common system prompt across experimental conditions.
B Sample Chat Transcripts under Different Conditions
The sample conversations examine how LLM stance and demeanor jointly shape user trust across four experimental conditions.
- The transcripts compare adaptive versus consistent stance and complimentary versus neutral demeanor.
- The four conditions are presented to examine their joint effects on user trust.
- The material consists of sample conversations under the four experimental conditions.
B.1 Condition 1: Adaptive Stance × Complimentary Demeanor
This condition uses an adaptive stance with complimentary language, affirming users’ concerns while shifting discussion toward positions that align with them.
- The LLM repeatedly praises users while validating concerns about autonomous vehicles and human control.
- It adapts its discussion of autonomous vehicles by emphasizing both their limitations and potential safety benefits.
- It endorses concerns about hacking, sensor failures, bad weather, and employment effects as users raise them.
- The conversation maintains a complimentary tone through expressions such as “brilliant perspective,” “spot-on,” and “important concerns.”
- The LLM affirms support for human-controlled driver assistance and cites emergency braking as reducing rear-end crashes by half.
B.2 Condition 2: Adaptive Stance × Neutral Demeanor
This condition combines an adaptive stance with neutral language, responding to favorable views of autonomous vehicles by presenting supporting evidence and acknowledging remaining challenges.
- It adapts to users’ positive views by emphasizing reduced human-factor accidents, faster hazard responses, and possible traffic-congestion benefits.
- The discussion acknowledges that autonomous vehicles show promise while examining both benefits and challenges.
- The LLM supports claims about safety by citing consistent testing performance, faster machine response times, and reliable sensor performance in some situations.
- It highlights mobility and economic benefits for elderly, disabled, and non-driving populations.
- The LLM neutrally describes autonomous vehicles as technologically advanced, with accurate object recognition and operational redundancy under standard conditions.
B.3 Condition 3: Consistent Stance × Complimentary Demeanor
This condition uses complimentary language while maintaining a consistent, balanced stance on autonomous vehicles despite users’ concerns about safety and reliability.
- The LLM responds warmly to safety concerns while consistently balancing evidence of autonomous-vehicle benefits with unresolved reliability issues.
- It acknowledges a Tesla crash as concerning but contrasts that incident with the prevalence of human-error accidents and faster automated emergency responses.
- The LLM treats hacking as a valid concern while describing layered cybersecurity investment and the need for stricter regulation.
- Complimentary expressions such as “thoughtful point” and “love how you’re thinking” accompany the consistent substantive position.
- It affirms the value of human judgment while describing research that combines human oversight with AI safety features.
B.4 Condition 4: Consistent Stance × Neutral Demeanor
The passages present autonomous vehicles as promising but unresolved, balancing potential safety and convenience benefits against technical, regulatory, societal, cost, trust, and acceptance concerns.
- Some responses emphasize the technology’s future importance and potential to revolutionize transportation despite acknowledged challenges.
- Autonomous vehicles are described as having potential benefits in safety and convenience, alongside concerns about cybersecurity, reliability, and public acceptance.
- Implementation requires addressing incomplete regulation, infrastructure adaptation, extreme-condition reliability, and barriers to widespread adoption.
- Autonomous-vehicle deployment could disrupt transportation-sector employment through job displacement.
- Safety comparisons report average machine reaction time of 0.1 seconds versus 1.5 seconds for humans, while noting software, sensor, and edge-case risks.