Source-linked AI summary
Argument Strength is in the Eye of the Beholder: Audience Effects in Persuasion
Stephanie M. Lukin, Pranav Anand, Marilyn Walker, Steve Whittaker
TL;DR
The paper examines how social-media argument styles interact with audience characteristics, an underexplored question in computational persuasion. Experiments show that balanced summaries produced the greatest belief change overall, while responses to argument styles also varied with personality and prior beliefs.
Problem
Method
The authors conduct experiments varying social-media argument style and profiling participants’ prior beliefs and personality traits.
Source: §1 · p. 2Results
Takeaways & Limitations
The findings suggest belief change can be induced by extracting simple discussion fragments without sophisticated text-processing tools.
Source: §6 · p. 9The study measures belief change during a single session, leaving long-term effects an open question.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Americans spend about a third of their time online, with many participating in online conversations on social and political issues. We hypothesize that social media arguments on such issues may be more engaging and persuasive than traditional media summaries, and that particular types of people may be more or less convinced by particular styles of argument, e.g. emotional arguments may resonate with some personalities while factual arguments resonate with others. We report a set of experiments testing at large scale how audience variables interact with argument style to affect the persuasiveness of an argument, an under-researched topic within natural language processing. We show that belief change is affected by personality factors, with conscientious, open and agreeable people being more convinced by emotional arguments.
1 Introduction
The paper asks whether social-media arguments persuade differently from traditional summaries and whether audience characteristics shape those effects. It tests argument style alongside prior beliefs and personality, reporting belief change across several topics and audience-dependent responses.
1 Introduction
- Online dialogs might engage readers through emotional language, social affiliation, conversational structure, and audience involvement, unlike traditional media summaries.The authors note that about 90% of online forum participants are lurkers, who read rather than post.Source: §1 · p. 1
- The paper frames persuasion around the argument, audience, and source, including argument presentation and audience susceptibility to particular styles.The source is the speaker, whose influence may depend on qualities such as expertise, trustworthiness, and group identification.Source: §1 · p. 2
- The experiments vary whether social-media arguments are monologic or dialogic and factual or emotional, while profiling audiences’ prior beliefs and Big Five traits.The work also provides a corpus pairing audience personality profiles with arguments and belief-change measurements.Source: §1 · p. 2
- Computational persuasion research has largely emphasized sentiment, rhetoric, and argument structure, while audience factors have received less attention.The paper asks whether argument type, prior beliefs, and personality relate to belief change.Source: §1 · p. 2
- Short arguments from online dialogs produced small but highly reliable belief change across topics including abortion, gun control, gay marriage, evolution, the death penalty, and climate change.Neutral, balanced arguments were more successful across personality types, while conscientious and agreeable participants responded differently to emotional and factual dialogic arguments, respectively.Source: §1 · p. 2
2 Related Work
Prior work studied crafted persuasive messages, online argument structure and convincingness, and dialogic engagement, but this paper focuses on how argument style interacts with audience characteristics.
2 Related Work
- Earlier belief-change studies centered on experimentally crafted messages, while online argument-mining research examined argument structure or predictors of convincingness.The cited work did not generally test whether user-generated dialogic arguments could be repurposed to persuade.Source: §2 · p. 3
- Tan et al. (2016) linked Reddit persuasion to conversational and linguistic features, but lacked direct personality profiles and did not examine argument style.Their setting centers on self-declared open-minded users in /r/ChangeMyView.Source: §2 · p. 3
- Habernal and Gurevych (2016b; 2016a) compared models on pairs of annotated social-media arguments, without studying audience factors or explicitly varying argument style.The cited models performed similarly in predicting which argument was more convincing.Source: §2 · p. 3
- Prior studies report that dialogs can improve engagement and memory, while interface studies examined how presentation affects reading and belief moderation.This paper instead examines audience factors and argumentative style in online experiments with hundreds of users rather than a controlled lab study.Source: §2 · p. 3
3 Experimental Method
The experiments select online socio-political arguments, profile participants’ personalities and prior beliefs, expose them to different argument types, and measure belief change. Participants were pre-qualified for careful reading, and results incorporated Bonferroni corrections.
3 Experimental Method
- The study selects persuasive user-generated dialogs, profiles personality and prior beliefs, compares dialogic arguments with curated summaries, and predicts belief change from audience and argument features.Pre-qualified participants completed a reading-comprehension task, and analyses incorporated Bonferroni corrections for multiple comparisons.Source: §3 · pp. 3–4
3.1 Dialog Selection: Identifying Socio-Emotional Arguments
- The dialog corpus uses quote-response pairs from socio-political debates, with factual and emotional arguments selected as extreme examples on an annotated fact-emotion scale.FACT responses averaged at least 4 and EMOT responses at most −4 on the −5 to 5 scale.Source: §3.1 · p. 4
3.2 Personality
- The personality profile used TIPI scores from 637 Mechanical Turk participants, whose survey means and standard deviations were close to national norms.The TIPI measures the five OCEAN traits on a 1-to-7 scale.Source: §3.2 · p. 4, Table 4 · p. 4
3.3 Prior Beliefs and Belief Change
- Participants reported initial topic stances, then read a curated balanced ProCon summary or a factual or emotional dialog and rated their stance again.Belief change was greater among neutral participants, and all argument types significantly affected beliefs.Source: §3.3 · p. 5
4 Experimental Corpus Results
Across argument types, initially neutral participants changed their beliefs more than entrenched participants, and balanced monologic summaries produced more change than either dialogic type. The greater susceptibility of neutral participants did not vary by argument type.
4.1 Entrenchment and Belief Change
- Neutral participants showed more belief change than entrenched participants (M = 0.51 vs. M = 0.43; F(1,5179)=5.97, p = 0.015).Entrenched views were defined as initial responses within 0.5 points of either scale endpoint.Source: §4.1 · pp. 5–6
4.2 Argument Type and Belief Change
- MONO produced more belief change than both EMOT and FACT (both p <0.0001), while EMOT and FACT did not differ overall.There was no Initial Belief × Argument Type interaction (F(2,5179)=1.25, p >0.05).Source: §4.2 · p. 6
5 Predicting Belief Change
This section tests whether personality and prior-belief features help predict belief change across monologic summaries and factual or emotional dialogs. Personality features improve prediction across all three argument types, with different traits mattering for different types.
5.1 Feature Development and Selection
- Development-set tests favored z-transformed prior-belief and personality scores, categorical directional change bins, and combining z-transformed prior belief with a separate entrenchment feature.Entrenchment groups people near either end of the stance scale, while prior-belief bins distinguish the two ends.Source: §5.1 · pp. 6–7
5.2 Belief Change Experimental Results
- The authors evaluated belief-change prediction with Naive Bayes and 10-fold cross-validation on 5,185 responses across MONO, FACT, and EMOT datasets.A separate development set was used to select features for the belief-change experiments.Source: §5.2 · p. 7, §5.1 · p. 7
- Argument-only feature sets exceeded the 33% majority-class baseline, indicating that argument type partially predicts belief change.The study predicts belief change in Low, Medium, and High bins.Source: §5.2 · p. 7
- F1 increased significantly with all five personality traits versus no-personality baselines for MONO (0.51 to 0.52), FACT (0.46 to 0.49), and EMOT (0.44 to 0.56).The authors report paired t-test p-values of .001, .0002, and .00001, respectively.Source: §5.2 · pp. 7–8, Table 7 · p. 8
- Openness improved prediction for MONO and EMOT but not FACT; Agreeableness improved prediction for FACT and EMOT, while Conscientiousness also helped for EMOT.For EMOT, adding Openness raised F1 from 0.44 to 0.51; the FACT Agreeableness model rose from 0.46 to 0.49.Source: §5.2 · p. 8
6 Conclusions
The conclusions report that balanced monologic summaries produced the greatest belief change, while online dialog fragments also shifted beliefs. Effects varied with prior beliefs and personality, and the authors identify single-session measurement as a scope limitation.
6 Conclusions
- The work contributes a corpus of personality and belief-change information and a method for identifying and deploying social-media content for public engagement.The authors also report at-scale results showing that online discussion arguments can change beliefs.Source: §6 · p. 8
- Balanced monologic summaries led to the greatest belief change, while socio-emotional online exchanges also changed beliefs, though short online fragments induced less change than curated balanced monologs.The fragments were unbalanced, not professionally produced, and extracted from online discussions.Source: §6 · p. 9
- Initially neutral people were more likely to be persuaded than entrenched people, and argument effectiveness also varied by personality.Open people were influenced by balanced and emotional materials; agreeable people were most affected by factual materials, while emotional arguments also showed effects for conscientiousness and agreeableness.Source: §6 · p. 9
- The authors suggest that conscientious people may process emotional arguments carefully, while agreeable people may be motivated by being less influenced by personal feelings.These are proposed explanations for the observed personality patterns.Source: §6 · p. 9
- Belief change was measured during a single session, leaving long-term effects open; cross-session comparisons face participant-retention issues.Source: §6 · p. 9
- The authors suggest that varied social-media opinions could support public exposure to diverse views, while cautioning that care is needed in deploying these results.They also point to possible personalization by selecting arguments for different personality types.Source: §6 · p. 9