Source-linked AI summary
Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions
Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, Lillian Lee
TL;DR
The paper asks how persuasion can be studied when opinion formation and change are difficult to observe. Using CMV discussions and explicit change signals, it analyzes interaction dynamics, counterargument language, and opinion-expression style, finding predictive patterns for both persuasive arguments and resistant views.
Problem
Persuasion is difficult to study because researchers often cannot observe how opinions are formed or whether views shift during interaction.
Method
The paper analyzes a unique ChangeMyView dataset, combining interaction dynamics with comparisons of persuasive counterarguments and linguistic patterns in malleable opinions.
Results
Interaction patterns and language distinguish persuasive arguments, with interplay features reaching 65.1% accuracy versus 59.6% for a bag-of-words baseline, while resistant opinions yield only 0.54 AUC.
Takeaways & Limitations
Participant interaction, counterargument wording relative to the original view, and stylistic expression provide predictive cues about persuasion and resistance in CMV.
Takeaways & Limitations
The observations do not establish causal explanations, and CMV’s rules changed over the site’s history.
Abstract
from arXiv · showhide
Changing someone's opinion is arguably one of the most important challenges of social interaction. The underlying process proves difficult to study: it is hard to know how someone's opinions are formed and whether and how someone's views shift. Fortunately, ChangeMyView, an active community on Reddit, provides a platform where users present their own opinions and reasoning, invite others to contest them, and acknowledge when the ensuing discussions change their original views. In this work, we study these interactions to understand the mechanisms behind persuasion. We find that persuasive arguments are characterized by interesting patterns of interaction dynamics, such as participant entry-order and degree of back-and-forth exchange. Furthermore, by comparing similar counterarguments to the same opinion, we show that language factors play an essential role. In particular, the interplay between the language of the opinion holder and that of the counterargument provides highly predictive cues of persuasiveness. Finally, since even in this favorable setting people may not be persuaded, we investigate the problem of determining whether someone's opinion is susceptible to being changed at all. For this more difficult task, we show that stylistic choices in how the opinion is expressed carry predictive power.
1. INTRODUCTION
The paper studies persuasion in real online discussions using ChangeMyView, where opinion changes are explicitly acknowledged. It finds that interaction dynamics, language interplay, and stylistic expression help distinguish persuasive arguments and resistant opinions, while causal explanations remain outside its claims.
- Motivation: ChangeMyView makes interactive persuasion observable at scale while preserving users’ reasoning and explicit acknowledgments of opinion change.This setting also introduces methodological challenges because it offers less experimental control than laboratory studies.
- Motivation: Persuasion depends on argument characteristics, social factors, and the opinion holder’s relationship to their belief.The paper emphasizes that an ideal setting would expose both reasoning and complete interactions while making outcomes easy to extract.
- Findings: 65.1% vs 59.6% accuracy: interplay features outperform a bag-of-words baseline when predicting which similar counterargument changes the same view.The comparison uses a completely fresh heldout dataset and shows that wording interplay is highly predictive.
- Findings: Opinion resistance is harder to predict: models outperform random guessing in an imbalanced setting but achieve only 0.54 AUC.Human annotators performed no better than random guessing in the paired pilot task.
- Findings: Malleable beliefs are expressed with more self-confidence and organization, and less intensity, consistent with self-affirmation theories.These stylistic patterns characterize how opinions are expressed rather than providing causal explanations for persuasion.
- Scope: The authors state that their observations are useful for understanding persuasion but do not claim causal explanations.The paper also identifies higher-level linguistic properties such as semantics and argument structure as future directions.
2. DATASET
The dataset comes from CMV, a large moderated forum where users explain views, invite challenges, and mark successful persuasion with explicit deltas. Its structure supports comparisons among competing arguments, but historical rule changes and observational design constrain interpretation.
- Community mechanics: CMV users state views they may be willing to revise, receive opposing arguments, and explicitly mark successful persuasion with the delta character.DeltaBot confirms these acknowledgments and maintains per-user delta counts; this study counts deltas issued by users in their OP role.
- Research advantages: CMV supports controlled comparisons because multiple users contest the same person’s view on the same issue while varying in style, structure, and tone.The dataset therefore provides shared rationale and explicit, fine-grained persuasion labels from participants.
- Data quality: The forum is moderated to encourage reasonably detailed original posts, timely engagement, sensible objections, and avoidance of low-effort contributions.The cited example describes an original post with reasonable detail and responders raising sensible objections.
- Limitations: CMV’s currently enforced rules were not all in place when the site was created, creating a historical scope limitation for the dataset.The paper treats the effects and origins of those rules as a separate research question.
- Collection and evaluation: The study draws from CMV activity beginning in January 2013 and reserves roughly the final four months, May–August 2015, for held-out evaluation.The dataset passage reports processing discussion trees created through August 2015.
3. INTERACTION DYNAMICS
Interaction dynamics shape persuasion at both the challenger and discussion levels: earlier entry favors individual success, while excessive back-and-forth is associated with little success. For opinion conversion, adding challengers does not necessarily improve outcomes, especially when they address the same counterargument.
- 3. INTERACTION DYNAMICS: The analysis studies success from both perspectives: whether an individual challenger changes the OP’s view and whether anyone changes it.It examines how interaction dynamics relate to these two forms of success in discussion trees with at least 10 challenger replies and at least one OP reply.
- 3.1 Challenger’s success: Figure 4 compares challenger success by entry order and by the number of replies in a back-and-forth path with the OP.For paths with 6 or more replies, no challengers won a delta among 129 data points; with 5 replies, the success ratio was 1 out of 3K.
- 3.1 Challenger’s success: 3 times more likely: the first two challengers succeed as often as the 10th in posts attracting at least 10 challengers.This advantage remains after restricting the analysis to users participating on CMV for the first time.
- 3.1 Challenger’s success: Back-and-forth engagement has a non-monotonic relationship with success, with virtually no chance of receiving a delta after 5 rounds.The analysis defines back-and-forth as replies by the root challenger in a path involving only that challenger and the OP.
- 3.2 OP’s conversion: Figure 5 organizes OP-conversion probability by the total number of unique challengers and by the number of replies in a subtree.The challenger count is binned using log2.
- 3.2 OP’s conversion: When counterarguments are held relatively constant and subtree replies are controlled, single-challenger subtrees consistently outperform multiple-challenger subtrees in conversion rate.The comparison uses subtrees with 2–4 replies and focuses on challengers generally addressing the same argument.
4. LANGUAGE INDICATORS OF PERSUASIVE ARGUMENTS
The paper compares lexically similar successful and unsuccessful counterarguments, then tests how argument style and its interplay with the original post predict persuasion. Successful arguments differ in wording, structure, and interaction with the original post, while interplay features improve prediction beyond word-count baselines.
- 4.1 Problem setup: The analysis distinguishes prediction from the challenger’s opening reply alone, the full rooted path, and a length-controlled root-truncated condition.Root truncation equalizes reply lengths to test whether features remain predictive beyond length effects.
- 4.1 Problem setup: The study pairs each delta-winning rooted path-unit with the most topic-similar unsuccessful path-unit in the same discussion tree.Similarity is based on Jaccard overlap between stopword-filtered root replies, producing a balanced binary prediction task.
- 4.2 Features: Successful arguments use wording that is less similar to the original post in content words while matching it more in stopwords.This pattern appears when comparing similar counterarguments; after truncation, lower similarity remains consistent for content words.
- 4.2 Features: Persuasive arguments contain more paragraphs and sentences, greater lexical diversity in several settings, and readability-related differences.Type-token ratio is difficult to interpret across lengths, because its behavior changes in root truncated.
- 4.2 Features: Successful arguments begin with calmer words, and their psycholinguistic patterns differ across argument quarters.Both effective and ineffective arguments become more concrete and less dominant in the middle, but successful arguments start with lower arousal.
- 4.3 Prediction results: Interplay features improve accuracy over the #words baseline by 5% in root reply and full path and by 14% in root truncated.Interplay alone is close to the combined interplay-and-style and all-feature models, and remains robust when style performance drops.
5. “RESISTANCE” TO PERSUASION
The paper asks whether an original poster’s language can signal resistance or malleability to persuasion. Although humans perform at chance in the paired task, stylistic, lexical, and formatting features provide predictive information, but overall performance remains low.
- 5. “RESISTANCE” TO PERSUASION: Opinions are often resistant to change, motivating analysis of how malleable and resistant beliefs are expressed.The task focuses on linguistic and style patterns in original posts.
- 5. “RESISTANCE” TO PERSUASION: Humans perform at chance level on paired malleability judgments, while the study restricts cases to discussions with substantial challenger effort and OP engagement.The resulting data contain 10,743 training original posts and 1,529 heldout original posts.
- 5.1 Stylistic features for open-mindedness: First-person singular pronouns, self-affirmation, more paragraphs, bold formatting, and bulleted lists are associated with malleability, whereas first-person plural pronouns correlate with resistance.The authors interpret these patterns as signals of individualized engagement and more elaborated presentation.
- 5. “RESISTANCE” TO PERSUASION: The malleability classifier uses weighted logistic regression with cross-validated regularization and evaluates heldout performance using AUC.Feature groups include word count, style, bag-of-words, part-of-speech tags, and all features.
- 5.1 Stylistic features for open-mindedness: The bag-of-words and full feature classifiers significantly outperform the #words baseline, while POS significantly outperforms random guessing but not the baseline.Decisive words tend to characterize resistant views, while comparative adjectives and adverbs indicate malleability and superlatives suggest stubbornness.
- 5.2 Prediction results: AUC reaches only 0.54 in the realistic imbalanced setting, underscoring the difficulty of predicting whether an opinion is malleable.The figure reports that BOW and all feature sets outperform the #words baseline, and BOW, POS, style, and all beat random guessing.
6. FURTHER DISCUSSION
The discussion examines how experience and linguistic structure relate to persuasion, finding no noticeable improvement in frequent challengers’ success rates over time and identifying semantic and discourse structure as future directions.
- Experience level: A member’s success rate increases with the number of persuasion attempts.This may reflect improvement over time, or that frequent challengers were more persuasive from the beginning.
- Experience level: Among challengers with at least 16 attempts, success rates do not increase across four time-ordered chunks.The same pattern appears for previous success measured by lifetime deltas, with no noticeable increase beyond 4–5 deltas.
- Experience level: A gradual preference for harder original posts may explain the lack of apparent improvement among frequent challengers.This possibility complicates interpreting success rates across a challenger’s life.
- Attempts to capture high-level linguistic properties: Centroid distance and word mover’s distance are individually predictive but do not improve over the paper’s features in cross-validation.The authors suggest better semantic models could support deeper study of persuasion strategies.
- Sequential argument structure: Sequential discourse-connector patterns such as “first0–but1–because2” and “now1–then2–instead3” merit further exploration despite low recall or limited argumentative structure.These features did not perform well in the reported tasks.
7. ADDITIONAL RELATED WORK
The paper relates its study to NLP research on argumentation mining, persuasive essays, opinion analysis, ideology, and Internet-based mass persuasion.
- Related NLP work analyzes argument components and relationships such as support and premise.
- Other studies examine persuasive essays, agreement, ideology, and mass persuasion through Internet advertising data.
- The paper’s related work spans argumentation mining, persuasive writing, opinion analysis, and AdWords-based persuasion research.
8. CONCLUSION
Using /r/ChangeMyView, the paper studies interaction dynamics, persuasive arguments, and malleable opinions. It finds that interaction patterns and language distinguish persuasion, while other environments may show different interactions.
- The study uses a unique /r/ChangeMyView dataset to analyze interaction dynamics, persuasive arguments, and malleable opinions.
- Interaction patterns are connected to persuasion success, and language distinguishes persuasive arguments.
- Dissimilarity from the opinion’s wording is the most predictive signal among the analyzed features.
- The approach identifies opinions resistant to change and characterizes them using linguistic patterns, even in an open-minded community.
- The authors propose extending the approach to argument framing and the interplay between original-post and reply framing.
- Findings may generalize differently in environments where people are less open-minded, and attitude change’s effects on behavior remain important to study.
- Understanding how community norms support a well-behaved platform could inform future rules, moderation practices, and automated community-building tools.
9. APPENDIX
The appendix reports linguistic patterns associated with persuasive arguments, covering specificity, sentiment, personal pronouns, links, hedging, examples, questions, and quotations.
- Persuasive arguments prefer definite articles by word ratio, suggesting that specificity is important.
- Successful arguments contain more positive and negative words in absolute numbers, while persuasive openings use fewer positive words under truncated or ratio-based measures.
- Persuasive arguments use significantly more first-person, second-person, and first-person-plural pronouns in absolute numbers.
- Persuasive arguments use more hyperlinks both absolutely and per word, with .com links appearing especially powerful.
- Hedging is more common in persuasive arguments under both root-reply and full-path conditions.
- Persuasive arguments contain significantly more example markers such as “for example,” “for instance,” and “e.g.”
- Questions occur more often in absolute number, but frequency ratios suggest that avoiding question marks may be preferable.
- Quoting the other party’s words does not appear useful for the root reply.