Source-linked AI summary
Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good
Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, Zhou Yu
TL;DR
Persuasive dialogue systems lack integration between computational dialogue research and social-scientific persuasion research, while personalized persuasion remains underexplored. This paper constructs a human-human donation dataset, models persuasion strategies, and examines psychological backgrounds; its classifier reaches 74.8% accuracy and 59.6% F1, supporting future personalized system design.
Problem
Persuasive agents remain underdeveloped because computational dialogue research and social-scientific persuasion research lack synergy, and personalized strategy effectiveness remains underexplored.
Method
The paper collects 1,017 donation-persuasion conversations, surveys psychological profiles, annotates strategies, and classifies 10 strategies using dialogue context and sentence-level features.
Results
74.8% accuracy and 59.6% F1 were achieved by the hybrid RCNN using sentence embedding, context embedding, turn position, sentiment, and character features.
Takeaways & Limitations
The dataset, strategy analyses, and psychological findings provide foundation elements for designing personalized persuasive dialogue systems.
Takeaways & Limitations
Deployment requires disclosure of the persuasive agent’s identity and data practices, active consent, and continuous monitoring for appropriate, nondiscriminative responses.
Abstract
from arXiv · showhide
Developing intelligent persuasive conversational agents to change people's opinions and actions for social good is the frontier in advancing the ethical development of automated dialogue systems. To do so, the first step is to understand the intricate organization of strategic disclosures and appeals employed in human persuasion conversations. We designed an online persuasion task where one participant was asked to persuade the other to donate to a specific charity. We collected a large dataset with 1,017 dialogues and annotated emerging persuasion strategies from a subset. Based on the annotation, we built a baseline classifier with context information and sentence-level features to predict the 10 persuasion strategies used in the corpus. Furthermore, to develop an understanding of personalized persuasion processes, we analyzed the relationships between individuals' demographic and psychological backgrounds including personality, morality, value systems, and their willingness for donation. Then, we analyzed which types of persuasion strategies led to a greater amount of donation depending on the individuals' personal backgrounds. This work lays the ground for developing a personalized persuasive dialogue system.
1 Introduction
The paper addresses gaps in automated personalized persuasion by studying how personal information and persuasion strategies relate to outcomes. It establishes a foundation through human persuasion data, strategy annotation, classification, and analysis of background-dependent strategy effectiveness.
- Personalized persuasion research had not established how personal information affects persuasion outcomes or which strategies work best for different user backgrounds.
- Automated persuasive agents lagged because computational dialogue research and social-scientific persuasion research lacked synergy.
- The study collected 1,017 human-human persuasion conversations involving real participant incentives and annotated persuasion strategies in a subset.
- The authors classified 10 persuasion strategies using sentence-level features and dialogue context, then examined psychological backgrounds, donation behavior, and background-dependent strategy effectiveness.
- The resulting insights were intended to support future personalized persuasive dialogue-system design.
2 Related Work
Prior work established foundations in personalized persuasion, persuasion detection, personalized dialogue, negotiation, and applied persuasive systems, but left important personalization gaps. In particular, existing persona descriptions lacked quantitative sociodemographic and psychological detail for analyzing interactions between users and dialogue policies.
- The Elaboration Likelihood Model motivates personalized persuasion by linking message engagement to individuals’ motivation, ability, and ways of processing information.
- Persuasion research has developed annotation and prediction methods for claims, premises, persuasion strategies, and persuasiveness in online forums and social media.
- Personalized dialogue systems have targeted user engagement and adaptation using user information such as sentiment.
- Existing persona descriptions lacked quantitative sociodemographic and psychological characteristics needed to analyze interactions between personalities and dialogue-policy preferences.
- Negotiation systems and persuasive dialogue systems pursue different goals: negotiation seeks mutual agreement, whereas persuasion seeks to change one person’s attitude or decision.
- Earlier persuasive dialogue systems often overlooked personalized design and deep learning, while one study linked personality factors to belief change from emotional arguments.
3 Data Collection
The authors created PERSUASIONFORGOOD through an online charity-donation task with surveys, assigned conversational roles, private donation reports, and post-task measures. The resulting dataset captured human persuasion conversations, including longer average persuader utterances than persuadee utterances.
- The online task paired participants in human-human persuasion conversations about donating to Save the Children.
- The study used pre-task, persuasion-dialogue, donation-confirmation, and post-task survey stages.
- Participants’ pre-task survey produced a 23-dimension psychological feature vector covering personality, moral foundations, values, and decision-making style.
- Randomly assigned persuader and persuadee roles helped reduce correlation between persuasion strategies and the targeted persuadee’s characteristics.
- Donation amounts were entered privately after the conversation, with the task payment as the maximum, followed by sociodemographic and engagement questions.
- 22.96 tokens versus 15.65 tokens: persuaders’ average utterances were longer than persuadees’ in PERSUASIONFORGOOD.
4 Annotation
The study annotated persuasion conversations with ten strategy categories spanning persuasive appeals and inquiries, then examined how these strategies are distributed across dialogue turns.
- Annotation Procedure: Each complete sentence was annotated rather than each conversation turn because many turns contained multiple sentences with different semantic meanings.The annotation choice addressed the complexity of typed conversations and their multi-sentence turns.
- Strategy Categories: The annotation scheme identified 10 persuasion strategy categories divided into persuasive appeals and persuasive inquiries, alongside non-persuasive dialogue acts.Non-persuasive acts included general behaviors such as greetings and task-specific acts such as donation propositions and confirmations.
- Persuasive Appeals: Persuasive appeals use psychological mechanisms to change attitudes or decisions, including logical, emotional, credibility, foot-in-the-door, and other strategies.The seven appeals include reasoning, emotion elicitation, credibility building, and staged donation requests.
- Persuasive Inquiries: Persuasive inquiries ask about the source, task, or personal experience to support personalized appeals and stronger interpersonal relationships.Source-related inquiry concerns organizational awareness; task-related inquiry concerns opinions or expectations; personal-related inquiry concerns relevant charity experiences.
- Turn Distributions: Credibility appeal appeared mainly at conversation beginnings, Donation information later, and Logical and Emotion appeals frequently in the middle.Personal story, Self-modeling, and Foot-in-the-door were more evenly distributed across conversations.
- Turn Distributions: Source-related inquiry mainly appeared in the first three turns, while the other two inquiry types had similar distributions.The inquiry distributions were examined separately from the seven persuasive appeals.
5 Donation Strategy Classification
The paper proposed a hybrid RCNN classifier for persuasion strategies that combines sentence representations, dialogue context, and sentence-level features.
- Model Architecture: The hybrid RCNN combines sentence embedding, context embedding, and sentence-level features through vector concatenation.Figure 3 distinguishes the sentence embedding, context embedding, and sentence-level feature components.
- Model Architecture: Sentence embedding uses an RCNN to capture effective information throughout the sentence after recurrent-convolutional processing and max pooling.The resulting sentence representation is transformed linearly before max pooling.
- Model Architecture: Context embedding uses the previous persuadee utterance, with the context LSTM’s last hidden state initializing the sentence RCNN.The design accounts for relatively long conversational context.
- Sentence-Level Features: The sentence-level features encode turn position, sentiment, and character information alongside neural embeddings.Turn position is represented with a 10-dimensional embedding, sentiment uses VADER scores, and character features come from a pretrained mLSTM.
6 Experiments
Experiments evaluated the classifier with cross-validation and macro F1 because the dataset was imbalanced, finding the hybrid RCNN strongest while revealing context and strategy-specific errors.
- Experimental Setup: Macro F1 and accuracy were evaluated using five-fold cross-validation because the dataset was highly imbalanced.The reported scores were averaged across folds for model comparison.
- Experimental Results: 74.8% accuracy and 59.6% F1 were achieved by the hybrid RCNN with all sentence, context, turn-position, sentiment, and character features.This configuration outperformed the baseline models that also used all features.
- Ablation Study: 74.4% accuracy and 59.3% F1 were obtained with RNN context embedding, the best among the four tested context extraction methods.The tested alternatives were CNN, mean word embeddings, RNN, and tf-idf.
- Ablation Study: Context incorporation improved performance slightly but not significantly, while all three sentence-level features improved the model’s F1.The authors suggest complex, relatively long sentences make context semantics difficult to encode.
- Error Analysis: Personal story was often confused with Emotion appeal, Task-related inquiry was difficult to classify, and Foot-in-the-door was confused with Logical appeal.These errors reflect sentences conveying multiple meanings and overlapping semantic cues.
7 Donation Outcome Analysis
The analysis modeled donation as a binary outcome and examined strategy effects, psychological predictors, and interactions between persuasion strategies and personal backgrounds.
- Persuasion Strategies and Donation: Donation information significantly increased donation probability (p < 0.05), partly because step-by-step instructions made donating more accessible.The analysis used logistic regression with donation coded as 1 for any donation and 0 for none.
- Psychological Backgrounds and Donation: Older, more agreeable, caring, benevolent, and rational participants were more likely to donate, whereas intuitive decision makers were less likely.Reported associations were significant for age, agreeableness, care, benevolence, and rational or intuitive decision style.
- Inconsistent Donation Behaviors: Among 236 persuadees who agreed to donate, 11% reduced their amount, 43% did not donate, and 3% donated more than stated.More agreeable participants were more likely to match their stated and actual donation behaviors, but this result was not significant in the relatively small sample.
- Interaction Effects of Persuasion Strategies and Psychological Backgrounds: The analysis expected persuasion strategies to have heterogeneous effects conditioned on persuadees’ psychological profiles.The focal profiles included agreeableness, care, benevolence, and rational decision-making style.
- Interaction Effects of Persuasion Strategies and Psychological Backgrounds: Emotion appeal positively affected donation probability among more extroverted participants, despite lacking a significant average main effect.This interaction was significant at p < 0.05.
- Interaction Effects of Persuasion Strategies and Psychological Backgrounds: Personal-related inquiry increased donation probability for more neurotic people but was negatively associated with donation probability among those endorsing authority more.Both interaction effects were reported as significant at p < 0.05, though the authors caution against overinterpretation pending more annotated data.
8 Ethical Considerations
The paper frames persuasion as ethically dual-use and argues that persuasive agents should benefit persuadees while preserving transparency, consent, and nondiscrimination.
- Ethical Principles: Persuasive agents should be developed with ethical intentions that benefit persuadees before optimizing conversational persuasive capability.The donation task was chosen as an initial context because it involves helping children.
- Deployment Requirements: Deployed agents should disclose their identity, offer access to the human team, explain personal-data use, obtain active consent, and produce appropriate nondiscriminative responses.The paper also calls for continuous conversation monitoring against universal and local ethical standards.
9 Conclusions and Future Work
The work addresses limited high-quality persuasion data by combining dialogue collection, strategy annotation, psychological profiling, and strategy classification as groundwork for user-adaptive agents.
- Contributions: The dataset contains rich human-human persuasion dialogues with psychological studies and persuasion-strategy annotations.The authors identify a lack of high-quality data and interdisciplinary research as key challenges in persuasion study.
- Classifier and Future Work: A classifier using sentence embeddings, context embeddings, and sentence-level features achieved good persuasion-strategy prediction results.The authors propose more annotations and dialogue context as future avenues for improving classifier performance.
- Personalization: Psychological backgrounds interacted with persuasion strategies; for example, Source-related inquiry may be considered for more open participants.The reported interaction findings motivate designing systems that choose strategies using user-profile information.
A.1 Annotation Scheme
The paper defines dialogue-act annotation schemes, excludes ubiquitous donation propositions from the strategy analysis, and documents the associated analyses and interfaces.
- Annotation Scheme: Tables 6 and 7 describe selected persuadee and persuader dialogue acts, respectively.The persuader scheme includes acts related to persuasive propositions and other non-strategy acts.
- Annotation Scheme: Donation propositions were excluded as strategies because they occurred in every conversation and could dilute outcome analyses.The scheme distinguishes propositions of donation, amount, confirmation, and more donation.
- Outcome Analysis: The analyses used logistic-regression coefficients for donation outcomes, with two-tailed significance thresholds of p < 0.05, p < 0.01, and p < 0.001.The reported models predict donation coded as 1 and no donation coded as 0.
- Strategy Classification: Figure 5 presents the confusion matrix for classifying ten persuasion strategies plus a non-strategy category with the hybrid RCNN model and all features.The figure is described as a classification confusion matrix.
- Data Collection Interface: Figures 6 and 7 show the persuader’s and persuadee’s chat interfaces used for data collection.The data-collection interface is described in the accompanying text.