CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems

• Negotiation systems must understand partners’ motives while reasoning and communicating effectively, a challenge for automated systems.Prior human-machine negotiation work primarily used strict protocols such as predefined menus; free-form dialogue efforts were limited.Source: §1 · p. 1
• Prior game-based negotiation dialogues can be tractable to evaluate but often reduce to exchanging offers between arbitrarily defined items.Without semantic context, they may miss small talk, preference elicitation, emotional expression, and arguments grounded in individual needs.Source: §1 · p. 1
• Open-ended buyer-seller negotiations encourage diverse, natural conversation, but make negotiation performance difficult to evaluate.That difficulty also inhibits practical applicability, including the ability to provide critical feedback in skills training.Source: §1 · pp. 1–2
• CaSiNo combines a tractable closed-domain task with a relatable camping scenario to elicit rich negotiation dialogue.Two campsite neighbors negotiate over additional Food, Water, and Firewood according to their individual preferences and requirements.Source: §1 · p. 2
• The corpus contains 1,030 English dialogues featuring rapport building, preference discussion, offers, emotion, and personal or logical persuasion.It also records participants’ outcome satisfaction and opponent likeness, and annotates nine persuasion strategies.Source: §1 · p. 2

Preparation

• Participants complete a tutorial on high offers, preference discussion, appropriate emotion, and arguments based on individual requirements.They prepare justifications for their preferences, drawing on examples such as hiking, recreation, group needs, or emergencies.Source: §2 · pp. 2–3

Task design

• The design aims for a meaningful, symmetric task with symmetric items, allowing either participant to identify with any assigned preference order.The scenario clarifies that basic essentials are already provided, mitigating bias toward any one item.Source: §2 · pp. 2–3
• Participants negotiate additional Food, Water, and Firewood packages, each with three units, in a multi-issue bargaining task.The items represent basic camping requirements, and participants negotiate for extras based on their individual plans and preferences.Source: §2 · p. 3

Dialogue and evaluation

• Randomly paired participants exchange at least 10 utterances, may use four basic-emotion emoticons, and can walk away without agreement.Points are the primary performance metric: High, Medium, and Low priority items are worth 5, 4, and 3 points, respectively, with a maximum of 36.Source: §2 · pp. 3–4
• The study collects satisfaction and opponent likeness on 5-point scales in addition to negotiators’ points.The authors propose considering these measures collectively because participants may be satisfied despite scoring poorly when they view a partner’s arguments as justified.Source: §2 · p. 4

Data collection

• The resulting dataset contains 1,030 dialogues from 846 participants, averaging 11.6 utterances and 22 tokens per utterance.Participants were recruited through Amazon Mechanical Turk; the participant pool was restricted to the United States with minimum assignment and approval requirements. Participants gave positive feedback and related well to the scenario.Source: §2 · p. 4

3 Strategy Annotations

• The annotation scheme labels utterances with one or more strategies, or Non-strategic when no strategy is evident.Utterances are the analysis unit because a strategy can span multiple sentences or several strategies can occur in one sentence.Source: §3 · p. 4

3.1 Prosocial

• Prosocial strategies include Small-Talk, Empathy, Coordination, No-Need, and Elicit-Pref.They cover rapport, acknowledgment of a partner’s personal context, coordination toward a deal, letting go of an unneeded item, and discovering the opponent’s preference order.Source: §3.1 · p. 5

3.2 Proself

• Proself strategies include UV-Part, Vouch-Fairness, Self-Need, and Other-Need.These capture undermining an opponent’s requirements, calling on fairness for personal benefit, and arguing for needs for oneself or another person.Source: §3.2 · p. 5
• Three expert annotators labeled 4,615 utterances from 396 dialogues, and the authors report inter-annotator agreement using nominal Krippendorff’s alpha.Coordination and Empathy are omitted from the paper’s analysis because their higher subjectivity produced relatively lower reliability scores.Source: Table 2 · p. 4, §3.2 · p. 5

Outcome variables

• Points scored correlate positively with participants’ satisfaction (r=0.376, p < 0.01) and opponent perception (r=0.276, p < 0.01).Similar trends appear for the corresponding variables of the negotiation partner.Source: §4 · p. 6

Integrative potential

• Participants achieve significantly more joint points as integrative potential increases (r = 0.425, p < 0.001).The paper describes this as participants discovering the integrative potential of their negotiations.Source: §4 · p. 6

Negotiation strategies

• Greater use of prosocial strategies is generally associated with higher satisfaction and likeness ratings, while Other-Need and Vouch-Fair are negatively correlated with ratings and points.The negative associations with points extend to participants, opponents, and joint points, and generally remain when integrative potential is controlled for.Source: §4 · p. 6
• Negotiators’ strategy use is significantly related to their opponents’ behavior, with both tending to use similar strategies (ps < 0.01).Prosocial strategies are more likely than proself strategies to be associated with prosocial opponent behavior and favorable outcomes in this negotiation scenario.Source: §4 · p. 6

5.2 Experiment Design

• The model predicts strategies from an utterances and its previous dialogue context, with positive-class F1 and Joint-A used to evaluate performance.Joint-A measures the percentage of utterances for which all strategies are predicted correctly.Source: §5 · p. 6, §5.2 · p. 7

5.1 Methodology

• The shared BERT encoder uses task-specific self-attention and turn-position embeddings, while in-domain pre-training incorporates unannotated CaSiNo dialogues.The task-specific attention layers can be visualized to show which input parts are attended for each strategy.Source: §5.1 · pp. 6–7, Figure 1 · p. 7

5.2 Experiment Design

• 68.3% overall F1 and 70.2% Joint Accuracy are achieved by Full+IDPT+OS, the best-performing evaluated model.The comparison uses five-fold cross-validation; Table 3 reports positive-class F1 for the strategy prediction task.Source: Table 3 · p. 8, §5.2 · p. 8
• Full improves F1 for all strategy labels, with more than 100% improvements for the most skewed labels, No-Need and UV-Part.Full reaches 36.4% F1 for No-Need and 44.5% F1 for UV-Part; the final model combines Full with IDPT and oversampling.Source: §5.2 · p. 8
• In two test examples, task-specific attention focuses on relevant utterance parts, including “we” when speakers refer to group needs.The authors suggest phrase retrieval from utterance-level annotations may be possible, but say this requires further investigation.Source: §5.2 · p. 8, Figure 2 · p. 8

6 Related Work

• Negotiation research has addressed game theory, human behavior, and automatic agents, with growing interest in human-agent interactions.Much prior work focused on agent-agent interactions (Cao et al., 2018).Source: §6 · pp. 8–9
• Negotiation dialogue datasets include game settings and buyer-seller bargaining (He et al., 2018).DeVault et al. (2015) study a similar multi-issue bargaining design through face-to-face interaction, speech, and virtual embodied systems.Source: §6 · p. 9
• Other related tasks address charity persuasion (Wang et al., 2019), anti-scam (Li et al., 2020), card collection, and finding a mutual friend (He et al., 2017).The paper distinguishes its focus as rich personal negotiations with different ultimate goals and downstream applications.Source: §6 · p. 9

7 Conclusions and Future Work

• CaSiNo’s campsite setting supports constrained yet linguistically rich and personal conversations.The authors describe future work on demographic and personality analysis and on automated systems that use free-form dialogue and strategies from negotiation research.Source: §7 · p. 9
• The authors identify adding more negotiating parties and modalities such as facial expressions or embodied agents as useful extensions for realistic settings.These changes would make the task more complicated.Source: §7 · p. 9
• The setup assumes each participant’s preferences remain fixed throughout the negotiation and does not model splitting an item into sub-parts.The authors identify both as avenues for future work.Source: §7 · p. 9

8.1 Data Collection

• The study received IRB approval, obtained informed consent, allowed withdrawal, and released anonymized data after removing personally identifying information.Participants were instructed not to use offensive or discriminative language, and compensation followed IRB fairness rules.Source: §8.1 · p. 9

8.2 Automatic Negotiation Systems

• Automated negotiation systems are presented as a way to reduce the cost of training and as a potential conversational-AI capability, while raising concerns about manipulation, deception, bias, and misrepresentation.The authors recommend transparency, regular ethical review, human monitoring, user contact options, and disclosure of data collection and use.Source: §8.2 · pp. 9–10

A Pre-Survey

• The CaSiNo dataset was collected on Amazon Mechanical Turk over a month, with 846 participants; the pool represented varied age groups, genders, ethnic backgrounds, and education levels.The Social Value Orientation counts were 364 Proself, 463 Prosocial, and 19 unclassified.Source: A Pre-Survey · p. 13

B Preparation Phase

• The preparation phase included a participant-facing camping scenario description and examples of arguments for individual requirements.The example categories are described as non-exhaustive.Source: B Preparation Phase · p. 13, Table 4 · p. 13

C Data Post-processing steps

• Incomplete dialogues were removed when either participant disconnected or required pre- or post-survey data were missing.Source: C Data Post-processing steps · p. 13
• Low-effort or irrelevant dialogues were removed, though in some cases irrelevant utterances alone were deleted to retain the rest of the dialogue and its metadata.Indicators included very short utterances, incorrect answers to preference questions, or discussion of the MTurk task rather than the negotiation.Source: C Data Post-processing steps · p. 13
• Dialogues containing inappropriate language identified using an English swear-word lexicon were removed from the final dataset.Source: C Data Post-processing steps · pp. 13–14
• Table 5 presents examples of arguments participants generated for their individual requirements, with categories that are not exhaustive.Source: Table 5 · p. 14

D Participant Feedback

• Manual inspection found generally positive feedback, with participants often describing the task as fun, interesting, or nice and reporting that they could relate to camping.The paper provides selected feedback responses in Table 6.Source: D Participant Feedback · p. 14

E Correlational Analysis

• The correlational analysis is presented in Tables 7, 8, 9, and 10.Source: E Correlational Analysis · p. 14

F.1 Architecture

• The architecture combines a 768-dimensional [CLS] representation and a 128-dimensional feature embedding, then predicts each strategy with feed-forward layers and a sigmoid output.The [CLS] representation comes from a single-layer, single-head BERT self-attention layer.Source: §F.1 · p. 14

F.2 Computing Infrastructure

• A single Nvidia Tesla V100 GPU trained each model across all cross-validation folds in two hours.Source: §F.2 · p. 15

F.3 Training Details

• Hyperparameters were selected using validation mean positive-class F1 averaged across labels and five cross-validation folds.The search varied learning rate, weight decay, and dropout; the chosen settings were used for all experiments.Source: §F.3 · p. 15

G Screenshots from the data collection interface

• The pre-survey used Qualtrics, while the rest of data collection used ParlAI.Source: G Screenshots from the data collection… · p. 15
• Figures 3–6 show the interface stages: task preview, participant onboarding, chat, and post-survey.Source: Figure 3 · p. 16, Figure 4 · p. 17, Figure 5 · p. 18
• The post-survey follows an accepted deal or a participant walking away, and the screenshot contains dummy responses.Source: Figure 6 · p. 19