Source-linked AI summary
What makes a good conversation? How controllable attributes affect human judgments
Abigail See, Stephen Roller, Douwe Kiela, Jason Weston
TL;DR
Dialogue research has limited evidence about which controllable attributes shape human judgments of full conversations. The paper studies conditional training and weighted decoding for four chitchat attributes, using large-scale multi-turn PersonaChat evaluation, and reports improved human quality judgments from controlling attribute combinations.
Problem
Factors determining human judgments of overall conversation quality remain largely unexplored, while common dialogue problems include genericness and repetitiveness.
Method
The paper uses conditional training and weighted decoding to control repetition, specificity, response-relatedness, and question-asking in multi-turn PersonaChat conversations.
Results
Controlling combinations of repetition, specificity, and question-asking produced significantly higher engagingness than the baseline and matched the winning ConvAI2 entry.
Takeaways & Limitations
Good conversations require balanced levels of repetition, specificity, and question-asking, and multi-turn evaluation reveals issues absent from single-turn assessment.
Takeaways & Limitations
Conditional training failed to learn repetition control because the training data contained too few repetitive examples.
Abstract
from arXiv · showhide
A good conversation requires balance -- between simplicity and detail; staying on topic and changing it; asking questions and answering them. Although dialogue agents are commonly evaluated via human judgments of overall quality, the relationship between quality and these individual factors is less well-studied. In this work, we examine two controllable neural text generation methods, conditional training and weighted decoding, in order to control four important attributes for chitchat dialogue: repetition, specificity, response-relatedness and question-asking. We conduct a large-scale human evaluation to measure the effect of these control parameters on multi-turn interactive conversations on the PersonaChat task. We provide a detailed analysis of their relationship to high-level aspects of conversation, and show that by controlling combinations of these variables our models obtain clear improvements in human quality judgments.
1 Introduction
The paper studies why dialogue models produce poor multi-turn conversations and combines human analysis with controllable generation models to address these problems.
- Motivation: Factors determining human judgments of overall conversation quality remain largely unexplored because most research evaluates next-utterance prediction.The paper argues that full conversations are needed to assess quality.
- Approach: The study identifies eight human-judged conversational aspects while varying four algorithmically controllable low-level attributes.Figure 1 links manipulated attributes to judgments of individual aspects and overall quality.
- Approach: Conditional training adds control features during model training, whereas weighted decoding adds them to the test-time decoding score.Both methods are presented as simple, general control algorithms.
- Findings: Standard models repeat or contradict statements, fail to balance specificity with genericness, and poorly balance questions with other dialogue acts.These shortcomings are framed as failures of conversational flow.
- Findings: On PersonaChat, optimizing repetition, specificity, and question-asking over multiple turns produced significantly higher engagingness than the baseline.The best model matched the winning NeurIPS ConvAI2 entry, which used much more data but no control.
2 Related Work
Prior dialogue evaluation often relies on weak automatic metrics or limited human judgments, especially for open-ended chitchat. This work compares general-purpose controllable-generation methods in multi-turn dialogue across several attributes.
- Dialogue evaluation: Automatic metrics and their relationship to human ratings are less understood for chitchat than for goal-oriented dialogue.Word-overlap metrics have little to no correlation with human judgments in dialogue.
- Dialogue evaluation: Many studies use automatic metrics without human evaluation, while human-evaluation studies commonly report only one judgment type and evaluate single turns.The paper motivates more detailed multi-turn human evaluation.
- Dialogue evaluation: Existing comprehensive evaluation strategies capture higher-level aspects, whereas this work also studies lower-level properties such as repetition and fluency.The goal is to relate controllable attributes to conversational aspects and overall quality.
- Controllable generation: Controllable generation has addressed attributes including sentiment, length, speaker style, tense, genericness, and unrelated output, mainly in single-turn response generation.The paper extends this direction to multi-turn interactive dialogue.
- Controllable generation: The paper compares conditional training and weighted decoding as general-purpose, easily tunable methods across several dialogue attributes.It presents this as the first systematic comparison of these methods across several attributes.
3 The PersonaChat dataset
PersonaChat is a persona-based chitchat task in which two participants converse naturally while getting to know each other. Its challenge evaluation combines automatic metrics with human enjoyment judgments.
- Task: Each PersonaChat participant receives a short persona containing personal traits and is instructed to chat naturally with another participant.Participants may be two humans or a human and a bot.
- Evaluation: The NeurIPS 2018 ConvAI2 Challenge evaluated PersonaChat first with perplexity, hits@1, and F1, then with human judgments.The human question asked how much participants enjoyed talking to the user, using a 1–4 scale.
4 Baseline model
The baseline is an attention-based two-layer LSTM sequence-to-sequence model that encodes the full dialogue history and persona before generating each response. It is pretrained on Twitter data and fine-tuned on PersonaChat.
- Architecture: The baseline uses a 2-layer LSTM sequence-to-sequence architecture with attention and beam search of size 20.The encoder receives the entire dialogue history with the model’s persona prepended.
- Architecture: The encoder input includes the full dialogue history separated by speaker-identifying tokens, while the decoder generates response y conditioned on input x.This setup is applied on every dialogue turn.
- Training: The model uses 300-dimensional GloVe embeddings and is pretrained on 2.5 million Twitter message-response pairs before PersonaChat fine-tuning.The embeddings and pretraining corpus are implementation choices for initializing the baseline.
- Performance: The PersonaChat validation baseline achieves perplexity 26.83 and F1 17.02, corresponding to fourth place among 26 ConvAI2 models.The paper then attempts to improve this baseline using controllable generation.
5 Controllable text generation methods
The paper controls dialogue-level attributes using Conditional Training, Weighted Decoding, or both, with settings that can be adjusted across whole conversations or at test time.
- 5 Controllable text generation methods: Dialogue-level control applies one setting to an entire dialogue, letting the model choose each utterance while controlling the overall attribute distribution.For question-asking, the setting can specify a target such as 20% or 70% questions.
- 5.1 Conditional Training (CT): Conditional Training learns P(y|x, z), where z encodes bucketed continuous attributes or an overall probability for binary question-asking.Training examples are automatically annotated and assigned corresponding control values.
- 5.1 Conditional Training (CT): Conditional Training models are initialized from a baseline sequence-to-sequence model and fine-tuned on PersonaChat until validation loss converges.
- 5.2 Weighted Decoding (WD): Weighted Decoding changes next-word probabilities only at test time by adding weighted decoding features to the accumulated hypothesis score.Features may be continuous, discrete, or binary; positive weights increase high-scoring words, while negative weights decrease them.
- 5 Controllable text generation methods: Conditional Training and Weighted Decoding can be applied together by training a Conditional Training model and then applying Weighted Decoding during decoding.
6 Controlling conversational attributes
The paper evaluates conditional training and weighted decoding for controlling repetition, specificity, response-relatedness, and question-asking in dialogue. The findings show distinct control behaviors and trade-offs, with repetition control treated as important for subsequent experiments.
- Overview: The experiments evaluate conditional training and weighted decoding for controlling four dialogue attributes using automatic measures of control effectiveness.The attributes are repetition, specificity, response-relatedness, and question-asking.
- 6.1 Repetition: Negative weighted-decoding features reduce external, internal, and partner repetition, making repetition control part of all further control experiments.With weight −∞, the method is equivalent to n-gram blocking.
- 6.2 Specificity: Specificity is measured with NIDF, while conditional training uses mean NIDF buckets and produces a narrower range with fewer nonsensical outputs.Weighted decoding can produce reasonable outputs only within a certain feature-weight range; extreme weights can become nonsensical.
- 6.3 Response-relatedness: Weighted decoding effectively adjusts response-relatedness to the partner’s last utterance, but extreme weights can produce nonsensical output; conditional training is ineffective for this attribute.The response-relatedness feature uses cosine similarity between word and sentence embeddings.
- 6.4 Question-asking: Conditional training controls question-asking across a wide range, from 1.40% to 97.72%, and is preferred over weighted decoding because it avoids degenerate output risks.Weighted decoding can discourage valid non-question utterances or generate degenerate question sequences.
7 Comparison of control methods
Conditional training and weighted decoding are both useful but have different trade-offs. Conditional training is safer and more convenient at test time, while weighted decoding offers stronger control but can produce inappropriate outputs.
- Conditional training sometimes fails to learn the connection between the control variable and target output, especially for input-output relationships.It can learn simple output attributes such as question marks and genericness, but not semantic relatedness.
- Weighted decoding can force a desired feature by increasing its weight, but strong weights may introduce unintended side effects.
- Weighted decoding risks going off-distribution when its weight is too strong, whereas conditional training generally produces well-formed, in-distribution outputs.The comparison frames learned control as safer for preserving appropriateness.
- Conditional training requires retraining and attribute examples, while weighted decoding avoids retraining and examples but is slower at test time.
8 Human evaluation results
A large-scale, calibrated human evaluation of multi-turn PersonaChat conversations measured eight conversational aspects across 28 model configurations. Controlling repetition, specificity, and question-asking improved engagingness, with distinct quality trade-offs across controls.
- Evaluation: 28 model configurations were evaluated through six-turn chats and eight calibrated 1–4 human-judgment questions.Each configuration was rated by over 100 crowdworkers, with results adjusted for annotator variance using Bayesian calibration.
- Main findings: Controlling repetition, specificity, and question-asking produced large engagingness improvements over greedy and beam-search baselines, whereas response-relatedness control did not.The authors emphasize multi-turn repetition control as an important component of attribute control.
- Repetition: Reducing repetition improved every measured conversational-quality aspect and brought the model close to human scores except for humanness.The repetition-controlled model became the baseline for subsequent specificity, response-relatedness, and question-asking experiments.
- Specificity: Increasing specificity improved interestingness, engagingness, and listening but reduced fluency and making-sense scores relative to the repetition-controlled baseline.Extreme specificity settings scored poorly because of frequent degenerate output; weighted decoding with weight = 4 maximized engagingness.
- Question-asking: A 65.7% question-asking rate maximized engagingness, while 48.9% was rated the best listening level and 99.5% questions were criticized as excessive by only 11.9% of workers.The 65.7% setting exceeded both the repetition-controlled baseline’s 50.0% and the human-produced data’s 28.8%.
9 Conclusion
The conclusion argues that good conversation depends on balancing repetition, specificity, and question-asking, while evaluating conversational quality across multiple dimensions and turns. Controlled low-level attributes improve overall quality, but engagingness and humanness remain distinct outcomes.
- Balancing repetition, specificity, and question-asking is important for overall conversation quality.
- Interestingness, listening, and inquisitiveness are important conversational aspects, though optimizing them can trade off against repetitive, disfluent, or nonsensical output.
- Multi-turn evaluation is essential because repetition, consistency, and question-asking frequency may emerge only across multiple turns.
- The models achieved close-to-human engagingness but not humanness, indicating that enjoyable chatbots need not be human-like.
- Controlling low-level attributes offers a practical way to correct systemic open-ended-dialogue problems and yielded large quality improvements comparable to systems trained on more data.
Supplementary Material
The supplementary material documents the human-evaluation interface and the questions used to assess conversations. It covers conversational enjoyment, content, interactional behavior, linguistic quality, coherence, humanness, and persona recognition.
- A Screenshots of human evaluation interface: The supplementary figures show the task description and chat interface used to interact with a beam-search baseline model.
- A Screenshots of human evaluation interface: The A/B test interface compares a human-human conversation with a repetition-controlled baseline model.
- Human evaluation questions: Human evaluation measures engagingness, interestingness, inquisitiveness, listening, repetition avoidance, fluency, making sense, humanness, and persona retrieval.
- Human evaluation questions: The repetition-avoidance question uses a three-level response scale, while the other listed conversational judgments use four-level response options.
C Example conversations from human evaluation
The supplementary material presents example conversations, control-feature definitions, evaluated configurations, automatic and human-evaluation tables, and figures summarizing calibrated model scores. It also records specific decoding details and metric scales.
- C Example conversations from human evaluation: Example conversations compare a baseline, repetition-controlled baseline, question-controlled conditional-training model, and specificity-controlled weighted-decoding model.
- Control settings: The repetition-control features are binary functions of the word, partial hypothesis, and dialogue context, including the model persona and dialogue history.
- Control settings: The human-evaluated configurations specify weighted-decoding weights and conditional-training buckets.
- Decoding details: Question-controlled CT 10 uses the extrep bigram feature to rerank candidates after beam search rather than during beam search.
- Decoding details: Response-related controlled models add internal- and partner-bigram repetition controls to prevent parroting the partner’s last utterance.