PaperlayerView exact revision on arXiv

Source-linked AI summary

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions

Stephen Roller, Y-Lan Boureau, Jason Weston, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, Pratik Ringshia, Kurt Shuster, Eric Michael Smith, Arthur Szlam, Jack Urbanek, Mary Williamson

arXiv:2006.12442v2cs.CLcs.AI

TL;DR

The paper targets open-domain agents people prefer to human partners, while noting that static benchmarks miss adaptation to changing topics. It surveys approaches to learning and engaging conversation, reports improvements to common chatbot weaknesses, and identifies unresolved limits in generative expression and dialogue flow.

Abstract

from arXiv · show

We present our view of what is necessary to build an engaging open-domain conversational agent: covering the qualities of such an agent, the pieces of the puzzle that have been built so far, and the gaping holes we have not filled yet. We present a biased view, focusing on work done by our own group, while citing related work in each area. In particular, we discuss in detail the properties of continual learning, providing engaging content, and being well-behaved -- and how to measure success in providing them. We end with a discussion of our experience and learnings, and our recommendations to the community.

Qualities of a Conversational Agent

This section describes qualities needed for open-domain conversational agents people prefer as speaking partners, focusing on continual learning, engaging content, and good behavior. It considers multimodal grounding, personality, personal continuity, integrated skills, and safe, empathetic, privacy-preserving interaction.

Qualities of a Conversational Agent

  • The goal is an agent preferred on average to a human speaking partner in open conversation, rather than one that passes the Turing test.The section identifies continual learning, engaging content, and good behavior as necessary high-level traits.Source: Qualities of a Conversational Agent · p. 1

Continually Learning

  • Continual learning must adapt agents to new users, contexts, topics, and knowledge while continually improving.It involves online model training, extracting learning signals from interaction, and updating knowledge; static fine-tuning does not dynamically adapt to new audiences or topics. Staying current also requires balancing recent data against retaining past-topic ability, while evaluating adaptation remains difficult without dynamic benchmarks.Source: Continually Learning · p. 2
  • Interactive feedback can improve learning, but open-domain conversation lacks reliable reward signals and proxy optimization can distort interaction quality.A self-feeding chatbot learned to estimate satisfaction and request feedback, with gains especially when initial data was limited. Implicit signals can reward sticky engagement rather than interaction quality, and inferred sentiment may concern events outside the conversation; optimizing proxies can discourage negative topics or inflate metrics.Source: Continually Learning · pp. 2–3

Engaging Content

  • Knowledgeable agents need broad familiarity or specialized expertise; Wizard of Wikipedia’s generative variants were rated 26% more engaging than knowledgeless counterparts.The task collected 22k dialogues across 1k topics, and Transformer Memory Networks outperformed systems without a knowledge-storing memory in automatic metrics and human evaluations. Transferring expertise to new tasks remains an open problem, tied to compositional architectures and continual learning.Source: Engaging Content · p. 4
  • Controllable generation can improve conversation flow and human quality judgments, but dialogue-level planning and rich, human-like expression remain open problems.Controls for repetition, specificity, response-relatedness, and question-asking improved human judgments on multi-turn PersonaChat; minimal length constraints also significantly improved ratings. Generative systems can still overuse common n-grams, underuse rare words, and struggle to plan dialogue flow.Source: Engaging Content · pp. 4–5
  • Dialogue NLI reranking yielded an average of 3x fewer contradictions across three test sets, and humans rated the models more consistent.The approach applies a classifier as post-processing and is limited to Persona-Chat; future systems should build consistency understanding into models and generalize beyond that domain.Source: Engaging Content · pp. 5–6
  • Current dialogue systems have limited memory, and effective static knowledge retrieval does not provide the read-write memory needed to form and retain new knowledge.Dialogue history is often truncated to a few turns; the section calls for architectures, datasets, and benchmarks for creating memories and for storing useful inferences over time.Source: Engaging Content · p. 6
  • Reasoning research has produced progress on NLI and commonsense benchmarks, but whether systems genuinely generalize remains uncertain.The section notes that adversarially collected benchmark rounds aim to expose and fix model flaws.Source: Engaging Content · p. 6
  • Reasoning methods remain under-transferred to dialogue, and likelihood and unlikelihood training have been proposed to reward correct reasoning and penalize incorrect reasoning (Li et al., 2020).The authors also argue that grounding language in other modalities can connect agents to users’ direct experiences, while substantial cross-modal research remains to be done.Source: Engaging Content · pp. 6–7
  • 49.5 percent of the time, annotators preferred Image-Chat model captions on the first turn over human-written captions (Shuster et al., 2018).Image-Chat contains 187k image-grounded dialogues with speakers adopting personalities, and its best system approached human performance in engagement and relevance.Source: Engaging Content · p. 7
  • 64.5% win rate: captions conditioned on personality were significantly more engaging than neutral captions (Shuster et al., 2018).Image-Chat models could mimic 215 personality traits; the authors favor adaptable multiple personalities, while generalization, change over time, and matching traits to users remain understudied.Source: Engaging Content · pp. 7–8
  • As relationships develop, users expect models to maintain reasonable continuity in how personal conversation is; end-to-end chatbots often forget simple attributes across turns, let alone sessions, and dialogue research should connect more closely with recommendation systems (Dodge et al., 2016; Kang et al., 2019).Source: Engaging Content · p. 8
  • Models blending knowledgeability, personality, and empathy were strongly preferred over models able to perform only one skill (Roller et al., 2020).The 12-task dodecaDialogue challenge combines knowledge, personality, and images; initial experiments suggest multitasking may support a single agent with these skills (Shuster et al., 2020).Source: Engaging Content · p. 8

Well-Behaved

  • Human-in-the-loop adversarial refinement improved toxic-content detection and robustness over three rounds, while crowdworker-data fine-tuning yielded less toxic models than social-media pre-training.Dialogue context affects whether an utterance is offensive; open problems include understanding individual boundaries and avoiding bland responses when optimizing against toxicity (Roller et al., 2020).Source: Well-Behaved · pp. 8–9
  • Training on an empathy benchmark yielded models rated as more empathetic, but the right balance with informing or entertaining—and users’ preferred empathy levels—remains unclear (Rashkin et al., 2019).A potential objective is controlling empathy according to conversational context and partner.Source: Well-Behaved · p. 9
  • Role-play can shield private information, but it may mismatch real private conversations; privacy-preserving learning and local personalization are proposed alternatives.These approaches require smaller, efficient models that can be updated on-device without communicating with external servers.Source: Well-Behaved · pp. 9–10

Measuring Success

Because open-ended multi-turn dialogue is difficult to evaluate, the section surveys human, automatic, and behavioral measures and argues they should be considered together with some granularity.

Human Evaluations

  • Single-turn pairwise tests are simple but miss multi-turn behavior, while Likert evaluations capture longer conversations at greater cost and with measurement difficulties.Likert ratings can struggle to detect incremental differences and can shift with annotator populations and sequential effects.Source: Measuring Success · p. 10
  • ACUTE-Eval compares complete dialogues pairwise, exposing multi-turn issues while reducing sequential effects and enabling reuse of collected conversations.Its two evaluation stages can raise costs, though reusing dialogues can reduce the number of conversations needed; self-chat results highly correlated with human-chat for most, but not all, systems.Source: Measuring Success · pp. 10–11, Figure 9 · p. 11

Automatic Metrics

  • Word-overlap metrics correlate poorly with human judgments, while perplexity and learned metrics have generation, comparability, or optimization caveats.Perplexity may not reflect beam-search or sampled output; learned metrics can be difficult to disentangle from the generation system they evaluate.Source: Measuring Success · pp. 11–12

Behavioral Metrics

  • Behavioral measures such as turn count or retention can misrepresent preference when system behaviors frustrate users or exploit engagement.The suitable implicit metrics depend heavily on the system's medium and design.Source: Measuring Success · p. 12

Discussion

  • The discussion recommends measuring human, automatic, and behavioral outcomes with enough granularity to understand trade-offs, while initially focusing deployed models on retention.Retention is proposed in the short term to maintain a steady stream of users for experimentation and iteration.Source: Measuring Success · p. 12

Discussion

The discussion sets openness, privacy, and practical research lessons alongside recommendations for shared tasks and software, aiming to accelerate accessible, reproducible open-domain dialogue research.

Values and Guiding Principles

  • Open publication of findings, models, code, and data is paired with ParlAI’s unified platform, which reduces development time across data collection, modeling, evaluation, and deployment.The authors describe openness and swiftness as guiding principles.Source: Values and Guiding Principles · p. 12
  • Privacy takes precedence over openness and swiftness in conflicts, with consent, anonymization, and future private-mode and federated-learning measures proposed.The authors suggest role-playing games as one way to reduce sensitive information in released data.Source: Values and Guiding Principles · pp. 12–13

Our Experiences

  • Pretraining benefits nearly every tested chitchat retrieval and generation variant, with dialogue-like pretraining outperforming generic corpora; deployment should also remain feasible on commodity hardware.The authors recommend that researchers be able to interact with models in real time on a mid-tier laptop.Source: Our Experiences · p. 13
  • Automatic testing, continuous integration, and code review help catch regressions and bugs, while ParlAI supports consistent quality and reproducibility across projects.The authors also credit a shared platform with helping preserve model performance and longevity.Source: Our Experiences · p. 13
  • Games requiring multiple human participants face pairing delays that can frustrate users and prompt abandonment; some interactions may work better without another person present.Users may want an immediate game response while taking their time to reply.Source: Our Experiences · p. 13
  • Adversarial users have tried to elicit offensive responses, underscoring response safety—especially safety-in-context—and the risks of online learning without strong safeguards.Some adversaries expected the bot to learn suggested responses quickly or asked sensitive questions to provoke failures.Source: Our Experiences · p. 13

Recommendations to the Community

  • Shared tasks should standardize measurements, grow to cover harder chatbot behaviors, and include interactive learning systems with more open data and participation.The authors point to DodecaDialogue as an evaluation framework spanning many of the discussed research challenges.Source: Recommendations to the Community · pp. 13–14
  • ParlAI standardizes task I/O and exposes teachers, models, crowdworkers, and users through one Agent API, connecting data collection, training, evaluation, and deployment.Its adoption has also created an ecosystem of support, feature development, and engineering improvements.Source: Recommendations to the Community · p. 14

Conclusion

The authors report that their models can improve common chatbot weaknesses and aim to combine these advances into a unified agent. They share models, training code, and datasets through ParlAI as a resource for the research community.

Conclusion

  • The authors report that models can improve some of today’s most common chatbot weaknesses.They describe bringing the subtasks together into one unified intelligent agent as future work.Source: Conclusion · p. 14
  • Their longer-term vision is an open-domain agent that is personable, consistent, empathetic, and engaging.Source: Conclusion · p. 14
  • The authors share models, training code, and datasets through ParlAI to support research advances and address open problems.Source: Conclusion · p. 14
Loading 2006.12442v2…