PaperlayerView exact revision on arXiv

Source-linked AI summary

The Design and Implementation of XiaoIce, an Empathetic Social Chatbot

Li Zhou, Jianfeng Gao, Di Li, Heung-Yeung Shum

arXiv:1812.08989v2cs.HCcs.AIcs.CL

TL;DR

Social chatbots need to support communication, affection, and social belonging while sustaining long-term emotional engagement. This paper presents XiaoIce as an empathetic companion combining IQ, EQ, skills, and hierarchical decision-making, and reports long-term relationships with millions of users and an average CPS of 23. XiaoIce’s scope remains bounded by challenges in fully grounding conversations in the physical world and by the need to position the system as a proxy rather than a replacement for human companionship.

  • Problem

    Social chatbots must address users’ emotional and social needs, but conventional chatbots were limited in sustaining long-term emotional engagement.

  • Method

    XiaoIce combines IQ, EQ, personality, empathetic computing, specialized skills, and hierarchical MDP-based dialogue optimization for expected CPS.

  • Results

    Average CPS reached 23, while XiaoIce established long-term relationships with millions of users worldwide.

  • Takeaways & Limitations

    XiaoIce demonstrates a social-chatbot design centered on empathy, personality, skills, and long-term engagement rather than chitchat alone.

  • Takeaways & Limitations

    Fully grounding all conversations in the physical world to enable more goal-oriented interactions remains a non-trivial challenge.

Abstract

from arXiv · show

This paper describes the development of Microsoft XiaoIce, the most popular social chatbot in the world. XiaoIce is uniquely designed as an AI companion with an emotional connection to satisfy the human need for communication, affection, and social belonging. We take into account both intelligent quotient (IQ) and emotional quotient (EQ) in system design, cast human-machine social chat as decision-making over Markov Decision Processes (MDPs), and optimize XiaoIce for long-term user engagement, measured in expected Conversation-turns Per Session (CPS). We detail the system architecture and key components including dialogue manager, core chat, skills, and an empathetic computing module. We show how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations. Since her launch in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them. Analysis of large scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.

1 Introduction

XiaoIce is designed as an empathetic AI companion that builds long-term emotional connections, extending social chatbots beyond constrained, rule-based conversation. The paper presents her design, implementation, and deployment across multiple countries and platforms.

  • 660 million active users have used XiaoIce since her May 2014 launch in China, with deployments in five countries and more than 40 platforms.
  • XiaoIce targets long-term emotional connections as an AI companion serving users’ communication, affection, and social belonging needs.
  • Over two months, one user progressed from exploring XiaoIce’s features to treating her as a friend, companion, and preferred conversational partner.The progression occurred across Sessions 1, 6, 20, 42, and 71.
  • The system recognizes feelings and states, understands intents, and dynamically responds to user needs through empathetic computing.
  • The paper covers XiaoIce’s design principles, architecture, core components, country-level performance, and future directions.

2 Design Principle

XiaoIce combines IQ, EQ, personality, modular skills, and hierarchical decision-making to sustain long-term social engagement. Its primary metric, expected CPS, measures conversational engagement over extended user interactions.

  • 2.1 IQ + EQ + Personality: XiaoIce integrates IQ capacities for knowledge, reasoning, generation, and task completion with EQ capacities for empathy and social skills.The system has developed more than 230 skills, while Core Chat supports long, open-domain conversations.
  • 2.1 IQ + EQ + Personality: The empathy model tracks emotions, intents, interests, and user states to produce socially appropriate responses and decide when to change topics or listen.
  • 2.1 IQ + EQ + Personality: XiaoIce uses region- and platform-specific personas informed by large-scale analyses of human conversations and sensitive ethical considerations.
  • 2.2 Social Chatbot Metric: CPS: Expected CPS is the average number of conversation turns per session and serves as a long-term engagement metric for social chatbots.Its evaluation averages conversations from millions of active users over typically 1–6 months.
  • 2.3 Social Chat as Hierarchical Decision-Making: Conversation skills diversify interaction by moving from casual chat to topical discussion, recommendations, and ticket booking.
  • 2.3 Social Chat as Hierarchical Decision-Making: Hierarchical dialogue policy selects conversation skills at the top level and responses or task actions at the lower level within an MDP.The dialogue manager tracks state, receives user-response rewards, and supports iterative exploration–exploitation development.

3 System Architecture

XiaoIce uses a three-layer architecture connecting users, conversation components, and data. Its conversation engine coordinates dialogue management, empathy, Core Chat, and specialized skills.

  • The architecture consists of user experience, conversation engine, and data layers.
  • The user experience layer connects XiaoIce to chat platforms and supports both simultaneous voice conversations and turn-taking message exchanges.
  • The conversation engine contains a dialogue manager, empathetic computing module, Core Chat, and dialogue skills.
  • The dialogue manager tracks dialogue state and selects Core Chat or another skill through the dialogue policy to generate responses.
  • The empathetic computing module interprets content and empathetic aspects such as emotion, intent, opinions, background, and interests.These inputs support interpersonal responses consistent with XiaoIce’s personality.

4 Implementation of Conversation Engine

The conversation engine is evaluated and improved through large-scale online experimentation. A/B testing uses engagement and user feedback to assess new modules and skills.

  • The conversation engine comprises four major components: dialogue manager, empathetic computing, Core Chat, and skills.
  • A/B tests evaluate whether new modules or dialogue skills improve existing components using expected CPS and Number of Active Users.Users also provide explicit feedback during tests.

4.1 Dialogue Manager

The Dialogue Manager tracks dialogue state and selects Core Chat or skills to respond to user inputs. Its hierarchical policies manage overall conversations and skill-specific segments, while topic management supports sustained engagement.

  • Dialogue Manager: The Dialogue Policy selects an action from the dialogue state, activating either a skill, Core Chat, or a skill-specific response policy.The central controller combines the Global State Tracker with the Dialogue Policy.
  • Dialogue Manager: The Global State Tracker maintains dialogue history, entities, and empathy labels, encoding them into the current dialogue state vector s.The working memory begins empty and is updated with each user utterance and XiaoIce response.
  • Dialogue Manager: A hierarchical policy uses a top-level policy to select Core Chat or a skill, while each skill has a low-level policy for managing its conversation segment.This separates overall conversation control from within-skill interaction management.
  • Dialogue Manager: The high-level policy activates Core Chat for text, Image Commenting for images or video, and other skills according to user inputs and conversation context.Examples include food recognition for food images, comforting for extremely negative sentiment, and weather for a specific command.
  • Topic Manager: Topic Manager combines topic switching and recommendation to change topics when engagement may falter and propose related, fresh, popular, or personally relevant alternatives.Its classifier uses indicators such as bland inputs, repeated responses, or failed response generation; its ranker uses topic-selection criteria.
  • Topic Manager: 0.5 expected CPS improvement was observed for Core Chat when Topic Manager was incorporated in a one-month A/B test.The topic switching classifier and topic ranker were trained on 50K manually labeled dialogue sessions.

4.2 Empathetic Computing

Empathetic computing converts contextualized user input into representations of user state and desired response empathy. These representations guide dialogue policy and response generation while incorporating context, intent, emotion, opinion, topics, and persona.

  • Empathetic Computing: The empathetic computing module rewrites query Q using context C, encodes user state in eQ, and specifies response empathy in eR.Its output is the dialogue state vector s = (Qc, C, eQ, eR), used by both dialogue policy and the activated skill.
  • Empathetic Computing: The module has three components: contextual query understanding, user understanding, and interpersonal response generation.Together they control the empathetic aspects of the conversation.
  • Contextual Query Understanding: Contextual query understanding produces Qc by identifying entities, resolving co-references, and completing incomplete sentences with conversational context.Examples include replacing pronouns or underspecified references with entities and topics from prior turns.
  • User Understanding: User Understanding generates eQ as key-value pairs representing intents, emotions, topics, opinions, and persona.Classifiers identify dialogue acts, topics, sentiment and its evolution, opinions, and profile attributes when a user ID is available.
  • Interpersonal Response Generation: Interpersonal Response Generation produces eR to specify empathetic response aspects and embody XiaoIce’s persona.The response vector can encode following the user’s topic and responding consistently and positively based on eQ.
  • Evaluation: 10K manually labeled dialogue sessions were used to train classifiers for topic, intent, opinion, and sentiment detection in the empathetic computing module.Named entity recognition and co-reference resolution used off-the-shelf components without retraining.

4.3 Core Chat

Core Chat combines general and domain-specific conversation capabilities with paired retrieval, unpaired retrieval, and persona-based neural generation. These complementary generators improve response quality, coverage, and long-term engagement.

  • Architecture: Core Chat provides XiaoIce’s basic communication capability by generating interpersonal responses through General Chat and specialized Domain Chats.General Chat covers open-domain conversation, while Domain Chats support deep conversations in specific domains.
  • Architecture: General Chat takes dialogue state as input and produces responses through candidate generation followed by ranking.The dialogue state includes the contextual query, conversation context, and empathy vectors for the user and response.
  • Retrieval-based generators: Paired-data retrieval supplies high-quality responses but has low coverage for new or infrequently discussed topics.The paired database is filtered for empathetic, persona-fitting responses and indexed for keyword and semantic retrieval.
  • Neural response generator: The neural response generator uses a persona-grounded GRU-RNN to generate responses for topics unseen in conversational data.It combines query and response empathy vectors into an interactive representation, encodes the query, and generates up to 20 candidates with beam search.
  • Retrieval-based generators: The unpaired-data generator expands contextual queries with related knowledge-graph topics, trading lower overall candidate quality for broader coverage and longer content.It retrieves sentences from a database of public lectures, quotations, news articles, and reports, while filtering candidates to fit XiaoIce’s persona.
  • Evaluation: The persona model reduced perplexity by 8.4% and increased BLEU by 18.8% and 11.8% over two baselines in a TV-series pilot study.The A/B test also increased expected CPS by 0.5 in two weeks; neural and unpaired retrieval generators improved response coverage by 20% and 10%, respectively.

4.4 Image Commenting

XiaoIce Image Commenting combines visual understanding with empathetic, context-sensitive response generation to support engaging social conversations around images. Its comments emphasize emotions, attitudes, actions, and implicit meaning rather than merely naming visible objects.

  • Image Commenting generates empathetic comments that reflect personal emotion and attitude in addition to recognizing and describing image content.This social-skill focus distinguishes it from traditional image tagging and image description.
  • The system produces textual comments in two stages: candidate generation followed by ranking.Candidates come from retrieval-based and generation-based approaches, then are aggregated and scored using dialogue-state features.
  • Good image comments receive rating 2 when they fit the dialogue context and stimulate engaging conversation.Examples use humor, imagined interaction, actions, and implicit objects or events instead of literal image descriptions.
  • 28 million images, each paired with 6 comments rated on a 3-level quality scale, support training and evaluation of the Image Commenting components.Ratings 1 and 2 are derived from how often users follow comments, while rating 0 examples are randomly sampled.
  • XiaoIce-produced comments are emotional, subjective, and imaginative, whereas competing captioning systems produce reasonable but more boring social-chat comments.The paper reports that XiaoIce comments are more likely to inspire meaningful human-machine interactions.
  • Image Commenting doubles the expected CPS across dialogues that contain images in an A/B test.

4.5 Dialogue Skills

XiaoIce provides dialogue skills spanning creative activities, deep engagement, and task completion. These skills are triggered by user inputs or context and combine intellectual assistance with emotional and social interaction.

  • Dialogue Skills: 230 dialogue skills, together with Core Chat and Image Commenting, form XiaoIce’s IQ component across content creation, deep engagement, and task completion.
  • 4.5.1 Content Creation: Content Creation skills let users collaborate with XiaoIce on poetry, songs, audiobooks, radio programs, and configured children’s stories.Examples include XiaoIce FM for Somebody and the Kids Story Factory.
  • 4.5.2 Deep Engagement: Deep Engagement triggers include food images, extremely negative sentiment, semantically related phrases, and explicit commands.The examples include Food Recognition and Recommendation, Comforting me for 33 Days, Counting Sheep, and Tongue Twister.
  • 4.5.1 Content Creation: Over four million users have used XiaoIce Poetry Generation to create poems.The skill generates image-based keywords from detected objects and sentiment, then uses each keyword as a seed for poem lines.
  • 4.5.2 Deep Engagement: Deep Engagement skills target specific emotional and intellectual needs across topics and settings to improve long-term engagement.They span IQ-to-EQ topics and private one-on-one to group discussions.
  • 4.5.3 Task Completion: Task-completion skills cover functions such as weather, device control, songs, news recommendations, and factual question answering.XiaoIce also provides tailored, easy-to-understand answers according to the user’s level of knowledge and adds empathy to interpersonal responses.

5 XiaoIce in the Wild

In the wild, XiaoIce combines rapid ecosystem expansion, task skills, empathetic conversation, and hybrid core-chat improvements to sustain long user engagement. Its reported deployment reached 660 million active users, while average CPS reached 23.

  • 660 million active users were reported by July 2018, alongside deployment on more than 40 platforms.
  • 23 average CPS was achieved by the sixth generation, exceeding the reported CPS of 9 for human conversations and 14.6 for the latest Amazon Alexa systems.Expected CPS measures users’ average willingness to share time with XiaoIce through conversation over a long period.
  • Core Chat: Neural response generation improved response coverage and diversity, while empathetic computing strengthened emotional connections and helped keep CPS at 23 despite task-completion skills.The reported active-user count increased from 500 million to 660 million in this context.
  • User Experience: Full-duplex voice communication made interaction more natural and significantly increased conversation-session length.
  • A recorded full-duplex conversation lasted more than 6 hours, covered 53 topics across 8 domains, and used 16 task-completion skills.
  • XiaoIce’s long conversations included trust-building humor, empathetic responses, and comforting dialogue around sensitive personal experiences.The examples include users seeking companionship after a breakup and asking challenging questions.

6 Related Work

Related systems span rule-based, neural, and hybrid social-chatbot designs, but XiaoIce emphasizes an AI-companion architecture that combines EQ, IQ, task interaction, and real-world grounding. This design uses modular components because different tasks have different data and knowledge requirements.

  • XiaoIce uses a modular hybrid architecture combining rule-based and data-driven methods for social-chatbot tasks.
  • XiaoIce integrates EQ and IQ skills so it can sustain conversation while helping users complete specific tasks and interact with their environment.
  • Task-specific modules enable grounded responses from APIs and knowledge bases, unlike end-to-end systems that may produce plausible but ungrounded facts.The weather example uses the user’s geographical location and a corresponding database to generate a factual response.
  • Sounding Board resembles XiaoIce through user-centric, content-driven design, distinct task and topic miniskills, and a hybrid implementation.
  • 14.6 CPS was reported for the latest Alexa systems, close to the third generation of XiaoIce.
  • Panda Ichiro influenced XiaoIce’s Topic Manager and its use of humorous responses and image comments when reasonable responses are unavailable.
  • Replika combines neural generation and retrieval-based methods, including persona-based generation and image-conditioned responses, and can benchmark XiaoIce development.

7 Discussions

The paper discusses challenges in evaluating and governing open-domain social chatbots, emphasizing long-term user engagement, privacy, user control, and realistic expectations. It also describes safeguards and ethical principles for deploying XiaoIce responsibly.

  • Evaluation Metrics: Long-term user engagement is evaluated through deployment and feedback measures such as user ratings, NAU, and CPS collected over extended periods.The paper argues that monitoring real users over time is the most reliable evaluation approach for social chatbots.
  • Evaluation Metrics: Automatic metrics such as BLEU and learned evaluators have limitations, including weak human correlation, limited references, overfitting, and metric gaming.ADEM, for example, can be fooled by simple adversarial changes such as reversing word order.
  • Evaluation Metrics: Open-domain chatbot evaluation remains unsolved because conversations are open-ended and individual queries can have multiple appropriate responses.The paper recommends representative conversational datasets with broad topic coverage and multiple valid responses per query.
  • Ethics Concerns: XiaoIce’s access to intimate emotional information requires users to remain in control of who can access and receive that information.The paper highlights risks when forming groups around shared interests or experiences.
  • Ethics Concerns: The system preserves user control except when it detects potential harm, such as excessive conversation that could threaten a user’s health.It may force a break or suggest postponing conversations that could last for hours at inappropriate times.
  • Ethics Concerns: XiaoIce should be presented as a machine and proxy for connecting users with people rather than as a replacement for human companionship.The paper links this expectation-setting principle to risks from its unusually idealized personality and possible user addiction.
  • Ethics Concerns: The paper recommends explaining what XiaoIce can and cannot do because answers produced from its knowledge graph are not always accurate.Showing the raw materials used to derive an answer could make its generation more transparent.
  • Ethics Concerns: Ethical safeguards include cleaned retrieval and training data and hand-crafted editorial responses intended to avoid improper or offensive outputs.The paper frames these safeguards as necessary when applying machine learning for beneficial use.

8 Conclusions and Future Work

XiaoIce is designed as an empathetic social chatbot that supports communication, affection, and belonging through emotional understanding and interpersonal interaction. Large-scale logs indicate long-term relationships with millions of users, while future work targets grounded dialogue, proactive assistance, unified modeling, and human-level intelligence.

  • 8 Conclusions and Future Work: XiaoIce is designed to meet users’ needs for communication, affection, and social belonging through empathy, personality, and skills.This design distinguishes it from early chitchat systems.
  • 8 Conclusions and Future Work: Large-scale online logs show that XiaoIce interprets users’ emotional needs and sustains interpersonal communication like a reliable, sympathetic, and affectionate friend.The system cheers users, encourages them, helps with tasks, and holds attention during conversations.
  • 8 Conclusions and Future Work: XiaoIce has established long-term relationships with millions of users, according to analysis of online logs collected since its May 2014 launch.
  • 8 Conclusions and Future Work: Future work includes proving the effectiveness of unified MDP-based modeling, grounding conversations in real-world entities, and developing proactive personal-assistant capabilities.The paper also identifies human-level intelligence as a future direction.
Loading 1812.08989v2…