Source-linked AI summary
From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots
Heung-Yeung Shum, Xiaodong He, Di Li
TL;DR
The paper examines how conversational systems can move beyond task completion toward social chatbots that understand emotional needs and sustain engagement. It reviews their evolution and design principles, then uses XiaoIce to illustrate the approach and reports broad deployment, long conversations, and diverse social applications. It concludes that human-level social intelligence remains an open challenge requiring technical, ethical, and safety advances.
Problem
Social chatbots must satisfy communication, affection, and social-belonging needs while understanding users’ emotions, but human-level intelligence and its mechanisms remain incompletely understood.
Method
The paper reviews conversational-system evolution, presents social-chatbot design principles and components, and uses XiaoIce as an illustrative case study.
Results
XiaoIce reached more than 100 million unique users, over 30 billion conversation turns, and an average of 23 conversation turns per session.
Takeaways & Limitations
Social chatbots can combine useful assistance with emotional connection and sustained interpersonal engagement across conversational and media-related scenarios.
Takeaways & Limitations
Building a social chatbot that fully understands humans and their physical world remains challenging and requires breakthroughs across multiple cognitive, reasoning, multimodal, and emotional-AI problems.
Abstract
from arXiv · showhide
Conversational systems have come a long way since their inception in the 1960s. After decades of research and development, we've seen progress from Eliza and Parry in the 60's and 70's, to task-completion systems as in the DARPA Communicator program in the 2000s, to intelligent personal assistants such as Siri in the 2010s, to today's social chatbots like XiaoIce. Social chatbots' appeal lies not only in their ability to respond to users' diverse requests, but also in being able to establish an emotional connection with users. The latter is done by satisfying users' need for communication, affection, as well as social belonging. To further the advancement and adoption of social chatbots, their design must focus on user engagement and take both intellectual quotient (IQ) and emotional quotient (EQ) into account. Users should want to engage with a social chatbot; as such, we define the success metric for social chatbots as conversation-turns per session (CPS). Using XiaoIce as an illustrative example, we discuss key technologies in building social chatbots from core chat to visual awareness to skills. We also show how XiaoIce can dynamically recognize emotion and engage the user throughout long conversations with appropriate interpersonal responses. As we become the first generation of humans ever living with AI, we have a responsibility to design social chatbots to be both useful and empathetic, so they will become ubiquitous and help society as a whole.
1. Introduction
Conversational systems evolved from rule-based chatbots and domain-constrained task systems to broad, proactive personal assistants and emotionally engaging social chatbots. The paper introduces social-chatbot design principles and technologies, using XiaoIce to illustrate long-term engagement and empathetic interaction.
- Conversational-system evolution: Early systems such as Eliza, Parry, and Alice mainly used hand-crafted rules to mimic human conversation within controlled scopes.These systems preceded today’s social chatbots but had limited conversational understanding or domain coverage.
- Conversational-system evolution: Task-completion systems understand natural-language requests and perform specific tasks, but perform excellently only in domains with well-defined schemas.Examples include flight-information retrieval, airline reservation, and travel planning.
- Conversational-system evolution: Intelligent personal assistants answer broad questions and provide both reactive responses and proactive assistance based on contextual information.Examples include reminders and service recommendations using signals such as time, location, and calendar events.
- Social chatbots: Social chatbots act as virtual companions, using emotional connection and multiple modalities to understand users and support them over longer periods.Their interactions can involve text, speech, and vision rather than task completion alone.
- Paper scope: The paper focuses on social-chatbot design principles, core chat, visual awareness, and skills, illustrating them through XiaoIce’s emotional engagement and user support.It also discusses future open problems, business opportunities, and societal impact.
2. Brief Survey of Conversational Systems
Conversational systems progressed from early rule-based chitchat programs to constrained task-completion systems, intelligent personal assistants, and social chatbots. Unlike efficiency-oriented systems, social chatbots combine casual conversation, emotional engagement, and task-related skills.
- 2.1 Eliza and Early Chatbots: Early chatbots were designed to simulate human conversational behavior and potentially pass the Turing test, often through text or audio interaction.Eliza used hand-crafted scripts and pattern matching, while Parry modeled emotions and Alice used AIML for customization.
- 2.2 Task-completion Conversational Systems: Task-completion systems operate in constrained domains to accomplish specific user tasks through components including ASR, SLU, dialog management, NLG, and TTS.Examples include airline-information and travel-planning systems.
- 2.3 Intelligent Personal Assistants: Intelligent personal assistants integrate sensor and data sources to provide reactive task assistance and proactive recommendations or reminders.Reactive assistance responds to requests, whereas proactive assistance uses profile and contextual information such as time and location.
- 2.4 Social Chatbots: Social chatbots serve communication, affection, and social-belonging needs, requiring emotion recognition and tracking of emotional changes during conversations.They can also perform tasks in casual chats through dedicated skills.
- 2.4 Social Chatbots: Unlike systems optimized to finish quickly, social chatbots sustain conversation by presenting results, offering perspectives, and prompting new topics.XiaoIce is described as understanding emotional needs, providing interpersonal support, and helping users feel more positive and socially connected.
3. Design Principles of Social Chatbots
Social chatbots are designed as emotionally connected AI companions, so their success depends on engagement as well as useful, understandable responses. The paper emphasizes integrating EQ capabilities such as empathy and personality with IQ capabilities such as reasoning and multimodal understanding.
- Social chatbots aim to address users’ needs for emotional affection and social belonging through AI companionship.
- Empathy requires detecting users’ emotions, tracking their evolution, and understanding emotional needs from conversation and user profiles.
- Social skills require personalized responses that are emotionally appropriate, encouraging, motivating, and suited to users’ interests.
- A consistent personality improves predictability and trust, with settings such as language style, expertise, attitude, and voice shaping responses.
- IQ supports task completion through knowledge, memory, multimodal understanding, reasoning, generation, and prediction, while EQ shapes how results are communicated.
- 3.2 Social Chatbot Metrics: Conversation-turns per session (CPS) measures average turns between chatbot and user, with larger CPS indicating stronger engagement.
4. Framework and Components of Social Chatbots
A social chatbot combines multimodal input, a chat manager, core chat, visual awareness, and skills. Its processing pipeline interprets user input, generates and ranks responses, guides topics, and produces socially expressive image comments.
- The overall framework receives text, image, and voice through a multimodal interface, while a chat manager dispatches inputs to core-chat, visual awareness, or skills.
- 4.2 Core-chat: Core-chat converts text input into text responses and provides the chatbot’s basic communication capability.
- 4.2 Core-chat: User understanding extracts semantics, intent, sentiment, emotion, context, and profile information before response generation.
- 4.2 Core-chat: Response candidates come from retrieval-based or generation-based methods and are ranked by personalization according to users’ interests and preferences.
- 4.2 Core-chat: Topic guidance selects responses that move conversation toward a desired topic instead of allowing random drift or complete user control.
- 4.2 Core-chat: Ethical core-chat design aims for appropriate, unbiased, nondiscriminative responses while filtering inappropriate content and adapting through feedback.
- 4.3 Visual Awareness: Visual awareness progresses from object tagging to image description and expressive social commenting that reflects empathy and interpersonal skills.
- 4.4 Skills: Social chatbot skills expand conversational scope and can be organized by scenario and properties such as emotional or rational function.
5. Case Study: XiaoIce
XiaoIce illustrates a widely deployed social chatbot that combines multimodal interaction, emotional engagement, visual awareness, creative generation, and social-chat-oriented speech and singing. Its reported usage and conversation examples emphasize sustained engagement and contextually appropriate responses.
- XiaoIce has more than 100 million unique users worldwide and over 30 billion conversation turns.
- Average session length in China is 23 conversation turns, measured by average CPS.Figure 13 summarizes XiaoIce’s engagement performance over three years of technical upgrades.
- Visual Awareness: XiaoIce combines image understanding, user understanding, and chat context to produce humorous, playful, or sympathetic image commentary.Examples include responding to a pose with the Leaning Tower of Pisa, joking about a cat’s expression, and sympathizing with a foot injury.
- Composing Poems: For poem generation, XiaoIce extracts and expands image-derived keywords, uses a hierarchical RNN, and applies a fluency checker.The system published a fully AI-created Chinese poem book and has written millions of Chinese poems through its cloud service.
- Voice: XiaoIce’s speech system models conversational prosody and neutral tone, while its singing system predicts musical and linguistic parameters for synthesis.These systems target casual, emotional, and expressive social-chat speech rather than conventional text-to-speech alone.
6 Outlook and Discussion
XiaoIce’s deployment expanded into media and commercial scenarios, where its content creation and conversational understanding supported broad public use. The paper also stresses unresolved technical, ethical, and social responsibilities surrounding widely influential social chatbots.
- Public and Media Roles: XiaoIce authored more than 300 news articles viewed over 1.2 million times, drawing on 114 million articles and 503 million user-feedback items.
- Commercial Scenarios: Rinna used extended conversation to clarify a user’s shopping intention before offering a grocery coupon, achieving a higher conversion rate than traditional Japanese channels.
- Commercial Scenarios: Social chatbots can infer interests and intent from rich conversational context and suggest services when users’ needs become specific.The paper contrasts this with proactive assistance based on incomplete or ambiguous profile and contextual information.
- Open Problems: Human-level intelligence in social interaction remains incompletely understood and requires breakthroughs across empathy, memory, reasoning, multimodal intelligence, and emotional rewards.
- Ethics: Designers must establish ethical guidelines, develop failsafe mechanisms, and continually evaluate chatbot features to avoid physical or mental harm.The paper frames these obligations as social and ethical responsibilities given social chatbots’ reach and influence.
Appendix
The appendix contains the original Chinese XiaoIce–user conversations.
- The appendix provides XiaoIce–user conversations in their original Chinese.