PaperlayerView exact revision on arXiv

Source-linked AI summary

More than Chit-Chat: Developing Robots for Small-Talk Interactions

Rebecca Ramnauth, Dražen Brščić, Brian Scassellati

arXiv:2412.18023v1cs.ROcs.AIcs.HC

TL;DR

Small-talk-capable robots are needed because current conversational systems can struggle to sustain natural, socially appropriate exchanges. The paper evaluates LLM small talk and develops an observer-based system that autonomously guides responses toward small-talk conventions. Across chatbot and human-robot evaluations, the system outperformed a baseline LLM and produced more natural, human-like dialogue, although the study reports a convenience-sampled participant population.

  • Problem

    Current LLMs and robots can redirect casual conversation toward task-oriented assistance instead of sustaining natural small talk.

    Source: §I · p. 1

  • Method

    The paper evaluates LLM small talk and develops an observer-based GPT-3.5 Turbo system that autonomously guides responses toward small-talk conventions.

    Source: §I · p. 1, B. Results · p. 3, §IV · p. 4

  • Results

    The system outperformed a baseline LLM in text chat and produced significantly more natural, human-like robot dialogue in online assessments.

    Source: §VII · p. 7

  • Takeaways & Limitations

    The system provides a framework for more authentic and contextually appropriate small-talk interactions between robots and users.

    Source: §VII · p. 7

Abstract

from arXiv · show

Beyond mere formality, small talk plays a pivotal role in social dynamics, serving as a verbal handshake for building rapport and understanding. For conversational AI and social robots, the ability to engage in small talk enhances their perceived sociability, leading to more comfortable and natural user interactions. In this study, we evaluate the capacity of current Large Language Models (LLMs) to drive the small talk of a social robot and identify key areas for improvement. We introduce a novel method that autonomously generates feedback and ensures LLM-generated responses align with small talk conventions. Through several evaluations -- involving chatbot interactions and human-robot interactions -- we demonstrate the system's effectiveness in guiding LLM-generated responses toward realistic, human-like, and natural small-talk exchanges.

I. INTRODUCTION

Small talk supports rapport, trust, comfort, and relationship-building, yet robots often redirect casual exchanges toward task-oriented assistance. This paper addresses that gap by developing a robot system for authentic, contextually appropriate small talk.

  • A care-home dialogue illustrates the problem: the robot answers friendly conversation with information and programmed assistance rather than reciprocal small talk.Source: §I · p. 1
  • Robots that respond naturally in small talk can foster rapport, enhance comfort, and make human-robot interaction more seamless.Source: Figure 1 · p. 1
  • Small talk builds rapport and trust, supports relationships, and serves professional networking and communication therapy.Source: §I · p. 1
  • Social robots could extend these benefits to communication therapy, professional development, and reducing social isolation beyond purely task-oriented assistance.Source: §I · p. 1
  • The paper presents a robot system designed to sustain authentic and contextually appropriate small-talk interactions with users.Source: §I · p. 1

II. BACKGROUND

Open-domain conversational agents make social chat increasingly feasible, but earlier small-talk systems relied heavily on scripts and lacked generalizable rules. The paper frames small talk as informal, light-hearted, socially purposeful, and governed by distinctive response characteristics.

A. Open-Domain, Conversational Agents

B. The Design of Small Talk Dialogue

III. EVALUATION OF CURRENT LLMS FOR SMALL TALK

The initial evaluation tested three LLMs in repeated small-talk conversations and assessed their responses against small-talk criteria and conversational motives. The design used randomized, blinded interactions, independent annotation, and preregistered procedures.

III. EVALUATION OF CURRENT LLMS FOR SMALL TALK

  • Three volunteers held 50 conversations with ChatGPT-3.5, Gemini Pro [33], and LLaMA-2 [28] using the same friendly-companion prompt.Source: §III · p. 2

A. Data Collection Procedures

B. Results

The evaluation found that LLM responses differed substantially from human small talk in style, motives, and conversational flow. GPT was the most human-like among the tested models, but its human-likeness was more variable, while qualitative feedback identified several recurring interaction failures.

B. Results

  • Across 150 conversations, the study produced 1547 annotated responses, with inter-rater agreement ranging from κ = 0.65 to 0.81 and ICC = 0.86 to 0.93.Source: B. Results · p. 3
  • LLM responses were less concise, slightly more positive, more specific, and less thematically coherent than human responses.Agent brevity was M = 4.55 versus human M = 1.23, while thematic coherence was M = 1.88 versus human M = 4.56.Source: B. Results · p. 3
  • LLM responses were more informative and assistive but less expressive and person-directed than human responses.Source: B. Results · p. 3
  • ChatGPT 3.5 was more concise, less specific, and more thematically coherent than LLaMA and Gemini Pro in pairwise comparisons.Source: B. Results · p. 3
  • GPT generated significantly more human-like responses than LLaMA and Gemini, but it also showed greater variability in human-likeness.The human-likeness scale ranges from 0, meaning no difference from human responses, to 4, the highest absolute difference.Source: B. Results · p. 3, Figure 2 · p. 4
  • The forgetfulness analysis modeled response order with conversation identifier and LLM name as random effects to examine whether performance changed across interactions.Source: B. Results · p. 3
  • Participants reported dead-end responses, excessive questions and topic shifts, emotional loops, and unbalanced advice that disrupted natural conversational flow.Source: B. Results · p. 4

IV. AN OBSERVER-BASED SYSTEM

The system addresses the gap between LLM conversational behavior and human small-talk expectations by using GPT-3.5 Turbo in an observer-based robot system. Its design targets succinct, expressive, appropriate, and relevant responses during small talk.

  • The system uses GPT-3.5 Turbo to develop a conversational system for small talk.Source: §IV · p. 4
  • The system targets a disparity between LLM conversational momentum and human expectations in small talk.Source: §IV · p. 4
  • Fig. 3 presents the architecture and processes that generate autonomous small-talk robot behaviors.Source: Figure 3 · p. 5

A. Monitoring Prompt Adherence

The observer monitors LLM responses against small-talk criteria because static prompts do not ensure dynamic, contextually appropriate behavior. It evaluates brevity, tone, specificity, coherence, and assistance-related language using feature-based measures.

B. Feedback Redirection

Feedback redirection lets an observer model detect deviations from small-talk criteria and guide the speaking model toward compliant responses. The mechanism combines corrective prompts with selective forced regeneration while monitoring system load.

  • Implicit feedback allows a response with corrective guidance, whereas forced feedback requires revision until the criteria are satisfied.Source: B. Feedback Redirection · p. 5
  • Forced feedback is used sparingly, with a maximum of three regeneration attempts, to facilitate timely responses.Source: B. Feedback Redirection · p. 5
  • The observer provides conversational feedback while maintaining system load, and Fig. 4 summarizes human-likeness comparisons between observer and base responses.Source: B. Feedback Redirection · p. 6, Figure 4 · p. 6

V. SYSTEM EVALUATION I: CHATBOT INTERACTIONS

The chatbot evaluation compared ChatGPT-3.5 responses with and without the observer across four small-talk criteria. Observer-guided responses were more human-like and less variable than base responses.

  • The evaluation used 50 observer-model conversations producing 499 transcribed responses, including 250 generated responses.The observer flagged 42.4% for implied feedback and 5.6% for forced feedback.Source: §V · p. 6
  • Observer responses were significantly more concise, positive, less specific, and more thematically coherent than base-system responses.Wilcoxon tests reported p ≤0.0001 for all four criteria.Source: §V · p. 6
  • Observer responses showed significantly less variability in human-likeness than base responses.The Brown-Forsythe test reported F′ = 15.47, p ≤0.0001.Source: §V · p. 6

VI. SYSTEM EVALUATION II: ROBOT INTERACTIONS

The system was deployed on Jibo for autonomous face-to-face small-talk interactions and evaluated against a base model in a within-subjects study. Participants generally viewed observer-model responses as more relevant and human-like, while response delays and embodied form shaped conversational impressions.

VI. SYSTEM EVALUATION II: ROBOT INTERACTIONS

  • The robot system used Jibo’s personified gaze, pose, and movement within a modular autonomous architecture running on ROS.A compact PC handled hardware communication and local data storage.Source: §VI · p. 6

A. In-Person Evaluation

  • Twenty-five participants completed three conversations with each model, producing 150 conversations and 1,725 responses in a randomized within-subjects evaluation.Each conversation lasted at least eight turns, and interactions averaged 40.5 minutes per participant.Source: A. In-Person Evaluation · p. 6
  • Twenty-three participants described observer responses as relevant, human-like, and natural, whereas 21 criticized the base model as overly assistive and verbose.Base conversations were characterized as “rambling,” “dry,” and “like speaking to a wall.”Source: A. In-Person Evaluation · p. 6
  • All six participants who commented on observer-condition delays described them positively, while all five who commented on base-condition delays described them negatively.The delays primarily arose from speech-to-text, text-to-speech, and possible Wi-Fi latency.Source: A. In-Person Evaluation · p. 6
  • Thirteen participants said Jibo’s animated, life-like movements affected conversational quality, although three noted that its embodied form lacked personality or experiences.The physical form was mostly viewed positively but did not consistently establish a convincing social identity.Source: A. In-Person Evaluation · p. 6

B. Online Evaluation

The online evaluation compared observer and base robot interactions using randomized video pairs and broad demographic participation. Observer behavior received significantly higher ratings across human-likeness, naturalness, responsiveness, and casualness, despite residual speaking delays and mixed reactions to embodiment.

B. Online Evaluation

  • The online study used five interaction-video pairs, randomized presentation, and 10-point Likert ratings from 100 participants aged 18 to 104.Participants rated human-likeness, naturalness, responsiveness, and casualness for observer and base conditions.Source: B. Online Evaluation · p. 7
  • Observer behavior scored higher than the base model on human-likeness (∆M = 1.53), naturalness (∆M = 1.50), responsiveness (∆M = 1.01), and casual chat (∆M = 1.84), all with p ≤0.0001.The comparisons used paired dependent t-tests across ratings from 100 online participants.Source: B. Online Evaluation · p. 7, Figure 5 · p. 7
  • Online feedback again distinguished the observer’s casual conversation from the base system’s fact-heavy and potentially condescending responses.Participants linked the base model to stating facts or being “somewhat snarky,” while describing the observer interaction as casual.Source: B. Online Evaluation · p. 7
  • Residual irregular delays from observer regeneration sometimes added perceived human-likeness, although participants also noted the delay remained after video editing.The observer was perceived as better mimicking human speech patterns and casualness despite the delay.Source: B. Online Evaluation · p. 7
  • Most participants viewed the robot’s life-like form positively, but some preferred the base model because its inanimate appearance seemed more consistent with lacking feelings.Embodiment therefore produced both affirming impressions and reservations about anthropomorphic presentation.Source: B. Online Evaluation · p. 7

VII. DISCUSSION

The discussion reports that feedback redirection improved small-talk behavior across text, in-person, and online evaluations. It presents the method as a practical framework for more natural robot dialogue while noting that its generalizability remains to be tested.

VII. DISCUSSION

  • The study introduces feedback redirection as a method for autonomously keeping LLM-generated responses aligned with small-talk conventions.The method is presented as enabling robots to sustain authentic and contextually appropriate small talk autonomously.Source: §VII · p. 7
  • Across three evaluations, the system outperformed a baseline LLM in text chat and produced more natural, human-like robot dialogue in online assessments.The in-person evaluation additionally showed robustness in real-world human-robot conversations and the inadequacy of an out-of-the-box LLM.Source: §VII · p. 7
  • The method’s effectiveness and generalizability across LLMs, robot platforms, behaviors, and domains remain open questions.Different LLM and platform designs may require different implementations, motivating future evaluation beyond ChatGPT 3.5 and Jibo.Source: §VII · p. 7

Related papers

Loading 2412.18023v1…