PaperlayerView exact revision on arXiv

Source-linked AI summary

RoboCaf\'e in the Open: Interaction Continuity in Long-Term Public Human-Robot Interaction

Kaitlynn Taylor Pineda, Kush Kumar Kushwaha, Jie Wang, Jiaming Du, Anvii Mishra, Emilie Basu Suri, Angela Guo, Chien-Ming Huang

arXiv:2609.27475v1cs.RO

TL;DR

Public service robots must preserve and correctly apply context as people and encounters change, yet an order’s task boundaries may not match its social context. RoboCafé combined coffee service, task-aware dialogue, perception, and prior-encounter memory in a 12-day deployment. Observations from repeated encounters motivate four requirements for interaction continuity: contextual interaction state, persistent person grounding, explicit interaction life-cycle management, and interaction observability.

  • Problem

    Public coffee-service orders provide task boundaries, but the surrounding social context does not necessarily begin and end with those orders.

    Source: §I · pp. 1–2

  • Method

    RoboCafé combined autonomous beverage preparation, task-aware dialogue, real-time multimodal perception, and prior-encounter memory in a 12-day deployment receiving 148 orders.

    Source: §I · p. 2

  • Results

    Deployment observations yielded four requirements: contextual interaction state, persistent person grounding, explicit interaction life-cycle management, and interaction observability.

    Source: §VII · p. 7, A. Interaction Membership is Dynamic · p. 7, B. Persistent IDs Needs More than… · p. 7

  • Takeaways & Limitations

    Maintaining public interaction continuity requires tracking participants, matching prior context to people, managing interaction boundaries, and checking internal state against the physical encounter.

    Source: §VII · p. 7

Abstract

from arXiv · show

As robots remain in public spaces over extended periods, they must maintain interaction continuity by preserving and correctly applying context as people, encounters, and circumstances change. To study interaction continuity in long-term public human-robot interactions, we developed RoboCaf\'e, an autonomous conversational coffee robot designed to support repeated interactions through task-aware dialogue, real-time multimodal perception, and memory of prior encounters. We deployed RoboCaf\'e for 12 days in a university building, where it received 148 orders. The deployment involved repeat customers, passersby, changing groups, and back-to-back orders that repeatedly crossed the boundaries assumed by the system's order-centered interaction model. We found that successful interaction continuity requires a robot to determine who is currently present, which prior context belongs to whom, where interactions begin and end, and whether its representation of an interaction matches what is occurring in the physical world. From these observations, we derive four system design requirements for maintaining interaction continuity in longitudinal public human-robot interactions: contextual interaction state, persistent person grounding, explicit interaction life-cycle management, and interaction observability.

I. INTRODUCTION

Interaction continuity means preserving and correctly applying context as people, encounters, and circumstances change. RoboCafé studies this problem through repeated public coffee-service interactions, where order boundaries do not necessarily match social boundaries.

I. INTRODUCTION

  • Interaction continuity is preserving and correctly applying context as people, encounters, and circumstances change.Public robots may meet people again and carry context across encounters, shifting the systems problem beyond completing a single task interaction.Source: §I · p. 1
  • Coffee orders offer a structured task boundary, but the surrounding social context may not begin or end with the order.The same customers may return, while people arrive together, interact around service, and move through the shared space.Source: §I · pp. 1–2
  • RoboCafé combined autonomous beverage preparation, task-aware dialogue, real-time multimodal perception, and prior-encounter memory in a 12-day deployment that received 148 orders.The deployment probe yielded four requirements: contextual interaction state, persistent person grounding, explicit interaction life-cycle management, and interaction observability.Source: §I · p. 2

A. Modern Human-Robot Conversations

  • Prior work reports that LLM-based agents can struggle with coherence, continuity, and context across sessions,.Recent advances have enabled more natural, open-ended dialogue in robots,, while studies of LLM-powered voice interfaces report recurring breakdowns.Source: A. Modern Human-Robot Conversations · p. 2

B. Interactions with Robots in Public Spaces

Public HRI unfolds amid unscripted participation, changing social conditions, and repeated encounters, while evidence about the same individuals interacting with conversational robots over time remains limited. RoboCafé is presented as a deployable platform for examining these conditions, with an initial design centered on one primary customer at a time.

B. Interactions with Robots in Public Spaces

  • Public interactions may be incidental, unscripted, group-based, or appropriated beyond a robot’s intended role, making the social environment part of public HRI.Prior deployments also report that people may leave conversations midway and test robot capabilities in unanticipated ways –.Source: B. Interactions with Robots in Public… · p. 2

C. Long-term and Repeated Human-Robot Interaction

  • Long-term deployments report changing engagement and expectations, but most studies focus on isolated encounters or aggregate users rather than repeated encounters with the same people.It remains unclear how increasingly conversational robots behave during sustained public use.Source: C. Long-term and Repeated Human-Robot… · p. 2

D. Socially responsible HRI

  • Public deployment raises social-responsibility concerns spanning safety, unpredictable responses, robot abuse, adversarial prompting, and privacy risks from conversational disclosures.The paper frames understanding robots’ effects on people and social environments as central to responsible public deployment.Source: D. Socially responsible HRI · p. 2

III. ROBOCAF´E

  • RoboCafé was developed as a deployable research platform for studying repeated conversational interaction in social coffee service.The research-owned platform was intended to support extended real-world operation while remaining adaptable to different research questions.Source: §III · p. 2
  • The system linked order-centered task execution with dialogue, live perception, and per-customer conversational summaries, while designing for one primary customer at a time.The dialogue used order and robot task state, presence and expression cues, and stored customer summaries; earlier conversations were converted into summaries for later context.Source: §III · pp. 3–4
  • The initial deployment excluded persistent biometric identity, co-presence as interaction membership, and multiparty speaker attribution.Face features were not retained over consent and storage concerns; people in view were not classified as group members, and accepted speech was not linked to a specific speaker.Source: §III · p. 4

IV. ROBOCAF´E AS A PROBE

RoboCafé served as a public barista-like probe for observing user behavior and service-robot conversational dynamics. The paper defines procedures and metrics for studying interaction continuity in such systems.

IV. ROBOCAF´E AS A PROBE

  • RoboCafé was developed to interact publicly with people placing orders, like a barista.The platform was intended as a probe into user behavior and social dynamics of conversation for service robots deployed in the wild.Source: §IV · p. 4
  • The study defines procedures and metrics for examining interaction continuity in public service-robot systems.Source: §IV · p. 4
  • The probe’s stated focus includes user behavior and the social dynamics of conversation in wild service-robot deployments.Source: §IV · p. 4

A. Interaction Study Procedure

The study ran on weekdays over three weeks, with orders limited to 11 AM–6 PM and recording consent required to order. It initially planned a donation-by-conversation-condition study, but the paper focuses on interactions before the donation message.

A. Interaction Study Procedure

  • The deployment ran weekdays from 11 AM to 6 PM for three weeks, and the kiosk accepted orders only during those hours.The kiosk automatically deactivated after 6 PM.Source: A. Interaction Study Procedure · p. 4
  • Users had to consent to recording to place an order, but publication permission for quotes or video was optional.Source: A. Interaction Study Procedure · pp. 4–5
  • The planned 2 × 2 study varied donation appeal and conversational style, with the donation request presented after beverage making.Appeals supported continued robot operation or a local charity, across social and task conversational conditions.Source: A. Interaction Study Procedure · p. 5
  • The paper dropped donation-behavior analysis because too few donations supported meaningful comparison, and focuses on interactions before the donation message.The authors identify the campus setting and the financial status of a majority-student population as likely reasons for the low donation count.Source: A. Interaction Study Procedure · p. 5

B. Analysis Definitions and Measures

The analysis defined orders, customers, repeat visits, encounter composition, and dialogue measures, then characterized them descriptively rather than experimentally.

Analysis Definitions and Measures

  • An order is one kiosk submission and its surrounding interaction; customer identities were reconstructed from video because email identifiers were unreliable.A unique customer had at least one coded order, and a repeat customer placed two or more coded orders; later orders by repeat customers were return visits.Source: B. Analysis Definitions and Measures · p. 5
  • A companion was someone the customer visibly interacted with, and an order counted as a group order if that person joined within 80 seconds and stayed until the order ended.The threshold was about 45% of the 180-second median order duration, requiring a companion to be present for at least 55% of the order.Source: B. Analysis Definitions and Measures · p. 5
  • Dialogue measures counted accepted transcripts answered, words in accepted transcripts, gated transcripts, and whether an order included any accepted transcript.The order window ran from the robot’s welcome message to its first order-ready message; because speakers were not identified, measures describe transcripts rather than specific people.Source: B. Analysis Definitions and Measures · p. 5
  • Encounter composition and visit history were described with counts, proportions, medians, and inter-quartile ranges, not treated as experimentally assigned comparisons.The analyses characterize observed interaction patterns rather than make causal comparisons.Source: B. Analysis Definitions and Measures · p. 5

V. DEPLOYMENT OBSERVATIONS

Three of 15 deployment days were excluded because of experimenter setup errors, leaving 12 days split evenly between social and task conditions.

Deployment Observations

  • Three of 15 deployment days were excluded for setup errors, leaving 12 days with six social and six task days.All social days fell in weeks 1 and 2, while all of week 3 ran the task condition.Source: §V · p. 5

A. Deployment Context

Across 12 deployment days, RoboCafé served 164 coffees across 148 orders placed by 108 unique customers.

Deployment Context

  • 148 orders from 108 unique customers produced 164 coffees over 12 days; 86 orders were on social days and 62 on task days.The median was 11 orders per day (13.5 on social days and 8 on task days); most orders contained one drink, and the median order window was 171.6 seconds.Source: A. Deployment Context · p. 5
  • Fig. 3 depicts repeat customer visits during the deployment.Source: Figure 3 · p. 6

B. Repeated Encounters and Identity

Repeat customers accounted for a substantial share of orders, but customer-provided identifiers often failed to preserve identity across visits.

Repeated Encounters and Identity

C. Group Encounters

Group encounters produced more speech and robot turns than solo encounters, but the system could not identify speakers. Deployment cases also exposed mismatches between the system’s order-centered model and encounters that crossed its assumed boundaries.

C. Group Encounters

  • Group orders had higher medians than solo orders for accepted transcripts (8 vs. 2), words in accepted transcripts (43 vs. 6), and robot turns (10 vs. 3), but speakers could not be identified.Group encounters accounted for 68 of 148 orders (45.9%), compared with 80 solo orders (54.1%).Source: C. Group Encounters · p. 6

D. Deployment Cases

  • Across four deployment cases, customers’ encounters with the system did not match the system’s interpretation.The cases included unusable identifiers, a group spanning multiple orders, and a user turning off the speaker.Source: D. Deployment Cases · p. 6
  • A regular visited on all 12 deployment days but was treated as new each time after entering a different gibberish identifier, leaving summaries under unrelated keys.The system required the same typed identifier to recognize a returning customer.Source: D. Deployment Cases · p. 6
  • Five people from one group placed five orders in 16 minutes, yet each order opened a new conversation despite one person appearing as a companion in three orders.The social encounter extended beyond an individual order’s life cycle.Source: D. Deployment Cases · p. 6
  • When a user turned off the speaker, dialogue continued and logged spoken turns even though the system did not observe that its output no longer reached the user.Dialogue state and the physical state of the output channel diverged.Source: D. Deployment Cases · p. 6

VI. DISCUSSION

The deployment surfaced challenges in maintaining interaction membership and identity across public encounters. The discussion calls for contextual interaction state and persistent person grounding as distinct capabilities.

VI. DISCUSSION

  • The bounded model relied on an order-linked primary customer and email-based recognition, while full multi-party dialogue was outside the deployment’s scope.The open public setting surfaced challenges including interaction membership, persistent memory and privacy, open interactions, and observability.Source: §VI · p. 6

A. Interaction Membership is Dynamic

  • A contextual interaction state should track who participates and how membership changes, because people may join or leave during orders, remain nearby between orders, or interact across back-to-back orders.Person detection alone cannot establish who is participating, who is speaking, or whether a newly detected person belongs to an ongoing interaction.Source: A. Interaction Membership is Dynamic · p. 7

B. Persistent IDs Needs More than Conversational Memory

  • Persistent person grounding should be separate from memory storage so a robot can determine whether prior information belongs to the person in the current interaction.Fifteen customers placed multiple orders, and identifiers were unusable in 60% of repeat orders; stronger identity mechanisms also raise privacy and consent considerations.Source: B. Persistent IDs Needs More than… · p. 7

C. Interactions Need Explicit Starts, Endings & Transitions

The paper distinguishes social interaction boundaries from service-task boundaries and calls for explicit interaction states. It also argues that deployed systems should expose whether their actions remain meaningful in the physical encounter.

C. Interactions Need Explicit Starts, Endings & Transitions

  • Interaction life-cycle management should distinguish social interaction progression from individual service-task boundaries.The deployment showed that an order is not necessarily equivalent to a social interaction.Source: C. Interactions Need Explicit Starts,… · p. 7
  • Interaction managers should represent start, continuation, transition, pause, and ending while preserving the broader social interaction across service tasks.Task-level state can be maintained alongside the wider interaction.Source: C. Interactions Need Explicit Starts,… · p. 7

D. Observability Must Extend to the Interaction

  • Interaction observability should reveal whether system actions remain meaningful in the physical interaction, not only whether individual components function.It should expose relevant state transitions and mismatches between system assumptions and the physical interaction.Source: D. Observability Must Extend to the… · p. 7

VII. CONCLUSION & FUTURE WORK

  • The deployment suggests that continuity requires tracking participants, linking context to the right person, representing interaction boundaries, and checking internal state against the physical encounter.Many properties could not be recovered reliably from system logs and required retrospective manual video analysis; speaker diarization alone would only partially address the problem.Source: §VII · p. 7
Loading 2609.27475v1…