Source-linked AI summary
Can People Distinguish Human and AI Agency in Humanoid Teleoperation? A Preliminary Study of Agency Perception
Xiang Li, Koya Dendo, Keigo Minamida, Yuto Nakamura, Per Ola Kristensson, Jun Rekimoto
TL;DR
The paper asks whether people can distinguish human from AI agency when both are presented through a consistent humanoid embodiment. It develops Ghost-in-the-Loop and evaluates short interaction clips; participants often struggled to distinguish the control sources. The findings motivate further study of agency perception and blended human–AI telepresence.
Problem
The paper asks whether people can distinguish human-operated from AI-generated behavior presented through a consistent embodied communication partner.
Source: §1 · p. 1Method
Ghost-in-the-Loop presents human-operated and AI-generated voice, facial animation, and gestures through a consistent humanoid embodiment, which 50 participants evaluated using short interaction clips.
Source: §1 · pp. 1–2, §3 · p. 2Results
Participants often struggled to distinguish the control sources in brief interactions, with judgments influenced by naturalness, temporal coordination, and cross-modal consistency.
Source: §1 · p. 2, §3.1 · p. 3Takeaways & Limitations
The preliminary findings motivate investigating agency perception in longer, more interactive encounters and dynamic human–AI control while preserving identity, transparency, and trust.
Source: §4 · p. 3
Abstract
from arXiv · showhide
Can people distinguish between human and AI agency in humanoid teleoperation? To explore this question, we developed \textit{Ghost-in-the-Loop}, a teleoperation framework that supports both human-operated and AI-generated control of a robot's voice, facial expressions, and gestures while maintaining a consistent embodiment. We conducted a preliminary online study ($N=50$) in which participants viewed short interaction clips generated by either a Human Operator or an AI Control and judged the perceived source of control. Results suggest that participants often struggled to distinguish between the two conditions in brief interactions. Qualitative responses indicate that judgments were primarily influenced by perceived naturalness, temporal coordination, and consistency across speech, facial expression, and gesture. These findings provide initial insights into agency perception in embodied human--AI communication and motivate future investigations of blended human--AI telepresence systems.
1 Introduction
Humanoid teleoperation can support rich remote communication, but continuous human involvement limits scalability. The section introduces Ghost-in-the-Loop to explore whether observers can distinguish human from AI control when both share an embodied interface.
1 Introduction
- Continuous human involvement in conventional teleoperation limits scalability and makes it difficult for one operator to support multiple users, robots, or interactions simultaneously.Recent generative AI advances raise the possibility of delegating parts of interaction while preserving a consistent communication partner.Source: §1 · p. 1
- Whether observers can reliably distinguish human-operated from AI-generated behavior through the same embodied interface, and which cues shape their judgments, remains unclear.Prior work has found AI behavior increasingly difficult to distinguish from human behavior in individual modalities.Source: §1 · p. 1
- Ghost-in-the-Loop supports switching between human-operated and AI-generated control while maintaining a consistent robotic embodiment.It combines voice cloning, facial animation, and speech-driven gesture generation through the same visual and behavioral interface.Source: §1 · pp. 1–2
- In an online study of 50 participants, observers often struggled to distinguish the two control sources in brief interaction clips.Judgments primarily drew on perceived naturalness, temporal coordination, and consistency across speech, facial expression, and gesture.Source: §1 · p. 2
2 Ghost-in-the-Loop
Ghost-in-the-Loop presents human and AI behavior through the same humanoid embodiment and output channels. Its AI pipeline generates speech, facial animation, and speech-driven gestures alongside the teleoperation components.
2 Ghost-in-the-Loop
- Ghost-in-the-Loop runs on a Unitree G1 with a facial display and camera, presenting human and AI control through shared voice, facial-expression, and upper-body-motion channels.The Local side contains the robot and participant-facing interface, the Remote side supports human teleoperation, and the AI side generates behavior.Source: §2 · p. 2
- Human control transmits speech, maps captured facial expressions to an ARKit-compatible blendshape representation, and teleoperates upper-body motion using a Meta Quest 3 setup.Speech uses LiveKit; facial capture uses MediaPipe Face-Landmarker.Source: §2 · p. 2
- AI control processes participant speech, synthesizes a cloned voice, generates facial animation, and produces hand gestures from speech using voice2hand.The outputs use the same presentation pipeline as human teleoperation; voice2hand derives from Speech2Gesture.Source: §2 · p. 2, Figure 2 · p. 2
3 Preliminary Evaluation
In a preliminary online evaluation using short clips, participants had limited ability to distinguish Human Operator from AI Control. Their judgments reflected perceived naturalness, temporal cues, and consistency across modalities.
3 Preliminary Evaluation
- The evaluation recruited 50 participants who viewed eight clips from Human Operator and AI Control conditions using the same robot, facial identity, voice identity, and recording environment.The clips covered task-oriented and casual conversation; participants also briefly explained their judgments.Source: §3 · p. 2
3.1 Findings
- Participants showed limited ability to distinguish Human Operator and AI Control conditions: the mean rating difference was 0.34 on a [−10, 10] scale (90% CI [−0.98, 1.67]); TOST with ±2-point equivalence bounds suggested practical equivalence (p= .021) in this short-video setting.Source: §3.1 · pp. 2–3
- Participants relied on perceived naturalness, temporal cues such as pacing and response timing, and consistency across speech, facial expression, and gesture.Rigid timing, repetitive movements, or cross-modal mismatches were associated with AI judgments, while coherent multimodal behavior was often judged as human-operated.Source: §3.1 · p. 3
- The findings suggest that human and AI agency can be difficult to distinguish through the same embodied interface, particularly in brief, structured interactions.Source: §3.1 · p. 3
4 Conclusion and Future Work
The conclusion presents Ghost-in-the-Loop as a framework for human-operated and AI-generated behavior through one robotic embodiment. Its preliminary study suggests people often struggle to distinguish the sources in short interactions, and it identifies longer, more interactive encounters as a direction for future work.
4 Conclusion and Future Work
- Ghost-in-the-Loop presents human-operated and AI-generated behavior through the same robotic embodiment.Source: §4 · p. 3
- A preliminary study with 50 participants suggests people often struggle to distinguish human from AI agency in short interactions.Source: §4 · p. 3
- Future work will examine agency perception in longer, more interactive encounters and dynamic combinations of human and AI control while preserving identity, transparency, and trust.Source: §4 · p. 3