PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Self-Repairing Ubiquitous Robots Using Goal-Oriented Agentic AI in Human-Robot Interactions

Morten Roed Frederiksen

arXiv:2609.26155v1cs.RO

TL;DR

User-led repair of ubiquitous robots is difficult when procedures are conditional and static instructions offer little support for clarification or recovery. This paper presents a goal-oriented dialogue architecture that structures repair goals and tracks progress. In a physical task, 19 of 20 participants completed repair, while helpfulness remained high despite lower social presence and trust than online.

  • Problem

    User-led repair is difficult because ubiquitous robots can have hidden dependencies and branching procedures, while static manuals poorly support conditional instructions, clarification, and recovery.

    Source: §I · p. 1

  • Method

    The architecture separates goal decomposition, persistent state tracking, strategic goal management, and conversational execution to guide users through situated repair dialogue.

    Source: §I · p. 1, A. System Architecture · p. 1

  • Results

    19 of 20 participants completed the physical repair, and helpfulness remained high while perceived social presence and trust were lower than in the online comparison.

    Source: §I · p. 1, §IV · p. 3

  • Takeaways & Limitations

    The study supports a narrower capability than autonomous diagnosis: maintaining coherent repair dialogue while non-experts perform physical actions.

    Source: §IV · p. 4

    The evaluation involved only 20 participants and a constrained wiring task, and the online baseline differed in sample size and interaction medium, limiting causal claims about embodiment.

    Source: §IV · p. 4

Abstract

from arXiv · show

Ubiquitous robotic systems often lack traditional visual interfaces, making natural language interaction important for maintenance and repair. This paper presents a goal-oriented agentic AI architecture that enables non-expert users to complete technical repair tasks through situated dialogue. The architecture separates pre-interaction goal decomposition, persistent state tracking, strategic goal management, and real-time conversational execution. We evaluated the system in a physical hardware repair task with twenty participants. Nineteen participants completed the task, corresponding to a 95% completion rate. Participants rated the system as helpful and competent, and the agent remained robust to conversational diversions such as meta-queries and code-switching. A comparison with a prior online baseline showed that physical interaction significantly reduced perceived social presence, (p=.0005), and trust and competence, (p=.037), while perceived helpfulness remained high.

I. INTRODUCTION

User-led repair is difficult because ubiquitous robots can have hidden dependencies and branching procedures, while static instructions handle clarification and recovery poorly. The paper proposes goal-oriented agentic dialogue and evaluates it in a physical repair task, with most participants completing the task.

I. INTRODUCTION

  • Ubiquitous robots face everyday wear, but hidden dependencies, branching procedures, and technical concepts can make user-led repair difficult for non-experts.Static manuals and linear tutorials are poorly suited to conditional instructions, clarification, and recovery from partial progress or mistakes.Source: §I · p. 1
  • The architecture converts natural-language repair requirements into structured goals, tracks progress, and guides users through the active objective rather than a fixed troubleshooting script.The paper uses “self-repairing” for robot-initiated, user-mediated physical repair, not autonomous repair execution.Source: §I · p. 1
  • Separating high-level goal reasoning from low-latency conversation is intended to keep dialogue responsive while maintaining a repair plan through questions, clarifications, and diversions.The design addresses the tension between rigid scripts and open-ended dialogue that can lose track of task state or skip checks.Source: §I · p. 1
  • 19 of 20 participants completed the physical repair task, while the physical setting lowered perceived social presence and trust relative to the online version despite preserving high helpfulness.The evaluation used a spoken-dialogue repair task with an animated robot interface.Source: §I · p. 1

A. System Architecture

The architecture separates offline goal structuring, persistent state, active-goal selection, and real-time conversation. Its agents preserve conditional repair logic while allowing users to ask questions, revisit steps, or divert briefly.

A. System Architecture

  • The four-agent architecture separates offline task structuring, state persistence, high-level goal selection, and real-time dialogue.The goal decomposition agent creates the initial hierarchy; persistence stores conversation and task state; strategy selects the active goal; conversation produces user-facing responses.Source: A. System Architecture · p. 1, Figure 1 · p. 2
  • The Goal Decomposition Agent turns unconstrained repair instructions into a pre-interaction hierarchy of goals and subgoals.Goals encode conditional steps and completion status, preserving repair-manual structure without requiring a strictly linear script.Source: A. System Architecture · pp. 1–2
  • The Persistence Agent stores conversation history, task variables, and intermediate goal states so the system can remain coherent across clarifications and partial physical progress.The stored state helps avoid redundant instructions and recover when spoken progress and hardware state diverge.Source: A. System Architecture · p. 2
  • The Strategic Agent selects and verifies the active repair objective, while the Conversational Agent responds only for that objective and reports completion to the planner.This division supports non-linear dialogue and revisits while preventing the conversational layer from independently skipping ahead.Source: A. System Architecture · p. 2

B. Experimental Procedure and Testbed

The study tested spoken repair guidance on a physical wiring task with twenty university participants. The protocol combined a warm-up, guided repair, questionnaires, interaction logging, and comparison with a prior online version using the same goal logic.

B. Experimental Procedure and Testbed

  • Twenty university participants aged 18–44 interacted with an animated robot face using spoken input and verbal responses.The setup used face detection, microphone thresholding, Whisper-based speech-to-text, and text-to-speech; the face also blinked, smiled, and shifted gaze.Source: B. Experimental Procedure and Testbed · p. 2
  • The testbed required participants to report a sticker color, follow the corresponding wiring instructions, and confirm that the status lamp was active.The lamp lit only when the correct four wires were connected, combining a diagnostic condition, physical actions, and an observable completion signal.Source: B. Experimental Procedure and Testbed · p. 2
  • The procedure included a pre-study questionnaire, a two-minute warm-up, the agent-guided repair, and a post-study questionnaire, with objective interaction measures logged.One objective interaction log was missing and excluded from turn-count analysis.Source: B. Experimental Procedure and Testbed · p. 2
  • The physical study was compared with a prior online baseline that used the same architecture and branching logic but a virtual wiring task.Holding task logic constant was intended to examine how situated physical interaction related to performance and user perception.Source: B. Experimental Procedure and Testbed · p. 2

A. Task Completion and User Perception

Most participants completed the branched physical repair, and questionnaire responses were positive on helpfulness and competence. Interaction logs also show extended, variable conversations, including recovery from diversions and code-switching.

A. Task Completion and User Perception

  • 95% (19 of 20) completed the physical repair, verified by successful termination of the repair sequence and an active status lamp.The task combined a multi-step, conditionally branched procedure with physical hardware manipulation.Source: A. Task Completion and User Perception · p. 3
  • Trust and Competence averaged 4.87 on the 7-point scale, while Social Presence averaged 4.09 and perceived self-efficacy averaged 4.95.Prior AI familiarity was not significantly correlated with Trust and Competence (r = −.07, p = .775).Source: A. Task Completion and User Perception · p. 3
  • 19.7 mean turns (SD = 20.6) exceeded the theoretical optimum of 9 turns (t(19) = 2.27, p = .035), reflecting clarifications, social diversions, and physical clip manipulation.Code-switching occurred in 10.5% of sessions, while meta-queries or chit-chat occurred in 21.1%; the agent mirrored language or handled diversions before redirecting to repair.Source: A. Task Completion and User Perception · p. 3

B. Comparison with Online Baseline

Physical interaction preserved high helpfulness and perceived understanding relative to the online baseline, but users rated social presence and trust and competence lower.

B. Comparison with Online Baseline

IV. DISCUSSION AND CONCLUSION

The discussion presents the architecture as support for user-led physical repair, while distinguishing its feasibility evidence from autonomous diagnosis or broad validation across repair contexts.

IV. DISCUSSION AND CONCLUSION

  • 95% of participants completed the physical repair, supporting feasibility while users manipulated hardware, asked clarifying questions, and occasionally diverged.Source: §IV · p. 3
  • Persistent goal status and routing dialogue through the active objective create a middle ground between static manuals and unconstrained chatbots.Source: §IV · p. 3
  • Physical repair preserved perceived usefulness and high completion but lowered social presence and trust; authors suggest managing confidence in hardware consequences.They suggest safety statements, progress markers, confirmation prompts, or sensor-based verification.Source: §IV · pp. 3–4
  • The demonstrated scope is coherent repair dialogue using user-reported observations and persistent state, not fully autonomous diagnosis.Source: §IV · p. 4
  • Twenty participants and a constrained wiring task make the results feasibility evidence, while differences in online sample size and medium limit causal claims about embodiment.Source: §IV · p. 4
  • The authors frame situated natural-language user-led maintenance as supporting the sustainability of body-worn and ubiquitous technologies.Source: §IV · p. 4
Loading 2609.26155v1…