Source-linked AI summary
Toward Trustworthy Physical AI for Human Interaction
Niccolò Pagliarani, Maximilian Stölzle, Cosima du Pasquier, Jeff Lui, Andrew Sabelhaus, Cecilia Laschi, Gentiane Venture, Daniela Rus, Matteo Cianchetti
TL;DR
Robots entering human environments must be safe, understandable in their behavior, and clear about their capabilities, while avoiding both harmful contact and interaction-limiting overcaution. The paper proposes co-designing safety, behavioral intelligibility, and perceptual alignment; it argues that trustworthy interaction depends on their alignment and that evaluation must account for limits in human-facing estimates.
Problem
Robots can pose risks through harmful contact or failed safeguards, while avoidance-only behavior can make meaningful collaboration impossible.
Source: Safety · p. 5Method
The framework links safety, behavioral intelligibility, and perceptual alignment through co-designed embodiment, control, cognition, and communication.
Source: Safety · p. 8, Behavioral Intelligibility · p. 10, Perceptual Alignment · p. 12Results
The paper argues that trustworthy interaction depends on aligning the three dimensions, since changes in embodiment, control, or design can affect them differently.
Source: Outlook: Toward a Science of… · p. 17Takeaways & Limitations
Users’ expectations should be calibrated to demonstrated capability, with evaluation pairing perceived and measured interaction envelopes across repeated interactions.
Source: Perceptual Alignment · p. 14Predictability estimates remain proxies whose dependence on population, context, and feedback protocol should be tested on held-out conditions.
Source: Behavioral Intelligibility · p. 10
Abstract
from arXiv · showhide
Robots are entering human spaces faster than we can establish when they deserve trust. We propose a framework for trustworthy physical AI that integrates Safety, Behavioral Intelligibility, and Perceptual Alignment across embodiment, control, cognition, and design. Trustworthiness emerges from aligning physical capabilities, observable behavior, and expectations people form during interaction.
Safety
Safety requires bounding physical risk in a task- and user-specific interaction envelope, including when contact occurs or nominal operation is disrupted. The section connects that goal to contact modeling, uncertainty-aware control, recovery, and actionable communication.
Safety
- Safety must address both the likelihood and severity of harm, since avoidance-only systems can prevent useful collaboration while active safeguards can fail under uncertainty or model mismatch.Contact can still produce harmful loads through speed, inertia, geometry, stored energy, gravity, or feedback.Source: Safety · p. 5
- Safety claims should specify intended users, task, environment, permitted contact modes, and assumptions because acceptable limits depend on the person and context.The formalization focuses on blunt impact and sustained contact; other hazards require additional criteria, and robot-specific loads require a closed-loop contact model.Source: Safety · p. 5
- The contact equations connect robot inertia, actuation, feedback, and surface pressure to contact forces and injury severity through coupled robot-human dynamics.The model includes robot configuration, contact geometry, pressure distribution, and robot, human-tissue, and contact-history properties.Source: Safety · pp. 5–6
- A context-specific force margin is positive when the contact-force limit is satisfied, zero at its boundary, and negative when violated.Analogous margins can be defined for pressure, compression, transferred energy, or acceleration; guarantees remain conditional on contact and injury models.Source: Safety · p. 6
- Uncertain human motion and contact conditions call for conservative margins and actions with a positive predicted lower safety bound, or a defined fallback.Risk control also uses state estimation, motion prediction, and trials that vary noise, occlusion, and conditions outside training data.Source: Safety · pp. 6–7
- Tactile sensing can support contact detection and reflexes, while timely communication about intent and recovery complements mechanical and control safeguards.Cues should be validated for the intended users and environment.Source: Safety · pp. 7–8
- Safety design combines lower contact loads, maintained margins and recovery, and actionable cues, with nominal safety planning rather than boundary-only filtering.Embodied compliance and energy dissipation can reduce impact severity; active compliance shapes yielding interaction, while stability or energy guarantees do not necessarily establish an injury threshold.Source: Safety · pp. 7–8
Behavioral Intelligibility
Behavioral intelligibility concerns whether people can anticipate robot motion and infer its goal, but correct task execution alone does not ensure either. The section proposes ways to estimate these qualities from demonstrations and interaction, while emphasizing validation and recognizable behavior under ambiguity.
Behavioral Intelligibility
- Predictability asks whether motion matches expectations for a known goal, whereas legibility asks whether motion reveals that goal; a robot may be safe yet still move unpredictably or abruptly.A legible goal does not guarantee predictable motion, and hesitation may reflect caution, yielding, replanning, or failure.Source: Behavioral Intelligibility · p. 8
- Evaluation should test rare or extreme behavior, including unexplained lunges or force spikes, and stress-test occlusions, delays, ambiguous goals, and out-of-distribution conditions.Both demonstration-only and interaction-refined estimates remain proxies that require held-out testing against validated self-report and behavioral measures.Source: Behavioral Intelligibility · pp. 9–10
- Trajectory log-likelihood estimates predictability, while legibility measures how quickly motion leads an observer to recognize the actual goal.Exaggerated motion can improve legibility while reducing predictability.Source: Behavioral Intelligibility · p. 9
- Human-endorsed demonstrations can serve as evidence of expected robot motion, with a selected density or generative model representing that expectation distribution.This avoids assuming that observers share the robot’s task cost or mentally reproduce its optimizer.Source: Behavioral Intelligibility · p. 10
- Interactive learning adds localized evidence during robot action, using interventions and optional corrected motions to refine demonstration-based predictability estimates.The combined model reweights demonstration-derived trajectories toward motions consistent with feedback; corrected motions and uncorrected rollouts provide positive evidence, while intervened trajectories provide negative evidence.Source: Behavioral Intelligibility · p. 10
- Robot motion should carry the primary burden: keep nominal, replanning, and recovery behavior recognizable, and use timely cues to resolve ambiguity rather than mask unintelligible motion.Outside validated conditions, fallback should be simple; when ambiguity remains, the robot should yield or request clarification.Source: Behavioral Intelligibility · pp. 9–10
- Embodied, motor-cognitive, and communicative design levers should be co-designed rather than applied independently.The section frames interaction as a closed loop in which robot motion, human response, and sparse cues progressively clarify intent and upcoming action.Source: Behavioral Intelligibility · pp. 10–11
Perceptual Alignment
Perceptual Alignment concerns whether users accurately understand a robot’s capabilities, limits, reliability, and role, while forming an appropriate interpersonal stance. The framework connects this alignment to design, evaluation, and co-design for specific interaction contexts.
Perceptual Alignment
- Perceptual Alignment couples users’ accurate beliefs about a robot’s capabilities, reliability, uncertainty, and role with their comfort and perceived social relationship.These epistemic and social-affective layers are distinct but practically coupled; the same appearance, motion, softness, or social behavior can shape both.Source: Perceptual Alignment · p. 12
- Misleading cues can produce both over-reliance and underuse, and a soft or approachable form alone guarantees neither bounded forces nor transparent failure recovery.Underestimation can suppress interaction and leave capabilities undiscovered, while overestimation can lead to reliance beyond demonstrated capability.Source: Perceptual Alignment · pp. 12–13
- Expressive product design can build trust before users observe capability, but whether these design choices survive the first year of deployment remains untested.A brief interaction with a soft gripper was enough to significantly increase trust in robots; expectations set high can sharpen disappointment when faults occur.Source: Perceptual Alignment · p. 13
- Evaluation should compare perceived and measured interaction properties, prioritize behavioral evidence, and span repeated interactions rather than a single session.Relevant properties include confidence, sensing limits, and contact tolerance; reliance, compliance, monitoring, and intervention are more diagnostic than attitudinal ratings.Source: Perceptual Alignment · pp. 13–14
- Calibration should match appearance and cues to capability, deliberately exposing robot boundaries and using shared signals for capabilities and limitations.Structured capability-onboarding sessions can let users experience boundaries before unsupervised use; uncertainty, mode changes, and intervention requests should be salient.Source: Perceptual Alignment · p. 14
Putting the framework into practice
- For a hospital cup handover, practitioners specify group-specific safety limits and require predictable motion and cues that convey force protection accurately.The example accounts for vulnerable patients, visitors, and medical personnel, including their abilities, expectations, contact preferences, and proximity.Source: Perceptual Alignment · p. 16
- The handover assessment estimates contact loads and safety margins, calibrates motion uncertainty, tests predictability with participants, and compares expected with measured contact forces.It also records comfort and withdrawal during the handover and requires a positive conservative safety-margin bound or a defined fallback.Source: Perceptual Alignment · p. 16
- Co-design balances compliance with payload and pinch-point protection, tunes motion priorities under uncertainty, and adapts behavior and cues to patient comfort.The example proposes optional demonstrations when patients overestimate risk and adaptation to their comfort and preferences.Source: Perceptual Alignment · p. 16
Outlook: Toward a Science of Trustworthy Physical Interaction
The outlook treats trustworthiness as an emergent property of the whole robot, with design choices coupling safety, behavioral intelligibility, and perceptual alignment. It calls for integrated, context-specific evaluation and sustained appropriate reliance, supported by shared ways to make capabilities and limits legible.
Outlook: Toward a Science of Trustworthy Physical Interaction
- Trustworthiness emerges from the whole robot: changes to embodiment, control, or design can improve one dimension while shifting others.A compliant body may soften contact while making motion harder to regulate; adaptability may increase capability while reducing predictability, and expressiveness may invite unsupported expectations.Source: Outlook: Toward a Science of… · p. 17
- Longitudinal studies can examine how initial impressions develop into stable reliance patterns, while digital twins could support pre-deployment assessment.The proposed digital twins integrate contact dynamics with models of human expectation and interpretation.Source: Outlook: Toward a Science of… · p. 17
- The framework proposes that recurring signals of confidence, adaptation, and required participation could let experience with one robot inform interaction with another.Shared insight across platforms could connect robot design, evaluation, and standardization around a common language of physical interaction.Source: Outlook: Toward a Science of… · p. 17
- Table 1 presents figures of merit for evaluating trustworthy physical interaction; interpret measures as part of a multidimensional profile, not a universal score.The outlook calls for evaluation tied to explicit users, tasks, and operating conditions.Source: Table 1 · p. 18, Outlook: Toward a Science of… · p. 18
- Evaluation should report a multidimensional trustworthiness profile for explicit users, tasks, and operating conditions rather than collapse results into a universal score.Future benchmarks could measure the profile during realistic physical tasks and show how design choices shift dimensions together.Source: Outlook: Toward a Science of… · pp. 17–18
- Table 2 maps system-side trustworthiness dimensions in rows to technical design pillars in columns, with off-diagonal cells highlighting cross-pillar couplings.Diagonal cells indicate intuitive anchor relationships.Source: Outlook: Toward a Science of… · p. 19, Table 2 · p. 19