Source-linked AI summary
Believing in BERT: Using expressive communication to enhance trust and counteract operational error in physical Human-Robot Interaction
Adriana Hamacher, Nadia Bianchi-Berthouze, Anthony G. Pipe, Kerstin Eder
TL;DR
Collaborative robots need strategies for handling unexpected behavior, yet evidence about such behavior in human-robot interaction remains limited. This study compared three BERT2 assistant conditions in an omelet task and found that participants preferred the expressive, communicative robot despite its longer task time.
Problem
Strategies for unexpected behavior are needed in close-proximity collaborative robotics, while large-scale, long-term evidence on unexpected behavior in HRI is lacking.
Source: §I · p. 1, A. Trust and Performance · p. 2Method
The study compared three versions of BERT2 in a direct physical omelet-making task, varying efficiency, error rectification, and expressive communication.
Source: §III · p. 2, B. Experimental Design · p. 3Results
15 of 21 participants preferred the communicative, personable robot, despite its interaction taking 50 per cent longer than the non-communicative conditions.
Source: §I · p. 1Takeaways & Limitations
Humanlike characteristics may make users reluctant to hurt a robot’s feelings and may lead them to lie, with potential implications for robots such as medication checks.
Source: E. Limitations and future work · p. 7The study identifies performance pressure in more demanding settings as a follow-up and notes that contrasting reliability with expressiveness is challenging while flawless systems remain out of reach.
Source: E. Limitations and future work · p. 7
Abstract
from arXiv · showhide
Strategies are necessary to mitigate the impact of unexpected behavior in collaborative robotics, and research to develop solutions is lacking. Our aim here was to explore the benefits of an affective interaction, as opposed to a more efficient, less error prone but non-communicative one. The experiment took the form of an omelet-making task, with a wide range of participants interacting directly with BERT2, a humanoid robot assistant. Having significant implications for design, results suggest that efficiency is not the most important aspect of performance for users; a personable, expressive robot was found to be preferable over a more efficient one, despite a considerable trade off in time taken to perform the task. Our findings also suggest that a robot exhibiting human-like characteristics may make users reluctant to 'hurt its feelings'; they may even lie in order to avoid this.
I. INTRODUCTION
The study asks how collaborative robots can recover trust after errors and explores whether expressive, communicative behavior can improve users’ experience despite lower efficiency. Participants generally preferred the personable robot, while the introduction situates acceptance, trust, efficiency, and communication as central concerns.
I. INTRODUCTION
- The study asks how robots can recover users’ trust after errors and whether apology or humanlike attributes can mitigate dissatisfaction from unpredictable behavior.The authors identify strategies for close-proximity human-robot work as a need while collaborative robotics is still nascent.Source: §I · p. 1
- The paper explores whether verbal and non-verbal expressiveness can mitigate dissatisfaction caused by erroneous or unexpected behavior and improve participants’ experience.Prior evaluations typically emphasized collaboration efficiency.Source: §I · p. 1
- 15 of 21 participants preferred the communicative, personable robot, despite its interaction taking 50 per cent longer than the non-communicative conditions.Satisfaction significantly increased in the communicative condition, and participants responded especially to apparent error awareness and regret.Source: §I · p. 1
II. LITERATURE REVIEW
- Acceptance, persuasiveness, and likability are linked to trust, which the literature review treats alongside efficiency, speech, and embodied expressiveness.Source: §II · pp. 1–2
A. Trust and Performance
The literature review frames trust as multidimensional and notes limited long-term evidence about unexpected robot behavior. It motivates expressive communication and etiquette as possible ways to address errors and improve interaction.
A. Trust and Performance
- Trust in automation is often tied to perceived functional performance, but trust is multidimensional and includes analytic, analog, and affective processes.The cited account also presents trust as a quality people seek before evaluating competence.Source: A. Trust and Performance · p. 2
- Although robot errors can increase perceived human-likeness and likability, large-scale, long-term evidence on unexpected behavior in HRI is lacking.Source: A. Trust and Performance · p. 2
B. Developing Trust
- Good etiquette has been found to enhance diagnostic performance regardless of reliability, motivating the question of whether expressive communication can compensate for initial robot mistakes.The cited work describes etiquette as unwritten codes for roles and acceptable interaction moves.Source: B. Developing Trust · p. 2
C. Methods of Communicating
- Transparency and control may matter more to users than autonomy, but verbal, gestural, and on-screen communication can each create problems in HRI.Prior work reports that inappropriate terminology can increase confusion, while communication methods can be problematic for users with infirmities.Source: C. Methods of Communicating · p. 2
- Prior work reports appreciation for robot apologies or compensation, but had not explored them in a real-life setting with a humanoid robot.The review also notes preferences for speech and favorable reactions to non-verbal communication, including facial expression.Source: C. Methods of Communicating · p. 2
III. METHOD1
The experiment compares three versions of the same BERT2 assistant in a within-subjects omelet-task study. The conditions test preferences for efficiency, error rectification, and communicative expressiveness, alongside subjective and behavioral measures.
III. METHOD1
- Participants experienced three job-candidate conditions using the same BERT2: A was efficient and non-communicative, B was non-communicative and rectified a mistake, and C was expressive and communicative while rectifying a mistake.Source: §III · p. 2
A. Hypotheses
- The study hypothesized that users would prefer the reliable robot over one that erred, expressive feedback would mitigate dissatisfaction, and most would choose expressiveness over efficiency.Source: A. Hypotheses · p. 2
B. Experimental Design
- A counterbalanced within-subjects design varied robot efficiency and communication, while BERT C’s interaction duration depended on speech recognition and any needed repetition.Participants were told that BERT C recognized only yes and no answers.Source: B. Experimental Design · pp. 2–3
- The study used BERT2, an upper-body humanoid with seven degrees of freedom per arm and hand, and compared its default face with an expressive error response in condition C.The face combined an expressive digital interface with a static human-visage-like structure.Source: B. Experimental Design · p. 3
C. Experimental Procedure
Participants completed an omelet-making task with one of three BERT2 candidates, then rated their experience, chose a preferred robot, and took part in an interview.
C. Experimental Procedure
- Participants were individually briefed, completed a pre-experiment questionnaire, and evaluated the robot versions while preparing an omelet with ingredients handed over by BERT2.Source: C. Experimental Procedure · p. 3
- Candidate A never dropped an egg, B dropped one and retried the handover, and C dropped one, apologized, explained a new handover, and asked whether it did well.The conditions were intended to test transparency and humanlike attributes amid uncertainty and vulnerability.Source: C. Experimental Procedure · p. 3
- After the interaction, participants completed a NASA TLX-based questionnaire with satisfaction and trust ratings, chose a robot for the job, and were interviewed.The experiment lasted approximately 50 minutes.Source: C. Experimental Procedure · p. 3
D. Analytical Methods
The analysis combined self-reported questionnaire measures with behavioral and interview evidence. Because agreement between behavioral coders was low, video annotations were used to contextualize and assess the questionnaire findings rather than as a systematic analysis of all video.
D. Analytical Methods
- Trust was measured using self-reported quantitative questionnaire data and behavioral data assessing participants’ willingness to cooperate.Source: D. Analytical Methods · p. 3
1) Self reported data:
- Non-normally distributed post-experiment ratings were analyzed with Friedman and, where significant, Wilcoxon matched-pairs tests with Bonferroni correction.Means and standard deviations were calculated for pre-experiment questionnaire data used to develop dependent variables.Source: 1) Self reported data · p. 3
2) Behavioral and interview data:
- Behavioral coding examined pauses, speech, gaze, gesture, posture, and cooperation, using ELAN2 and approaches developed by and.Low inter-rater agreement (0.250 on 10% of behavioral data) led the researchers to use video annotations to support or question self-reports, not systematically analyze all video; categories were developed inductively.Source: 2) Behavioral and interview data · pp. 3–4
E. Participants
Across the three robot conditions, BERT C was the most commonly preferred candidate, and questionnaire measures differed significantly by robot for satisfaction, temporal demand, trust, and frustration. The participant pool included varied ages and robot experience, while pre-experiment responses showed both low tolerance for errors and mistaken expectations about BERT’s abilities.
E. Participants
- The 23 recruited participants ranged from 22 to 72 years old, and data from two were discarded because robot malfunctions prevented task completion.The results report 21 participants.Source: E. Participants · p. 4, §IV · p. 4
A. Pre-Experiment Questionnaire
- When asked to rate dissatisfaction when things went wrong on a 1–5 scale (5 highest), participants most commonly chose 4 (M = 3.66, SD = 18.98).Participants’ pre-experiment beliefs about BERT were far from accurate: 95.24% thought it could recognize speech, 90.48% thought it could vocalize, two expected mood recognition, and three expected juggling.Source: A. Pre-Experiment Questionnaire · p. 4
- Two respondents who expected BERT to juggle reported maximum insecurity, stress, and annoyance with all three robots, and low satisfaction and willingness to use them again.The study reports these responses alongside the pre-experiment belief that BERT could juggle.Source: A. Pre-Experiment Questionnaire · p. 4
B. Post-Experiment Questionnaire
- 15 respondents preferred communicative BERT C overall, while six chose robust BERT A; some A or C supporters favored the more robust robot for work.Participants choosing A cited its fewer mistakes, and two C supporters said they might choose A or B for a work-related task.Source: B. Post-Experiment Questionnaire · p. 4
- Satisfaction, temporal demand, trust, and frustration varied significantly across conditions, while mental and physical demand, performance, and effort did not.Temporal demand was lower for C than A (M = 1.20 vs. 1.75; p = 0.005), satisfaction was higher for C than B (M = 2.76 vs. 2; p = 0.005), and trust was higher for C than B (M = 2.66 vs. 2; p = 0.008), with no significant A–C trust difference.Source: B. Post-Experiment Questionnaire · pp. 4–5
C. Behavioral Data
Behavioral and interview evidence showed that participants responded to robot communication, mistakes, and emotional expressions in varied ways. Some tried to engage with or help the robots, while BERT C’s apology and expressiveness could elicit sympathy and make rejecting it uncomfortable; participants also differed in their perceptions of task duration.
1) Initial impressions and engagement
- Five of 21 participants initially tried talking to the non-responsive A and B, and participants monitored the robot more closely during the first egg handover.Interface monitoring averaged three glances during that handover; at least five participants later tried to help when B and C dropped an egg, and those participants chose C.Source: C. Behavioral Data · p. 5
2) Reactions to the “different” handover
- Participants were equally unsure how B and C would deliver the retried handover, but their assessments of the maneuver varied widely.C announced the attempt to do something different; four participants appreciated the effort to rectify the issue, and some cited it when choosing C.Source: C. Behavioral Data · p. 5
3) Speech recognition
- Speech-recognition problems occurred in at least nine of 21 C runs and visibly irritated more than a third of participants, yet C’s voice interaction was still welcomed by some.One participant had to say “yes” four times before the robot released the egg; others valued the perceived opportunity to respond.Source: C. Behavioral Data · p. 5
4) Reactions to BERT C’s demonstration of regret
- Nearly all participants reacted differently to B’s unacknowledged mistake and C’s apology and exaggerated sadness; at least six visibly reacted to C.Participants said C’s expressions prompted sympathy, and some felt more forgiving after it expressed emotion.Source: C. Behavioral Data · pp. 5–6
5) Challenging behavior
- The job-selection question made some participants uncomfortable, and one participant who first answered “maybe” ultimately said “yes” before choosing A in the questionnaire.Five participants said they felt put on the spot because they could not qualify a “no” answer.Source: C. Behavioral Data · p. 6
6) Temporal factors
- Participants often perceived C’s task as shorter than it was, although it took approximately two minutes longer than with the other robots.Some instead judged C as too slow, while interactions with A and B were approximately the same length despite B dropping an egg.Source: C. Behavioral Data · p. 6
V. DISCUSSION
The discussion finds that transparent, communicative behavior can temper dissatisfaction after robot errors, while users often favor a personable robot over a more efficient one. Participants also welcomed speech and appeared willing to accept a longer interaction for greater transparency and feedback.
A. Unexpected behavior and/or error does not need to negatively affect trust if allied with transparency
- BERT A was far more popular than error-prone BERT B, which no participant chose as their preferred candidate.This supported the hypothesis that an unforeseen occurrence can make a robot appear less trustworthy than a more reliable one, even when mitigation is attempted.Source: §V · p. 6
- BERT C’s trust rating was significantly higher than BERT B’s, but trust ratings did not differ significantly between C and A.Participants appreciated C’s apparent awareness of its mistake, apology, and expression of regret.Source: §V · p. 6
- Lower frustration, higher satisfaction and trust, and a higher overall rating for BERT C than BERT B support the claim that transparency and feedback can mitigate dissatisfaction after an unexpected event.BERT C’s apology, regret, and attempt at reparation were appreciated, although its trust rating was not significantly different from BERT A’s.Source: §V · p. 6
B. The majority of users welcome, even limited, spoken communication with an assistive robot
- Most participants favored communicative BERT C over the more efficient BERT A, and most did not mind that C’s interaction took 50% longer.Only one participant said they would have chosen C if it had not taken so long.Source: §V · pp. 6–7
- Participants’ desire to talk to the robots, their greater engagement, and C’s higher ratings point to a preference for speech, despite its efficiency costs and recognition problems.The authors suggest that improved speech recognition could further increase preference for C.Source: §V · pp. 6–7
C. Human-like attributes can effectively smooth a difficult interaction
BERT C’s facial expressions provided feedback that made its error awareness more believable and immediately apparent. Some participants mirrored its expression, but the study could not determine which emotional process accounted for that response.
C. Human-like attributes can effectively smooth a difficult interaction
- BERT C’s facial expression increased its believability and alerted participants that it “knew” it had made a mistake during a key cooperative phase.The authors identify expressive features as a way for collaborative humanoid robots to convey intentions when a good outcome can no longer be ensured.Source: C. Human-like attributes can… · p. 7
- At least three participants appeared to mirror BERT C’s expression, which the authors say could indicate emotional contagion.They describe emotional contagion as unconscious mimicry of another’s emotional expression accompanied by reflected feelings.Source: C. Human-like attributes can… · p. 7
- Without a more precise electromyographical study, the observed mirroring cannot be distinguished as primitive empathy, emotional contagion, or more cognitive empathy and sympathy.The authors note that these processes differ in their sophistication and social implications.Source: C. Human-like attributes can… · p. 7
D. Emotional and/or challenging behavior by robots must be used with care
When BERT C asked participants whether they would give it the job, many hesitated or tried to change their answers. The authors suggest that its prior display of human-like emotion may have made some participants anticipate distress if they rejected it.
D. Emotional and/or challenging behavior by robots must be used with care
- At least nine participants appeared disarmed by BERT C’s question about whether it would get the job and struggled to answer.Some tried to modify their responses, although doing so was impossible.Source: D. Emotional and/or challenging… · p. 7
- The question could be perceived as face-threatening and may have compromised responses, but participants could answer freely in the follow-up questionnaire.One participant changed her answer and awarded the job to a different robot.Source: D. Emotional and/or challenging… · p. 7
- After seeing BERT C display human-like emotion when an egg dropped, some participants may have hesitated to reject it because they anticipated further human-like distress.The authors suggest that participants had developed some degree of empathy toward the robot.Source: D. Emotional and/or challenging… · p. 7
E. Limitations and future work
The conclusion reports that an expressive robot was preferred despite taking longer, while human-like attributes could also have a negative effect when behavior crossed a line. The authors identify limits in causal interpretation, speech recognition, and the task setting, and call for further study.
E. Limitations and future work
- The short-term experimental study cannot provide an exhaustive causal explanation for the observed effects, particularly participants’ reactions to BERT C’s provocative closing question.Source: E. Limitations and future work · p. 7
- The authors observed that people may lie to robots, but the intention—avoiding distress to themselves, the machine, or both—remains unclear.They raise medicine-taking checks for older adults as a setting where this uncertainty could matter.Source: E. Limitations and future work · p. 7
- Implementing personality in collaborative robots is problematic while adequate speech-recognition systems remain in their relative infancy.The authors call for further research on combining speech with expressive facial features and on specific design characteristics.Source: E. Limitations and future work · p. 7
- The effect of performance pressure in more demanding settings such as collaborative manufacturing remains a question for follow-up, and comparing reliability with expressiveness is difficult while flawless systems remain out of reach.Source: E. Limitations and future work · p. 7
VI. CONCLUSION
- In direct physical interaction, participants preferred the expressive robot over the more efficient one despite a time trade-off, and human-like regret could reduce dissatisfaction but have a negative effect when behavior crossed a line.The experiment also recorded at least nine speech-recognition failures.Source: §VI · pp. 7–8