Source-linked AI summary
Better Situational Awareness in AR-HRC? A Comparative Study of Augmented Reality and Mobile Interfaces for Human-Robot Collaboration
Zhehan Qu, Christian Fronk, Jaewoong Jeong, Pavel Manakhov, Maria Gorlatova
TL;DR
In safety-critical HRC, AR may reshape visual attention in ways that affect both robot and environmental situational awareness, while prior evidence has been limited. This study compares equivalent AR and mobile interfaces using SAGAT and eye tracking, finding a selective robot-awareness benefit without broader SA gains. The findings indicate that moving attention away from a map does not necessarily direct it toward the physical environment.
Problem
Prior AR-HRC evaluations provided limited rigorous evidence jointly assessing robot and environmental SA across perception, comprehension, and projection.
Method
A between-subjects study with 30 participants compared custom AR and an information-equivalent mobile interface using SAGAT probes and concurrent eye tracking in a SAR-inspired HRC task.
Results
AR improved perception-level robot SA but produced no gains in higher-level robot SA or environmental SA relative to the mobile baseline.
Takeaways & Limitations
AR’s SA benefits were selective because freed attention was re-invested in conformal visuals rather than the physical environment.
Takeaways & Limitations
The Magic Leap 2’s restricted peripheral field of view may have hindered environmental awareness, although the same headset was used in both conditions.
Abstract
from arXiv · showhide
Augmented reality (AR) facilitates human-robot collaboration (HRC) by enabling in-situ spatial visualizations of the robot and the joint task. However, in safety-critical HRC scenarios such as search-and-rescue, spatial visualizations may also reshape visual attention in ways that create competing situational awareness (SA) demands, potentially introducing new safety concerns. While prior AR-HRC work suggests potential benefits for SA, rigorous evaluations that jointly consider robot and environmental awareness across multiple levels of SA remain limited. We address this through a between-subjects study with 30 participants comparing custom AR and mobile interfaces presenting equivalent information, measuring robot and environmental SA with the Situation Awareness Global Assessment Technique (SAGAT) across all three levels, with concurrent eye tracking to identify the attentional mechanisms underlying any SA differences. Both interfaces achieved high usability; relative to the mobile baseline, AR improved perception-level awareness of the robot but yielded no gains in higher-level robot awareness or in environmental awareness at any level. Gaze analysis explained this: AR freed attention from the map, but that attention was re-invested in the conformal visuals rather than the physical environment. Freeing the eyes from a screen is not the same as directing them to the world, a distinction AR interfaces for safety-critical HRC must design around.
1 INTRODUCTION
The study examines whether AR preserves and improves robot and environmental situational awareness in dynamic, safety-critical HRC. It compares AR with an information-equivalent mobile interface and uses SAGAT plus gaze analysis to identify selective benefits and their attentional basis.
- AR spatial visualizations can communicate robot state, intent, workspace constraints, and task objectives in situ, but must also preserve situational awareness for safe collaboration.
- Spatially conformal AR may ease attention division between the robot and the physical world, yet its visuals can compete with hazards and unexpected obstacles.
- Prior evaluations often relied on usability, performance, or robot-only measures rather than structured assessments spanning targets and SA levels in dynamic HRC.
- The study compares custom AR with an information-equivalent mobile baseline in a SAR-inspired task involving dynamic hazards and constrained visibility.Thirty participants completed the between-subjects study; SAGAT probes assessed SA and eye movements addressed attentional allocation.
- AR improved perception-level robot SA but not higher-level robot SA or environmental SA relative to the mobile baseline.
- Gaze analysis linked AR’s selective effects to attentional allocation, including more active visual exploration and attention drawn toward conformal visuals.
2 RELATED WORK
Related work presents AR as a way to embed robot and safety information in shared workspaces, while highlighting limited rigorous evaluation of both robot and environmental SA across Endsley’s levels. Eye tracking offers a means to study the attention mechanisms involved.
- AR-HRC systems have supported safety, trust, ergonomics, robot-state understanding, intent communication, and collaborative-task performance.
- AR search-and-rescue systems have often treated SA as information provided by the interface or measured it through subjective feedback, without comprehensive robot and environmental assessment.
- SA comprises perception of environmental elements, comprehension of their significance, and projection of their future states.
- Prior AR-HRC research has focused mainly on robot SA and used coarse measures rather than established frameworks such as Endsley’s model or SAGAT.
- Spatially conformal graphics are proposed to divide attention between interface content and the physical world more effectively than screen-fixed displays.
- The study layout includes physical room halves, table polygons, real and fake stations, and separate user and robot paths.
- Eye-tracking measures such as fixations, saccades, gaze transitions, and stationary entropy characterize visual exploration, cognitive load, and attention distribution.
3 STUDY DESIGN
The study design tests AR’s effects on robot and environmental SA across three levels and examines whether gaze differences explain those effects. SAGAT freeze probes and continuous eye tracking provide the corresponding outcome and attentional measures.
- RQ1 compares spatially conformal AR with an information-equivalent mobile interface for robot and environmental SA in a dynamic HRC task.
- Robot and environmental SA are evaluated across perception, comprehension, and projection using SAGAT freeze probes.
- RQ2 examines how AR reallocates visual attention and whether attentional differences explain its effects on SA.
- Participants’ eye movements are continuously recorded to analyze gaze dynamics, attention allocation, and fixation transitions.
3.1 Study Task and Environment
The SAR-inspired task places participants and a robot in a cluttered, partially observable workspace with independent but interdependent duties. Dynamic station discovery, navigation, subtasks, occlusions, and fake stations create ongoing demands for joint situational awareness.
- The robot explores the environment and scans stations while participants navigate to stations and complete word-search subtasks.
- Participants are asked to maintain awareness while operating in a hazardous, partially observable environment and moving between stations.
- A 9.22×7.12 m room was configured as a maze-like obstacle course with tables, chairs, boxes, occlusions, and constrained movement.
- The robot continued exploring while participants completed subtasks lasting between 30 and 60 s.
- Of 10 stations, 5 were real and 5 were visually identical fake stations, preventing participants from anticipating destinations before scanning.
- The design required participants to integrate robot discoveries, personal navigation, and environmental information throughout the joint task.
3.2 Interface Design
The study developed information-equivalent mobile and AR interfaces supporting navigation and robot monitoring, while exploiting each platform’s distinct affordances. The AR design combined spatially registered visuals, a persistent minimap, and off-screen robot cues.
- Both interfaces provided equivalent information and functionality for navigation to stations and monitoring the robot’s state, location, and next actions.
- The mobile interface used a user-centered 2D top-down map showing the environment, obstacles, user and robot positions, paths, and subtask stations.
- The AR interface combined spatially registered conformal visuals, a persistent minimap, and an off-screen robot indicator for directional cues.
- AR robot cues included an occlusion-resistant red sphere, a 4.5 m trajectory visualization, and a directional paw indicator when the robot was outside the field of view.
- AR navigation cues used scan-progress indicators, completed-station markers, and static ground waypoints for user and robot paths to reduce flicker and visual noise.
- The AR minimap matched the mobile map, remained aligned with the user’s heading, and was placed at the headset’s bottom-right to reduce focus switching.
3.3 Evaluation of Situational Awareness
Situational awareness was evaluated with freeze probes during dynamic navigation and robot-monitoring events. The design jointly assessed robot and environmental awareness across perception, comprehension, and projection.
- Three SAGAT freeze probes assessed participants’ robot and environmental situational awareness while they navigated and monitored interface information.
- Each probe contained six questions: three about the environment and three about the robot, spanning perception, comprehension, and projection.
- Robot questions covered relative position, motion, and intended future behavior, including distance trend, heading, and next room region.
- The first probe introduced an unexpected RC car crossing the participant’s path while the robot moved toward another station, requiring simultaneous monitoring of dynamic agents.
- The study controlled robot trajectories, timing, and scanning behavior to ensure consistent motion and probe conditions across trials.
3.4 Participants
Thirty participants were randomly assigned equally to the AR and mobile groups, with varied prior experience using robots and optical see-through AR.
- Thirty participants were randomly assigned to AR or mobile conditions, with 15 participants in each group.
- The sample included 19 men and 11 women, with a mean age of 25.6 years.
- Participants reported varied prior experience with robotic systems and optical see-through AR.
3.5 Study Procedure
Participants completed consent, demographic and experience surveys, training, eye-tracking calibration, a practice questionnaire, the task with three SAGAT probes, and a post-study survey. Sessions lasted approximately one hour.
- Participants completed informed consent and a pre-study survey covering demographics and prior experience with robots and AR.
- After group-specific task instruction and eye-tracking calibration, participants practiced answering questions through the assigned interface.
- Participants then completed the trial with three SAGAT probes before taking a post-study survey.
- Each experimental session lasted approximately one hour.
3.6 Data Collection and Metrics
The study measured situational awareness with binary SAGAT correctness scores, navigation time excluding word-search time, and world-coordinate gaze recorded at 60 Hz. Post-study surveys assessed workload, usability, experience, information amount, attentional focus, and environment tracking.
- SAGAT responses were compared with correct answers for each probe, producing a binary score per question.
- Navigation time was defined as trial completion time excluding time spent on word searches.
- World-coordinate cyclopean gaze was recorded at 60 Hz using the Magic Leap OpenXR Eye Tracker Feature.
- Six gaze segments per participant supported metrics for fixation, saccades, and blinks across task transitions and probes.Segments covered periods between subtask completion and probes, and between probes and the next subtask start.
- After each trial, participants completed NASA-TLX, UMUX-Lite, UEQ-S, attention-allocation sliders, and environment-tracking ratings.UMUX-Lite used a 0–100 scale, UEQ-S used a −3 to 3 scale, and agreement statements used a 5-point Likert scale.
4 STUDY RESULTS
The results showed selective AR benefits: robot perception improved, but higher-level robot awareness and environmental awareness did not. Gaze measures indicate that AR changed visual exploration and attention allocation in ways associated with the robot-perception benefit.
- 4.1 Interface Validation: Both interfaces provided a fair comparison basis, with no significant differences in reported information amount or workload metrics.The reported information ratings were MAR = 45.8 and MMobile = 45.07, p = .88.
- 4.2 RQ1: Effect of Interface on Situational Awareness: AR improved robot-related SA relative to mobile, but the three-way interaction showed that this benefit was reduced at the projection level.The interface × target contrast was β = 1.70, p = .028, while the three-way interaction was β = −1.99, p = .046.
- 4.2.1 Robot SA (RQ1a): χ2(1,N = 30) = 5.4, p = .02: 13 of 15 AR participants answered the first robot-position probe correctly versus 7 of 15 mobile participants.This significant difference was selective to robot perception and did not extend to comprehension or projection.
- 4.2.2 Environmental SA (RQ1b): Environmental SA showed no significant AR advantage or disadvantage across perception, comprehension, or projection.No significant group differences appeared for any of the 9 environmental SA questions; projection correctness was lower than perception, β = −1.22, p = .012.
- 4.3 RQ2: Attention Allocation and Gaze Behavior: AR users showed lower fixation duration and blink rate but higher fixation rate, saccade amplitude, and saccade velocity than mobile users.The significant overall effects were Fix-Dur β = −.063 s, FR β = .611, Sacc-Amp β = 1.695°, Sacc-Vel β = 23.00°/s, and BR β = −.261.
- 4.3 RQ2: Attention Allocation and Gaze Behavior: AR increased environmental fixation metrics while reducing map attention and increasing inter-AOI and within-environment fixation switching.Environment fixation rate and ratio increased, whereas map Fix-Share and Dwell-Share decreased; within-environment Fix-Trans-Freq increased with β = 0.795, p < .001.
- 4.3.2 Gaze Analysis: Mean saccade velocity fully mediated AR’s effect on robot perception in the first probe.The mediation analysis followed 10,000 bootstrap resamples and linked increased AR Sacc-Vel to improved robot perception.
5 DISCUSSION
AR improved perception-level robot awareness, but not higher-level robot or environmental awareness. Gaze findings indicate that attention freed from the map was redirected to conformal visuals rather than the physical environment.
- AR improved perception-level robot SA but produced no gains in higher-level robot SA or environmental SA relative to the mobile baseline.
- AR users showed more active visual exploration, with higher fixation-rate and saccade measures than mobile users.Mobile users showed longer fixations and fewer saccades, consistent with focused attention on the phone screen.
- In mediation analysis, increased saccade velocity fully mediated AR’s benefit for robot perception.Higher saccade velocity reflected switching between spatially distant environmental targets or between the environment and the map.
- AR users allocated significantly less fixation and dwell time to the minimap than mobile users allocated to the phone screen.Minimap fixation time was 35.0% for AR versus 64.7% for mobile users, while dwell time was 32.0% versus 54.7%, respectively.
- Physical-environment viewing did not differ significantly between groups because AR attention was divided between the environment and conformal visuals.AR users allocated 28.3% of fixation time and 26.0% of dwell time to conformal visuals.
- Because current optical see-through displays separate virtual content from the real environment, conformal visuals may need to be used more sparingly.The authors suggest discrete billboards at key path points instead of continuous navigation paths, while noting that this requires further investigation.
- The study’s SAGAT probes revealed stronger perception-level performance than comprehension- or projection-level performance.Probe wording and the compact physical space may have limited isolation of SA components.
- The Magic Leap 2 provided about 108° of unobstructed diagonal real-world view, potentially hindering environmental awareness.Using the same headset for both conditions controlled this factor, but broader-field-of-view devices remain a future direction.
6 CONCLUSION
In a real-world search-and-rescue-style study, the authors compared spatially conformal AR with an information-equivalent mobile interface and examined both SA and visual attention. AR improved perception-level robot SA through more active visual exploration, but not higher-level robot or environmental SA.
- AR significantly improved perception-level robot SA compared with the mobile baseline, with the benefit mediated by more active visual exploration.
- AR’s perceptual benefit did not extend to comprehension- or projection-level robot SA or environmental SA.
- AR-HRC interfaces should consider minimalist conformal designs when environmental SA is a concern.