Source-linked AI summary
Towards Intent-Aware Human-Robot Teaming: A Platform for Search-and-Rescue Operations
Rohith Prem Maben, Ayesha Jena, Björn Olofsson, Stefan Reitmann, Jacek Malec, Rogier Woltjer, Elin Anna Topp
TL;DR
Heterogeneous SAR teams make it challenging for operators to oversee and control multiple agents, and intent inference in this setting remains limited. The paper develops a JCF-based interaction platform to study operator behavior and intent inference, then demonstrates through a mission scenario how operator choices reveal function allocation and could inform adaptive assistance.
Problem
Heterogeneous teams increase the challenge of maintaining operator oversight and control, while how to infer and use intent in SAR coordination remains limited.
Source: §1 · pp. 1–2Method
Results
Takeaways & Limitations
Abstract
from arXiv · showhide
We investigate the challenges of enabling effective collaboration between human operators and heterogeneous autonomous agents in complex, dynamic environments by developing an interaction platform that allows study of operator behavior and supports intent inference and decision-making using state-of-the-art frameworks. We demonstrate the extent to which the operator's perception, decisions, and actions could be supported by autonomous systems during search-and-rescue operations with our platform.
1 Introduction
Heterogeneous UAV-UGV teams can extend SAR reach, but they increase the demands of operator oversight and control in uncertain environments. The paper presents a platform for studying operator intent in SAR and supporting adaptive decision-making.
1 Introduction
- Heterogeneous teams challenge operators to maintain balanced oversight and control of multiple agents in dynamic, uncertain environments.Examining operator decisions and the intent behind them is identified as one way to provide decision support.Source: §1 · p. 1
- Intent inference remains limited in SAR human-robot teaming, including how to use inferred intent to adapt control and coordination across heterogeneous agents.Prior probabilistic and learning-based work has explored anticipating operator decisions and adapting system behavior.Source: §1 · pp. 1–2
- The platform combines interaction data collection, configurable UAV-UGV SAR scenarios, and real-time intent inference using the Joint Control Framework.A representative scenario illustrates how actions reflect reliance on robot autonomy versus pre-planned procedures, and how that understanding supports adaptive decision support.Source: §1 · p. 2
2 Related work
Prior work addresses operator support and task allocation in single- and multi-robot systems, while complex heterogeneous missions motivate more integrated frameworks. The paper builds a Unity simulation platform around the JCF to study operator behavior and intent inference in UAV-UGV SAR operations.
2 Related work
- Human-robot teams face operator cognitive-load, efficiency, oversight, and workload-allocation challenges in dynamic environments.Shared mental models and dynamically adaptive automation are described as ways to support collaboration and task goals.Source: §2 · p. 2
- Complex heterogeneous missions motivate unified frameworks that infer operator intent and translate it into coordinated behavior across multiple agents.Earlier work has addressed operator support, task efficiency, and task allocation in single- and multi-robot systems.Source: §2 · p. 2
- The JCF distributes sensing, deciding, and acting across cognitive levels and agents using Levels of Autonomy in Cognitive Control.Although JCF operationalization for multi-robot SAR has been studied, its potential for operator intent inference has not been extensively examined.Source: §2 · p. 2
- The authors implement a Unity simulation platform with operator interfaces and data logging that communicates with multiple robot instances for controlled UAV-UGV SAR studies.The platform addresses the stated gap in studying operator behavior and intent inference in this setting.Source: §2 · p. 2
3 System architecture
The platform connects UAV and UGV telemetry to a Unity visualization and operator interfaces, while recording synchronized mission data. Its architecture currently uses one UAV and one UGV, with independent telemetry mapping designed to scale to additional agents.
3 System architecture
- ArduPilot telemetry from one UAV and one UGV reaches Unity through MAVLink v2 over UDP, where decoded frames update each vehicle model in real time.The four message types consumed by Unity are position, orientation, motor outputs, and flight data.Source: §3 · p. 3, Figure 1 · p. 3
- Operators use a MAVLink-enabled ground control station and camera viewer to select the UGV feed, UAV feed, or both.The platform maps ArduPilot's North-East-Down coordinates to Unity's left-hand scene coordinates.Source: §3 · p. 3
- A separate Unity acquisition module merges mission state with synchronized MAVLink telemetry into a unified per-episode log.The current implementation presents one UAV and one UGV, while the architecture supports additional agents on dedicated UDP ports with independent telemetry mapping.Source: §3 · p. 3
4 Holistic intent modeling in UAV-UGV teaming
The study applies the JCF to model operator interaction across cognitive-control levels and analyzes a SAR mission as perception, decision, and action events. The example traces how the operator searches, responds to debris, and chooses an obstacle-avoidance strategy.
4 Holistic intent modeling in UAV-UGV teaming
- The JCF is used to model temporal human-robot interaction at different Levels of Autonomy in Cognitive Control, with levels operationalized for the SAR study.Table 1 summarizes the study's control-level operationalizations.Source: §4 · p. 4, Table 1 · p. 4
- HMI-T decomposes the logged SAR scenario into perception, decision, and action points across search, obstacle identification, and obstacle avoidance.The operator is the supervisory-control subject and the UGV is the controlled object.Source: §4 · p. 4, Figure 2 · p. 4
- During search, the operator inspects UGV video, registers survivor coordinates, checks for casualties with heat sensors, then shifts from careful scanning to faster direct travel.The described perception, decision, and action events are assigned different cognitive-control levels.Source: §4 · pp. 4–5
- At the obstacle, the operator retreats after detecting debris, chooses not to use UAV assistance to save time, assesses the debris, and steers around it prioritizing stability over speed.The platform logs sensor use, camera selection, automation levels, route commitments, and action commands; the analysis identifies function allocation between operator and UGV and suggests a basis for UAV adaptive assistance.Source: §4 · p. 5
5 Discussion and Conclusion
The platform evaluates holistic operator-intent inference in SAR teaming, capturing context-rich interactions and structuring intention space for temporal analysis. It points toward UAV decision support aligned with operator objectives, with real-world deployment and Bayesian inference left for future work.
5 Discussion and Conclusion
- The platform uses the Joint Control Framework to infer holistic operator intent across multiple abstraction and cognitive-control levels in SAR operations.It is designed to evaluate holistic intent-structure inference in human-autonomy teaming.Source: §5 · p. 5
- Platform data capture operator actions, mission context, and multi-agent interactions to support intent models reflecting SAR decision-making dynamics.Source: §5 · p. 5
- Temporal analysis of operator-UGV interactions structures the intention space, opening a path to UAV support aligned with current operator objectives.Source: §5 · p. 5
- The setup represents broader scalable UAV-UGV configurations, while real-world SAR deployment and Bayesian probability updates over intent space remain future work.The authors are working toward an end-to-end adaptive-support framework aligned with LACC levels.Source: §5 · p. 5