PaperlayerView exact revision on arXiv

Source-linked AI summary

Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision

Seabin Lee, Sujeong Park, Nayoung Kim, Sungjin Park, Haechan Jung, Changjoo Nam

arXiv:2610.00897v1cs.RO

TL;DR

Fixed supervisory capacity overlooks changing workload, fatigue, and differences among operators, although human supervisors remain necessary for unexpected robot events. HAMA updates operator capacity and allocates intervention tasks online using human-state estimates. In the study, it improved task performance and reduced blink counts relative to a fixed-capacity baseline.

  • Problem

    Existing methods often assume fixed operator throughput, overlooking changes in workload and fatigue and differences in operator performance.

    Source: §I · p. 1

  • Method

    HAMA estimates operator workload and fatigue, updates supervisory capacity periodically, and assigns teleoperation tasks by priority, visual demand, and current capacity.

    Source: §I · pp. 1–2

  • Results

    Task performance rose from 16.22 to 17.86 and blink count fell from 66.39 to 55.92 under HAMA versus baseline across sessions 3–10.

    Source: D. Algorithm · p. 8

  • Takeaways & Limitations

    Higher throughput did not require higher individual capacity: HAMA often assigned fewer concurrent robots per operator while performance improved.

    Source: A. Efficiency Gains through Adaptive… · p. 8

    The study used a VR interface, two intervention tasks, and participants with limited prior experience; capacity updates occurred only at session boundaries.

    Source: C. Limitations · p. 8

Abstract

from arXiv · show

We propose a human-factor-aware method of allocating robot supervision tasks to multiple human operators. In scenarios where multiple operators occasionally teleoperate multiple robots to help the robots overcome difficulties, the allocation of the supervisory control tasks to humans needs to consider the real-time cognitive states of individual operators. However, most existing methods assume fixed supervisory capacity per operator and overlook fluctuations in the human factors such as workload and fatigue. As a result, workload distribution can be unbalanced where some operators become overloaded while the others remain underused. Our method dynamically regulates supervisory capacity and allocates tasks in a way that maintains balanced mental workload, prevents overload, and improves overall team performance. The allocation method uses a greedy strategy that minimizes estimated operator workloads with task prioritization. Robots are assigned to operators by reflecting their current supervisory capacity where the required effort depends on the types of tasks. In the user study, the analysis across predefined time intervals shows that the proposed method consistently achieves higher performance and lower behavioral signs of fatigue compared to a baseline method that does not consider human factors. These results highlight adaptive capacity adjustment as an effective preventive mechanism for sustaining operator performance in long-duration, high-demand settings.

I. INTRODUCTION

Multi-robot systems still need human intervention for unexpected events, creating a second allocation problem: assigning supervision tasks to operators. HAMA adapts operator capacity to human states, while prior work and interface support leave real-time, individual differences insufficiently addressed.

I. INTRODUCTION

  • Multi-robot systems still need human supervisors for unexpected events, so dynamically emerging intervention tasks must also be allocated among operators.Robot task allocation improves throughput and coordination, but human intervention adds an operator-assignment problem.Source: §I · p. 1
  • Fixed operator throughput overlooks workload, fatigue, and performance differences, despite cognitive resources being limited and potentially changing during prolonged supervision.The authors identify a need to account for both real-time human factors and technical capacity.Source: §I · p. 1
  • HAMA updates each operator’s supervisory capacity from human-state estimates and assigns robots requiring teleoperation according to task priority, visual demand, and current capacity.It supports reallocation as mission priorities change.Source: §I · pp. 1–2
  • In user studies, HAMA improved team performance by about 10% and reduced blink counts by 16% versus a fixed-capacity baseline.The method combines subjective reports, behavioral indicators, and physiological signals to update individualized capacity.Source: §I · p. 2

A. Task Allocation Strategies

  • Prior human-in-the-loop allocation work models operator assignments and performance constraints, motivating capacity as a human-factor-dependent rather than fixed quantity.Earlier task-allocation research largely allocates tasks among robots.Source: A. Task Allocation Strategies · p. 2

B. Human-in-the-Loop Control

  • Interface support often responds only after performance degrades, while operators differ in how quickly fatigue or overload develops.The paper frames real-time, individual-aware task allocation as a remaining research need.Source: B. Human-in-the-Loop Control · p. 2

C. Human Factors in Measuring Cognitive States

Operator state is not captured by workload alone: fatigue and workload interact, and multiple signals can represent distinct aspects of cognition. The section describes subjective and objective workload measures alongside fatigue indicators suited to prolonged monitoring.

C. Human Factors in Measuring Cognitive States

  • Workload and fatigue interact, so relying on a single measure can limit sensitivity and interpretability when estimating operator state.The paper argues that both should be considered to support consistent supervision and prevent overload.Source: C. Human Factors in Measuring… · p. 2

1) Mental Workload:

  • NASA-TLX measures workload subjectively but lacks the temporal resolution to track operational changes continuously.AOI dwell time reflects time spent focusing on interface elements; combining it with subjective ratings can provide a more robust workload estimate.Source: 1) Mental Workload · pp. 2–3

2) Cognitive Fatigue:

  • Cognitive fatigue is a gradual decline in cognitive efficiency during prolonged task engagement, associated with reduced alertness, attention span, and decision speed.Subjective scales are periodic; blink rate is a non-intrusive, continuously measurable indicator used to complement self-reports.Source: 2) Cognitive Fatigue · p. 3

III. PROBLEM DEFINITION

The problem is to assign queued intervention robots to operators while reducing mental workload and respecting changing operator capacities. HAMA estimates those capacities from human-state indicators and updates them periodically.

III. PROBLEM DEFINITION

  • The system assigns queued deadlock or manual-docking robots to operators to reduce mental workload while respecting each operator’s concurrent-supervision capacity.The warehouse scenario uses narrow passages and docking interventions; each monitoring window displays one supervised robot.Source: §III · pp. 3–4

IV. HAMA: HUMAN-FACTOR-AWARE MULTI-ROBOT ALLOCATION

  • HAMA combines periodic capacity updates from workload, fatigue, and blink indicators with an allocation method based on estimated capacity.Initial capacity is proportional to cognitive ability measured with three 2-back test results, capped by the monitoring-window limit.Source: §IV · p. 4, A. Human-factor-aware Capacity Update · p. 4

A. Human-factor-aware Capacity Update

  • At each session, NASA-TLX workload and subjective fatigue are combined into q_o, while completed tasks and blinks provide additional indicators.The method compares percentage changes from the previous session; when a previous value is zero, its change is set to zero.Source: A. Human-factor-aware Capacity Update · p. 4
  • Capacity changes by one unit according to the direction of the combined score change, bounded between one and the monitoring-window limit.Fixed increments or decrements are intended to balance responsiveness and stability rather than react to signal magnitude.Source: A. Human-factor-aware Capacity Update · p. 4

B. Task Allocation Strategy

HAMA greedily assigns prioritized supervision requests to the operator with the lowest relative workload after assignment, accounting for operator-specific task difficulty and deadlock precedence.

B. Task Allocation Strategy

  • The online greedy allocator assigns the highest-priority queued task to an available operator whose relative workload would be lowest afterward.The workload comparison needs only to identify the lowest among operators, not an absolute workload scale.Source: B. Task Allocation Strategy · p. 4
  • Task priority reflects type and urgency; docking can outrank deadlock because delayed intervention risks critical battery depletion.Deadlock priority also favors robots farther from their goals, encouraging wider distribution and local free space.Source: B. Task Allocation Strategy · p. 4
  • AOI dwell-time ratios from the previous session scale each operator’s workload weights for different task types.For example, a 30-second docking average and 45-second deadlock average produce weights of 1 and 1.5, respectively.Source: B. Task Allocation Strategy · p. 5
  • During deadlock resolution, boundary robots must move before inner robots, so HAMA updates priorities as the cluster changes.Boundary robots are identified on the cluster’s convex hull; operators assigned inner robots are switched to currently identified outer robots.Source: B. Task Allocation Strategy · p. 5, Figure 3 · p. 5

V. USER STUDY

The user study examines the proposed allocation method with respect to operator state and task performance.

  • The study evaluates the effectiveness of an allocation method that considers human factors.Source: §V · p. 5
  • Task performance is one outcome used to evaluate the method.Source: §V · p. 5
  • Operator cognitive workload and fatigue are also evaluation targets.Source: §V · p. 5

A. Participants

The study involved teams of three participants and used a VR warehouse-supervision task. Teams completed repeated sessions under two allocation conditions, with a break between conditions.

A. Participants

  • Thirty participants formed 10 teams of three operators; participants were 20–35 years old and had no prior experience with the proposed allocation system.Mean age was 26.4 years (SD = 3.23).Source: A. Participants · p. 5

B. Apparatus

  • The apparatus combined a Meta Quest Pro VR headset with a keyboard, and the headset's eye tracker recorded gaze data.The vendor provided no validated accuracy specification, so the study calibrated the tracker and focused on relative measures.Source: B. Apparatus · p. 5

C. Procedure

  • Teams supervised robots in a shared warehouse workspace, handling deadlock resolution and manual docking after participants completed three 2-back tests.The tests provided the initial operator-capacity measure.Source: C. Procedure · p. 5
  • Each team completed 10 four-minute sessions per method, with a brief survey after each session and a scheduled break between conditions.The study lasted about 90 minutes per team, including approximately 80 minutes of task execution.Source: C. Procedure · pp. 5–6, Figure 4 · p. 5
  • The VR interface showed multiple robot camera feeds, while Table I identifies the allocation features used in the two conditions.Source: Figure 5 · p. 6, Table I · p. 6

D. Experimental Design

Each team experienced both allocation conditions in counterbalanced order while supervising 20 mobile robots. The study collected self-reported workload and fatigue ratings alongside other measures.

D. Experimental Design

  • Each team experienced both conditions in a within-subjects design, with condition order counterbalanced to mitigate learning effects.Source: D. Experimental Design · p. 6
  • Operators supervised 20 mobile robots through VR monitoring windows and selected robots for intervention using a physical keyboard.Source: D. Experimental Design · p. 6
  • The baseline fixed operator capacity at four and applied equal weighting to task types, while retaining boundary-based reallocation.This configuration was intended to isolate capacity updates and attention-based weighting.Source: D. Experimental Design · p. 6

1) Self-reported Questionnaire:

2) Human Factor Indicator:

The study recorded blink count, task-specific AOI dwell time, and completed tasks, then analyzed sessions 3–10 because capacity adaptation used prior-session data.

2) Human Factor Indicator:

  • Blink count was recorded per session as a behavioral indicator that can reflect fatigue or reduced vigilance during prolonged supervision.Source: 2) Human Factor Indicator · p. 6
  • AOI dwell time in the supervised robot's ego-view and top-view windows estimated operator-specific relative task difficulty for task-type weighting.Source: 2) Human Factor Indicator · p. 6

3) Task Performance:

  • Task performance was measured as the number of tasks each operator successfully completed per session.Source: 3) Task Performance · p. 6

F. Analysis

  • Because capacity updates required prior-session data, HAMA adaptation began after session two and repeated-measures ANOVA used sessions 3–10.Source: F. Analysis · p. 6

VI. RESULTS

  • The results section presents session-wise means and standard deviations in Table II and two-way repeated-measures ANOVA results in Table III.Figure 6 shows session trends and the overall analysis.Source: §VI · p. 6, Table II · p. 7, Table III · p. 7

A. Self-reported Questionnaire

HAMA produced significantly fewer blinks than baseline, while self-reported NASA-TLX and cognitive-fatigue ratings did not show significant algorithm effects. Task-type difficulty was estimated from AOI dwell time, with manual docking requiring longer sustained attention than deadlock resolution.

A. Self-reported Questionnaire

  • NASA-TLX showed no significant algorithm effect, but ratings varied by session without a significant algorithm-by-session interaction.The reported algorithm test was F(1, 29) = 0.001, p = 0.975; the session effect was F(7, 23) = 3.342, p = 0.013.Source: A. Self-reported Questionnaire · p. 7
  • Cognitive-fatigue ratings were consistently lower with HAMA, but the algorithm effect was not significant; ratings varied by session.The algorithm test was F(1, 29) = 1.546, p = 0.224, and the session effect was significant at p < 0.001.Source: A. Self-reported Questionnaire · p. 7

B. Human Factor Indicators

  • HAMA resulted in significantly fewer blinks, with no significant session effect or algorithm-by-session interaction.Figure 6 presents session-wise trends and overall means with SD; its asterisks mark algorithm effects for blink count and performance.Source: B. Human Factor Indicators · p. 7, Figure 6 · p. 7
  • The relative difficulty ratio was deadlock-handling AOI dwell time divided by manual-docking AOI dwell time, and its session medians stayed below 1.This indicates that manual docking required longer sustained attention throughout the experiment; Figure 7 plots the ratio by session.Source: B. Human Factor Indicators · p. 7, Figure 7 · p. 8

C. Task Performance

Across sessions 3–10, HAMA increased average task performance and reduced blink count relative to baseline, while NASA-TLX changed by less than 1% and cognitive fatigue decreased without a significant algorithm effect. Allocation latency was 1.34 ms for HAMA and 1.26 ms for baseline.

C. Task Performance

  • HAMA yielded more successfully completed tasks than baseline (algorithm p = 0.046); performance also varied by session (p = 0.043), with no significant algorithm-by-session interaction (p = 0.245).The algorithm main effect was F(1, 29) = 4.362; the session main effect was F(7, 23) = 2.541.Source: C. Task Performance · p. 8

D. Algorithm

  • Across sessions 3–10, HAMA increased average task performance from baseline 16.22 to 17.86 and decreased blink count from baseline 66.39 to 55.92.These changes were about a 10% performance increase and about a 16% blink-count reduction.Source: D. Algorithm · p. 8
  • NASA-TLX changed by less than 1% (30.90 to 30.93), while cognitive-fatigue ratings decreased about 7% (5.54 to 5.15) without a significant algorithm effect.The supplied overall comparison reports these measures across sessions 3–10.Source: D. Algorithm · p. 8
  • Average allocation latency was 1.34 ms for HAMA and 1.26 ms for baseline, a difference the authors describe as negligible.The authors state that HAMA maintained real-time responsiveness despite processing additional human-factor data.Source: D. Algorithm · p. 8

VII. DISCUSSION

HAMA improved task performance and reduced blink counts without increasing perceived workload, while often assigning fewer concurrent robots per operator. The discussion highlights complementary workload measures and limits from the study setting and session-level updates.

VII. DISCUSSION

  • HAMA increased successfully completed tasks versus a fixed-capacity baseline and reduced blink counts without increasing perceived workload.The discussion characterizes the blink-count change as a reduction in behavioral signs of fatigue.Source: §VII · p. 8

A. Efficiency Gains through Adaptive Allocation

  • Higher throughput did not require higher individual capacity: HAMA often assigned fewer concurrent robots per operator and shifted demand toward operators with stable performance signals.The authors suggest overload can reduce the quality and speed of interventions.Source: A. Efficiency Gains through Adaptive… · p. 8

B. Behavioral Sensitivity Beyond Subjective Workload Ratings

  • Blink counts changed while NASA-TLX and fatigue ratings did not; post-session reports may miss within-session fluctuations, motivating complementary measures.The authors note that subjective ratings may become less sensitive when cognitive resources are saturated.Source: B. Behavioral Sensitivity Beyond… · p. 8

C. Limitations

  • The evaluation used VR, only deadlock resolution and manual docking, and participants with limited prior experience; session-boundary updates reduce sensitivity to transient fluctuations.A shorter update interval is suggested for settings requiring faster adaptation.Source: C. Limitations · p. 8

VIII. CONCLUSION

  • HAMA is lightweight and real-time with negligible computational overhead; longer deployments and more diverse operator populations remain future tests.The conclusion also points to more principled weighting of integrated human-factor measures.Source: §VIII · p. 8
Loading 2610.00897v1…