PaperlayerView exact revision on arXiv

Source-linked AI summary

GazeFS: Target-Centered Gaze-Trajectory Forecasting and Stabilization from Gaze-Head History

Yaozheng Xia, Zaiping Zhu, Bo Pang, Minghao Xie, Hui Li, Shaorong Wang, Sheng Li

arXiv:2609.03868v1cs.HCcs.AI

TL;DR

Gaze interaction must address persistent target-relative offsets and task-aligned acquisition dynamics, not only frame-to-frame fluctuation. GazeFS forecasts target-center direction and short-horizon Search/Focus phase from variable-length gaze–head history without target information at inference, improving Focus target centering while treating temporal smoothness separately. The study supports complementary history and progress context, but broader continuous-stream and cross-setting generalization remain open.

  • Problem

    Gaze estimates can be smooth yet persistently offset from the intended target, while acquisition changes across confirmation-anchored Search and Focus phases.

  • Method

    GazeFS uses variable-length gaze–head history with parallel target-direction and short-horizon phase prediction, phase-aligned training, and no target information in the inference tensor.

  • Results

    0.182°, 0.257°, and 0.400° reductions in Focus episode bias, within-episode dispersion, and P90 target error versus raw hold were observed across participant-grouped out-of-fold evaluation.

  • Takeaways & Limitations

    GazeFS improves Focus target centering and empirical residual contraction, while recent history contributes spatial information beyond observable progress cues.

  • Takeaways & Limitations

    Evaluation covers segmented successful acquisitions from one HoloLens 2 task, leaving open-stream false activation and cross-layout, session, task, and device generalization.

Abstract

from arXiv · show

Target-centered gaze interaction requires more than suppressing frame-to-frame fluctuations: target acquisition produces task-aligned changes in gaze-head dynamics, while a gaze trace may retain a persistent target-relative residual direction. We formulate gaze correction as online target-centered gaze-trajectory forecasting and stabilization and introduce GazeFS, which maps a variable-length gaze-head history to the next target-center direction and a short-horizon Search/Focus estimate without target information at inference. Across 7,960 acquisition episodes from 30 participants, Search-Focus differences remain stable under quality control, onset exclusion, and duration matching. History windows improve phase decoding over the current endpoint, but explicit task progress remains a strong control. Under the 30-participant, five-fold grouped out-of-fold protocol across three seeds, the reductions relative to raw hold in Focus episode bias, within-episode dispersion, and P90 target error are 0.182 degrees, 0.257 degrees, and 0.400 degrees, with participant-bootstrap 95% confidence intervals excluding zero. Endpoint-free replay from empty history preserves the Focus advantage and yields raw-network phase balanced accuracy/AUPRC of 0.925/0.993; coordinate controls further show that recent history contributes beyond explicit progress metadata. GazeFS therefore improves Focus target centering and empirical residual contraction while leaving temporal smoothness as a separate objective.

Gaze–Head History

GazeFS uses task-aligned gaze–head history to forecast target-centered gaze behavior, as illustrated by its overview figure.

  • GazeFS maps gaze–head history to the next target-center direction and short-horizon Focus probability without target geometry at inference.

Introduction

The paper distinguishes transient gaze fluctuation from persistent target-relative offset and asks whether history can jointly support target centering and acquisition-phase inference. GazeFS addresses this with target-agnostic forecasting and reports improved Focus-stage centering under participant-grouped evaluation.

  • Gaze estimates contain both short-timescale fluctuation and persistent target-relative offset, which can leave smooth gaze systematically off target.The paper defines stabilization as target-relative centering and residual contraction, while evaluating temporal smoothness separately.
  • The central question is whether gaze-head history can infer both target-center direction and entry into the short-horizon Focus phase.Search and Focus are confirmation-anchored task-progress labels rather than assumed physiological states.
  • Prior approaches typically separate spatial correction from acquisition timing and may use interaction feedback or candidate targets at runtime.
  • Recent history improves phase decoding over a current endpoint, while explicit progress explains much of phase predictability and frame-order perturbation causes little loss.
  • GazeFS combines masked variable-length history encoding, parallel direction and phase prediction, and phase-aligned coordinate learning.The design targets progressively growing acquisition histories and phase-dependent behavior.
  • 0.182°, 0.257°, 0.109°, and 0.400° are the reductions in Focus episode bias, dispersion, median error, and P90 error versus raw hold.The evaluation uses all 30 participants, five participant-grouped outer folds, and three seeds; endpoint-free replay preserves the centering advantage.
  • GazeFS demonstrates improved Focus-stage target centering without future or endpoint information at inference.

Related Work

Related work spans temporal filtering, eye–head pointing, spatial correction, gaze forecasting, and fixation or intent modeling. GazeFS combines target-direction correction with task-aligned phase prediction from shared historical context.

  • Temporal filters reduce rapid fluctuation but remain anchored to measured direction, while eye–head pointing combines gaze responsiveness with head-motion stability.
  • Candidate-aware spatial methods infer intended objects or landing positions using scene geometry, target candidates, or endpoint distributions.
  • Temporal gaze-forecasting research uses gaze, head motion, and scene history to anticipate future viewing behavior and model short-horizon time-series patterns.
  • Fixation and saccade methods identify physiological events, whereas intent and adaptive-dwell models estimate interaction readiness or activation timing.
  • GazeFS predicts confirmation-anchored Search/Focus acquisition progress jointly with target direction from a shared historical representation.These labels encode task progress rather than an intrinsic oculomotor regime.

Data and Task Definition

The study analyzes gaze–head recordings from HoloLens 2 target-acquisition episodes and defines Search and Focus from confirmation-anchored target-centered traces. Target direction supports offline labeling and evaluation but is excluded from inference.

  • Data: 7,960 60-Hz episodes from 30 HoloLens 2 participants form the study dataset.
  • Data: The input contains head-local gaze yaw–pitch, angular rates, and head yaw–pitch–roll rates.
  • Task Definition: Target-center direction is expressed in the same frame but used only for offline label construction, supervision, and evaluation.It is never provided at inference.
  • Task Definition: A backward scan from participant confirmation locates the earliest stable target-centered segment, separating preceding Search frames from subsequent Focus frames.The labels represent acquisition progress rather than physiological fixation or saccade states.
  • Analysis: Natural history-window probes compare endpoint-only, ordered-history, order-invariant, and explicit-progress controls using frozen endpoints.
  • Ethics: The studies received institutional ethics approval, and all participants provided informed consent.

Task-Aligned Gaze–Head Structure

Gaze–head dynamics differ systematically between Search and Focus, while Focus residuals retain a stable direction and recent history improves phase decoding beyond a single endpoint.

  • Focus has lower gaze-speed tails, head motion, and local dispersion than Search, with effect directions preserved after onset exclusion and duration matching.The comparison uses all 30 participants and remains stable after removing ±300 ms around the phase boundary and matching phase durations.
  • The observed Search–Focus structure supports task-aligned non-stationarity during acquisition.Sensitivity to label-construction thresholds and retained membership is reported separately.
  • An early/late Focus bias-vector cosine of 0.886 and persistent-offset ratio of 0.977 indicate directionally stable target-relative residuals.Both intervals are well above zero, supporting target-center forecasting rather than local smoothing alone.
  • A recent gaze–head window raises balanced accuracy from 0.640 for the current 7D observation to 0.927/0.928 at 64/128 frames.Elapsed time is competitive with the full window, and frame or short-block shuffling causes essentially no loss.

Forecasting and Stabilization

GazeFS uses variable-length gaze–head history to forecast next-frame target-center directions and short-horizon Search/Focus phases online, without target information at inference. Its multi-scale causal architecture combines parallel trajectory and phase heads with streaming-prefix refinement and phase-aligned training.

  • Problem formulation: GazeFS predicts a dense sequence of next target-center directions and patch-level Search/Focus logits from the currently observed gaze–head prefix.The target direction is head-local yaw–pitch, and the phase output concerns whether the next short interval contains Focus rather than the current frame.
  • Multi-scale history encoding: Separate fine- and coarse-scale gaze and head tokens are fused before a shared Transformer encoder uses causal and key-padding masks for online variable-length histories.Fine tokens capture rapid acquisition changes, while coarse tokens provide longer local motion context; causal masking excludes future anchors.
  • Parallel prediction heads: The trajectory and Search/Focus heads share an encoded representation but remain parallel, so predicted phase never feeds the direction forecast.Deployment uses one encoder pass, with inference paths excluding target information and phase labels.
  • Streaming-prefix refinement: Short-prefix residual modules refine the base trajectory forecast using raw gaze–head history and additional first-stage direction and prefix descriptors.The residual gates decay as context accumulates, leaving the base forecast unchanged outside the short prefix before inverse standardization.
  • Task-aligned learning: The phase head combines encoded context with temporally pooled gaze- and head-motion magnitudes, while training-only masking and geometry terms regularize learning.The coordinate objective pairs a phase-weighted Huber displacement term with a directional cosine term, using label-derived Focus weights rather than predicted phase.

Experiments

Grouped out-of-fold experiments show that GazeFS improves Focus-stage target centering over raw hold, while benefits vary by phase and temporal smoothness remains a separate objective.

  • Focus outcomes: 0.182°, 0.257°, 0.109°, and 0.400° reductions in Raw bias, dispersion, median error, and P90 error, respectively, are reported for common-support Focus samples.Participant-bootstrap and simultaneous intervals remain below zero, and the improvements also hold against several comparator methods.
  • Temporal behavior: Full GazeFS increases mean frame-to-frame target-relative residual motion by 0.0578° and residual path by 7.003° relative to Raw.These metrics describe temporal movement of the residual trajectory, not spatial spread or device-level pointer jitter.
  • Phase dependence: Only Focus improves jointly across centering, spread, tail error, and residual motion; Search and transition do not improve consistently.The phase-harm audit therefore localizes the joint spatial benefit to Focus.
  • Phase and onset: Full phase outputs achieve 0.903 balanced accuracy, 0.936 Focus F1, and 0.990 AUPRC, with 248/585 ms median/P90 absolute onset error.The reported phase and onset outcomes represent a trade-off rather than uniform dominance.
  • Online replay: Endpoint-free replay from empty history preserves the fixed-window Focus centering advantage, while exact-prefix gains are not uniform at every prefix length.Results improve over Raw at 8, 16, 64, and 128 frames, but the 32-frame interval crosses zero.
  • History and progress: A history model without explicit progress improves bias, dispersion, median, and P90 over Endpoint+Progress by 0.159°, 0.132°, 0.098°, and 0.303°, respectively.Restoring progress and position context primarily improves dispersion, P90, frame-to-frame residual motion, and phase/onset reliability, although the bundled control does not isolate exact frame order.
  • Temporal localization: Relative to Raw, P90 is worse during the first 250 ms after Focus onset (+0.360°) but better after 500 ms (−0.455°).This localizes the Raw-relative improvement to sustained Focus rather than initial target discovery.
  • Closed-loop feasibility: In a 12-participant study, GazeFS is 0.196 s faster than One Euro in the all-trial min(T, 5 s) sensitivity, while comparisons with native pointing and Weighted Pointing cross zero.Completion is near ceiling, and only GazeFS resets history/state after the relevant interaction boundary.

Limitations

The evaluation is constrained to confirmation-anchored phases and segmented successful acquisitions from one HoloLens 2 task, with several generalization and validation boundaries. Grouped out-of-fold results remain internal to the same corpus, while the closed-loop comparison does not isolate reset from learned correction and excludes some deployed latency components.

  • Evaluation scope: Evaluation covers segmented successful acquisitions from one HoloLens 2 task, with Search and Focus defined as confirmation-anchored task phases.These labels represent task-defined acquisition phases rather than assumed physiological states.
  • Generalization boundaries: Open-stream false activation and cross-layout, session, task, and device generalization remain to be evaluated.A single-seed held-quadrant screen supports extrapolation only within the recorded geometry.
  • Validation scope: Grouped out-of-fold validation is internal to the same corpus rather than confirmation from an independent cohort.Model development used a fixed split from that corpus.
  • Closed-loop interpretation: The closed-loop comparison does not isolate reset effects from learned correction, and centering retains a Raw-relative motion cost.These conditions limit attribution of the deployed comparison to learned correction alone.
  • Latency scope: PC-side ONNX latency excludes sensing, network transport, and Unity rendering.The reported latency therefore does not represent complete end-to-end interaction latency.

Conclusion

GazeFS formulates target-centered gaze-trajectory forecasting and stabilization from gaze-head history without target information at inference. Across grouped out-of-fold evaluation and endpoint-free replay, it improves Focus-stage target centering, while history and observable progress contribute complementary information and temporal smoothness remains separate.

  • Conclusion: GazeFS jointly estimates target-center direction and Search/Focus phase from gaze-head history without target information at inference.Its design combines multi-scale history encoding, parallel direction and phase prediction, and phase-aligned training.
  • Conclusion: Recent gaze-head history contributes spatial information beyond observable progress cues, whose role remains complementary rather than ordered by a strict law.The paper separates temporal smoothness from the target-centering objective.
Loading 2609.03868v1…