PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion models for eye-gaze trajectory generation using position and velocity representations

Laxman Basnet, Alexander Szorkovszky, Pedro G. Lind, Anis Yazidi, Shailendra Bhandari

arXiv:2609.05522v1cs.CVcs.AIcs.NE

TL;DR

Eye-gaze recordings are costly to collect and difficult to share, and prior diffusion gaze models are predominantly stimulus- or task-conditioned. This paper compares unconditional DDPMs for position and velocity sequences; the position-space model outperforms a kinematic Markov baseline on mean JS divergence and Fréchet Gaze Distance. The two analyses characterize local gaze dynamics recovered by unconditional diffusion and where explicit conditioning may help.

  • Problem

    Eye-gaze recordings require specialized hardware and controlled conditions, are difficult to share because of privacy constraints, and existing diffusion gaze models are mostly stimulus- or task-conditioned.

    Source: §I · p. 1

  • Method

    The study trains two unconditional DDPMs on gaze-position and two-component velocity sequences from the same eye-tracking dataset and visual-search task.

    Source: §I · p. 2, §II · p. 2

  • Results

    Position-space DDPM mean JS is 0.016 versus 0.221 for the kinematic Markov baseline, and its FGD is 13.79 ± 2.97 versus 467.49 for the positional Markov baseline.

    Source: A. Position-space model · p. 6

  • Takeaways & Limitations

    The two analyses characterize which aspects of gaze dynamics unconditional diffusion recovers and where explicit conditioning may provide further benefit.

    Source: §I · p. 2

    The conclusion contrasts more accurate recovery of local gaze kinematics with less accurate recovery of long-range properties, including cumulative path geometry.

    Source: §VI · pp. 7–8

Abstract

from arXiv · show

Eye-tracking data are expensive to collect, requiring specialized hardware and controlled laboratory conditions, and difficult to share because of privacy constraints. We address this using two complementary denoising diffusion probabilistic models (DDPMs) for unconditional generation of eye-gaze dynamics from visual-search data. Both use an identical FiLM-conditioned one-dimensional U-Net with self-attention (19.35,M parameters), trained on 8,s sliding-window sequences from 28 participants. One model generates raw two-dimensional gaze-position sequences, while the other generates two-component velocity sequences; each uses representation-specific preprocessing, training settings, data partitions, and evaluation protocols. Both are evaluated across three independent training seeds, with aggregated metrics reported as mean,$\pm$,SD. The position-space model achieves a mean Jensen-Shannon (JS) divergence of $0.016\pm0.004$ across nine kinematic features, with the highest feature-wise mean below $0.030$, fixation duration within 2% of real data, and a Fr'echet Gaze Distance more than an order of magnitude below statistical and Markovian baselines. Under a Train-on-Synthetic-Test-on-Real protocol, synthetic-only training achieves $R^2=0.66\pm0.02$, or 82.7% of the real-data $R^2$ point estimate. The velocity-space model achieves a mean JS divergence of $0.0065$ across velocity components, speed, log-speed, and turning angle, with a maximum of $0.015\pm0.005$. Reconstructed path length is less accurate ($0.21\pm0.02$ versus $0.03\pm0.01$ in position space), although the protocols differ. Overall, unconditional diffusion captures local gaze kinematics and short-range temporal and directional structure, while long-range properties such as saccade counts and cumulative path geometry remain targets for future conditioned models.

I. INTRODUCTION

Eye-gaze recordings are costly to collect and difficult to share, while existing diffusion gaze models are mostly stimulus- or task-conditioned. This study compares unconditional position- and velocity-space DDPMs, while treating their differing evaluation protocols as non-comparable.

I. INTRODUCTION

  • Gaze recordings require specialized hardware, controlled laboratory conditions, and substantial time, while privacy constraints limit sharing and dataset size, diversity, and reproducibility.Gaze also varies substantially across people under identical viewing conditions, making behavioral diversity hard to collect at scale.Source: §I · p. 1
  • Classical Markov and autoregressive models capture gaze’s non-stationary fixation–saccade dynamics only over short horizons, while GANs can suffer mode collapse and training instability.Diffusion offers a non-adversarial alternative and has been extended to probabilistic time-series forecasting and imputation.Source: §I · pp. 1–2
  • Unconditional diffusion gaze models address a gap in how well intrinsic kinematic and temporal structure is recovered without stimulus or participant conditioning.Earlier diffusion gaze approaches are predominantly stimulus- or task-conditioned, with evaluations focused on agreement with the corresponding visual stimulus.Source: §I · p. 2
  • The study trains unconditional DDPMs on position and velocity representations, using the same underlying dataset and task but representation-specific preprocessing and training configurations.Because validation sets and evaluation protocols differ, the results are not treated as a controlled representation ablation.Source: §I · p. 2

II. BACKGROUND AND THE DATASET

  • Both models use 2,000-sample, 8 s windows at 250 Hz from a visual-search task whose unscripted scanpaths preserve inter-individual variation.The task has long continuous trajectories with fixation and saccade structure; position-space processing yields 4,401 valid segments from 28 participants, while the velocity split contains 425 training and 127 validation segments.Source: §II · pp. 2–3

III. METHODS

Both representation-specific denoisers share a FiLM-conditioned one-dimensional U-Net that predicts noise from a noisy two-channel gaze window and diffusion step.

III. METHODS

  • The shared one-dimensional U-Net takes a noisy two-channel window and diffusion step, then outputs a same-shaped two-channel noise estimate.Its three encoder stages use widths 128, 256, and 512, followed by a residual-block and self-attention bottleneck and a mirrored decoder with skip connections.Source: §III · p. 3
  • 19,350,914 parameters make up the backbone, whose residual blocks use group normalization, SiLU activations, and FiLM conditioning from a sinusoidal diffusion-step embedding.Bottleneck self-attention has four heads with head dimension 32, allowing temporal positions to attend across the window.Source: §III · p. 3
  • Both models use 1,000 diffusion steps, a linear noise schedule, AdamW optimization, the ε-prediction objective, and deterministic DDIM sampling.Training recipes, inference budgets, and data partitions differ between representations.Source: §III · p. 3

A. Position-space training recipe

The position-space model is trained with flips and temporal reversal as augmentation, uses EMA weights for inference, and generates trajectories with 100 DDIM steps.

A. Position-space training recipe

  • The position-space model trains for 500 epochs with batch size 8, learning rate 10^-4, weight decay 10^-4, gradient clipping at 1.0, and cosine annealing after a 10-epoch warm-up.Training uses automatic mixed precision.Source: A. Position-space training recipe · p. 3
  • Independent horizontal and vertical flips each occur with probability 0.5, and temporal reversal with probability 0.3, reflecting approximate population-level symmetries in the visual-search data.The model uses EMA weights with decay 0.9999 for inference and generates samples with 100 DDIM steps.Source: A. Position-space training recipe · p. 3, Figure 2 · p. 4
  • Three random seeds (42, 123, and 2024) use the same participant-stratified 80/20 split, with Section V A metrics reported as mean ± SD across runs.The split uses seed 42.Source: A. Position-space training recipe · pp. 3–4

B. Velocity-space training recipe

The velocity-space model uses Min-SNR-weighted training and EMA inference, then generates rescaled velocity windows with 50 DDIM steps.

B. Velocity-space training recipe

  • The velocity-space model trains for 500 epochs with batch size 16, learning rate 10^-4, weight decay 10^-4, gradient clipping at 1.0, and a 10-epoch warm-up followed by cosine annealing.The excerpt specifies the forward noising relation used during training.Source: B. Velocity-space training recipe · p. 4
  • Training applies Min-SNR loss weighting with γ = 5, and EMA weights with decay 0.999 are used for inference.The model uses three random seeds and a fixed participant-level train-validation split; metrics are reported as mean ± SD.Source: B. Velocity-space training recipe · p. 4
  • Synthetic velocity windows are generated with 50 DDIM steps and rescaled by the training-set factor cv, without smoothing or mean-centering.Complete settings for both models are listed in Appendix Table III.Source: B. Velocity-space training recipe · pp. 4–5

IV. EVALUATION PROTOCOLS

The evaluation uses representation-specific metrics and baselines, so results across position and velocity space are not a controlled comparison. In position space, the DDPM substantially outperforms statistical and Markov baselines on feature-space distance and captures directional structure better than the kinematic Markov model.

IV. EVALUATION PROTOCOLS

  • Because the models use different physical quantities, validation sets, and metric suites, their numerical results are not a controlled ablation.The position-space feature-level table reports KS and JS agreement over three seeds on a fixed held-out split.Source: §IV · p. 5, Table I · p. 5
  • The position-space protocol compares nine feature distributions and uses FGD, fixation detection, downstream TSTR, and four descriptors of directional and temporal structure.The four descriptors are speed, speed autocorrelation, movement direction, and signed turning angle; the protocol compares 1,136 real and 1,136 generated validation segments.Source: §IV · p. 5

A. Position-space model

  • Figure 4 compares empirical gaze, the position-space DDPM, and the kinematic Markov baseline on speed, speed autocorrelation, movement direction, and turning angle.The DDPM reproduces the central turning-angle peak at ϕ = 0 more closely than the kinematic Markov baseline.Source: Figure 4 · p. 6, A. Position-space model · p. 6
  • FGD = 13.79 ± 2.97, versus 467.49 for the strongest baseline, positional Markov, and 1,648.77–2,752.88 for the other three baselines.The DDPM also improves on the kinematic Markov baseline across eight distinct features: mean JS is 0.016 versus 0.221; Figure 4 compares their directional and temporal statistics.Source: A. Position-space model · p. 6, Figure 4 · p. 6
  • The ε-prediction ablation yields FGD 11.86 versus 16,782 for direct x0-prediction, while synthetic-only TSTR achieves R2 = 0.6612 ± 0.0219.The TSTR score is 83% of the real-to-real reference, 0.80 ± 0.05.Source: A. Position-space model · p. 7

B. Velocity-space model

The velocity-space model closely matches local velocity and turning-angle distributions, as well as short-lag temporal and spectral structure. Reconstructed cumulative path length remains its clearest mismatch.

B. Velocity-space model

  • JS divergence is 0.0022 ± 0.0005 for vx, 0.0073 ± 0.0009 for vy, 0.0083 ± 0.0014 for speed, and 0.0003 ± 0.0002 for signed turning angle.Log-speed JS is 0.0146 ± 0.0049.Source: B. Velocity-space model · p. 7
  • Reconstructed path length has JS = 0.207 ± 0.020, the clearest mismatch, while lag-one autocorrelation mismatches remain below 0.06 for all four quantities.Normalized spectral distances are of order 10^-4; small speed differences accumulate over the full 8 s window.Source: B. Velocity-space model · p. 7
  • Across both representations, local signed and directional statistics are better matched than integrated long-range structure, including saccade counts and cumulative path length.The models share an architecture but use different validation-set sizes and metric suites, so their numeric results are not a direct comparison.Source: B. Velocity-space model · p. 7

VI. DISCUSSION, LIMITATIONS AND CONCLUSIONS

Unconditional diffusion captures local gaze kinematics more accurately than long-range trajectory structure, while the position-space model outperforms statistical and Markov baselines. The findings are limited to one visual-search task and to the models’ unconditional setting.

VI. DISCUSSION, LIMITATIONS AND CONCLUSIONS

  • Position-space generation underestimates saccade count by approximately 15%, comparable to the 13% undercount reported for DiffGaze.The discussion says discrete saccade recovery remains difficult for diffusion-based gaze models.Source: §VI · p. 8
  • Both models are unconditional, and evaluation is restricted to the Where’s Waldo? visual-search task, leaving generalization to reading and free viewing untested.FGD uses the position model’s own feature space, while fixation and saccade counts depend on a noise- and preprocessing-sensitive fixed I-VT threshold.Source: §VI · p. 8
  • The position-space DDPM reproduces marginal kinematics, fixation timing, and directional structure while substantially outperforming statistical and Markov baselines.The velocity-space DDPM closely matches velocity marginals, speed, turning angle, autocorrelation, and spectral structure.Source: §VI · p. 8

COMPUTATIONAL ENVIRONMENT

  • Both models were implemented in Python with PyTorch; position-space training used an NVIDIA A40 GPU and velocity-space training used an NVIDIA RTX A6000 GPU.The velocity-space implementation ran through GitLab CI/CD in a Docker container.Source: COMPUTATIONAL ENVIRONMENT · p. 8

Appendix A: Gaussian posterior of the forward diffusion process

The appendix derives the exact Gaussian reverse posterior for the forward diffusion process when the clean sample is known. During sampling, the learned network approximates the unknown clean sample through noise prediction.

Appendix A: Gaussian posterior of the forward diffusion process

  • The appendix derives q(x_{t-1} | x_t, x_0) by combining the forward transition and the marginal conditioned on the clean sample.Both factors are Gaussian, so the posterior is Gaussian as well.Source: Appendix A · p. 10
  • The resulting posterior is the exact one-step reverse distribution when x_0 is known; during sampling, noise prediction supplies the needed approximation.The posterior is written as q(x_{t-1} | x_t, x_0) = N(x_{t-1}; μ̃_t(x_t, x_0), β̃_t I).Source: Appendix A · p. 10

Appendix B: Variational lower bound and the denoising objective

The appendix decomposes the variational lower bound into terminal, reverse-kernel, and reconstruction terms. Its DDPM objective trains the network to predict injected noise, with the clean input defined differently for position- and velocity-space models.

Appendix B: Variational lower bound and the denoising objective

  • Table III identifies the combined hyperparameter configuration used for the experiments, with architecture and diffusion settings shared across both models.Single-value rows are shared; rows with two values differ between models.Source: Table III · p. 11
  • The negative variational bound decomposes into a terminal prior-matching term, intermediate reverse-kernel terms, and a reconstruction term.The intermediate terms train learned reverse kernels to match the exact Gaussian posteriors.Source: Appendix B · pp. 10–11
  • The network is trained to recover injected noise from the noisy sample and diffusion step using the simplified DDPM objective.For velocity space, x_0 is the normalized velocity window; for position space, it is the normalized coordinate window.Source: Appendix B · p. 11

Appendix C: Implementation details and hyperparameters

The listed values are configurations used for the reported experiments.

Appendix C: Implementation details and hyperparameters

  • The listed values are configurations used for the reported experiments.Source: Appendix C · p. 11

Appendix D: Additional results

Appendix D reports position-space comparisons with a kinematic Markov baseline, downstream prediction performance, event-level statistics, and sampling ablations. Distributional metrics aggregate three seeds, while the ablations and I-VT analyses are within-run comparisons.

Appendix D: Additional results

  • Increasing DDIM steps generally improves agreement, with 100 steps selected as a quality–efficiency compromise; deterministic sampling outperforms stochastic sampling.The EMA shadow model reduces mean-speed KS from 0.277 to 0.158 versus the raw checkpoint. These ablations use one reference run rather than independent training seeds.Source: Appendix D · pp. 11–13, Table VI · p. 14
  • Mean KS falls from 0.481 to 0.138 ± 0.013 and mean JS from 0.221 to 0.0155 ± 0.0038 versus the kinematic Markov baseline.The DDPM improves on the baseline for every individual feature across eight shared non-redundant features.Source: Appendix D · pp. 11–12, Table IV · p. 13
  • Wasserstein-1 distances are reported in each feature’s own units and should not be compared directly across rows with different units.Total path length is 1,999× mean speed by construction, explaining its correspondingly larger W1.Source: Appendix D · p. 12, Table V · p. 13
  • Mean fixation duration (0.351 s vs 0.344 s), fixation count (21.41 vs 21.81), and mean saccade peak speed (0.0618 vs 0.0624 normalized units) closely match real data.Fixation dispersion is approximately 46% higher and saccade count approximately 14.8% lower in generated data; these I-VT results depend on a fixed velocity threshold.Source: Appendix D · p. 13, Table VII · p. 14
  • TSTR fixation-count prediction reaches R2 = 0.6612 ± 0.0219, or 82.7% of the TRTR R2 point estimate of 0.7999 ± 0.0467.The TSTR MAE is 2.7423 ± 0.0970 versus 1.9219 for TRTR; the TRTR spread is a five-fold cross-validation SD, not an upper bound.Source: Appendix D · p. 13, Table VIII · p. 14
Loading 2609.05522v1…